What it does

This repo implements a custom RNN layer with attention in Keras, trained on a toy problem: converting human-readable dates (“Saturday January 5 2016”) into machine formats (“2016-01-05”). The real product is the visualization script that plots attention weights as heatmaps, showing which input tokens the model considered relevant for each output token.

The interesting bit

The attention maps reveal concrete failure modes. One example shows the model correctly learning that “Saturday” carries no predictive value; another shows it hallucinating “02” from the “20” in “2016” when parsing a mangled input. It’s interpretability as bug report.

Key highlights

• Custom Keras attention layer with full training and visualization pipeline
• Generates synthetic date data with configurable vocabulary
• Pre-trained weights and example files included so you can run visualize.py immediately
• Accompanied by a Medium tutorial walking through the attention mechanism
• Python 3.4+, GPU strongly recommended (CPU training is “prohibitively slow”)

Caveats

• Status: “Reference code only. No updates expected” — the authors have explicitly abandoned maintenance
• Built for an older Keras/TensorFlow ecosystem; requirements files may need pinning to run today
• The run.py help text has a copy-paste error (“Location of validation data” appears twice, once for batch size)

Verdict

Worth a look if you’re teaching or learning attention mechanisms and want concrete visual intuition. Skip it if you need production-grade tooling — this is a 2017-era educational artifact, not a maintained library.