Shakespeare by the word, not the byte
A word-level LSTM language model that swaps character-by-character generation for token-by-token output, with beam search to keep things coherent.

What it does Trains multi-layer LSTM/RNN networks on word-level text corpora, then generates new text in the same style. The default dataset is tinyshakespeare, so out of the box you get Elizabethan-ish dialogue that almost makes sense. Sampling supports greedy selection, probabilistic picking, or beam search with configurable width.
The interesting bit The project is explicitly “mostly reused code” from a char-rnn-tensorflow port of Karpathy’s original char-rnn. The twist is swapping character tokens for word tokens—same architecture, different granularity. Beam search is the one addition that feels genuinely useful: it keeps multiple candidate sequences alive during generation, which visibly reduces the “and and and and” repetition loops that plague greedy word-level models.
Key highlights
- Word-level tokenization instead of character-level, producing more readable (if not more sensible) output
- Beam search with configurable width (
--pick 2 --width 4) for less repetitive generation - Simple CLI:
python train.pythenpython sample.py - One known downstream project: a Korean poem generator at bot.wpoem.com
- ~1,300 stars, suggesting it served as a common reference implementation circa 2017
Caveats
- Requires TensorFlow 1.1.0rc0—a very old release, so expect dependency archaeology
- The README admits this is “mostly reused code,” so don’t expect architectural novelty
- Beam search still produces occasional repetition and nonsense; the sample outputs show “you, and and and and” slipping through even with width=4
Verdict Worth a look if you’re studying how early TensorFlow LSTM implementations worked, or if you need a minimal word-level baseline to compare against modern transformers. Skip it if you want production-ready text generation—this is a 2017-era educational artifact.