← all repositories
hunkim/word-rnn-tensorflow

Shakespeare by the word, not the byte

A word-level LSTM language model that swaps character-by-character generation for token-by-token output, with beam search to keep things coherent.

1.3k stars Python Language Models
word-rnn-tensorflow
Velocity · 7d
+0.4
★ / day
Trend
steady
star history

What it does Trains multi-layer LSTM/RNN networks on word-level text corpora, then generates new text in the same style. The default dataset is tinyshakespeare, so out of the box you get Elizabethan-ish dialogue that almost makes sense. Sampling supports greedy selection, probabilistic picking, or beam search with configurable width.

The interesting bit The project is explicitly “mostly reused code” from a char-rnn-tensorflow port of Karpathy’s original char-rnn. The twist is swapping character tokens for word tokens—same architecture, different granularity. Beam search is the one addition that feels genuinely useful: it keeps multiple candidate sequences alive during generation, which visibly reduces the “and and and and” repetition loops that plague greedy word-level models.

Key highlights

  • Word-level tokenization instead of character-level, producing more readable (if not more sensible) output
  • Beam search with configurable width (--pick 2 --width 4) for less repetitive generation
  • Simple CLI: python train.py then python sample.py
  • One known downstream project: a Korean poem generator at bot.wpoem.com
  • ~1,300 stars, suggesting it served as a common reference implementation circa 2017

Caveats

  • Requires TensorFlow 1.1.0rc0—a very old release, so expect dependency archaeology
  • The README admits this is “mostly reused code,” so don’t expect architectural novelty
  • Beam search still produces occasional repetition and nonsense; the sample outputs show “you, and and and and” slipping through even with width=4

Verdict Worth a look if you’re studying how early TensorFlow LSTM implementations worked, or if you need a minimal word-level baseline to compare against modern transformers. Skip it if you want production-ready text generation—this is a 2017-era educational artifact.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.