← all repositories
carpedm20/NTM-tensorflow

A 2014 deep-learning curiosity, ported to TensorFlow with honest scars

An early TensorFlow implementation of DeepMind's Neural Turing Machine that the author warns is "very poorly designed" for variable-length inputs.

1k stars Jupyter Notebook ML Frameworks
NTM-tensorflow
Velocity · 7d
+0.3
★ / day
Trend
steady
star history

What it does

Implements the Neural Turing Machine paper from DeepMind (2014) in TensorFlow 1.1, using an LSTM controller and supporting multiple read/write heads. Ships with copy and recall tasks to test whether the network can learn to read from and write to an external memory matrix.

The interesting bit

The README’s candor is the real feature. The author flags two known issues upfront: loss can explode to NaN despite gradient clipping, and the code structure can’t handle variable-length sequences gracefully. That second warning—“Just use this code as a reference”—saves you from treating it as production scaffolding.

Key highlights

  • TensorFlow 1.1.0 implementation with LSTM controller
  • Supports multiple read/write heads per the original NTM paper
  • Includes copy task training and quick-test modes
  • Results notebook (ipynb/NTM Test.ipynb) with visualized outputs
  • Referenced against a parallel Torch implementation for cross-checking

Caveats

  • Loss instability to NaN is a documented, unresolved issue (#2)
  • Codebase is explicitly not designed for variable-length NTM inputs
  • Recall task results are marked “(in progress)”
  • Requires Python 2.7 or 3.3+ and the now-legacy TensorFlow 1.x

Verdict

Worth a look if you’re studying the NTM architecture historically or need a readable reference implementation. Skip it if you need something battle-tested for variable-length sequences or modern TensorFlow compatibility.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.