Generating waveforms with dilated convolutions and patience

What it does ibab/tensorflow-wavenet is a from-scratch rebuild of DeepMind’s generative audio model. It learns to synthesize sound by predicting each raw sample conditioned on every sample that came before it, using stacks of causal dilated convolutions. Training happens on plain WAV corpora—mostly tested against the VCTK speech dataset—and the resulting model can emit new audio or mimic specific speakers if trained with global conditioning.

The interesting bit The model treats audio generation as a sequence-modeling problem over categorical integers: it quantizes waveforms, one-hot encodes the amplitudes, and runs them through convolutions that only look backward in time. A softmax layer predicts the next sample’s probability distribution, which means the network is essentially doing next-token prediction at 16,000 tokens per second.

Key highlights

• Supports global conditioning on speaker ID, enabling multi-speaker training and voice-specific generation (currently hard-wired to VCTK filename conventions).
• Includes fast generation borrowed from the Fast Wavenet project, cutting synthesis time down to minutes rather than hours.
• Outputs can be streamed to TensorBoard for playback or saved directly as WAV files.
• The core architecture lives in a single model.py file, making the network topology easy to inspect.

Caveats

• The code targets TensorFlow 1.0.1 and Python 2.7/3.5, so expect a distinctly 2017 flavor of dependency management.
• Global conditioning logic is tightly coupled to VCTK corpus naming patterns; adapting it to other datasets requires manual tweaks.
• Local conditioning is not implemented, meaning you cannot control linguistic content or guide the model with text prompts.

Verdict Worth a look if you want a readable, self-contained implementation of WaveNet’s core architecture. Skip it if you need modern TensorFlow, local conditioning, or a production-ready speech pipeline.