DeepMind’s neural network kit that ships without a trainer

What it does

Sonnet is a TensorFlow 2 library built by DeepMind researchers for constructing neural networks. It centers on snt.Module, a self-contained abstraction that holds parameters, sub-modules, and computation methods. The library ships with common layers like snt.Linear and snt.Conv2D, plus pre-built networks such as snt.nets.MLP, but leaves all wiring decisions to the user.

The interesting bit

Most frameworks smuggle in assumptions about how you train; Sonnet deliberately refuses. It does not ship with a training framework, does not automatically average gradients during distributed training, and does not sync batch normalization statistics across replicas. The README explicitly states that users should be in full control of these aspects, which means the library is essentially a bag of decoupled parts rather than a pipeline.

Key highlights

• Modules are self-contained and entirely decoupled from one another
• No built-in training framework; users build or adopt their own loops
• Distributed training support via custom TensorFlow distribution strategies, but modules do not change behavior under distribution (no automatic gradient all-reduce or batch norm sync)
• Supports TensorFlow checkpointing and SavedModel serialization for model export
• Includes predefined modules and networks, with strong encouragement to subclass snt.Module for custom components

Caveats

• Python pickle serialization is supported but explicitly discouraged as brittle and poorly supported by many TensorFlow components
• Distributed training requires manual implementation of gradient all-reduce or swapping in explicitly distribution-aware modules like snt.distribute.CrossReplicaBatchNorm

Verdict

Sonnet is for DeepMind-style researchers who want modular LEGO bricks without a prescribed training architecture. Those seeking a framework that automatically handles distributed strategy details will find it too bare-bones.