Neural nets to distributed training, written in raw numpy

What it does This repository is a set of educational notebooks that implement core machine-learning primitives—neural networks, transformers, and distributed-model patterns—using only numpy. The README walks through forward passes, loss calculation, and backpropagation with plain Python classes like Neuron and Layer, pairing code with architecture diagrams and partial-derivative intuition.

The interesting bit The project insists on hand-rolling gradient descent inside a Neuron.backward method rather than calling a framework autograd function. That deliberate friction forces you to confront exactly how each weight and bias nudges the loss—clarity that a black-box .backward() usually spares you.

Key highlights

• Implements a full feed-forward network from scratch with ReLU activations, MSE loss, and mini-batch gradient descent.
• Explains backpropagation via the chain rule (∂C/∂w = ∂C/∂a × ∂a/∂z × ∂z/∂w) rather than treating it as a library primitive.
• Visual-heavy documentation with architecture diagrams, weight-and-bias illustrations, and gradient-descent plots.
• Also covers transformers and distributed-model patterns, supported by extensive diagram assets for attention mechanisms and parallelism strategies.
• Runs fully offline with no GPU framework or cloud dependency—just numpy and the standard library.

Caveats

• The transformer and distributed-model sections are truncated in the provided README snippet, so the full depth of those implementations is not visible in the text alone.
• The neural-network code is strictly educational: no regularization, advanced optimizers, or numerical-stability tricks are included.

Verdict Worth bookmarking if you learn best by stripping away framework magic. Skip it if you need production-grade model code or a drop-in PyTorch replacement.