After-hours deep learning: capsules, gradient norms, and SELU

What it does This repository is a monorepo of standalone deep learning experiments built in Keras and TensorFlow. It bundles four specific projects—capsule pooling for images, a gradient optimizer that normalizes by gradient norms, SELU weight regularizers for Dense and Conv2D layers, and an oversampling example using the tf.data.Dataset API—alongside slide decks from internal company seminars. Think of it as a working lab notebook rather than a unified framework.

The interesting bit The value is in the specifics: instead of yet another MNIST tutorial, you get a max-normed-optimizer and a Keras regularizer that forces self-normalizing properties mid-training, which the README notes can preserve SELU behavior even as weights drift. The capsule pooling experiment is explicitly labeled an “after-hours” attempt, which at least promises honest curiosity over polished marketing.

Key highlights

• max-normed-optimizer: experimental gradient descent variant that normalizes gradients according to their norms, with included experiments.
• selu-regularization: custom Keras regularizer layers for Dense and Conv2D that enforce self-normalizing weight properties during training.
• keras-capsule-pooling: an image-focused attempt at capsule pooling, included as a working reference.
• tf-oversampling: a practical pattern for oversampling using the tf.data.Dataset API.
• seminars: a folder of presentations given internally, presumably on these topics.

Caveats

• The repo is a loose collection, not a cohesive library; expect to extract code rather than import a package.
• The README is sparse on details like test coverage or maintenance status; what you see is what you get.

Verdict Worth a bookmark if you are building an oddball optimizer or regularizer and need a working Keras reference; skip it if you need a maintained, versioned library.