← all repositories
sicara/tf-explain

X-ray specs for your Keras model, no PhD required

tf-explain wraps seven neural-network interpretability methods into plain TensorFlow 2.x callbacks so you can watch your model think while it trains.

1k stars Python ML FrameworksLLMOps · Eval
tf-explain
Velocity · 7d
+0.4
★ / day
Trend
steady
star history

What it does

tf-explain gives you GradCAM, SmoothGrad, Integrated Gradients, and four other techniques as drop-in Keras callbacks or standalone explainers. Point it at a trained model to generate heatmaps, or hook it into model.fit and watch explanations land in TensorBoard every epoch. It is pure plumbing: the methods themselves are well-known papers; the value is the packaging.

The interesting bit

The callback design is the quiet win. Most interpretability libraries make you write boilerplate to bridge training loops and visualization; here you pass GradCAMCallback(...) to fit and get automatic logging. It is the kind of integration that only looks obvious after someone else builds it.

Key highlights

  • Seven methods: Activations Visualization, Vanilla Gradients, Gradients×Inputs, Occlusion Sensitivity, Grad CAM, SmoothGrad, Integrated Gradients
  • Two APIs: one-off explainer.explain(data, model) or recurring Callback during training
  • TensorBoard-native output when used as a callback
  • Supports Python 3.6–3.8 and TensorFlow 2.x (you bring your own tensorflow install)
  • OpenCV and the library itself are pip-installable; no conda gymnastics

Caveats

  • Subclassing API support is on the roadmap but not yet implemented
  • Python support tops out at 3.8, which is increasingly dated
  • Auto-generated API docs are also still pending

Verdict

Worth a look if you are already in TensorFlow 2.x and want interpretability without leaving the Keras ecosystem. Skip it if you need PyTorch, modern Python, or subclassed-model support today.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.