Google's BERT, rebuilt for Keras diehards

What it does

This is a native Keras implementation of BERT that ingests Google’s official pre-trained checkpoints. It exposes helpers to tokenize text, extract token- or sentence-level embeddings, and fine-tune the model on tasks like masked-word prediction or classification. You can also train a transformer from scratch, though the documentation focuses on loading existing weights.

The interesting bit

The library acts as a compatibility layer: it reconstructs the BERT architecture in Keras and maps the official weights so the outputs match Google’s reference implementation. It also bundles an AdamWarmup optimizer and utilities for downloading checkpoints and calculating warmup steps, saving you from re-implementing the training boilerplate BERT usually demands.

Key highlights

• Loads official pre-trained BERT models and reproduces their extraction results.
• Provides extract_embeddings for quick feature extraction from raw text or paired sentences.
• Includes a custom Tokenizer handling WordPiece segmentation and special tokens like [CLS] and [SEP].
• Supports TPU execution via conversion helpers (demos linked in the repo).
• Bundles AdamWarmup with decay scheduling and a step calculator.

Caveats

• The README is almost entirely quick-start snippets; architectural details, performance notes, and version compatibility guidance are absent.
• Training examples rely on model.fit_generator without clarifying supported TensorFlow or Keras versions.

Verdict

Worth a look if you are maintaining a Keras codebase and need BERT embeddings or fine-tuning without migrating to the Hugging Face ecosystem. If you are already using modern transformers libraries, this is largely redundant.