WWDC 2019’s on-device BERT demo, now archived

What it does

This archived repo houses Swift code and Core ML 3 models for running BERT, DistilBERT, GPT-2, and DistilGPT-2 locally on Apple hardware. It ships a Swift implementation of the BERT tokenizer (BasicTokenizer and WordpieceTokenizer), SQuAD parsing utilities, and Python scripts that convert PyTorch-trained Hugging Face checkpoints into Core ML. The payoff is two demo apps: one for question-answering and another for on-device text generation.

The interesting bit

The BERT SQuAD model was demoed on stage at WWDC 2019 and linked from Apple’s official Core ML model page, giving this repo a small place in on-device ML history. Most of the value hides in the unglamorous plumbing—custom tokenizers and byte-pair encoders in Swift—that lets a Core ML model actually understand text rather than just crunch tensors.

Key highlights

• Archived; Hugging Face points to swift-transformers as the in-development replacement.
• Includes conversion scripts from PyTorch (transformers) to Core ML for GPT-2 and DistilGPT-2.
• Supports greedy and Top-K decoding strategies for text generation.
• BERT and DistilBERT models are fine-tuned for question-answering on the SQuAD dataset.
• BERTSQUADFP16 was an official Apple Core ML 3 launch model.

Caveats

• The repository is archived and no longer actively maintained.
• The README still promises “other Transformers coming soon” despite the archive status.
• Large model files live outside standard Git, so a vanilla checkout does not include the full weights.

Verdict

Worth a look if you are maintaining legacy Core ML 3 apps or studying how to bridge Hugging Face tokenizers to Swift. Everyone else should jump straight to the successor swift-transformers repository instead.