From MobileViT to OpenELM: Apple’s training stack goes wide

What it does

CoreNet is a PyTorch-based toolkit for training small and large neural networks across vision, language, and audio tasks, including foundation models like CLIP and LLMs. It evolved from Apple’s earlier CVNets library, expanding beyond pure computer vision. The repository contains model implementations, dataset loaders, loss functions, optimizers, and training pipelines, all wired together through a registry-and-YAML configuration system.

The interesting bit

The project functions as a cookbook for Apple’s recent ML research. Each publication—from FastViT and MobileViT to OpenELM and KV Prediction—ships with its own projects/ folder containing reproducible training recipes, configs, and links to pretrained weights. There are even MLX examples for running CLIP and OpenELM on Apple Silicon, suggesting the toolkit is not just for Linux server farms.

Key highlights

• Grew from a computer vision library (CVNets) into a generalist training framework for LLMs and multimodal models.
• Ships with reproducible training recipes and pretrained checkpoints for over a dozen Apple research papers.
• Organized by task via a registry pattern: models, datasets, and pipelines are registered and referenced in YAML configs.
• Includes MLX ports for running CLIP and OpenELM on Apple Silicon.
• Provides Jupyter notebook tutorials for training from scratch, CLIP, detection, and segmentation.

Caveats

• The README notes that Git LFS is required to run tests and notebooks, which adds friction.
• It warns that macOS case-insensitive filesystems can confuse Git when working with the repository.

Verdict

Researchers trying to reproduce Apple’s specific papers or train models across vision and language in a single, unified framework will find this useful. If you are already settled into Hugging Face, Lightning, or Detectron2 and do not need Apple’s exact recipes, this may be redundant.