The official CodeLlama loader is deliberately no-frills

What it does This repository is Meta’s official minimal inference loader for the Code Llama family—foundation, Python-tuned, and instruction-following variants ranging from 7B to 70B parameters. It provides example scripts for code completion, infilling (available on 7B and 13B models), and instruction chat, plus the llama package that actually loads the weights. You supply the checkpoints, which require a Meta license and manual download; the repo supplies the PyTorch wiring and model-parallel sharding logic.

The interesting bit The project treats transparency as a feature. Because it is explicitly a “minimal example,” there is almost no abstraction: you can see exactly how the 70B instruct model’s turn-based prompt format diverges from the smaller variants, and how cache pre-allocation ties directly to your hardware limits. That makes it less of a framework and more of a reference implementation you are meant to read, then replace.

Key highlights

• Three model flavors: base code completion, Python specialization, and instruction-following chat.
• 7B and 13B variants support code infilling—generating the middle of a function from surrounding context.
• Most models handle up to 100k tokens in practice, though the 70B Python and instruct variants do not share this long-context ability.
• Weights are cleared for commercial use, but accessing them means signing Meta’s license and using signed URLs that expire after 24 hours.
• Safety hooks are stubbed out, pointing to the llama-recipes repo for input and output filtering examples.

Caveats

• This is reference glue, not a production framework; expect to outgrow it if you are building a product.
• Weight downloads are gated behind Meta’s approval flow and finicky expiring links.
• Prompt formatting is size-sensitive: the 70B instruct model uses a separate turn-based format from its smaller siblings.

Verdict Use this if you need an unopinionated, official baseline for reproducing Code Llama research or benchmarking. Look elsewhere—like Hugging Face or llama.cpp—if you want a polished, batteries-included inference stack.