TencentARC/LLaMA-Pro
A research project introducing block expansion to progressively extend LLaMA models, published at ACL 2024.
Not currently ranked — collecting fresh signals.
star history
This project presents a method for progressively extending LLaMA models by inserting and training new transformer blocks, enabling efficient model capacity expansion without full retraining. The work includes released model weights on HuggingFace and demonstrates improvements on code and math benchmarks. Extensions to Mistral models are also provided.
Frequently asked
- What is TencentARC/LLaMA-Pro?
- A research project introducing block expansion to progressively extend LLaMA models, published at ACL 2024.
- Is LLaMA-Pro open source?
- Yes — TencentARC/LLaMA-Pro is open source, released under the Apache-2.0 license.
- What language is LLaMA-Pro written in?
- TencentARC/LLaMA-Pro is primarily written in Python.
- How popular is LLaMA-Pro?
- TencentARC/LLaMA-Pro has 513 stars on GitHub.
- Where can I find LLaMA-Pro?
- TencentARC/LLaMA-Pro is on GitHub at https://github.com/TencentARC/LLaMA-Pro.