JittorLLMs is a large language model inference library developed by the Jittor team. It provides a unified runtime for running various open-source LLMs such as ChatGLM-6B, LLaMA/LLaMA2, ChatRWKV, and Pangu, aiming to reduce hardware barriers so models can run on machines with as little as 2GB RAM without a GPU. It leverages Jittor’s zero-copy and automatic meta-operator compilation to accelerate model loading and throughput compared to同类 frameworks.