Is mlc-llm open source?

Yes — mlc-ai/mlc-llm is open source, released under the Apache-2.0 license.

What language is mlc-llm written in?

mlc-ai/mlc-llm is primarily written in Python.

How popular is mlc-llm?

mlc-ai/mlc-llm has 23k stars on GitHub.

Where can I find mlc-llm?

mlc-ai/mlc-llm is on GitHub at https://github.com/mlc-ai/mlc-llm.

← all repositories

mlc-ai/mlc-llm

One Compiler Stack to Run LLMs on Phones, Browsers, and GPUs

It exists so you can compile and deploy large language models to phones, browsers, and nearly any consumer GPU from a single stack.

★23k stars Python Inference · Serving Language Models

View on GitHub ↗ Homepage ↗

Not currently ranked — collecting fresh signals.

star history

What it does

MLC LLM is a machine learning compiler and high-performance deployment engine for large language models. It compiles models down to MLCEngine, a unified inference runtime that exposes an OpenAI-compatible API across REST, Python, JavaScript, iOS, and Android. The stated mission is to let developers optimize and deploy AI natively on nearly any platform without fragmenting their serving stack.

The interesting bit

Rather than hand-writing GPU kernels for every vendor or wrapping a dozen platform SDKs, the project leans on the TVM compiler ecosystem—specifically TensorIR and MetaSchedule—to generate optimized code for each target automatically. That mechanical, unglamorous work is exactly what makes the “run anywhere” claim credible.

Key highlights

Supports AMD (ROCm/Vulkan), NVIDIA (CUDA/Vulkan), Apple (Metal), Intel (Vulkan/Metal), and mobile GPUs (OpenCL on Adreno and Mali).
Runs in web browsers via WebGPU and WASM, and on iOS/iPadOS via Metal on Apple A-series chips.
Offers a single MLCEngine backend with an OpenAI-compatible API across REST, Python, JavaScript, iOS, and Android.
Built on top of the TVM compiler stack, leveraging TensorIR and MetaSchedule for automatic tensor program optimization.

Verdict

Developers who need to ship LLMs to edge devices, mobile apps, or heterogeneous GPU fleets without fragmenting their serving code should look here. If you are already locked into a single-vendor datacenter stack and only care about throughput on the latest NVIDIA hardware, this is probably overkill.

Frequently asked

What is mlc-ai/mlc-llm?: It exists so you can compile and deploy large language models to phones, browsers, and nearly any consumer GPU from a single stack.
Is mlc-llm open source?: Yes — mlc-ai/mlc-llm is open source, released under the Apache-2.0 license.
What language is mlc-llm written in?: mlc-ai/mlc-llm is primarily written in Python.
How popular is mlc-llm?: mlc-ai/mlc-llm has 23k stars on GitHub.
Where can I find mlc-llm?: mlc-ai/mlc-llm is on GitHub at https://github.com/mlc-ai/mlc-llm.