Someone wrote a llama.cpp competitor in C#
Local inference is C++ and Python territory; TensorSharp bets a full GGUF engine — text, image, video, embeddings, agents — can live in .NET.

What it does TensorSharp is an LLM inference engine and agent runtime built on .NET, loading GGUF models across a broad roster: DeepSeek V4/V4.1 Flash, GLM 5.x, Gemma 4, Qwen 3.5 through 3.8, GPT OSS, Nemotron-H, Mistral 3, and more. It goes past autoregressive text into DiffusionGemma-style text diffusion, Qwen-Image-2.1 image generation and editing, MiniMax-H3 video with native 32 kHz stereo audio, and BERT/XLM-R embeddings. Everything is reachable from a CLI, a browser chat UI, Ollama/OpenAI-compatible HTTP APIs, and an iPhone app, with CUDA, Metal, and Vulkan backends spanning Windows, macOS, Linux, and iOS.
The interesting bit The unusual part is where the code lives: the CPU path is managed .NET, with native GGML/CUDA/Metal/Vulkan backends underneath for the GPU heavy lifting, and the authors publish head-to-head benchmarks against llama.cpp on identical GGUF files and hardware. It’s also a teaching artifact — two books walk through building this engine from tensors and tokenizers to serving. The agent layer stands on its own: bounded tool loops, subagents with private workspaces and read-only defaults, so sandboxing is a design constraint rather than an afterthought.
Key highlights
- Continuous batching and a paged, Radix prefix-shared KV cache ship on by default; speculative decoding and tensor parallelism are opt-in.
- Tensor parallelism spans multiple GPUs and multiple machines — two nodes × two GPUs yields a global TP degree of 4.
TensorSharp.AgentHostadds Agent Skills, sandboxed file and shell tools, and automatic subagent delegation with independent contexts.- Ollama/OpenAI-compatible APIs cover chat plus batch embeddings (Snowflake Arctic Embed L, MiniLM).
- TensorAgent extends the same chat and agent experience to iPhone and iPad via the iOS
ggml_metalbackend.
Caveats
- Support is uneven by design: backend, modality, and validation coverage vary model by model, so check the model cards before committing.
- The GB10/DGX Spark build’s one real-hardware check predates the current code, and hosted CI only re-verifies CPU and archive paths — GPU paths aren’t continuously certified.
- The llama.cpp comparison is honest but scoped: results are specific to the measured model, backend, and workload.
Verdict .NET teams wanting local inference — or anyone who wants to learn how an inference engine actually works, with two books as the lab manual — should pay attention. If llama.cpp already covers your needs, this is a well-documented curiosity with a genuinely interesting agent runtime attached.
Frequently asked
- What is zhongkaifu/TensorSharp?
- Local inference is C++ and Python territory; TensorSharp bets a full GGUF engine — text, image, video, embeddings, agents — can live in .NET.
- Is TensorSharp open source?
- Yes — zhongkaifu/TensorSharp is open source, released under the BSD-3-Clause license.
- What language is TensorSharp written in?
- zhongkaifu/TensorSharp is primarily written in C#.
- How popular is TensorSharp?
- zhongkaifu/TensorSharp has 681 stars on GitHub.
- Where can I find TensorSharp?
- zhongkaifu/TensorSharp is on GitHub at https://github.com/zhongkaifu/TensorSharp.