What it does

llm-ls is a Rust-based LSP server that sits between your editor and an LLM backend, handling code completion requests. It tokenizes prompts to stay within context windows, parses ASTs to decide whether to suggest single-line, multi-line, or no completions, and supports “fill in the middle” prompting. The project explicitly aims to be shared infrastructure for IDE extensions rather than a standalone tool.

The interesting bit

The AST-aware completion filtering is the quietly useful part: instead of blindly dumping LLM output into your editor, it tries to match the suggestion format to what the code structure actually needs. The telemetry logging to a local file (not external export) is also a thoughtful touch for anyone thinking about fine-tuning later.

Key highlights

• Supports multiple backends: Hugging Face Inference API, TGI, ollama, and OpenAI-compatible APIs including llama.cpp
• Existing integrations for Neovim, VS Code, and IntelliJ; Jupyter planned
• Tokenization-aware context window management
• Local-only telemetry logging to ~/.cache/llm_ls/llm-ls.log
• Written in Rust, which matters if you’re running this persistently alongside your editor

Caveats

• README opens with “expect things to be broken” — this is pre-1.0 software
• Workspace-wide context is on the roadmap, not implemented; currently limited to the current file
• “More?” in the description hints at broader ambitions, but the README only documents completion

Verdict

Worth a look if you’re building or maintaining an IDE extension and tired of reimplementing LLM client logic. Skip it if you want a polished, batteries-included copilot replacement; this is plumbing, not a faucet.