Elixir’s LLM Framework Refuses to Pretend It’s Python

What it does

This library connects Elixir applications to large language models—both commercial APIs like Claude, GPT-4, Grok, and Gemini, and self-hosted options via Bumblebee and Ollama. It provides modular components and pre-built chains for common tasks, centered around the LLMChain module. You can also expose your own Elixir functions to the LLM as tools, complete with permission-aware context.

The interesting bit

Unlike ports of the Python and JavaScript LangChain libraries, this one embraces Elixir’s functional nature rather than fighting it. The author explicitly rejects object-oriented serialization parity and skips the conversation-history hacks needed back when LLMs lacked native chat support. That makes it less a translation and more a rethink of how LLM plumbing should work on the BEAM.

Key highlights

• Supports providers including Anthropic Claude, OpenAI, xAI Grok, Google Gemini, DeepSeek, Mistral, and AWS Bedrock Mantle (with third-party models like Moonshot Kimi K2)
• Self-hosting options via Bumblebee (Nx) and Ollama for fully offline inference
• LangChain.Function exposes custom application code to LLMs with user-scoped context and permissions
• Prompt caching support for ChatGPT, Claude, and DeepSeek; streaming and tool calling across major providers
• Built on the Req HTTP client and requires Elixir 1.17+

Caveats

• Gemini context caching is explicitly not supported (requires a separate API call)
• No aim for parity with JS/Python LangChain, so cross-language serialization of prompts and chains will not work
• Provider-specific quirks (e.g., AWS Bedrock Mantle reasoning extraction) may require reading module-level documentation

Verdict

Worth a look if you run Elixir in production and want to add LLM features without maintaining Python microservices. Skip it if you need drop-in compatibility with existing Python or JavaScript LangChain ecosystems, or if you are not on Elixir 1.17+.