ReqLLM turns 21 LLM APIs into one predictable Elixir interface

What it does ReqLLM is an Elixir HTTP client for LLMs built atop Req and Finch. It exposes a unified API—generate_text, stream_text, generate_object, and more—that normalizes requests and responses across 21 provider integrations covering over 1,200 models. Provider-specific callbacks translate canonical options into each backend’s native parameters, while Finch handles streaming and Req handles the rest.

The interesting bit The library tracks 92 non-text operation models—embedding, image generation, speech, transcription, rerank, and OCR—and maintains a fixture-backed compatibility ledger of 619 recorded model specs. It also auto-translates canonical options like max_tokens into provider-specific variants (for example, max_completion_tokens for OpenAI’s o1 and o3), saving you from tedious parameter mapping.

Key highlights

• Typed canonical structs (Context, Message, Tool, Response, etc.) implement Jason.Encoder for easy persistence.
• generate_object/4 uses NimbleOptions-compiled schemas, including OpenAI native structured output modes.
• stream_text/3 streams via Finch directly for HTTP/2 multiplexing, working around known Req SSE limitations.
• Layered key management (ReqLLM.Keys) falls back through per-request overrides, in-memory storage, config, env vars, and .env files.
• Best-effort usage and USD cost tracking from provider responses and model metadata.

Caveats

• Streaming uses Finch directly because of known Req limitations with SSE responses.
• Batch embeddings and other operations are not supported by every provider.
• The OpenAI realtime WebSocket session API is experimental and low-level.

Verdict Elixir developers tired of maintaining N different HTTP client wrappers should look here; if you only ever call OpenAI, it’s likely overkill.