Teaching OpenTelemetry to speak fluent LLM

What it does — OpenInference is a specification and plugin set that layers on top of OpenTelemetry to make AI applications observable. It defines semantic conventions for LLM invocations, vector-store retrievals, and external tool usage, then ships Python auto-instrumentation packages for popular frameworks so those events emit standard traces without hand-rolled instrumentation.

The interesting bit — Instead of building a bespoke telemetry stack, it piggybacks on OpenTelemetry’s existing infrastructure. The spec is transport- and format-agnostic, meaning traces can land in Arize Phoenix, Jaeger, or any OTLP-compatible backend without vendor lock-in. It is essentially the adapter that teaches generic observability tools to read prompts, embeddings, and agent loops.

Key highlights

• Broad framework coverage in Python: OpenAI, LangChain, LlamaIndex, AWS Bedrock, VertexAI, DSPy, Haystack, MCP, CrewAI, and others (per the package table)
• Backend-agnostic by design: works with any OpenTelemetry-compatible collector, not just Arize Phoenix
• Reusable instrumentation utilities and decorators via the openinference-instrumentation base package
• Specification is format-agnostic, intended to work alongside JSON, ProtoBuf, and DataFrames

Caveats

• The README claims support for “a variety of languages,” but only Python instrumentations are visible in the provided source
• The package list is truncated, so the full breadth of supported frameworks is unclear from the README alone

Verdict — Reach for this if you already run OpenTelemetry and need LLM-specific traces without building custom spans. Look elsewhere if you want a turnkey observability dashboard; this is plumbing, not porcelain.