Your Mac, Dressed Up as OpenAI and Anthropic

What it does

MLX Omni Server is a FastAPI-based inference backend built on Apple’s MLX framework. It exposes local models through endpoints that mirror the OpenAI and Anthropic API shapes, so existing client libraries can talk to your M-series Mac instead of the cloud. The server handles chat completions, speech-to-text, text-to-speech, image generation, and embeddings.

The interesting bit

Most local inference projects pick one API dialect; this one translates both. That means you can keep using openai or anthropic Python SDKs and simply swap the base_url to localhost. The server auto-discovers MLX models in your local Hugging Face cache and handles on-demand loading.

Key highlights

• Dual API compatibility: supports OpenAI /v1/* and Anthropic /anthropic/v1/* endpoints including streaming, function calling, and structured output.
• Runs entirely offline on Apple Silicon (M1–M4) with hardware-accelerated MLX inference.
• Covers a full stack: chat, embeddings, audio transcription/synthesis, and image generation.
• Auto-discovers and caches models from the Hugging Face mlx-community hub.

Caveats

• Requires an Apple Silicon Mac and Python 3.11+; there is no fallback for Intel Macs or other platforms.
• The README notes it is not affiliated with OpenAI, Anthropic, or Apple.

Verdict

Worth a look if you already build against OpenAI or Anthropic APIs and want a private, local staging environment on a modern Mac. Skip it if you are not on Apple Silicon or need production-grade multi-user serving.