Envoy AI Gateway implements a two-tier gateway pattern for generative AI traffic. The Tier One gateway handles authentication, global routing, and rate limiting as a centralized entry point. The Tier Two gateway provides fine-grained control over self-hosted model access with endpoint picker support for LLM inference optimization. It supports a wide range of AI providers including OpenAI, Azure OpenAI, Google Gemini, Vertex AI, AWS Bedrock, and Mistral.