Alibaba's Envoy gateway that routes LLMs and hosts MCP servers

What it does Higress is a cloud-native API gateway built on Istio and Envoy that can be extended with Wasm plugins written in Go, Rust, or JavaScript. It acts as a Kubernetes ingress controller, a microservice gateway with registry discovery, and a security gateway with WAF and authentication plugins. More recently, it has been positioned as an “AI Native API Gateway” that unifies access to mainstream LLM providers and can host Model Context Protocol (MCP) servers via its plugin system.

The interesting bit The project originated inside Alibaba to solve a specific, painful problem: Tengine reloads caused traffic jitter that broke long-connection services like gRPC and Dubbo, and configuration changes took too long to propagate. Higress replaces that model with Envoy’s hot-restart-friendly architecture, adding true streaming request/response body processing through Wasm plugins—a feature that matters when you are proxying high-bandwidth Server-Sent Events from AI models. It also ships with an openapi-to-mcp tool that essentially turns existing REST APIs into remote MCP servers, giving AI agents a standardized way to call them.

Key highlights

• Supports mainstream LLM providers with unified protocol, token rate limiting, caching, and multi-model load balancing.
• Hosts MCP servers dynamically through Wasm plugins, with unified auth, rate limiting, and audit logging for tool calls.
• Compatible with Kubernetes Ingress API and Gateway API, with claims of significantly lower resource overhead and faster route updates than ingress-nginx.
• Integrates with Alibaba’s microservice stack (Dubbo, Nacos, Sentinel) and supports service discovery from ZooKeeper, Consul, and Eureka.
• Sandboxed Wasm plugins allow traffic-lossless hot updates and support multi-language extensions without restarting the gateway.

Caveats

• The documentation and default registries are heavily oriented toward Alibaba Cloud, which can make the project feel like an on-ramp to a commercial platform rather than a neutral open-source core.
• The README recommends the Alibaba Cloud enterprise edition for cloud deployments, so self-hosters should verify which features are open-source and which require the paid tier.

Verdict If you are running AI workloads or MCP infrastructure on Kubernetes and need an Envoy-based ingress controller with Wasm extensibility, Higress is worth evaluating—especially if you are already using Alibaba’s ecosystem. If you are looking for a vendor-neutral, minimal gateway with no cloud vendor affiliations, the Alibaba Cloud branding may give you pause.