jina-ai/serve
A framework for building and deploying multimodal AI services via gRPC, HTTP and WebSockets with native LLM serving and cloud-native orchestration.

Jina-Serve provides infrastructure for developing and deploying AI services at scale. It supports all major ML frameworks and data types with built-in features for streaming, dynamic batching, and high-performance service design. The framework includes Docker integration, Kubernetes support, and one-click deployment to cloud platforms, enabling developers to focus on core AI logic while handling orchestration, scaling, and monitoring.