tensorflow/serving
TensorFlow Serving is an open-source system for serving trained machine learning models in production environments with versioning, batching, and gRPC/HTTP endpoints.
Not currently ranked — collecting fresh signals.
star history
It provides a flexible, high-performance serving system designed for machine learning model inference in production. The system manages model lifetimes, supports versioned access to multiple models simultaneously, and includes a scheduler that batches inference requests for efficient GPU execution. TensorFlow Serving offers gRPC and HTTP inference endpoints, enables canary deployments and A/B testing of new model versions, and integrates natively with TensorFlow while remaining extensible to other ML frameworks and model types.
Frequently asked
- What is tensorflow/serving?
- TensorFlow Serving is an open-source system for serving trained machine learning models in production environments with versioning, batching, and gRPC/HTTP endpoints.
- Is serving open source?
- Yes — tensorflow/serving is open source, released under the Apache-2.0 license.
- What language is serving written in?
- tensorflow/serving is primarily written in C++.
- How popular is serving?
- tensorflow/serving has 6.4k stars on GitHub.
- Where can I find serving?
- tensorflow/serving is on GitHub at https://github.com/tensorflow/serving.