TensorRT LLM provides a Python API for defining Large Language Models and performs inference efficiently on NVIDIA GPUs through state-of-the-art optimizations. It includes specialized kernels for common operations and Python/C++ runtime components that orchestrate performant inference execution. The framework supports MoE architectures and integrates with PyTorch.