This is Intel’s official extension for transformers that provides state-of-the-art compression techniques (INT4, weight-only quantization) to run LLMs efficiently on Intel platforms including CPU, GPU (Gaudi3/Habana), and Xeon processors. It offers NeuralChat for building chatbots within minutes, supports RAG-based retrieval applications, and includes optimizations like speculative decoding and StreamingLLM. The toolkit supports popular models such as Qwen2, Llama 3, and GPT-J, targeting performance improvements for LLM inference workloads.