This repository provides demos and packages for running efficient inference on the BLOOM large language model. It supports inference via HuggingFace accelerate and DeepSpeed Inference, with options for fp16/bf16 and int8 quantized deployment on multi-GPU setups. It leverages LLM.int8() and ZeroQuant techniques for post-training quantization to reduce memory footprint while maintaining generation quality.