This repository accompanies the paper Crosslingual Generalization through Multitask Finetuning and provides all components used to create the BLOOMZ and mT0 model families. It includes the xP3 multilingual dataset for multitask training, training scripts for finetuning T5-based and BLOOM-based models on diverse tasks across 46+ languages, and evaluation pipelines for assessing crosslingual generalization capabilities. The work enables zero-shot transfer of language models to non-English tasks through instruction tuning.