Pattern-Exploiting Training (PET) is a semi-supervised training procedure for language models that reformulates input examples as cloze-style phrases. It outperforms regular supervised training and even GPT-3 in low-resource settings while requiring 99.9% fewer parameters. The iterative variant iPET trains multiple generations of models and can operate without any training data, making it effective for few-shot text classification and natural language inference tasks.