zihangdai/xlnet
Official implementation of XLNet, a transformer-based language model using permutation language modeling for natural language understanding.

XLNet provides an unsupervised language representation learning method based on a generalized permutation language modeling objective. It uses Transformer-XL as its backbone architecture, enabling effective handling of long-range dependencies in text. The model achieves state-of-the-art results on diverse downstream NLP tasks including question answering, text classification, natural language inference, and document ranking, outperforming BERT on 20 tasks.