ChemBERTa provides BERT/RoBERTa transformer models pre-trained on molecular datasets (ZINC, PubChem) for cheminformatics tasks. The models apply masked language modeling to chemical SMILES notation to learn molecular representations. Pre-trained weights for various dataset sizes are available on HuggingFace for downstream fine-tuning on property prediction and drug discovery tasks.