BERT learns to speak fluent finance

What it does

FinBERT takes standard BERT and runs it through extra pre-training on a large financial corpus—a subset of the Reuters TRC2 dataset—then fine-tunes it for sentiment classification on financial phrases. The result is a model that assigns one of three sentiment labels to sentences from a text file and emits a score calculated as the probability of positive minus the probability of negative.

The interesting bit

The training notebook applies discriminate and gradual_unfreeze techniques specifically to combat catastrophic forgetting, so the model learns finance jargon without overwriting its general language knowledge. It is a defensive training detail that matters when domain vocabulary and ordinary English overlap.

Key highlights

• Further pre-trained on a Reuters TRC2 subset, then fine-tuned on the Financial PhraseBank dataset.
• Hosted on the Hugging Face Hub alongside downloadable model binaries.
• Includes a predict.py script that consumes raw .txt files and produces a CSV with per-sentence probabilities and a composite sentiment score.
• Training configuration and catastrophic-forgetting mitigations are exposed in the finbert_training.ipynb notebook.

Caveats

• The codebase still depends on the legacy pytorch_pretrained_bert library rather than modern transformers; the authors note migration is a priority but not yet completed.
• The Reuters TRC2 dataset used for language-model training is not publicly available; researchers must apply for access to replicate that exact step.
• This is explicitly an intern research project, not an official Prosus product.

Verdict

Worth a look if you need off-the-shelf sentiment analysis for financial documents and can tolerate legacy dependencies. Skip it if you require a library built on the current transformers stack.