← all repositories

voidism/DoLa

A decoding strategy for large language models that reduces hallucinations by contrasting differences in logits across transformer layers.

554 stars Python Language ModelsLLMOps · Eval
DoLa
Velocity · 7d
+0.5
★ / day
Trend
steady
star history

This repository provides the official implementation of DoLa, a decoding method that improves factuality in LLMs without requiring additional fine-tuning or external knowledge retrieval. The approach contrasts logits between early and later transformer layers to identify and suppress hallucinated tokens. The code integrates with Hugging Face Transformers and includes evaluation scripts for benchmarking factuality improvements.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.