← all repositories

kmeng01/rome

A research implementation of Rank-One Model Editing for modifying factual knowledge in GPT-2 XL and GPT-J language models.

764 stars Python Language ModelsML Frameworks
rome
Velocity · 7d
+0.5
★ / day
Trend
steady
star history

This repository implements ROME, a technique for locating and editing factual associations stored in auto-regressive transformer models. It provides causal tracing tools to identify internal representations of specific facts in GPT models, then applies rank-one updates to the MLP layers to alter stored knowledge. The framework supports GPT-2 XL (1.5B) and GPT-J (6B), includes evaluation on the CounterFact dataset, and offers notebook demonstrations for both causal tracing and model editing.

heatdrop uses Google Analytics to see which pages get read — nothing else. Your call. How we handle data.