kmeng01/rome
A research implementation of Rank-One Model Editing for modifying factual knowledge in GPT-2 XL and GPT-J language models.

This repository implements ROME, a technique for locating and editing factual associations stored in auto-regressive transformer models. It provides causal tracing tools to identify internal representations of specific facts in GPT models, then applies rank-one updates to the MLP layers to alter stored knowledge. The framework supports GPT-2 XL (1.5B) and GPT-J (6B), includes evaluation on the CounterFact dataset, and offers notebook demonstrations for both causal tracing and model editing.