kmeng01/memit
A method for batch-editing thousands of factual associations in transformer models like GPT-J.

MEMIT implements a technique for efficiently editing multiple facts in a transformer model’s internal knowledge representation at once. It allows users to specify rewrite requests with prompts, subjects, and new target values, which are then applied to the model’s MLP layers via rank-one weight updates. The evaluation suite tests methods on datasets like ZsRE and can handle up to 10,000 simultaneous edits while maintaining model performance on unrelated tasks. This work originates from a 2023 ICLR publication.