jianghoucheng/AlphaEdit
AlphaEdit implements null-space constrained parameter projection for editing large language models while preserving their existing knowledge.

AlphaEdit is a model editing technique for language models that projects parameter perturbations onto the null space of key matrices to minimize disruption to preserved knowledge. It removes output error related to existing knowledge from the editing objective, allowing the model to focus on knowledge updates without trade-off. The method maintains invariant hidden representation distributions after edits, enabling post-edited LLMs to handle both knowledge update and preservation simultaneously. The codebase includes experiments for editing models like Llama3-8B-instruct on counterfact datasets.