PiSSA (Principal Singular values and Singular vectors Adaptation) is a PEFT method for LLMs that optimizes the essential singular values and vectors while freezing the noisy parts, in contrast to LoRA which freezes the original matrix. The technique achieves faster convergence and better performance across benchmarks like GSM8K, surpassing LoRA by up to 5.16% on Mistral-7B. It also integrates with quantization (QLoRA) to reduce quantization error by 18.97% in LLaMA 2-7B and has been merged into the HuggingFace peft library.