Catastrophic forgetting is a phenomenon in which a neural network loses capabilities it acquired earlier while learning a new task. The cause is the nature of distributed representation: knowledge is not stored in separate pigeonholes but spread across the same weights, so gradient descent optimizing the new task overwrites those weights without scruple. The problem is especially acute in incremental learning and in the fine-tuning of large language models, where intensive training on a narrow domain can damage general linguistic and reasoning abilities or bypass safety behaviour instilled by RLHF. Several defensive strategies exist. Data replay mixes samples from the old task into the new one. Regularization methods such as elastic weight consolidation identify the weights important for the earlier task and penalize changes to them. Architectural approaches give the new knowledge its own capacity – which is exactly how LoRA works, leaving the base weights frozen and training only a small adapter. The most effective practical measure remains a low learning rate and a short fine-tune.
Imagine an excellent pianist who decides to spend a year practising jazz intensively. He comes back, sits down at the piano and finds he cannot remember the Chopin he played for ten years. He did not forget it the way you forget a phone number – his fingers and musical instincts were rebuilt around the new style and overwrote the old one. Had he played at least a little Chopin during that jazz year, it would not have happened. Better still would have been to cultivate a separate jazz mode he could switch on and off, leaving the foundation untouched.