Catastrophic forgetting in LLM fine-tuning is the loss of abilities a model already had when it is trained on new data. It is worst when a model is fine-tuned in sequence, round after round, which is what continual learning asks of it. Replay, regularisation and orthogonal subspace methods reduce it, and an evaluation gate with a regression set decides whether a new version replaces the current one.
This post covers what catastrophic forgetting is, why sequential fine-tuning causes it, the published mitigations with their sources, and how a release gate keeps a forgetful version out of production.
What is catastrophic forgetting in LLMs?
Catastrophic forgetting is a sharp drop in a model's performance on earlier tasks after it is trained on new ones. IBM's explainer on catastrophic forgetting dates the first observation to Michael McCloskey and Neal J. Cohen in 1989, under the name catastrophic interference. In an LLM it shows up as a fine-tuned model that does the new task well and has lost something it used to do: arithmetic, a language, a safety behaviour, or an output format another system depends on.
It has been measured in modern LLMs. Luo et al.'s empirical study of forgetting during continual fine-tuning (2023) found it across models from 1 billion to 7 billion parameters during continual instruction tuning, and within that range the larger models forgot more.
Why does fine-tuning cause catastrophic forgetting?
Fine-tuning updates weights that every task shares, and its training objective measures only the new data. Gradient descent moves the weights in whatever direction lowers the loss on the fine-tuning set, with no term for the abilities the model had before, so an update that helps the new task can overwrite what an earlier task relied on. A high learning rate, many epochs and a narrow dataset all make the overwrite larger.
Sequential fine-tuning compounds it. Each round starts from the previous round's model and moves it again, so the losses add up. Red Hat's OSFT explainer (28 July 2026) cites a case from the Sculpting Subspaces paper: a model that solved 29% of grade-school math problems fell to 2% after fine-tuning on eight new tasks. Google Research's sleep framework (27 October 2026) reports the same pattern, with supervised fine-tuning baselines suffering severe forgetting in sequential translation tasks.
That is why forgetting matters more for continual learning for AI agents than for a single fine-tune. An agent's model retrained on its traffic every few weeks is a sequence of fine-tunes, and each one is a fresh chance to lose something.
How do you prevent catastrophic forgetting during fine-tuning?
The published mitigations fall into five families, and each one reduces forgetting by limiting how far an update can move the abilities the model already has. A measurement still has to confirm it worked.
| Method | How it works | Source |
|---|---|---|
| Replay | Mixes earlier training data into each new round | Scialom et al., 2022 |
| Regularisation | Slows learning on weights important to earlier tasks | Kirkpatrick et al., 2016 |
| Parameter-efficient training | Trains small adapters on top of frozen weights | Biderman et al., 2024 |
| Orthogonal subspaces | Keeps updates away from directions earlier tasks use | Wang et al., 2023; Nayak et al., 2025 |
| Added capacity | Writes new knowledge into newly added parameters | Google Research, 2026 |
Replay is the cheapest to run on production traffic, and the families can be combined, for example replay with LoRA.
Replay. Replay, also called rehearsal, keeps a buffer of earlier training examples and mixes them into each new round, so the model keeps practising what it learned before. In "Fine-tuned Language Models are Continual Learners" (2022), Scialom et al. trained a T0 model on new tasks in sequence with a memory buffer of 1% of the earlier data and kept its performance on the earlier tasks. For a model retrained on agent traffic, the approved examples from earlier rounds are already on hand, which makes replay nearly free.
Regularisation. Elastic weight consolidation, from Kirkpatrick et al.'s "Overcoming catastrophic forgetting in neural networks" (2016), adds a penalty for changing the weights that matter most to earlier tasks, so learning slows exactly where forgetting would do damage.
Parameter-efficient training. LoRA trains small low-rank adapters and leaves the base weights untouched. Biderman et al. found in "LoRA Learns Less and Forgets Less" (2024) that on code and math, LoRA learned less of the target domain than full fine-tuning and kept more of the base model's performance outside it, and that it reduced forgetting more than weight decay or dropout.
Orthogonal subspaces. O-LoRA (Wang et al., 2023) learns each new task in a low-rank subspace kept orthogonal to the earlier ones, and needs no stored data for replay. Sculpting Subspaces (Nayak et al., 2025) applies the idea to full fine-tuning: it finds the weight directions that earlier tasks depend on and constrains updates to be orthogonal to them. Red Hat calls the method Orthogonal Subspace Fine-Tuning (OSFT), and its explainer reports that on TRACE, a benchmark of eight tasks learned in sequence, a LLaMA-2-7B-Chat model reached 48.4% average accuracy with OSFT against 23.0% with standard fine-tuning, on the same model and data.
Added capacity. Some methods give new knowledge new parameters to live in. Google Research's sleep framework adds low-rank experts to a mixture-of-experts model during an offline phase and consolidates new information there, leaving older parameters in place. These methods are the newest of the five and mostly at research stage as of October 2026.
Conservative training settings help alongside all five: a lower learning rate and fewer epochs make each update smaller.
Does LoRA prevent catastrophic forgetting?
LoRA reduces catastrophic forgetting and leaves some of it in place. Biderman et al. found it kept more of the base model's performance outside the target domain than full fine-tuning, and paid for that by learning less of the target domain. Red Hat's explainer makes the same point from the other side: LoRA changes less of the model, and has no mechanism that protects a particular capability. Across a sequence of training rounds, combine LoRA with replay and measure every version. Our glossary entry on LoRA covers how the adapters work.
How does an evaluation gate catch catastrophic forgetting?
An evaluation gate scores every new version against a fixed set of cases before it takes traffic, and the regression set inside that gate is what catches forgetting. The mitigations lower the odds of forgetting; the gate shows whether it happened.
A regression set for an agent's model holds four kinds of case:
- Cases the current model gets right on the agent's main task.
- Earlier fixes. The cases each previous round was trained to fix, so a later round cannot undo them unseen.
- Behaviour other systems depend on. Output schemas, tool-call formats and refusals.
- Unrelated cases. A few cases outside the area being fixed, which Arize's guide to continual learning also recommends, to catch a regression the targeted test set would miss.
The gate rule: the new version must hold the quality bar on the task set and stay within a noise margin of the current model on the regression set, with both thresholds set before the new version is scored. If it fails either test, the current model stays in service, and the failed cases go back into the next round's training data. Keeping the previous version deployable is what makes this safe; our post on model versioning, rollback and drift covers the mechanics.
The noise margin needs measuring. In our QMSum study (arXiv 2609.25028), changing only the training seed moved validation ROUGE-1 by 1.59 points across two runs, and five checkpoints from one run spanned 1.7 points. A drop smaller than that spread can be chance. Set the margin too tight and chance drops block good versions; set it too loose and real forgetting gets through.
Is catastrophic forgetting still a problem in 2026?
Yes. LangChain's Harrison Chase calls it a central challenge of continual learning at the model layer and an open research problem in "Continual learning for AI agents" (5 April 2026), and Google Research's sleep framework post (27 October 2026) calls it a fundamental issue. Mitigations have improved, with OSFT and capacity expansion among the newest, and no method makes forgetting impossible. Production systems therefore pair mitigation with measurement.
How does the Ertas endpoint handle forgetting?
The Ertas adaptive model endpoint keeps the current model in service until a new version holds the bar. It serves a specialist model trained on your agents' work through one API, and it learns continually from your agents' traffic: each new version trains in the background and is scored against your evaluations, including regression cases the current model already handles. You approve the training examples and every release. If a new version falls short, the current model stays in service.
It runs Ertas-managed, in your cloud or on-premise. The endpoint is currently invitation only, and we are developing a more streamlined version for wider access.
Frequently Asked Questions
What is catastrophic forgetting in simple terms?
A model learns something new and loses something it already knew. In LLM fine-tuning, a model trained on a new task can get worse at tasks it handled well before, because the same weights store both.
Does fine-tuning make a model worse at other tasks?
It can. A narrow fine-tune with a high learning rate and many epochs often degrades abilities outside the target task. Replay, LoRA, conservative settings and an evaluation that covers more than the target task keep the damage small and visible.
How much old data should be replayed?
A small share can be enough. Scialom et al. kept earlier-task performance with a buffer of 1% of the earlier training data. Start small, and raise the share if the regression set shows losses.
Can catastrophic forgetting be undone?
Yes, if the previous version was kept. Roll back to it, then retrain from that version with replay of the examples the forgetful version lost. Keeping every released version and its evaluation results is what makes this possible.
Does RAG cause catastrophic forgetting?
No. Retrieval adds information to the prompt and leaves the model's weights unchanged, so nothing the model learned is overwritten. That is one reason facts that change often belong in retrieval, with training reserved for behaviour, format and task skill.
The Short Version
Catastrophic forgetting is the price of updating shared weights, and sequential fine-tuning pays it round after round. Replay, regularisation, LoRA and orthogonal subspace methods such as OSFT reduce it. A regression set in the release gate shows whether a new version forgot anything, and the current model keeps serving until a version passes.
Sources, read 29 October 2026: IBM, "What is catastrophic forgetting?"; Red Hat Developer, "OSFT explained" (28 July 2026); Google Research, "The sleep framework" (27 October 2026); LangChain, "Continual learning for AI agents" (5 April 2026); Arize, "Continual learning for AI agents and LLM systems" (last reviewed 16 August 2026); papers on arXiv by Luo et al. (2308.08747), Scialom et al. (2205.12393), Kirkpatrick et al. (1612.00796), Biderman et al. (2405.09673), Wang et al. (2310.14152) and Nayak et al. (2504.07097). Our research: the QMSum study.