Catastrophic Forgetting

Catastrophic forgetting describes the tendency of a neural network to lose proficiency on tasks it has previously mastered once it is further trained on new data or objectives. In conventional supervised training, model parameters are adjusted to reduce error on the current dataset; when a subsequent dataset arrives that emphasizes different patterns, these adjustments can overwrite the weight configurations that encoded earlier knowledge, leading to a sharp drop in performance on the original tasks.

The phenomenon matters because many real‑world AI systems must evolve over time without sacrificing what they have already learned. Applications such as personal assistants, autonomous robots, recommendation engines, and any service that updates its model continually face the risk that fresh updates erase past capabilities, forcing costly retraining from scratch or resulting in erratic behavior.

Catastrophic forgetting appears wherever a single parameterised model is expected to handle a sequence of distinct learning episodes—known as continual or lifelong learning settings. It also surfaces in transfer learning pipelines, multi‑task training regimes that are updated incrementally, and reinforcement‑learning agents that must adapt to new environments while retaining prior policies.