Problem

Most existing machine unlearning methods are designed for single-phase unlearning, where data deletion occurs only once. However, real-world applications often require continual unlearning, where deletion requests arise repeatedly over time. In this more realistic setting, two critical issues emerge: Knowledge Erosion, which is the progressive degradation of accuracy on retained data as unlearning phases proceed, and Forgetting Reversal, where data previously successfully forgotten becomes recognizable again in later unlearning phases. These phenomena threaten both the reliability of AI models and their compliance with privacy regulations like the "right to be forgotten."

Method

The authors propose SAFER (StAbility-preserving Forgetting with Effective Regularization), a continual unlearning framework designed to tackle Knowledge Erosion and Forgetting Reversal. SAFER achieves this through two primary optimization objectives. First, to mitigate knowledge erosion, it enhances the clusterability of retain data representations by reducing intra-class variation and increasing inter-class separation. This is achieved using a class-conditioned latent-variable module that ensures feature smoothness and consistency, with regularization terms promoting compact intra-class distributions and well-separated class boundaries. Second, to prevent forgetting reversal, SAFER enforces negative logit margins for forget data. For each forget sample, it suppresses the logit of its original class and distributes probability mass to retain classes using a random target vector, ensuring that forgotten samples remain suboptimal and unrecognizable in the decision space across multiple unlearning phases.

Results

Extensive experiments were conducted on diverse datasets, including CIFAR-100, VGGFace2 (class-aligned), and MUFAC (class-misaligned), using various backbone models like ResNet-18, ResNet-50, and Vision Transformer (ViT). SAFER demonstrated superior performance compared to existing state-of-the-art baselines. Specifically, SAFER showed virtually no knowledge erosion, maintaining stable accuracy on retain data across multiple unlearning phases. It also effectively mitigated forgetting reversal, with accuracy on forget and forgotten data remaining close to zero in class-aligned settings. While some drift towards zero was observed in class-misaligned scenarios, SAFER remained robust. Furthermore, SAFER achieved Membership Inference Attack (MIA) scores similar to models retrained from scratch, indicating strong unlearning efficacy. Analysis of the Davies–Bouldin Index (DBI) confirmed that SAFER preserves the stability and clusterability of retain data representations. Although not the fastest, SAFER's computational efficiency was comparable to most fine-tuning methods, making it practical for real-world applications. Ablation studies highlighted the critical contribution of both clusterability enhancement and negative unlearning margin enforcement to SAFER's robust performance.

Implications

The SAFER framework offers a significant advancement in machine unlearning by providing a robust and practically deployable solution for scenarios requiring continuous data deletion. By effectively addressing knowledge erosion and forgetting reversal, SAFER ensures that AI systems can reliably comply with privacy regulations, such as the "right to be forgotten," without compromising their overall utility and performance over their lifecycle. This work contributes to building more trustworthy and adaptable AI models that can dynamically manage and update learned knowledge in response to evolving data privacy and ethical requirements, fostering greater confidence in AI deployment in sensitive applications.