Unlearning Isn’t Forgetting: Revealing Hidden Leakage in Class Unlearning Evaluations

We identify a critical gap in how class unlearning is evaluated, demonstrating that overlooking the underlying class geometry can cause information leakage about the forgotten class. We introduce a Class Membership Inference Attack (CMIA) that exploits neighboring class probabilities to detect unlearned samples, showing that existing methods remain vulnerable. To address this, we propose Tilted REWeighting (TREW), which uses inter-class similarity estimation to approximate how a retrained model would distribute probabilities across the remaining classes. On CIFAR-10, TREW reduces performance gaps by approximately 19-46% compared to state-of-the-art methods.

Recommended citation: Ebrahimpour-Boroojeny, A., Wang, Y., & Sundaram, H. (2026). Unlearning Isn't Forgetting: Revealing Hidden Leakage in Class Unlearning Evaluations. In Forty-third International Conference on Machine Learning (ICML 2026).
Download Paper