Self-Distillation Enables Continual Learning [pdf]

Computer Science > Machine Learning [Submitted on 27 Jan 2026] Title:Self-Distillation Enables Continual Learning View PDF HTML (experimental)Abstract:Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explici...