Thursday 27 March 2025
In a significant breakthrough in the field of machine learning, researchers have developed a novel approach to tackle the issue of catastrophic forgetting in federated continual learning. The problem of catastrophic forgetting arises when a model is trained on multiple tasks sequentially, and it forgets previously learned knowledge as new information is incorporated. This phenomenon can lead to a decline in performance on previous tasks, which can be detrimental in real-world applications.
The researchers’ solution, dubbed Accurate Forgetting for Federated Continual Learning (AF-FCL), employs a generative-replay mechanism that selectively retains relevant information from previous tasks while discarding biased features. The method leverages a normalizing flow model to estimate the credibility of previously learned knowledge and adaptively adjusts its influence on the current learning process.
In federated continual learning, multiple clients collaborate to learn a shared global model by sharing their local datasets. This approach enables efficient training on large-scale datasets while preserving data privacy. However, it also introduces new challenges, such as statistical heterogeneity across clients and potential label noise. AF-FCL addresses these issues by incorporating adaptive forgetting mechanisms that selectively retain knowledge from previous tasks.
The researchers evaluated AF-FCL on several benchmark datasets, including EMNIST-LTP, EMNIST-shuffle, CIFAR100, and ImageNet-subset. The results show that AF-FCL outperforms existing baselines in terms of accuracy and resistance to forgetting. For instance, on the EMNIST-LTP dataset, AF-FCL achieved an average accuracy of 47.5%, compared to 43.3% for the next best method.
AF-FCL’s performance is attributed to its ability to selectively retain relevant information from previous tasks while discarding biased features. The normalizing flow model enables accurate estimation of credibility and adaptively adjusts its influence on the current learning process. This approach effectively mitigates catastrophic forgetting, allowing the model to learn new tasks without compromising performance on previously learned ones.
The researchers also explored the impact of noisy clients on AF-FCL’s performance. They found that even in the presence of label noise, AF-FCL consistently outperformed baselines. This is a significant improvement over existing methods, which often suffer from catastrophic forgetting when faced with noisy data.
In addition to its technical merits, AF-FCL has important implications for real-world applications. Federated continual learning is a key enabler of edge AI, where devices and IoT sensors collaborate to learn shared models without compromising user privacy.
Cite this article: “Federated Continual Learning with Accurate Forgetting”, The Science Archive, 2025.
Machine Learning, Catastrophic Forgetting, Federated Continual Learning, Accurate Forgetting For Federated Continual Learning, Generative-Replay Mechanism, Normalizing Flow Model, Credibility Estimation, Adaptive Forgetting, Label Noise, Edge Ai.
