Thursday 08 May 2025
Researchers have devised a novel attack that can compromise federated learning, a technique used to train AI models on decentralized data without sharing it directly. The method, called COLLAPOIS, exploits the differences in data distribution among clients and can cause even a small number of compromised clients to infect the global model.
Federated learning is designed to allow multiple organizations or individuals to contribute their data to a shared goal, such as improving medical diagnosis or natural language processing. However, this approach has been shown to be vulnerable to attacks, where malicious actors can manipulate the training process to inject backdoors into the model.
COLLAPOIS works by identifying and targeting clients with diverse data distributions, which are more likely to have unique patterns that can be exploited. The attacker then creates a Trojaned model that is designed to mimic the behavior of the legitimate global model, but with subtle differences that allow it to perform poorly on benign data while still achieving high accuracy on malicious input.
The attack is particularly effective when the compromised clients are selected based on their similarity to the auxiliary data used to train the Trojaned model. This can be achieved by analyzing the distribution of the data and selecting clients that have similar patterns or characteristics.
Experiments conducted on two popular datasets, Sentiment and FEMNIST, demonstrated the effectiveness of COLLAPOIS in compromising federated learning. The results showed that even with a small number of compromised clients, the attack can still achieve high success rates, making it challenging for defenses to detect.
The findings highlight the need for robust security measures to protect against such attacks. Researchers have proposed various defense mechanisms, including data poisoning detection and model smoothing, but these methods may not be effective against sophisticated attacks like COLLAPOIS.
The vulnerability of federated learning to attacks like COLLAPOIS underscores the importance of developing more secure and resilient AI systems. As the use of decentralized data becomes increasingly prevalent, it is essential to ensure that the training process can withstand malicious attempts to compromise the integrity of the model.
The researchers’ work provides a sobering reminder of the ongoing challenges in securing AI models against attacks. As the field continues to evolve, it is crucial to develop innovative solutions that can detect and mitigate such threats, ensuring that AI systems are not only powerful but also trustworthy.
Cite this article: “COLLAPOIS: A Novel Attack on Federated Learning”, The Science Archive, 2025.
Federated Learning, Ai, Attacks, Collapois, Trojaned Model, Data Distribution, Compromised Clients, Sentiment Dataset, Femnist Dataset, Security Measures
