Tuesday 08 April 2025
Researchers have made significant strides in developing a new approach to Federated Learning (FL), a technique that enables multiple devices or organizations to collaborate on training artificial intelligence models without sharing their data. The innovation, dubbed FedZGE, has been shown to be more effective than existing methods in certain scenarios.
The key challenge in FL is that each device or organization may have its own unique dataset, which can lead to a lack of diversity and accuracy in the trained model. To address this issue, FedZGE employs a generator network that produces synthetic data samples, allowing devices to share knowledge without revealing their individual datasets. This approach has several advantages over traditional methods.
Firstly, FedZGE reduces the communication overhead between devices, as only the generated synthetic data is exchanged rather than entire datasets. This can lead to significant bandwidth savings and reduced energy consumption. Secondly, the generator network helps to improve the diversity of the trained model by introducing new patterns and features that may not be present in individual datasets.
The researchers tested FedZGE on several popular image classification benchmarks, including CIFAR-10 and CIFAR-100, using different architectures such as ResNet-18 and ResNet-50. The results showed that FedZGE consistently outperformed existing FL methods in terms of test accuracy, while also reducing the communication overhead.
The team also explored the effects of three scaling factors on the performance of FedZGE. These factors control the contribution of adversarial loss, diversity loss, and information entropy loss to the training of the generator network. The results revealed that the values of these scaling factors have a significant impact on the accuracy of the trained model.
For instance, increasing the value of the adversarial loss factor can improve the performance of FedZGE, but too high a value can lead to overfitting. Similarly, increasing the value of the diversity loss factor can enhance the diversity of the generated synthetic data, but too high a value can result in decreased accuracy.
The researchers believe that their approach has significant potential for real-world applications, particularly in industries where data sharing is restricted or sensitive information needs to be protected. For example, healthcare organizations may use FedZGE to collaborate on developing AI-powered diagnostic tools without sharing patient data.
Overall, the development of FedZGE represents an important step towards more efficient and effective Federated Learning. By reducing communication overhead and improving diversity, this approach has the potential to accelerate the adoption of FL in various industries and domains.
Cite this article: “Data-Free Federated Learning via Zeroth-Order Gradient Estimation: A Scalable and Efficient Approach to Heterogeneous Distributed Machine Learning”, The Science Archive, 2025.
Federated Learning, Synthetic Data, Generator Network, Communication Overhead, Artificial Intelligence, Image Classification, Resnet-18, Resnet-50, Adversarial Loss, Diversity Loss.
