Friday 28 March 2025
Federated learning, a technique that enables multiple parties to share their data without revealing sensitive information, has taken a significant leap forward. Researchers have developed a new method, called Federated Locate-then-Edit Knowledge Editing (FLEKE), which allows clients to collaborate on updating large language models while preserving privacy and reducing computational costs.
The FLEKE system consists of two stages: editing and re-editing. In the editing stage, clients locally perform knowledge editing and upload mediator knowledge vectors (MKVs) to a central server. These MKVs contain information about the edits made by each client, which are then aggregated by the server. The re-editing stage involves retrieving relevant MKVs from the server based on cosine similarity, allowing clients to refine their edits.
One of the key challenges in federated learning is dealing with heterogeneous data across different clients. FLEKE addresses this issue by using clustering to select MKVs and re-editing conditions to improve the knowledge editing process. The system has been tested on two benchmark datasets, zsRE and COUNTERFACT, and has shown promising results.
The potential applications of FLEKE are vast. It could be used in a variety of fields, including natural language processing, computer vision, and recommender systems. For example, it could enable multiple hospitals to collaborate on updating their medical records while preserving patient privacy.
FLEKE’s success is due in part to its ability to balance the competing demands of accuracy, efficiency, and privacy. By using MKVs and cosine similarity, the system can accurately capture the edits made by each client while minimizing the amount of data that needs to be shared.
The researchers behind FLEKE have also developed a novel framework called FedEdit, which enables clients to edit their knowledge models in a decentralized manner. This framework consists of two stages: editing and re-editing, similar to FLEKE. However, FedEdit uses a different approach to select MKVs and re-editing conditions.
The future of FLEKE is bright, with many potential applications waiting to be explored. As the technology continues to evolve, it’s likely that we’ll see even more innovative uses for federated learning in a variety of fields.
Cite this article: “Federated Knowledge Editing: A New Approach to Secure and Efficient Collaborative Learning”, The Science Archive, 2025.
Federated Learning, Knowledge Editing, Natural Language Processing, Computer Vision, Recommender Systems, Patient Privacy, Medical Records, Accuracy, Efficiency, Privacy
