Friday 28 February 2025
Researchers have been working on a solution to make large language models, like those used in chatbots and virtual assistants, more secure without sacrificing their ability to learn and adapt. This is crucial because these models are designed to understand and respond to vast amounts of user input, making them vulnerable to data breaches and unauthorized access.
The problem lies in the fact that these models rely on complex algorithms and massive datasets, which can be difficult to protect. Moreover, when users interact with these systems, their personal data and conversations are stored and processed by the servers hosting the models. This raises concerns about privacy and security.
To address this issue, scientists have developed a method called Fully Homomorphic Encryption (FHE), which allows for computations to be performed on encrypted data without decrypting it first. This means that sensitive information remains confidential while still enabling the model to learn from user input.
The researchers combined FHE with another technique called Parameter-Efficient Fine-Tuning, which involves reducing the size of the model’s parameters while maintaining its ability to perform tasks. By combining these two methods, they created a secure inference scheme for large language models that protects both user-side inputs and server-side private weights.
One key innovation is the separation of the model into public and private parts. The public part is accessible and can be deployed on client devices, while the private part remains encrypted and only processed on the server-side. This approach significantly reduces the overhead of computing on private data.
The team also developed a transformation method called Private Linear Layer (PLL), which converts linear layers in the model to preserve their original functionality while providing security against model extraction attacks. These attacks involve attempting to reverse-engineer the model’s weights and biases to access sensitive information.
To test their approach, the researchers fine-tuned a popular language model using LoRA, a technique that reduces the size of the model’s parameters. They then applied their secure inference scheme to classify sensitive data, achieving an accuracy rate of around 97%.
The results show that this method is not only more secure but also efficient, with the inference process taking less than 1.61 seconds per token when processing large amounts of text.
This breakthrough has significant implications for the development of AI-powered systems that require both security and performance. By making it possible to perform computations on encrypted data without compromising confidentiality, this research opens up new possibilities for secure communication and data analysis in various fields, from healthcare and finance to education and entertainment.
Cite this article: “Secure Inference Scheme for Large Language Models”, The Science Archive, 2025.
Large Language Models, Fully Homomorphic Encryption, Parameter-Efficient Fine-Tuning, Secure Inference Scheme, Public And Private Parts, Private Linear Layer, Model Extraction Attacks, Lora, Confidentiality, Ai-Secured Systems
