Friday 28 March 2025
Researchers have been working on developing artificial intelligence that can learn and reason like humans, but they’ve hit a roadblock when it comes to robustness under noisy observations. In other words, AI models are great at processing clean data, but when faced with errors or inconsistencies in the data, they tend to break down.
To tackle this problem, scientists have been testing large language models on various tasks that require reasoning and abstraction. These models were given a set of examples, some of which contained noise, and asked to generate rules that explained how the input-output pairs worked.
The results showed that while these AI models are capable of learning complex patterns, they’re not very good at handling noisy data. In fact, when presented with incorrect information, many of the models completely failed to produce a correct rule or even crashed.
One of the tasks tested was arithmetic, where the model had to generate rules for basic math operations like addition and subtraction. However, when the input numbers were expressed in bases other than 10 (like base-7 or base-9), the models struggled to understand what was going on. They often produced incorrect answers or got stuck in an infinite loop.
Another task was cryptography, where the model had to decipher encrypted text using various ciphers like Caesar and Atbash. However, even with simple encryption techniques, the models often failed to produce the correct decryption rules.
So why do these AI models struggle with noisy data? One reason is that they’re trained on large amounts of clean data, which makes them sensitive to errors in the training set. Another reason is that they’re not designed to handle uncertainty or ambiguity in the data.
To overcome this limitation, researchers are exploring new approaches like iterative refinement and feedback mechanisms. These methods involve gradually refining the model’s output based on additional information or corrections from a human expert.
In one experiment, a sample-steered rule refinement method was used to improve the model’s performance. This approach involved providing the model with initial rules and then iteratively refining them based on feedback from the user. The results showed that this approach significantly improved the model’s ability to handle noisy data and generate accurate rules.
These findings have important implications for the development of artificial intelligence. As AI models become increasingly sophisticated, they’ll need to be able to handle real-world data with all its imperfections. By developing more robust and flexible models, researchers can create AI systems that are better equipped to deal with uncertainty and ambiguity.
Cite this article: “Artificial Intelligences Achilles Heel: Handling Noisy Data”, The Science Archive, 2025.
Artificial Intelligence, Noisy Data, Machine Learning, Robustness, Language Models, Reasoning, Abstraction, Arithmetic, Cryptography, Uncertainty, Ambiguity
