Thursday 13 March 2025
The noisy world of machine learning is about to get a whole lot quieter. A new paper has proposed a novel approach to dealing with label noise, where data is mislabeled or incorrect, which plagues many machine learning models.
Label noise can occur in various ways. For instance, human annotators may make mistakes when labeling data, or the labels themselves might be noisy due to variations in the way they were collected. This type of noise can significantly impact the performance and accuracy of machine learning models, making it difficult for them to learn from the data.
The researchers behind this new paper propose a method called Adaptive Estimation of Instance-Dependent In-Distribution and Out-of-Distribution Label Noise (AEON). AEON is designed to dynamically estimate the noise rates in both in-distribution (ID) and out-of-distribution (OOD) data, which are crucial for robust learning.
In-distribution data refers to instances that belong to the same class or category as the training data. Out-of-distribution data, on the other hand, includes instances from unseen classes or categories. AEON is capable of distinguishing between these two types of data and adjusting its noise estimation accordingly.
The method works by first identifying instances with high confidence in their labels. These instances are then used to train a neural network that can distinguish between ID and OOD data. The output of this network is then used to estimate the noise rates for each instance.
To combat label noise, AEON uses a novel sigmoid-based soft masking mechanism. This mechanism allows the model to adaptively weigh the importance of each instance based on its confidence in its own predictions. This ensures that noisy instances do not dominate the training process and negatively impact the model’s performance.
The researchers tested AEON on several benchmark datasets, including Clothing1M and mini-WebVision, which are known for their high levels of label noise. The results show that AEON significantly outperforms other state-of-the-art methods in terms of accuracy and robustness to noisy labels.
This research has significant implications for the field of machine learning. It provides a new approach to dealing with label noise, which is a pervasive problem in many real-world applications. By developing more robust models that can effectively handle noisy data, researchers can create more accurate and reliable systems that are better equipped to tackle complex tasks.
In addition to its practical significance, AEON also offers insights into the nature of label noise and how it affects machine learning models.
Cite this article: “Silencing the Noise: A Novel Approach to Dealing with Label Noise in Machine Learning”, The Science Archive, 2025.
Machine Learning, Label Noise, Adaptive Estimation, Instance-Dependent, Out-Of-Distribution, In-Distribution, Neural Network, Soft Masking, Sigmoid-Based, Robustness
