Sunday 02 March 2025
A new approach has been developed to compress deep neural networks, allowing them to be stored and transmitted more efficiently without sacrificing their performance. The technique, known as multi-objective uniform binning (MO-UB), uses a combination of clustering and optimization algorithms to reduce the number of weights required by a network while maintaining its accuracy.
The problem of compressing neural networks is becoming increasingly important as they are being used in a wider range of applications, from self-driving cars to medical devices. However, these models can be massive, with millions or even billions of parameters that require significant storage and computational resources.
MO-UB addresses this issue by dividing the network’s weights into small clusters, called bins, which are then quantized using a uniform distribution. This reduces the number of unique values required, making it possible to store and transmit the network more efficiently.
The approach is particularly effective when combined with two additional techniques: iterative merge and Huffman coding. Iterative merge involves repeatedly merging nearby bins until the desired level of compression is achieved, while Huffman coding assigns shorter codes to more frequently occurring values in the network.
In tests using a range of neural networks, including ResNet-18, ResNet-34, and ViT-B-16, MO-UB was shown to achieve significant reductions in memory usage without sacrificing accuracy. For example, on the CIFAR-10 dataset, MO-UB reduced the number of parameters required by ResNet-18 from 11.2 million to just 8.0 million while maintaining a test F1 score of 94.6%.
The approach also performed well on more complex datasets such as ImageNet-1K, where it was able to compress AlexNet and ViT-B-16 models without compromising their performance.
While MO-UB is not the first technique to be developed for compressing neural networks, its combination of clustering, optimization, and encoding algorithms makes it a powerful tool for reducing the size and computational requirements of these models. As the use of neural networks continues to grow, the need for efficient compression techniques like MO-UB will only become more pressing.
The researchers behind MO-UB are now working on further improving the technique, including exploring its potential applications in areas such as autonomous vehicles and medical imaging. With the ability to compress neural networks more efficiently, these technologies could soon be deployed in a wider range of devices and scenarios, leading to new possibilities for artificial intelligence and machine learning.
Cite this article: “Efficient Neural Network Compression with Multi-Objective Uniform Binning”, The Science Archive, 2025.
Neural Networks, Compression, Deep Learning, Multi-Objective Uniform Binning, Clustering, Optimization, Quantization, Huffman Coding, Iterative Merge, Machine Learning
