Saturday 15 March 2025
A team of researchers has developed a novel approach for estimating the energy consumption of deep neural networks (DNNs) during training, which could have significant implications for the development of artificial intelligence.
Currently, estimating the energy consumption of DNNs is a complex task, as it depends on various factors such as the type of device being used, the complexity of the network, and the data being processed. This makes it challenging to design efficient algorithms that minimize energy consumption while still achieving good performance.
The researchers have proposed a generic approach called THOR (Total Hardware Optimization using Regression), which uses Gaussian Process regression to model the energy consumption of DNNs during training. The approach is based on profiling the energy consumption of the device and fitting a Gaussian Process model to the data.
The team used five different deep neural network models, including LeNet-5 and ResNet, and trained them on various devices, including smartphones, development boards, and a Windows server. They found that their approach was able to accurately estimate the energy consumption of the DNNs during training, with an average mean absolute percentage error (MAPE) of 10%.
The results suggest that THOR could be used to develop more efficient algorithms for training DNNs on devices with limited power resources, such as smartphones. This could enable the development of more powerful AI systems that can be deployed in a wider range of environments.
One of the key advantages of THOR is its ability to adapt to different devices and models. The approach does not require any modifications to the device or the DNN model, making it easy to deploy and use. Additionally, THOR can handle complex energy consumption profiles, which makes it suitable for a wide range of applications.
The researchers also experimented with varying the number of sampled data points used in the Gaussian Process regression, finding that increasing the number of samples generally improved the accuracy of the estimates. However, they also found that there was a point of diminishing returns, beyond which additional samples did not significantly improve the results.
In addition to its potential applications in AI research, THOR could also have implications for the development of more energy-efficient computing systems. By accurately estimating the energy consumption of DNNs during training, developers can design algorithms that minimize power consumption while still achieving good performance.
Overall, the researchers’ approach offers a promising new way to estimate the energy consumption of deep neural networks during training, which could have significant implications for the development of artificial intelligence and more energy-efficient computing systems.
Cite this article: “Estimating Energy Consumption of Deep Neural Networks During Training with THOR”, The Science Archive, 2025.
Deep Neural Networks, Energy Consumption, Training, Gaussian Process Regression, Thor, Total Hardware Optimization Using Regression, Artificial Intelligence, Machine Learning, Energy Efficiency, Computing Systems.
