Thursday 10 April 2025
The world of smart contracts has long been plagued by numerical defects, which can lead to unexpected and often disastrous consequences. These errors can arise from a variety of sources, including integer overflows, division issues, and precision loss, making it difficult for developers to create reliable and secure contracts.
In an effort to address this issue, researchers have developed a tool called NumScout, which utilizes large language models (LLMs) to detect five new types of numerical defects in smart contracts. These defects include div in path, operator order issue, minor amount retention, exchange problem, and precision loss trend.
The tool works by first pruning functions unrelated to numerical operations using GPT-4o, a large language model. This allows NumScout to quickly enter the target function and improve runtime speed by 28.4%. The tool then performs symbolic execution at the bytecode level, combined with source code features for analysis. It constructs and analyzes expression operator order trees, extracts conditional statements of comparison from the bytecode, and token flows.
NumScout’s detection algorithm is designed to identify defects in real-world smart contracts, including those used by popular decentralized applications (DApps). The tool has been tested on 6,617 contracts and was able to detect 1,774 containing at least one of the five defined defects. It achieved an overall precision of 89.7%, indicating a high degree of accuracy.
The detection of these numerical defects is critical in the world of smart contracts, as they can have significant financial implications. For example, a division issue could result in a contract incorrectly calculating the amount of tokens to be transferred, leading to a loss of funds for users.
The development of NumScout highlights the importance of using LLMs in software engineering tasks. The tool’s ability to quickly and accurately detect numerical defects demonstrates the potential for LLMs to improve the reliability and security of smart contracts.
In addition to its use in detecting numerical defects, NumScout also has the potential to be used as a educational tool. By analyzing the source code and bytecode of smart contracts, developers can gain a better understanding of how these defects occur and how they can be prevented.
Overall, the development of NumScout represents an important step forward in the field of smart contract security. The tool’s ability to detect numerical defects quickly and accurately makes it a valuable asset for developers looking to create reliable and secure contracts.
Cite this article: “Uncovering Hidden Faults: A Large-Scale Study of Numerical Defects in Smart Contracts Using LLM-Based Pruning and Symbolic Execution”, The Science Archive, 2025.
Smart Contracts, Numerical Defects, Large Language Models, Llms, Software Engineering, Security, Precision Loss, Integer Overflows, Div In Path, Operator Order Issue
