Monday 10 March 2025
Researchers have developed a novel approach to testing refactoring engines, which are critical tools used in software development. These engines analyze and transform existing code to improve its structure and maintainability. However, they can be prone to bugs and errors, leading to issues that are difficult to detect.
The new method, called RETESTER, leverages historical bug reports and error-prone input program characteristics to generate diverse test cases for refactoring engines. This approach is designed to identify previously unknown bugs and improve the reliability of these critical tools.
RETESTER starts by collecting a dataset of known bug reports from popular refactoring engines like Eclipse and IntelliJ IDEA. The researchers then analyzed the bug reports to identify common patterns and characteristics that are associated with errors. They found that certain types of input programs, such as those containing anonymous classes or lambda expressions, tend to be more prone to bugs.
Using this information, RETESTER generates a wide range of test cases that incorporate these error-prone characteristics. The tool uses a large language model (LLM) to propose potential refactoring operations and then applies these operations to the input programs. This process is repeated multiple times to generate a diverse set of test cases.
The researchers tested RETESTER on two popular refactoring engines, Eclipse and IntelliJ IDEA, and found that it successfully identified 18 new bugs in each engine. Seven of these bugs were confirmed by the developers of the engines, while three have already been fixed.
One of the key benefits of RETESTER is its ability to identify bugs that are difficult or impossible to detect using traditional testing methods. The tool’s use of historical bug reports and error-prone input program characteristics allows it to target specific areas where errors are more likely to occur.
The development of RETESTER highlights the need for more sophisticated testing approaches in software development. As refactoring engines become increasingly complex, they require robust testing tools that can identify even the most subtle errors. By leveraging LLMs and historical bug reports, researchers like those behind RETESTER are helping to advance the field of software testing and improve the reliability of critical tools.
The implications of this research extend beyond the realm of refactoring engines. As AI-powered testing tools become more prevalent in software development, they will require sophisticated approaches like RETESTER to identify errors and improve overall system reliability. By developing more effective testing methods, researchers can help ensure that AI-powered systems are trustworthy and reliable for a wide range of applications.
Cite this article: “Enhancing Refactoring Engine Testing with RETESTER: A Novel Approach to Identifying Bugs and Improving Reliability”, The Science Archive, 2025.
Refactoring Engines, Software Development, Testing, Bugs, Errors, Reliability, Language Models, Machine Learning, Artificial Intelligence, Code Analysis
