Thursday 20 March 2025
A team of researchers has made a significant breakthrough in the field of software engineering, developing an innovative approach to fault localization that leverages the power of large language models.
Fault localization is a critical task in software development, as it enables developers to quickly and accurately identify the source of errors or bugs in their code. This process typically involves analyzing logs, testing and debugging, and manual review, which can be time-consuming and labor-intensive. The new approach, dubbed COSMosFL, aims to streamline this process by harnessing the capabilities of large language models (LLMs) to automatically pinpoint faults.
The researchers began by evaluating the performance of various LLMs, including GPT-3.5, Llama 3, Mistral NeMo, and Qwen2.5-Coder, on a range of software engineering tasks. They found that each model exhibited unique strengths and weaknesses, with some performing better than others on specific tasks.
Building on this insight, the team developed COSMosFL, an ensemble approach that combines the predictions of multiple LLMs to achieve more accurate fault localization results. By weighting the outputs from different models, COSMosFL can identify faults that individual models might miss, resulting in improved overall performance.
To test the effectiveness of COSMosFL, the researchers conducted a series of experiments using the Defects4J dataset, a widely used benchmark for evaluating software engineering techniques. The results were impressive: COSMosFL outperformed single-model approaches and achieved comparable or better results than existing ensemble methods.
One of the key advantages of COSMosFL is its ability to adapt to different software development scenarios. By incorporating multiple LLMs with diverse strengths, the approach can handle a wide range of fault types and complexity levels. This flexibility makes it an attractive solution for developers working on large-scale projects or those dealing with complex codebases.
Another significant benefit of COSMosFL is its potential to reduce the environmental impact of software development. Large language models require significant computational resources and energy consumption, which can contribute to greenhouse gas emissions. By leveraging multiple LLMs in a single approach, COSMosFL may help minimize this environmental footprint while still delivering accurate results.
The implications of COSMosFL are far-reaching, with potential applications extending beyond software engineering. The approach could be adapted for use in other domains where fault localization is critical, such as medical diagnosis or cybersecurity analysis.
Cite this article: “COSMosFL: A Novel Ensemble Approach to Fault Localization Leveraging Large Language Models”, The Science Archive, 2025.
Software Engineering, Large Language Models, Fault Localization, Ensemble Approach, Defects4J Dataset, Benchmarking, Computational Resources, Energy Consumption, Environmental Impact, Software Development.
