Friday 21 March 2025
The quest for efficient and accurate requirements traceability has long been a challenge in software development. With the increasing complexity of modern systems, ensuring that software engineers can identify and link specific requirements to their corresponding implementation has become a crucial task. In recent years, researchers have explored various approaches to tackle this problem, from natural language processing (NLP) techniques to machine learning algorithms.
A new study published in a leading computer science journal presents an innovative approach to requirements traceability using large language models (LLMs). The authors propose two automated solutions based on LLMs to predict links between software requirements and legal provisions. These solutions leverage the power of LLMs to analyze technical requirements against target artifacts, such as regulations or standards.
The first solution, dubbed Kashιf, employs sentence transformers and semantic similarity to identify trace links. Kashιf relies on a classifier trained on a large dataset of requirements and legal provisions to predict potential linkages between them. The study finds that Kashιf can accurately identify trace links with an average recall of approximately 67%, outperforming a baseline classifier by a substantial margin.
However, when tested on unseen, more complex requirements documents related to the European General Data Protection Regulation (GDPR), Kashιf’s performance deteriorates significantly. In this case, the solution yields an average recall of only 15%. This suggests that simple classification-based approaches may not be sufficient for tackling the intricacies of real-world requirements traceability.
The authors then turn their attention to a second solution, built upon the Rice framework for prompt engineering. This approach leverages the power of LLMs to generate prompts that elicit specific information from the models. In this case, the prompts are designed to retrieve relevant legal provisions related to software requirements. The study finds that the Rice-based solution can accurately identify trace links with an average recall of 84%, significantly outperforming Kashιf on the GDPR dataset.
The findings of this research have significant implications for software development and requirements engineering. They highlight the potential benefits of using LLMs for requirements traceability, particularly when combined with careful prompt engineering. However, they also underscore the limitations of simple classification-based approaches and the need for more sophisticated techniques to tackle complex real-world scenarios.
In practical terms, this research suggests that developers may be able to use LLMs to automate aspects of requirements traceability, potentially reducing the time and effort required for this task.
Cite this article: “Unlocking Efficient Requirements Traceability with Large Language Models”, The Science Archive, 2025.
Software Development, Requirements Engineering, Natural Language Processing, Machine Learning, Large Language Models, Requirements Traceability, Automated Solutions, Sentence Transformers, Prompt Engineering, European General Data Protection Regulation
