Challenges in applying large language models to requirements engineering tasks

Norheim, Johannes J. and Rebentisch, Eric and Xiao, Dekai and Draeger, Lorenz and Kerbrat, Alain and de Weck, Olivier L. (2024) Challenges in applying large language models to requirements engineering tasks. Design Science, 10. e16. ISSN 2053-4701 (https://doi.org/10.1017/dsj.2024.8)

[thumbnail of Norheim-etal-challenges-in-applying-large-language-models-to-requirements-engineering-tasks]
Preview
 Text. Filename: Norheim-etal-challenges-in-applying-large-language-models-to-requirements-engineering-tasks.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (622kB)| Preview

Abstract

Growth in the complexity of advanced systems is mirrored by a growth in the number of engineering requirements and related upstream and downstream tasks. These requirements are typically expressed in natural language and require human expertise to manage. Natural language processing (NLP) technology has long been seen as promising to increase requirements engineering (RE) productivity but has yet to demonstrate substantive benefits. The recent addition of large language models (LLMs) to the NLP toolbox is now generating renewed enthusiasm in the hope that it will overcome past shortcomings. This article scrutinizes this claim by reviewing the application of LLMs for engineering requirements tasks. We survey the success of applying LLMs and the scale to which they have been used. We also identify groups of challenges shared across different engineering requirement tasks. These challenges show how this technology has been applied to RE tasks that need reassessment. We finalize by drawing a parallel to other engineering fields with similar challenges and how they have been overcome in the past – and suggest these as future directions to be investigated.

ORCID iDs

Norheim, Johannes J. ORCID logoORCID: https://orcid.org/0009-0004-9989-0740, Rebentisch, Eric, Xiao, Dekai, Draeger, Lorenz, Kerbrat, Alain and de Weck, Olivier L.;
CORE (COnnecting REpositories)