A critical evaluation of 42, large-scale, science and technology foresight Delphi surveys
Belton, Ian and Cuhls, Kerstin and Wright, George (2022) A critical evaluation of 42, large-scale, science and technology foresight Delphi surveys. Futures & Foresight Science, 4 (2). e2118. ISSN 2573-5152 (https://doi.org/10.1002/ffo2.118)
Preview |
Text.
Filename: Belton_etal_FFS_2021_A_critical_evaluation_of_forty_two_large_scale_science_and_technology_foresight_Delphi_surveys.pdf
Accepted Author Manuscript Download (901kB)| Preview |
Abstract
Large-scale Delphi surveys on technology foresight started in the 1960s and involve an average of about 2000 participants answering, potentially, up to about 450 items. This contrasts sharply with the participation and content of the more common, smaller-scale Delphi surveys. Previously, Belton et al. developed “six steps” to underpin a well-founded and defensible Delphi process and we apply these steps in a novel evaluation of the quality of 42 large-scale technology foresight surveys. Using a detailed analysis of two exemplar studies and a content analysis of all 42 surveys, we explore whether such surveys differ systematically from “traditional” smaller-scale Delphi surveys and, if so, why this may be and what it may mean for the quality of data produced. We conclude that there are some problematic issues within these surveys—to do with (i) data quality in both the numerical summarizing of participant's between-round feedback and in the reporting of final round numerical responses, (ii) the infrequent elicitation of rationales to justify participants' proffered numerical responses, and, when such rationales are elicited, (iii) the between-round summary and presentation of the rationales. We speculate on the reasons for these design differences in the extant large-scale surveys and conclude that extra-survey political influences, such as differing objectives and the need to demonstrate wide-ranging expert participation, may be the underlying cause. We conclude with considerations and recommendations for the design of future large-scale Delphi surveys to enable the underlying process to become better-founded and more defensible to procedural evaluation.
ORCID iDs
Belton, Ian ORCID: https://orcid.org/0000-0002-2479-6563, Cuhls, Kerstin and Wright, George ORCID: https://orcid.org/0000-0002-4350-7800;-
-
Item type: Article ID code: 78986 Dates: DateEvent30 June 2022Published15 January 2022Published Online18 November 2021AcceptedSubjects: Social Sciences > Social Sciences (General) Department: Strathclyde Business School > Management Science Depositing user: Pure Administrator Date deposited: 20 Dec 2021 13:22 Last modified: 03 Oct 2024 00:36 URI: https://strathprints.strath.ac.uk/id/eprint/78986