The "analysis of competing hypotheses" in intelligence analysis

Dhami, Mandeep K. and Belton, Ian K. and Mandel, David R. (2019) The "analysis of competing hypotheses" in intelligence analysis. Applied Cognitive Psychology, 33 (6). pp. 1080-1090. ISSN 0888-4080 (https://doi.org/10.1002/acp.3550)

[thumbnail of Dhami-etal-ACP-2019-The-analysis-of-competing-hypotheses-in-intelligence]
Preview
 Text. Filename: Dhami_etal_ACP_2019_The_analysis_of_competing_hypotheses_in_intelligence.pdf
Final Published Version
License: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 logo
Download (210kB)| Preview

Abstract

The intelligence community uses "structured analytic techniques" to help analysts think critically and avoid cognitive bias. However, little evidence exists of how techniques are applied and whether they are effective. We examined the use of the analysis of competing hypotheses (ACH)—a technique designed to reduce "confirmation bias". Fifty intelligence analysts were randomly assigned to use ACH or not when completing a hypothesis testing task that had probabilistic ground truth. Data on analysts' judgement processes and conclusions were collected using written protocols that were then coded for statistical analyses. We found that ACH-trained analysts did not follow all of the steps of ACH. There was mixed evidence for ACH's ability to reduce confirmation bias, and we observed that ACH may increase judgement inconsistency and error. It may be prudent for the intelligence community to consider the conditions under which ACH would prove useful and to explore alternatives.

ORCID iDs

Dhami, Mandeep K., Belton, Ian K. ORCID logoORCID: https://orcid.org/0000-0002-2479-6563 and Mandel, David R.;
CORE (COnnecting REpositories)