Picture of mobile phone running fintech app

Fintech: Open Access research exploring new frontiers in financial technology

Strathprints makes available Open Access scholarly outputs by the Department of Accounting & Finance at Strathclyde. Particular research specialisms include financial risk management and investment strategies.

The Department also hosts the Centre for Financial Regulation and Innovation (CeFRI), demonstrating research expertise in fintech and capital markets. It also aims to provide a strategic link between academia, policy-makers, regulators and other financial industry participants.

Explore all Strathclyde Open Access research...

Evaluating epistemic uncertainty under incomplete assessments

Baillie, Mark and Azzopardi, Leif and Ruthven, Ian (2008) Evaluating epistemic uncertainty under incomplete assessments. Information Processing and Management, 44 (2). pp. 811-837. ISSN 0306-4573

Text (strathprints003418)
Accepted Author Manuscript

Download (1MB) | Preview


The thesis of this study is to propose an extended methodology for laboratory based Information Retrieval evaluation under incomplete relevance assessments. This new methodology aims to identify potential uncertainty during system comparison that may result from incompleteness. The adoption of this methodology is advantageous, because the detection of epistemic uncertainty - the amount of knowledge (or ignorance) we have about the estimate of a system's performance - during the evaluation process can guide and direct researchers when evaluating new systems over existing and future test collections. Across a series of experiments we demonstrate how this methodology can lead towards a finer grained analysis of systems. In particular, we show through experimentation how the current practice in Information Retrieval evaluation of using a measurement depth larger than the pooling depth increases uncertainty during system comparison.