Picture of a black hole

Strathclyde Open Access research that creates ripples...

The Strathprints institutional repository is a digital archive of University of Strathclyde's Open Access research outputs. Strathprints provides access to thousands of research papers by University of Strathclyde researchers, including by Strathclyde physicists involved in observing gravitational waves and black hole mergers as part of the Laser Interferometer Gravitational-Wave Observatory (LIGO) - but also other internationally significant research from the Department of Physics. Discover why Strathclyde's physics research is making ripples...

Strathprints also exposes world leading research from the Faculties of Science, Engineering, Humanities & Social Sciences, and from the Strathclyde Business School.

Discover more...

A retrieval evaluation methodology for incomplete relevance assessments

Baillie, M. and Azzopardi, L. and Ruthven, I. (2007) A retrieval evaluation methodology for incomplete relevance assessments. In: Advances in Information Retrieval: 29th European Conference on IR Research (ECIR 2007). Lecture Notes in Computer Science, 4425 . Springer. ISBN 978-3-540-71494-1

[img]
Preview
PDF (strathprints003158.pdf)
strathprints003158.pdf

Download (323kB) | Preview

Abstract

In this paper we a propose an extended methodology for laboratory based Information Retrieval evaluation under in complete relevance assessments. This new protocol aims to identify potential uncertainty during system comparison that may result from incompleteness. We demonstrate how this methodology can lead towards a finer grained analysis of systems. This is advantageous, because the detection of uncertainty during the evaluation process can guide and direct researchers when evaluating new systems over existing and future test collections.