A retrieval evaluation methodology for incomplete relevance assessments

Baillie, M. and Azzopardi, L. and Ruthven, I. (2007) A retrieval evaluation methodology for incomplete relevance assessments. In: Advances in Information Retrieval: 29th European Conference on IR Research (ECIR 2007). Lecture Notes in Computer Science, 4425 . Springer. ISBN 978-3-540-71494-1

[img]
Preview
 PDF (strathprints003158.pdf)
strathprints003158.pdf
Download (323kB)| Preview

    Abstract

    In this paper we a propose an extended methodology for laboratory based Information Retrieval evaluation under in complete relevance assessments. This new protocol aims to identify potential uncertainty during system comparison that may result from incompleteness. We demonstrate how this methodology can lead towards a finer grained analysis of systems. This is advantageous, because the detection of uncertainty during the evaluation process can guide and direct researchers when evaluating new systems over existing and future test collections.

    Author(s):Baillie, M., Azzopardi, L. and Ruthven, I. ORCID logoORCID: https://orcid.org/0000-0001-6669-5376
    Item type:Book Section
    ID code:3158
    Keywords:information retrieval, document retrieval, relevance ranking, searching, Electronic information resources, Electronic computers. Computer science
    Subjects:Bibliography. Library Science. Information Resources > Information resources > Electronic information resources
    Science > Mathematics > Electronic computers. Computer science
    Department:Faculty of Science > Computer and Information Sciences
    Depositing user: Strathprints Administrator
    Date deposited:30 Apr 2007
    Last modified:30 Sep 2019 03:31
    URI:https://strathprints.strath.ac.uk/id/eprint/3158
    Export data:
    CORE (COnnecting REpositories)