A topical approach to retrievability bias estimation

Wilkie, Colin and Azzopardi, Leif; (2016) A topical approach to retrievability bias estimation. In: Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval. ACM, New York, NY, USA, pp. 119-122. ISBN 9781450344975 (https://doi.org/10.1145/2970398.2970437)

Preview

Text. Filename: Wilkie_Azzopardi_ACM_ICTIR_2016_A_topical_approach_to_retrievability_bias.pdf Accepted Author Manuscript Download (167kB)| Preview

Abstract

Retrievability is an independent evaluation measure that offers insights to an aspect of retrieval systems that performance and efficiency measures do not. Retrievability is often used to calculate the retrievability bias, an indication of how accessible a system makes all the documents in a collection. Generally, computing the retrievability bias of a system requires a colossal number of queries to be issued for the system to gain an accurate estimate of the bias. However, it is often the case that the accuracy of the estimate is not of importance, but the relationship between the estimate of bias and performance when tuning a systems parameters. As such, reaching a stable estimation of bias for the system is more important than getting very accurate retrievability scores for individual documents. This work explores the idea of using topical subsets of the collection for query generation and bias estimation to form a local estimate of bias which correlates with the global estimate of retrievability bias. By using topical subsets, it would be possible to reduce the volume of queries required to reach an accurate estimate of retrievability bias, reducing the time and resources required to perform a retrievability analysis. Findings suggest that this is a viable approach to estimating retrievability bias and that the number of queries required can be reduced to less than a quarter of what was previously thought necessary.

• Share and Export
  

  RefWorksRIS (EndNote Online, Zotero, Ref manager, etc)RDF+XMLEndNoteASCII CitationOpenURL ContextObject in SpanMPEG-21 DIDLRIOXX2 XMLBibTeXAtomEP3 XMLDublin CoreRDF+N-TriplesMODSHTML CitationJSONData Cite XMLRDF+N3Multiline CSVOpenURL ContextObjectReferSimple MetadataMETS
• Item metadata

  Item type:	Book Section
  ID code:	58951
  Dates:	Date Event 12 September 2016 Published
  Notes:	© 2016 Copyright held by the owner/author(s). Publication rights licensed to ACM. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 2016 ACM International Conference on the Theory of Information Retrievalhttp://dx.doi.org/10.1145/2970398.2970437.
  Subjects:	Bibliography. Library Science. Information Resources > Library Science. Information Science
  Department:	Faculty of Science > Computer and Information Sciences
  Depositing user:	Pure Administrator
  Date deposited:	05 Dec 2016 10:59
  Last modified:	17 Nov 2024 01:28
  Related URLs:	Event
  URI:	https://strathprints.strath.ac.uk/id/eprint/58951