Picture of smart phone in human hand

World leading smartphone and mobile technology research at Strathclyde...

The Strathprints institutional repository is a digital archive of University of Strathclyde's Open Access research outputs. Strathprints provides access to thousands of Open Access research papers by University of Strathclyde researchers, including by Strathclyde researchers from the Department of Computer & Information Sciences involved in researching exciting new applications for mobile and smartphone technology. But the transformative application of mobile technologies is also the focus of research within disciplines as diverse as Electronic & Electrical Engineering, Marketing, Human Resource Management and Biomedical Enginering, among others.

Explore Strathclyde's Open Access research on smartphone technology now...

Video test collection with graded relevance assessments

Qiying, Weng and Halvey, Martin and Villa, Robert (2016) Video test collection with graded relevance assessments. In: CHIIR'16. ACM, New York, pp. 309-312. ISBN 9781450337519

Text (Qiying-Halvey-Villa-ACMSIGIR-video-test-collection-with-graded-relevance-assessments)
Qiying_Halvey_Villa_ACMSIGIR_video_test_collection_with_graded_relevance_assessments.pdf - Accepted Author Manuscript

Download (634kB) | Preview


Relevance is a complex, but core, concept within the field of Information Retrieval. In order to allow system comparisons the many factors that influence relevance are often discarded to allow abstraction to a single score relating to relevance. This means that a great wealth of information is often discarded. In this paper we outline the creation of a video test collection with graded relevance assessments, to the best of our knowledge the first example of such a test collection for video retrieval. To directly address the shortcoming above we also gathered behavioural and perceptual data from assessors during the assessment process. All of this information along with judgements are available for download. Our intention is to allow other researchers to supplement the judgements to help create an adaptive test collection which contains supplementary information rather than a completely static collection with binary judgements.