Query variation performance prediction for systematic reviews
Scells, Harrisen and Azzopardi, Leif and Zuccon, Guido and Koopman, Bevan; (2018) Query variation performance prediction for systematic reviews. In: 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018. ACM, USA, pp. 1089-1092. ISBN 9781450356572 (https://doi.org/10.1145/3209978.3210078)
Preview |
Text.
Filename: Scells_etal_SIGIR_2018_Query_variation_performance_prediction_for_systematic_reviews.pdf
Accepted Author Manuscript Download (1MB)| Preview |
Abstract
When conducting systematic reviews, medical researchers heavily deliberate over the final query to pose to the information retrieval system. Given the possible query variations that they could construct, selecting the best performing query is difficult. This motivates a new type of query performance prediction (QPP) task where the challenge is to estimate the performance of a set of query variations given a particular topic. Query variations are the reductions, expansions and modifications of a given seed query under the hypothesis that there exists some variations (either generated from permutations or hand crafted) which will improve retrieval effectiveness over the original query. We use the CLEF 2017 TAR Collection, to evaluate sixteen pre and post retrieval predictors for the task of Query Variation Performance Prediction (QVPP). Our findings show the IDF based QPPs exhibits the strongest correlations with performance. However, when using QPPs to select the best query, little improvement over the original query can be obtained, despite the fact that there are query variations which perform significantly better. Our findings highlight the difficulty in identifying effective queries within the context of this new task, and motivates further research to develop more accurate methods to help systematic review researchers in the query selection process.
-
-
Item type: Book Section ID code: 66484 Dates: DateEvent27 June 2018Published29 January 2018AcceptedSubjects: Science > Mathematics > Electronic computers. Computer science
Bibliography. Library Science. Information Resources > Library Science. Information ScienceDepartment: Faculty of Science > Computer and Information Sciences Depositing user: Pure Administrator Date deposited: 09 Jan 2019 10:56 Last modified: 04 Oct 2024 00:12 URI: https://strathprints.strath.ac.uk/id/eprint/66484