CLEF 2018 technologically assisted reviews in empirical medicine overview

Kanoulas, Evangelos and Li, Dan and Azzopardi, Leif and Spijker, Rene (2018) CLEF 2018 technologically assisted reviews in empirical medicine overview. CEUR Workshop Proceedings, 2125. ISSN 1613-0073

[thumbnail of Kanoulas-etal-CEUR2018-CLEF-2018-technologically-assisted-reviews-in-empirical-medicine]
Preview
 Text. Filename: Kanoulas_etal_CEUR2018_CLEF_2018_technologically_assisted_reviews_in_empirical_medicine.pdf
Final Published Version
Download (1MB)| Preview

Abstract

Conducting a systematic review is a widely used method to obtain an overview over the current scientific consensus on a topic of interest, by bringing together multiple studies in a reliable, transparent way. The large and growing number of published studies, and their increasing rate of publication, makes the task of identifying all relevant studies in an unbiased way both complex and time consuming to the extent that jeopardizes the validity of their findings and the ability to inform policy and practice in a timely manner. The CLEF 2018 e-Health Technology Assisted Reviews in Empirical Medicine task aims at evaluating search algorithms that seek to identify all studies relevant for conducting a systematic review in empirical medicine. The task had a focus on Diagnostic Test Accuracy (DTA) reviews, and consisted of two subtasks: 1) given a number of relevance criteria as described in a systematic review protocol, search a large medical database of article abstracts (PubMed) to find the studies to be included in the review, and 2) given the article abstracts retrieved by a carefully designed Boolean Query, prioritize them to reduce the effort required by experts to screen the abstracts for inclusion in the review. Seven teams participated in the task, with a total of 12 runs submitted for subtask 1 and 19 runs for subtask 2. This paper reports both the methodology used to construct the benchmark collection, and the results of the evaluation.

CORE (COnnecting REpositories)