Use of text-mining methods to improve efficiency in the calculation of drug exposure to support pharmacoepidemiology studies

McTaggart, Stuart and Nangle, Clifford and Caldwell, Jacqueline and Alvarez-Madrazo, Samantha and Colhoun, Helen and Bennie, Marion (2018) Use of text-mining methods to improve efficiency in the calculation of drug exposure to support pharmacoepidemiology studies. International Journal of Epidemiology, 47 (2). pp. 617-624. ISSN 0300-5771 (https://doi.org/10.1093/ije/dyx264)

[thumbnail of Bennie-etal-IJE-2017-text-mining-methods-to-improve-efficiency-in-the-calculation-of-drug-exposure]
Preview
 Text. Filename: Bennie_etal_IJE_2017_text_mining_methods_to_improve_efficiency_in_the_calculation_of_drug_exposure.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (314kB)| Preview

Abstract

Background: Efficient generation of structured dose instructions that enable researchers to calculate drug exposure is central to pharmacoepidemiology studies. The aim was to design and test an algorithm to codify dose instructions, applied to the NHS Scotland Prescribing Information System (PIS) that records approximately 100 million prescriptions per annum. Methods: a natural language processing (NLP) algorithm was developed enabling free-text dose instructions to be represented by three attributes: quantity; frequency; and qualifier, each specified by a set of variables. This was tested on a sample of 15 593 distinct dose instructions and manually validated. The final algorithm was then applied to the full dataset. Results: the dataset comprised 458 227 687 prescriptions, of which 99.67% had dose instructions represented by 4 964 083 distinct free-text dose instructions; 13 593 (0.27%) of these occurred ≥1000 times accounting for 88.85% of all prescriptions. Reviewers identified 767 (5.83%) instances where the structured output (n=13 152) was incorrect, an accuracy of 94.2%. Application of the final NLP algorithm to the dataset generated an overall structured output of 92.3% which varied by therapeutic area (86.7% central nervous system to 96.8% cardiovascular). Conclusion: We adopted a zero assumption approach to create an NLP algorithm, operational at scale, to produce structured output which enables data users maximum flexibility to formulate, test and apply their own assumptions according to the medicines under investigation. Text mining approaches can provide a solution to the safe and efficient management and provisioning of large volumes of data generated through our health systems.

CORE (COnnecting REpositories)