Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level

Meakin, Sophie and Abbott, Sam and Bosse, Nikos and Munday, James and Gruson, Hugo and Hellewell, Joel and Sherratt, Katharine and Chapman, Lloyd A.C. and Prem, Kiesha and Klepac, Petra and Jombart, Thibaut and Knight, Gwenan M. and Jafari, Yalda and Flasche, Stefan and Waites, William and Jit, Mark and Eggo, Rosalind M. and Villabona-Arenas, C. Julian and Russell, Timothy W. and Medley, Graham and Edmunds, W. John and Davies, Nicholas G. and Liu, Yang and Hué, Stéphane and Brady, Oliver and Pung, Rachael and Abbas, Kaja and Gimma, Amy and Mee, Paul and Endo, Akira and Clifford, Samuel and Sun, Fiona Yueqian and McCarthy, Ciara V. and Quilty, Billy J. and Rosello, Alicia and Sandmann, Frank G. and Barnard, Rosanna C. and Kucharski, Adam J. and Procter, Simon R. and Jarvis, Christopher I. and Gibbs, Hamish P. and Hodgson, David and Lowe, Rachel and Atkins, Katherine E. and Koltai, Mihaly and Pearson, Carl A.B. and Finch, Emilie and Wong, Kerry L.M. and Quaife, Matthew and O’Reilly, Kathleen, CMMID COVID-19 Working Group (2022) Comparative assessment of methods for short-term forecasts of COVID-19 hospital admissions in England at the local level. BMC Medicine, 20 (1). 86. ISSN 1741-7015 (https://doi.org/10.1186/s12916-022-02271-x)

[thumbnail of Meakin-etal-BMCM-2022-Comparative-assessment-of-methods-for-short-term-forecasts-of-COVID-19-hospital-admissions-in-England]
Preview
 Text. Filename: Meakin_etal_BMCM_2022_Comparative_assessment_of_methods_for_short_term_forecasts_of_COVID_19_hospital_admissions_in_England.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (3MB)| Preview

Abstract

Background: Forecasting healthcare demand is essential in epidemic settings, both to inform situational awareness and facilitate resource planning. Ideally, forecasts should be robust across time and locations. During the COVID-19 pandemic in England, it is an ongoing concern that demand for hospital care for COVID-19 patients in England will exceed available resources. Methods: We made weekly forecasts of daily COVID-19 hospital admissions for National Health Service (NHS) Trusts in England between August 2020 and April 2021 using three disease-agnostic forecasting models: a mean ensemble of autoregressive time series models, a linear regression model with 7-day-lagged local cases as a predictor, and a scaled convolution of local cases and a delay distribution. We compared their point and probabilistic accuracy to a mean-ensemble of them all and to a simple baseline model of no change from the last day of admissions. We measured predictive performance using the weighted interval score (WIS) and considered how this changed in different scenarios (the length of the predictive horizon, the date on which the forecast was made, and by location), as well as how much admissions forecasts improved when future cases were known. Results: All models outperformed the baseline in the majority of scenarios. Forecasting accuracy varied by forecast date and location, depending on the trajectory of the outbreak, and all individual models had instances where they were the top- or bottom-ranked model. Forecasts produced by the mean-ensemble were both the most accurate and most consistently accurate forecasts amongst all the models considered. Forecasting accuracy was improved when using future observed, rather than forecast, cases, especially at longer forecast horizons. Conclusions: Assuming no change in current admissions is rarely better than including at least a trend. Using confirmed COVID-19 cases as a predictor can improve admissions forecasts in some scenarios, but this is variable and depends on the ability to make consistently good case forecasts. However, ensemble forecasts can make forecasts that make consistently more accurate forecasts across time and locations. Given minimal requirements on data and computation, our admissions forecasting ensemble could be used to anticipate healthcare needs in future epidemic or pandemic settings.

ORCID iDs

Meakin, Sophie, Abbott, Sam, Bosse, Nikos, Munday, James, Gruson, Hugo, Hellewell, Joel, Sherratt, Katharine, Chapman, Lloyd A.C., Prem, Kiesha, Klepac, Petra, Jombart, Thibaut, Knight, Gwenan M., Jafari, Yalda, Flasche, Stefan, Waites, William ORCID logoORCID: https://orcid.org/0000-0002-7759-6805, Jit, Mark, Eggo, Rosalind M., Villabona-Arenas, C. Julian, Russell, Timothy W., Medley, Graham, Edmunds, W. John, Davies, Nicholas G., Liu, Yang, Hué, Stéphane, Brady, Oliver, Pung, Rachael, Abbas, Kaja, Gimma, Amy, Mee, Paul, Endo, Akira, Clifford, Samuel, Sun, Fiona Yueqian, McCarthy, Ciara V., Quilty, Billy J., Rosello, Alicia, Sandmann, Frank G., Barnard, Rosanna C., Kucharski, Adam J., Procter, Simon R., Jarvis, Christopher I., Gibbs, Hamish P., Hodgson, David, Lowe, Rachel, Atkins, Katherine E., Koltai, Mihaly, Pearson, Carl A.B., Finch, Emilie, Wong, Kerry L.M., Quaife, Matthew and O’Reilly, Kathleen;
CORE (COnnecting REpositories)