Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

eLife, Journal Year: 2023, Volume and Issue: 12

Published: April 21, 2023

Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields recent insights epidemiology, one maximise the predictive performance such if multiple models are combined into an ensemble. Here, we report ensembles predicting COVID-19 cases deaths across Europe between 08 March 2021 07 2022.

Language: Английский

---

Comparing trained and untrained probabilistic ensemble forecasts of COVID-19 cases and deaths in the United States

International Journal of Forecasting, Journal Year: 2022, Volume and Issue: 39(3), P. 1366 - 1383

Published: July 1, 2022

The U.S. COVID-19 Forecast Hub aggregates forecasts of the short-term burden in United States from many contributing teams. We study methods for building an ensemble that combines these These experiments have informed used by Hub. To be most useful to policymakers, must stable performance presence two key characteristics component forecasts: (1) occasional misalignment with reported data, and (2) instability relative forecasters over time. Our results indicate challenges, untrained robust approach ensembling using equally weighted median all is a good choice support public health decision-makers. In settings where some record performance, trained ensembles give those higher weight can also helpful.

Language: Английский

---

Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty

Nature Communications, Journal Year: 2023, Volume and Issue: 14(1)

Published: Nov. 20, 2023

Our ability to forecast epidemics far into the future is constrained by many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify state critical epidemic drivers. Since December 2020, U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams make months ahead SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function both scenario validity model calibration. We show remained close reality for 22 weeks on average before arrival unanticipated variants invalidated key assumptions. An ensemble participating models preserved variation between (using linear opinion pool method) was consistently more reliable than any single in periods valid assumptions, while projection interval coverage near target levels. were used guide pandemic response, illustrating value collaborative hubs

Language: Английский

---

Incorporating variant frequencies data into short-term forecasting for COVID-19 cases and deaths in the USA: a deep learning approach

EBioMedicine, Journal Year: 2023, Volume and Issue: 89, P. 104482 - 104482

Published: Feb. 22, 2023

Language: Английский

---

Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations

Nature Communications, Journal Year: 2024, Volume and Issue: 15(1)

Published: July 26, 2024

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021-22 and 2022-23 seasons, 26 forecasting teams provided national jurisdiction-specific probabilistic predictions of weekly confirmed hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using Weighted Interval Score (WIS), relative WIS, coverage. Six out 23 models outperform baseline model across forecast locations in 12 18 2022-23. Averaging all targets, FluSight ensemble 2

Language: Английский

---

Challenges of COVID-19 Case Forecasting in the US, 2020–2021

PLoS Computational Biology, Journal Year: 2024, Volume and Issue: 20(5), P. e1011200 - e1011200

Published: May 6, 2024

During the COVID-19 pandemic, forecasting trends to support planning and response was a priority for scientists decision makers alike. In United States, coordinated by large group of universities, companies, government entities led Centers Disease Control Prevention US Forecast Hub ( https://covid19forecasthub.org ). We evaluated approximately 9.7 million forecasts weekly state-level cases predictions 1–4 weeks into future submitted 24 teams from August 2020 December 2021. assessed coverage central prediction intervals weighted interval scores (WIS), adjusting missing relative baseline forecast, used Gaussian generalized estimating equation (GEE) model evaluate differences in skill across epidemic phases that were defined effective reproduction number. Overall, we found high variation individual models, with ensemble-based outperforming other approaches. generally higher larger jurisdictions (e.g., states compared counties). Over time, performed worst periods rapid changes reported (either increasing or decreasing phases) 95% dropping below 50% during growth winter 2020, Delta, Omicron waves. Ideally, case could serve as leading indicator transmission dynamics. However, while most outperformed naïve model, even accurate unreliable key phases. Further research improve indicators, like cases, leveraging additional real-time data, addressing performance phases, improving characterization forecast confidence, ensuring coherent spatial scales. meantime, it is critical users appreciate current limitations use broad set indicators inform pandemic-related making.

Language: Английский

---

A multiscale modeling framework for Scenario Modeling: Characterizing the heterogeneity of the COVID-19 epidemic in the US

Epidemics, Journal Year: 2024, Volume and Issue: 47, P. 100757 - 100757

Published: March 5, 2024

The Scenario Modeling Hub (SMH) initiative provides projections of potential epidemic scenarios in the United States (US) by using a multi-model approach. Our contribution to SMH is generated multiscale model that combines global metapopulation modeling approach (GLEAM) with local and mobility US (LEAM-US), first introduced here. LEAM-US consists 3142 subpopulations each representing single county across 50 states District Columbia, enabling us project state national trajectories COVID-19 cases, hospitalizations, deaths under different scenarios. age-structured, multi-strain. It integrates data on vaccine administration, human mobility, non-pharmaceutical interventions. contributed all 17 rounds SMH, allows for mechanistic characterization spatio-temporal heterogeneities observed during pandemic. Here we describe mathematical computational structure underpinning our model, present as case study results concerning emergence SARS-CoV-2 Alpha variant (lineage designation B.1.1.7). findings reveal considerable spatial temporal heterogeneity introduction diffusion variant, both at level individual combined statistical areas, it competes against ancestral lineage. We discuss key factors driving time required rise dominance within population, quantify significant impact had effective reproduction number level. Overall, show able capture complexity pandemic response US.

Language: Английский

---

Usage of Compartmental Models in Predicting COVID-19 Outbreaks

The AAPS Journal, Journal Year: 2022, Volume and Issue: 24(5)

Published: Sept. 2, 2022

Language: Английский

---

Model Diagnostics and Forecast Evaluation for Quantiles

Annual Review of Statistics and Its Application, Journal Year: 2022, Volume and Issue: 10(1), P. 597 - 621

Published: Nov. 1, 2022

Model diagnostics and forecast evaluation are closely related tasks, with the former concerning in-sample goodness (or lack) of fit latter addressing predictive performance out-of-sample. We review ubiquitous setting in which forecasts cast form quantiles or quantile-bounded prediction intervals. distinguish unconditional calibration, corresponds to classical coverage criteria, from stronger notion conditional as can be visualized quantile reliability diagrams. Consistent scoring functions—including, but not limited to, widely used asymmetricpiecewise linear score pinball loss—provide for comparative assessment ranking, link coefficient determination skill scores. illustrate use these tools on Engel's food expenditure data, Global Energy Forecasting Competition 2014, US COVID-19 Forecast Hub.

Language: Английский

---

Real-time mechanistic Bayesian forecasts of COVID-19 mortality

The Annals of Applied Statistics, Journal Year: 2023, Volume and Issue: 17(3)

Published: Sept. 1, 2023

The COVID-19 pandemic emerged in late December 2019. In the first six months of global outbreak, U.S. reported more cases and deaths than any other country world. Effective modeling course can help assist with public health resource planning, intervention efforts, vaccine clinical trials. However, building applied forecasting models presents unique challenges during a pandemic. First, case data available to real time represent nonstationary fraction true incidence due changes diagnostic tests test-seeking behavior. Second, interventions varied across geography leading large transmissibility over We propose mechanistic Bayesian model that builds upon classic compartmental susceptible–exposed–infected–recovered (SEIR) operationalize time. This framework includes nonparametric varying transmission rates, death discrepancies testing reporting issues, joint observation likelihood on new counts deaths; it is implemented probabilistic programming language automate use reasoning for quantifying uncertainty forecasts. has been used submit forecasts Centers Disease Control through Forecast Hub under name MechBayes. examine performance relative baseline as well alternate submitted forecast hub. Additionally, we include an ablation test our extensions SEIR model. demonstrate significant gain both point scoring measures using MechBayes, when compared model, show MechBayes ranks one top two out nine which regularly duration pandemic, trailing only ensemble part.

Language: Английский

---