Cargando…

Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations

Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and have often lacked transparency in terms of prioritization of false-positive versus...

Descripción completa

Detalles Bibliográficos
Autores principales: Bilinski, Alyssa M., Salomon, Joshua A., Hatfield, Laura A.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: National Academy of Sciences 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10410764/
https://www.ncbi.nlm.nih.gov/pubmed/37527346
http://dx.doi.org/10.1073/pnas.2302528120
_version_ 1785086521061146624
author Bilinski, Alyssa M.
Salomon, Joshua A.
Hatfield, Laura A.
author_facet Bilinski, Alyssa M.
Salomon, Joshua A.
Hatfield, Laura A.
author_sort Bilinski, Alyssa M.
collection PubMed
description Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and have often lacked transparency in terms of prioritization of false-positive versus false-negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false-negative versus false-positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a method to update risk thresholds in real time. We show that this adaptive approach to designating areas as “high risk” improves performance over static metrics in predicting 3-wk-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-wk-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context.
format Online
Article
Text
id pubmed-10410764
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher National Academy of Sciences
record_format MEDLINE/PubMed
spelling pubmed-104107642023-08-10 Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations Bilinski, Alyssa M. Salomon, Joshua A. Hatfield, Laura A. Proc Natl Acad Sci U S A Biological Sciences Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and have often lacked transparency in terms of prioritization of false-positive versus false-negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false-negative versus false-positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a method to update risk thresholds in real time. We show that this adaptive approach to designating areas as “high risk” improves performance over static metrics in predicting 3-wk-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-wk-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context. National Academy of Sciences 2023-08-01 2023-08-08 /pmc/articles/PMC10410764/ /pubmed/37527346 http://dx.doi.org/10.1073/pnas.2302528120 Text en Copyright © 2023 the Author(s). Published by PNAS. https://creativecommons.org/licenses/by-nc-nd/4.0/This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND) (https://creativecommons.org/licenses/by-nc-nd/4.0/) .
spellingShingle Biological Sciences
Bilinski, Alyssa M.
Salomon, Joshua A.
Hatfield, Laura A.
Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations
title Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations
title_full Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations
title_fullStr Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations
title_full_unstemmed Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations
title_short Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations
title_sort adaptive metrics for an evolving pandemic: a dynamic approach to area-level covid-19 risk designations
topic Biological Sciences
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10410764/
https://www.ncbi.nlm.nih.gov/pubmed/37527346
http://dx.doi.org/10.1073/pnas.2302528120
work_keys_str_mv AT bilinskialyssam adaptivemetricsforanevolvingpandemicadynamicapproachtoarealevelcovid19riskdesignations
AT salomonjoshuaa adaptivemetricsforanevolvingpandemicadynamicapproachtoarealevelcovid19riskdesignations
AT hatfieldlauraa adaptivemetricsforanevolvingpandemicadynamicapproachtoarealevelcovid19riskdesignations