COVID-19 PROGNOSIS

COVID-19 PROGNOSIS

Scope of Use (only research)
  • Due to the overcrowding of hospitals during the covid-19 pandemic, researchers have put efforts in developing AI models that predict the progression of the disease, with the aim of aiding with triage in times where resources are scarce.
  • Most researched prognostics models aim to estimate mortality risk and progression to a severe or critical state of the disease.
  • Other models, however, also aim to predict recovery, length of hospital stay, intensive care admission, intubation, the duration of mechanical ventilation, acute respiratory distress syndrome, cardiac injury or thrombotic complication [1].
  • The most frequently used categories of prognostic indicators include age, comorbidities, vital signs, image features, sex, lymphocyte count, and C reactive protein [1].
  • The models differ in prediction horizons (between one and 37 days) [1].
Technological robustness and efficacy (no evidence)
  • The COVID-PRECISE group reviewed and appraised the technical robustness and efficacy of published and preprint reports of 107 prognostic models.
  • The review showed that most models had a high or unclear risk of bias due to various reasons.
  • Some suffered from dichotomization of predictors, or inappropriate inclusions/exclusions of study participants, leading to inappropriate sample sizes [1].
Impact on citizens and society (blue=pos, red=neg)
  • Although the impact of accurately developing prognostics tools that could save lives is fairly positive, the state of research suggests that most of the models being developed suffer from a high degree of bias.
  • The impact of using biased models can cause more harm than good, thus having a negative impact on citizens and society.
Governance and Accountability (importance)
  • Similarly to research on AI for diagnostics or identification of population vulnerability, the PRECISE group shows that there is an urgent need for researchers to adhere to reporting guidelines (i.e. TRIPOD) and be assessed against bias with tools such as PROBAST to ensure that models are appropriately calibration, validated, and using good modelling practices.
  • Additionally, many reports did not report relevant information clearly [1].
Technological robustness and efficacy (no evidence)
  • The COVID-PRECISE group reviewed and appraised the technical robustness and efficacy of published and preprint reports of 107 prognostic models.
  • The review showed that most models had a high or unclear risk of bias due to various reasons.
  • Some suffered from dichotomization of predictors, or inappropriate inclusions/exclusions of study participants, leading to inappropriate sample sizes [1].
Impact on citizens and society (blue=pos, red=neg)
  • Although the impact of accurately developing prognostics tools that could save lives is fairly positive, the state of research suggests that most of the models being developed suffer from a high degree of bias.
  • The impact of using biased models can cause more harm than good, thus having a negative impact on citizens and society.
Governance and Accountability (importance)
  • Similarly to research on AI for diagnostics or identification of population vulnerability, the PRECISE group shows that there is an urgent need for researchers to adhere to reporting guidelines (i.e. TRIPOD) and be assessed against bias with tools such as PROBAST to ensure that models are appropriately calibration, validated, and using good modelling practices.
  • Additionally, many reports did not report relevant information clearly [1].

Assessment prepared by Mónica Fernández-Peñalver

COVID-PRECISE

The COVID-PRECISE project is the most comprehensive systematic review of all COVID-19 related diagnostic, prognostic and general population prediction models, including accuracy, quality (risk of bias) and applicability assessment. This living review will be frequently updated in BMJ.

The project is led by Prof. Laure Wynants of Maastricht University who presented the project at the AI & Corona Webinar at the EESC, that was part of this project.