COVID-19 DIAGNOSIS

Scope of Use

Viral nucleic acid testing and chest computed tomography imaging are standard methods for diagnosing Covid-19 but are time-consuming [1].
Thus, researchers have been developing diagnostic models to predict the presence and severity of covid-19 in patients in a more efficient manner.
These models are developed using single country data or international (combined) data [1].
Because pneumonia-signs on lung CT scans is one of the most common manifestations of Covid-19, several diagnostic applications focus on the use of image-recognition software to accelerate the reading of lung X-rays and CT scans [2]. There are more than 100 publications in MedRxiv and bioRxiv dedicated to this medical application [2].
However, there are other models being proposed that do not use images to diagnose the presence or severity of the virus. Some models work by using predictors such as vital signs (e.g. temperature, heart rate, respiratory rate, oxygen saturation, blood pressure), or flu-like signs and symptoms (e.g. shiver, fatigue), among others [1].
Multiple different AI-based diagnostic tools have been proposed during the course of the pandemic [2]. Yet, most of them have not been implemented at a large scale but only in the context of small trials [2].

Technological robustness and efficacy

The COVID-PRECISE group reviewed and appraised the validity and usefulness of published and preprint reports of 118 diagnostics models. This review showed that most models had a high or unclear risk of bias due to various reasons. Some models used inappropriate data sources, or other general research malpractices resulting in a high risk of bias. Some of these also did not have clear reporting resulting in an unclear risk of bias.
Most models showed a lack of transparency and reproducibility due to a lack of descriptions of model specifications and subsequent estimations [1]
A high risk of bias implies that the reported accuracy of these models is quite optimistic, thus the performance of these models in new samples will probably be worse than that reported by the researchers.

Impact on citizens and society (blue=pos, red=neg)

In principle, this type of AI use can have a positive impact on citizens and society if developed responsibly. If however the methodologies are inadequate and the robustness is insufficient (as appears to be the case with many AI driven diagnostics models) the impact on citizens and society is negative, as it can result in incorrect, insufficient or lack of medical treatment.
Given that diagnostics tools are used mainly for medical triage, high biases on these prediction tools can easily create space for unfair treatment among patients.
Using unrepresentative data during the development of these models (as reviewed by the PRECISE group) can make the unrepresented population group be at a disadvantage when it comes to diagnosing them. In times of corona, this can lead to fatal consequences.
Thus, a high or unclear risk of bias negatively impacts the right of citizens to be treated in a fair manner and potentially their health and life.

Governance and Accountability

In January 2021, the FDA summarized its approach to a pre-market review of AI-driven software used for the diagnosis and treatment of disease [2].
However, the PRECISE-group showed that there is a high/unclear degree of bias that is hidden behind claims of high accurate predictions in most studies. Such claims could allow models to be accepted through the pre-market review of AI-driven software.
Therefore, there is an urgent need for researchers to adhere to reporting guidelines (i.e. TRIPOD or MINIMAR) and to the use of appropriate calibration, and validations of their models as part of existing methodological guidance for prediction modelling studies.

COVID-PRECISE

The COVID-PRECISE project is the most comprehensive systematic review of all COVID-19 related diagnostic, prognostic and general population prediction models, including accuracy, quality (risk of bias) and applicability assessment. This living review will be frequently updated in BMJ.

The project is led by Prof. Laure Wynants of Maastricht University who presented the project at the AI & Corona Webinar at the EESC, that was part of this project.

COVID-19 DIAGNOSIS

COVID-19 DIAGNOSIS

COVID-PRECISE

CONTACT

FOLLOW US

MAILING LIST*