Who’s better: your doctor or AI?

A novel systematic review and meta-analysis suggests that artificial intelligence (AI) technology may as effectively diagnose disease conditions based on medical images as trained healthcare professionals.

Artificial intelligence (AI) — also referred to as ‘deep learning’ — employs algorithms, computing power and big data to mimic human learning, intelligence and performance. In a novel study, an international team of researchers from London, Birmingham, Switzerland and California have conducted the largest, most inclusive analysis of literature evidence concerning the performance level of AI technology compared to human healthcare professionals. The results demonstrate that AI appears to successfully detect diseases from medical imaging with comparable levels of accuracy as healthcare providers.

AI can identify and predict disease patterns based on thousands of medical images used to ‘train’ the technology. AI therefore offers great potential to aid the timely and accurate diagnosis of disease; to date, 30 AI technologies have received US FDA approval for implementation in routine clinical practice.

• How can AI be applied in the real-world evidence field?

In the study, researchers sought to determine to what extent previous studies comparing the diagnostic accuracy of AI and healthcare professionals may be biased or poorly reflective of real-world clinical practices.

Researchers identified more than 31,000 publications from 2012—2019 concerning the diagnostic potential of AI compared to healthcare professionals based on medical images. However, data from only 69 such articles were analyzed in the study as most submissions were deemed to be of poor quality, lacking in sufficient data or not directly comparing the diagnostic accuracy of AI with healthcare professionals specifically.

Out-of-sample external validation was performed for 25 of the studies analyzed. Pooled sensitivity scores for AI and healthcare professionals, when the analysis was restricted to the contingency table for each study, were 92.5% and 90.5% respectively.

Lead study author Alastair Denniston, a Professor from University Hospitals Birmingham NHS Foundation Trust (Birmingham, UK), commented that despite the small study sample size, “…we found that deep learning could indeed detect diseases ranging from cancers to eye diseases as accurately as health professionals.”

However, Denniston continued: “…it’s important to note that AI did not substantially out-perform human diagnosis.”

---

Source:

Liu X, Faes L, Kale AU et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. doi: 10.1016/S2589-7500(19)30123-2. (Epub ahead of print) (2019).

---

How can AI be applied in the real-world evidence field?

AI and machine learning technologies are being increasingly applied across the clinical pipeline, from preclinical discovery to clinical development, to market access and commercial use cases. AI has a well established place in supporting complex analyses in very large data sets or extracting more ‘signal’ from smaller and more complex data.

Read more on AI in real-world evidence in this interview with Jeff Elton>>