Diagnostic Radiology Guided by Machine Learning

Author: Ruchi Maniar Edited by: Burcu Anil Kirmizitas

The humble chest x-ray is one of the main investigation methods used to diagnose several diseases in the human body, and as such can offer valuable information about lung diseases such as pneumonia. Pneumonia is one such infection where the diagnosis of the disease is unfortunately often missed by radiologists, and can, if untreated, lead to hospitalisation in the intensive care unit. In the UK, respiratory disease-related deaths accounted for 20% of all deaths, with pneumonia in general causing 5.1 % of them across the country between 2001 and 2010 according to the British Lung Foundation.

Early detection of pneumonia therefore can prevent some of the deaths attributed to this deadly disease, and the emergence of integrating artificial intelligence and machine learning into diagnostic radiology has been substantial, for example in cardiology, dermatology and diabetes.

This November, Professor Andrew Ng’s Stanford-based machine learning group published their findings on a clever neural network they developed - CheXNet -  inspired by a dataset of over 100,000 CXR images of 30,805 unique patients, released by the National Institutes of Health in September. The frontal-view images in this data set were labelled with 14 different thoracic pathologies along with some preliminary algorithms. CheXNet then labelled images that have pneumonia as one of the annotated pathologies as positive examples and labelled all other images as negative examples for the pneumonia detection task. Pneumonia was chosen since chest x-rays are known to be currently the best available method for diagnosing the condition.

The research group then collected a test set of 420 frontal chest x-rays, with annotations obtained independently from four practicing Stanford radiologists to label all 14 pathologies. The performance of an individual radiologist was then evaluated by using the majority vote of the other three radiologists as ground truth. Similarly, the group evaluated CheXNet using the majority vote of 3 out of 4 radiologists, repeated four times to cover all groups of 3.

The performance of CheXNet was compared to that of the four practising academic radiologists on a test set against sensitivity and specificity. The computer-based tool works by producing heat maps, where instead of temperature in the regular heat maps, the variety of colours represent areas that the algorithm determines are most likely to show pneumonia. A single radiologist’s performance is represented by an orange marker, while the average is represented by green. CheXNet outputs the probability of detecting pneumonia in a chest x-ray, and a blue curve is then generated by altering the thresholds used for the classification boundary. The heat maps subsequently showed the sensitivity-specificity point for each radiologist and for the average, lie below the blue curve, indicating that CheXNet was able to detect pneumonia at a level matching or exceeding radiologists.

The research group has promised to extend CheXNet across the remaining other 14 pathologies within the dataset. It would also be interesting to see how deep machine learning can be incorporated into diagnostic radiology, considering the importance of imaging examination tools used in practice, screening, diagnosis and management of diseases. With automation at the level of experts, neural networks may be able to improve healthcare delivery and increase access to medical imaging expertise where access to skilled radiologists may be limited.