Deep learning, computer-aided radiography reading for tuberculosis: a diagnostic accuracy study from a tertiary hospital in India

Madlen Nash, Rajagopal Kadavigere, Jasbon Andrade, Cynthia Amrutha Sukumar, Kiran Chawla, Vishnu Prasad Shenoy, Tripti Pande, Sophie Huddart, Madhukar Pai, Kavitha Saravu

Research output: Contribution to journalArticle

Abstract

In general, chest radiographs (CXR) have high sensitivity and moderate specificity for active pulmonary tuberculosis (PTB) screening when interpreted by human readers. However, they are challenging to scale due to hardware costs and the dearth of professionals available to interpret CXR in low-resource, high PTB burden settings. Recently, several computer-aided detection (CAD) programs have been developed to facilitate automated CXR interpretation. We conducted a retrospective case-control study to assess the diagnostic accuracy of a CAD software (qXR, Qure.ai, Mumbai, India) using microbiologically-confirmed PTB as the reference standard. To assess overall accuracy of qXR, receiver operating characteristic (ROC) analysis was used to determine the area under the curve (AUC), along with 95% confidence intervals (CI). Kappa coefficients, and associated 95% CI, were used to investigate inter-rater reliability of the radiologists for detection of specific chest abnormalities. In total, 317 cases and 612 controls were included in the analysis. The AUC for qXR for the detection of microbiologically-confirmed PTB was 0.81 (95% CI: 0.78, 0.84). Using the threshold that maximized sensitivity and specificity of qXR simultaneously, the software achieved a sensitivity and specificity of 71% (95% CI: 66%, 76%) and 80% (95% CI: 77%, 83%), respectively. The sensitivity and specificity of radiologists for the detection of microbiologically-confirmed PTB was 56% (95% CI: 50%, 62%) and 80% (95% CI: 77%, 83%), respectively. For detection of key PTB-related abnormalities 'pleural effusion' and 'cavity', qXR achieved an AUC of 0.94 (95% CI: 0.92, 0.96) and 0.84 (95% CI: 0.82, 0.87), respectively. For the other abnormalities, the AUC ranged from 0.75 (95% CI: 0.70, 0.80) to 0.94 (95% CI: 0.91, 0.96). The controls had a high prevalence of other lung diseases which can cause radiological manifestations similar to PTB (e.g., 26% had pneumonia, 15% had lung malignancy, etc.). In a tertiary hospital in India, qXR demonstrated moderate sensitivity and specificity for the detection of PTB. There is likely a larger role for CAD software as a triage test for PTB at the primary care level in settings where access to radiologists in limited. Larger prospective studies that can better assess heterogeneity in important subgroups are needed.

Original languageEnglish
Number of pages1
JournalScientific Reports
Volume10
Issue number1
DOIs
Publication statusPublished - 14-01-2020

Fingerprint

Tertiary Care Centers
Radiography
Reading
India
Tuberculosis
Pulmonary Tuberculosis
Learning
Confidence Intervals
Area Under Curve
Sensitivity and Specificity
Software
Thorax
Pleural Cavity
Triage
Pleural Effusion
ROC Curve
Lung Diseases
Case-Control Studies
Primary Health Care
Pneumonia

All Science Journal Classification (ASJC) codes

  • General

Cite this

@article{d331a2d442d74da489f33b7b85ad1c57,
title = "Deep learning, computer-aided radiography reading for tuberculosis: a diagnostic accuracy study from a tertiary hospital in India",
abstract = "In general, chest radiographs (CXR) have high sensitivity and moderate specificity for active pulmonary tuberculosis (PTB) screening when interpreted by human readers. However, they are challenging to scale due to hardware costs and the dearth of professionals available to interpret CXR in low-resource, high PTB burden settings. Recently, several computer-aided detection (CAD) programs have been developed to facilitate automated CXR interpretation. We conducted a retrospective case-control study to assess the diagnostic accuracy of a CAD software (qXR, Qure.ai, Mumbai, India) using microbiologically-confirmed PTB as the reference standard. To assess overall accuracy of qXR, receiver operating characteristic (ROC) analysis was used to determine the area under the curve (AUC), along with 95{\%} confidence intervals (CI). Kappa coefficients, and associated 95{\%} CI, were used to investigate inter-rater reliability of the radiologists for detection of specific chest abnormalities. In total, 317 cases and 612 controls were included in the analysis. The AUC for qXR for the detection of microbiologically-confirmed PTB was 0.81 (95{\%} CI: 0.78, 0.84). Using the threshold that maximized sensitivity and specificity of qXR simultaneously, the software achieved a sensitivity and specificity of 71{\%} (95{\%} CI: 66{\%}, 76{\%}) and 80{\%} (95{\%} CI: 77{\%}, 83{\%}), respectively. The sensitivity and specificity of radiologists for the detection of microbiologically-confirmed PTB was 56{\%} (95{\%} CI: 50{\%}, 62{\%}) and 80{\%} (95{\%} CI: 77{\%}, 83{\%}), respectively. For detection of key PTB-related abnormalities 'pleural effusion' and 'cavity', qXR achieved an AUC of 0.94 (95{\%} CI: 0.92, 0.96) and 0.84 (95{\%} CI: 0.82, 0.87), respectively. For the other abnormalities, the AUC ranged from 0.75 (95{\%} CI: 0.70, 0.80) to 0.94 (95{\%} CI: 0.91, 0.96). The controls had a high prevalence of other lung diseases which can cause radiological manifestations similar to PTB (e.g., 26{\%} had pneumonia, 15{\%} had lung malignancy, etc.). In a tertiary hospital in India, qXR demonstrated moderate sensitivity and specificity for the detection of PTB. There is likely a larger role for CAD software as a triage test for PTB at the primary care level in settings where access to radiologists in limited. Larger prospective studies that can better assess heterogeneity in important subgroups are needed.",
author = "Madlen Nash and Rajagopal Kadavigere and Jasbon Andrade and Sukumar, {Cynthia Amrutha} and Kiran Chawla and Shenoy, {Vishnu Prasad} and Tripti Pande and Sophie Huddart and Madhukar Pai and Kavitha Saravu",
year = "2020",
month = "1",
day = "14",
doi = "10.1038/s41598-019-56589-3",
language = "English",
volume = "10",
journal = "Scientific Reports",
issn = "2045-2322",
publisher = "Nature Publishing Group",
number = "1",

}

Deep learning, computer-aided radiography reading for tuberculosis : a diagnostic accuracy study from a tertiary hospital in India. / Nash, Madlen; Kadavigere, Rajagopal; Andrade, Jasbon; Sukumar, Cynthia Amrutha; Chawla, Kiran; Shenoy, Vishnu Prasad; Pande, Tripti; Huddart, Sophie; Pai, Madhukar; Saravu, Kavitha.

In: Scientific Reports, Vol. 10, No. 1, 14.01.2020.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Deep learning, computer-aided radiography reading for tuberculosis

T2 - a diagnostic accuracy study from a tertiary hospital in India

AU - Nash, Madlen

AU - Kadavigere, Rajagopal

AU - Andrade, Jasbon

AU - Sukumar, Cynthia Amrutha

AU - Chawla, Kiran

AU - Shenoy, Vishnu Prasad

AU - Pande, Tripti

AU - Huddart, Sophie

AU - Pai, Madhukar

AU - Saravu, Kavitha

PY - 2020/1/14

Y1 - 2020/1/14

N2 - In general, chest radiographs (CXR) have high sensitivity and moderate specificity for active pulmonary tuberculosis (PTB) screening when interpreted by human readers. However, they are challenging to scale due to hardware costs and the dearth of professionals available to interpret CXR in low-resource, high PTB burden settings. Recently, several computer-aided detection (CAD) programs have been developed to facilitate automated CXR interpretation. We conducted a retrospective case-control study to assess the diagnostic accuracy of a CAD software (qXR, Qure.ai, Mumbai, India) using microbiologically-confirmed PTB as the reference standard. To assess overall accuracy of qXR, receiver operating characteristic (ROC) analysis was used to determine the area under the curve (AUC), along with 95% confidence intervals (CI). Kappa coefficients, and associated 95% CI, were used to investigate inter-rater reliability of the radiologists for detection of specific chest abnormalities. In total, 317 cases and 612 controls were included in the analysis. The AUC for qXR for the detection of microbiologically-confirmed PTB was 0.81 (95% CI: 0.78, 0.84). Using the threshold that maximized sensitivity and specificity of qXR simultaneously, the software achieved a sensitivity and specificity of 71% (95% CI: 66%, 76%) and 80% (95% CI: 77%, 83%), respectively. The sensitivity and specificity of radiologists for the detection of microbiologically-confirmed PTB was 56% (95% CI: 50%, 62%) and 80% (95% CI: 77%, 83%), respectively. For detection of key PTB-related abnormalities 'pleural effusion' and 'cavity', qXR achieved an AUC of 0.94 (95% CI: 0.92, 0.96) and 0.84 (95% CI: 0.82, 0.87), respectively. For the other abnormalities, the AUC ranged from 0.75 (95% CI: 0.70, 0.80) to 0.94 (95% CI: 0.91, 0.96). The controls had a high prevalence of other lung diseases which can cause radiological manifestations similar to PTB (e.g., 26% had pneumonia, 15% had lung malignancy, etc.). In a tertiary hospital in India, qXR demonstrated moderate sensitivity and specificity for the detection of PTB. There is likely a larger role for CAD software as a triage test for PTB at the primary care level in settings where access to radiologists in limited. Larger prospective studies that can better assess heterogeneity in important subgroups are needed.

AB - In general, chest radiographs (CXR) have high sensitivity and moderate specificity for active pulmonary tuberculosis (PTB) screening when interpreted by human readers. However, they are challenging to scale due to hardware costs and the dearth of professionals available to interpret CXR in low-resource, high PTB burden settings. Recently, several computer-aided detection (CAD) programs have been developed to facilitate automated CXR interpretation. We conducted a retrospective case-control study to assess the diagnostic accuracy of a CAD software (qXR, Qure.ai, Mumbai, India) using microbiologically-confirmed PTB as the reference standard. To assess overall accuracy of qXR, receiver operating characteristic (ROC) analysis was used to determine the area under the curve (AUC), along with 95% confidence intervals (CI). Kappa coefficients, and associated 95% CI, were used to investigate inter-rater reliability of the radiologists for detection of specific chest abnormalities. In total, 317 cases and 612 controls were included in the analysis. The AUC for qXR for the detection of microbiologically-confirmed PTB was 0.81 (95% CI: 0.78, 0.84). Using the threshold that maximized sensitivity and specificity of qXR simultaneously, the software achieved a sensitivity and specificity of 71% (95% CI: 66%, 76%) and 80% (95% CI: 77%, 83%), respectively. The sensitivity and specificity of radiologists for the detection of microbiologically-confirmed PTB was 56% (95% CI: 50%, 62%) and 80% (95% CI: 77%, 83%), respectively. For detection of key PTB-related abnormalities 'pleural effusion' and 'cavity', qXR achieved an AUC of 0.94 (95% CI: 0.92, 0.96) and 0.84 (95% CI: 0.82, 0.87), respectively. For the other abnormalities, the AUC ranged from 0.75 (95% CI: 0.70, 0.80) to 0.94 (95% CI: 0.91, 0.96). The controls had a high prevalence of other lung diseases which can cause radiological manifestations similar to PTB (e.g., 26% had pneumonia, 15% had lung malignancy, etc.). In a tertiary hospital in India, qXR demonstrated moderate sensitivity and specificity for the detection of PTB. There is likely a larger role for CAD software as a triage test for PTB at the primary care level in settings where access to radiologists in limited. Larger prospective studies that can better assess heterogeneity in important subgroups are needed.

UR - http://www.scopus.com/inward/record.url?scp=85077840341&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85077840341&partnerID=8YFLogxK

U2 - 10.1038/s41598-019-56589-3

DO - 10.1038/s41598-019-56589-3

M3 - Article

C2 - 31937802

AN - SCOPUS:85077840341

VL - 10

JO - Scientific Reports

JF - Scientific Reports

SN - 2045-2322

IS - 1

ER -