In this blog post we will delve further into the performance results of the TBDx™ evaluation in South Africa. Additionally, we will provide access to the slides presented by Dr. Nazir Ismail during the Union/CDC Late Breaker session on the final day of the 44th Union World Conference on Lung Health.
In the Union/CDC Later Breaker session, each panelist was given ten minutes to present their research and five minutes to respond to audience questions. We will report more details when the results are produced in a peer-reviewed publication.
The Performance Numbers In Context
- A 2011 Proof of Concept study illustrated the potential of TBDx™ and automated sputum microscopy. At that time the technology achieved a sensitivity of 75.8%. However, the specificity performance was far below routine microscopy and required improvement.
- The purpose of the 2013 evaluation was to measure the performance improvements of important software changes implemented since 2011. The 2013 evaluation also assessed whether TBDx™ could be a screening technology, where very specific scanty cases would be confirmed by GeneXpert technology. Additionally, a preliminary analysis of the economic impact was to be developed.
Key Performance Tables
The performance of TBDx™ was compared against culture, considered the diagnostic “gold standard”. This is reflected in the first table, treating scanty 1 either as positive or negative.
TBDx vs. Culture
For an assessment of TBDx as a screening technology, an algorithm was developed whereby all TBDx™ negative cases were treated as normal, all P1-P3 cases were treated as positive, and all scanty cases were tested using GeneXpert. Scanty 1 cases were treated either as positive or negative. The table below illustrates the number of GeneXpert tests that would be required and the combined performance of these two technologies.
TBDX+(Scanty) > GXP vs. Culture
Key Observations of the Evaluation Results
- In analyzing the performance improvements since the 2011 evaluation, the latest step-wise binary classifiers implemented in the detection software, shows a 120% improvement in specificity (43.5% to 95.7 %). The differences in sensitivity were immaterial, just 2%. This demonstrates substantial improvement. When combined with GeneXpert, specificity improves to 98.8%.
- In spite of a bias in the study, where the routine smear microscopy performance was unusually high (two microscopists reading the case slides each had 40 years of experience), the sensitivity performance delivered by the combination of TBDx™ and GeneXpert represents a 14% improvement (77.6% vs. 68.2%).
- As a screening technology, used in combination with GeneXpert, TBDx™ produced a sensitivity of 77.6%. When compared to the average sensitivity performance of a routine microscopist (60%) the combined performance represents an improvement of 30%. This is a significant improvement over today’s laboratory environment in highly TB-burdened countries. Additionally, this combination produces a specificity equal to an experienced microscopist.
- Preliminary economic impact analysis shows that using TBDx™ as a screening technology reduces the number of GeneXpert tests by 79.5% (207/1006) or by as much as 95.2% (49/1006). This performance results in a material economic savings.
The Potential of Automated Computer-Vision Microscopy
In reviewing these performance improvements we realize that advances can be made both in sensitivity and specificity. Combining TBDx™ with GeneXpert offers the potential to acquire additional fields of view, perhaps increasing from 300 to 600, which could further improve sensitivity. This leverages both the uniqueness of image analysis in detecting MTB and the strengths of molecular tests, which is very specific and can remove false positive cases. Can these combined technologies produce a sensitivity of 90% and a specificity that is above 95%? We believe such performance is imminently possible and represents the next set of benchmarks for image analysis technology.
Why These Results and This Diagnostic Technology is Important
According to the 2013 Global Tuberculosis Report published by WHO, “About 3 million people who developed TB in 2012 were missed by national notification systems.” Affordable, fast, and accurate diagnostic tests such as TBDx™ are desperately needed, and must be combined with more active case findings, where mass screening in high risk settings can begin to positively impact transmission rates.