The TB Clinical Diagnostic Research Consortium (CDRC), an organization funded by the National Institute for Allergy and Infectious Diseases of the National Institutes of Health and managed by Johns Hopkins University, recently challenged the company to correctly identify 60 patient slides prepared from cases gathered from their work in Uganda. The diagnosis, unknown to the company, was independently verified by two sets of 3 microscopists in two laboratories in Uganda.
The CDRC conducts feasibility studies of new diagnostic technologies. Members include the Johns Hopkins University; Imperial College of London in the United Kingdom; Infectious Diseases Institute in Kampala, Uganda; Boston Medical Center, University of Cape Town, South Africa; Universidade Federal do Espirito Santo in Vitoria, Brazil; University of Medicine and Dentistry of New Jersey – New Jersey Medical School; National Masan Tuberculosis Hospital-Yonsei University College of Medicine in the Republic of Korea; Foundation for Innovative New Diagnostics (FIND), and Westat Inc.
This test was of special interest to the company as it continues to further validate its TBDx platform and the recent advances made in its detection algorithms. Additionally this test follows recent successes in the evaluation of 100 panel slides provided by the Research Institute of Japan and precedes a larger evaluation of the technology that will take place at the National Health Laboratory Services schedule for April in Johannesburg, South Africa.
- Using patent-pending technologies it took the camera 1 minute, 15 seconds to establish the one-time auto-focus needed to acquire the images. Once this calibration is determined no additional focusing is necessary.
- Separately, TBDx captured 100 Fields of View (FOV) from 3 different acquisition patterns, for a total of 300 digital images.
- TBDx took 1 minute, 10 seconds to acquire each set of 100 images, or 3.5 minutes. In total, to focus, capture 300 FOVs and classify the cases as positive or negative was 5 minutes.
- TBDx classified 51 of the 60 slides provided by CDRC. The 9 unprocessed slides contained smears that did not have sufficient sputum for computer-vision technology to calibrate the autofocus for the camera.
- The TBDx case results were submitted to CDRC, who maintained the correct classifications.
In evaluating performance the two key measurements are sensitivity and specificity. How accurately can the technology correctly classify a case as positive (sensitivity)? How accurately can the technology correctly classify a case as negative or normal (specificity)? TBDx performed as follows:
|Truth +||Truth –||TOTAL|
Sensitivity= TP / TP+FN or 26/27 or 96.30%
Specificity = TN / TN+FP or 21/24 or 87.50%
- TBDx was 100% correct in classifying the heavily TB-burdened cases ( 1+ / 2+/ 3+ )
- In a review of the data, TBDx requires fewer than 100 FOVs to classify 2+ and 3+ cases. Equally, correctly classifying cases as 1+ and Scanty was improved when acquiring 300 FOVs.
- Acquiring 300 FOV did not produce a large percentage of false positive cases, adversely impacting specificity. The technology correctly classified 21 of the 24 normal cases.
- In a CDRC review of the false positive cases (3), specifically the objects that TBDx identified as Acid Fast Bacilli, they agreed that one object likely was AFB and missed by the microscopists.
These results are a very positive validation that reflects the improvements made in the TBDx detection technology. Also, the results obtained mirror internal test results. This consistent level of performance is a very good indicator as we move into upcoming technology evaluations in South Africa, Nigeria, and Uganda.
These tests continue to validate our long-held view that computer vision technology can greatly aid laboratory processes that rely upon human vision for diagnostic purposes. There are great opportunities for this technology, not only for TB but also for a host of infectious disease applications that utilize slides and microscopes.