Sensitivity vs. specificity: The eternal AI debate

Which is more important when working with healthcare AI detection: that it never misses something on a patient scan, or that it never identifies something that isn’t there?

By ELAD WALACH

Which is more important when working with healthcare AI detection: that it never misses something on a patient scan, or that it never identifies something that isn’t there? In other words, which should you be more concerned with: AI sensitivity or specificity?

As one comes at the expense of the other, the question should be what the right balance is of the two. But if you asked a 100 people, I believe that most of them would say sensitivity holds the utmost importance. Indeed, missing something critical on a scan could lead to a disaster. Depending on how a particular AI solution fits into the healthcare pipeline, one miss could cause a patient their life. On the other hand, if an AI system flags a false abnormality, it could cause extra expenses on unnecessary tests. Would this be so bad?

Radiologists know that there’s no simple answer regarding the importance of high sensitivity or specificity. The specific use case, the role of the solution in the healthcare journey, and the prevalence of a disease detected all have an impact on what makes ‘good’ specificity. Ultimately, AI accuracy must be tailored to the specific use-case and pathology, in order to truly provide value ‘in the wild.’

Specificity is compounded