Uncertainty is More Important to AI than Explainability

Someday in the near future, a primary care physician will sit down in front of a computer to pull up all of the medical information about the patient who just walked into her office. The information won't just be dumped to the screen as a big table of numbers. It will be thoughtfully curated and presented by artificial intelligence to help guide the physician towards the best diagnosis and treatment for her patient.

One way to make A.I. more palatable is to use systems that can explain why they make a particular prediction. The idea is that the physician can take the reasoning into account. She will be more likely to follow a recommended treatment if the evidence used by the algorithm came from a recently published systematic review than if it came from a horoscope. That makes sense.

But, restricting technologies to those that are easily explainable may be putting our algorithms in handcuffs. Human biology is incredibly complex and it is likely that complex algorithms will be required to make sense of it. If that is true, then the physician may end up disregarding all recommendations of the explainable A.I. because it simply doesn't work.

This vision for A.I. enabled personalized medicine is quite alluring. Patients receive better care at lower cost. Doctors caring for patients instead of combing through charts. Everybody wins - unless they lose.

Machine learning systems make mistakes. They pass along any biases from the data they were trained on to the problems to which they are applied. It is inevitable that A.I. systems will make mistakes in health care, and these mistakes will have real - potentially life-threatening - costs. As a result, many people are concerned about the role that "black box" machine learning models will play in the future of medicine.

Uncertainty is an alternative to explainability that places no constraints on the complexity of the underlying algorithm. That is, algorithms should provide more than just a prediction - they need to describe how confident they are in that prediction. After all, isn't that all we wanted in the first place? For the physician to know how much she should trust the algorithm?

A.I. and machine learning are going to play an important role in the future of medicine. Luckily, we already have the tools to build models that can describe their degree of confidence, even if they can't explain why.

Enter your email address to download paper.

Click the link to begin download.
Oops! Something went wrong while submitting the form.
Blog

Embracing Innovation to Move Forward

Podcasts

Your Digital Twin - UnlearnAI

Podcasts

Using AI Digital Twins for Drug Testing

Dr. Charles Fisher, CEO of Unlearn AI, discusses creating digital clones by using artificial intelligence for use in clinical drug trials.
A fascinating approach to the problem of how to make clinical trials more efficient, and understand more about what may be possible with more and better patient data.
At Unlearn, our goal is to use the data available from historical trials, to generate new evidence to inform and advance research.