Why IBM’s artificial intelligence “Watson” could not replace a physician

“Watson”, is an artificial intelligence computer system that answers questions posed in natural language, and is a product of IBM’s R&D department. Much of the buzz surrounding Watson occurred when it handily defeated two Jeopardy contestants last week.

After the defeat, IBM announced they would be working with Nuance, Columbia University Medical Center, and the University of Maryland Medical School on health care analytics research. The goal of the collaboration is to develop a commercial offering in the next 18 to 24 months that will exploit Watson’s capabilities to aid in the diagnosis and treatment of patients.

The win on Jeopardy and the announcement of this healthcare initiative for Watson has led some in the media to feel Watson can actually replace the diagnosing and treating physicians do with their patients — a CNN anchor even postulated that “Watson could do everything but operate” .

Does Watson have the potential to be helpful in healthcare?  Yes, but only if we understand its limitations. The following exchange between Google’s CEO Eric Schmidt and Surgeon Atul Gawande highlights the problems with using computer based algorithms in medicine.

Last year at the President’s Council of Advisors on Science and Technology (PCAST) meeting, Google’s Schmidt was befuddled as to why physicians hadn’t adopted the use of computer algorithms to diagnose patients. He stated:

“So when you show up at the doctor with some set of symptoms, in my ideal world what would happen is that the doctor would type in the symptoms he or she also observes, and it would be matched against the data in this repository…….As computer scientists, this is a platform database problem, and we do these very, very well, as a general rule. And it befuddles me why medicine hasn’t organized itself around these platform opportunities.”

Dr. Atul Gawande, a Harvard University surgeon, and author of The Checklist Manifesto, responded by saying:

I think part of the bafflement occurs because the folks who know how to make such systems don’t understand how the clinical encounter actually operates.”

He went on to state that the bigger issue with these types of algorithm searches is they produce more information than needed for a physician, who usually has 15 minutes to manage six problems. Dr. Gawande didn’t dismiss this type of computer decision support though — and finished his response to Google’s Schmidt by saying he would welcome a smartphone app that could actually help with patient care.

This type of exchange, showing a computer scientists understanding of clinical medicine, highlights why reports of Watson’s role in medicine are likely over exaggerated.  Medicine cannot be reduced to a set of complex algorithms because much of the data for these algorithms cannot even be inputted.  Those without training in medicine do not understand the multifaceted “behind the scenes” analysis that actually occurs when talking to a patient.

When physicians are asking patient’s their symptoms, we’re analyzing a complex amount of information that is not tangible and cannot be spoken or inputted into an algorithm:  Eye contact; Subtle physical movements; How they respond to questions – does their tone change when describing a particular symtom, leading me to believe I’ll uncover more information if I ask more about this; How they smell; How they are sitting; The reaction of family members when the patient responds to a particular question; What they are wearing; Any signs of underlying trauma; and much more.

There are so many more things being analyzed that are not included in the above list — and it all occurs within seconds.  And depending on each of the above and more, my questions for the History and Physical (H&P) will change, as will my treatment plan.  It’s why we’re taught in medical school that the H&P is the most important part of the exam.

No matter how good you are at diagnosing and treating, unless you asked the right questions in a timely manner, all the knowledge in the world won’t be helpful.  I’m sure an artificial intelligence program could produce a rudimentary H&P, but far from a focused and disease specific H&P a trained physician produces hundreds of times a month.  Some would argue it’s why physicians have a minimum of 7 years of post-graduate training (medical school + residency) before we have the sole responsibility of a patient.

At the end of the day, algorithms are only a guide, and you have to use your own clinical judgement, because each patient is unique in their own way.  And one of the points Dr. Gawande mentioned in his response to Google’s Schmidt speaks volumes – “Time”.  You don’t necessarily have time to input all the “data” to even utilize a computer support system.

One of the reasons I love Emergency Medicine is often you don’t have the necessary time to talk to a patient for more than a few moments before starting some sort of life saving treatment or procedure. You have a finite amount of time to save a person’s life, so you better ask the right questions, and no matter how well tuned an artificial intelligence program is, adding another layer to treatment requires time — and with the acuity of some medical emergencies, minutes could be the difference between life and death.

So could Watson be used in healthcare? As a decision support tool that is combined with an electronic medical record — sure.  But to replace a physician – negative.

CNN video of Watson in Medicine.


Iltifat Husain, MD

Founder, Editor-in-Chief of iMedicalApps.com. Assistant Professor of Emergency Medicine and Director of Mobile App curriculum at Wake Forest University School of Medicine. He is also the founder of iPrescribeApps, a platform for prescribing apps to patients. Dr. Husain has given lectures on digital medicine globally. He went to North Carolina State University for undergrad and went to medical school at Wake Forest School of Medicine.

Follow Me

Click to view 22 Comments

22 Responses to Why IBM’s artificial intelligence “Watson” could not replace a physician

  1. Nayana Somaratna February 21, 2011 at 8:08 am #

    Anyone who has worked with an experienced doctor knows that it is his / her ‘clinical sense’ which plays a key role in the mangement of the patient – not just knowledge per se. Sometimes the diagnoses made / management ordered appears to be illogical and counter-intuitive – until they turn out to be correct in the end !

    Unfortunately, it is only medical professionals who understand this – just try explaining it to a vanilla software developer who looks at a book of diagnostic algorithms and thinks ‘aha, this is simple to implement as an expert system’.

  2. Marc-Emile February 21, 2011 at 4:43 pm #

    I agree. I think the human interaction is very difficult to put in an algorithm. The more I practice medicine, the more obvious this becomes.

    However, I would be careful in saying it can’t be done. Never say never 😉 Maybe not in 10-20 years, but I expect I’ll see something like that (a “robot doctor”) in my lifetime. We’ll see :-)

    • Iltifat Husain February 22, 2011 at 1:23 pm #

      “Never Say Never” – Very true!….should have changed it to…”not in the near future”..

  3. panderer March 2, 2011 at 1:07 am #

    I understand that human interaction will have a supremely important role for a very long time. I even understand the illustration that an ER doc maybe has minutes/seconds in which their intuition guides them down the correct diagnostic path.

    However, I work in a hospital laboratory where I see the benefit of a Watson-like computer system. Often at great cost to the patient, we see large batteries of tests that are ordered on a patient as a shotgun approach. The tests may even have similar function (sed rate, CRP, rheumatoid factor, etc), some with less specificity than others. And surprisingly, sometimes we even have ER docs, as well as other clinicians, call the lab to inquire on what a specific test is used for and more often, what those results mean.

    Watson is meant to “listen over the doctor’s shoulder” as he is doing his H&P, and later offer ideas both on treatment as well as proper diagnostic paths. This would be powered by Watson’s Nuanced ability to understand natural language. The doctor can even interact with Watson in the same way as talking to another physician for a 2nd opinion. Later I imagine that lab and other diagnostic test results can also be gathered back in for an even more accurate diagnosis.

    Watson for me is legitimately exciting and not to be blown off because it can’t do everything as well as a human. Maybe not everything, but certainly in some ways it can be superior to what we do today.

    • Iltifat Husain, MD March 18, 2012 at 1:45 am #

      Yep, great points. And that’s exactly what I say in the last few sentences of the article — how Watson won’t replace, but can be a good decision support.

  4. Saorr March 5, 2011 at 5:45 am #

    As a reasonably experienced clinician I would suggest that the clinical interaction with the patient and their family is full of subtle findings and nuance. Often the presenting complain is mismatched to the classical presenting complaint for a given disease. I am humbled nearly every day at atypical symptoms presenting atypically and misinterpreted by the patient.

    Diseases manifest differently in different patients.

    Symptoms present differently and are experienced differently in different patients.

    Different patients have concomitant diseases that modify how disease affects their physiology. Some patients are partially impaired and “just feel sick”.

    Language for most patients inadequately describes and inaccurately explains the patient’s unique experience.

    I have gone back and forth on how many tests to order. It is not cost effective to order too many tests and it is not cost effective to have the patient keep coming back and tip-toe one test at a time, especially if an ambulatory patient works or is busy.

    I would like to see some real, high quality studies on the accuracy of Watson in the real world, in the real combat-like situation of clinician’s offices. Since I have evaluated over 100,000 patient visits already I have a clinical experience that does give me a sense of when to order a scheduled MRI of the brain versus an urgent MRI… or other tests. I have been audited – the conclusion –> I was NOT ordering enough studies given my yields. Keep in mind, if 100% of a given test ordered demonstrate clinical pathology we are not ordering enough of that test since we would be missing pathology.

    If Watson’s accuracy approaches an experienced clinicians, then we should adopt it. Over 20 years ago authored an article on autonomous planes. 100% of readers surveyed would not ride in an aircraft piloted by an AI computer (with no human pilot on board). If Watson can match or exceed a human clinicians judgement and decision-making I would gladly accept a computer doctor, most people, even if it is cost effective would be reluctant to be managed by a computer.

    Also –> “I like my doctor” versus “My doctor is competent” are two completely different concepts.

    Likable doctors are not necessarily the most competent. Competent doctors are not necessarily the most likable. We live in an emotional world – patients’ relationship with their doctor functions on a trust level and their main way of assessing whether or not they can trust their doctor is by their interpretation of how they feel about their doctor largely based on features of emotional compatibility. This is largely not a cognitive process. Why? Because we evolved in systems originally engineered as a trusted tribe. If the tribe hunted, fished, cooked and sheltered together in a trusting situation there chances of survival improved. These traits are inherent in the human experience and apply to the doctor-patient relationship.

    When someone becomes ill their ability to self-assess becomes impaired, partially. As a result, their issued communications are less accurate. How can Watson interpret a distorted symptom set?

    I would suggest that a better application is to engineer Watson a doctor-colleague. As a wise sage able to converse intelligently with clinicians on unusual, rare cases. Many of these cases have hundreds of lab tests, dozens of scans, dozens of doctors… Coordination of care can be problematic.

    The other issue is I have invited “outsiders” to be with me for a day and shadow a high volume, intense clinical environment. I think Eric Schmidt would become a lot more befuddled at how we can actually accomplish as much as we can, given the sea of uncertainty that we swim in every day.

    Many more comments later on

    • Felasfa Wodajo March 6, 2011 at 1:22 pm #

      This is a great comment. An essay really.

      Communication is at the root of the doctor-patient interaction. The ability of caregivers, doctors and nurses alike, to make the patients feel like they are being treated with courtesy and respect, and most of all, that they are being listened to, dramatically affects patients’ perception of the quality of their care.

      Patients already assume that their doctor is competent.

      So, I think you’re right. Computerizing the doctor will not make the patient feel better. But, having a real smart colleague who remembers every detail, would indeed be great for doctors.

  5. 15lk November 11, 2011 at 10:08 am #

    artificial doctors can replace human physicians.they can be trained for that but they must be under general human control

  6. dan March 17, 2012 at 7:56 pm #

    I agree with Iltifat Husain. Of course doctors don’t want to be told their expertise is less valuable now after they spend hundreds of thousands of dollars and over a decade getting it, but the fact is that, given a proper list of symptoms and patient history, a computer like Watson armed with sufficient historical data can give a list of diagnoses with percentage likelihoods in a much more accurate manner than a doctor ever could. Do we still need a trained professional to make sure symptoms are properly accounted for? Of course, and I’m sure the patient appreciates the human touch. However, rather than nursing their wounded pride by blasting this technology, doctors need to start thinking about how best to implement this technology to provide the best patient care possible.

  7. John Doe March 25, 2012 at 12:11 pm #

    The NHS Direct website already offers a “self-service” diagnostic tool that simply allows you to point to the various areas of the body and it’ll ask you various questions.

    It is an absolute success, very popular here in the UK and this isn’t even using IBM’s incredibly powerful Watson tool.

    The simple fact is General Practitioners are incredibly overpaid for the service that they offer. Everybody thinks their GP can diagnose off the top of their heads, but the amount of them who just look your symptoms up on the internet anyway is astounding.

    Pharmacists and treatment practitioners are far more useful, and they, in combination with IBM’s Watson, will be the future of medical practice in the developed and developing worlds.

  8. RandyL March 28, 2012 at 2:52 am #

    I’m going to propose an even more radical future ….

    When both the New York Presbyterian Hospitals and the Sloan-Kettering Cancer Center incorporate Watson-like technologies into their daily rounds of internal medicine cases, starting 2013-2015 time frame, in the decade which will follow, much of internal medicine will be revolutionized to use expert systems in diagnosis.

    Here’s why … no single person has the breath of experience of an entire history of a collection of medical centers’ case files. Thus, accuracy will go up and treatment options will be more targeted. As a result, the costs override of misdiagnosis or shall we say, chasing one’s tail, will go down dramatically.

    This is the future and it’s already in motion. Thus, doctors better increase their level of specialization because general internal medicine will fade away and PAs will be able to handle a lot of the basic cold/flu, body ache type of patients. Every doctor will need to have a specialty or else, they may find themselves earning the salary of an ordinary engineer or postdoc-type in industry than a person in a highly protected profession like medicine.

    • Felasfa Wodajo, MD March 28, 2012 at 10:28 am #

      Those are insightful comments, thank you. I propose they will only be partially realized. I argue the explosion of health information will only make the role of a specialist more important not, less. However, there will likely be fewer of them since they can perform more of their practice at a distance.

      The role of primary care providers is even more critical. Without correct inputs, no software can function and a computer cannot elicit an accurate history from a nervous or imprecise patient. As for decreasing physician income, there are larger economic forces than software.

      Anybody can download free legal documents but they still take a lawyer to court. Similarly, sophisticated decision making software will assist highly trained specialists even more powerfully than lay people or generalists.

    • Iltifat Husain, MD March 28, 2012 at 11:30 am #

      Also, most primary care docs finishing residency currently are already making the same salary as middle to high end engineers and usually less than PHD’s in the pharma industry (i have friends who are in the job market, not what it used to be) — it’s why so many primary care spots for residency go unfilled. Seeing them make even less than engineers and PHD’s in industry after a minimum 7 year post graduate education and $400K in debt would be tough for them — just sayin.

  9. Dr Eman Mann April 4, 2012 at 12:35 am #

    All this does sound fascinating!…..but what about the financials, the cost involved and who will this cost be tranferred to….most probably the patients, who will end up paying for the human as well as the machine. AI has been a boon to mankind, but is it (???) so evolved that we can trust ERs to it yet!!

  10. Jake June 13, 2012 at 11:42 am #

    It is of course inevitable that computers will replace physicians (eventually). Even surgeons (eventually). I don’t think either will happen to any great extent in my lifetime, but with the expansion and increasing affordability of diagnostic tests … it will come to a point where a computer does a MUCH better job analyzing the mountains of data. Throw someone’s genome, epigenetic changes, existance / concentrations of every molecule, virus, bacteria, etc. and how that changed since the last visit, etc. (all things that will eventually be cheap and instant, perhaps within a few decades) at a human, ANY human, and its just too much. A computer could extract a lot out of that though, and weigh much better the relative risks and benefits of any drugs or therapy. All the visual (normal or from diagnostic scans / radiology) and social cues would be MUCH harder to get a computer to fully understand, but I don’t see any reason why AI couldn’t eventually do that better than humans as well.

    I don’t think any doctor alive today has to worry much though. Even if tools / computers start replacing some of their job fuctions, it will probably just open the door to more specialists and research.

  11. naga July 29, 2012 at 6:09 am #

    I am really surprised when i read that artificial intelligence and other technology (which is only going to be better in the near future) can not eventually replace doctors.I think that most docs are really sure about their profession and that is bad.we will see more automation in all fields and that is only beginning…

Leave a Reply