AI chatbots are emerging as invaluable tools for helping providers make disease diagnoses and navigate difficult medical decisions. Whereas conditions commonly follow diagnostic algorithms or clinical practice guidelines, recommendations for treatment-for instance, preoperative stoppage of anticoagulants for procedures-may call for judgment calls. One research team led by Jonathan H. Chen, MD, PhD, sought to determine whether chatbots could assist in these decisions, and whether they made a difference in the way a physician performed.
What does the report say?
The report indicated that the AI-powered chatbot beat out physicians who consulted only the medical literature or conducted searches through alternative medical databases. However, the chatbot users performed at par with the AI, thus emphasizing human-computer collaboration over the disparate actions of each side. This supports the belief that AI best serves as an enhancement of human expertise, rather than a substitute thereof.
The study builds on a previous paper in JAMA Network Open, which made the case for chatbots diagnosing more accurately than doctors. The current study assessed “clinical management reasoning”-that is, the decision-making process underpinning treatment plans, which balances risk, patient preference, and medical best practices.
What was the trial result?
To test AI’s capabilities, researchers designed a trial with three groups: chatbots alone, doctors with chatbot support, and doctors using only traditional resources. Each group was given five de-identified patient cases and had to determine the best course of action. Their responses were evaluated by a panel of board-certified doctors using a standardized rubric. The chatbot alone performed better than doctors relying solely on medical references, while doctors with AI assistance performed equally as well as the chatbot itself.
The findings raise an important question: Are AI doctors the future? While chatbots can provide valuable insights, Chen stresses they should not replace human physicians. Instead, AI should serve as an assistive tool, enhancing clinical decision-making while doctors ensure credibility and contextual understanding. As AI continues to evolve, its role in medicine is likely to expand, offering a promising future where technology and human expertise work hand in hand.
