Recent findings in a cross-sectional JAMA Network study reveal that AI chatbots could provide high-quality and empathetic responses to patient inquiries. In this study, a team of licensed healthcare professionals compared responses to patient questions generated by both the AI chatbot and physicians. The chatbot’s responses were longer and preferred over the physicians’ responses, with significantly higher marks for both information quality and empathy. This calls for further investigation into the role of AI assistants in clinical settings. JAMA’s question was, “Can an artificial intelligence chatbot assistant, provide responses to patient questions that are of comparable quality and empathy to those written by physicians?”
As an AI Conversational UI/UX designer, I am excited to see how ChatGPT AI-powered conversational chatbots can enhance the future of patient care.
1. JAMA Network Study – Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions
The study evaluated the ability of an AI chatbot (ChatGPT) to provide quality and empathetic responses to 195 patient questions randomly drawn from Reddit’s r/AskDocs, a public online forum. The researchers then compared the AI chatbot responses to the original physician responses from the forum. The study took place in a blinded manner, with licensed healthcare professionals evaluating the responses.
2. Key Findings
-
-
Chatbot responses were longer and preferred 78.6% of the time.
-
Chatbot responses were rated significantly higher for information quality and empathy compared to physician responses.
-
The study suggests the potential benefits of AI chatbots in drafting responses to patient inquiries.
-
Chatbot responses were rated significantly higher for both quality and empathy, even when compared with the longest physician-authored responses.
However, limitations of the study include using a public forum for question samples (which may not reflect the typical patient-physician questions) and possible bias among assessors who were also coauthors of the study.
3. Future Implications
While the study shows promising possibilities for AI assistants in drafting responses to patient questions, further research is needed to evaluate the effectiveness of such chatbots in clinical settings. Incorporating AI chats to complement existing processes for message-based care delivery could potentially improve healthcare professionals.