Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions (2024)

Associated Data

Supplementary Materials

Key Points

This cross-sectional study of question-answer pairs in an online health forum for eye care questions compares AI-generated responses from a large language model chatbot with advice from American Academy of Ophthalmology–affiliated physicians.

Abstract

Introduction

In recent years, large language models (LLMs) have revolutionized natural language processing, helping computers to interact with text and spoken words just like humans, resulting in the creation of the chatbot. These models, including bidirectional encoder representations from transformers (BERT) and generative pretrained transformer 3 (GPT-3), are trained on massive amounts of text data and excel at natural language processing tasks such as text summarization or responding to queries.^1,2 They have been used for a wide range of applications in health care, including predicting length of postsurgical hospital stay, captioning medical images, summarizing radiology reports, and named entity recognition of electronic health record notes.^3,4,5,6

Among these models, ChatGPT (OpenAI) has emerged as a particularly powerful tool based on GPT-3.5 that was designed specifically for the task of generating natural and contextually appropriate responses in a conversational setting. Building on the GPT-3 model, GPT-3.5 was trained on a larger corpus of textual data and with additional training techniques like Reinforcement Learning from Human Feedback (RLHF), which incorporates human knowledge and expertise into the model.^7,8 This chatbot is an implementation of GPT-3.5 fine-tuned on conversational data, allowing it to generate appropriate responses to user input in a conversational context.⁸ Since its release in November 2022, it has been applied to simplify radiology reports, write discharge summaries, and transcribe patient notes.^9,10,11

Commensurate with this exciting potential is the need for prudence; incorporation of LLMs into clinical practice necessitates cautionary measures. Patients commonly turn to the internet for quick and accessible health information or advice,¹² and a major concern is whether information generated by LLM chatbots are safe and comparable with information from a physician. Concerns with chatbot use in health care include limited, outdated knowledge, incorrect citations, and inaccurate content with risk of hallucination—outputs that sound convincingly plausible yet are factually inaccurate.¹³ Chatbots have the capability of producing empathetic-sounding responses of high quality,¹⁴ but in ophthalmology, a recent study found that they were only 45% accurate as a source of patient information on common retinal diseases.¹⁵ Given the rapidly evolving landscape of artificial intelligence (AI)-driven health care and the potential for both transformative advancements and unintended consequences, rigorous studies examining the clinical effectiveness, safety, and ethical implications of AI-powered technologies are essential for optimizing patient outcomes and mitigating harm. This study evaluates how an LLM chatbot can be used to answer patient questions related to eye health, and how its answers compare with those of board-certified ophthalmologists.

Methods

Results

From the initial data set of 4747 question-answer pairs, the final sample comprised 200 questions with a median (IQR) length of 101.5 (76.0-143.25) words. The median (IQR) length of chatbot responses was significantly longer than human answers (chatbot, 129.0 [101.0-162.5] words vs human, 77.5 [57.0-104.0] words; t = 6.71; P < .001). Human answers were written by 9 unique board-certified ophthalmologists, with a median (IQR) of 30.7 (28.3) years of board certification, with practices in comprehensive ophthalmology, cornea, glaucoma, pediatrics and strabismus, and retina.

As a group, the expert panel was able to distinguish between the chatbot and human answers, more frequently rating AI answers as probably or definitely written by AI compared with human answers (prevalence ratio [PR], 1.72; 95% CI, 1.52-1.93) (Figure 1). However, a high percentage of responses were rated as definitely AI-written, including 320 (40.0%) of the human-written answers. The mean accuracy of the expert panel for distinguishing between AI and human answers was 61.3%, with individual rater accuracies ranging from 45% to 74%.

The expert panel rated the chatbot and human answers similarly on whether they contained incorrect information, aligned with the perceived consensus in the medical community, were likely to cause harm, and the extent of harm (Table). Thus, there were no statistically significant differences between chatbot and AI answers on these measures of quality and harm. Compared with human answers, chatbot answers were equally likely to contain incorrect or inappropriate material in the answer (PR, 0.92; 95% CI, 0.77-1.10). Compared with human answers, chatbot answers did not have a different likelihood of harm as human answers (PR, 0.84; 95% CI, 0.67-1.07). Chatbot answers were not perceived to be significantly more harmful than human answers (PR, 0.99; 95% CI, 0.80-1.22) (Figure 2).

Several patient questions exemplify answers given by either the chatbot or humans that raters deemed to contain incorrect information, were opposed to perceived consensus in the medical community, or had likelihood of causing some degree of harm (eTable 3 and eTable 4 in Supplement 1). For instance, a chatbot response to a forum post describing an eye shrinkage following cataract surgery incorrectly asserts that removal of the cataract can cause a decrease in the size of the eye, whereas a correct response would have described the risk of ptosis following cataract surgery, which may have made the eye appear smaller but not actually shrinking the eye. In another error, the chatbot states that posterior vitreous detachment could change astigmatism and results in need for updated glasses prescription, thus providing inaccurate information. These errors suggest that chatbots may hallucinate incorrect information in their responses.

Discussion

Our study is the first to evaluate the quality of ophthalmology advice generated by an LLM chatbot in comparison with ophthalmologist-written advice. A panel of expert ophthalmologist reviewers could discern human vs chatbot-written responses with approximately 61% accuracy, but chatbot and human responses did not significantly differ in terms of presence of incorrect information, likelihood of causing harm, extent of harm, or agreement with perceived consensus in the medical community. Although the chatbot was trained on a general domain of corpora,⁸ it could generate surprisingly coherent and correct answers to many ophthalmology questions, some of which were quite detailed and specialized. Our results suggest that LLMs can provide appropriate ophthalmic advice to common patient questions of varying complexity.

Our findings were consistent with several previous studies that suggested LLMs are capable of performing a wide variety of medical tasks. Google Research developed and tuned their Pathways Language Model (PaLM) to the medical domain using instruction prompt tuning, resulting in Med-PaLM, which scored 85% on US medical licensing examination–style questions, suggesting that LLMs can apply medical knowledge.^19,21 Chatbots have demonstrated passing examination scores in multiple specialty domains, including pathology, oncology, obstetrics and gynecology, and in ophthalmology, on the Ophthalmic Knowledge Assessment Program.^{15,22,23,24,25} Other LLM applications include simplifying radiology reports, writing discharge summaries, drafting operative notes, and facilitating palliative care discussions.^9,10,26,27 Given the high proportion of adults seeking health information online, it is highly likely that patients have already begun submitting medical questions to LLM chatbots.¹² Several previous studies have investigated the ability of chatbots to respond to patient questions, but generally without comparison with human physicians’ answers. Samaan et al²⁸ found that 86.8% of chatbot responses to questions related to bariatric surgery were “accurate and comprehensive.” A panel of 4 plastic surgeons found the LLM chatbot capable of providing “coherent answers that were easily comprehended and sufficiently informed” for common questions related to rhinoplasty.²⁹ Chatbot answers to 4 common cardiovascular patient questions were deemed trustworthy, valuable to patients, and of minimal harm.³⁰ However, other studies highlight limitations on the ability of chatbots to educate patients. Two transplantation hepatologists queried an LLM chatbot, finding it only 79.1% accurate and 47.3% comprehensive on questions related to cirrhosis, and 74.0% accurate and 41.1% comprehensive on questions on hepatocellular carcinoma.³¹ In ophthalmology, 2 clinicians assessed chatbot responses to questions on vernal keratoconjunctivitis, finding potentially dangerous responses related to treatment, such as omitting information on key adverse effects.³² However, it is likely that safer responses could have been produced using careful prompt design. For example, our prompt to the chatbot sought to eliminate false confidence through explicit instruction to “refer the patient to see their ophthalmologist” if “not highly confident in their response to the patient.” To our knowledge, only 1 prior study directly compared advice produced by an LLM chatbot with advice written by humans. Ayers et al¹⁴ collected data from an online social media forum, Reddit’s AskDocs subreddit, where users ask questions to moderator-verified health care professionals. One hundred ninety-five user questions and responses from an LLM chatbot and physicians were reviewed in a side-by-side fashion by an expert panel, who preferred chatbot over physician responses for 78.6% of questions. They also rated chatbot responses of higher quality and empathy.¹⁴ A strength of our study is that we also were able to compare physician and chatbot answers to patient questions; in addition, we masked our expert reviewers as to the origin of the answer and found that neither chatbot nor human-written responses were superior.

The implications of this study are significant for the field of ophthalmology and health care more broadly. With the increasing use of digital technologies in health care, including chatbots and other AI-powered tools, it is crucial to assess the accuracy, safety, and acceptability of these systems to both patients and physicians. Regardless of whether such tools are officially endorsed by health care providers, patients are likely to turn to these chatbots for medical advice, as they already search for medical advice online.¹² While LLM-based systems are not designed to replace human ophthalmologists, there may be a future in which they augment ophthalmologists’ work and provide support for patient education under appropriate supervision. For example, LLMs could engage with patients prior to their ophthalmologist appointment, offering preliminary information on common eye health concerns. After the visit, LLMs could summarize the key points, allowing the ophthalmologist to direct their attention toward the more complex and nuanced aspects of the patient’s needs. This approach could empower the patient with personalized education while saving the ophthalmologist valuable time that could be applied to more complex and challenging cases. Additionally, LLM integration could enhance clinical workflows by augmenting electronic health record communication systems, such as In Basket (Epic Healthcare Systems).³³ LLMs can draft messages for patients asking for medical advice in an asynchronous fashion, and can generate suggested responses for common queries or requests from patients or other care team members, a feature particularly useful for routine inquiries or administrative tasks. LLMs could also potentially improve access to health care advice for patients who may not have access to an ophthalmologist, particularly in underserved areas; indeed, some patient questions explicitly mentioned that they turned to the Eye Care forum for advice because they did not have easy access to a local ophthalmologist. However, prior to deployment in clinical settings, these potential use cases would need rigorous study in specific contexts to determine their feasibility and safety.

While LLMs undoubtedly hold tremendous promise, it is crucial to examine its potential drawbacks and the potential harm. LLMs are prone to generating incorrect text, known as “hallucinations.”³⁴ In our study, an example that may be considered a hallucination could be the chatbot’s assertion that “removal of the cataract can cause a decrease in the size of the eye,” made in response to a patient query about whether cataract surgery could “shrink” the eye (eTable 3 in Supplement 1). Although this study demonstrates that the overall potential harm of chatbot-generated ophthalmology advice does not significantly differ from human-written advice, it still shows that LLM chatbots can generate outputs with the potential to cause harm. This potential harm is compounded by how the output generated by chatbots often exhibits a striking resemblance to human-written text; although our results suggested that raters were more likely to rate human answers as coming from humans and AI answers as coming from AI, the overall accuracy was still fairly low and for some individual ophthalmologists was less than 50%. Hence, LLM output would be even more likely to deceive nonexpert patients into believing that that content was written by humans, leading to a false sense of trust. Thus, the ideal approach for clinical applications of LLMs may be in aiding ophthalmologists, rather than serving as a patient-facing AI that substitutes their judgment. Moreover, although the chatbot was instructed to answer as a human and not to reveal its identity as an AI for the sake of the current study, any advice provided by LLMs in actual deployment should disclose its AI-generated nature. Another potential harm revolves around patient data, protected health information (PHI). To generate advice that is tailored to the patient’s unique circ*mstances, PHI would need to be input to the LLM. As ChatGPT is not an open model, it is important to note that OpenAI’s privacy policy explains that they “may collect Personal Information that is included in the input.”³⁵ In the clinical deployment of LLMs, policies should include strategies to protect PHI.

Conclusions

LLMs hold significant promise for improving the quality and efficiency of patient care with its ability to generate responses to often complex and nuanced medical queries, as demonstrated in this study of patients’ ophthalmology queries. Additional research is needed to assess patient attitudes toward LLM-augmented ophthalmology, to evaluate clarity and acceptability of LLM-generated answers from the patient perspective, test the performance of LLMs in a greater variety of clinical contexts, and to determine an optimal manner of utilizing LLMs that is ethical and minimizes harm. Furthermore, it may soon be possible to develop domain-specialized LLMs via fine-tuning models on multimodal ophthalmologic data. Ultimately, the undeniable reality is that LLMs have emerged and are accessible to the general public. We intend for this study to catalyze more extensive and nuanced dialogue and joint efforts surrounding the use of LLMs in ophthalmology among various health care stakeholders, including patients, clinicians, researchers, and policy makers. The primary goal is to prudently leverage these early research findings to shape the responsible implementation of LLMs in the field of ophthalmology.

Notes

References