| A real-world primary care study tests Google’s AMIE diagnostic AI with patients at home, finding promising accuracy but clear safety and workflow limits. |
| (Nanowerk News) The aging population, workforce changes, and patients’ increasingly complex medical needs are all factors straining primary care systems around the world. Artificial intelligence tools have emerged as a potential means of extending the capabilities of health care workers, including in primary care settings. Large language models (LLMs), also known as chatbots, have performed extremely well in simulations. However, little is known about how these systems perform with real patients. |
| Researchers at Beth Israel Deaconess Medical Center (BIDMC) have conducted what they believe is the first prospective real-world study of a patient-facing conversational AI system in primary care, taking an important step toward evaluating the technology through the same kind of rigorous clinical testing used for other health care interventions. |
| The findings appear in The Lancet (“Conversational diagnostic artificial intelligence in ambulatory primary care: a prospective feasibility study”). |
| “Despite the theoretical performance of patient-facing AI in experimental settings, it is unclear how conversationally safe these encounters are in a real-world setting,” said Adam Rodman, MD, Director of AI Programs at the Carl J. Shapiro Institute for Research and Education at BIDMC. “Before we begin evaluating them as workload interventions, AI tools must first be shown to operate safely and acceptably in real care workflows. With this research, we set out to assess whether this kind of technology could improve both patient and physician experience.” |
| Rodman and colleagues in collaboration with researchers at Google evaluated the Articulate Medical Intelligence Explorer (AMIE), a medical AI chatbot that patients used from home after scheduling an urgent primary care appointment with a BIDMC physician. Through a secure text-chat interface, the system asked patients about their symptoms, gathered medical histories, provided possible diagnoses for patients to review with their doctor, and generated a summary for clinician review prior to the visit. |
| Conducted between April and November 2025, the study enrolled 114 patients, 98 of whom completed both the AI interaction and their primary care appointment. Every conversation was monitored in real time by a board-certified internal medicine physician who could intervene if a patient appeared at risk of harm, experienced significant emotional distress, asked to end the session, or if the physician identified a potential safety concern — such as the need to clarify symptoms, provide emergency care instructions, or address concern about a serious possible diagnosis. |
| Across all 98 completed encounters, none of the conversations required a safety-stop intervention. Supervising physicians identified one hallucination and provided additional clinical clarification in five cases. |
| “This study helps establish the baseline characteristics of such real-world conversations,” said co-first author Peter Brodeur, MD, a clinical fellow in cardiovascular medicine at BIDMC. “These baseline characteristics provide a foundation for all future prospective trials aimed at discovering when human-on-the-loop workflows may be appropriate.” |
| Patients appeared receptive to the technology. Patients rated the quality of AMIE’s conversations favorably — as did physician evaluators — across most measures. Moreover, patient attitudes toward AI improved after interacting with the chatbot and remained elevated following their appointment with a clinician. |
| “One of the most important questions was simply how patients would engage with a conversational AI system in a real clinical setting,” said co-first author Jacob Koshy, MD, MPH, a hospitalist in the Division of General Medicine at BIDMC. “By prospectively observing these interactions, we were able to better understand both the opportunities and the limitations of the technology in the context of routine patient care.” |
| While patients gave high marks to the system’s ability to listen, explain information, and help them feel at ease, concerns remained around two areas: trust in the confidentiality of information shared with the system and trust in the chatbot’s honesty and trustworthiness. Researchers said those findings underscore the importance of continued work to build patient trust as AI tools move into clinical care. |
| “Research suggests that patient trust in medical AI systems flows from relationships with their human providers,” said Rodman. “Future research will need to explore which interaction characteristics can build patient trust and how AI interactions can enhance the patient-physician relationship.” |
| The findings also suggest the technology may have practical value in clinical workflows. |
| Primary care providers were able to review an AI-generated transcript or summary before seeing their patients for 44 cases. In 75 percent of cases, physicians said doing so helped them prepare for the visit and in 57 percent of cases, they reported it may have influenced their clinical approach. In only one case did a provider report the interaction as somewhat harmful, citing concern that a patient may have experienced anxiety after AMIE included lymphoma among its possible diagnoses. |
| Rodman and colleagues emphasized that the study was designed to evaluate feasibility rather than to determine whether AI improves health outcomes. |
| “Our primary care practice — and our hospital — has a long history of rigorously monitoring the safety of the clinical care we provide, and this study serves as an exemplar of how the safety of AI may be assessed prior to clinical implementation,” said co-senior author Marc L. Cohen, MD, Clinical Chief of Primary Care, Division of General Medicine at BIDMC. “We believe the future involves continued involvement of human clinicians in patient care, but see a future where the triad of patient, human clinician, and AI help to provide our patients with the very best evidenced-based care possible,” said Cohen. |
