17% Use AI Chatbots for Health Questions Despite Hallucination Risk

For machine learning engineers and AI product teams, this advisory is not merely a consumer warning—it is a public evaluation failure. Users are over-trusting confident outputs from ChatGPT, Microsoft Copilot, and Google Gemini despite documented fabrication and mixed-up details. The finding that many people cannot distinguish an AI-generated medical answer from a physician's raises hard questions about safety, evaluation benchmarks, and deployment guardrails for general-purpose models in high-stakes domains.

Consumer Reports has issued a consumer advisory warning that the growing use of general-purpose AI chatbots for health questions is outpacing evidence of their reliability. The guidance, distributed through local news affiliates on August 11 and August 12, 2026, highlights a survey finding that 17 percent of adults now ask AI tools such as ChatGPT, Microsoft Copilot, and Google Gemini health questions at least once a month. The appeal is easy to understand: the systems return detailed, readable answers in seconds, and they often sound authoritative. But that authority is exactly the problem, because the underlying models can confidently produce fabrications or mix up critical details in ways that are difficult for a nonexpert to detect.

“

Users are over-trusting confident outputs from ChatGPT, Microsoft Copilot, and Google Gemini despite documented fabrication and mixed-up details.

Kevin Loria, a health reporter at Consumer Reports, acknowledges that chatbots "can be quite good at generating a lot of information" and are "very comprehensive and easy to understand." Yet he also warns that "it's really hard to know when there are inaccuracies" and that "fabrications can be hard to detect." This asymmetry is compounded by a separate study finding cited in the consumer warning: people tend to over-trust AI-generated medical advice, and many respondents could not tell the difference between an answer written by an AI and one from a real doctor. For a patient worried about chest pain, a new medication, or a child's fever, that inability to discriminate carries real clinical risk.

The trust problem interacts with a privacy problem. Loria notes that "AI systems are not bound by the same privacy laws as your doctors' offices." General-purpose chatbots are not typically covered by HIPAA, so the symptoms, medications, or genetic details a user types into a prompt may be retained, used for training, sold by data brokers, or exposed in a breach. Consumer Reports therefore advises users to keep sensitive personal health information private, to treat AI output only as a starting point, and to verify any information through trusted sources such as MedlinePlus, the CDC, or a recognized medical organization. The guidance is unambiguous on clinical boundaries: chatbots should not be used for diagnosis or treatment, should never replace a conversation with a doctor when a decision will affect one's health, and should never be used in a medical emergency, when the correct action is to call 9-1-1.

For the healthcare system, the practical implication is that a substantial minority of patients may arrive at appointments influenced by chatbot-generated material they cannot fully evaluate. Clinicians may need to ask patients what online tools they have used, correct misinformation, and document AI-mediated health concerns. Health IT and telehealth platforms, in turn, face pressure to integrate validated, citeable medical knowledge bases and to make their own AI features explainable and privacy-protective. At the same time, this is not a call to reject AI in health entirely. The Consumer Reports framing treats chatbots as useful for orientation and patient education, but not as a substitute for licensed clinical judgment.