Google’s AI search features did not consistently direct young users toward emergency or medical help when their questions suggested a mental health crisis. In a safety test, AI Overview delivered appropriate emergency referrals in only about 58% of cases that were expected to trigger them.
AI Mode performed better at roughly 77%, yet it also remained below the researchers’ 95% safety benchmark. The findings raise concerns because an inadequate response may fail to recognize an urgent warning from a child or teenager.
Emergency Referral Results
| Google Feature | Appropriate Emergency Referrals | Research Benchmark |
|---|---|---|
| AI Overview | About 58% | 95% |
| AI Mode | About 77% | 95% |
The Youth AI Safety Institute at Common Sense Media conducted about 2,600 interactions with AI Overview and AI Mode between May and July 2026. Researchers used simulated accounts representing users aged 11 and 15.
Both test accounts had SafeSearch enabled and parental controls through Google Family Link. The results therefore concerned systems tested with child-focused safeguards already in place.
The research included 214 prompts that, according to the institute, should have resulted in a referral to emergency services or a medical professional. The prompts covered suicide, self-harm, eating disorders, psychosis, and child sexual exploitation.
Responses to High-Risk Language
One cited example involved slang that indicated suicidal intent. AI Overview responded with guidance on managing digital accounts after a person dies instead of directing the user to crisis support.
The institute said AI Overview and AI Mode failed to adequately protect children experiencing a crisis. Its report also said the systems could miss signs of suicidal intent, reinforce indications of psychosis and mania, validate eating disorders, and celebrate cannabis use.
The issue carries greater weight as AI-generated answers become more common in everyday searching. Common Sense Media reported that about three-quarters of children and teenagers had used AI answers in Google Search in 2026.
Inconsistent Answers for Schoolwork
The testing identified problems beyond crisis handling. Across more than 275 history questions, AI Overview and AI Mode gave different answers to the same prompt in about 43% of tests.
Those differences included accuracy, the depth of information, and the perspective presented to the user. Such variation may make it harder for students to decide whether a Google AI answer is reliable for learning.
Researchers also tested AI Mode on school assignments and said it completed every mathematics question and essay task presented to it. That capability created separate concerns about potential academic cheating among students.
Common Sense Media recommended that schools and families have the option to disable AI Overview and AI Mode for child users. For elementary school students, it recommended library databases or search engines that do not provide AI-generated answers.
Google Challenges the Methodology
Google rejected the report’s conclusion, saying the test methodology did not reflect real-world Google Search use. In a statement to Mashable, the company said many of the prompts were ambiguous and artificially constructed.
Google said its own internal testing using the same queries produced better-quality answers. The company also maintained that its AI Search tools are designed to help children and teenagers learn, explore information, and understand the world.
Parents can still use parental controls to restrict access to search services on Android devices and Chrome browsers. However, family and school supervision remains important while these AI features are available to underage users.
