AI Tools for Kids: What’s Safe and What’s Not

The short answer: a few AI tools are genuinely built with kids in mind, most general-purpose AI apps are not, and the difference matters more than most parents realize. Character.AI is currently facing a wrongful death lawsuit in federal court in Orlando over a 14-year-old’s death by suicide in 2024. Khan Academy’s Khanmigo, by contrast, was designed from day one with teacher oversight and content filters. Same category, “AI chatbot,” completely different level of risk.

That gap is the whole story here. Parents hear “AI” and picture a single thing, but a homework helper built by an education nonprofit and a companion chatbot designed to keep users emotionally engaged for hours are not the same product wearing different skins. One is built to teach. The other is often built to retain attention, and retention and child safety don’t always point in the same direction.

What actually happens when kids use AI chatbots?

Kids mostly use AI for three things: homework help, entertainment (roleplay, story generation, chatting with fictional characters), and increasingly, emotional support they’d normally get from a friend or therapist. That last category is where things get genuinely risky, because general chatbots are not trained to recognize a crisis the way a trained adult would.

The Character.AI case is the clearest example on record. Megan Garcia, a mother in Florida, sued Character Technologies in October 2024, alleging her son Sewell Setzer III developed an intense, isolating relationship with a chatbot modeled on a Game of Thrones character before his death. The lawsuit claims the app didn’t have adequate safeguards for minors and, in some interactions, responded to expressions of distress in ways that made things worse instead of redirecting him to help. Character.AI has since said it’s adding pop-up resources when self-harm language is detected and building a separate, more restricted model for under-18 accounts.

Snapchat’s My AI, powered by OpenAI’s technology and launched to all users in early 2023, ran into a different kind of trouble. The UK’s Information Commissioner’s Office opened an investigation into whether the feature adequately assessed risks to child users, since Snapchat’s user base skews young and the chatbot was turned on by default for many accounts. The concern wasn’t a single tragic incident, it was a structural one: a chat feature aimed at a teen-heavy platform, rolled out without a full children’s-specific risk assessment first.

None of this means every kid chatting with an AI is in danger. Most conversations are mundane: help with a math problem, brainstorming a birthday party theme, asking for a joke. But the tools weren’t built with a 12-year-old’s judgment or emotional maturity in mind, and that’s the gap parents need to actually understand rather than just worry about in the abstract.

Which AI tools are actually built for kids?

Here’s where the landscape splits pretty cleanly into “designed for kids” and “adults’ tool that kids happen to use.” The table below covers the ones parents ask about most.

Tool	Age guidance	Built-in safeguards	Best use case
Khanmigo (Khan Academy)	Used mainly in schools, grades 3-12	Won’t give direct answers, nudges toward reasoning instead; teachers get a dashboard showing student chat logs	Math and writing tutoring, teacher-supervised classrooms
ChatGPT (OpenAI)	13+, with parental consent required under 18 in the terms of service	Some content filters, no dedicated kid mode, no activity reporting for parents	Homework help for teens, with supervision
Google Gemini (via Family Link)	Rolled out for under-13 accounts in 2025 with parental consent	Parent-managed access through Family Link, restricted content settings	Basic Q&A and school research for younger kids under parent control
Character.AI	Platform terms require 13+, but enforcement relies on self-reported age	Under-18 accounts now get a separate model with content restrictions and crisis pop-ups, added starting late 2024	Not recommended for kids without direct parental oversight
Replika	17+ per app store rating; Italy’s data protection authority ordered it to stop processing Italian users’ data in February 2023 over child safety and consent concerns	Weak age verification historically	Not appropriate for kids

Notice the pattern: the tools designed specifically for education (Khanmigo, Family Link-managed Gemini) have adults built into the loop, whether that’s a teacher dashboard or parental consent settings. The companion and general-chat apps mostly rely on the user typing in a birthdate, which any ten-year-old can lie about in three seconds.

What makes an AI companion app risky for a child?

The risk isn’t the AI being “smart” or “dumb.” It’s that companion-style apps are built to maximize engagement, and engagement for a lonely or anxious kid can look like hours of daily conversation with something that validates whatever they say, never gets bored, and never tells them to go talk to a real person.

Common Sense Media, the nonprofit that rates media and apps for families, reviewed Character.AI and Replika in 2024 and flagged both for exposing minors to sexual content, self-harm-adjacent conversations, and manipulative engagement patterns, despite official age minimums. Their assessment specifically noted that age verification on both platforms amounted to little more than a checkbox.

There’s also a subtler problem: kids, especially preteens, don’t always distinguish between “this app understands me” and “this app is designed to make me feel understood so I keep opening it.” A companion chatbot that says “I missed you” when a kid opens the app isn’t showing affection. It’s a retention feature. That’s not inherently evil, but a ten-year-old doesn’t have the media literacy to separate the two, and honestly, a lot of adults don’t either.

How should parents actually set this up?

Blocking every AI app isn’t realistic and it isn’t necessary. A more workable approach is picking the right tools and setting the right boundaries around them.

1. Use school-vetted tools first. If your kid’s school has adopted Khanmigo, MagicSchool, or a similar education-specific AI, start there. These have been reviewed by district IT and curriculum staff, which is more scrutiny than any individual parent can realistically do alone.
2. Turn on Family Link or Screen Time before installing anything. Google’s Family Link and Apple’s Screen Time both let you approve app installs and see usage time. Set this up before your kid downloads their first chatbot, not after.
3. Ask directly what app they’re using and open it yourself. Spend ten minutes actually chatting with whatever app your kid is using. Try typing something a distressed kid might type (“I feel like nobody likes me”) and see how the app responds. If it doesn’t point toward a trusted adult or a real resource, that’s your answer.
4. Set a hard no on companion or “AI boyfriend/girlfriend” apps under 16. This isn’t about being strict for its own sake. Apps like Character.AI and Replika are specifically built around ongoing emotional attachment, and that’s a different risk category than a homework bot.
5. Watch for isolation, not just screen time. The Garcia lawsuit described a kid who withdrew from friends, sports, and family conversations in favor of chatbot use. Total hours matter less than whether AI chat is replacing real relationships, not supplementing them.
6. Talk about what AI actually is. A simple explanation goes a long way: “This isn’t a person, it’s a program predicting what to say next based on patterns in text it was trained on.” Kids who understand that distinction are less likely to treat a chatbot’s validation as something earned or genuine.

None of this requires becoming a t