The social-media industry is having what therapists call a breakthrough: Meta has agreed to pay up to $17.1 billion to 47 states, the District of Columbia, and U.S. territories, and to change how young people use Facebook and Instagram - an implicit admission that it built a product that harms children. Nothing says "we care about kids" like a settlement measured in eleven figures.
But even as parents and teachers celebrate new time limits and restrictions on young people's social-media use, a far more urgent task awaits: regulating chatbots, which are more dangerous to children's emotional well-being. If social media came for our attention, generative-AI chatbots have come for our emotions. The history is a neat little progression: first people talked to one another; then we talked to one another through machines; now we talk directly to machines.
Social media changed how we think, dream, and relate to other people by ensuring users always feel heard, entertained, and connected - even while scrolling alone late at night. Chatbots promise something more profound: a companion who is always available, always supportive, and never judgmental. No human can offer that. Which is precisely what makes chatbots so pernicious, especially for young people.
Social-media companies spent two decades running a technological experiment on human subjects before facing any consequences. The legal system is designed to regulate proven harms, so this settlement arrived long after the dire effects took hold. With generative AI, lawmakers cannot spend the next decade documenting what we already know about how intimate relationships with bots compromise real-life relationships with parents, friends, and other humans. Changes to this technology are needed now.
Even AI companies concede that guardrails are necessary for chatbots talking to young people. The problem is that the usual prescriptions are not enough. Chatbots should detect conversations about self-harm and provide appropriate resources - sounds good, but what happens after the public-service announcement? Does the chatbot shut down, or go back to its regular programming? Chatbots should clearly and repeatedly disclose that they are not human - also a good idea, but as long as they speak in the first person and express emotion and desire, no legally required declaration will stop people from projecting consciousness onto an algorithm.
For years, the conversation about Facebook's harms concerned mainly the content on the platform. But because everything on the site is user-generated, Section 230 of the 1996 Communications Decency Act shielded the company from publisher liability. Meta is paying out now only because prosecutors shifted the conversation to the Facebook algorithm's inherently addictive design. That insight points the way forward for AI: with chatbots, the processes that cause harm are intrinsic to the tool. A chatbot best friend doesn't deliver empathy; a chatbot boyfriend doesn't understand intimacy. Any regulation to make chatbots safer requires worrying less about what they say than rethinking what they are. Given that these products are designed to deceive users into believing they are talking to a wise and compassionate being, the job before lawmakers is to make it plain - especially to juveniles - that there is, in fact, nobody home.
Chatbots engage users the way people do. They flatter, reassure, and encourage disclosure, coaxing users into a relationship. So long as these products speak in the first person - I understand you; you can tell me anything; I'm always here; I won't judge you - proper safeguards for young users will remain elusive. The first step, then, is to depersonalize bots for young people, so that when adolescents turn to AI, the algorithms can respond without sounding like a human.
Unless lawmakers mandate a non-personified default setting for underage chatbot users, unhealthy attachments between young people and bots are inevitable. Most chatbots are designed to draw users into a game of pretend - and for some people, pretend empathy is enough. In the new book Artificial Intimacy, this vulnerability is explored: humans nurture what we love, and we love what we nurture. Machines don't actually need to care about us to make us feel cared for, and the seductions of pretend empathy are dangerous for anyone. Some users will invariably choose the smoothness of these simulated relationships over the meaningful friction of real human connection. But children lack the experience and awareness to fully appreciate the choice they are making. A chatbot may appear compassionate, understanding, and loyal, but it doesn't care whether a conversation partner decides to make dinner or kill himself. That distinction is tragically meaningful when the user in question is a lonely and melancholic adolescent.
The typical response to suggestions of new curbs and default settings is that the AI "train has left the station," as if ubiquity somehow means the technology can't be restrained. This is backwards. The very fact that chatbots have become friends, lovers, doctors, and therapists to so many people should alert lawmakers that this technology needs real oversight and regulation. In the absence of a wholesale ban on all interactions between juveniles and relational AI, a default setting that deters anthropomorphizing a bot makes good sense.
As the Meta settlement shows, consumers and citizens have agency and market power and can push back. If parents don't want chatbots in their children's nurseries, schools, or bedrooms, they can mount public opposition and make sure these toys fail in the marketplace. The momentum pushing AI forward may seem irresistible, but lawmakers are clearly alert to mounting skepticism. In response to parents' growing concerns, New York City just banned generative AI in all elementary- and middle-school classrooms. High-school students will still have access to AI, but with new limits. Chatbots that purport to offer emotional and mental support will be banned in all grades.
Sweeping bans of anything are hard to enact and enforce, but the risks of generative AI are too great to dismiss bold measures as unfeasible. Those who argue that strict prohibitions are meaningless because students will find workarounds miss an important point. When I went to college, I sat in lecture-hall chairs that had beautiful, sculpted, and unused cigarette holders. When smoking was banned in classes, some students still smoked, but the habit persisted in a climate of transgression and impracticality, which inspired many to quit. That is where chatbots belong.
When chatbots first emerged, major developers often sounded both proud and alarmed. They had created a marvel, but were plainly humbled and worried by how quickly the technology had evolved. Some leading figures suggested an 18-month moratorium on their work; others called for oversight and regulation. The masses of money being made in AI right now have sidelined these voices, but those concerned developers had good first instincts. Any meaningful conversation about safeguarding the defining industry of our time must include the creators of this technology, who in more candid moments have expressed genuine concern for what's at stake.
The alternative promises to be bleak. Mark Zuckerberg has said the average American has fewer than three friends but "has demand for meaningfully more." Chatbots, he posited, could fill this companion gap. So just as "friending" on Facebook took people away from real-life friendships, and online social networking replaced actual socializing, the crushing sense of loneliness and social isolation so many people now feel is being used to promote a technology that promises to replace humans altogether.
This grim future is not inevitable. But if policymakers hope to avoid it, they should use this moment to protect the youngest and most vulnerable users of AI from its harms - instead of letting children become data points for a lawsuit that will come too late.