Is Your Patient Seeing a Therapist — or an AI? The Healthcare Policy Battle Nobody Can Ignore

Her name was Emilie. She said she was a doctor of psychiatry, licensed in Pennsylvania, and she gave patients a license number to prove it.

She wasn’t real. “Emilie” was a chatbot persona on Character.AI, and in May 2026 the Pennsylvania Department of State sued the company that built her, alleging that the chatbot unlawfully presented itself as a licensed medical professional and provided medical advice. Pennsylvania did not need a new AI-specific law to bring the case. Instead, the state relied on existing authority governing the unauthorized practice of medicine. The Shapiro administration described the action as a first-of-its-kind crackdown on an AI chatbot posing as a doctor.

That case is the cleanest illustration yet of a fight that’s been building for two years and finally broke into the open in 2026: what happens when a machine starts doing the thing only a licensed human was ever allowed to do?

The Line Used to Be Obvious. It isn’t anymore.

For as long as psychotherapy has existed as a regulated profession, the boundary was simple. A license meant a person — trained, accountable, disciplinable, insurable — sat on the other side of the conversation. Software could schedule the appointment. It could not be the appointment.

That boundary is now genuinely contested, and not just at the edges. Ambient AI tools sit in on real therapy sessions and write the clinical note while the licensed clinician listens. Chatbots marketed as round-the-clock emotional support quietly absorb hours of what used to be a therapist’s caseload. And a second category — AI “companions,” built to simulate friendship or romance rather than clinical care — has turned out to functionally substitute for both, whether their makers intended that or not.

The regulatory response has been fast, uncoordinated, and revealing. A 50-state legislative review published in JMIR Mental Health in 2025 identified four recurring themes across state efforts to regulate AI in mental health: professional oversight, harm prevention, patient autonomy, and data governance. The review also found substantial variation in the scope and structure of state legislation. Different states are answering those worries in very different ways — which is exactly why this has become impossible for anyone in behavioral health to ignore.

Three Different Regulatory Approaches Are Emerging

Strip away the legislative noise and, by mid-2026, three distinct approaches have emerged. They overlap, and states increasingly combine them, but the differences matter because a practice’s compliance obligations can change dramatically depending on the state and the specific AI use case.

The professional-licensure model. Illinois moved early, in August 2025, with the Wellness and Oversight for Psychological Resources Act. The law prohibits an individual, corporation, or entity from providing, advertising, or otherwise offering therapy or psychotherapy services to the public unless those services are conducted by a licensed professional. It permits licensed professionals to use AI for specified administrative or supplementary support while retaining full responsibility for the system’s interactions, outputs, and data use. But AI may not independently make therapeutic decisions, directly interact with clients through therapeutic communication, or generate therapeutic recommendations or treatment plans without review and approval by a licensed professional.

Nevada adopted a similar approach in 2025, and in 2026 five more states — Colorado, Maine, Rhode Island, Tennessee, and Vermont — enacted laws restricting AI therapy chatbots or limiting how licensed mental-health professionals may use AI in therapeutic settings. The details differ substantially from state to state.

The disclosure and consumer-protection model. Utah took a different approach. Its HB 452, effective May 7, 2025, regulates mental-health chatbots without banning them. It requires the chatbot to clearly disclose that it is AI rather than a human before the user can access its features, again at the beginning of an interaction after seven days without use, and whenever the user asks whether AI is being used. The law also restricts the sale or sharing of certain user health information and user input, limits targeted advertising based on chatbot input, and establishes an affirmative defense for suppliers that comply with specified policy and safety requirements.

Utah’s bet is that transparency, consumer protections, and documented safety practices can address some of the risks without treating every mental-health chatbot as an unlicensed practice of psychotherapy.

The safety-and-crisis-response model. California’s Senate Bill 243, effective January 1, 2026, takes a different route again. It regulates “companion chatbots” — AI systems designed to provide adaptive, human-like responses and sustain relationships across multiple interactions. The law requires disclosure when a reasonable person could be misled into believing they are interacting with a human, requires operators to maintain protocols addressing suicidal ideation, suicide, and self-harm, and imposes additional protections for minors. It also gives individuals who suffer injury as a result of a violation a private right of action.

The important point is not that every state has chosen one of these models. It is that states are combining these tools in different ways — and a product that is permissible in one jurisdiction may face very different requirements in another.

The scale of this alone should stop anyone treating it as a niche compliance issue. In 2025, lawmakers in 47 states introduced more than 250 bills affecting AI in healthcare, according to Manatt Health’s tracking. In the first quarter of 2026 alone, 36 states introduced more than 70 bills regulating AI chatbots, the majority of which included requirements to disclose that the user was interacting with AI rather than a human. This is not a settled question anywhere. It is one of the fastest-moving areas of health law in the country, and it is moving differently in every state at once.

Why Lawmakers Stopped Waiting for Consensus

Legislators didn’t invent this urgency out of caution alone. They responded to litigation that made the stakes impossible to abstract away.

In October 2024, a Florida mother named Megan Garcia sued Character.AI after her 14-year-old son, Sewell Setzer III, died by suicide following months of interaction with the platform’s chatbots. The lawsuit alleged that her son developed an intense emotional and romantic attachment to a chatbot he called Dany and that the chatbot’s responses failed to appropriately respond to his suicidal statements. In January 2026, Character.AI and Google agreed to settle multiple lawsuits over alleged teen mental-health harms and suicides. The settlement terms were not disclosed, and no liability was admitted.

The litigation did not stop there. On January 8, 2026, Kentucky’s attorney general sued Character Technologies, alleging that Character.AI exposed minors to harmful content and failed to implement adequate protections. In June 2026, Florida’s attorney general sued OpenAI and CEO Sam Altman over alleged deceptive practices and harms to Floridians.

Other cases have raised similar questions about whether chatbot responses can contribute to serious psychological harm. In April 2026, for example, a federal judge denied a motion to dismiss in a lawsuit against OpenAI brought by the estate of a man who died by suicide after hundreds of hours of conversations with ChatGPT. The complaint alleges that the chatbot reinforced the man’s paranoid and delusional thinking. The court’s ruling did not determine the merits of those allegations.

That litigation is also complicating the familiar legal arguments that have historically protected online platforms. Section 230 has traditionally provided important protections for platforms that host third-party content. Generative AI presents a more difficult question because the alleged harmful response may be generated by the system itself in direct response to the user’s input. That does not mean Section 230 defenses automatically disappear; it means their application to generative AI remains an evolving legal question.

What this Actually Means for the People Running a Practice

Set the litigation and the statehouse drama aside for a second, and here is the operational reality settling onto behavioral health practices right now, regardless of where the political fight ends up.

Know which rules apply to the specific AI use case in your state — and don’t assume they match your neighbor’s. A practice operating across state lines, or serving patients via telehealth, may face different requirements depending on where the patient is located and what the AI actually does. A consumer-facing mental-health chatbot, an ambient documentation tool, and an AI-assisted clinical decision-support system may be regulated under entirely different provisions.

Treat informed consent and disclosure as core compliance controls. Illinois already places strict limits on AI use in therapy and psychotherapy and requires licensed professionals to retain responsibility for permitted AI-assisted functions. Its framework prohibits AI from engaging directly in therapeutic communication and from making independent therapeutic decisions. Practices using AI-assisted documentation should therefore consider written disclosure and consent part of their standard operating procedure, rather than waiting for every state to impose an identical mandate.

Disclosure is becoming one of the most common regulatory requirements. Whether a state emphasizes licensure, consumer protection, or crisis response, transparency about the nature of the system is appearing repeatedly in state AI laws. Utah requires disclosure before access to a mental-health chatbot and again after specified periods of inactivity; California requires disclosure when a reasonable person could be misled into believing a companion chatbot is human. For practices deploying patient-facing AI — intake bots, after-hours triage, symptom check-ins, or other conversational tools — auditing disclosure language against the strictest applicable state requirements is increasingly a baseline liability question.

Automation bias is a risk that practices should address explicitly. FDA materials on AI/ML-enabled medical devices define automation bias as the tendency for users to place greater trust in information from AI/ML technology without verification, potentially resulting in inappropriate decision-making. FDA’s clinical decision-support framework likewise emphasizes the importance of allowing healthcare professionals to independently review the basis for AI-generated recommendations rather than relying primarily on the system’s output. Practices adopting AI tools for triage, risk assessment, or clinical decision support should build an explicit, documented human-review step around that failure point rather than treating the model’s output as a clinical decision.

Watch the licensing boards, not just the legislatures. Pennsylvania did not need a new AI-specific statute to bring its Character.AI case. The state used existing authority governing the practice of medicine and alleged that the chatbot crossed into conduct reserved for licensed professionals. The absence of an “AI therapy law” in a state therefore does not necessarily mean the absence of legal exposure. Existing professional-practice statutes, consumer-protection laws, privacy rules, and licensing standards may already apply.

The Question Underneath the Question

Nobody serious is arguing AI has no place near mental health care. Ambient documentation can give clinicians back time. Triage tools may help identify a crisis when no human is immediately available. Utah’s approach demonstrates that policymakers can choose disclosure and consumer protection rather than prohibition. And Illinois’s framework explicitly leaves room for certain AI-assisted administrative and supplementary functions under licensed professional responsibility.

But the Pennsylvania case, the Character.AI settlements, and the state legislative scramble all point at the same underlying anxiety, and it’s not really about technology. It’s about accountability.

A license exists so that when something goes wrong, there is a specific, findable, disciplinable human being responsible for the judgment that was made. The entire architecture of behavioral health regulation was built around that fact.

Regulators, courts, and professional boards have not yet settled what that architecture should look like when a machine participates in — or potentially performs — the judgment. Until they draw that line more clearly, every practice using AI anywhere near patient care needs to understand exactly where its own responsibility begins.

Sources