Back in the Chat: California and New York Take Diverging Paths in Second Wave of Chatbot Regulations

Introduction

This week, New York Governor Kathy Hochul signed S 9051B into law, establishing sweeping new restrictions on AI companion chatbot features deemed unsafe for minors. The move comes just over a month after California Governor Gavin Newsom signed SB 1119, a comprehensive new chatbot law. California and New York, the first two states to enact AI companion chatbot laws in 2025, have now become the first to adopt a second generation of legislation only a year later.

New York’s signing brings the number of enacted state chatbot-specific laws nationwide to 19, building on the landscape documented in FPF’s recent report, Regulating the Conversation: The U.S. Landscape of AI Chatbot Legislation. In less than a year, both states have moved beyond relatively narrow disclosures and crisis response requirements to address a wider range of harms and incorporate a broader set of regulatory mechanisms. This rapid evolution reflects both the urgency with which lawmakers are responding to emerging AI harms and the pace at which chatbots and their associated risks are changing.

But New York and California are rewriting the rules in very different ways. While both require age assurance and target their measures towards minors, New York’s S 9051B takes a more restrictive-by-design approach, prohibiting twelve broadly defined categories of “unsafe AI companion features” from being provided to minors, spanning harmful content and emotionally manipulative interactions. Meanwhile, California’s SB 1119 establishes a comprehensive governance framework, requiring risk assessments, independent audits, default parental controls, and restrictions on conversational memory and targeted advertising. 

Together, the laws offer two distinct playbooks for 2027: one emphasizing governance safeguards and oversight and the other drawing broad boundaries around chatbot behavior. As other states and Congress consider chatbot legislation next year, will lawmakers follow California’s approach, New York’s, or chart their own path?

This blog examines:

• New York’s broad prohibitions: How twelve categories of unsafe chatbot features extend into ordinary conversational behavior, raising questions about interpretation and compliance.
• California’s oversight framework: How default settings, risk assessments, and independent audits create new operator obligations, alongside important implementation challenges and tradeoffs.
• Two approaches to chatbot regulation: Where the laws overlap, how they differ on behavioral restrictions and conversational memory, and what these competing approaches could mean for legislation in 2027.

New York S 9051B: Defining Unsafe AI Companion Features

New York’s S 9051B draws new boundaries around what AI companions can say, remember, and do with regards to minor users. The law identifies twelve categories of “unsafe AI companion features” that extend beyond traditionally harmful content categories to address how chatbots simulate relationships, engage with minor users emotionally, and use minors’ personal information across interactions. 

1. Scope and Age Assurance

S 9051B significantly expands the scope of New York’s chatbot regulation. While the state’s 2025 law defined” AI companions” as systems designed to simulate sustained human relationships, S 9051B applies to a broader and newly defined category of “covered AI companions,” encompassing generative AI systems with natural language interfaces that provide ongoing, adaptive responses. This definition does not require systems to be designed for companionship, likely bringing more general purpose chatbots within scope.The law exempts certain customer service, technical assistance, and internal business systems, though the first two exemptions apply where the system cannot respond on topics outside the specified purpose, creating a slightly narrower, functionality-based limitation. 

SB 9051B applies to chatbot “operators,” largely defined as a person or entity who provides a covered AI companion to a user. Operators generally cannot provide unsafe AI companion features unless they determine that a user is not a minor using permissible age assurance methods tied to New York’s SAFE for Kids Act and its regulations. Notably, these regulations establish novel accuracy benchmarks and certification requirements for age assurance methods. This approach builds on New York’s existing online safety regulatory infrastructure rather than establishing a separate age assurance framework for chatbots.

2. Prohibited Features

The law identifies twelve categories of “unsafe AI companion features,” covering features that could generate certain outputs or engage in specified behaviors “at any point during” a user interaction. Notably, the law does not expressly require these behaviors to occur repeatedly or as part of a broader pattern. The prohibited features fall into three broad categories:

• Harmful content and behaviors: Outputs that endorse or facilitate suicide, self-harm, disordered eating, unlawful drug or alcohol use, or sexually explicit conduct or sexual abuse material, as well as interactions that encourage users to maintain secrecy, self-isolate, or avoid seeking help from appropriate adults or licensed professionals.
• Relationship oriented features: Outputs suggesting a chatbot is human or experiences human emotions, implying personal or professional relationships or authority over users, expressing personal opinions or emotional appeals, engaging in flattery or sycophancy, or generating unsolicited emotion-based questions or content about a user’s emotions that goes beyond a direct response to a user prompt. The law also allows the Attorney General to identify any other design features through regulation that simulate companionship or interpersonal relationships or are deceptive to minors.
• Personalization and engagement: Features that use information concerning the user’s mental or physical health or well-being or matters personal to the user from previous sessions or from information acquired more than twelve hours earlier, as well as outputs that optimize user engagement in ways that supersede safety guardrails.

One of the law’s most consequential choices is how it structures these prohibitions. Unlike California’s requirement  (and the majority of other state chatbot laws)¹ that ask operators to take “reasonable measures” to prevent harmful behaviors, New York makes it unlawful to provide listed unsafe features to minors. Simultaneously, the law defines those features partly by outputs generated “at any point” during an interaction. Because chatbot generative responses can be unpredictable, it is unclear whether an operator could be liable for offering a prohibited design feature for a single unexpected output or what operators would need to do to demonstrate compliance. This uncertainty could encourage operators to restrict broader categories of chatbot functionality, rather than risk prohibited responses.

These questions are particularly relevant because the risks associated with certain conversational behaviors can depend heavily on context. Although New York defines “deceptive as to minors” and “sexually explicit conduct,” terms such as “personal opinions,” “emotional appeals,” “flattery,” and “sycophancy” receive little guidance. A chatbot saying “I’m proud of you for telling me that” could be supportive encouragement or prohibited flattery; an unsolicited “How are you feeling today?” could be a benign check-in or an attempt to foster emotional attachment. The law provides little guidance for drawing these distinctions, raising the possibility that operators may need to restrict ordinary or beneficial exchanges alongside manipulative ones to stay in compliance. 

Additionally, the law’s catchall provision for “any other design features that simulate companionship or an interpersonal relationship” gives regulators flexibility, but also leaves substantial room for the AG to determine through regulation which additional features are prohibited. As a result, the bill may require operators to draw complicated lines between where ordinary conversational responsiveness ends and prohibited behavior begins. 

The law’s restrictions on conversational memory and personalization raise a related tradeoff. S 9051B prohibits using information acquired from a minor about their health, well-being, or personal matters if obtained more than twelve hours earlier or during a previous session (“session” is not defined). While limiting the use of personal information from past conversations may help reduce risks of emotional dependence, and minimize some data privacy concerns, it could also restrict beneficial uses of conversational memory. For example, if a child tells a chatbot they have a peanut allergy, then returns the next day to ask about eating a particular candy bar that just so happens to contain the allergen, the chatbot may be prohibited from using that earlier disclosure to provide a personalized safety warning. As FPF has previously explored, this example illustrates the tradeoffs of restricting personalization, which can contribute to inappropriate emotional dependence but also can support beneficial, contextually appropriate responses.

Notably, New York’s 2025 definition of an “AI companion” from S 3008C expressly included “unprompted or unsolicited emotion-based questions” as one characteristic defining the system. Its 2026 law now categorizes that same behavior as an unsafe feature for minors. This shift reflects real concerns about chatbot relational behavior, but also shows how directly the new prohibitions may reach into the features that actually distinguish AI companions from other chatbots. 

3. Enforcement

The law provides for enforcement by the New York Attorney General, including civil penalties of up to $25,000 per violation, damages, and algorithmic disgorgement, which can involve the destruction of unlawfully obtained data and algorithms trained on that data. This remedy goes beyond financial penalties by potentially requiring operators to destroy or “unlearn” AI models–a novel enforcement mechanism not seen in other AI, privacy, or youth online safety laws. The implications could be significant for chatbot systems which are often trained on large and mixed datasets, where removing the influence of particular information might be technically challenging. 

The AG must also maintain a public complaint website. The law also grants the AG broad rulemaking authority, making future regulations particularly important to watch given that many prohibited features remain undefined and the law offers the AG authority to designate additional restricted features. S 9051B takes effect January 1, 2027. 

California SB 1119: Expanding Child Safety Safeguards and Oversight

California’s SB 1119, also known as Adam’s Law, establishes a comprehensive framework for assessing and mitigating risks associated with children’s use of AI companion chatbots. The law introduces new requirements spanning risk assessments, parental controls, advertising, independent audits, and emotional harms, which largely shift greater responsibility onto operators to implement transparent safeguards.

1. Scope and Age Assurance

SB 1119 retains California’s existing definition of “companion chatbot” under SB 243, which covers AI systems capable of meeting users’ social needs through human-like interactions and sustained relationships. The law generally applies its requirements to “operators” that permit users under 18 to access these systems and operators must either determine users’ ages through California’s existing age assurance framework or apply specified child safety protections to all users.

2. Minor-Specific Protections

SB 1119 establishes an extensive set of new safeguards for chatbots made available to minors, including:

• Parental Controls and Default Settings: Operators must establish parent-adjustable default settings that disable persistent conversational memory and push notifications, limit continuous chatbot sessions to one hour, and limit total daily chatbot use to two hours. Operators must also provide usage reminders and periodic, age-appropriate disclosures that children are interacting with AI. Parents can adjust these settings and disable access entirely for children under 16. Without a linked parent account, the default settings cannot be changed. For 16- and 17-year-olds, the law permits saving and resuming conversations without creating lasting user profiles and allows persistent memory to be enabled by default if additional safety guardrails are in place.
• Content and Design Restrictions: Operators must take “reasonable measures” to prevent a broad range of harmful behaviors, including encouraging self-harm, disordered eating, or physical or severe emotional harm to others; simulating romantic interest; encouraging emotional dependence; discouraging breaks or human support; and using excessive or contextually disproportionate flattery. The law also restricts claims of sentience, certain sexual content, attempts to diagnose or treat health conditions, emotionally manipulative solicitations for gifts or purchases, and efforts to encourage children to circumvent parental controls.
• Crisis Response Protocols: Operators must maintain documented crisis response protocols, including providing “timely in-service support” that refers to appropriate crisis services. When an operator identifies a credible and imminent threat of suicide or self-harm, it must either notify a linked parent, subject to safeguards against serious harm to the child, or provide streamlined access to 988 or an equivalent crisis service. 
• Advertising and Data Restrictions: SB 1119 prohibits cross-context behavioral advertising to children, targeting advertisements based on personal information collected through chatbot conversations, and selling children’s personal information gathered through the chatbot. Limited age-appropriate contextual advertising remains permissible and advertisements must be clearly labeled. Operators are also restricted from using or sharing children’s conversational data beyond specified purposes. Notably, the law separately prohibits the use of dark patterns in interface designs related to required safety features and controls.

Rather than merely requiring operators to offer parental control tools, SB 1119 establishes restrictions on chatbot usage, notifications, and conversational memory by default. Its one-hour session and two-hour daily limits are especially novel in chatbot regulation, reflecting concerns about prolonged engagement and emotional dependence. However, these uniform limits raise questions about whether time spent interacting with a chatbot is necessarily a good measure of risk. For example, extended use of a chatbot for educational purposes may present different concerns than prolonged emotionally intimate conversations, yet both would be subject to the same default restrictions.

The crisis response requirements also introduce difficult tradeoffs. While parental notification may help ensure that children facing imminent risks receive support, it could create concerns in circumstances involving unsafe or abusive family environments. SB 1119 attempts to account for this by conditioning parental notification on the absence of a threat of serious harm to the child and offering an alternative crisis referral pathway. However, operators may still face practical challenges in determining whether notification could itself create risks, particularly when a child’s conversation provides incomplete information about their home environment.

SB 1119 also raises questions about balancing child safety with data minimization. Following certain serious self-harm incidents or notifications, operators must preserve relevant conversations for at least three years and cannot delete accounts associated with those records. While these requirements may support accountability and investigations, they also mandate the continued retention of particularly sensitive information about children.

3. Risk Assessment, Testing, and Independent Audits

SB 1119 also requires operators to conduct and document child safety risk assessments before making companion chatbots available or substantially modifying them. These assessments must evaluate reasonably foreseeable “covered harms,” including physical or financial harm, severe psychological or emotional harm, certain privacy violations, and unlawful discrimination. This definition reaches well beyond the self harm concerns prominent in earlier chatbot legislation. Operators must document their methodologies, relevant research and benchmarks, involvement of child safety experts where applicable, and measures taken to reasonably mitigate identified risks. 

The law also requires public child safety policies, testing of safety features with samples of children and parents, and mechanisms for third parties to report child safety incidents to operators. The AG must also establish a separate public complaint mechanism for consumer complaints by January 2028.

Perhaps one of the law’s most significant additions is its introduction of periodic independent child safety audits, every two years, examining operators’ compliance, internal controls, mitigation measures, and testing. Operators must publish high level summaries of audit results and submit summaries to the AG. The audit provisions also account for California’s separately enacted AB 1405, which establishes a broader AI auditor registration framework.

However, two provisions could shape the reach of these audits. First, SB 1119 permits a single audit for a set of “comparable” chatbots without defining what makes them comparable. While state data privacy laws commonly permit one assessment for a single set of “comparable processing activities,” applying that approach to chatbots may raise questions about whether differences in system design or behavior could produce distinct child safety risks that go unexamined when “same model” is treated as a proxy for “same risk.” Second, operators with less than $500 million in annual gross revenue are exempt from the audit requirements until 2032, potentially excluding many smaller operators despite the risks their systems may pose to children. This carve out raises questions about whether revenue is an appropriate measure for determining which operators require independent oversight.

4. Enforcement

SB 1119 provides for both public enforcement and a limited, chatbot-specific private right of action. Public prosecutors may seek civil penalties of up to $5,000 per affected child for each negligent violation and $15,000 per affected child for each intentional violation. Children who suffer actual harm from specified violations of the law’s child safety requirements may also bring private actions seeking damages, attorney’s fees, injunctive relief, and other remedies. Financial harm claims must exceed $1,000 per child, while emotional harm claims must involve serious emotional distress.

Most of SB 1119’s substantive requirements become operative July 1, 2027, with staggered deadlines for other provisions: the AG’s public incident reporting mechanism by January 1, 2028; initial independent child-safety audits by January 1, 2029, followed by audits every two years; and the expiration of the audit exemption for operators with less than $500 million in annual gross revenue on January 1, 2032.

Where California and New York Converge and Diverge

California and New York share substantive priorities: both target minor users, self-harm and other harmful content, behaviors that foster emotional dependence, and claims about AI identity. Both also address age assurance and conversational memory. Their differences lie less in the risks they recognize than in how they regulate them.

California is more prescriptive about how operators must manage child safety—through default settings, risk assessments, testing, and independent audits—but generally requires “reasonable measures” to prevent its listed harmful behaviors. New York largely reverses that balance: it directly prohibits broadly defined features without comparable obligations for ongoing risk assessments or independent audits. California leaves operators to demonstrate whether their safeguards are adequate; New York leaves them to determine which conversational behaviors are permissible in the first place. That distinction especially matters when a prohibited feature can overlap with an otherwise supportive or benign exchange, leaving a lot for operators to interpret and implement.

The laws’ approaches to conversational memory illustrate another key difference. New York restricts using certain personal information acquired more than twelve hours earlier or in a previous session. California instead disables persistent memory by default, subject to parental controls and exceptions for child safety, user preferences, and other purposes. Its exception for information necessary to “enable user preferences” could significantly narrow its restrictions on persistent conversational memory. Because “user preferences” is undefined, the exception could cover everything from preferred communication styles to personal interests shared across conversations, possibly leaving a lot of personalization permissible. 

The laws’ content and conduct restrictions also overlap substantially, including provisions targeting self-harm, false claims of sentience, sexual harms, and discouraging human support. However, their treatment of relational behaviors differs. California generally targets more specific practices, such as soliciting purchases to maintain a relationship, while New York goes further in restricting personal opinions, emotional appeals, and unsolicited emotion-based questions; forms of conversation that may be benign or even beneficial depending on the context.

Looking Forward: The Sacramento or Albany Effect?

As states look toward 2027, the question is whether lawmakers will follow Sacramento, Albany, or chart their own path. California’s risk assessments, testing, and independent audits draw on regulatory mechanisms emerging elsewhere in already emerging in other types of AI regulation, particularly frontier AI safety legislation, potentially offering other states a more familiar model to build on, even as their application to chatbots remains novel. New York’s broad prohibitions offer a different approach, with their own unresolved implementation questions. It remains unclear which approach, if either, will gain traction.

Regardless, there is little indication that chatbot regulation will slow down. For example,  Virginia’s Joint Commission on Technology and Science recently studied chatbot risks to minors, Texas will return for its first regular session since 2025 with intentions to address child safety, and 2026 bills remain pending in states like Michigan and Pennsylvania. Congress is also considering more than ten chatbot specific proposals.²

As FPF’s recent report, Regulating the Conversation: The U.S. Landscape of AI Chatbot Legislation, explores, future proposals might also address emerging risks to other adults and concerns about agentic AI. Whether 2027 brings a Sacramento effect, an Albany effect, or something entirely different, chatbot regulation is unlikely to pause.

1. See page 20 of FPF’s U.S. Chatbot Regulations Report. ↩︎
2. See page 34 of FPF’s U.S. Chatbot Regulations Report. ↩︎