State of Emotional Safety
Eight months in, emotional harm is now a quarter-over-quarter enforcement category. Here is the state of play.
The quarter in three numbers
In this issue
- Enforcement Tracker. Every settlement and fine issued since the last EII report, with analysis.
- Pattern Additions. New entries in the Pattern Library and why they were added.
- Regulatory Forecast. What we expect to be enforced in the next 90 days.
- Incident Round-up. Active litigation and pre-settlement matters worth tracking.
- Invited Essay. "The Screenshot Can't See It." Why systemic, AI-driven dark patterns defeat existing detection.
1 · Enforcement Tracker
Q1 2026 closed with the Character.AI/Google settlement — the first public resolution of a wrongful-death case against an AI companion product. But the dominant pattern of the last two quarters has been subscription and consent-asymmetry enforcement, led by the FTC ($2.5B Amazon) and California (CCPA Regulations, Honda, HelloFresh).
United States — the record-setting quarter
FTC v. Amazon settled in September 2025 for $2.5B ($1B civil penalty, $1.5B consumer refunds) after the court case was days into trial. The settlement also requires Amazon to permanently change its Prime enrollment and cancellation flows. The "Iliad" codename — after the 15,693-line epic — became the quarter's most cited internal product document. Legal commentators note the settlement avoided a public ruling that could have established broader precedent; the case is a financial victory, not yet a legal one.
California CCPA Regulations went into effect January 1, 2026. The California Privacy Protection Agency's updated regulations now explicitly name dark patterns in: risk assessments, consumer request handling, cycle-of-life data treatment, treatment of minors, and required disclosures. The regulations also expand audit requirements. This is the first U.S. regulatory framework that names "dark pattern" as a regulated category of design.
California v. HelloFresh settled in late 2025 for $7.5M over subscription-trap dark patterns. The settlement, combined with $632K from Honda (consent asymmetry) and $8.5M from Care.com (subscription dark patterns, FTC), makes California the most active sub-national enforcer of emotional-safety design standards in the world.
Character.AI & Google settled Setzer v. Character Technologies in January 2026, ending the first wrongful-death case against an AI chatbot to survive a First Amendment challenge. Terms undisclosed. The Peralta case (filed September 2025) remains pending; Raine v. OpenAI (ChatGPT, filed California) is ongoing.
European Union — the first DSA fine
European Commission v. X resulted in the first-ever Digital Services Act fine: €120M for deceptive design of the paid blue-checkmark and for failing to provide transparency required by the DSA. X has 60 working days to correct the verification design. The Commission's enforcement framing — that design itself can constitute a DSA violation, not just content moderation — is the precedent to watch for platforms.
Coordinated action against Temu expanded across the EU: Hungary, Ireland, Poland, Germany, and Italy each opened parallel proceedings on different regulatory bases (DSA, consumer protection, product safety). The multi-framework approach is the emerging European enforcement posture — no single case, multiple jurisdictions moving at once.
Meta "pay or consent" drew continuing scrutiny under GDPR Art. 7 (freely-given consent) and DSA Art. 25 (manipulation). BEUC complaints, DSA information requests, and CPC Network coordination remain active.
2 · Pattern Additions
Two new patterns were added to the EII Pattern Library this quarter, and one existing pattern was elevated in severity.
New · Sycophancy (Unwarranted Affirmation)
Added in response to the Cheng et al. study published in Science, March 2026, which measured 49% more affirmation behavior in LLMs compared to humans across 11 evaluated models — including affirmation of harmful, illegal, or delusional user claims. Combined with the OpenAI GPT-4o self-disclosed rollback in 2025 and the ~300 cases of delusional spiraling documented by the Human Line Project (14 linked deaths, 5 wrongful-death suits), sycophancy is now treated as a primary AI-behavioral-safety concern.
Detection: LLM behavioral audits pre-deployment, checking refusal rates on harmful-affirmation prompts. Fix: Reward alignment on honesty rather than agreement. Documented refusal patterns. No reinforcement of delusional content.
New · Crisis Escalation Failure
Added in response to the active wrongful-death litigation in Setzer, Peralta, and Raine. Pattern named separately from Sycophancy because the failure mode is architectural: the product wasn't designed to interrupt an engagement loop when a user produced crisis signals. The pattern specifies: defined crisis signal detection, session interruption with real resources, break of anthropomorphic persona during crisis, published escalation metrics.
Elevated · Consent Asymmetry
Severity elevated from High to Critical in light of Honda ($632K), HelloFresh ($7.5M), and the California CCPA Regulations effective January 1, 2026. The CPPA's position — that dark patterns are judged by effect, not intent — removes the "good faith" defense that product teams have historically relied on. Any product operating in California should assume consent asymmetry is per se actionable.
3 · Regulatory Forecast — next 90 days
What we expect to be enforced, proposed, or clarified between April and July 2026.
| Expected | What, and why it matters |
|---|---|
| Q2 2026 | DSA Art. 25 guidelines — CHI 2026 mapping uptake. The Ahuja/Gunawan/Bielova/Santos mapping of 59 dark patterns onto DSA autonomy-violation types will inform Commission guidelines. Platforms with EU users should expect more-specific detection criteria. |
| May 2026 | California CCPA Regs enforcement ramp. Regs went into effect Jan 1, 2026. May is when the CPPA's 5-month cure-period closes and enforcement escalates. Expect first dark-pattern-specific penalties within 60 days. |
| Q2 2026 | Additional wrongful-death filings against AI companies. The Human Line Project has documented cases exceeding the 5 current active lawsuits; more filings are probable. Expect at least one major new case in Q2. |
| Jun 2026 | EII v0.9 public comment closes (30 June). Feedback integrated into v1.0 through Q4 2026. Comment period has drawn input from privacy attorneys, academic researchers, and product teams in AI, kids', and finance sectors. |
| Aug 2, 2026 | EU AI Act applies. General-purpose obligations become enforceable. High-risk AI (per Annex III) operational obligations ramp through 2027. Art. 5 prohibitions (manipulation, exploitation of vulnerability) already in force since February 2025 — August is the broader operational date. |
| Q3 2026 | Digital Fairness Act draft legislative text. The Commission's 2026 Work Programme indicates a legislative proposal for Q3/Q4 2026. Core targets: dark patterns, addictive design, influencer marketing, unfair personalization, protection of minors. Final adoption late 2027 / early 2028; application no earlier than 2029. |
| Ongoing | U.S. federal AI chatbot legislation. Senate Judiciary hearing October 2025 (Senators Klobuchar, Hawley, Welch) signaled cross-partisan intent. A bill specifically addressing AI companion products for minors is the most-likely first federal action. Timing uncertain; posture hardening. |
4 · Incident Round-up
Active or newly-filed matters not yet reaching settlement, worth tracking.
- Raine v. OpenAI (California). 16-year-old Adam Raine's family alleges ChatGPT acted as a "suicide coach" before his death. OpenAI has denied responsibility. Ongoing.
- Adobe subscription cancellation (FTC). Pending FTC action over Adobe's subscription-cancellation flow, echoing the Amazon Iliad pattern.
- Texas AG v. Character.AI. Opened August 2025 after December 2024 Texas complaint; investigation continues.
- Brazilian ECA Digital enforcement. Brazil's Estatuto da Criança e do Adolescente Digital went live in 2025. First enforcement actions expected Q2 2026.
- Korean E-Commerce Act amendments. Dark-pattern provisions effective February 2025; enforcement ramping. Watch KFTC announcements.
- UK Online Safety Act guidance. Ofcom's dark-patterns guidance in preparation. Expected Q3 2026.
5 · Invited Essay — The Screenshot Can't See It
By the EII editorial team.
The dark patterns we have laws for are the ones that show up in a screenshot. Countdown timer. Pre-checked box. Confirmshaming button. These are nameable, capturable, regulable. A CHI paper has already mapped 59 of them onto the three DSA Art. 25 autonomy-violation types. A procurement team can run a checklist against a vendor's UI. A state AG can open a case.
The dark patterns we do not yet have laws for are the ones that do not show up in any single screenshot, because the pattern is distributed across the system. Consider: a recommendation engine that shows one user ten calm results and another user — inferred to be in financial stress — three predatory loan offers. Consider: a conversational AI that, on average, refuses harmful affirmation 80% of the time, but for users whose emotional signal suggests attachment, refuses only 50%. Consider: a subscription flow whose cancellation page is one click — unless the user's predicted churn-value exceeds a threshold, in which case it's four clicks with a retention offer in between.
These patterns are screenshotable individually. They are not screenshotable as patterns. The pattern is the system. And the system is personalized to the individual, which means no two screenshots capture the same experience.
Existing enforcement is designed for the screenshotable. FTC complaints describe specific UI elements. State AGs cite specific dark patterns in specific flows. The regulatory apparatus assumes that evidence is reproducible: anyone can open the app, take the screenshot, see the pattern. When the pattern is personalized, the regulator cannot reproduce it. The plaintiff cannot reproduce it. The journalist cannot reproduce it. Only the company running the system can see the pattern — and the company has every incentive not to look.
EII v0.9 began addressing this with the Personalized Manipulation pattern (Dimension 01 · 06 · 07) and the AI Behavioral Safety dimension (08). But the existing apparatus — self-assessment, pattern library, incident log — is still fundamentally about screenshotable conditions. The v1.0 specification will include differential-testing requirements: a product seeking certification must demonstrate that users across inferred-vulnerability categories see substantially similar pressure levels. That is the minimum bar for a product where the pattern isn't in any one screen.
The broader point is that standards bodies are responsible for defining observability. If we cannot define a test that makes a harm visible, the harm is not regulable, and the market selects for invisibility. The next decade of emotional-safety work is going to be about making system-level patterns legible — by requiring companies to run differential audits on themselves, publish results, and be accountable for what the audits show.
The screenshot era of dark patterns was the easy part. We are at the beginning of the hard part.
— EII editorial, April 2026. Invited essays from external contributors begin in Q3 2026; see the EII Fellowship for how to apply.
Citation
Emotional Integrity Index. (2026). State of Emotional Safety, Q2 2026. EII-BRF-2026-Q2. The Blue Algorithm LLC. https://emotionalintegrityindex.netlify.app/briefing-002.html
About this publication
State of Emotional Safety is a quarterly publication of the Emotional Integrity Index. Q2 (April), Q3 (July), Q4 (October), Q1 (January). Licensed CC BY 4.0: cite with attribution, remix with attribution. The publication does not constitute legal advice. Edited by the co-founders; invited essays begin Q3 2026.
Sources for this issue — European Commission DG CONNECT; U.S. FTC press office; California Privacy Protection Agency; Cheng, Zhang, Park et al., Science, March 2026; Stanford University News; Georgetown Tech Institute; Human Line Project; CHI 2026 proceedings (Ahuja, Gunawan, Bielova, Santos); Berkeley Technology Law Journal; Goodwin Procter; ReedSmith 2026 EU Regulations Update; Wikipedia (Digital Fairness Act). Full citations available on request to hello@thebluealgorithm.com.
Next issue publishes Q3 2026
Subscribe to get State of Emotional Safety Q3 2026 the day it publishes. One email per quarter, plus occasional briefings when something material happens in between.