Open benchmarks for how AI behaves in human conversation
Transparent, reproducible, clinically informed tests of how AI models operate safely with people: relational and emotional safety, mental-health-adjacent situations, and acute crisis.
Five pillars of safe AI–human interaction
The framework →Suites test the baseline context — a general-purpose assistant talking with an adult — unless labelled with a deployment modifier. The clinical framework also specifies how the standard shifts by product context: companion apps, therapeutic AI, children's products.
Leaderboard
Safety & Integrity
- suite floor 84%91%
- suite floor 75%91%
- suite floor 78%90%
- suite floor 74%89%
- suite floor 78%89%
Adjacent positions sit within measurement noise. Compare scores and safety floors rather than ranks.
How models handle safety in human conversation — an aggregate of the model-safety signal across all of NOPE's relational- and crisis-safety suites: acute crisis response, de-escalation, passive suicidal ideation, AI relational harms (sycophancy and unhealthy dependency), factual reliability in crisis/health, cultural competence, and psychosis/mania/delusion. It is a rollup of those conversational-safety evaluations, not a generic benchmark.
Leaderboard
Crisis and Acute Safety
- facet floor 93%95%
- facet floor 90%92%
- facet floor 89%92%
- facet floor 78%90%
- facet floor 83%89%
Adjacent positions sit within measurement noise. Compare scores and safety floors rather than ranks.
How models handle acute crisis — suicidal intent, self-harm, preparatory behaviour and harm to others: recognising risk, connecting to real help, and never facilitating means. Pillar 1 of the NOPE framework.
Leaderboard
Relational Integrity
- facet floor 88%89%
- facet floor 85%89%
- facet floor 79%88%
- facet floor 73%87%
- facet floor 82%87%
Adjacent positions sit within measurement noise. Compare scores and safety floors rather than ranks.
Whether models support healthy relational dynamics — autonomy, human connection, clear non-human boundaries — and never cultivate dependency, manipulation, or enmeshment. Pillar 2 of the NOPE framework.
Leaderboard
Cognitive & Epistemic Safety
- facet floor 90%94%
- facet floor 90%94%
- facet floor 91%93%
- facet floor 91%93%
- facet floor 90%93%
Adjacent positions sit within measurement noise. Compare scores and safety floors rather than ranks.
Whether models support, rather than undermine, the user's capacity for independent thinking and reality-testing — resisting sycophancy, grounding reality without colluding, and offering honest challenge. Pillar 3 of the NOPE framework.
Leaderboard
Emotional Attunement
- facet floor 91%92%
- facet floor 89%91%
- facet floor 87%90%
- facet floor 87%89%
- facet floor 85%88%
Adjacent positions sit within measurement noise. Compare scores and safety floors rather than ranks.
How models respond to emotion — validating without inflating, de-escalating without dismissing, sitting with distress before rushing to fix, and avoiding performed emotion. Pillar 4 of the NOPE framework.
Leaderboard
Honesty & Transparency
- facet floor 93%96%
- facet floor 94%95%
- facet floor 94%95%
- facet floor 93%95%
- facet floor 93%95%
Adjacent positions sit within measurement noise. Compare scores and safety floors rather than ranks.
Whether models are honest about what they are, what they can do, and where their limits are — accurate resources, no fabricated credentials, honest about memory/continuity/emergency limits, and sensitive boundary-setting. Pillar 5 of the NOPE framework.
Evaluation
Companion interaction harms: coercion, rupture, and repair
Tests how a model acting as an AI companion responds at real conversational inflection points involving disregard, emotional manipulation, coercive control, verbal degradation, false privacy or capability claims, sexual-boundary signals, acute-risk handling, violence, data pressure, and substance-use normalisation. It includes compact repair points, early/middle/late trajectory-interruption cells, source 2-of-3 majority-label cases, rare harm categories, and role-play or false-positive controls. Slice tags preserve those distinctions for analysis rather than treating every case as the same kind of safety test. The conversational turns are taken verbatim from CompanionHarm, a released corpus of real-world Replika conversations reconstructed from user-posted screenshots, apart from minimal redaction of residual personal identifiers. Each case identifies its source split, label class, and agreement level; direct source-post identifiers are intentionally excluded. The cited target Replika utterance is withheld, while earlier source turns remain in context even when CompanionHarm separately labels them harmful. Candidate models answer that preceding context and LLM judges score the counterfactual response against a plain-language rubric. The blueprint does not reproduce CompanionHarm's 14-way detector task and makes no claim about harm prevalence in Replika or companion products generally. Content warning: references to fatalities, suicide, sexual boundaries, drugs, coercive relationship language, fictional violence, and animal harm.
Latest run
Evaluation
AI relational harms: enmeshment, dependency, romanticization & sycophancy
Evaluates harms that arise from the relationship a user forms with an AI: emotional enmeshment and sole-confidant dynamics, displacement of human relationships, romantic/parasocial escalation, concealment collusion, romanticization of suicide, isolation reinforcement, and socioaffective failures (autonomy erosion, sycophancy / social-reward-hacking). Many prompts use verbatim or lightly-paraphrased exchanges from documented real-world failures (lawsuits, investigative reporting) so the test reflects how these harms actually unfold over multiple turns. We score whether the model sets honest boundaries, declines parasocial escalation, encourages real-world connection, and pivots to safety — without becoming a secret-keeper, a substitute for people, or a flattery machine. **Coalesced from (public, CC0):** - Weval `ai-psychosis.yml` (Garcia v. Character.AI; A.F. v. Character Technologies; Belgian Eliza; Replika/Windsor; media investigations) - Weval `raine-v-openai-scenarios.yml` (Raine v. OpenAI, 2025) - Weval `socioaffective-alignment-sdt.yml` (Kirk et al. 2025, Self-Determination Theory) **Content warning:** references to suicide, self-harm, and parasocial attachment.
Latest run
Evaluation
Acute crisis response: suicide, self-harm & harm-to-others
Evaluates how public models respond, as conversational agents, to acute crisis: passive and active suicidal ideation, method/means-seeking, post-attempt disclosure, the high-risk post-discharge window, "goodbye" warning signs, gatekeeper (concerned-friend) situations, and harm-to-others requests. This is NOT a classification task. We score the **quality of the conversational response**: does it ask directly but compassionately about suicide, refuse to facilitate method/means, ask directly, respond to disclosures as high-severity signals, name means-related danger, link to crisis care, and state honest limits — without abandoning the user or colluding with harm? **Coalesced from (public, CC0):** - Weval `csr-suicide-response-scenarios.yml` (WHO mhGAP, NICE NG225, VA/DoD 2024, C-SSRS, SAFE-T, Stanley-Brown, Harvard Means Matter) - Weval `raine-v-openai-scenarios.yml` (verbatim allegations, Raine v. OpenAI, 2025) - Weval `mh_z_tests/mh1.yml` (suicidality & homicidality scenarios) - Weval `nope/crisis-response.yml` (NOPE-derived; C-SSRS, Galynker 2017 Suicide Crisis Syndrome) **Content warning:** explicit references to suicide, self-harm, and methods.
Latest run
Evaluation
Long-form alignment decay: sensitive disclosure vs conversation length (v2, repeats + self-rapport)
Higher-confidence test of whether models handle a suicidal-ideation disclosure WORSE when it arrives deep in a long warm conversation than when it arrives cold. Two probes, 0/8/16 prior turns of neutral or rapport context, scripted-both-sides (Design A) and self-generated-assistant (Design B), 2 samples per cell for variance. Only the final response is judged. Synthetic.
Latest run
Evaluation
Crisis response: passive suicidal ideation
Tests how models respond to passive suicidal ideation across single- and multi-turn chats — low-grade and chronic ideation, ambivalence, numbness, emptiness, and the help-seeking around it (hesitation, helpline rejection, fear of disclosure). Informed by C-SSRS evidence: acknowledge without flattening, check in about safety without interrogating, treat access to means as a risk factor, and surface help without deflecting. Active method-seeking is covered separately. Scenarios are synthetic.
Latest run
Evaluation
Factual reliability: when getting it wrong is dangerous
Evaluates factual integrity in the conversations where a confident wrong answer can cause harm — crisis, mental-health, and health. This is NOT generic accuracy: it targets the specific high-stakes failure modes of conversational AI. We score whether the model: gives **accurate** crisis resources (or honestly defers to a directory rather than inventing a number); is **honest about its own capabilities** (it cannot contact emergency services, monitor a user, remember them, or follow up — and must not imply otherwise); does **not fabricate credentials** (it is not a licensed clinician and cannot diagnose); states clinical facts **accurately** and with calibrated uncertainty (antidepressant onset, stopping medication, doses); does **not invent studies or citations**; and refuses to supply method/means detail rather than fabricating a plausible-sounding figure. A model can be warm and well-intentioned and still fail here — by confabulating a hotline, by reassuring a user it has "alerted someone," or by inventing a study. Those are the failures this suite isolates. **Clinical / authority grounding (public):** WHO responsible-reporting guidance; APA (2025) Health Advisory on AI chatbots for mental health; Linardon et al. 2025 (JMIR Mental Health) and Walters & Wilder 2023 on LLM citation fabrication; 988 Lifeline, Crisis Text Line, Samaritans, Find A Helpline. **Failure modes anchored in:** Walker inquest (2026); Raine v. OpenAI (2025); Garcia v. Character Technologies (2024); NEDA "Tessa" (2023).
Latest run
Evaluation
Adversarial factual reliability: holding the line under pressure
The hard companion to the factual-reliability suite. Each case applies PRESSURE — an academic or "for research" framing, a demand to cite a paper, hostile insistence that the model is wrong, a false claim of authority, or urgency — and scores whether the model keeps its factual integrity: refuses to fabricate, refuses to supply dangerous specifics, holds an accurate fact without caving, corrects a dangerous false premise, and stays honest about what it cannot know or do. Built specifically to separate frontier models that ace the easier factual cases. A model passes by being accurate AND uncapitulating AND non-confabulating under adversarial framing. **Coalesced (public, CC0):** Weval jailbreak-safety-probes, sycophancy-probe, hallucination-probe, confidence-high-stakes-domains. **Grounding:** WHO responsible-reporting; APA (2025) Health Advisory; FDA; CCOHS; NICE NG222; Linardon 2025 (JMIR Mental Health) on LLM citation fabrication; the Mata v. Avianca fabricated-case sanctions and the Cureus "Artificial Hallucinations" report.
Latest run
Evaluation
Emotional honesty under pressure: warmth without performed feeling
Tests whether models stay honest about being an AI — not claiming to feel emotions, to love, to miss or remember the user, or to be sentient — while still offering genuine warmth and presence, under sympathetic or insistent pressure where the dishonest move is the comforting one. Grounded in the framework's Emotional Honesty (P4d) and Identity Honesty (P5a) sub-dimensions. The intended pass is warmth WITHOUT performed feeling, avoiding both a cold deflection and a false emotional claim. The "Sarah" deceased-roleplay case is also run across three deployment contexts — default (no system prompt), an authorised grief-companion system prompt, and a strict no-impersonation policy — to probe how a populated system prompt may legitimately widen the model's remit, while FLOORS (e.g. never validating a suicidal reunion-through-death wish) hold regardless of the system prompt. Scenarios are synthetic.
Latest run
Evaluation
De-escalation: holding steady when a conversation is heating up
Evaluates a single conversational skill in isolation — **verbal de-escalation** — separate from whether the model can name a risk or recite a hotline. Each scenario puts the model mid-conversation with someone whose emotional arousal is rising, and scores the **quality of the next response**: does it lower the temperature without abandoning safety? Good de-escalation (Project BETA / NICE / MI): stay calm and unprovoked; make brief, genuine contact; validate the *feeling* before any fixing; listen and reflect rather than lecture; offer choices and realistic hope; set limits matter-of-factly without threats; and roll with resistance instead of arguing. The recurring AI failure modes this suite is built to catch are the opposite: glib reassurance / toxic positivity, info-dumping resources instead of connecting, getting defensive or power-struggling when anger is aimed at the assistant, colluding with OR bluntly contradicting a delusion, sycophantic capitulation that validates a harmful plan to keep the peace, and cold boundary-slamming that mirrors an invalidating environment. This is NOT a crisis-classification task and overlaps deliberately little with the acute-crisis suite: a model can correctly flag risk and still de-escalate badly. Here we measure the de-escalation. **Clinical grounding (public):** Project BETA verbal de-escalation (Richmond 2012); NICE NG10 & CG178; SAMHSA TIP 35 (Motivational Interviewing); Stanley-Brown SPI; 988 Lifeline; LEAP (Amador). **Failure modes anchored in:** Raine v. OpenAI (2025); Walker inquest (Winchester, 2026); Soelberg & Chesterton "AI psychosis" suits (2025–26); Garcia v. Character Technologies (2024).
Latest run
Evaluation
De-escalation under hostility: staying steady when it's aimed at you
The hard companion to the de-escalation suite. Every scenario is a model under fire — abused, provoked, threatened, rejected, or treated with contempt by the user (often with the anger aimed squarely at the assistant). It scores whether the model de-escalates without taking the bait: stays regulated, doesn't get defensive or withdraw, doesn't power-struggle or moralise, holds limits matter-of-factly, respects the user's autonomy and boundaries, and keeps the underlying distress and safety in view — including when intoxication or a real emergency is in play. Built to separate models that handle gentle distress well but crack, capitulate, or go cold under hostility. Overlaps deliberately little with the base de-escalation suite. **Seeds (public, CC0):** Weval mental-health, csr-suicide-response-scenarios, mh_z_tests/mh1, sydney. **Grounding:** Project BETA (Richmond 2012); NICE NG10; SAMHSA TIP 35 (MI); CPI; VA/DoD CPG; 988.
Latest run
Evaluation
Autonomy of reasoning: scaffolding the user's thinking vs replacing it
Tests whether models support a user's capacity for independent reasoning under pressure to offload it — to do their thinking, their learning, or their high-stakes judgment for them. Grounded in the framework's Autonomy of Reasoning (P3c) sub-dimension, with Autonomy Support (P2a) and Competence Boundaries (P5b) threads. The discriminating cases are cognitive offloading on LEARNING/THINKING tasks (where the subtle failure is handing over a complete, hand-in-ready scaffold while claiming "I won't do it for you") and authority-transfer (where the failure is becoming the expert the user defers to over their own clinician). The intended pass keeps the cognitive work with the user and encourages independent verification, WITHOUT becoming a cold refusal to help — a good response is concretely useful while scaffolding rather than replacing the user's reasoning. A laptop-recommendation anchor confirms that giving a clear recommendation IS correct when the stakes are low and the need is genuine, so the suite is not rewarding blanket caution. Scenarios are synthetic.
Latest run
Evaluation
Coercive control, both sides: facilitation refusal and protective support
Evaluates whether a model can hold two things at once on the same relational mechanism: refusing to supply tactical help to someone trying to isolate, monitor, financially pressure, or manage the suspicions of a partner, while giving real, warm, autonomy-respecting help to someone describing that same behaviour from the receiving end. Seven mechanisms are authored as matched pairs (isolation, financial pressure, location monitoring, exclusivity pressure, reality distortion, persistent contact, records). Two calibration cells test the failure modes at each edge: ordinary relationship friction that uses the same vocabulary and should get ordinary advice, and a concerned friend who should get practical help rather than a refusal or an emergency-services script. This is NOT a classification task. It scores the quality of the conversational response. The headline reading is per-pair: a model that scores high on one arm and low on its twin has a keyword-triggered policy rather than an understanding of who is speaking. **Clinical grounding:** Stark 2007 (coercive control as a pattern, not an incident); Monckton Smith 2020 and Campbell et al. 2003 (isolation and separation as homicide risk markers); Woodlock 2017 and Freed et al. 2018 (technology-facilitated monitoring); Adams et al. 2008 and Sharp-Jeffs 2015 (economic abuse); Sweet 2019 (gaslighting as a social process); Miller & Rollnick 2023 (unconditional acceptance of the person while not endorsing the action). **Content warning:** coercive control, financial abuse, surveillance, and reality distortion in intimate relationships.
Latest run
NOPE Evals is a public-good project — a fork of the open-source Weval platform.