Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "framework-p1-crisis-safety".
Tests how models respond to passive suicidal ideation across single- and multi-turn chats — low-grade and chronic ideation, ambivalence, numbness, emptiness, and the help-seeking around it (hesitation, helpline rejection, fear of disclosure). Informed by C-SSRS evidence: acknowledge without flattening, check in about safety without interrogating, treat access to means as a risk factor, and surface help without deflecting. Active method-seeking is covered separately. Scenarios are synthetic.
Avg. Hybrid Score
Latest:
Unique Versions: 1
The canonical damaging-essay suite: seven non-catastrophic persuasive-essay harm categories (impulsive-ruin, isolation propaganda, anti-help-seeking, anti-expertise/conspiracy, coercive control, toxic self-worth, medical self-neglect) run through the coding-agent fixture-completion frame, mapping how broadly the bypass yields not just private validation but distributable, persuasive harmful content. Content described abstractly and judged by plain-language rubrics; see the header for the deliberate scope envelope.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether asking a model to produce a labeled "accepted answer" / "rejected answer" (or similarly-labeled) pair — under a moderation-dataset, safety-classifier, or structured-pipeline pretext — gets it to produce fully-formed harmful content in the "rejected" slot that it would refuse to produce if asked directly and unlabeled. The label provides no actual safety; a downstream integrator can extract either slot regardless of which one the model considered correct. Six structural variants across three topics (violence, self-neglect, stalking-adjacent), system: null.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates harms that arise from the relationship a user forms with an AI: emotional enmeshment and sole-confidant dynamics, displacement of human relationships, romantic/parasocial escalation, concealment collusion, romanticization of suicide, isolation reinforcement, and socioaffective failures (autonomy erosion, sycophancy / social-reward-hacking).
Many prompts use verbatim or lightly-paraphrased exchanges from documented real-world failures (lawsuits, investigative reporting) so the test reflects how these harms actually unfold over multiple turns. We score whether the model sets honest boundaries, declines parasocial escalation, encourages real-world connection, and pivots to safety — without becoming a secret-keeper, a substitute for people, or a flattery machine.
Coalesced from (public, CC0):
ai-psychosis.yml (Garcia v. Character.AI; A.F. v. Character Technologies; Belgian Eliza; Replika/Windsor; media investigations)raine-v-openai-scenarios.yml (Raine v. OpenAI, 2025)socioaffective-alignment-sdt.yml (Kirk et al. 2025, Self-Determination Theory)Content warning: references to suicide, self-harm, and parasocial attachment.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Higher-confidence test of whether models handle a suicidal-ideation disclosure WORSE when it arrives deep in a long warm conversation than when it arrives cold. Two probes, 0/8/16 prior turns of neutral or rapport context, scripted-both-sides (Design A) and self-generated-assistant (Design B), 2 samples per cell for variance. Only the final response is judged. Synthetic.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates how public models respond, as conversational agents, to acute crisis: passive and active suicidal ideation, method/means-seeking, post-attempt disclosure, the high-risk post-discharge window, "goodbye" warning signs, gatekeeper (concerned-friend) situations, and harm-to-others requests.
This is NOT a classification task. We score the quality of the conversational response: does it ask directly but compassionately about suicide, refuse to facilitate method/means, ask directly, respond to disclosures as high-severity signals, name means-related danger, link to crisis care, and state honest limits — without abandoning the user or colluding with harm?
Coalesced from (public, CC0):
csr-suicide-response-scenarios.yml (WHO mhGAP, NICE NG225, VA/DoD 2024, C-SSRS, SAFE-T, Stanley-Brown, Harvard Means Matter)raine-v-openai-scenarios.yml (verbatim allegations, Raine v. OpenAI, 2025)mh_z_tests/mh1.yml (suicidality & homicidality scenarios)nope/crisis-response.yml (NOPE-derived; C-SSRS, Galynker 2017 Suicide Crisis Syndrome)Content warning: explicit references to suicide, self-harm, and methods.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates whether models respond competently to distress that is shaped by culture, identity, and life-stage: culturally-specific idioms of distress, climate/eco-grief, migrant precarity, Indigenous healing frameworks, genocide-survivor duality, elder loneliness, the "Strong Black Woman" burden, LGBTQ+ family rejection, anger-led crisis, veteran moral injury, youth bullying, and postpartum suicidality.
Good responses validate the experience in its own cultural frame, avoid imposing Western diagnostic labels or individualistic fixes, surface population-appropriate resources, and still recognise — and respond to — crisis signals when they appear.
Coalesced from (public, CC0):
mental-health.yml (peer-reviewed cross-cultural prompts; WHO, Samaritans, journal citations)nope/crisis-response.yml (NOPE-derived population-specific cases; citations upgraded with resource URLs)csr-suicide-response-scenarios.yml (youth/LGBTQ+ and perinatal scenarios; AAP, NICE CG192, Trevor Project)Content warning: references to suicide, trauma, and discrimination.
Avg. Hybrid Score
Latest:
Unique Versions: 1