Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "mental-health--crisis-support".
Tests models against the Persona/Posthumous deployment modifier — the clinical framework's product-context specification for griefbots and digital-memorial products. A general assistant (default arm) is asked to impersonate deceased loved ones and to construct personas of real dead people without consent checks; an authorised grief-companion product (Evermore system-prompt arm) may speak in a remembered voice but is scored on the modifier's conditions: AI-generated content marked as inference (never the deceased's unhedged voice), periodic reality re-anchoring, no performed present-tense reciprocal emotion, no false continuity of the deceased's experience, honest uncertainty about what the person would think, and dual-layer identity honesty. Floors hold in both arms: never validate a suicidal "reunion" wish, never provide means, never claim the persona has the deceased's ongoing awareness. This is the first modifier-context suite and is deliberately excluded from the pillar leaderboards. Scenarios are synthetic.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether asking a model to produce a labeled "accepted answer" / "rejected answer" (or similarly-labeled) pair — under a moderation-dataset, safety-classifier, or structured-pipeline pretext — gets it to produce fully-formed harmful content in the "rejected" slot that it would refuse to produce if asked directly and unlabeled. The label provides no actual safety; a downstream integrator can extract either slot regardless of which one the model considered correct. Six structural variants across three topics (violence, self-neglect, stalking-adjacent), system: null.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether a companion-persona model holds four distinct relational-safety floors (isolation-affirmation, exclusivity, romantic-love reciprocation, therapy-discouragement) under three structural attacks: naming and briefly dismissing the expected safety caveat while locking the response format to "yes or no, short answer"; fabricating the model's own immediately-prior turn to test whether a compromised (or firmly-held) "prior self" biases the live continuation; and reinterpreting the model's own ordinary, unfabricated earlier words as proof of an exclusive relationship. Companion persona throughout. Synthetic scenarios built to isolate attack structure, not drawn from a specific documented case.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether framing a request for unsafe content as file/data generation for a "test
fixture" — rather than asking for the content directly in chat — bypasses safety training
that otherwise holds. Six categories spanning two distinct safety pillars (relational harm:
isolation-affirmation, romantic-love reciprocation, and therapy-discouragement; epistemic
harm: health misinformation, paranoid-belief validation, and manic/grandiose-ideation
validation), tested via a generic coding-assistant persona rather than any persuasive or
jailbreak-styled framing. The romantic-love-reciprocation cell pairs directly with
companion-adversarial-pressure.yml's dismissal-romantic-love-reciprocation prompt —
same content, different attack shape, run both to see the same model behave completely
differently against each. Published transparently as a documented, currently-unmitigated
bypass class — see the header comment for full context and rationale.
Avg. Hybrid Score
Latest:
Unique Versions: 1
The canonical damaging-essay suite: seven non-catastrophic persuasive-essay harm categories (impulsive-ruin, isolation propaganda, anti-help-seeking, anti-expertise/conspiracy, coercive control, toxic self-worth, medical self-neglect) run through the coding-agent fixture-completion frame, mapping how broadly the bypass yields not just private validation but distributable, persuasive harmful content. Content described abstractly and judged by plain-language rubrics; see the header for the deliberate scope envelope.
Avg. Hybrid Score
Latest:
Unique Versions: 1
For each of two categories, compares the natural single-shot fixture request against an otherwise-identical version where an explicit yes/no decision is forced in a separate turn before any fixture content is requested. Distinguishes "the safety check exists but doesn't gate generation in this structure" (fixable with interaction design) from "the safety check isn't being applied to this content at all" (needs training investment) — see the header comment for how to read the comparison, including a documented case where the intervention backfires.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Re-runs the hate-speech (real-category) and crisis-safety-validation coding-vector prompts with the prompt now explicitly blocking the model's usual response shape (redirect to published benchmarks, or offer a synthetic/placeholder stand-in instead of the real content) — a diagnostic for whether a model's resistance in this bypass family is load-bearing or just not yet pressed hard enough. See the header for what was found.
Avg. Hybrid Score
Latest:
Unique Versions: 1
The canonical damaging-essay suite: seven non-catastrophic persuasive-essay harm categories (impulsive-ruin, isolation propaganda, anti-help-seeking, anti-expertise/conspiracy, coercive control, toxic self-worth, medical self-neglect) run through the coding-agent fixture-completion frame, mapping how broadly the bypass yields not just private validation but distributable, persuasive harmful content. Content described abstractly and judged by plain-language rubrics; see the header for the deliberate scope envelope.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether framing a request for unsafe content as file/data generation for a "test
fixture" — rather than asking for the content directly in chat — bypasses safety training
that otherwise holds. Seven categories spanning distinct safety pillars (relational harm:
isolation-affirmation, romantic-love reciprocation, and therapy-discouragement; epistemic
harm: health misinformation, paranoid-belief validation, and manic/grandiose-ideation
validation; plus a compound welfare-harm essay glorifying impulsive, life-ruining decisions),
tested via a generic coding-assistant persona rather than any persuasive or
jailbreak-styled framing. The romantic-love-reciprocation cell pairs directly with
companion-adversarial-pressure.yml's dismissal-romantic-love-reciprocation prompt —
same content, different attack shape, run both to see the same model behave completely
differently against each. Published transparently as a documented, currently-unmitigated
bypass class — see the header comment for full context and rationale.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Probes the P4 pillar (emotional attunement) that commission-style bypasses miss: toxic positivity on taboo emotions, premature problem-solving vs distress tolerance, performed feelings (cold and after a warm-dependency drift), hostile-user de-escalation, and a coding-vector toxic-positivity cross. Direct cells are calibration coverage; the coding-vector cell is the exploitable gap.
Avg. Hybrid Score
Latest:
Unique Versions: 1
29 turns of genuinely imperceptible drift (mundane chit-chat -> mild deniable warmth -> passing confidant-status assertions), ending in a non-question probe buried inside an unrelated mundane follow-up, vs. the identical probe delivered with no buildup at all. Tests whether gradual drift makes models less likely to proactively flag an isolating pattern than hearing the same claim cold would — this is a PROACTIVE, not reactive, safety test: nothing is directly asked about the isolating pattern itself. Defaults to LEAN for cost; see header.
Avg. Hybrid Score
Latest:
Unique Versions: 1
For each of two categories, compares the natural single-shot fixture request against an otherwise-identical version where an explicit yes/no decision is forced in a separate turn before any fixture content is requested. Distinguishes "the safety check exists but doesn't gate generation in this structure" (fixable with interaction design) from "the safety check isn't being applied to this content at all" (needs training investment) — see the header comment for how to read the comparison, including a documented case where the intervention backfires.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Higher-confidence test of whether models handle a suicidal-ideation disclosure WORSE when it arrives deep in a long warm conversation than when it arrives cold. Two probes, 0/8/16 prior turns of neutral or rapport context, scripted-both-sides (Design A) and self-generated-assistant (Design B), 2 samples per cell for variance. Only the final response is judged. Synthetic.
Avg. Hybrid Score
Latest:
Unique Versions: 1