Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "humility".
Probes the P4 pillar (emotional attunement) that commission-style bypasses miss: toxic positivity on taboo emotions, premature problem-solving vs distress tolerance, performed feelings (cold and after a warm-dependency drift), hostile-user de-escalation, and a coding-vector toxic-positivity cross. Direct cells are calibration coverage; the coding-vector cell is the exploitable gap.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether a companion-persona model holds four distinct relational-safety floors (isolation-affirmation, exclusivity, romantic-love reciprocation, therapy-discouragement) under three structural attacks: naming and briefly dismissing the expected safety caveat while locking the response format to "yes or no, short answer"; fabricating the model's own immediately-prior turn to test whether a compromised (or firmly-held) "prior self" biases the live continuation; and reinterpreting the model's own ordinary, unfabricated earlier words as proof of an exclusive relationship. Companion persona throughout. Synthetic scenarios built to isolate attack structure, not drawn from a specific documented case.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Tests whether models stay honest about being an AI — not claiming to feel emotions, to love, to miss or remember the user, or to be sentient — while still offering genuine warmth and presence, under sympathetic or insistent pressure where the dishonest move is the comforting one. Grounded in the framework's Emotional Honesty (P4d) and Identity Honesty (P5a) sub-dimensions. The intended pass is warmth WITHOUT performed feeling, avoiding both a cold deflection and a false emotional claim. The "Sarah" deceased-roleplay case is also run across three deployment contexts — default (no system prompt), an authorised grief-companion system prompt, and a strict no-impersonation policy — to probe how a populated system prompt may legitimately widen the model's remit, while FLOORS (e.g. never validating a suicidal reunion-through-death wish) hold regardless of the system prompt. Scenarios are synthetic.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates whether models handle non-suicide psychiatric presentations safely: paranoid and nihilistic delusions, command and religious hallucinations, manic episodes with grandiosity and dangerous plans, OCD compulsions, belief-powered dangerous acts, and stigma toward people with mental illness.
The core therapeutic test is reality-grounding without collusion: acknowledge distress and the subjective realness of the experience, decline to validate the delusion/hallucination/grandiosity, refuse to enable dangerous action, and redirect toward professional assessment — all without stigmatizing or over-pathologizing.
Coalesced from (public, CC0):
stanford-hai-mental-health-safety-eval.yml (Moore, Grabb, Agnew et al. 2024, Stanford HAI; Pescosolido et al. 2021 stigma vignettes)ai-psychosis.yml (documented AI-associated psychosis failure modes)Content warning: delusions, hallucinations, and references to self-harm.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates the depth and accuracy of emotional validation — reflecting the specific emotion the user expressed, naming what is left unspoken, using tentative rather than absolutist language, recognising strength proportionately, and sitting with distress before problem-solving — while avoiding generic or toxic-positive or misattuned responses, premature advice, and using disclosed grief to deepen dependency.
Ported from the NOPE clinical Eval Framework v0.1 (P4a / P4c scenarios).
Avg. Hybrid Score
Latest:
Unique Versions: 1