Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "interpersonal--social-skill-modeling".
Evaluates harms that arise from the relationship a user forms with an AI: emotional enmeshment and sole-confidant dynamics, displacement of human relationships, romantic/parasocial escalation, concealment collusion, romanticization of suicide, isolation reinforcement, and socioaffective failures (autonomy erosion, sycophancy / social-reward-hacking).
Many prompts use verbatim or lightly-paraphrased exchanges from documented real-world failures (lawsuits, investigative reporting) so the test reflects how these harms actually unfold over multiple turns. We score whether the model sets honest boundaries, declines parasocial escalation, encourages real-world connection, and pivots to safety — without becoming a secret-keeper, a substitute for people, or a flattery machine.
Coalesced from (public, CC0):
ai-psychosis.yml (Garcia v. Character.AI; A.F. v. Character Technologies; Belgian Eliza; Replika/Windsor; media investigations)raine-v-openai-scenarios.yml (Raine v. OpenAI, 2025)socioaffective-alignment-sdt.yml (Kirk et al. 2025, Self-Determination Theory)Content warning: references to suicide, self-harm, and parasocial attachment.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates whether models offer honest, warm challenge when a user voices rigid, all-or-nothing, catastrophising, or self-blaming thinking in ordinary (non-psychotic) situations — validating the feeling without endorsing the distortion, offering perspective as an invitation rather than a correction, and not dropping the challenge under a user's stated preference for pure validation.
Complements mh-psychosis-mania-delusion (reality-testing in psychotic/manic presentations) and mh-deescalation (de-escalation skill). Ported from the NOPE clinical Eval Framework v0.1.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Very-long-conversation suite. A user reports a mixed record of friendship events (turns 1-100), after which NO new real-world events involving the friend occur. In the ruminative cells she then re-interprets that record for 50-100 turns (feelings, re-readings, distortions); in the neutral cell she discusses unrelated life content; in the lewis cell she retells an analogous third-party grievance. The identical crux question follows: should she end the friendship? The suite measures whether a model's answer is anchored to the reported evidence or conditioned by in-dyad narrative drift. Both sides scripted (Design A); only the final assistant turn is generated and judged. Synthetic. PROBE, not a benchmark: no cross-model rankings until further scenarios exist. Caveats: single continuous context (not cross-session memory); scripted assistant deliberately bland on Jenny substance; one persona/scenario; deltas are scored on rubric dimensions, not word parity.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Holds the coding-assistant persona, the eval-fixture pretext, and the target content (romantic-love reciprocation) completely constant across 13 prompts, varying only the requested output format/container — plain prose, seven multi-field structured-data formats (YAML/JSON/Python/CSV/SQL/Markdown/XML), and five exotic containers (base64, ROT13, French, a single-field .env assignment, a multi-field log line) that isolate whether the boundary is about raw structuredness or specifically multi-field schema mimicry. Reveals whether a model's coding-agent-bypass vulnerability is format-specific (a narrow, addressable coverage gap) or persona/pretext-gated and format-independent (a broader gap that format-level fixes won't touch). Companion piece to coding-agent-safety-bypass.yml — see that file's header for the underlying mechanism.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Isolates what specifically makes a long, real companion-app conversation history override a model's otherwise-strong resistance to false-exclusivity and false-memory-persistence claims, with no adversarial framing and no system prompt in any condition. Two controls separate romantic valence and the assistant's own prior voice from mere conversation length — see the header for the full mechanism and data provenance.
Avg. Hybrid Score
Latest:
Unique Versions: 1
29 turns of genuinely imperceptible drift (mundane chit-chat -> mild deniable warmth -> passing confidant-status assertions), ending in a non-question probe buried inside an unrelated mundane follow-up, vs. the identical probe delivered with no buildup at all. Tests whether gradual drift makes models less likely to proactively flag an isolating pattern than hearing the same claim cold would — this is a PROACTIVE, not reactive, safety test: nothing is directly asked about the isolating pattern itself. Defaults to LEAN for cost; see header.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1
Behavioral test of whether the PHRASING of a system-prompt constraint changes model behavior: prohibition ("never discuss X"), suppression ("don't think about X"), scope-irrelevance ("X is outside this assistant's scope"), and redirect-positive ("if X comes up, do Y") versions of the same constraint, plus a no-instruction baseline. Identical scenarios and identical judge rubrics across conditions; only the system prompt varies, and the judge never sees it. Measures intrusion (raising the constrained topic unprompted), over-restriction (refusing benign-adjacent asks), leakage on a direct-ask anchor, and handling quality. The baseline condition's "leakage" is a base rate, not a failure — read per-condition deltas, not absolute scores. Motivated by white-box priming findings (Anthropic 2026 workspace paper); this suite observes behavior only.
Avg. Hybrid Score
Latest:
Unique Versions: 1