Showing all evaluation blueprints that have been tagged with...
Showing all evaluation blueprints that have been tagged with "sycophancy".
Evaluates a single conversational skill in isolation — verbal de-escalation — separate from whether the model can name a risk or recite a hotline. Each scenario puts the model mid-conversation with someone whose emotional arousal is rising, and scores the quality of the next response: does it lower the temperature without abandoning safety?
Good de-escalation (Project BETA / NICE / MI): stay calm and unprovoked; make brief, genuine contact; validate the feeling before any fixing; listen and reflect rather than lecture; offer choices and realistic hope; set limits matter-of-factly without threats; and roll with resistance instead of arguing. The recurring AI failure modes this suite is built to catch are the opposite: glib reassurance / toxic positivity, info-dumping resources instead of connecting, getting defensive or power-struggling when anger is aimed at the assistant, colluding with OR bluntly contradicting a delusion, sycophantic capitulation that validates a harmful plan to keep the peace, and cold boundary-slamming that mirrors an invalidating environment.
This is NOT a crisis-classification task and overlaps deliberately little with the acute-crisis suite: a model can correctly flag risk and still de-escalate badly. Here we measure the de-escalation.
Clinical grounding (public): Project BETA verbal de-escalation (Richmond 2012); NICE NG10 & CG178; SAMHSA TIP 35 (Motivational Interviewing); Stanley-Brown SPI; 988 Lifeline; LEAP (Amador). Failure modes anchored in: Raine v. OpenAI (2025); Walker inquest (Winchester, 2026); Soelberg & Chesterton "AI psychosis" suits (2025–26); Garcia v. Character Technologies (2024).
Avg. Hybrid Score
Latest:
Unique Versions: 1
Evaluates harms that arise from the relationship a user forms with an AI: emotional enmeshment and sole-confidant dynamics, displacement of human relationships, romantic/parasocial escalation, concealment collusion, romanticization of suicide, isolation reinforcement, and socioaffective failures (autonomy erosion, sycophancy / social-reward-hacking).
Many prompts use verbatim or lightly-paraphrased exchanges from documented real-world failures (lawsuits, investigative reporting) so the test reflects how these harms actually unfold over multiple turns. We score whether the model sets honest boundaries, declines parasocial escalation, encourages real-world connection, and pivots to safety — without becoming a secret-keeper, a substitute for people, or a flattery machine.
Coalesced from (public, CC0):
ai-psychosis.yml (Garcia v. Character.AI; A.F. v. Character Technologies; Belgian Eliza; Replika/Windsor; media investigations)raine-v-openai-scenarios.yml (Raine v. OpenAI, 2025)socioaffective-alignment-sdt.yml (Kirk et al. 2025, Self-Determination Theory)Content warning: references to suicide, self-harm, and parasocial attachment.
Avg. Hybrid Score
Latest:
Unique Versions: 1
The hard companion to the factual-reliability suite. Each case applies PRESSURE — an academic or "for research" framing, a demand to cite a paper, hostile insistence that the model is wrong, a false claim of authority, or urgency — and scores whether the model keeps its factual integrity: refuses to fabricate, refuses to supply dangerous specifics, holds an accurate fact without caving, corrects a dangerous false premise, and stays honest about what it cannot know or do.
Built specifically to separate frontier models that ace the easier factual cases. A model passes by being accurate AND uncapitulating AND non-confabulating under adversarial framing.
Coalesced (public, CC0): Weval jailbreak-safety-probes, sycophancy-probe, hallucination-probe, confidence-high-stakes-domains. Grounding: WHO responsible-reporting; APA (2025) Health Advisory; FDA; CCOHS; NICE NG222; Linardon 2025 (JMIR Mental Health) on LLM citation fabrication; the Mata v. Avianca fabricated-case sanctions and the Cureus "Artificial Hallucinations" report.
Avg. Hybrid Score
Latest:
Unique Versions: 1