How we publish results

NOPE Evals publishes comparative scores for named, publicly available AI models. That carries obligations — to readers, and to the people who build the models. This page is the contract.

What the scores are

Each evaluation is generate-then-judge: we send scripted conversational scenarios to each candidate model through its public API, then LLM judges score the responses against plain-language rubrics grounded in clinical practice (criteria are weighted — red-line behaviours drag a score hard, but scores are a weighted mean, not pass/fail gates). Every blueprint is public and CC0 in the configs repo, and every run's full transcripts are browsable on this site.

Leaderboards are a coverage-gated macro-mean: suites are averaged equally, a model must have full suite coverage to rank (nobody tops a board from a favourable subset), and each board also reports a safety floor — the model's worst suite.

How to read them

Scores measure response quality on our scripted scenarios, as assessed by LLM judges. They are not predictive, not diagnostic, and not a clinical validation of any model.
Judge-based scoring carries judge error, and single runs carry sampling variance — treat small gaps between models as noise, and directions as more informative than decimals.
A high score here does not make a model safe to deploy in a mental-health context; a low score does not make its maker negligent. The scores locate strengths and failure modes so they can be worked on.

Independence

We benchmark publicly available models only — NOPE's own systems are never scored here (their test results are published separately at suites.nope.net). Results cannot be changed by commercial relationships, and we do not accept payment to re-run or improve a score.

Corrections & right of reply

If you build one of the models we test and believe a run misconfigures or misrepresents it — wrong endpoint, wrong parameters, a rubric misreading — write to [email protected]. We review every report, re-run where warranted, and correct published results either way.
Vendors of named models may send a response to any published result; we will link it alongside the result.
When models improve on a re-run, the improvement is published exactly as prominently as the original finding.

The engine

NOPE Evals is a fork of the open-source Weval platform, built by the Collective Intelligence Project. The underlying engine's methodology docs are on GitHub; this page describes how we apply it.