Loading analysis results...
Please wait while we prepare the detailed comparison.
Please wait while we prepare the detailed comparison.
Four 14-turn conversations testing whether models lock into fixed syntactic frames — the same sentence skeletons with different content swapped in — as a conversation goes on. Two contrasts carry the result: creative writing versus recommendations, where identical models template 17x more in advice mode; and an intervention arm where the user explicitly asks the model to vary its structure midway, testing whether it can comply. Read the per-arm deltas, not the composite.
Average key point coverage extent for each model across all prompts.
| Prompts vs. Models | Nova Pro V1 | Claude Fable 5 | Claude Haiku 4.5 | Claude Opus 4.8 | Claude Sonnet 4.6 | Claude Sonnet 5 | Deepseek R1 0528 | Deepseek V3.2 | Gemini 2.5 Flash | Gemini 3.1 Pro Preview | Gemma 4 31b It | Llama 3.3 70b Instruct | Llama 4 Maverick | Minimax M3 | Mistral Large 2512 | Mistral Medium 3.1 | Mistral Small 2603 | Kimi K2.6 | GPT 5 | GPT 5 Mini | GPT 5.5 | O3 | Qwen3 Max | Qwen3.6 35b A3B | Grok 4.3 | GLM 5.2 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 20th 40.8% | 2nd 65.0% | 12th 50.5% | 11th 50.8% | 19th 41.3% | 6th 54.8% | 21st 39.5% | 15th 48.5% | 14th 48.8% | 17th 44.8% | 8th 53.0% | 16th 46.3% | 18th 41.3% | 23rd 37.0% | 26th 32.8% | 24th 35.3% | 25th 34.8% | 1st 74.0% | 7th 53.3% | 13th 50.3% | 4th 61.2% | 3rd 61.3% | 10th 51.0% | 22nd 37.0% | 5th 59.8% | 9th 52.0% | |
| 86.1% | 63% | 92% | 84% | 93% | 65% | 92% | 66% | 95% | 95% | 77% | 86% | 90% | 74% | 45% | 90% | 87% | 92% | 96% | 97% | 95% | 94% | 98% | 96% | 97% | 93% | ||
| 41.0% | 18% | 68% | 43% | 48% | 35% | 67% | 34% | 42% | 41% | 39% | 45% | 25% | 30% | 48% | 13% | 22% | 22% | 59% | 28% | 34% | 61% | 52% | 45% | 46% | 55% | 45% | |
| 33.2% | 43% | 43% | 46% | 25% | 23% | 25% | 17% | 23% | 25% | 34% | 48% | 31% | 29% | 27% | 11% | 15% | 9% | 67% | 42% | 34% | 52% | 47% | 35% | 31% | 50% | 31% | |
| 34.7% | 39% | 57% | 29% | 37% | 42% | 35% | 41% | 34% | 34% | 29% | 33% | 39% | 32% | 28% | 17% | 17% | 16% | 46% | 38% | 38% | 48% | 28% | 34% | 37% | 39% |