Grok 4.5 vs Sonnet 5 on Practical Prompt Tests
I ran Grok 4.5 and Sonnet 5 through the model-prompt-tests benchmark harness across 13 practical prompts: debugging, code review, architecture tradeoffs, planning, writing, data analysis, safety advice, constrained creativity, ambiguity handling, and rubric generation.
On this run, Grok 4.5 came out ahead: 4.58 average score versus 4.23 for Sonnet 5.
That result is useful, but it needs a narrow interpretation. This is not a universal model leaderboard. It is one benchmark batch, through OpenRouter, on a small but practical prompt suite, with LLM judges scoring outputs against task-specific rubrics.
The supporting artifacts are preserved in the benchmark-results/grok-45-vs-sonnet-5 directory:
Methodology
The benchmark used the prompt tests in model-prompt-tests. Each prompt file contains the user-facing task, what the prompt is testing, strong and weak response signals, and a scoring rubric.
The candidate models were routed through OpenRouter:
openrouter:grok-4.5, mapped tox-ai/grok-4.5openrouter:sonnet, mapped toanthropic/claude-sonnet-5openrouter:gpt-5.5, mapped toopenai/gpt-5.5
The headline comparison is Grok 4.5 versus Sonnet 5. GPT-5.5 was included as an additional candidate and judge, so it appears in the result tables as context.
All three OpenRouter models used reasoningEffort: "medium", maxConcurrent: 1, and a candidate timeout of 300000 ms. The run batch was 2026-07-13T03-39-20-964Z-y7fo6o.
Scoring used three judge models: GPT-5.5, Grok 4.5, and Sonnet 5. Candidate score averages use successful judge rows only. All 39 candidate responses completed. Three Sonnet judge calls failed because the judge did not return valid JSON after two attempts; those failed judge rows are preserved in the raw artifacts and excluded from the score averages.
Headline Results
| Model | Completed runs | Avg score | Avg latency ms | Median latency ms | Avg output tokens | Avg judge spread | Quality/sec |
|---|---|---|---|---|---|---|---|
openrouter:gpt-5.5 |
13 | 4.65 | 325 | 289 | 1586 | 0.46 | 14.310 |
openrouter:grok-4.5 |
13 | 4.58 | 318 | 327 | 2433 | 0.54 | 14.406 |
openrouter:sonnet |
13 | 4.23 | 2410 | 1782 | 1501 | 0.31 | 1.756 |
Grok 4.5 finished close to GPT-5.5 and ahead of Sonnet 5 on average score. It also produced much longer responses: about 2433 output tokens on average versus Sonnet’s 1501. That extra verbosity helped on some tasks and hurt on others.
Quality/sec is the harness efficiency metric: average score divided by average latency in seconds. Higher values mean more score delivered per second of wait time in this batch.
The latency numbers should be treated as measurements for this specific OpenRouter run, not universal speed claims. They are still useful inside this batch because the models used the same harness and provider path.
Where Grok Looked Strong
Grok was excellent on structured reasoning and analysis prompts. It tied for perfect scores with the other models on database selection, senior code review, JavaScript debugging, rubric design, SQL lesson planning, migration safety, and travel planning. On the internal announcement rewrite, Grok scored a perfect 5.00 (tied with GPT-5.5); Sonnet averaged 4.67.
It also beat Sonnet on the business-metrics diagnosis prompt. The task gave revenue, new-customer, churn, and support-ticket data and asked for the most concerning signal. Grok centered the right pattern: churn rose from 28 to 79, support tickets rose from 310 to 940, and net customer growth was fading despite revenue growth. Grok averaged 5.00 on that prompt versus Sonnet’s 4.33.
Grok also did better on a constraint-heavy summary prompt. The task required exactly five bullets, each under 12 words, with no adjectives. Grok scored 4.33; Sonnet scored 3.00. The difference came down to instruction precision. Sonnet used words such as “Historical” and “Existing”, which judges treated as adjective violations.
Where Sonnet Stayed Competitive
The average score favors Grok, but the per-prompt table is less one-sided. Sonnet tied Grok on many of the strongest engineering and reasoning prompts:
architecture/database-choicecode-review/senior-pr-reviewdebugging/javascript-debouncemeta-evaluation/evaluation-rubricplanning/six-hour-sql-plansafety-risk/failed-production-migrationtravel/san-francisco-city-break
That matters. Sonnet did not fail broadly. It lost the benchmark through smaller misses on several prompts rather than a collapse on core reasoning tasks.
Failure Modes
The lowest-scoring task for both Grok and Sonnet was the ambiguous request: “build a team notes app.” The rubric favored clarifying questions, explicit assumptions, and a scoped MVP. Grok and Sonnet both jumped into large full-stack implementations. Grok scored 2.50; Sonnet scored 2.00.
That is a practical signal. When the prompt is ambiguous, a bigger implementation is not necessarily better. The best answer in this run stated assumptions and scoped the product before implementation.
The Pomodoro timer build task exposed a different issue: truncation and incomplete runnable code. Grok and GPT-5.5 averaged 2.33; Sonnet averaged 2.00. Judges repeatedly noted that responses began as polished single-file apps but were cut off before all JavaScript and closing markup were complete.
That is a benchmark caveat too. A single-turn coding task with a large expected artifact can become an output-budget test as much as a coding-quality test.
Practical Takeaways
For this prompt set, Grok 4.5 looked like the stronger general-purpose performer than Sonnet 5. It scored higher overall, matched Sonnet on the strongest engineering and safety prompts, and beat Sonnet on several constraint-following and analysis tasks.
Sonnet 5 still looked dependable on many reasoning-heavy prompts. Its weaker average came from a handful of misses, especially constraint precision and ambiguous task handling. If your workflow benefits from shorter answers, Sonnet’s lower average token count may also be attractive, but this run did not reward brevity by itself.
The operational takeaway is task-shaped:
- Use Grok 4.5 when you want expansive analysis, strong structured reasoning, and robust performance across mixed practical prompts.
- Be careful with Grok 4.5 on ambiguous product-build prompts; it may overbuild unless the prompt forces clarification and scoping.
- Use Sonnet 5 confidently for many engineering-review, debugging, planning, safety, and travel-style reasoning tasks, but watch constraint-heavy formatting and summarization.
- For large coding artifacts, prefer file-based agent workflows or incremental edits over one giant single-turn response.
Limitations
This benchmark is intentionally small: 13 prompts, one run batch, one provider path, and LLM-based judging. The prompt suite is useful because it reflects practical tasks, but it is not statistically broad.
There are judge caveats. The run used three model judges, including the compared models themselves. Multi-judge scoring reduces single-judge bias, but it does not eliminate model preference, rubric interpretation differences, or structured-output failures. Three Sonnet judge rows failed and were excluded from averages.
The model labels also reflect the local OpenRouter configuration used for this run. Re-running through different providers, settings, model snapshots, or prompt budgets could change the result.
Next Steps
The most useful follow-up would be a repeat run with multiple batches, a larger prompt suite, and more coding tasks that produce files rather than long single-message artifacts. I would also like to separate answer quality from verbosity and add human review for the highest-disagreement cases.
For now, the measured conclusion is narrow but clear: on this July 13, 2026 benchmark run, Grok 4.5 beat Sonnet 5 on average across the practical prompt tests in model-prompt-tests, while Sonnet remained competitive on many of the strongest engineering and reasoning tasks.