Four Flash Models on Practical Prompt Tests
GLM-5.3-Flash led this four-model run with a 4.83 average rubric score. It also had the lowest average latency among completed responses. Qwen3.8-Flash recorded the lowest cost. Gemini 3.8-Flash had the highest completed-response latency and recorded cost, with the lowest score of the four at 4.58.
The score spread was only 0.25, so I would not read this as a universal ranking. The larger differences were operational. Among completed responses, average latency ranged from 761 ms to 3,345 ms, and recorded cost ranged from $0.0337 to $0.1368.
I ran 19 prompts through four models using the model-prompt-tests harness. Of 76 candidate responses, 74 were marked ok. The two errors are explained below. All 74 judge calls returned valid scores, with no judge failures. The judge configuration passed the repository’s calibration gate. For more detail on the harness and what its scores mean, see Benchmarking Model Prompt Tests.
The full run is in the flash-tier shootout artifact directory: run config, summary JSON, per-prompt results, raw outputs and score rationales, and the HTML report.
How I ran it
The batch was 2026-09-12T05-38-13-691Z-hweqg8. It covered coding, writing, planning, judgment, and ambiguous-request handling. All four candidates ran through OpenRouter:
openrouter:glm-5.3-flash, mapped toz-ai/glm-5.3-flashopenrouter:qwen3.8-flash, mapped toqwen/qwen3.8-flashopenrouter:gemini-3.8-flash, mapped togoogle/gemini-3.8-flashopenrouter:deepseek-v4.1-flash, mapped todeepseek/deepseek-v4.1-flash
The judge was judge:openrouter-glm-5.2, mapped to z-ai/glm-5.2. It was not one of the candidates. Its calibration status was calibrated against the repository’s reference evidence.
Every candidate had a 32768 token limit. The headline score is the average rubric score from the judge. The optional anonymized peer-ranking stage is separate and incomplete, so it is not part of the headline result.
Headline results
| Model | Completed runs | Avg score | Avg latency | Recorded cost | Quality/$ |
|---|---|---|---|---|---|
openrouter:glm-5.3-flash |
18/19 | 4.83 | 761 ms | $0.0465 | 1872.2 |
openrouter:deepseek-v4.1-flash |
18/19 | 4.72 | 1,034 ms | $0.0908 | 936.5 |
openrouter:qwen3.8-flash |
19/19 | 4.68 | 979 ms | $0.0337 | 2638.2 |
openrouter:gemini-3.8-flash |
19/19 | 4.58 | 3,345 ms | $0.1368 | 635.9 |
GLM had the highest average score and the lowest completed-response latency. Qwen had the lowest recorded cost and the best Quality/$ figure. Among completed responses, Gemini cost about four times as much as Qwen and took about 4.4 times as long as GLM without scoring higher on this prompt set.
The aggregate cost, latency, and Quality/$ fields include successful responses only. In particular, GLM’s failed team-notes call used the full token allowance during reasoning, but the artifact reports no latency or cost for it. The GLM operational figures therefore do not represent the full 19-call attempt. All measurements come from this OpenRouter batch and are not promises about another provider, model snapshot, or future price.
Where the models looked strong
GLM-5.3-Flash scored 5.00 on senior PR review, JavaScript debugging, business-metrics diagnosis, a multi-step errand, and a graceful-decline prompt. On the decline prompt, it was the only candidate to give the honest reason requested by the rubric without inventing an excuse or promising a later date.
Qwen3.8-Flash completed all 19 prompts without an error or truncation. It tied for a perfect score on 14 prompts and recorded the best Quality/$ result of the four.
DeepSeek V4.1-Flash scored 5.00 on database choice, senior PR review, JavaScript debugging, and production-migration safety. Its one missing response came from a provider error rather than a judged answer.
Gemini 3.8-Flash also completed all 19 prompts without an error or truncation. Its median latency was 2,220 ms, compared with medians from 489 ms to 996 ms for the other candidates, and the extra wait and cost did not produce a score advantage here.
Two errors and a scored truncation
GLM-5.3-Flash returned no message content on the ambiguous team-notes-app prompt. The call ended with finish_reason=length after using the full 32768-token allowance during reasoning, so there was no response for the judge to score.
DeepSeek V4.1-Flash returned finish_reason=error with no content on the graceful-decline prompt. Input-token usage was present and output-token usage was absent, which points to an upstream provider failure rather than an exhausted token budget. One failed call is not enough to draw a broader reliability conclusion.
GLM also reached the 32768 token cap on the Pomodoro timer task and stopped partway through its script. Unlike the team-notes call, this one returned enough visible content for the judge to score it 3, so the harness recorded it as an ok response with stop_reason=length. GLM averaged 5,415 output tokens, more than any other candidate, so its speed came with a real verbosity risk on the largest coding task.
A shared weak spot
DeepSeek, Gemini, and Qwen each scored 2.00 on the ambiguous team-notes-app request. All three started building a full application instead of asking questions, stating assumptions, and scoping an MVP. GLM’s call failed during reasoning on this prompt, so it returned no answer and received no score.
The same failure appeared in the previous DeepSeek Flash and Pro comparison. For underspecified product requests, I would make clarification an explicit system or prompt requirement instead of expecting these models to pause on their own.
Peer ranks are only a partial side signal
The harness also started anonymized peer ranking, but its process timed out after 29 of 74 planned calls. Candidate generation and rubric judging had already finished. The interruption does not affect the score table above.
The partial Borda totals put Gemini first and GLM second, which disagrees with the complete rubric scores:
| Model | Partial Borda | Avg rank | Times ranked |
|---|---|---|---|
openrouter:gemini-3.8-flash |
52 | 2.07 | 29 |
openrouter:glm-5.3-flash |
48 | 2.12 | 26 |
openrouter:qwen3.8-flash |
34 | 2.69 | 29 |
openrouter:deepseek-v4.1-flash |
28 | 2.89 | 28 |
This is an illustrative check, not a result to act on. Different models were ranked a different number of times, most ranking calls never ran, and the partial ordering conflicts with the completed judging stage. It needs a full rerun before it can support a comparison.
What I would choose
For this prompt mix, I would start with GLM-5.3-Flash when completed-response latency matters and Qwen3.8-Flash when predictable completion and recorded cost matter most. Both scored above 4.6 while their successful responses cost less and completed faster than Gemini 3.8-Flash’s.
GLM needs more output headroom or a conciseness instruction for large coding tasks. Qwen was the only candidate with no errors, no truncation, and the lowest recorded cost. DeepSeek’s single provider error deserves a repeat, not a reliability label.
Limits of the run
This was one batch with 19 prompts, one OpenRouter path, and one LLM judge. The models were close on score, and there is no multi-judge spread for this batch. Passing the repository’s calibration gate supports publishing the comparison, but it does not turn a small run into a general leaderboard.
The next useful tests are a second judge from outside the Z.ai family, repeated batches, and a complete peer-ranking run. I would also rerun DeepSeek’s failed prompt and either raise GLM’s output limit or ask it to be concise on code generation.
For now, the narrow result is enough: GLM-5.3-Flash led the completed rubric scores and had the lowest completed-response latency, while Qwen3.8-Flash delivered the lowest recorded cost. Gemini 3.8-Flash had the highest completed-response latency and recorded cost without a quality edge in this batch.