groq/openai/gpt-oss-120b is the default choice today: 21/21 passing at about $0.04 per 1k runs.
groq/openai/gpt-oss-120b is the only Pareto-frontier model, passing 21/21 tests at an average of $0.0000406 per run (about $0.0406 per 1k) and 629ms average latency. No regressions were detected. The most surprising finding is openrouter/qwen/qwen3.8-max-0902:batch, which had the lowest median latency at 86ms but failed all 6 of its tests, so its speed is not usable. Several providers, including deepinfra, openrouter (some models), and together (some models), have empty or sparse results.
Cost-vs-Quality Frontier
1 optimal models
groq
Only groq/openai/gpt-oss-120b sits on the cost-quality Pareto frontier, with 100% pass rate, $0.0000406 average cost per run, and 629ms average latency across 21 tests. No other model dominates it on both cost and quality in this snapshot.
Per-Task Recommendations
For tool_use and context, groq/openai/gpt-oss-120b is cheapest and fastest (about 280–305ms) at 100% pass rate. For speed/long generation, groq/openai/gpt-oss-120b also leads at 403 tokens/sec with 100% pass. For code and json, groq/openai/gpt-oss-120b passes at 659–683ms; for code generation specifically, the other passing options (e.g., openai/gpt-4o-mini at 2039ms) are slower.
Code
cheapest$0.042groqgpt-oss-120b
fastest659msgroqgpt-oss-120b
tok/s346groqgpt-oss-120b
JSON
cheapest$0.028groq
Surprising Findings
openrouter/qwen/qwen3.8-max-0902:batch has the lowest median latency at 86ms but failed all 6 tests, so the speed number reflects failed responses rather than usable output.
·groq/openai/gpt-oss-120b passed 21/21 tests with 629ms average latency, the only Pareto-frontier model.
·openrouter/qwen/qwen3.8-max-0902:batch failed all 6 tests despite an 86ms median latency, which is the fastest reading overall but not usable.
·nvidia/google/diffusiongemma-26b-a4b-it failed all 6 tests at about 20s each; nvidia/deepseek-ai/deepseek-v4.1-flash failed all 6 at about 60s each.
·nvidia/mistralai/mistral-large-4-1t-a52b showed 60s timeouts on speed and reasoning tests.
·gemini/gemini-3.1-pro-preview-customtools speed test failed at 20s; gemini/gemini-3.7-flash and gemini-3.8-flash passed all tests at 2.4–11s latency.
·deepinfra and together routes show no current results for many models; deepinfra shows no results at all.
Recommendation
Route all default traffic to groq/openai/gpt-oss-120b, which passes every test at $0.0406 per 1k with the highest throughput, and use it as the fallback path as well.