Loading benchmark data…
EyesInAI·Loading live benchmark data
Recent vs. prior changes in latency, cost, and reliability across all tested models — each model's latest results compared against its earlier ones.
| Provider | Model | Test | Runs | Latency | $ / 1k | Pass | Trend |
|---|---|---|---|---|---|---|---|
| GLM-5.2 | 🔧 Tool Use | 2 | 1003ms | $0.388 | 100% | ||
| GLM-5.2 | 🔍 Context Recall | 2 | 2186ms | $1.61 | 100% | ||
| GLM-5.2 | 🚀 Throughput | 2 | 5476ms | $1.54 | 100% | ||
| GLM-5.2 | 💻 Code Gen | 2 | 3863ms | $1.21 | 100% | ||
| GLM-5.2 | { } JSON Output | 2 | 2480ms | $1.11 | 100% | ||
| GLM-5.2 | 🧮 Reasoning | 2 | 7979ms | $1.49 | 100% | ||
| GLM-5.2 | ⚡ Ping | 2 | 1614ms | $0.066 | 100% | ||
| gpt-oss-120b | 🔧 Tool Use | 2 | 1169ms | $0.031 | 100% | ||
| gpt-oss-120b | 🔍 Context Recall | 2 | 954ms | $0.022 | 100% | ||
| gpt-oss-120b | 🚀 Throughput | 2 | 2841ms | $0.089 | 100% | ||
| gpt-oss-120b | 💻 Code Gen | 2 | 1888ms | $0.046 | 100% | ||
| gpt-oss-120b | { } JSON Output | 2 | 1003ms | $0.028 | 100% | ||
| gpt-oss-120b | 🧮 Reasoning | 2 | 877ms | $0.021 | 100% | ||
| gpt-oss-120b | ⚡ Ping | 2 | 716ms | <$0.01 | 100% | ||
| cogito-v2-1-671b | 🔧 Tool Use | 2 | — | — | 0% | ||
| cogito-v2-1-671b | 🔍 Context Recall | 2 | 1154ms | $0.315 | 100% | ||
| cogito-v2-1-671b | 🚀 Throughput | 2 | 5551ms | $0.698 | 100% | ||
| cogito-v2-1-671b | 💻 Code Gen | 2 | 1604ms | $0.216 | 100% | ||
| cogito-v2-1-671b | { } JSON Output | 2 | 766ms | $0.149 | 100% | ||
| cogito-v2-1-671b | 🧮 Reasoning | 2 | 1149ms | $0.105 | 100% | ||
| cogito-v2-1-671b | ⚡ Ping | 2 | 573ms | $0.046 | 100% | ||
| Qwen3.5-9B | 🔧 Tool Use | 2 | — | — | 0% | ||
| Qwen3.5-9B | 🔍 Context Recall | 2 | — | — | 0% | ||
| Qwen3.5-9B | 🚀 Throughput | 2 | 13954ms | $0.079 | 100% | ||
| Qwen3.5-9B | 💻 Code Gen | 2 | — | — | 0% | ||
| Qwen3.5-9B | { } JSON Output | 2 | — | — | 0% | ||
| Qwen3.5-9B | 🧮 Reasoning | 2 | — | — | 0% | ||
| Qwen3.5-9B | ⚡ Ping | 2 | 896ms | <$0.01 | 100% | ||
| MiniMax-M3 | 🔧 Tool Use | 2 | — | — | 0% | ||
| MiniMax-M3 | 🔍 Context Recall | 2 | — | — | 0% | ||
| MiniMax-M3 | 🚀 Throughput | 2 | 3155ms | $0.647 | 100% | ||
| MiniMax-M3 | 💻 Code Gen | 2 | 2259ms | $0.474 | 100% | ||
| MiniMax-M3 | { } JSON Output | 2 | 1231ms | $0.177 | 100% | ||
| MiniMax-M3 | 🧮 Reasoning | 2 | 1568ms | $0.131 | 100% | ||
| MiniMax-M3 | ⚡ Ping | 2 | 1366ms | $0.059 | 100% | ||
| MiniMax-M2.7 | 🔧 Tool Use | 2 | — | — | 0% | ||
| MiniMax-M2.7 | 🔍 Context Recall | 2 | — | — | 0% | ||
| MiniMax-M2.7 | 🚀 Throughput | 2 | — | — | 0% | ||
| MiniMax-M2.7 | 💻 Code Gen | 2 | — | — | 0% | ||
| MiniMax-M2.7 | { } JSON Output | 2 | — | — | 0% | ||
| MiniMax-M2.7 | 🧮 Reasoning | 2 | — | — | 0% | ||
| MiniMax-M2.7 | ⚡ Ping | 2 | — | — | 0% | ||
| glm-5v-turbo | 🔍 Context Recall | 2 | 1756ms | $1.64 | 100% | ||
| glm-5v-turbo | 🚀 Throughput | 2 | 1655ms | $2.04 | 100% | ||
| glm-5v-turbo | 💻 Code Gen | 2 | 1966ms | $1.26 | 100% | ||
| glm-5v-turbo | { } JSON Output | 2 | 2381ms | $0.578 | 100% | ||
| glm-5v-turbo | 🧮 Reasoning | 2 | 1710ms | $1.01 | 100% | ||
| glm-5v-turbo | ⚡ Ping | 2 | 1506ms | $0.078 | 100% | ||
| glm-4.6v | 🔍 Context Recall | 2 | — | — | 0% | ||
| glm-4.6v | 🚀 Throughput | 2 | 934ms | $0.461 | 100% | ||
| glm-4.6v | 💻 Code Gen | 2 | — | — | 0% | ||
| glm-4.6v | { } JSON Output | 2 | — | — | 0% | ||
| glm-4.6v | 🧮 Reasoning | 2 | 901ms | $0.099 | 100% | ||
| glm-4.6v | ⚡ Ping | 2 | 1166ms | $0.018 | 100% | ||
| grok-build-0.1 | 🔍 Context Recall | 2 | 709ms | $1.37 | 100% | ||
| grok-build-0.1 | 🚀 Throughput | 2 | 634ms | $3.44 | 100% | ||
| grok-build-0.1 | 💻 Code Gen | 2 | 759ms | $1.13 | 100% | ||
| grok-build-0.1 | { } JSON Output | 2 | 729ms | $1.83 | 100% | ||
| grok-build-0.1 | 🧮 Reasoning | 2 | 688ms | $1.45 | 100% | ||
| grok-build-0.1 | ⚡ Ping | 2 | 714ms | $0.457 | 100% | ||
| unslopnemo-12b | 🔍 Context Recall | 2 | 814ms | $0.105 | 100% | ||
| unslopnemo-12b | 🚀 Throughput | 2 | 497ms | $0.214 | 100% | ||
| unslopnemo-12b | 💻 Code Gen | 2 | 544ms | $0.052 | 100% | ||
| unslopnemo-12b | { } JSON Output | 2 | 571ms | $0.039 | 100% | ||
| unslopnemo-12b | 🧮 Reasoning | 2 | — | — | 0% | ||
| unslopnemo-12b | ⚡ Ping | 2 | 595ms | <$0.01 | 100% | ||
| Meta-Llama-3.3-70B-Instruct | 🔧 Tool Use | 2 | 1029ms | $0.161 | 100% | ||
| Meta-Llama-3.3-70B-Instruct | 🔍 Context Recall | 2 | 896ms | $0.187 | 100% | ||
| Meta-Llama-3.3-70B-Instruct | 🚀 Throughput | 2 | 2368ms | $0.639 | 100% | ||
| Meta-Llama-3.3-70B-Instruct | 💻 Code Gen | 2 | 1199ms | $0.275 | 100% | ||
| Meta-Llama-3.3-70B-Instruct | { } JSON Output | 2 | 1271ms | $0.086 | 100% | ||
| Meta-Llama-3.3-70B-Instruct | 🧮 Reasoning | 2 | 885ms | $0.074 | 100% | ||
| Meta-Llama-3.3-70B-Instruct | ⚡ Ping | 2 | 909ms | $0.026 | 100% | ||
| glm-5.2 | 🔍 Context Recall | 2 | 514ms | free | 100% | ||
| glm-5.2 | 🚀 Throughput | 2 | 1318ms | free | 100% | ||
| glm-5.2 | 💻 Code Gen | 2 | 891ms | free | 100% | ||
| glm-5.2 | { } JSON Output | 2 | 381ms | free | 100% | ||
| glm-5.2 | 🧮 Reasoning | 2 | 363ms | free | 100% | ||
| glm-5.2 | ⚡ Ping | 2 | 1544ms | free | 100% | ||
| glm-4.6 | 🔍 Context Recall | 2 | 2563ms | $1.34 | 100% | ||
| glm-4.6 | 🚀 Throughput | 2 | 670ms | $1.76 | 100% | ||
| glm-4.6 | 💻 Code Gen | 2 | — | — | 0% | ||
| glm-4.6 | { } JSON Output | 2 | 1654ms | $0.991 | 100% | ||
| glm-4.6 | 🧮 Reasoning | 2 | 8300ms | $1.82 | 100% | ||
| glm-4.6 | ⚡ Ping | 2 | 1956ms | $0.216 | 100% | ||
| grok-4.3 | 🔍 Context Recall | 2 | 621ms | $1.26 | 100% | ||
| grok-4.3 | 🚀 Throughput | 2 | 977ms | $2.50 | 100% | ||
| grok-4.3 | 💻 Code Gen | 2 | 663ms | $1.05 | 100% | ||
| grok-4.3 | { } JSON Output | 2 | 1093ms | $0.912 | 100% | ||
| grok-4.3 | 🧮 Reasoning | 2 | 710ms | $1.17 | 100% | ||
| grok-4.3 | ⚡ Ping | 2 | 707ms | $1.02 | 100% | ||
| skyfall-36b-v2 | 🔍 Context Recall | 2 | 536ms | $0.235 | 100% | ||
| skyfall-36b-v2 | 🚀 Throughput | 2 | 539ms | $0.514 | 100% | ||
| skyfall-36b-v2 | 💻 Code Gen | 2 | — | — | 0% | ||
| skyfall-36b-v2 | { } JSON Output | 2 | 489ms | $0.156 | 100% | ||
| skyfall-36b-v2 | 🧮 Reasoning | 2 | 776ms | $0.177 | 100% | ||
| skyfall-36b-v2 | ⚡ Ping | 2 | 494ms | $0.103 | 100% | ||
| step-3.5-flash | 🔍 Context Recall | 2 | — | — | 0% | ||
| step-3.5-flash | 🚀 Throughput | 2 | 2673ms | $0.154 | 100% | ||
| step-3.5-flash | 💻 Code Gen | 2 | — | — | 0% | ||
| step-3.5-flash | { } JSON Output | 2 | — | — | 0% | ||
| step-3.5-flash | 🧮 Reasoning | 2 | — | — | 0% | ||
| step-3.5-flash | ⚡ Ping | 2 | 10123ms | — | 100% | ||
| relace-apply-3 | 🔍 Context Recall | 2 | — | — | 0% | ||
| relace-apply-3 | 🚀 Throughput | 2 | — | — | 0% | ||
| relace-apply-3 | 💻 Code Gen | 2 | — | — | 0% | ||
| relace-apply-3 | { } JSON Output | 2 | — | — | 0% | ||
| relace-apply-3 | 🧮 Reasoning | 2 | — | — | 0% | ||
| relace-apply-3 | ⚡ Ping | 2 | — | — | 0% | ||
| qwen3.6-flash | 🔍 Context Recall | 2 | 493ms | $1.10 | 100% | ||
| qwen3.6-flash | 🚀 Throughput | 2 | 494ms | $2.88 | 100% | ||
| qwen3.6-flash | 💻 Code Gen | 2 | 509ms | $1.48 | 100% | ||
| qwen3.6-flash | { } JSON Output | 2 | 460ms | $1.29 | 100% | ||
| qwen3.6-flash | 🧮 Reasoning | 2 | 479ms | $1.06 | 100% | ||
| qwen3.6-flash | ⚡ Ping | 2 | 491ms | $0.133 | 100% | ||
| qwen3.5-397b-a17b | 🔍 Context Recall | 2 | — | — | 0% | ||
| qwen3.5-397b-a17b | 🚀 Throughput | 2 | 1692ms | free | 100% | ||
| qwen3.5-397b-a17b | 💻 Code Gen | 2 | 1107ms | free | 100% | ||
| qwen3.5-397b-a17b | { } JSON Output | 2 | 971ms | free | 100% | ||
| qwen3.5-397b-a17b | 🧮 Reasoning | 2 | — | — | 0% | ||
| qwen3.5-397b-a17b | ⚡ Ping | 2 | 994ms | free | 100% | ||
| qwen3-vl-30b-a3b-thinking | 🔍 Context Recall | 2 | 1634ms | $1.27 | 100% | ||
| qwen3-vl-30b-a3b-thinking | 🚀 Throughput | 2 | 301ms | $1.84 | 100% | ||
| qwen3-vl-30b-a3b-thinking | 💻 Code Gen | 2 | 380ms | $8.99 | 100% | ||
| qwen3-vl-30b-a3b-thinking | { } JSON Output | 2 | 351ms | $0.297 | 100% | ||
| qwen3-vl-30b-a3b-thinking | 🧮 Reasoning | 2 | 312ms | $0.451 | 100% | ||
| qwen3-vl-30b-a3b-thinking | ⚡ Ping | 2 | 3237ms | $0.361 | 100% | ||
| qwen3-max-thinking | 🔍 Context Recall | 2 | 1600ms | $0.276 | 100% | ||
| qwen3-max-thinking | 🚀 Throughput | 2 | 899ms | $1.98 | 100% | ||
| qwen3-max-thinking | 💻 Code Gen | 2 | 1014ms | $0.839 | 100% | ||
| qwen3-max-thinking | { } JSON Output | 2 | 814ms | $0.160 | 100% | ||
| qwen3-max-thinking | 🧮 Reasoning | 2 | 905ms | $0.314 | 100% | ||
| qwen3-max-thinking | ⚡ Ping | 2 | 1535ms | $0.016 | 100% | ||
| qwen3-coder | 🔍 Context Recall | 2 | 754ms | $0.119 | 100% | ||
| qwen3-coder | 🚀 Throughput | 2 | 383ms | $0.908 | 100% | ||
| qwen3-coder | 💻 Code Gen | 2 | 729ms | $0.347 | 100% | ||
| qwen3-coder | { } JSON Output | 2 | 371ms | $0.076 | 100% | ||
| qwen3-coder | 🧮 Reasoning | 2 | 4907ms | $0.095 | 100% | ||
| qwen3-coder | ⚡ Ping | 2 | 2218ms | <$0.01 | 100% | ||
| qwen3-235b-a22b-2507 | 🔍 Context Recall | 2 | 828ms | $0.022 | 100% | ||
| qwen3-235b-a22b-2507 | 🚀 Throughput | 2 | 1544ms | $0.053 | 100% | ||
| qwen3-235b-a22b-2507 | 💻 Code Gen | 2 | 510ms | $0.023 | 100% | ||
| qwen3-235b-a22b-2507 | { } JSON Output | 2 | 637ms | <$0.01 | 100% | ||
| qwen3-235b-a22b-2507 | 🧮 Reasoning | 2 | 1120ms | <$0.01 | 100% | ||
| qwen3-235b-a22b-2507 | ⚡ Ping | 2 | 2676ms | <$0.01 | 100% | ||
| qwen-2.5-coder-32b-instruct | 🔍 Context Recall | 2 | 363ms | $0.177 | 100% | ||
| qwen-2.5-coder-32b-instruct | 🚀 Throughput | 2 | 370ms | $0.539 | 100% | ||
| qwen-2.5-coder-32b-instruct | 💻 Code Gen | 2 | — | — | 0% | ||
| qwen-2.5-coder-32b-instruct | { } JSON Output | 2 | 368ms | $0.075 | 100% | ||
| qwen-2.5-coder-32b-instruct | 🧮 Reasoning | 2 | — | — | 0% | ||
| qwen-2.5-coder-32b-instruct | ⚡ Ping | 2 | 854ms | $0.026 | 100% | ||
| laguna-m.1:free | 🔍 Context Recall | 2 | — | — | 0% | ||
| laguna-m.1:free | 🚀 Throughput | 2 | 578ms | free | 100% | ||
| laguna-m.1:free | 💻 Code Gen | 2 | — | — | 0% | ||
| laguna-m.1:free | { } JSON Output | 2 | — | — | 0% | ||
| laguna-m.1:free | 🧮 Reasoning | 2 | — | — | 0% | ||
| laguna-m.1:free | ⚡ Ping | 2 | 472ms | free | 100% | ||
| perceptron-mk1 | 🔍 Context Recall | 2 | 773ms | $0.101 | 100% | ||
| perceptron-mk1 | 🚀 Throughput | 2 | 838ms | $0.755 | 100% | ||
| perceptron-mk1 | 💻 Code Gen | 2 | 1220ms | $0.174 | 100% | ||
| perceptron-mk1 | { } JSON Output | 2 | 826ms | $0.076 | 100% | ||
| perceptron-mk1 | 🧮 Reasoning | 2 | 786ms | $0.065 | 100% | ||
| perceptron-mk1 | ⚡ Ping | 2 | 866ms | <$0.01 | 100% | ||
| gpt-oss-20b:free | 🔍 Context Recall | 2 | — | — | 0% | ||
| gpt-oss-20b:free | 🚀 Throughput | 2 | 2096ms | free | 100% | ||
| gpt-oss-20b:free | 💻 Code Gen | 2 | 1973ms | free | 100% | ||
| gpt-oss-20b:free | { } JSON Output | 2 | — | — | 0% | ||
| gpt-oss-20b:free | 🧮 Reasoning | 2 | 757ms | free | 100% | ||
| gpt-oss-20b:free | ⚡ Ping | 2 | 3538ms | free | 100% | ||
| nemotron-nano-9b-v2:free | 🔍 Context Recall | 2 | — | — | 0% | ||
| nemotron-nano-9b-v2:free | 🚀 Throughput | 2 | 1140ms | free | 100% | ||
| nemotron-nano-9b-v2:free | 💻 Code Gen | 2 | — | — | 0% | ||
| nemotron-nano-9b-v2:free | { } JSON Output | 2 | — | — | 0% | ||
| nemotron-nano-9b-v2:free | 🧮 Reasoning | 2 | — | — | 0% | ||
| nemotron-nano-9b-v2:free | ⚡ Ping | 2 | 903ms | free | 100% | ||
| nemotron-3-nano-omni-30b-a3b-reasoning:free | 🔍 Context Recall | 2 | 332ms | free | 100% | ||
| nemotron-3-nano-omni-30b-a3b-reasoning:free | 🚀 Throughput | 2 | 350ms | free | 100% | ||
| nemotron-3-nano-omni-30b-a3b-reasoning:free | 💻 Code Gen | 2 | 337ms | free | 100% | ||
| nemotron-3-nano-omni-30b-a3b-reasoning:free | { } JSON Output | 2 | — | — | 0% | ||
| nemotron-3-nano-omni-30b-a3b-reasoning:free | 🧮 Reasoning | 2 | 345ms | free | 100% | ||
| nemotron-3-nano-omni-30b-a3b-reasoning:free | ⚡ Ping | 2 | 352ms | free | 100% | ||
| hermes-3-llama-3.1-405b:free | 🔍 Context Recall | 2 | — | — | 0% | ||
| hermes-3-llama-3.1-405b:free | 🚀 Throughput | 2 | — | — | 0% | ||
| hermes-3-llama-3.1-405b:free | 💻 Code Gen | 2 | — | — | 0% | ||
| hermes-3-llama-3.1-405b:free | { } JSON Output | 2 | — | — | 0% | ||
| hermes-3-llama-3.1-405b:free | 🧮 Reasoning | 2 | — | — | 0% | ||
| hermes-3-llama-3.1-405b:free | ⚡ Ping | 2 | — | — | 0% | ||
| kimi-k2.5 | 🔍 Context Recall | 2 | 1010ms | $0.697 | 100% | ||
| kimi-k2.5 | 🚀 Throughput | 2 | 385ms | $1.02 | 100% | ||
| kimi-k2.5 | 💻 Code Gen | 2 | 889ms | $2.35 | 100% | ||
| kimi-k2.5 | { } JSON Output | 2 | 1227ms | $0.803 | 100% | ||
| kimi-k2.5 | 🧮 Reasoning | 2 | 1075ms | $1.87 | 100% | ||
| kimi-k2.5 | ⚡ Ping | 2 | 744ms | $0.038 | 100% | ||
| mistral-small-3.1-24b-instruct | 🔍 Context Recall | 2 | — | — | 0% | ||
| mistral-small-3.1-24b-instruct | 🚀 Throughput | 2 | 556ms | — | 100% | ||
| mistral-small-3.1-24b-instruct | 💻 Code Gen | 2 | — | — | 0% | ||
| mistral-small-3.1-24b-instruct | { } JSON Output | 2 | — | — | 0% | ||
| mistral-small-3.1-24b-instruct | 🧮 Reasoning | 2 | — | — | 0% | ||
| mistral-small-3.1-24b-instruct | ⚡ Ping | 2 | 363ms | <$0.01 | 100% | ||
| mistral-medium-3 | 🔍 Context Recall | 2 | 544ms | $0.110 | 100% | ||
| mistral-medium-3 | 🚀 Throughput | 2 | 575ms | $1.01 | 100% | ||
| mistral-medium-3 | 💻 Code Gen | 2 | 528ms | $0.238 | 100% | ||
| mistral-medium-3 | { } JSON Output | 2 | 520ms | $0.095 | 100% | ||
| mistral-medium-3 | 🧮 Reasoning | 2 | 507ms | $0.141 | 100% | ||
| mistral-medium-3 | ⚡ Ping | 2 | 614ms | <$0.01 | 100% | ||
| devstral-2512 | 🔍 Context Recall | 2 | 509ms | $0.110 | 100% | ||
| devstral-2512 | 🚀 Throughput | 2 | 650ms | $1.01 | 100% | ||
| devstral-2512 | 💻 Code Gen | 2 | 508ms | $0.238 | 100% | ||
| devstral-2512 | { } JSON Output | 2 | 546ms | $0.105 | 100% | ||
| devstral-2512 | 🧮 Reasoning | 2 | 506ms | $0.145 | 100% | ||
| devstral-2512 | ⚡ Ping | 2 | 580ms | <$0.01 | 100% | ||
| minimax-m2 | 🔍 Context Recall | 2 | — | — | 0% | ||
| minimax-m2 | 🚀 Throughput | 2 | 1334ms | $0.528 | 100% | ||
| minimax-m2 | 💻 Code Gen | 2 | — | — | 0% | ||
| minimax-m2 | { } JSON Output | 2 | — | — | 0% | ||
| minimax-m2 | 🧮 Reasoning | 2 | — | — | 0% | ||
| minimax-m2 | ⚡ Ping | 2 | 1846ms | $0.028 | 100% | ||
| llama-4-maverick | 🔍 Context Recall | 2 | 1591ms | $0.058 | 100% | ||
| llama-4-maverick | 🚀 Throughput | 2 | 358ms | $0.306 | 100% | ||
| llama-4-maverick | 💻 Code Gen | 2 | 514ms | $0.070 | 100% | ||
| llama-4-maverick | { } JSON Output | 2 | 1801ms | $0.027 | 100% | ||
| llama-4-maverick | 🧮 Reasoning | 2 | 779ms | $0.025 | 100% | ||
| llama-4-maverick | ⚡ Ping | 2 | 341ms | <$0.01 | 100% | ||
| llama-3.1-8b-instruct | 🔍 Context Recall | 2 | 382ms | <$0.01 | 100% | ||
| llama-3.1-8b-instruct | 🚀 Throughput | 2 | 792ms | $0.016 | 100% | ||
| llama-3.1-8b-instruct | 💻 Code Gen | 2 | 391ms | <$0.01 | 100% | ||
| llama-3.1-8b-instruct | { } JSON Output | 2 | 405ms | <$0.01 | 100% | ||
| llama-3.1-8b-instruct | 🧮 Reasoning | 2 | 372ms | <$0.01 | 100% | ||
| llama-3.1-8b-instruct | ⚡ Ping | 2 | 782ms | <$0.01 | 100% | ||
| kat-coder-pro-v2 | 🔍 Context Recall | 2 | 538ms | $0.084 | 100% | ||
| kat-coder-pro-v2 | 🚀 Throughput | 2 | 1066ms | $0.611 | 100% | ||
| kat-coder-pro-v2 | 💻 Code Gen | 2 | 505ms | $0.222 | 100% | ||
| kat-coder-pro-v2 | { } JSON Output | 2 | 552ms | $0.043 | 100% | ||
| kat-coder-pro-v2 | 🧮 Reasoning | 2 | 1048ms | $0.065 | 100% | ||
| kat-coder-pro-v2 | ⚡ Ping | 2 | 1143ms | <$0.01 | 100% | ||
| granite-4.1-8b | 🔍 Context Recall | 2 | 380ms | $0.014 | 100% | ||
| granite-4.1-8b | 🚀 Throughput | 2 | 307ms | $0.052 | 100% | ||
| granite-4.1-8b | 💻 Code Gen | 2 | 327ms | $0.013 | 100% | ||
| granite-4.1-8b | { } JSON Output | 2 | 429ms | <$0.01 | 100% | ||
| granite-4.1-8b | 🧮 Reasoning | 2 | 376ms | <$0.01 | 100% | ||
| granite-4.1-8b | ⚡ Ping | 2 | 337ms | <$0.01 | 100% | ||
| gemma-4-26b-a4b-it | 🔍 Context Recall | 2 | 482ms | $0.017 | 100% | ||
| gemma-4-26b-a4b-it | 🚀 Throughput | 2 | 1687ms | $0.168 | 100% | ||
| gemma-4-26b-a4b-it | 💻 Code Gen | 2 | 1281ms | $0.047 | 100% | ||
| gemma-4-26b-a4b-it | { } JSON Output | 2 | 642ms | $0.014 | 100% | ||
| gemma-4-26b-a4b-it | 🧮 Reasoning | 2 | 392ms | $0.017 | 100% | ||
| gemma-4-26b-a4b-it | ⚡ Ping | 2 | 433ms | <$0.01 | 100% | ||
| deepseek-v4-flash | 🔍 Context Recall | 2 | 1386ms | $0.026 | 100% | ||
| deepseek-v4-flash | 🚀 Throughput | 2 | 1008ms | $0.093 | 100% | ||
| deepseek-v4-flash | 💻 Code Gen | 2 | 1051ms | $0.090 | 100% | ||
| deepseek-v4-flash | { } JSON Output | 2 | 969ms | <$0.01 | 100% | ||
| deepseek-v4-flash | 🧮 Reasoning | 2 | 993ms | $0.017 | 100% | ||
| deepseek-v4-flash | ⚡ Ping | 2 | 1363ms | <$0.01 | 100% | ||
| deepseek-chat-v3.1 | 🔍 Context Recall | 2 | 1382ms | $0.065 | 100% | ||
| deepseek-chat-v3.1 | 🚀 Throughput | 2 | 1148ms | $0.403 | 100% | ||
| deepseek-chat-v3.1 | 💻 Code Gen | 2 | 739ms | $0.094 | 100% | ||
| deepseek-chat-v3.1 | { } JSON Output | 2 | 1128ms | $0.036 | 100% | ||
| deepseek-chat-v3.1 | 🧮 Reasoning | 2 | 4430ms | $0.029 | 100% | ||
| deepseek-chat-v3.1 | ⚡ Ping | 2 | 1960ms | <$0.01 | 100% | ||
| command-r-08-2024 | 🔍 Context Recall | 2 | 789ms | $0.059 | 100% | ||
| command-r-08-2024 | 🚀 Throughput | 2 | 763ms | $0.304 | 100% | ||
| command-r-08-2024 | 💻 Code Gen | 2 | 755ms | $0.066 | 100% | ||
| command-r-08-2024 | { } JSON Output | 2 | 787ms | $0.030 | 100% | ||
| command-r-08-2024 | 🧮 Reasoning | 2 | 759ms | $0.031 | 100% | ||
| command-r-08-2024 | ⚡ Ping | 2 | 818ms | <$0.01 | 100% | ||
| seed-1.6 | 🔍 Context Recall | 2 | 1528ms | $0.308 | 100% | ||
| seed-1.6 | 🚀 Throughput | 2 | 536ms | $4.72 | 100% | ||
| seed-1.6 | 💻 Code Gen | 2 | 1072ms | $1.82 | 100% | ||
| seed-1.6 | { } JSON Output | 2 | 569ms | $0.249 | 100% | ||
| seed-1.6 | 🧮 Reasoning | 2 | 501ms | $0.510 | 100% | ||
| seed-1.6 | ⚡ Ping | 2 | 833ms | $0.106 | 100% | ||
| nova-micro-v1 | 🔍 Context Recall | 2 | 1193ms | $0.020 | 100% | ||
| nova-micro-v1 | 🚀 Throughput | 2 | 986ms | $0.071 | 100% | ||
| nova-micro-v1 | 💻 Code Gen | 2 | 861ms | $0.018 | 100% | ||
| nova-micro-v1 | { } JSON Output | 2 | 655ms | <$0.01 | 100% | ||
| nova-micro-v1 | 🧮 Reasoning | 2 | 1014ms | $0.013 | 100% | ||
| nova-micro-v1 | ⚡ Ping | 2 | 1057ms | <$0.01 | 100% | ||
| aion-1.0 | 🔍 Context Recall | 2 | — | — | 0% | ||
| aion-1.0 | 🚀 Throughput | 2 | — | — | 0% | ||
| aion-1.0 | 💻 Code Gen | 2 | — | — | 0% | ||
| aion-1.0 | { } JSON Output | 2 | — | — | 0% | ||
| aion-1.0 | 🧮 Reasoning | 2 | — | — | 0% | ||
| aion-1.0 | ⚡ Ping | 2 | — | — | 0% | ||
| o3 | 🔧 Tool Use | 2 | 952ms | $1.71 | 100% | ||
| o3 | 🔍 Context Recall | 2 | 1331ms | $3.09 | 100% | ||
| o3 | 🚀 Throughput | 2 | 3563ms | $20.36 | 100% | ||
| o3 | 💻 Code Gen | 2 | 3123ms | $20.79 | 100% | ||
| o3 | { } JSON Output | 2 | 1470ms | $4.68 | 100% | ||
| o3 | 🧮 Reasoning | 2 | 2731ms | $15.45 | 100% | ||
| o3 | ⚡ Ping | 2 | 1375ms | $0.770 | 100% | ||
| gpt-4o-2024-11-20 | 🔧 Tool Use | 2 | 541ms | $0.338 | 100% | ||
| gpt-4o-2024-11-20 | 🔍 Context Recall | 2 | 1083ms | $0.885 | 100% | ||
| gpt-4o-2024-11-20 | 🚀 Throughput | 2 | 2522ms | $5.09 | 100% | ||
| gpt-4o-2024-11-20 | 💻 Code Gen | 2 | 964ms | $1.58 | 100% | ||
| gpt-4o-2024-11-20 | { } JSON Output | 2 | 890ms | $0.492 | 100% | ||
| gpt-4o-2024-11-20 | 🧮 Reasoning | 2 | 923ms | $0.665 | 100% | ||
| gpt-4o-2024-11-20 | ⚡ Ping | 2 | 573ms | $0.045 | 100% | ||
| gpt-4.1-mini | 🔧 Tool Use | 2 | 849ms | $0.062 | 100% | ||
| gpt-4.1-mini | 🔍 Context Recall | 2 | 939ms | $0.124 | 100% | ||
| gpt-4.1-mini | 🚀 Throughput | 2 | 4980ms | $0.815 | 100% | ||
| gpt-4.1-mini | 💻 Code Gen | 2 | 1973ms | $0.179 | 100% | ||
| gpt-4.1-mini | { } JSON Output | 2 | 819ms | $0.058 | 100% | ||
| gpt-4.1-mini | 🧮 Reasoning | 2 | 939ms | $0.060 | 100% | ||
| gpt-4.1-mini | ⚡ Ping | 2 | 641ms | <$0.01 | 100% | ||
| gpt-3.5-turbo-16k | 🔧 Tool Use | 2 | 1193ms | — | 100% | ||
| gpt-3.5-turbo-16k | 🔍 Context Recall | 2 | 1348ms | — | 100% | ||
| gpt-3.5-turbo-16k | 🚀 Throughput | 2 | 4136ms | — | 100% | ||
| gpt-3.5-turbo-16k | 💻 Code Gen | 2 | 2435ms | — | 100% | ||
| gpt-3.5-turbo-16k | { } JSON Output | 2 | 2315ms | — | 100% | ||
| gpt-3.5-turbo-16k | 🧮 Reasoning | 2 | 2813ms | — | 100% | ||
| gpt-3.5-turbo-16k | ⚡ Ping | 2 | 3306ms | — | 100% | ||
| step-3.7-flash | 🔍 Context Recall | 2 | — | — | 0% | ||
| step-3.7-flash | 🚀 Throughput | 2 | 2904ms | free | 100% | ||
| step-3.7-flash | 💻 Code Gen | 2 | — | — | 0% | ||
| step-3.7-flash | { } JSON Output | 2 | — | — | 0% | ||
| step-3.7-flash | 🧮 Reasoning | 2 | 1220ms | free | 100% | ||
| step-3.7-flash | ⚡ Ping | 2 | 495ms | free | 100% | ||
| gpt-oss-120b | 🔍 Context Recall | 2 | — | — | 0% | ||
| gpt-oss-120b | 🚀 Throughput | 2 | — | — | 0% | ||
| gpt-oss-120b | 💻 Code Gen | 2 | — | — | 0% | ||
| gpt-oss-120b | { } JSON Output | 2 | — | — | 0% | ||
| gpt-oss-120b | 🧮 Reasoning | 2 | — | — | 0% | ||
| gpt-oss-120b | ⚡ Ping | 2 | — | — | 0% | ||
| nemotron-3-super-120b-a12b | 🔍 Context Recall | 2 | 1080ms | free | 100% | ||
| nemotron-3-super-120b-a12b | 🚀 Throughput | 2 | — | — | 0% | ||
| nemotron-3-super-120b-a12b | 💻 Code Gen | 2 | 2118ms | free | 100% | ||
| nemotron-3-super-120b-a12b | { } JSON Output | 2 | — | — | 0% | ||
| nemotron-3-super-120b-a12b | 🧮 Reasoning | 2 | 1093ms | free | 100% | ||
| nemotron-3-super-120b-a12b | ⚡ Ping | 2 | 649ms | free | 100% | ||
| llama-3.1-nemotron-nano-8b-v1 | 🔍 Context Recall | 2 | — | — | 0% | ||
| llama-3.1-nemotron-nano-8b-v1 | 🚀 Throughput | 2 | — | — | 0% | ||
| llama-3.1-nemotron-nano-8b-v1 | 💻 Code Gen | 2 | — | — | 0% | ||
| llama-3.1-nemotron-nano-8b-v1 | { } JSON Output | 2 | — | — | 0% | ||
| llama-3.1-nemotron-nano-8b-v1 | 🧮 Reasoning | 2 | 3640ms | free | 100% | ||
| llama-3.1-nemotron-nano-8b-v1 | ⚡ Ping | 2 | — | — | 0% | ||
| mistral-medium-3.5-128b | 🔍 Context Recall | 2 | 792ms | free | 100% | ||
| mistral-medium-3.5-128b | 🚀 Throughput | 2 | 12018ms | free | 100% | ||
| mistral-medium-3.5-128b | 💻 Code Gen | 2 | 2136ms | free | 100% | ||
| mistral-medium-3.5-128b | { } JSON Output | 2 | — | — | 0% | ||
| mistral-medium-3.5-128b | 🧮 Reasoning | 2 | 15070ms | free | 100% | ||
| mistral-medium-3.5-128b | ⚡ Ping | 2 | 541ms | free | 100% | ||
| llama-3.3-70b-instruct | 🔍 Context Recall | 2 | — | — | 0% | ||
| llama-3.3-70b-instruct | 🚀 Throughput | 2 | — | — | 0% | ||
| llama-3.3-70b-instruct | 💻 Code Gen | 2 | — | — | 0% | ||
| llama-3.3-70b-instruct | { } JSON Output | 2 | — | — | 0% | ||
| llama-3.3-70b-instruct | 🧮 Reasoning | 2 | — | — | 0% | ||
| llama-3.3-70b-instruct | ⚡ Ping | 2 | — | — | 0% | ||
| gemma-3n-e2b-it | 🔍 Context Recall | 2 | — | — | 0% | ||
| gemma-3n-e2b-it | 🚀 Throughput | 2 | — | — | 0% | ||
| gemma-3n-e2b-it | 💻 Code Gen | 2 | — | — | 0% | ||
| gemma-3n-e2b-it | { } JSON Output | 2 | — | — | 0% | ||
| gemma-3n-e2b-it | 🧮 Reasoning | 2 | — | — | 0% | ||
| gemma-3n-e2b-it | ⚡ Ping | 2 | — | — | 0% | ||
| dracarys-llama-3.1-70b-instruct | 🔍 Context Recall | 2 | 8821ms | free | 100% | ||
| dracarys-llama-3.1-70b-instruct | 🚀 Throughput | 2 | — | — | 0% | ||
| dracarys-llama-3.1-70b-instruct | 💻 Code Gen | 2 | — | — | 0% | ||
| dracarys-llama-3.1-70b-instruct | { } JSON Output | 2 | 7253ms | free | 100% | ||
| dracarys-llama-3.1-70b-instruct | 🧮 Reasoning | 2 | 4333ms | free | 100% | ||
| dracarys-llama-3.1-70b-instruct | ⚡ Ping | 2 | 3845ms | free | 100% | ||
| llama-3.1-8b-instant | 🔧 Tool Use | 2 | 260ms | $0.014 | 100% | ||
| llama-3.1-8b-instant | 🔍 Context Recall | 2 | 265ms | $0.014 | 100% | ||
| llama-3.1-8b-instant | 🚀 Throughput | 2 | 969ms | $0.043 | 100% | ||
| llama-3.1-8b-instant | 💻 Code Gen | 2 | 334ms | $0.011 | 100% | ||
| llama-3.1-8b-instant | { } JSON Output | 2 | 252ms | <$0.01 | 100% | ||
| llama-3.1-8b-instant | 🧮 Reasoning | 2 | 331ms | <$0.01 | 100% | ||
| llama-3.1-8b-instant | ⚡ Ping | 2 | 243ms | <$0.01 | 100% | ||
| gemini-omni-flash-preview | 🔧 Tool Use | 2 | — | — | 0% | ||
| gemini-omni-flash-preview | 🔍 Context Recall | 2 | — | — | 0% | ||
| gemini-omni-flash-preview | 🚀 Throughput | 2 | — | — | 0% | ||
| gemini-omni-flash-preview | 💻 Code Gen | 2 | — | — | 0% | ||
| gemini-omni-flash-preview | { } JSON Output | 2 | — | — | 0% | ||
| gemini-omni-flash-preview | 🧮 Reasoning | 2 | — | — | 0% | ||
| gemini-omni-flash-preview | ⚡ Ping | 2 | — | — | 0% | ||
| gemini-3.1-flash-lite-image | 🔧 Tool Use | 2 | 794ms | — | 100% | ||
| gemini-3.1-flash-lite-image | 🔍 Context Recall | 2 | 821ms | — | 100% | ||
| gemini-3.1-flash-lite-image | 🚀 Throughput | 2 | 2612ms | — | 100% | ||
| gemini-3.1-flash-lite-image | 💻 Code Gen | 2 | 962ms | — | 100% | ||
| gemini-3.1-flash-lite-image | { } JSON Output | 2 | 853ms | — | 100% | ||
| gemini-3.1-flash-lite-image | 🧮 Reasoning | 2 | 933ms | — | 100% | ||
| gemini-3.1-flash-lite-image | ⚡ Ping | 2 | 650ms | — | 100% | ||
| gemini-2.5-pro | 🔧 Tool Use | 2 | 3845ms | $0.269 | 100% | ||
| gemini-2.5-pro | 🔍 Context Recall | 2 | 4673ms | $0.711 | 100% | ||
| gemini-2.5-pro | 🚀 Throughput | 2 | — | — | 0% | ||
| gemini-2.5-pro | 💻 Code Gen | 2 | 12618ms | $3.15 | 100% | ||
| gemini-2.5-pro | { } JSON Output | 2 | 6247ms | $0.515 | 100% | ||
| gemini-2.5-pro | 🧮 Reasoning | 2 | 6675ms | $0.491 | 100% | ||
| gemini-2.5-pro | ⚡ Ping | 2 | 2560ms | $0.019 | 100% | ||
| gpt-oss-120b | 🔧 Tool Use | 2 | 845ms | $0.052 | 100% | ||
| gpt-oss-120b | 🔍 Context Recall | 2 | 1262ms | $0.063 | 100% | ||
| gpt-oss-120b | 🚀 Throughput | 2 | 3218ms | $0.315 | 100% | ||
| gpt-oss-120b | 💻 Code Gen | 2 | 1746ms | $0.202 | 100% | ||
| gpt-oss-120b | { } JSON Output | 2 | 1019ms | $0.090 | 100% | ||
| gpt-oss-120b | 🧮 Reasoning | 2 | — | — | 0% | ||
| gpt-oss-120b | ⚡ Ping | 2 | 858ms | $0.021 | 100% | ||
| GLM-4.7-Flash | 🔧 Tool Use | 2 | 2407ms | $0.050 | 100% | ||
| GLM-4.7-Flash | 🔍 Context Recall | 2 | 4592ms | $0.135 | 100% | ||
| GLM-4.7-Flash | 🚀 Throughput | 2 | 8106ms | $0.202 | 100% | ||
| GLM-4.7-Flash | 💻 Code Gen | 2 | 16714ms | $0.269 | 100% | ||
| GLM-4.7-Flash | { } JSON Output | 2 | 6630ms | $0.150 | 100% | ||
| GLM-4.7-Flash | 🧮 Reasoning | 2 | 6683ms | $0.165 | 100% | ||
| GLM-4.7-Flash | ⚡ Ping | 2 | 2658ms | <$0.01 | 100% | ||
| Nemotron-Content-Safety-3.5 | 🔧 Tool Use | 2 | — | — | 0% | ||
| Nemotron-Content-Safety-3.5 | 🔍 Context Recall | 2 | — | — | 0% | ||
| Nemotron-Content-Safety-3.5 | 🚀 Throughput | 2 | 1297ms | $0.100 | 100% | ||
| Nemotron-Content-Safety-3.5 | 💻 Code Gen | 2 | — | — | 0% | ||
| Nemotron-Content-Safety-3.5 | { } JSON Output | 2 | — | — | 0% | ||
| Nemotron-Content-Safety-3.5 | 🧮 Reasoning | 2 | — | — | 0% | ||
| Nemotron-Content-Safety-3.5 | ⚡ Ping | 2 | 756ms | $0.096 | 100% | ||
| Kimi-K2.6 | 🔧 Tool Use | 2 | 2593ms | $0.423 | 100% | ||
| Kimi-K2.6 | 🔍 Context Recall | 2 | 4039ms | $1.30 | 100% | ||
| Kimi-K2.6 | 🚀 Throughput | 2 | 6206ms | $1.78 | 100% | ||
| Kimi-K2.6 | 💻 Code Gen | 2 | 9156ms | $2.70 | 100% | ||
| Kimi-K2.6 | { } JSON Output | 2 | 3307ms | $0.785 | 100% | ||
| Kimi-K2.6 | 🧮 Reasoning | 2 | 17326ms | $6.53 | 100% | ||
| Kimi-K2.6 | ⚡ Ping | 2 | 1160ms | $0.067 | 100% | ||
| Meta-Llama-3.1-8B-Instruct | 🔧 Tool Use | 2 | 9573ms | <$0.01 | 100% | ||
| Meta-Llama-3.1-8B-Instruct | 🔍 Context Recall | 2 | 4452ms | <$0.01 | 100% | ||
| Meta-Llama-3.1-8B-Instruct | 🚀 Throughput | 2 | — | — | 0% | ||
| Meta-Llama-3.1-8B-Instruct | 💻 Code Gen | 2 | 19786ms | <$0.01 | 100% | ||
| Meta-Llama-3.1-8B-Instruct | { } JSON Output | 2 | 3598ms | <$0.01 | 100% | ||
| Meta-Llama-3.1-8B-Instruct | 🧮 Reasoning | 2 | 5787ms | <$0.01 | 100% | ||
| Meta-Llama-3.1-8B-Instruct | ⚡ Ping | 2 | 2199ms | <$0.01 | 100% | ||
| gemma-4-31B-it | 🔧 Tool Use | 2 | 1222ms | $0.019 | 100% | ||
| gemma-4-31B-it | 🔍 Context Recall | 2 | 1061ms | $0.033 | 100% | ||
| gemma-4-31B-it | 🚀 Throughput | 2 | 16816ms | $0.196 | 100% | ||
| gemma-4-31B-it | 💻 Code Gen | 2 | 3750ms | $0.054 | 100% | ||
| gemma-4-31B-it | { } JSON Output | 2 | 2176ms | $0.020 | 100% | ||
| gemma-4-31B-it | 🧮 Reasoning | 2 | 1673ms | $0.022 | 100% | ||
| gemma-4-31B-it | ⚡ Ping | 2 | 791ms | <$0.01 | 100% | ||
| DeepSeek-V3.2 | 🔧 Tool Use | 2 | 6026ms | $0.118 | 100% | ||
| DeepSeek-V3.2 | 🔍 Context Recall | 2 | 1914ms | $0.055 | 100% | ||
| DeepSeek-V3.2 | 🚀 Throughput | 2 | — | — | 0% | ||
| DeepSeek-V3.2 | 💻 Code Gen | 2 | — | — | 0% | ||
| DeepSeek-V3.2 | { } JSON Output | 2 | 4779ms | $0.026 | 100% | ||
| DeepSeek-V3.2 | 🧮 Reasoning | 2 | 4428ms | $0.028 | 100% | ||
| DeepSeek-V3.2 | ⚡ Ping | 2 | 1615ms | <$0.01 | 100% | ||
| MiMo-V2.5 | 🔧 Tool Use | 2 | 2029ms | $0.228 | 100% | ||
| MiMo-V2.5 | 🔍 Context Recall | 2 | 1176ms | $0.180 | 100% | ||
| MiMo-V2.5 | 🚀 Throughput | 2 | 4658ms | $1.02 | 100% | ||
| MiMo-V2.5 | 💻 Code Gen | 2 | 2425ms | $0.427 | 100% | ||
| MiMo-V2.5 | { } JSON Output | 2 | 1393ms | $0.098 | 100% | ||
| MiMo-V2.5 | 🧮 Reasoning | 2 | 1313ms | $0.176 | 100% | ||
| MiMo-V2.5 | ⚡ Ping | 2 | 988ms | $0.044 | 100% | ||
| Qwen3.5-397B-A17B | 🔧 Tool Use | 2 | 2832ms | $0.422 | 100% | ||
| Qwen3.5-397B-A17B | 🔍 Context Recall | 2 | — | — | 0% | ||
| Qwen3.5-397B-A17B | 🚀 Throughput | 2 | 10056ms | $1.52 | 100% | ||
| Qwen3.5-397B-A17B | 💻 Code Gen | 2 | — | — | 0% | ||
| Qwen3.5-397B-A17B | { } JSON Output | 2 | — | — | 0% | ||
| Qwen3.5-397B-A17B | 🧮 Reasoning | 2 | — | — | 0% | ||
| Qwen3.5-397B-A17B | ⚡ Ping | 2 | 1027ms | $0.056 | 100% | ||
| Qwen3-Max-Thinking | 🔧 Tool Use | 2 | 2582ms | $0.552 | 100% | ||
| Qwen3-Max-Thinking | 🔍 Context Recall | 2 | 2596ms | $0.449 | 100% | ||
| Qwen3-Max-Thinking | 🚀 Throughput | 2 | 7284ms | $3.05 | 100% | ||
| Qwen3-Max-Thinking | 💻 Code Gen | 2 | 4562ms | $0.907 | 100% | ||
| Qwen3-Max-Thinking | { } JSON Output | 2 | 2977ms | $0.246 | 100% | ||
| Qwen3-Max-Thinking | 🧮 Reasoning | 2 | 3425ms | $0.484 | 100% | ||
| Qwen3-Max-Thinking | ⚡ Ping | 2 | 2401ms | $0.024 | 100% | ||
| Qwen3-14B | 🔧 Tool Use | 2 | — | — | 0% | ||
| Qwen3-14B | 🔍 Context Recall | 2 | — | — | 0% | ||
| Qwen3-14B | 🚀 Throughput | 2 | 7997ms | $0.125 | 100% | ||
| Qwen3-14B | 💻 Code Gen | 2 | — | — | 0% | ||
| Qwen3-14B | { } JSON Output | 2 | — | — | 0% | ||
| Qwen3-14B | 🧮 Reasoning | 2 | — | — | 0% | ||
| Qwen3-14B | ⚡ Ping | 2 | 973ms | <$0.01 | 100% |