EyesInAI

Switchyard

Benchmark your use caseAI Benchmark Intelligence

Loading models…

EyesInAI·Loading models…

Loading benchmark data…

EyesInAI·Loading live benchmark data

EyesInAI

An independent, measured view of AI models — real cost, real quality, no vendor claims.

Switchyard Benchmark your use case Benchmarks Case studies Task routing Learn News Methodology Contact Privacy

Home Benchmark

Text

Loading models…

EyesInAI·Loading models…

All models

together

google/gemma-4-31B-it

Last tested Jul 4, 2026

Overall pass

57%

Avg latency

2777 ms

Context

262k

Tools

Input $/1M

$0.39

Output $/1M

$0.97

Tests run

Passed

4/7

AI summary

Hardware to run it

Hosted: $0.39 in / $0.9700000000000001 out per 1M

Self-host:Single GPU26B MoE (also a 31B dense variant)

Q4 ~16GB

~16GB RAM — a mini-PC or 24GB Mac; the 31B dense variant on a 24GB GPU

The best consumer-hardware model — runs in ~16GB RAM, competes with models several times its size. Apache 2.0.

Test results

⚡Ping

Latency & availability — single-word reply

520 ms

🧮Reasoning

Basic math reasoning — show work, give answer

fail

{ }JSON Output

Structured output compliance — valid JSON with required keys

fail

💻Code Gen

Python function generation with docstring

5663 ms

🚀Throughput

Token generation speed — 500-token long-form response

5978 ms

🔍Context Recall

Retrieval from in-context data — 20-item list Q&A

Compare with another model See the leaderboard

fail

🔧Tool Use

Function/tool calling — get_weather invocation

1570 ms