EyesInAI

Switchyard

Benchmark your use caseAI Benchmark Intelligence

Loading models…

EyesInAI·Loading models…

Loading benchmark data…

EyesInAI·Loading live benchmark data

EyesInAI

An independent, measured view of AI models — real cost, real quality, no vendor claims.

Switchyard Benchmark your use case Benchmarks Case studies Task routing Learn News Methodology Contact Privacy

Home Benchmark

Text

Loading models…

EyesInAI·Loading models…

All models

deepinfra

meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8

Last tested Jul 1, 2026

Overall pass

71%

Avg latency

4705 ms

Context

1000k

Tools

Yes

Input $/1M

$0.15

Output $/1M

$0.60

Tests run

Passed

5/7

AI summary

Hardware to run it

Hosted: $0.15 in / $0.6 out per 1M

Self-host:GPU cluster400B total / 17B active (MoE)

Q4 ~48GB

~48GB RAM minimum at Q4; a cluster (or M4 Ultra) for full quality

Broadest ecosystem of any open model. Q4 runs on a high-RAM workstation; full precision wants data-center GPUs.

Test results

⚡Ping

Latency & availability — single-word reply

969 ms

🧮Reasoning

Basic math reasoning — show work, give answer

2074 ms

{ }JSON Output

Structured output compliance — valid JSON with required keys

1858 ms

💻Code Gen

Python function generation with docstring

4454 ms

🚀Throughput

Token generation speed — 500-token long-form response

fail

🔍Context Recall

Retrieval from in-context data — 20-item list Q&A

Compare with another model See the leaderboard

2391 ms

🔧Tool Use

Function/tool calling — get_weather invocation

fail