Loading benchmark data…
EyesInAI·Loading live benchmark data
We would like to count visits with Google Analytics, which sets a cookie. It is entirely optional — the site works exactly the same either way, and we never use it for advertising. What we set, and why.
The internet rewards a big GitHub star count, but stars measure hype, not fit. The question that actually matters for a team is narrower: would this survive contact with my stack, my hardware, my data rules, and the tools I already run? Most of the time — honestly — the answer is no. This is the ledger of tools we evaluated properly and chose notto ship, with the concrete reason on each. It's the same measured posture as the rest of the site: we show the receipts, including the ones that end in “we passed.”
A fairness rule we hold to: every entry is framed as “didn't fit us,”never “this is a bad project.” Most of these are genuinely good tools — the mismatch is with our constraints, not their quality. Some we deferred rather than rejected; some we kept as a reference architecture without installing. The counterpart to this page is Automation & Agents — the harness tools we did keep.
We evaluated it fairly, then realized we already have a path that does this — often with stronger guardrails.
Genuine merit. Blocked on hardware, timing, or a dependency rather than quality — teed up for when the blocker clears.
Worth studying for its architecture or packaging, but there was nothing here to actually deploy for us.
Overlaps — and would conflict — with a capability already in our stack. Adding it would be sprawl, not value.
A small eval framework: define a suite as a directory of YAML files, run it across model configs, grade the runs against checks, then serve or build a static HTML report.
Why it was worth a look: The vocabulary is the best we have seen anyone put on this problem — eval, task, config, run, runner, grader, check, checker, each doing exactly one job. It is also genuinely agent-native: `uvx smevals docs` prints the README so a coding agent can learn the tool and then build you a suite. Small, MIT, no infrastructure.
Why we passed: We already run this, one layer deeper: 36 tests (18 built-in, 18 custom) with 8 data validators, an LLM judge for open-ended answers, a proposed→approved workflow, results in Supabase feeding the public leaderboard, and a tests-eye lens over the lot. Adopting smevals would mean a second eval system beside a more mature one, with no path back to that history or that UI — sprawl, not value. What we DID take is its best idea: a "config" that varies the system prompt and parameters, not just the model, so a prompt-vs-prompt bake-off is an ordinary run rather than a bespoke script. That axis is now in our own runner. Borrowing the abstraction was the right adoption; installing the tool was not.
Two well-regarded, MIT design-assistant "skills" for coding agents: one redesign/audit-oriented with CI slop-detection, one greenfield/marketing-page oriented with a tunable taste system.
Why it was worth a look: Both are legitimately good at what they do, and a design layer for an AI coding agent is a real want when you are shipping a lot of UI.
Why we passed: We already run a mature, canonical design-skill stack (a house UI/UX system plus per-project design guidelines and an official frontend-design skill). Installing a fourth and fifth design assistant globally would mean overlapping, conflicting triggers — the exact skill-sprawl that makes an agent pick the wrong tool. The right number of global design skills for us was the ones we had, not more.
Could not be adopted the way it was pitched: unavailable on our routes, or barred by our data-sovereignty rules.