@ilijabogunovic image post · 2026-09-21
Jev on 450 WikiRace games: cheap, fast, short on world knowledge
@TypeSafeAI just released Jev, a new System 1 model, with Wikirace as a headline demo. So we ran it on ours: LLM-Wikirace benchmark. 450 games. 8 hours of wall-clock time. Under $1 total. Verdict: fast and absurdly cheap, but short on the world knowledge and planning that…
Why it is here
Somebody ran the vendor's headline demo on their own benchmark and published the verdict that hurts: fast and absurdly cheap, and short on the world knowledge and planning the demo implies. Four hundred and fifty games, eight hours, under a dollar — the cost claim holds even where the capability one does not.
What we checked
- the post read through X's public syndication endpoint, 2026-09-25
- like count and date read from the same endpoint the same day
What we did not check. We did not re-run the thing the post describes, so every figure on this page is the author's own and every claim is theirs — the note above is what we make of it after reading, not a measurement. The like count is a snapshot read on 2026-09-23 and it has moved since; 22 is what it said when we looked.