Field notes · Jev

How people actually use Jev

Eleven repositories on GitHub build on TypeSafe's Jev. I read them, and the files most people don't open — docs/performance.md, the limits sections, a benchmark buried in a repository with no stars. I ran none of them. Everything below comes from what their authors wrote and measured, and where there is a number you can go and check it.

The pitch, and the sentence under it

The version going around is that Jev flips the browser-automation industry. The repository that started that — Browser Use's, eight and a half thousand stars — says something narrower, in its first paragraph:

"Give it one goal. TypeSafe's Jev picks an operation and an element. A small LLM writes text only when the operation is TYPE_TEXT."

Read that twice. Jev picks. It does not write, it does not plan, and it does not check its own work. Five projects in this list — written by people who have never met — draw that same line in the same place, and every one of them says it out loud. It is the difference between "a model that runs your agent" and a model you can actually put inside a loop.

Where everyone draws the same line

I did not go looking for this. I went looking for how each project uses the model, and kept hitting the same sentence in different words. It is the most consistent thing in the whole field.

The piano project puts the sharpest version of it in a line about what the model is allowed to answer with: option ids from the list the page offered, never a key name or a coordinate. That is not modesty. A model that returns a coordinate can return a wrong coordinate, and you have no way to check it. A model that returns "option 7" can only be wrong about which of the things you offered it is right — and the executor can still refuse.

The seven seconds nobody earned

The demo everyone shared is a Google Flights search from Zürich to London that completes in about seven seconds. It is real — there is a video at 1× speed with continuous frames and the timing left in. What is missing from the coverage is what the same repository says about it two files later.

Put those rows together and the story changes shape. The matched comparison is a median of 9.450 s down to 7.092 s — a quarter faster, and the author writes out the sign test result himself: p = 0.25, three pairs, "not a broad agent benchmark". The number that moved hardest was not model latency at all. It was browser round trips, which fell from 1,092 to 101, because the new version reads the page in one call instead of walking the accessibility tree repeatedly. The commit where that landed is titled, in the author's own words, Reduce browser round trips and record a 7-second Flights demo.

This is not a takedown. A 25% end-to-end improvement on a real task is a good result, and publishing the p-value next to it is better practice than most research. It just means the headline number belongs to the DOM reader, not to the model.

178, 431, and a half-second head start

Two projects measured the same thing and got different answers, which is the useful part. Browser Use reports a median decision latency of 178 ms. A separate triage project measured roughly 431 ms per call, API only. Different inputs, different endpoints, no shared harness — so take the pair as a range, not a specification.

The piano project shows what that latency costs when the thing you are driving does not wait for you. It gives each hand its own model, and every 100 ms slice of the piece is a question: where does this hand sit, and which key is each finger on. The answer takes about 400 ms — four times slower than the slice. So it asks about each slice 500 ms before that slice is due, keeps several requests in flight, and applies the answer at the moment it was asked about rather than when it lands.

That is the most useful engineering note in the whole field, and it is in a repository with one star. If your loop needs a decision faster than the model returns one, you do not make the model faster — you ask earlier.

A cent a day, with a screenshot

Cost claims are usually the vaguest part of any AI project page. Two of these are not. The MCP server publishes an OpenRouter billing dashboard reading one cent for a day of work, and states plainly what that replaces: "one tool call instead of twenty, three seconds instead of a minute, a cent instead of a frontier model's context." The headless browser project reports $0.0016 for the Wikipedia navigation it uses as its own demo.

That fits the pricing we read from the catalogue: $0.042 per million input tokens and free output, which is what makes a per-step decision cheap enough to do on every element of every page. It also means the cache price matters more than the headline price here — if your harness re-sends the same instruction block on every step, the cached rate is the one that shows up on the bill. We publish those rates for every model in the directory, including this one, in the cached-pricing table.

Someone finally tested the calibration claim

The vendor's boldest claim is not speed, it is that the probabilities are calibrated — that 0.8 means roughly 80%. Almost nobody tests that, because it needs labelled data and patience. One repository does. It has zero stars.

The setup: 120 synthetic support tickets, three judgements each — department, urgency, frustration — run through the same workflow twice, once with Jev and once with a current small LLM.

"Jev's p(urgent) is well calibrated (bucket hit-rates 0 / 21 / 41 / 75 / 100%), so the workflow can route 0.35–0.65 to human_review instead of guessing."

The bucket hit-rates are the part that matters. Score buckets that actually land 0, 21, 41, 75 and 100 percent of the time mean you can write a rule that sends the uncertain middle to a human and lets the confident ends through — which is the thing you cannot do with a model that says "urgent" and nothing else. One caveat, from the repository itself: the tickets are synthetic and the labels were generated by a larger model, so this measures agreement with a model, not with reality.

Where it breaks

Limits sections are the least-read paragraphs on GitHub and the most useful. Quoted, because paraphrasing these would soften them:

"A valid operation can still be wrong, and DONE is never independent evidence of success" is the sentence I would put on the box. It is the author of the fastest implementation telling you not to trust its own success signal.

The fair objection

Not everyone is convinced, and the sceptical version is worth stating properly because it is partly right. The most-shared objection on the day I read these — from an AI commentator with a large following — was that X is full of flashy demos that "don't amount to much actual value when you look behind the scenes", and that the hard part is going from a demo to a loop you can depend on.

The evidence in this page splits the difference. On one side: three of the eleven repositories have no stars at all, one has no README, and the famous demo is a quarter faster, not a hundred times faster. On the other: people are pointing it at pianos and iOS simulators and QA harnesses within three days, several of them measured their work and published the failures, and the cost is low enough that trying it is not a decision you need a meeting about.

The honest summary is that the demos are real, the difficulty has moved rather than disappeared, and the part that moved is the part you would have had to write anyway: the table of things that can be acted on, and the rule that decides when you are finished.

We ran it, twenty times

Everything above is read from other people's files. This part is not. Our waitlist cleared on 2026-09-20, and the first thing worth doing with a key is the thing nobody does: send the same request over and over and see what moves.

One support ticket as the state, three questions in one call — a Noul, a Choice and a Score — and then 20 byte-identical repeats.

The Noul did not move at all. The Choice's label did not move either — technical, 20 times out of 20. What moved is the number next to it: the confidence ran from 0.38 to 0.58, which under TypeSafe's own three-band guidance — act, proceed with caution, do not act — is the difference between two different behaviours on an input nobody changed. A pipeline that gates on confidence, as the documentation tells you to, routes the same ticket two ways. The label it logs will be identical every time.

Our whole experiment — 20 calls, 420 input tokens each — cost $0.000353. Wall time from our machine was a median of 783 ms, which includes the network and is not comparable to the 70–500 ms the vendor quotes from their own machines. The full response body, and the one field their cookbook sends that the API rejected, are on the reference page.

We also cloned browser-use/jev-ultrafast and ran its own test suite, because that is the cheapest question anyone can ask about a repository and almost nobody reports the answer: 31 passed, in 0.44 s, via uv sync && uv run pytest -q. It is an offline suite, not a live run. What we did not run is the live browser agent — its harness daemon was not running here, and the live path wants a second API key we chose not to spend; every browser number on this page is still the author's, and still says so.

The eleven

Questions people actually ask

Where to go next

The project that started it → Jev reference Model directory