# Evals (/docs/evals)

Every run below actually happened. The prompt is what a person typed, the calls are
what the model wrote against the library, and the answer is what came back off the
live sites. Nothing is hand-written and **no prompt names a Bowmark function** — the
model gets one lookup and has to find its own way there.

They all ran on **GPT-5.6 Luna**, the cheapest tier OpenAI publishes, at $0.20 per
million input tokens. That is deliberate. The hard part of these tasks is the website,
not the model: the agent is calling a typed function, so it never has to be clever
enough to drive a browser, read a rendered page, or recover from a layout change.

Every prompt was run five times. **The pass count on each row is the real one** — ten of
these went five for five, and one did not; that row says so rather than rounding it up.
The run shown is the fastest from the most recent batch, and the timing is real
wall-clock for the whole exchange: the lookup, the script, every live site it touched,
and the written answer.

A call that errored is marked. They are left in because the run genuinely made them, and
a page claiming every step succeeded would be a different page from this one.

<EvalBrowser />

## Running them yourself

The prompts and the checks that grade them ship in the repo, in
`apps/demos/browser-use`. `node eval.mjs` runs the lot on the cheap model; pass
`--model` for a bigger one.

Three rules the checks enforce, because they are what make a pass mean anything:

* **A check never trusts the prose.** It pulls the links and prices out of the answer
  and requires each one to appear in what Bowmark actually returned on that run. An
  answer that invents a number fails, even when the number happens to be right.
* **Some checks require a specific function.** The month-of-fares prompt fails if the
  agent ran thirty searches instead of reading the price graph, even when the answer is
  correct. Getting there the slow way is the thing being measured.
* **Only passing runs are published here.** Failures stay in the repo and on the board,
  which is where they are useful.