Harness notes: what makes a Jev game harness perform

Working notes from building the Jev Wall (27 → 36 real-time games played by TypeSafe's Jev, one batched request per ~180 ms tick). Everything here was learned the hard way on this wall; each rule names the game that taught it. Jev makes every decision; code never overrides a choice, it only executes it and keeps the world honest.

The one principle: Jev picks the intent, code closes the loop

Asteroids was the best-playing tile from day one and the reason is structural. Jev is asked which rock to shoot or whether to evade; code does the lead aiming, the turning, the firing. A choice stays valid for a second or more while the world moves, so a 300 ms decision delay costs nothing. Every harness that asked for a raw control at a moment in time (a direction at this cell, climb or dive now, hop now) was worse, and every one of them improved when it was rewritten to the Asteroids shape:

Turn-based games (Snake, Tetris, Q*bert, Match-3, Bubbles, Artillery, Digger) do not need this: the game waits for the answer, so a per-move choice is already the right shape.

When raw controls are unavoidable: measure the delay and simulate through it

Pinball, Slalom and Sumo keep raw controls on purpose (they are the latency demo). The harness measures its own decision delay with an EMA around the ask (performance.now() before and after, scaled by the governor's game speed), and every option is simulated forward: the current control keeps running for the delay, then the candidate applies. The option text reports the outcome at that future moment ("at the gate you would be 17 px right of centre, THROUGH the gate"), and the state says the delay in words ("your choice takes effect about 0.25 s from now; until then you keep leaning straight"). Pinball runs an A/B (odd balls delay-aware, even balls naive) so the wall itself accumulates the evidence.

Option design

• Few options, four to six. Asteroids offers the three nearest rocks plus evade. Football's ball carrier gets pass targets, shoot and dribble, nothing else. More options dilute the probability mass and cost tokens.
• Facts in the model's frame, computed by code. Distance, turn needed, time to impact, pellets on the route, nearest ghost to the route, "reachable in time" or "TOO LATE". Jev is not asked to do arithmetic or geometry; it is asked to weigh outcomes.
• Capitalised outcome words (DANGER, HITS, TOO LATE, THROUGH, OUT OF THE RING) make the decisive fact easy to find in a long option string.
• Include a no-op when standing still is legitimate (Frogger's wait, Tower Defense's save, Whack-a-Mole's wait) and make it concrete: "wait (do nothing this tick): grass, safe; the next log reaches this column in 1.4 s". Without it the model is forced to act.
• Include a code-executed macro when the right move is "wait for X then act" (Frogger's board_log). It removes the timing problem entirely and is still Jev's decision.
• Candidates must cover the good answer. Doodle fell when the chosen platform was unreachable; the fix was filtering candidates by real flight time, not a smarter prompt.

Persona and instructions

• State priorities in order with one concrete threshold: "evade whenever an asteroid will hit you within about two seconds, otherwise shoot the nearest one".
• Say what the code will do with the answer: "the code walks the route square by square and asks again when you arrive or every second".
• Name the option set in the instruction with a backticked path (options, targets, lanes).

Timing and the world

• Sub-step movement for grid games (Pac-Man moved in 0.05-cell steps after a frame-rate drop skipped the cell-centre check). Detect crossings with previous-vs-current position, never with a predicted next position (Doodle passed through aligned platforms on frame jitter).
• Re-ask on events, not only on a timer: arrival at the target, the target vanished, a ghost within two squares, a ball heading your way.
• Keep an intent sticky between answers and never block the world on the answer. A pending ask must not be a reason to stand still (Pac-Man's old harness waited at junctions).
• One in-flight ask per tile; if the previous answer has not arrived, keep executing it.

Difficulty and the governor

• A wall tile calls JA.win() / JA.loss() on real outcomes (gate passed, ghost caught, goal conceded). The governor scales that tile's virtual clock between 0.55x and 1.3x to hold a target win/loss ratio, and needs at least two losses before slowing anything.
• Balance so that a wrong choice rarely kills instantly: keeper blocks, respawn invulnerability, a throw-in for a stuck ball. Instant death teaches nothing and reads as noise on the wall.
• Symmetric two-persona games (Pong, Sumo, Tron, Football) need something that forces contact (Sumo's shrinking ring) or both sides pick the safe option forever.

Cost and limits (measured 2026-09-19)

• jev-latest: 64k tokens per request, 32k for state plus the longest question; 1,200 requests per minute; 250k tokens per second. A 250 KB state is refused (max_tokens_exceeded); 400 small questions in one request (27k tokens) are accepted.
• The wall batches every tile's pending questions into one request per 180 ms window per host part. 27 games ≈ 520 calls/min at ~7 questions and ~3k tokens per call, about $4/hour. The binding limit is requests per minute, so more games mostly add questions per call, not calls; each extra host part adds ~175 calls/min.
• Ask only when there is a decision to make (Tower Defense asks only with gold for a tower; Whack-a-Mole only with a mole up). A tile that asks four times a second with nothing to decide is the most expensive thing on the wall.