Draft, critique, revise, check

Asking a model to "make it better" and then asking again tends to make a draft different rather than better. Each pass fixes something the last one noticed and quietly moves something else. The film shows the loop that keeps iteration honest, often called self-refine or evaluator-optimiser: a writer model drafts, a critic model scores the draft against a written rubric and returns a structured verdict, your code picks one target, the writer revises, and the loop stops when every check passes or its lap budget runs out.

The task is a customer-facing status update for a payments incident, written from an internal log that invites trouble on purpose: it names the on-call engineer, a paging mistake, and the vendor whose key rotation caused the outage. The rubric has five checks. The update must say who is affected and what they cannot do, say when it began (14:02 UTC), say when the next update comes (15:00 UTC), stay under 80 words, and blame no person, team or vendor. Start time, next update and length are checked in code. Impact and blame need judgement, so the critic decides them.

Every text and verdict in the film is real. Both parts were played by `google/gemini-3.8-flash` through OpenRouter at temperature 0 on 11 September 2026, the critic in its own call. It got the rubric and the draft under a strict JSON schema, a pass and a one-line note for every criterion plus a list of what fails, and it ran three times per version. The three runs agreed on every criterion of every version. One further reply arrived cut off mid-JSON and failed to parse, which is exactly why the loop reads the reply with `Verdict.parse(JSON.parse(raw))` and asks again rather than trusting whatever came back.

The first draft ran to 136 words, and the critic failed it on length. It passed the start time, in all three runs, with the note "Mentions orders placed between 14:02 and 14:31 UTC": the only 14:02 in the draft is about which orders might show as payment pending, not when the outage began. The code check needs 14:02 stated as a start, so it fails, and because the loop spreads the code checks over the critic's scores, code wins wherever code can count. The critic also miscounted words, 35 for a draft of 43 and 52 for one of 58. The loop then took one target, the first failing criterion in rubric order, and sent it to the writer with the whole rubric. Version two came back in 43 words, the loop re-scored all five criteria rather than only the target, every one passed, and the loop stopped and kept it.

Then the failure mode, which we set up on purpose and want to be plain about. An earlier recording, where each lap simply fixed the next failing criterion with the rubric in view, produced no regression at all. So the third version comes from the kind of request that often arrives mid-review: one narrow stakeholder ask, "make the impact clearer: tell customers exactly what went wrong and why", sent with the update but without the rubric. The model did what was asked. Version three is clearer, and it grew to 103 words and explained that the outage "was caused by an unannounced security key update from our third-party payment vendor". The word count failed in code and the critic voted three to nothing that it blamed the vendor. Two checks that version two passed turned red, and neither was what the edit aimed at. The fix sent the edit again with the rubric back in the prompt, version two in view as the last version that passed, and an instruction to lose nothing that had passed. Version four keeps the new detail about pending orders and duplicate charges, drops the blame, fits in 58 words and passes all five.

Two honest notes. The code checks are only as good as their patterns: a regular expression that matched 14:02 anywhere would have passed the first draft exactly as the critic did, so the check has to say what it means, a start. And the code on screen is the shape of what ran rather than every line of it; the lap budget of four never bit here, because the loop converged on its second lap, but a loop without one can spend forever on a rubric its writer cannot meet.

What to keep: let code decide whatever it can count, put the whole rubric in the prompt with every edit, re-score every criterion after it, keep the last version that passed so a bad edit can be undone, and cap the laps.