The Exemplar Ratchet
Extract the Shape of Greatness, Not the Steps
How an accidental overnight loop becomes a repeatable harness
A self-improving loop gets better only in the directions you can name and only against a bar you hold still.
So keep the shape — written judgement, gradient steering, outsider critic, frozen costed exemplar, blind pairs — not the steps of the accident.
By the end of this book you can
- ✓ Take apart a long agent run into eight reusable ingredients
- ✓ Write a gradient: north star, named weakness, named delight
- ✓ Freeze a costed exemplar and judge recreations in blind pairs
- ✓ Stand up an outsider critic and a cold-arrival reader as named roles
- ✓ Read a finished run as a replay simulator for scheduler policy
- ✓ Tune against a human expert’s hand-made work — with consent
The Wrong Souvenir
An astonishing overnight agent run is not the asset. The asset is the shape that made improvement directional — because a loop only gets better in directions you can name, against a bar you hold still.
I prompted it poorly to finish at 10am — and when it got there it thought the finish was the next 10am, so it kept going.
That is not a cute origin myth. It is a receipt. The review loop on Venture World — a simulation of Perch Software, an early-1980s games studio with bird-agents, letters, invoices, and a kid tester named Mags — ran because a deadline was written as a time of day without a date. Roughly eight hours of self-directed work followed. Across the wider build there were 338 commits in about five days. Half a Max plan burned. I was flabbergasted with the changes.
The story became more consistent and fluent. Gaps filled themselves. Mags, being a kid, turned up after school and only played a few hours. Her bug reports were hilarious and honest. Disks came back broken in the post asking for replacement, not refund. Letters arrived opened, with envelopes still in the frame. Newspaper clippings showed reverse print bleed from the back page and a crease in the paper. The postman brought packages. The birds got up and walked across to talk. They had a budget, spent it, and ran an overdraft.
If you have ever left an agent running overnight and woken up to work you would pay for again, you know the feeling that comes next. You want the night back. You want to bottle it. Most teams then keep the wrong souvenir.
What people keep instead
They keep the astonishment. They try to replay the steps. They A/B a few prompt variants and call the exercise self-improvement. They tell the model to “run in a loop and try more variations,” as if undirected variation were a strategy. Sometimes they get another expensive surprise. Sometimes they get a thinner copy of the first night and conclude that lightning does not strike twice.
The useful extract is narrower, and it travels.
Now that we have the shape of greatness, we can extract it, rather than repeat it step by step.
That line is the spine of this book. Farming exhaust — treating a run’s by-products as raw material rather than waste — is the practice underneath it. The method identity is simple enough to say out loud: see the pattern, generalise the solution (sort of like a TRIZ master), and feed it back in reverse to fix the original machine.1 The overnight miracle was never the product. The product is a harness that can find the next miracle on purpose.
The claim
A self-improving loop gets better only in the directions you can name and only against a bar you hold still. So the repeatable asset from a surprising run is not its steps but its shape: written judgement before code, a gradient that pairs a named weakness with a named delight, a stronger outsider who reviews what actually ran, and a frozen exemplar that every new version must beat in blind pairs at lower cost.
What you should be able to do afterwards
This is a deliberately short volume — live-tier material, not a capstone that tries to hold a whole company doctrine. After it, you should be able to:
- Take apart a successful long-running agent session into reusable ingredients, using the as-run record rather than recollection.
- Stand up an outsider critic and a cold-arrival reader as named roles in the harness.
- Freeze an exemplar with its cost, and judge recreations by blind pairwise comparison rather than scores out of seven.
- Tune a harness against a human expert’s hand-made work without asking that expert to invent criteria from cold recall.
What this book is not
- A tutorial on cron hooks, worktrees, or a particular CLI.
- A re-teach of general review-loop anatomy, handover-note discipline, or company/service doctrine already covered elsewhere.
- Permission to point a blunt outsider critic at a colleague’s shared work and call it process.
- A claim that coherence is the same thing as truth, pivot, or strategy.
The question the record forces
If you only keep the feeling of the night, you will mythologise it. The session log for that run — opened Sunday evening on the build machine as e37513db — does not support “left it running and magic happened.” Structure was visible. Judgement had been written down. A gradient had been named. An outsider was already on a clock. Cold readers were already in the notes. Gates existed. So did a buggy sense of time.
The next chapter walks those ingredients as they appear in the record, including two corrections to memory that matter as much as the ingredients themselves. Memory flatters loops. The as-run record is what keeps the extraction honest.
Eight Ingredients, From the Record
The session log does not support “left it running and magic happened.” Structure was visible — and two corrections to memory are part of the method.
Chapter 1 left a question hanging: what was the night actually made of? The answer is not a vibe. Session e37513db opened Sunday 13 September at about 21:43 on the build machine. Even a partial read — roughly thirty of more than a hundred and sixty turns — is enough to see the scaffold. What follows uses the wording the record actually supplies. Where a full critic brief file was named in conversation but not reproduced in the source material available here, we refuse to invent it.
1. Judgement written before code
Before the loop started grinding features, the author agent was told it held authority for taste, review and content, and to document so the work could be audited later. It produced on the order of 34,000 words across twenty documents, plus four tests every feature had to pass: theatre, business, looking interesting, and scripted.
That is not ceremony for its own sake. It is the difference between a loop that improvises taste on every turn and a loop that can be checked against something written while the humans were still awake.
2. Gradient steering instead of a spec
The second ingredient is the one almost nobody writes down. The north star was simple: is it interesting and fun. The named weakness was sharper: too much playing a game, and not ‘real business’. The named delight was the line that paid for itself: when the magazine cover/review arrived… so perhaps other things have an image.
That third sentence is where the letters, the broken disks and the clippings came from. Chapter 3 will treat gradient steering as doctrine. Here it is enough to notice that the record already had a direction of improvement, not merely a destination checklist.
3. Represented as it actually ran
Critics did not review the script’s intentions. They reviewed what happened on screen. As-run representation sounds obvious until you watch how often review loops argue with a plan document while the live artefact has already drifted. The transcript tools made the screen the evidence.
4. An hourly outsider critic
On a schedule, a stronger model arrived to judge progress against the north star — and explicitly not to judge code quality. In the harness it was nicknamed Gilfoyle; as a role it is the outsider critic. Chapter 6 takes the properties apart. In the record, the important facts are already present: stronger than the author agent, aimed at taste and drama, recurring, outside the author’s sunk cost.
5. Cold-arrival readers
Fresh agents with no context. The handover notes record multiple reader passes. These are not the same role as the outsider critic. The outsider is harsh and continuous. The cold-arrival reader is ignorant on purpose. Both matter; they answer different questions.
6. One author holding quality
Quality responsibility stayed with one author agent: review changes that came back from sub-agents against git. Later the brief tightened further — control and write everything, with bounded coding jobs elsewhere. Parallelism without a single quality owner is how you get a pile of locally clever diffs and a globally incoherent show.
7. Mechanical gates before every commit
Check. End-to-end. Reproduce. The taste loop did not replace engineering hygiene; it sat on top of it. That pairing matters when people hear “north star, not code quality” and quietly delete the tests. The same instinct shows up when skill libraries grow from exhaust: promote what has been verified, and keep ungated appends from silently corrupting later behaviour.2
8. A clock
“Sprint to 6am,” then “continue… to 11am,” with a hook that re-issued the brief. The runaway came from the clock. A deadline given as a time of day with no date is temporal blindness showing up in your own tooling. The extractable fix is boring and non-negotiable: an absolute datetime, plus a spend cap. Without both, you do not have a harness. You have a bonfire with a calendar invite.
The eight ingredients
- Judgement written before code (documents + pass/fail tests)
- Gradient steering (north star + named weakness + named delight)
- As-run representation for critics
- Stronger-model outsider critic on the north star
- Cold-arrival readers
- One author with quality responsibility
- Mechanical gates before every commit
- A clock with an absolute end and a spend cap
Two corrections memory will fight you on
The charming details are easy to misremember as overnight discoveries. The record is less flattering, and more useful.
Mags was not invented by the loop at 3am. He was designed as non-staff from the start: spawn at the door, play, give a verdict, leave. When labour laws came up later in the session, the agent could point at an earlier episode that already said he was not on the payroll. The world was coherent before anyone audited it. That is stronger evidence than a lucky find, because it shows the gradient and the judgement documents doing their job.
The pivot belonged to the humans. A question — why is episode one Perch and Post? — led to counting, then to the conclusion that the alpha was better than any of the generated thirty, then to a reframe: do not port the alpha; recover how it was produced. The loop polished. It did not choose the turn.
The loop polished the work. It did not pivot it.
Pitfall
If you write the story from astonishment alone, you will credit the loop with discovery and pivot. The record usually shows design-in and human judgement doing those jobs, with the loop doing something narrower and still valuable: coherence under a named gradient.
Expect junk
A later overnight run ended by keeping the useful fixes and removing a speculative change. That sentence belongs in the template. Loops produce waste alongside value. Morning review is not a soft cultural habit; it is how you stop speculative decoding from becoming doctrine because it arrived while you slept.
What is new here
Much of the surrounding loop craft — multi-surface review, adversarial closing, handover discipline, replay-derived reading — already exists as practice elsewhere. This volume does not re-teach those pages. What looks newly load-bearing in this record, and what the rest of the book develops, is narrower: gradient steering as a written triad, the exemplar benchmark with cost, and the cold-arrival reader as a named role beside the outsider critic.
Of the eight, ingredient two is the underrated jewel. Specs tell a system what done looks like. Scores pretend to measure it. A gradient tells a moving loop which way is up. That is the next chapter.
Gradient Steering
A north star alone is not enough. Pair a named weakness with a named delight — that triad is what tells a moving loop which way is up.
Chapter 2 called ingredient two the underrated jewel. Here is why, without retelling the overnight inventory.
Most teams already have two substitutes for direction, and both fail mid-flight. A specification says what to build when the world is still. A scoring rubric pretends to measure greatness when the work is already moving. Neither answers the question a long-running agent actually faces hour after hour: improve toward what?
Definition
Gradient steering
Write three lines before the loop starts: a north star (what “better” means in plain language), a named weakness (the failure mode you are currently sliding into), and a named delight (the positive attractor you want more of). The triad steers search under uncertainty. It is not a substitute for gates, and it is not a twelve-point rubric.
Why the delight line earns its keep
A north star without a weakness drifts into vibes. “Make it great” licenses everything and constrains nothing. A weakness without a delight produces austerity: the loop learns what to suppress and becomes expert at removing texture. The delight line is the positive attractor. In the Venture World record it was the magazine-cover observation — and that is the clause where letters, broken disks and clippings came from. The weakness kept the show from collapsing into pure toy. The delight kept “real business” from collapsing into spreadsheets with no world.
The pairing of named weakness and named delight tells a loop which direction to improve in, and almost nobody writes it down.
A provocation, not a specification
The triad has to be strong enough to steer and weak enough not to over-constrain every branch. Think of it as a provocation, not a specification. That is the same instinct as refusing to stuff directional “lessons from history” into the explorer’s prompt and then wondering why parallel search collapses into one local optimum. Dream-RSI later measured that failure mode directly: explicit directional guidance in the explorer prompt underperformed having no such guidance, because strong semantic priors over-constrained parallel discovery.3 Chapter 7 returns to the full reading. The practical rule arrives earlier: write the gradient so a stronger critic can argue with it, not so a weaker agent can hide inside it.
| If you only write… | You tend to get… |
|---|---|
| North star alone | Pleasant drift; no brake |
| Weakness alone | Austere correctness; texture dies |
| Delight alone | Ornament without stakes |
| All three | Directional improvement you can argue about |
Try this before the next long run
Gradient worksheet
For the artefact you are about to leave running, write:
- North star: In one sentence, what does “better” mean?
- Named weakness: What is it doing too much of right now?
- Named delight: What specific moment made you lean forward — and what would “more like that” mean?
Revisit the three lines when pairwise reviews start handing you “why” answers. Those whys are candidate gradient updates. Promotion of a new exemplar (Chapter 5) is also a moment to rewrite the triad, because the bar moved and the old weakness may no longer be the live one.
Myth vs reality
Myth: We already have a rubric, so we are steering.
Reality: A rubric scores after the fact. A gradient steers while the search is still open. You usually need both, and they are not the same document.
What a gradient cannot do
It cannot replace mechanical gates. It cannot invent a pivot the humans refuse to ask for. It cannot stop a contradiction-hunting loop from optimising coherence when what you needed was drama or business truth. Naming a direction is not the same as knowing what your gate is actually selecting for.
That is the next honesty check. Once the Venture World critic’s absolute scores proved too unstable to gate on, the useful signal became contradictions quoted from the as-run log. At that moment the loop’s job quietly specialised. Chapter 4 names what it became.
The Contradiction Hunter
When absolute scores fail and quoted contradictions become the gate, you have chosen an optimiser — whether you named it or not.
Chapter 3 ended on an honesty check. The Venture World record had already made the substitution: critic scores were too unstable to use as a gate; the useful signal was contradictions pulled from the as-run log. That is not a small implementation detail. It is the moment the loop’s job specialised.
Call the thing by its name. You were running a contradiction hunter. Absolute scores were the wrong instrument for the gate — a finding that also shows up when language-model judges are asked for lonely Likert numbers instead of pairwise preference.4
A contradiction hunter can only push the work one way, toward coherence — consistent, fluent, effortless, gaps filled.
What coherence buys you
Coherence is not nothing. In a simulated world it produces the feeling that made the overnight run worth talking about: things hang together; small facts imply schedules; artefacts look like they belong on the same desk. Under a written gradient, that hang-together-ness can be aimed. Without a gradient, it still hangs together — it just hangs together in whatever direction the contradictions happened to point.
So the first discipline is diagnostic. Before you leave a loop running, write one sentence: this loop optimises ____. If you cannot finish the sentence, you do not yet know what success will look like when you wake up.
Key takeaway
Name your optimiser in one sentence before you leave it running. If the useful gate is “quote contradictions from the as-run log,” you are buying coherence. Plan accordingly.
What coherence will not find
A contradiction hunter will not find drama on purpose. It will not find business truth on purpose. It will not ask whether a good player and a good episode are the same objective. Those are different axes. Coherence is a third one: does this hang together?
That limit is easy to miss when the output is charming. Charming is what coherence feels like from the outside. It is also what memory uses to over-credit the loop for pivots and discoveries the record attributes elsewhere.
Myth vs reality
Myth: The loop found the story.
Reality: The loop polished coherence under a gradient. Humans asked the pivot question. Design work placed key world facts before the night began.
The company-copy failure mode
Point the same hunter at a company’s public materials and it will still do its job. Ambition stated as fact will read more smoothly. Gaps between promise and practice will get prose bridges. The site will feel more consistent while becoming less honest.
Pitfall
Coherence is not honesty. A contradiction hunter aimed at marketing copy is a fluency machine for claims that may not deserve fluency.
The useful company aim is different: drift across surfaces. What the site promises, what the sales documents say, what the ledger shows, what sits in the folders. That is still contradiction hunting, but the contradictions are between institutional artefacts rather than between beats in a scene. Chapter 10 returns to that loop as an application. The doctrine point belongs here: choose the contradictions on purpose.
Why the ratchet needs a bar outside the hunter
A coherence loop cannot ask the pivot question by itself. It can make the current world tighter. It cannot, from inside its own objective, decide that the current world is the wrong world. That is why extraction needs a check that sits outside the hunter: a frozen exemplar, with cost attached, against which new work is judged in pairs.
Without that external bar, “better” means “more coherent with itself.” Self-consistency is a weak definition of greatness. It is also how loops quietly certify themselves.
Morning review belongs in the same family of outside checks. Even a well-aimed hunter emits speculative fixes beside useful ones. Expect junk. Keep the useful. Remove the rest before it becomes load-bearing.
Chapter 5 turns the external bar into a mechanism with three honesty conditions: freeze, pair, transfer.
Freeze, Pair, Transfer
If the benchmark keeps changing, it stops being a benchmark. The exemplar ratchet stays honest only when you freeze the bar — including its cost — compare blind in pairs, and test transfer on a different world.
Chapter 4 argued that a contradiction hunter needs a bar outside itself. Here is the mechanism.
Call it the exemplar ratchet. One version exposes another as weaker. The next version must beat the current one. In Venture World the alpha exposed the generated thirty; the current show beat the alpha; the next recreation has to beat the current show. A ratchet only works if the pawl holds. Three honesty conditions are the pawl.
1. Freeze the exemplar — and its cost
Tag the commit. Snapshot the as-run logs and the picture set. Treat one build as the reference copy. If the “gold” keeps sliding because someone improved it on Tuesday afternoon, you are no longer measuring extraction. You are measuring whether your memory can track a moving target.
The goal of extraction is the same bar for a fraction of that effort, which means the cost number belongs in the benchmark alongside the quality.
For the recorded run, cost was not a vibe. It was on the order of five days and half a Max plan. Extraction that reproduces the bar at the same spend has not extracted anything economically interesting. It has replayed a purchase.
2. Compare pairwise
Skip “rate this out of seven.” Absolute scores were already too unstable to gate on in the record that produced this doctrine. Cold-arrival readers get one stretch from the exemplar and one from the recreation, and answer which they would keep watching — and why.
“Which is better” tends to be a steadier judgement than a lonely number. The “why” is not decoration. It is the next gradient update (Chapter 3). Optional support from preference-judging work on language models points the same way: pairwise preference can be more reliable than direct scoring when the judge is itself a model.4 You do not need that paper to justify pairs. You need the Venture World finding that scores thrashed.
Pitfall — critic-taste drift
Promotion to exemplar stays with a human. If the loop’s own critics decide what counts as greatness, the ratchet drifts toward the critics’ taste. That is specification gaming with a friendly face.
3. Test transfer
If the template reproduces a 1983 software house, that may only show it memorised one. The real test is a different world judged against the same bar — a modern company with no CRM in sight, a home-care advocate whose folders hold family context instead of pipeline stages. Same bar. Different costume. If the pairs come out even at a fraction of the effort, you extracted the shape. If they only win when the wallpaper matches, you extracted wallpaper.
→ blind_pairs(exemplar, recreation) + why
→ human_promotes?
→ transfer_world(same_bar)
→ morning_review(keep_useful / drop_speculative)
Farming exhaust into libraries
The same honesty conditions apply when the exhaust is code and routines rather than episodes. Voyager’s ever-growing library of executable skills compounds an agent’s abilities and eases forgetting.5 Agent Workflow Memory induces commonly reused routines from trajectories and feeds them forward to guide later work.6
The discipline that keeps compounding from becoming corruption is gated promotion. Review promotions, not writing. Work on durable agent workflows warns that appending distilled lessons to memory without lifecycle gating lets a single erroneous update silently corrupt future behaviour; governed evolution puts verification and rollback in front of the write.2
A protocol you can run this month
- Choose your current best artefact. Tag it. Snapshot the evidence that proves what it was.
- Write down what it cost — hours, dollars, plan fraction, whatever unit you actually pay in.
- Attempt one recreation under a spend cap and an absolute end time.
- Run blind pairs with cold-arrival readers. Capture the why.
- Decide promotion yourself. If you promote, freeze the new bar and its new cost.
- Schedule a transfer test on a different world before you declare the shape extracted.
- Morning-review every unattended run. Keep useful. Drop speculative.
Key insight
Cost is part of the definition of quality for extraction. A bar you cannot price is a bar you cannot beat on purpose.
Who applies pressure against that bar while the work is moving? Not a peer-strength reviewer writing polite notes nobody reads. A stronger outsider — and, when the work is shared, a cold-arrival reader in a customer’s voice. Chapter 6 names both roles.
The Outsider Critic and the Cold-Arrival Reader
Harsh review becomes useful when the reviewer is stronger, blunt by licence, aimed at the north star, reading as-run output from outside the author’s context, holding no commit authority — and entertaining enough that humans actually read it.
Chapter 5 put a frozen bar on the table. Someone still has to press on the work while it moves. In the Venture World harness that someone was nicknamed Gilfoyle — hard, straight-in-your-face, stronger model, and, crucially, fun. The nickname is a blunt TV-character licence. The portable role name is the outsider critic. This chapter uses both on purpose.
Six properties that transfer
Definition — outsider critic
- Stronger than the author. Most review loops hire a peer or something weaker.
- A persona that licenses bluntness. Naming the character removes the default politeness tax.
- Judges the north star, not the code. Drama and aesthetics are in scope when the brief says so.
- Reads what actually ran, from outside the author’s context. Output, not intentions. No sunk cost.
- No commit authority. Advises; the author decides. Harshness is safe because it cannot ship.
- Entertaining enough to be read. A dutiful report gets skimmed. A reviewer you look forward to keeps the human in the loop.
Strength without bluntness produces careful notes that change nothing. Bluntness without strength produces theatre. North-star judgement without as-run evidence produces arguments about intent. As-run reading with commit authority produces fear — and fear makes authors hide work from the reviewer. Remove any one property and the harness softens into something that looks like review while behaving like reassurance.
Treat entertainment value as a governance feature.
That sixth property is easy to dismiss as personality. It is not. Governance that nobody reads is not governance. The outsider critic has to clear the same bar as any other control: it must actually be used.
The no-commit rule is the old chairmanship pattern in miniature — advise without the merge button. One clause is enough here. The point is mechanical: severity is tolerable when it cannot unilaterally rewrite the work. The same authority instinct appears when self-improvement is aimed at the scheduler rather than the gates: change how exploration spends, leave the evaluator and interfaces fixed.3
Where to aim the harsh one
Aim Gilfoyle — the outsider critic — at your own work: the episode, the ebook, the doctrine page you are about to publish under your name. That is the clean use.
Pitfall
Do not run the harsh outsider over a colleague’s shared website and call it process. That automates the exact interpersonal argument many teams are trying to stop having.
The cold-arrival reader
Shared work needs a different role. The cold-arrival reader arrives with no context on purpose, often in a customer’s voice. Example shape: I’m a nurse who was made redundant last month; here is what I understood from your homepage and where I stopped reading.
That is evidence about a reader, not your opinion. It depersonalises critique. A brand person can work from a transcript. A co-founder can argue with a reader’s confusion without having to accept your taste as the court of appeal.
Cold-reader protocol
After about twenty seconds with the page, ask:
- What does this company do for you?
- What would you do next?
If they cannot answer, that finding comes from the reader — not from your need to win an edit war. This is also the outcome dimension that is hard to game when you later tune against a human exemplar (Chapter 8).
Stand-up notes
| Role | Brief must say |
|---|---|
| Outsider critic / Gilfoyle | Stronger model; judge north star not code; read as-run artefacts; no merge rights; persona that permits bluntness; schedule cadence |
| Cold-arrival reader | Fresh context; named customer persona; two questions; time box; produce evidence transcript, not a rewrite |
Independence inside company agent teams
The same stronger-model outsider eventually belongs inside company boxes. When the reviewer shares both model and evidence with the producer, you have a closer without much independence. An hourly question — would this customer feel looked after? — restores a gap the current arrangement quietly deletes. Chapter 10 returns to that surface as an application.
You now have a bar and named critics. The trees you already paid for can still teach a further lesson: not how to rewrite the gates, but how to spend exploration budget. That requires reading a recent paper carefully enough to reject the misread that first suggests itself. Chapter 7 does that correction.
A Finished Run Is a Simulator
Read Dream-RSI carefully: it does not vary the whole harness or reverse-engineer prompts from known answers. It improves the exploration policy by treating completed discovery history as a replay simulator.
After a great run, a tempting sentence appears: now that we know a good result, vary the harness until we can get there cheaper. That sentence is close enough to be useful and wrong enough to send you into the wrong tool. It was also the first summary of Dream-RSI in the source conversation — and it was corrected against the paper mid-thread. This chapter keeps the correction.
Correction
Dream-RSI does not vary the whole harness, and it does not ask whether you could have prompted differently once you know the answer. It improves an explicit exploration policy — which branches to pursue, how many to run in parallel, when to stop — while leaving the underlying coding agent unchanged.3
History as a replay simulator
The key move is almost insultingly practical. A completed discovery history already records a tree of attempts and outcomes. Organise that history, and it becomes a replay simulator: candidate exploration policies can walk the saved tree, choose different subsets of branches, different parallel groupings, different stopping points, and read outcomes that were already paid for. One expensive online run can support many cheap offline evaluations.3
Only the exploration-policy code changes; the underlying models, evaluator, and execution interfaces remain fixed.
That sentence is the authority rule for self-improvement in this book. Lessons may change the scheduler. They do not get to rewrite the gates.
Hard limit: replay only reveals what was recorded
A simulator made of history cannot tell you what a never-generated prompt would have produced. It can tell you that you should have stopped a branch earlier, or run more readers and fewer builders. It cannot invent branches nobody grew. If you need a different machine, you need a different method — Chapter 8’s human-exemplar tuning sits closer to whole-harness improvement than Dream-RSI does.
You already own the trees
Overnight worktree trials are discovery trees. Hundreds of Venture World commits are nodes with saved workspaces. The paper’s trees arrive with scores; yours may not. You can add scores after the fact with pairwise-against-exemplar checks (Chapter 5). Once nodes are scored, you can evaluate policies offline on a night you already paid for.
Offline policies worth evaluating
- Stop after three outsider-critic reviews with no new contradictions.
- Schedule cold-arrival readers every N commits, not every commit.
- Stop-or-escalate when retries burn past a fixed dollar threshold.
The smallest honest cousin already on file is a sales rehearsal that burned about nine dollars on retries before escalation existed. A stop-or-escalate policy evaluated offline over recorded runs is farming exhaust made rigorous — not as inspiration, as an experiment you can actually run.
Lessons belong in the scheduler
Dream-RSI tested a natural alternative: summarise history into directional hints and inject them into the explorer’s prompt. Explicit directional guidance consistently underperformed having no such guidance. Strong semantic priors over-constrained parallel exploration.3
Myth vs reality
Myth: Stuffing history into the explorer prompt is how the system learns.
Reality: In long-horizon parallel discovery, that move can shrink the search. Put lessons in the scheduler — which branches, how many, when to stop — and keep the explorer freer. This matches steering by gradient as a provocation, not a specification (Chapter 3).
Two limits on that result, stated plainly. The guidance analysis sits inside the paper’s discovery setting (the figure is reported on a kernel-engineering task). And discovery is not delivery. When you are serving a particular client, context still helps. Do not universalise “no guidance” into “never give the agent the folder.”
The learned policy got frugal
In one reported trajectory, the exploration policy reduced evaluated attempts per round from 110 to 50 while scores kept improving, then spent more again when progress plateaued.3 That is the spend cap the 10am accident lacked — expressed as policy rather than as a human watching the bill in horror.
Not the same tool as Meta-Harness
Whole-harness search is a different line of work. Meta-Harness treats the harness code that stores, retrieves and presents information as its own optimisation surface, searched by an outer loop with access to prior candidates’ code, scores and traces.7 Keep the distinction clean: Dream-RSI teaches you how to explore trees you already grew; harness tuning against a gold exemplar changes the machine that grows them.
Authority rule
Self-improvement touches the scheduler. It never rewrites the gates.
When the gold is not a scored discovery tree but a human expert’s hand-made pages, you are in that second tool. Chapter 8 builds it.
Gold from a Human Hand
A human expert’s hand-made work is not a detour to automate away. It is the training signal — if you judge by pairs, not by wording, and decide holistically after the rubric has done its diagnostic job.
Chapter 7 left a fork in the road. Replay teaches you how to spend exploration over trees you already grew. Sometimes the better idea is different: take pages a human expert has already hand-made for a live lead, and tune not only prompts but the harness against that gold. Quarantine the names. What travels is the role: a human expert’s hand-made exemplar.
This is supervised harness tuning with a person’s taste at the top of the ratchet. The expert stops being the person who must do every instance and becomes the person whose judgement the system learns. That solves a technical problem and a capacity problem at the same time.
One exemplar teaches the instance, not the person
One exemplar teaches the instance, not the person.
Tune on one gold set. Test on a second the same expert has also hand-done. Better still, show a blind pair for a new case and see which one they pick. If you treat a single win as proof you have captured a whole human, the ratchet will flatter you until the next lead arrives.
Never measure similarity to the text
Wording will never converge. That is not a bug in the harness. It is not the idea. Diffs that chase lexical overlap train the system to imitate surface. Ask instead: what did the human do that the harness did not? That answer is your next weakness-and-delight gradient (Chapter 3). It is also the only kind of comparison that survives when two pages are a hundred percent different on purpose.
Dimensions for pairs, not scores out of ten
Keep a dozen dimensions if you need them — readability, benefit clarity, customer language, image use, engagement, brevity versus explanation, craft. Use them comparatively. On customer language, which page wins, and why? Absolute 1–10 scores across two pages will mostly be noise; the Venture World record already taught that lesson. A win-rate profile per dimension is steadier, and the whys steer.
Mine criteria from the exemplar; confirm by recognition
Your first list of criteria will reflect your taste. The expert’s taste is in the pages. Have a model describe what the gold does that a generic generated page does not. That produces candidate criteria. Then get the expert to confirm and weight them by showing pairs and asking which they prefer and why.
If you ask cold for a scoring list, you are asking for recall. Many experts work by recognition. Pairs meet them where they already are.
The dimension you cannot stand
Fonts. Colour. Craft flourishes you personally dislike. That distaste is a signal. It often marks tacit expertise you do not have — which makes it the most valuable thing you could capture. Judge it on rendered screenshots with a vision model, not on the HTML source. Your eyes were never going to be fair to it anyway.
Add the dimension that is not about the page
Every craft checklist can still miss the outcome. Add a cold-reader test (Chapter 6): after twenty seconds, what does this company do for you, and what would you do next? It is the closest proxy many teams have to the real reader outcome, and it is hard to game with prettier margins.
Diagnose with the rubric; decide holistically
Pitfall
If you optimise against a twelve-point rubric, you will get pages that score well on every point and are bland. Use the rubric to diagnose. Decide between pages with a single question: which would you rather receive?
Families with different owners
Definition — rubric families
- Craft — visual and verbal making; often owned by the human expert whose pages set the bar.
- Doctrine — whether the page carries the company’s actual commitments and level of offer; owned by whoever holds that doctrine.
- Care — whether the experience is safe, true, non-manipulative and fit for people who may be vulnerable; owned as a first-class family, not as an afterthought.
A page has to win across families, not average them into mush. Splitting ownership is also how you stop one person’s taste from silently vetoing another person’s responsibility.
A worked cycle (shape, not invented scores)
- Freeze the human pages as gold. Note the effort that went into them.
- Mine candidate criteria from what the gold does that a generic page does not.
- Run a short pairwise confirmation session with the expert (Chapter 9).
- Assign each surviving criterion to craft, doctrine or care.
- Let the harness produce a challenger under a spend cap.
- Blind pairs per dimension; capture whys as gradient fuel.
- Ask the holistic question: which would you rather receive?
- A human promotes or rejects. If you promote, freeze the new bar and its cost.
Report the cost ratio when you have one — expert hours versus harness hours to a blind-pair tie. Until you have a measured ratio, state the test shape and refuse to invent a percentage. Honesty about missing numbers is part of the ratchet.
None of this is durable if the expert discovers they were a silent training source. Chapter 9 is the consent rule that keeps the ratchet from eating the relationship.
Tell the Expert
Silent farming of a colleague’s exhaust works technically and costs the relationship. The durable move is a small, flattering protocol that makes them co-author of the engine.
Chapter 8 built the tuning cycle. It assumed access to gold. Access is not the same thing as consent.
The temptation is obvious. The pages are already there. The engine needs a bar. Why interrupt someone who is busy making excellent instances? Silent farming will produce a technically usable signal. It will also turn creative abrasion into extraction the moment they find out their work was a training target they were never asked about.
Pitfall
Technical success is not social success. A ratchet that eats trust is not a company asset.
The ask
Your pages are the best thing we’ve got. I want the engine to hit that standard. Give me fifteen minutes a week picking between pairs and telling me why.
That is the whole protocol in three sentences. It is flattering because it is true when the pages are good. It is small because recognition is dense per minute. It makes the expert a co-author of the engine rather than a source they never agreed to be.
Why pairs fit how experts work
Many people who hand-make excellent artefacts are weak at cold lists of scoring criteria and strong at choosing between two things in front of them. Chapter 8 already preferred recognition over recall for that reason. The consent ask should match the judgement style you need: not “write me a rubric,” but “pick, and tell me why.”
Fifteen-minute session script
- Show three to five blind pairs (gold vs challenger, or two challengers).
- On each pair: which would you rather receive, and why?
- One holistic pick across the set if time remains.
- Capture the whys as gradient fuel; do not argue them away in the room.
- Close by showing one improvement from last week so the loop is visible.
What you are offering — and not asking
| Offering | Not asking |
|---|---|
| Co-authorship of the engine | A cold criteria essay |
| Their taste captured without replacing them | Unlimited review labour |
| Proof they can see and steer | Permission for you to remote-edit their public pages |
If they say no
Then you do not have a moral licence to tune on their work. Hand-make your own exemplar first, or wait. The conversation that corrected a false assumption about existing gold was blunt on this point: when nobody has done the thing by hand yet, there is no gold to tune against. Inventing consent does not invent an exemplar. Even outer-loop harness search still needs a scoring signal from somewhere; human pairwise picks are that signal when the gold is craft, not a benchmark score.7
Failure modes beside silence
- Aiming the Gilfoyle outsider critic at their shared work (Chapter 6’s aiming rule).
- Sliding back into wording-convergence metrics after promising pairs.
- Never promoting, so their fifteen minutes feel like theatre.
- Expanding the meeting into the interpersonal conflict the consent rule was meant to avoid.
Broader co-founder dynamics, resentment, and edit wars belong in another volume. This chapter’s job is narrower: without the ask, the human-exemplar ratchet is extractive. With the ask, it is a partnership mechanism.
Co-author vs source
The silent route makes someone a source. The pairwise route makes them a co-author of the engine. Only one of those compounds trust.
Consent, rules, ratchet, outsider roles, replay reading — pack them into something you can run next week. Chapter 10 is the field template, and the three surfaces where the same geometry goes next.
The Extraction Template
What you can run next week without the author in the room: take a finished run apart, freeze a costed bar, stand up named critics, recreate under a spend cap, judge in blind pairs, transfer worlds, and tune against human gold only with consent.
Chapters 1–9 argued a single claim along one route: from a recorded overnight astonishment to a reusable shape. This chapter is the operating artefact. It assumes the doctrine landed. It does not re-derive it.
Field checklist
The extraction template
- Pull one finished run. List the eight ingredients as they actually appeared: exact north star, weakness and delight; critic fragments you truly have; cold-reader passes; quality ownership; gates; clock. Do not invent brief text you cannot find.
- Freeze the current best artefact. Tag the commit. Snapshot as-run evidence and pictures. Write the cost beside the quality.
- Stand up two roles. Outsider critic / Gilfoyle on your work. Cold-arrival reader for shared work. No commit authority. Make the critic readable enough that humans will actually open the reviews.
- Recreate under constraint. Absolute datetime. Spend cap. Morning review that expects junk.
- Judge in blind pairs. Which would you keep / rather receive, and why? The why updates the gradient.
- Test transfer. Different world, same bar. If you only win when the costume matches, you memorised a costume. Report a cost ratio when you have measured one; until then, state the test shape and invent nothing.
- If human gold exists: make the consent ask; mine criteria; confirm by pairs; assign craft / doctrine / care; decide holistically; only humans promote.
- Refuse loop self-certification. Scheduler may learn. Gates do not rewrite themselves.
+ frozen_costed_exemplar
+ outsider_critic (no merge)
+ cold_arrival_reader
+ blind_pairs + why
+ transfer_test
+ morning_review
+ human_promotion
= exemplar_ratchet
Where the shape goes next
The same geometry is not limited to simulated 1983 studios.
1. A coherence loop over a company box
Point the contradiction hunter at drift, not at fluency: what the site promises, what the sales documents say, what the ledger shows, what sits in the folders. Catch mismatches unattended. Do not use the hunter to make ambition-stated-as-fact read more smoothly (Chapter 4).
2. An outsider critic inside company agent teams
When the reviewer shares model and evidence with the producer, independence is theatre. An hourly outsider asking whether the customer would feel looked after restores a gap. Same properties as Chapter 6: stronger, blunt enough to be useful, north-star not code-nit, as-run evidence, no commit authority, readable.
3. Self-improvement as a core mechanic
Customer outcomes arrive slowly and sparsely. Start with proxies the boxes can already compute:
- First-pass approval rate
- Escalations per matter
- Cost per closed responsibility
- Human decisions required
Evaluate stop-or-escalate and spend policies offline over runs you already paid for (Chapter 7). The nine-dollar retry burn is the smallest honest shape of that idea: not a slogan, a recorded failure you can replay policies against.
Authority rule, final form
Self-improvement may change how you explore and when you stop. It does not get to rewrite the gates that keep promotion honest.3
What “done” looks like for a reader
Key takeaways
- Keep the shape of a great run, not the steps.
- Write the gradient: north star, named weakness, named delight.
- Know what your loop optimises — coherence is not truth.
- Freeze a costed exemplar; judge in blind pairs; humans promote; transfer to another world.
- Pair the outsider critic with the cold-arrival reader; aim them correctly; tell the expert.
The claim, one last time
A loop improves only in directions you can name, and only against a bar you hold still. The overnight miracle was never the product. The product is a harness that can find the next one on purpose — cheaper, in a world you have not built yet — because you named the gradient, froze the bar, and refused to let the loop certify itself.
References & Sources
The evidence base behind every claim — primary research, industry analysis, and technical specifications
Research Methodology
This ebook draws on primary research from standards bodies, independent research firms, enterprise technology vendors, and consulting firms. Statistics cited throughout have been cross-referenced against primary sources.
Frameworks and interpretive analysis developed by Scott Farrell / LeverageAI are listed separately below — these represent the practitioner lens through which external research is interpreted, and are not cited inline to avoid self-promotional appearance.
Primary Research & Standards Bodies
Genrikh Altshuller / Technical Innovation Center — And Suddenly the Inventor Appeared: TRIZ, the Theory of Inventive Problem Solving [1]
TRIZ as systematic inventive problem solving — simile only for pattern→generalise→feed back
https://archive.org/details/andsuddenlyinven0000alts
El Hattami, Chapados, Pal, arXiv:2606.08049 — SKILL.nb: Selective Formalization and Gated Execution for Durable Agent Workflows [2]
Lifecycle gating before memory/skill promotion
https://arxiv.org/abs/2606.08049
Zheng et al., arXiv:2609.14858 — Dream-RSI §5.1 [3]
Directional prompt guidance underperforms in parallel discovery
https://arxiv.org/html/2609.14858v1
Liu et al., arXiv:2403.16950 — Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators [4]
Pairwise preference vs direct scoring for LLM evaluators
https://arxiv.org/abs/2403.16950
Wang et al., arXiv:2305.16291 — Voyager: An Open-Ended Embodied Agent with Large Language Models [5]
Ever-growing skill library of executable code compounds abilities
https://arxiv.org/abs/2305.16291
Wang, Mao, Fried, Neubig, arXiv:2409.07429 — Agent Workflow Memory [6]
Induce commonly reused workflows from trajectories
https://arxiv.org/abs/2409.07429
Lee et al., arXiv:2603.28052 — Meta-Harness: End-to-End Optimization of Model Harnesses [7]
Harness code as optimisation surface, not only model weights
https://arxiv.org/abs/2603.28052
About This Reference List
Compiled September 2026. All URLs verified at time of compilation. Regulatory documents and standards specifications are subject to revision — check primary sources for the most current versions.
Some links to academic papers and vendor research may require free registration. Government and standards body publications are freely accessible.