Code as Proxy

The most consequential thing AI learned to do this year wasn't writing software. It was writing the machinery through which other things can be built — and then changed, one piece at a time, while everything else stays exactly where it was.

You can see it if you know where to look. The week I started this book, a coding model's three-dimensional ocean simulator — the waves, the light, a boat moving through it, all of it code — was shown off to a large audience under the line "3D simulation is solved".¹ Most people received it as a model trick: look how good the new model is at graphics. I'll tell you what I actually saw in it in Chapter 18. For now, the short version is that it wasn't a trick. It was a medium arriving in plain sight.

That medium is code. Not code as in "the app we're building", but code as the thing an AI writes when it wants an idea to behave — to have physics, or timing, or rules, or a layout — and to keep behaving the same way the next time you run it. Once you see that, "AI is good at coding" stops sounding like news about programmers. It starts sounding like news about everything.

The question this book answers

Here it is, plainly: why does AI's coding ability matter for work that isn't software — and what limits improvement once everything can be built as code?

Both halves matter. The first half is the opportunity, and it's bigger than most people have noticed. The second half is the catch, and it's more interesting than the opportunity, because it tells you where to spend your effort next. Most organisations are still spending it on the generator — the model, the prompt, the next release. That's the one place the ceiling isn't.

Code as proxy

Think about what "machinery through which knowledge can act" means in practice. There are four versions I keep running into:

• Physics and light in a simulation. A program that holds the boat, the water, the light and the camera as relationships, so that changing one condition changes the others to match.
• Timing, attention and emotion in a film. A composition in which the hold after a line, the order of scenes and the level of the music under the voice are all named values you can change independently.
• The decisions of a gate. A security check on inbound mail whose rules, thresholds and model calls are explicit, testable and replayable.
• The words, layout and behaviour of a page. A site whose structure can be rendered, inspected and regenerated rather than admired and then patched.

None of those is "an app". All of them are code. And in each case what changed is that the output has become something with parts — parts you can point at, measure and alter without starting again. A nicer output is the least of it.

I said it out loud in the middle of a very busy week, and I haven't found a better way to say it since:

"I've just been seeing more and more uses of code, and AI is so good at generating code. Code is becoming like a proxy for everything."

Why "AI is good at coding" undersells it

When people hear that AI is good at coding, they hear a claim about software teams. Fewer developers. Faster features. Cheaper apps. All of that may be true and none of it is the point. It's a category error — like hearing that the printing press was good at making copies of the Bible and concluding it was a story about monasteries.

The advantage of AI code isn't that it's building good code. It's that code has become the medium in which AI can make ideas behave. That's why "AI is good at coding" increasingly undersells the capability. Coding becomes a route into other domains because it supplies the machinery through which domain knowledge can act. Code is becoming an editable medium between intention and experience — not merely a way to build software.

That last sentence is the one I'd underline. A digital interface to users. Once a film, a demonstration or a simulation is built as code, it stops being a finished object you hand over and becomes an interface: to the audience, who experience it, and to the machine, which can inspect it, reason about it and change it. Both sides of that interface are where this book lives.

Where improvement stops

If code makes things cheap to construct and precise to change, you'd expect improvement to become unlimited. Run the loop, get better, run it again. It doesn't work like that, and the reason it doesn't is the most useful thing I learned building with it.

Once making is cheap and controllable, the generator stops being the constraint. Improvement doesn't disappear; it runs into three ceilings, one after the other.

The first ceiling is perception. An improvement loop constructs something, looks at it, reasons about it and changes it. But the looking is done by something — a model reading a transcript, a model watching the video, a script measuring the soundtrack, a person. If that judge can't perceive a dimension, the loop can't improve it. It can only drift — and a faster loop just drifts faster. Part II is about this ceiling, and about how you find out what your judges can and can't perceive.

The second ceiling is the audience. Even a perfect judge only tells you whether the thing was built the way you intended. It can't tell you what a person will do with it. Code lets you make the delivery exact — the same frames, the same words, the same pause, every time. It can't make the response exact. Part III is about keeping those separate: what you intended, what you delivered and what people actually did, and how to turn the differences into the next experiment rather than a story told after the fact.

The third ceiling is attention. Every improvement still passes through a person at some point: someone who asks for it, reads the result, decides whether to trust it. The scarce input stops being tokens or compute and becomes the human attention each completed, trustworthy improvement consumes. Part IV is about that — including a claim I published about effort that I now think is wrong, and why.

Then Part V follows the argument to its product consequence: when you hold a model of a situation instead of a fixed animation, the artefact becomes one way of playing that situation. A film is one path through it. A demonstration, a sales conversation and a training exercise are others.

Films are the specimen, not the subject

A lot of the evidence in this book comes from films — one-minute brand films built entirely with code, for companies in my own portfolio and for one small client. It would be easy to read this as a book about AI video. It isn't.

Strip away the films, the marketing, the client and my own projects, and ask what survives. What survives is this: code makes an intention editable and observable, and once it does, the ceiling on improvement moves to perception, to audience response and to human attention. Films are simply the sharpest instance of that, because a film unfolds in time and lives on attention and emotion — which is exactly what makes it hard to judge. If the doctrine holds for films, it holds with room to spare for the easier cases.

So wherever you see a film in these pages, you can substitute your own artefact: a training module, a product demonstration, an onboarding flow, an explainer, an interface, a model of your operations. Anything whose effect is judged by a person. Part VII does that substitution explicitly, six times.

The first thing code gives you

If code is the medium, the obvious next question is what, precisely, it gives you that a generator doesn't. The tempting answer is "better output". That's not it. Generators produce very good output now, and sometimes better-looking output than anything built with code.

What code gives you is more specific than quality, and more useful. It gives you handles. That's where the doctrine starts.

For a long time I thought AI video worked one way, and I suspect most people still do.

There is still some of that, and it isn't useless. Generators make remarkable things. But look at what the request does to the intention behind it, because that's where the difference lives.

What a prompt does to an intention

Take the moment in a brand film where the promise lands — the line that names what the company will do for you. You want that moment to hit. So you ask a generator to make it emotionally powerful.

That request bundles everything together. The words, the voice, the music, the image, the timing, what the viewer has just seen, what they see next — all of it fused into one instruction and handed to a system that returns one appearance. If the appearance is nearly right, you accept it. If it isn't, your only move is to ask again and hope the new roll keeps the good parts while fixing the bad one. Often it doesn't. The voice you liked changes. The image you liked changes. The pause you were trying to fix comes back different, and so does everything around it.

That's the whole control surface of a generated appearance: accept or reject. I'll call it what it is — re-rolling — and I think it's the default most organisations are still running.

The trouble is how much there is to get across. Here's what I was trying to say about one film, in the order it came out of my mouth:

"Connecting to the audience, providing a wow factor, timing a pause when we hit the brand promise, connecting the emotions to our application, taking the user on a journey — it's just so hard to get that much into a prompt to an LLM video generator. And they might not understand it either."

A prompt is a very narrow pipe for an intention that complicated. And even when the intention gets through, the generator's answer comes back as a single, finished surface. You can't reach inside it and adjust the thing you meant.

The same intention, constructed

Now build the same moment as code. Something interesting happens to the request: it unbundles. "Make it emotionally powerful" turns into a set of steps you can actually state:

1. Show the customer's situation resolving.

   The thing the viewer recognised at the start gets its answer on screen.

2. Let the narrator finish.

   No words on screen competing with the last spoken line.

3. Stop introducing information.

   Nothing new arrives while the promise is landing.

4. Hold the image long enough for the promise to register.

   A beat of stillness — silence, or music under nothing.

5. Only then move to the next step.

   The cut comes after the hold, not during it.

Each of those steps is now something in a file. Which means the machine has handles on the intention. It can change the hold without changing the voice. It can change the music without changing the words. It can change the opening situation without losing the ending. It can try the promise earlier, or later, and leave everything else exactly as it was.

This is the move the whole book depends on, so it's worth saying precisely. The important shift is from asking AI to produce an output to asking it to construct something it can run, inspect, intervene in and improve. Four verbs, and a generated appearance supports none of them.

The creative idea has become something you can intervene in, rather than something you can only accept or reject.

The medium is already here

None of this is hypothetical, and it doesn't need exotic tools. The engine I've been using for films is HeyGen's open-source HyperFrames, and its authoring model makes the point better than I can. Its documentation says: "Define a video as HTML. Add data attributes for timing and tracks." And the renderer "seeks each frame in headless Chrome and encodes the result with FFmpeg, so the same input produces the same video."²

Read that the way a builder should. In a composition like that, every clip has its start time, duration, track and volume written down as plain attributes. The hold after the promise is a number. The level of the music under the voice is a number. The order of scenes is the order of elements in a file. The handles from the list above aren't a metaphor — they're literally fields.

HeyGen's own engineers explain why they built it that way, and it's worth hearing, labelled as the vendor's account of their own choice. "Creating video is hard, but LLMs are great at writing code specifically HTML." They tried a more structured framework first and found that the more guardrails they added, the safer and more repetitive the output got: "When we dropped back to plain HTML/CSS/JS, the creativity came back."³ The medium was shaped around what agents can author well. That matters, because it means the handles are in a language the machine is fluent in.

Two honest qualifications. First, generated assets haven't gone anywhere. The stills in my films were generated; the voices were synthetic. They sit inside the construction, and the advantage is that you keep control over how they relate — which still appears when, under which line, for how long. Second, "deterministic" is an engineering property with conditions, not magic: HeyGen notes that its fully deterministic capture path runs on headless Linux, with fallbacks on other platforms. Same input, same frames — where the setup supports it.

"But a generator can iterate too"

This is the objection I'd raise if I were reading this, so let me take it seriously. You can absolutely put a video generator in a loop. Generate, look, adjust the prompt, generate again. People do it every day, and the results improve.

So the difference isn't the loop. What code adds is not the existence of a loop, but more precise control over what changes and what stays the same.

When you iterate a generator, each pass can change everything. The new version might be better — but you can't say why, because the voice moved, the framing moved, the timing moved and the image moved, all at once. You got a better output and learned nothing you can carry to the next one. When you iterate a construction, you change one handle and hold the rest. If the new version is better, you know what made it better. Chapter 4 turns that into an experimental discipline; for now, notice that it's the precondition for learning anything at all.

Behaviour that stays editable

Put the pieces together and you get the formulation I'd hold onto for everything that follows:

"Behaviour" is the important word. A film has timing and progression: what the viewer knows at second ten depends on what they saw at second five. A simulation has dynamics: what the water does depends on the wind and the hull. A page has states: what the reader sees depends on what they clicked. A generated appearance has none of that in a form you can reach. It's a picture of behaviour. A construction has behaviour, exposes it, and lets you change it selectively.

Relationships, not just pictures — and what that doesn't prove

This is also why a code-built simulation is a different kind of object from a generated clip, even when the clip looks just as good. A rendered frame shows a result. A program keeps the relationships between the boat, the water, the light and the camera, so changing one condition — a stronger wind, a lower sun — produces corresponding changes everywhere else. That's a manipulable model rather than an impressive appearance.

And here I want to refuse a tempting overclaim, because the whole book depends on not making it. Keeping relationships in code does not make them right. Researchers building physics benchmarks put it bluntly: current models "frequently produce code that runs but simulates the wrong physics."⁴ Beautiful water and accurate water are different claims. What code gives you is something another group of researchers describes precisely: "the inferred world representation is directly inspectable, editable, and falsifiable."⁵

Inspectable, editable, falsifiable. That's the honest gain. Code doesn't make a thing true. It makes it testable — which is why most of this book is about testing, and about what the tester can and can't see.

One guard, planted early

Handles are powerful enough to be misused. Once every creative decision is a parameter, it's tempting to specify all of them — to write the film as a list of values and call that direction. That caps the result at the first idea of whoever wrote the list. In Demonstrate Over Tell I argued for specifying facts and nudging taste — naming what the audience needs and leaving the machine room over how to express it. That still holds. Handles are for experiments and repairs: for changing one thing on purpose and learning from it. Chapter 17 comes back to this properly.

What you do with handles

A handle on its own is just a knob. What makes it valuable is a process that uses it well: one that builds the thing, looks at what it built, thinks about what it sees and changes one handle at a time. That process has four moves, and our own earlier work turns out to have contained most of them already — pointed in a different direction.

The first of those arguments was about code pointed inward, at the machine's own thinking.

Code as the machine's inward apparatus

In The Generative Pendulum I argued something that surprised me when I first worked it through: better AI produces more deterministic code, not less. Deterministic code in a hybrid system was never rationed by need. There was always more of it worth having. It was rationed by authoring cost — the decision, the implementation, the test, the review, the maintenance obligation. Drop that cost to seconds and the number of small, narrow components shoots up, the lifetime of each one collapses, and most of them become disposable instruments fitted to the question in front of you. "AI makes deterministic computation disposable, abundant, and dynamically fitted to the current thought."

The deterministic code in that argument was doing something specific. In The Agent's Retina I described it as the agent forging a probe — a query, a filter, a small script — so that thirteen gigabytes of mess becomes a dozen patterns a model can hold. The code didn't solve the problem. It "reshaped the world into a representation the intelligence can reason about."

That was code pointed inward — at the machine's own cognition. It made the world easier for the model to think about.

Code as an artefact you can inspect

The second argument pointed the other way, and it started with something small. In The Skeleton of a Visual I compared two ways of making quote cards for articles. One asked an image model to produce the card directly. The other had a model write styled HTML, which we rendered and photographed. On first-sample appeal, the direct image often edged it. The HTML path came out "almost as good".

But the HTML path won the thing that mattered, because we owned its structure. We could flatten what it produced back into its skeleton, check that every word survived and nothing clipped, and regenerate it cleanly when it failed. A directly generated image is a dead end for that kind of checking. The line I'd still stand behind: "'Almost as good' but fully inspectable beats 'slightly better' but opaque, every time you have to run it a thousand times unattended."

That also answers an objection I hear often about code-built media — that it looks a bit worse than what a generator produces. Sometimes it does, on the first sample. Judge it on the whole improvement process instead of the first sample and the comparison flips, because only one of the two can be improved on purpose.

That was code pointed outward — but at a static object. A card. Space, not time.

The convergence: outward, into time

Films push both arguments somewhere new. A film isn't a card; it unfolds. What the viewer understands at the moment the promise lands depends on what they saw a few seconds before. Whether a pause works depends on what came just ahead of it. Films carry the same logic from spatial arrangement into time, attention and emotional progression.

And once a film is built as code, both earlier ideas apply at once. The construction is an artefact people experience — pointed outward. It's also something the machine can reshape into representations it can reason about — frame sheets, timelines, measurements — pointed inward. Code turned outward into experiences that unfold in time, and still legible to the machine that made them. That's the convergence, and it produces a loop.

The loop

Put plainly: AI constructs the experience, constructs a useful view of the experience, reasons about the result, and changes the construction. Then it goes round again.

I noticed this from the builder's side before I had the words for it. The films were impressive, but that wasn't what excited me:

Really understand what it's doing. A generated video can be watched. A constructed one can be watched, read, measured and diffed — by you and by the machine. The path is the point.

The move everyone skips: the view

Most "improvement loops" I see are really construct → score → change. The scoring happens on whatever representation was lying around — often a transcript, or a single screenshot, or the model's own description of what it made. The view never gets designed. It's treated as a by-product of the output rather than an instrument in its own right.

That's a mistake, and the medium is on your side in fixing it. HyperFrames' renderer never simply plays a video; it seeks. In HeyGen's words, the renderer "calls seek(0), screenshots, seek(1/30), screenshots, seek(2/30), screenshots", and the preview uses the same mechanism: "When a user scrubs the timeline, the preview iframe calls seek. When the renderer captures frame 147, it calls seek(147 / fps). Same code path. Same output."³ A film you can seek is a film a model can inspect moment by moment — at the exact instant a line lands, not at whatever frame a sampler happened to grab.

Researchers building educational video with code arrived at the same shape independently. Their system has a planner and a coder that construct the video, and a critic — a vision-language model — that looks at the render and sends corrections back into the code. Their reasoning is exactly this chapter's: the requirements of such videos "are better addressed through the manipulation of a renderable environment, which can be explicitly controlled via logical commands (e.g., code)."⁶

The same breath at every scale

If this loop feels familiar, it should. In Cycle Compression I described a pattern I kept seeing at every scale of AI-assisted work: abstraction descends until it hits reality, reality pushes back with evidence, intelligence rises enough to reinterpret, then descends again with a better instrument. "Pure LLM without contact is plausible talk. Pure determinism without interpretation is row dumps. The freakish capability is the ping-pong: AI narrows the question; determinism narrows the world; together they stay in contact with reality."

Construct, view, reason, change is that breath applied to an artefact. And it's one of the reasons so much of my work has been drifting into code:

"I'm doing more and more auto-research loops with AI, and that works great in code. So that's another reason why everything's turning into code, because it's so good at those sort of loops."

The strongest outside parallel

The clearest external version of this loop comes from mathematics and computer science, not marketing. DeepMind's AlphaEvolve "pairs the creative problem-solving capabilities of our Gemini models with automated evaluators that verify answers, and uses an evolutionary framework to improve upon the most promising ideas."⁷ Models propose programs. Evaluators run and score them. The scores decide which programs seed the next round. The gains come from the combined search-and-evaluation process, not from trusting the first program a model writes.

I'm not claiming a brand film has the truth conditions of a sorting algorithm. It doesn't. The parallel is structural:

An executable candidate gives the search something concrete to work on.

A generated appearance gives the search nothing to hold. A construction gives it a thing with parts, and every pass can move a part.

Where the loop stops

There's a sentence in DeepMind's own paper that I think is the most important line in this chapter, and it's about the limit rather than the achievement. Relying on an automated evaluation metric is AlphaEvolve's key advantage, the authors write, but "it is also a limitation—in particular, it puts tasks that require manual experimentation out of our scope." And: "While AlphaEvolve does allow for LLM-provided evaluation of ideas, this is not a setting we have optimized for."⁸

Read that carefully. The loop is only as good as the thing doing the viewing and the reasoning. Where a metric can check the answer, the search is powerful. Where the evaluator has to be a model's judgment — or a person's — the search is only as good as that judgment. Films, pages, demonstrations and training modules live on exactly that side of the line. So will most of what you want to build this way.

Part II is about what happens there. But before the ceiling, there's one more thing the loop makes cheap, and it's the thing that turns iteration into learning: holding everything still and changing one thing.

Two ways to improve a film, and they cost the same. The first produces a new artefact. The second produces knowledge.

That difference used to be expensive to act on. Changing one element of a traditionally produced film meant going back into the edit. So most teams — reasonably — did the other thing: made a new version, compared it to the old one and kept the winner. With the film as code, the informative version costs no more than the uninformative one. It's an edit to a value.

The variations code makes nearly free

Here's what "change one thing" looks like when the film is a construction:

Each of those is an attribute edit, not a regeneration. And each asks the film a question: does this particular thing matter, for this audience, at this point?

Why holding the rest is the whole point

The people who run online experiments at scale have been clear about this for a long time. If an experiment is designed and run properly, "the only thing consistently different between the two variants is the change between the Control and Treatment, so any differences in the OEC are inevitably the result of this assignment, establishing causality."⁹ (The OEC is their overall evaluation criterion — whatever you've decided success means.)

Be precise about which half of that code gives you. It gives you the discipline on the stimulus side: two versions that differ in exactly one respect, for the price of an edit. It does not give you the other half — randomising who sees which version, and measuring what they do. That's the audience's side, and Part III deals with it. Keep the two halves separate in your head, because a lot of confident claims come from pretending the first half is the whole thing.

Controlled descendants, now as a diff

In Semantic Experiment Graph I argued against what I called mutation theatre. After a post or campaign does unusually well, the instinct is to make more of whatever it looked like — more of the same image style, the same hook, the same kind of subject. That's how a lucky measurement becomes a brand costume. The alternative was a controlled descendant: a new stimulus that holds the concept, the carrier or the structure steady and varies one class, specifically to hurt a wrong explanation. "Mutation theatre reenacts winners. Controlled descendants try to kill wrong explanations." And the rule that goes with it: "If the brief cannot name what it is trying to hurt, it is not an experiment. It is content."

The same book asked for one more discipline before anything ships: name the stimulus twice, once for its semantic composition — the idea — and once for its surface composition — the presentation. Otherwise "every stakeholder invents a different hero after the fact", and one number produces "three incompatible 'learnings'".

For posts on a professional network, a controlled descendant was a new post you had to write and schedule, and the surface composition was something you had to describe. With media built as code, both change character. A controlled descendant is a diff against the parent composition. And the surface composition is explicit by construction, because the composition is the code. You don't have to remember what you changed. The file tells you.

Two products of a good iteration

This is the part that changed how I think about the whole process.

And then learn. A good iteration doesn't just leave a better film behind. It leaves something else as well.

A good iteration should produce two things: a better artefact, and a better account of what deserves to be retained.

The difference between those two accounts is the difference between a learning system and a template factory. Here's what it looks like in practice:

The account is longer, and it's conditional, and that's what makes it useful. It says what worked, why it's believed to have worked, and under what conditions. A template says what to repeat. The template is how organisations end up with a house style that everybody follows and nobody can justify.

The media descendant brief

Put together, those ideas give you a short brief to fill in before any variation is built. If you can't fill it in, you're about to make content rather than run an experiment.

Most of that brief takes a minute to fill in. Line six is the one that stops people. If you can't name an instrument that can perceive the change you're making, don't run the variation yet — you'll get a result and it won't mean anything. That line is the subject of the next four chapters.

The catch

Here's the uncomfortable consequence of everything above. A variation is only informative if something can perceive the difference. So the cheaper variation gets, the more it matters what's doing the perceiving. Code drives the cost of asking a precise question towards zero — and in the same motion, it moves all the difficulty onto the judge.

I found this out the direct way. I changed one pause in a film — one number in one file — and handed the versions with and without it to my judges. What happened is in Chapter 19. It's not flattering to my setup.

A note on restraint

Holding everything and changing one thing is an experimental discipline. It isn't a creative method, and it shouldn't become one. If every expressive choice in a film turns into a parameter someone set, you've swapped a generator's blandness for a spreadsheet's. Keep the handles for questions you actually want answered — the ones where the answer would change the next film. Let the rest stay loose. Chapter 17 comes back to where that line sits.

That's the proxy, as far as it goes: a medium with handles, a loop that looks before it changes, and variations precise enough to teach. Everything in it depends on one thing that I'd assumed rather than checked — that the loop can see what changed. That assumption is where improvement stops first.

Here's the rule, and I'll spend this chapter showing why it's structural rather than a gripe about this year's models.

"Reliably" is doing work in that sentence. A loop can still drift in the right direction on a dimension its judge can't see — by luck, or because that dimension happens to move with something the judge can see. What it can't do is steer it. And the loop will go on producing confident scores the whole time, because nothing in it knows what it's missing.

Chapter 3 ended on DeepMind's own statement of where their search stops: where automated evaluation stops. Creative work lives on the far side of that line. There, the evaluator is a model's judgment or a person's, and what that judge can perceive becomes the ceiling on everything upstream of it.

The wrong judge

The failure mode is easy to describe and hard to notice from inside:

A thousand iterations against the wrong view may simply produce something increasingly optimised for the wrong judge.

Cheap iteration makes that worse, not better. When each pass is expensive, you look closely at the result before running another. When each pass is nearly free, you let the loop run, and the loop goes wherever its judge's blind spots allow.

Anthropic's engineers saw a version of this building a harness that generates and evaluates designs. Scores improved over the iterations, but the relationship wasn't clean: "I regularly saw cases where I preferred a middle iteration over the last one."¹⁰ The score and the person had come apart. That's what the wrong judge looks like from the outside: the number goes up and the thing you actually wanted doesn't.

I have my own version of that, with dates and numbers, and it's less polite. When I tested my film judges against my own verdicts, the judge that was given the transcript preferred a voice I'd rejected every single time — and the judge that could actually listen missed a pause I had cut out of the film. The full record is in Chapter 19. For now, hold onto the shape of it: neither judge was stupid. Each was blind to something specific, and neither said so.

We'd already seen this, at a smaller scale

The pattern isn't new to our own work. It turned up first with a static picture.

In How to Read a YouTube Video I told the story of a quote card whose attribution line had slid up and overlapped the quote itself — obviously broken to a human eye. We asked a vision model to check it. It read the attribution line perfectly, understood the quote, and never flagged the overlap. Then a plain optical-character-recognition pass measured how many of the known words it could recover from the card. Coverage dropped to 63%, and the defect was caught. "The 'dumber' tool was the more accurate one — because it was the right tool."

In Agent-Native Computing I put the mechanism more sharply: "A language model reads mangled, overlapping, badly-kerned text perfectly — which is exactly why it is the wrong instrument for telling you that a rendered card is broken. Its strength at recovering meaning from a mess is precisely what makes it blind to the mess." The same book drew the boundary that matters for this chapter. Turning pixels into text is usually right — except when "colour, layout, motion or spatial arrangement is the payload".

Now extend that to things that unfold in time. In a film, timing and sound are the payload. The pause after the promise. The voice's warmth. The level of the music between words. Whether the soundtrack sits in sync with the picture. A transcript is a beautifully clean text version of everything in a film except those things. Hand a judge the transcript and ask it to assess the film, and you've given it an excellent view of what was said and no view at all of how. A judge built to read meaning will then do what meaning-readers do: find meaning, report success, and miss the form entirely.

Five ways a judge fails silently

My own record is one dated instance. The research literature documents each class of failure in it independently, which is worth knowing, because it means you should expect these in your own loops too. None of these papers tested my configurations; they corroborate the kind of failure, not my numbers.

Notice what those five have in common. Not one of them announces itself. Each produces a confident, well-formed verdict. A dashboard of scores from these judges would look exactly like a dashboard of scores from judges that work.

The general form: what the evaluator is grounded in

A small pilot study of long-running agent loops puts the whole thing in one finding. An agent ran fifty-four improvement cycles and "claimed improvement every time, yet 56 percent had a measured delta of zero or below." The authors call it the progress mirage, and their diagnosis is the right one: "it is a question of what the evaluator is grounded in."¹⁷

That's the ceiling, stated generally. It isn't the judge's intelligence. It's the judge's contact with the dimension being judged. A brilliant judge reading a transcript has no contact with the pause. A modest meter with the audio file has perfect contact with the pause and no opinion about whether it worked.

Find the zero

In Give the Agent a Workshop I argued that an AI system's outcome capability is a product, not a sum: model times goal quality times persistence times access to reality times tools. "People say the new model is twenty percent better. Effective outcome is often not plus twenty percent. It is a product of several factors — and any zero collapses the lot."

An improvement loop has the same shape. A judge that can't perceive a dimension is a zero on that dimension's feedback. You can multiply that zero by a better generator as many times as you like. You'll get a more impressive film that's no better on the thing you couldn't see.

Which tells you where the next improvement usually comes from — not a stronger generator, but a better observation, or a better question.

In practice that's a habit change. When a loop plateaus, or starts disagreeing with your own judgment, the reflex is to reach for the newer model. Audit the view first. Ask what the judge was actually given, and whether the thing you care about was in it.

"Isn't this just evals?"

It's the natural objection, and it's half right. Evaluations are the right idea. But almost every evaluation practice I've seen quietly assumes the evaluator can perceive what it's scoring, and then worries about the other properties — consistency, bias, independence. The failures above are upstream of all that. They're silent, and they're per-dimension: a judge can be excellent on claims and deaf on timing in the same verdict, and nothing in its output will tell you which parts to trust.

So perceptual coverage can't be assumed. It has to be measured, dimension by dimension, for each judge and each way of feeding it. Which means treating judges less like oracles and more like instruments — with ranges you find out by testing.

In an earlier book I corrected myself about what makes a check trustworthy. The films made me correct the correction.

What a closer already needs

In Designing Loops, Not Prompts I used the word closer for the part of a loop that can stop it: "any durable, mechanically independent check that can veto the producer's output." A test suite. A gate that goes red. A separate evaluator. A person who has to click before anything ships.

The first edition of that book said a closer needs independence — author and reviewer as different instances, so nothing grades its own homework. The second edition corrected it: "Two properties hold a check up, not one." Independence is necessary. The other property is information asymmetry: the reviewer knows something the author structurally can't. In Hidden Gates I made that the whole design — intent visible to the worker, rubric held back by the reviewer — because "a gate the worker can't see is a gate it can't game."

That chapter's title asked a question I didn't fully answer at the time: something that can say no — and whether it can see you. The films answered it.

The third property

Put it against the other two and the order becomes obvious. Information asymmetry presumes perception. A reviewer holding a secret answer key about pauses is useless if it can't hear pauses. An independent judge is only independently blind if it can't perceive the thing. So in practice perceptual coverage is the first property you check, even though it was the last one I named.

The instrument is the model plus what you feed it

One implication catches people out. Perceptual coverage isn't a property of a model. It's a property of a model and the input you give it.

In Agent-Native Computing I argued that the unit worth benchmarking had moved from the model to the model plus its harness: "A model benchmark increasingly tells you which CPU you bought. It does not tell you what computer you built." Judges work the same way. The same model reading a transcript, looking at a sheet of keyframes and listening to the audio file is three different instruments. It will have three different ranges. So name your judges by configuration, not by model — "the listening judge", "the transcript-and-frames judge" — because that's the thing you're actually trusting.

I got there in practice before I got there in principle. That week I was running several judging setups side by side without being sure which one to believe:

Keeping them all was right, but for a reason I only saw later:

They are not automatically extra votes. They are different instruments whose useful range you are learning.

That's also why piling on judges isn't the fix it looks like. Three copies of the same configuration agreeing with each other is the correlated-checkers pitfall — "an echo, not a check". Three different configurations disagreeing with each other is information: it tells you where at least one of them is out of range.

Finding a range: counterfactual audits

How do you find out what a judge can perceive? You change one perceptual thing, hold everything else, and see whether the judge notices.

Researchers auditing audio-language models as judges do exactly this. "Each audit item holds the transcript fixed while varying affect, prosody, or the timing of an affective shift, forcing a valid judge to track the audio cue rather than lexical content or response style." Their finding is a warning worth pinning up: "contrastive success often overstates native judge reliability, and… similar aggregate accuracies can hide different failure modes."¹⁸ A judge's overall accuracy tells you very little about which dimensions it's actually tracking.

This is where code pays twice. Chapter 4 said a controlled variation costs an edit. The same edit makes an audit item. A copy of the film with one pause cut. A copy with the soundtrack slid half a second late. A copy with the music under the voice pushed up. A copy with a frozen opening. Each is a broken copy that differs from the original in exactly one way — which makes each one a test fixture for a judge. If the judge can't tell the broken copy from the original, you've found the edge of its range.

Calibrate against your own verdicts

The other half of the method is ground truth, and for perception questions the best ground truth you have is often your own. Not your taste in general — your specific verdict that this version has the pause and that one doesn't, or that this voice is right and that one isn't.

Anthropic's harness report describes the same move. "I calibrated the evaluator using few-shot examples with detailed score breakdowns," and the tuning loop was "to read the evaluator's logs, find examples where its judgment diverged from mine, and update the QAs prompt." Even after several rounds, the output "showed the limits of the model's QAing capabilities."¹⁰ Calibrate, and expect a residual range the judge still can't cover. The point of calibrating isn't to make the judge perfect. It's to know exactly where it isn't.

One fence, so this chapter doesn't wander. What a system learns about good — how taste is formed, carried and tested — is its own subject and another book. This chapter owns something narrower and more mechanical: whether the judge can perceive the thing at all.

Rules for using judges you've measured

Once you know a judge has a range, you stop treating its single verdict as an answer. Here are the rules I'd run with any model judge in a loop. (The specific thresholds I used, and why, are in Chapter 19.)

Code measures the facts. The model judges.

Some of what no model judge could perceive in my films, ordinary code could measure exactly. That points to a division of labour I'd now apply everywhere: anything that can be measured exactly should be measured by code and handed to the judge as a fact. Not left to the model's impression. And not converted into a score.

In The Agent's Retina I described deterministic code as the agent growing temporary sense organs for the question it has right now: "Grew an eye. Looked. Didn't like the view. Grew a different eye." A measuring script is exactly that: an eye built for one dimension.

The pattern generalises well beyond film. When I ran a security-review agent that rewrote its own method over eight rounds, the honest accounting afterwards was that the genuinely independent checks were the deterministic ones — hashes, excerpt anchoring, budgets, schemas — and that a sceptical second pass by the same model was "a second opinion from a correlated mind, not verification." Mechanical independence again. Code that measures is a different mechanism from a model that judges, which is exactly why you want both.

The eyes have ranges too

It would be convenient if measuring code were simply right. It isn't. It's an instrument, and it needs calibrating like any other. A tool that measures pauses can under-read a scripted pause, or miss a short one entirely. An optical-character-recognition service that returns lines of text without their box sizes can't tell you how big the smallest text is. Both happened to me that week — Chapters 19 and 20 have the details.

The rule that follows is small and important: when a fact can't be measured, the facts file says so. "Not measured, and here's why" is a fact. A guessed number presented as a measurement is worse than no number at all, because it borrows the authority of code for something code didn't do.

The perceptual coverage audit

Put all of that into a procedure you can run next week:

1. List the dimensions

   Everything your loop is trying to improve: timing, voice, mix, legibility, claims, sequence, tone. Be specific — "timing" is really several dimensions.

2. Assign an instrument to each

   Code, a named judge configuration, or a person. If you can't name one, you've already found a gap.

3. Build the fixtures

   Five controlled pairs you already have a verdict on, plus one broken copy per dimension, each differing from its original in exactly one way.

4. Run the judges properly

   Both orders, fresh, several repeats. Record every verdict, including refusals.

5. Mark each dimension

   Perceived, not perceived, or invented (the judge reported a difference that isn't there). Anything not perceived gets a code-measured fact, a better view, or a person — and goes on the list of things the loop can't improve on its own.

The output is a coverage grid — dimension against instrument — that belongs in the loop's documentation and gets re-run whenever you change a model or change what you feed it.

Two objections

"Judges are getting better fast. This is temporary." The specific findings are temporary — that's why every one in this book carries a date. The practice isn't. Measure the range, trust by dimension, re-run the audit when something changes. That doesn't expire when a model improves; it's how you find out that it has.

"Won't a stronger model just hear the pause?" Maybe. You'll find out from the audit, not from the release notes. The release notes will tell you what the model is better at on average. The audit will tell you whether it can perceive the one thing your loop depends on.

From judges to what they're fed

A judge's range depends on what you give it — the transcript, the frames, the audio, the facts. That makes the packaging part of the instrument. Which means the unglamorous work of deciding how evidence is bundled for a model isn't plumbing at all. It decides what the intelligence can notice.

Most of my earlier work was about packaging machine output for people. The films pushed me the other way.

It's the same discipline running in opposite directions. On the way out, you organise meaning into something a person can experience: a page, a diagram, a film. On the way in, you organise an experience into something a machine can reason about. Chapter 6 ended on the reason the way in matters so much: a judge's range depends on what you give it. So the packaging is part of the instrument, whether you designed it or not.

Not fewer frames — the right unit

The instinct, when evidence is too big for a model, is to reduce it. Fewer frames. A shorter transcript. A smaller payload. Reduction is sometimes necessary, but it isn't the significant step. The significant step is choosing the unit.

A single frame has too little context: it's a picture with no idea what was being said while it was on screen. A whole recording has too much: every moment at once, with the model left to find the moments that matter. In between is a unit that closes — a screen state, the small changes within it, and the narration that belongs with it. That's something a model can reason about as one thing.

My video reader in Extract Conversation made exactly that switch. It stopped sending the model a frame every few seconds and started sending sections of the video where the screen stays nearly the same, joined with the words spoken over each section. The point, as I put it at the time, was that the model "better understood what it was looking at" — because it was finally being shown something that meant something.

Timing is meaning

In How to Read a YouTube Video I made the case that the most valuable move in reading a screen recording isn't reading the screen or the speech better. It's putting them on the same timeline. Recover the on-screen text, turn the audio into a timestamped transcript, and interleave the two by start time — "and temporal adjacency stops being metadata and becomes meaning". When the narrator says "look at this", the "this" lands right next to it. "Neither stream is the video. A transcript lost the screen; an OCR dump lost the talk. The interleave — two timestamped text streams on one timeline — is the video."

For judging a film, that interleave is also a view in the Chapter 3 sense: what's on screen at the moment each line is spoken. A judge given that view can see a caption animating over the promise line. A judge given the transcript alone can't know there was a caption at all.

Grouping changes the evidence

The deeper point comes from Your Life Compiles to One Language. One spam email can look vaguely plausible. Three hundred similar ones in a cluster — same shape, same links, same timing — are obvious. Nothing about the model changed between those two cases. What changed was the object it was shown. "Grouping changes the epistemic quality of the evidence before the AI thinks." And the line I'd put on the wall of anyone building a judging pipeline: "Clever prompts on wrong units are cosplay. Show the model the right grain or do not bother asking."

For media that unfolds in time, the unit that closes is often a moment: a line, what's on screen with it, and what came just before. That last part matters more than people expect, and Part III will show why — a pause after a demonstration and a pause after a slogan are the same length and completely different events.

Keep the bronze, cut the silver per question

There's one more piece that makes all this workable. In Keep the Bronze I borrowed the data engineers' medallion vocabulary — bronze for raw, silver for validated and shaped, gold for enriched — and argued that the raw layer should be kept, immutable, because every useful view can be built on top of it "later, cheaply, and as many times as you like". "Storage is cheap, comprehension is now cheap — so deletion is the only irreversible operation in the whole stack."

That's what frees you to stop looking for one perfect representation. If the renders, the audio stems, the transcripts, the frame sheets and the judges' outputs are all kept, you can cut a different silver view for each question you want to ask — rather than choosing one packaging up front and living with what it destroys.

Silver, defined

Sometimes that means removing duplication — the three hundred near-identical frames become one screen state with its small changes. Sometimes it means joining things that belong together — the screen and the speech on one timeline. Sometimes it means preserving a relationship that ordinary extraction would destroy — the fact that the promise line came after the demonstration, not just that both occurred.

Different questions need different silver

Here's the practical consequence, and it's the same lesson as Chapter 6's coverage audit, arriving from the input side. Each kind of question needs the silver to preserve something specific — and a naive package destroys exactly that.

Read down the middle column and you can see why no single representation serves every judge. Claims want text. Timing wants a timeline. Voice wants audio. Layout wants pixels or a structural map. Sequence wants a record of order. Pick one universal package and at least one row goes dark — and whichever judge depends on that row will score it confidently anyway.

Find the units with cheap evidence first

How do you find where a unit closes? Not by asking a model to look at everything. The pattern that works is to segment with cheap, deterministic evidence first — where the screen changes, where the speech pauses, where the vocabulary shifts — and only then spend model judgment on the bundle. Your Life Compiles to One Language put it as a build order: identify the structural glue, materialise the closed bundle as an object the model can read once, then ask for significance against the bundle.

And treat the unit as a hypothesis about where meaning closes, not a law. You can test it the same way you test a judge: package the same material two ways and see whether the judge perceives what matters with one and not the other.

"Why not just send the whole video?"

Because a multimodal model that can take a video isn't thereby a judge that perceives everything in it. Three reasons, all already on the table. Models lean on text when text is present, so a video with a transcript attached may be judged mostly from the transcript. Pixels are the right representation only where pixels are the payload — the boundary from Agent-Native Computing in Chapter 5. And the representation is part of the instrument: if you don't choose it on purpose, you get one by accident, and you won't know what it can't see.

Good packaging is not whatever produces the smallest input. It is whatever preserves the distinctions the next judgment needs.

From reconstruction to birth

Everything in this chapter is about reconstructing structure from media someone else made. That's unavoidable for external footage. But for media your own system authors, the structure was known at the moment of making — every scene, line, hold and claim was a decision. Throwing all that away at export and then extracting it back is the most expensive way there is to learn something you already knew.

So give the media a sibling.

When a film leaves production, let it leave with a structured account of itself sitting beside it — written by the process that made it, while that process still knew everything. The film is for people. The sibling is for whatever reads the film next: a judge, a builder making a revision, a tracker trying to connect it to how the audience responded.

I want to be careful about one thing up front. The sibling is not a substitute for watching the film. What the builder intended and what actually lands can differ — that difference is half of what Part III is about. The sibling is a shared set of references for creation and review. And, as you'll see, the gap between what it declares and what a viewer perceives is itself a useful test.

What the sibling carries

Most of those fields fall out of the construction for free. A film built as code already knows what appears when and what's said when; the sibling just refuses to throw that away at export. The measured facts come from code. The two fields a person has to write — the intent note and the answer key — take minutes, and they're the two that make everything else meaningful.

Why "what each moment is for" is the field that matters

It's tempting to think the source code of a film is its account. It isn't, and the reason is the same one I gave in The Prompt Is Source. What counts as "source" depends on which stage of the process you're standing at. Generated code is the output of the intention that produced it and the input to whatever runs it. It only becomes a disposable intermediate — safely regenerable — to the extent the upstream intent can reproduce it: "Until then, parts of it are still the source of truth — whether you admit it or not."

Apply that to a film. The composition file will tell you there's a 0.8-second silence after the pitch line. It won't tell you why: that the silence exists so the viewer has a moment to connect what they've just been shown to the name of the thing that does it. Source code preserves what happened, but it may not preserve why it mattered.

Without the purpose field, the next agent can reproduce the pause but not the reason. It will keep the pause in this film — and, left to itself, start putting pauses everywhere, because nothing told it which pauses were load-bearing. That's the template factory from Chapter 4 arriving by a side door.

Born structured

This is a specific case of something I've argued at the scale of whole organisations. In Born Structured Exhaust the observation was that AI-assisted work pulls things into the open that ordinary corporate writing suppresses — intent, alternatives, what was rejected and why, what's still unresolved — because getting useful help requires stating them. "The document is the what; the deliberation is the why."

The sibling is born-structured exhaust for authored media. The intent, the declared moments, the reasons — captured at the moment of making, while they're still available, not reconstructed afterwards by someone guessing.

Write the intent before you build, and the answer key before it exists

Two of the fields have to be written first, or they're worth much less.

The intent note, before the build. Writing down the film's one job, its audience, the moment the viewer should recognise themselves and the moment they should understand how it works does three things. It forces the builder to decide what it's making before it starts making it. It gives every judge something to check the film against. And it gives the audience loop in Part III its "intended" account — the record you'll compare with what actually happened.

The answer key, before the artefact. What should a viewer who arrives cold be able to say afterwards? Write that before the film exists and the key can't be shaped by the film. It's the same discipline as Hidden Gates — the grader's criteria never leak into the thing being graded — applied to creative work.

Both are forms of preregistration. In Prediction Receipts I put the reason for writing expectations down in advance as bluntly as I could: "A forecast that cannot name how it could be wrong is not a prior. It is a hope with a decimal." The intent note is the producer's receipt: what we meant, written where we can't quietly revise it after we see the result.

The cost of skipping it has now been measured

Every artefact gets handed to a next agent. A judge. A reviser. A tracker. Without the sibling, each of them reverse-engineers what the maker already knew. That cost used to be invisible; it's now been measured, at least for coding agents. In a study of agents taking over interrupted tasks, "context-bearing handoffs reduce median agent events by 20-59% and cumulative prompt tokens by 42-63% relative to repository-only takeover."¹⁹ The authors draw the right conclusion: evaluation should report not only whether a task was solved, "but also how costly that work is for another agent to resume."

A film without its sibling is a repository without a handoff note. Every agent that touches it pays the rediscovery cost again.

What the sibling makes possible

The fields are dull. What they unlock isn't. Four things downstream get easier the moment the sibling exists:

I said as much while working out how to learn from the films:

That's the point in a sentence. A tracker trying to learn from a film no longer has to infer the whole experimental intention from a finished video file. It was written down at birth.

Do not make the next agent reverse-engineer everything the previous agent already knew.

"Isn't this just documentation overhead?"

It would be, if a person wrote it after the fact. It isn't written after the fact. Most of it is emitted during construction, by the builder and by code, from information that already exists at that moment. The part a person writes — the intent note and the answer key — is short, and it's written first, which is when it's cheapest and most honest. What it replaces is the expensive part: extraction, re-explanation, and a series of later agents guessing at what the film was for.

The end of the machine's ceiling

That's Part II. Better views, judges treated as instruments with measured ranges, facts measured by code, evidence packaged so the distinctions survive, and media that carries its own account from birth. All of it improves one loop: whether the thing was built the way it was meant to be built.

None of it tells you what a person will do when they watch it. That's a different ceiling, and no amount of code moves it.

Imagine the perfect judge. Every pause heard, every drift measured, every claim checked against its source, every version compared fairly in both orders. What would it tell you?

It would tell you, with total reliability, whether the film you built is the film you meant to build. That's enormously valuable, and it's still not the question you actually care about. The question you care about is whether the film does something for the person watching it. That's a different loop, and it's worth drawing the line between them firmly, because almost every confident claim I see about AI-made media blurs it.

Two loops

The production loop asks whether the intended experience was constructed successfully: the right information, a coherent sequence, a readable screen, usable sound. Everything in Part II — views, judges, facts, siblings — lives here.

The audience loop asks whether the experience actually did its job: did someone recognise their own situation, understand the offer, believe only what's justified, and know what to do next?

You can make the first loop very good. It's fast, it's cheap, and with care it's honest. The second loop is where the value is decided, and code has almost no direct grip on it.

A calibrated judge is still a production instrument

In Demonstrate Over Tell I drew the line for pages: a fixed judge is a comparison instrument. Hold it steady, run versions past it, and it tells you which version it prefers. That's how you find out whether a change made things better or worse. But "a judge's score is not evidence that a reader understood anything. It tells you what the judge rewards. If the judge is a model, it tells you what a model rewards." The rule from that book still stands: "Keep the judge for comparison. Add a comprehension test for truth."

I won't re-explain that comprehension test here; that book owns it. The extension this book adds is uncomfortable for anyone — including me — who has just spent Part II making judges better. Calibration doesn't move a judge across the line. A judge calibrated against my own ears can tell me, reliably, whether the pause is there. It can't tell me whether the pause helped anyone understand anything. Better perception makes the production loop more honest. It doesn't turn it into the audience loop.

Code gives you the same frames, the same words, the same 0.8 seconds of silence, every time, for every viewer. What it can't give you is the same viewer. People arrive with different concerns, different attention, different days. What becomes programmable is an increasingly precise hypothesis about an experience. The audience still supplies the answer.

A small pilot study of self-evaluating agent loops, which I cited in Chapter 5 for a different reason, puts the principle in a way that carries straight over: for open-ended objectives whose "success signal lives outside the transcript", no amount of scaling up the judge closes the gap.¹⁷ For a film, the success signal lives in a person. You have to go and get it.

What experiments can and can't tell you

The obvious next move is to run experiments with real audiences, and it's the right move. It's also worth being clear about what it gives you. The people who've run more online experiments than almost anyone put the limit plainly: "It is possible to know which variant is better, and by how much, but not 'why.'"⁹ Their recommendation is to pair experiments with studies that capture what people actually say — because the number alone won't explain itself.

That's where deterministic delivery earns its keep on this side of the line. When your variants differ in exactly one respect, a difference in response at least points at one candidate cause. It still doesn't tell you why that cause worked. For that you need records that keep what you intended, what you delivered and what came back apart — the subject of the next chapter — and a habit of turning each difference into the next question rather than a story.

One film, both loops

It helps to see the two loops disagree on something concrete. Take a one-minute film for a service, and suppose the production loop is completely satisfied with it.

Nothing on the left predicts which answer on the right you'll get. That's not a flaw in the production loop. It's the boundary of what it can know. And notice that the three answers aren't a scale from bad to good — they're different kinds of response, each pointing at a different next move. A single engagement number would have blended them into one figure and thrown that difference away.

The valuable wow

This is also where one of the words I used loosely all through that week got sharper. I kept asking for films with a "wow factor". The production loop is very good at rewarding one kind of wow: polish, motion, richness, that-looked-expensive. Judges love it, and so do the people who commission films.

The wow that matters is a different thing. It isn't "that looked expensive". It's "that understands my situation — and now I see what this would do for me." The first is a property of the artefact. The second is something that happens in a viewer, and it can only be observed in the audience loop.

The spectacle has to carry the promise, not compete with it.

That's exactly where "impressive" and "effective" come apart, and it's why a production loop left to itself will drift toward the wrong wow. Every judge in it can see polish. None of them can see a viewer suddenly recognising their own week.

What this doesn't mean

It doesn't mean judges are useless. They make the production loop fast and honest, and a production loop that's honest is the precondition for learning anything from the audience at all. If you can't say precisely what you delivered, you can't attribute what came back.

It doesn't mean you should wait for the audience before improving anything. The two loops run on different clocks — minutes against weeks — and you improve the production side continuously while audience evidence accumulates. Chapter 12 deals with holding both clocks at once.

And it doesn't mean the audience's answer is a mystery. It means it's a measurement you haven't taken yet, and that the most useful thing you can do before you take it is make the production side legible enough that the answer, when it comes, has something precise to attach to.

"A cold AI reader is good enough"

It's a tempting shortcut: hand the finished film to a fresh model with no context, ask what it understood, and treat that as the audience. Use it — it catches obvious confusion cheaply, before any person sees anything. But treat it for what it is. Demonstrate Over Tell called it "a screen, not a verdict", and I'd keep that exactly. A model watching a film is not a customer watching a film. It doesn't have the customer's week, or their worry, or their reason for scrolling past.

Different records for different things

If intention, delivery and response are three different things, they need three different records. Not one record with three columns that everybody edits, but three accounts, written by different hands at different times, and allowed to disagree. The disagreements turn out to be the most valuable thing you'll write down.

That week I described what I wanted my media-tracking system to do, and I think the description holds for anyone making things people are meant to respond to.

An experiment has at least three records: what you meant to test, what you actually ran, and what came back. Most teams keep one of them — usually the third, as a dashboard — and let the other two live in people's heads. Then the person who made the film explains the numbers, and the explanation quietly becomes both the intention and the finding.

The three accounts

The columns on the right matter as much as the ones on the left. Each account has a different author and a different time of writing, and that's what stops them contaminating each other. The intended account is written before anyone knows how the film turned out. The delivered account is written by the construction and by code, not by the person hoping it worked. The observed account is written by people who weren't in the room.

The disagreements are the findings

Once the accounts are separate, you can compare them in pairs. Each pair reveals a different kind of problem.

None of those gaps is visible in a single engagement number, and none of them is visible if the maker writes all three accounts after the fact. Each is visible the moment the three are kept apart and put side by side.

The intention is not evidence that the experience landed. The response is not self-explanatory. The join between them is the investigation.

Code helps with exactly one of the three. It makes the delivered account precise — you can say what the film did at every second — and it gives you more control over the next variation. It does not, by itself, establish why an audience responded the way it did. The join between the accounts is where that work happens, and it's work, not a readout.

Heat is a relationship, now with a production side

This isn't the first time I've argued that a result belongs to a relationship rather than to an artefact. In Heat Is a Relationship the claim was that a post's engagement isn't a property of the post. It arises from concept, framing, source, carrier, audience, surrounding discourse and time together, and ends in a response that has a type — a save isn't a click, a hard question isn't a like. The post is just where the relationship became observable: "the join key to the raw platform event — not the object that 'has' heat." And the rule: "File the score as evidence about a relationship; do not let the score become the identity of the lesson."

What changes when the stimulus is built as code is that several of those factors stop being things you reconstruct afterwards and start being things you set, on purpose, before publication. The framing, the structure, the timing, which cut went to which channel — all written into the construction and its sibling. The delivered account is the production side of the heat relationship, recorded rather than inferred.

The intended account is a receipt

The first account deserves a name of its own, because it's the one teams most often skip. In Prediction Receipts I argued for writing expectations down before publishing, with a way they could turn out to be wrong, so that the difference between expected and actual has somewhere to land. Semantic Experiment Graph added the discipline of naming a stimulus twice — once for its idea, once for its surface. The intended account is exactly that, written by the producer at build time: the receipt for what you meant.

Keeping them apart

The accounts only work if they stay separate. That sounds obvious and fails constantly, usually through good intentions. Here are the operating rules I'd hold to.

"Preregistering intent slows creative work"

It takes minutes, and it's written before the build, when the maker knows most clearly what they're trying to do. The alternative has a cost too; it's just paid later and less visibly — it's the one Chapter 4 named, where a single result produces several incompatible lessons because nobody wrote down what was being tested. A few minutes up front is cheaper than a quarter spent arguing about which lesson was real.

In Chapter 20 I fill this table in for one real film — including the third row, which is still blank, and says so.

The unit the accounts need

The three accounts give you structure. They don't yet give you the right grain. If the intended account says "trust" and the observed account says "enquiries went up", you've kept everything apart and still learned nothing, because neither side names anything a film could test. The unit inside a film isn't a topic. It's a change in someone's understanding.

Picture a one-minute film for an operations service — the kind of business that takes care of loose ends so a manager doesn't have to.

It opens on a feeling most managers know: something has fallen between the cracks. There's an unresolved matter nobody quite owns. The viewer then watches that specific matter acquire an owner, a next action and a visible follow-through — the thing moving, on screen, without anyone having to chase it. And the change the film is trying to produce in the viewer, in their words rather than the company's, is something like: "I would no longer have to hold all of this in my own head." Only then does the brand promise arrive — as the name for something the viewer has just understood, rather than a slogan they're asked to believe.

That's an illustration, not a case; I've built it to show the shape. But notice what it lets you say about the film that a list of topics can't.

Topics tell you almost nothing

Tag that film with its topics and you'd write something like reliability, trust, peace of mind. Accurate, and nearly useless. Every operations film ever made could carry the same tags. If the film worked, the tags can't tell you why; if it didn't, they can't tell you what to change.

The topic is reliability. The mechanism is something much more specific: making held responsibility visible, so the audience can imagine relinquishing some vigilance. That's a claim about how the film is supposed to work on a person — which means it's a claim you can test.

Written as a template, so it can sit in the intended account from Chapter 10:

Candidate explanations, not established effects

The fifth line is where discipline matters. For the illustrative film, there are at least three candidate explanations for why it might work, and they're different claims:

• The recognisable opening matters: viewers who see their own situation in the first few seconds keep watching and extend credit to what follows.
• The visible follow-through makes the promise credible: watching the matter move is what turns "we'll take care of it" from a slogan into a claim.
• The pause after the demonstration gives the viewer time to make the connection before the promise names it.

None of those is an established psychological effect. Each is a candidate. The value of writing them down separately is that each one suggests a different experiment — and a film built as code can run each of them by changing one thing.

Moments depend on each other

Here's the part that's easiest to underrate. A pause is not inherently valuable. Neither is a promise, or a demonstration. Their value depends on what comes before them.

Same pause, same promise, same price — different events, depending on the sequence. Which means the most important thing about a moment can be the relationship between it and the moment before.

The promise became intelligible because the demonstration preceded it.

That sentence is a different kind of record from "demonstration: strong; promise: strong". Scoring moments separately — as most judging rubrics do — throws away the because. And the because is the only part that travels: "promise: strong" tells the next film nothing; "the promise worked because the demonstration came first" tells it where to put the promise.

What the record has to keep

So the intended and delivered accounts need a few fields that most media records don't have:

• Order — the sequence of moments, not just their presence.
• Depends on — for each important moment, which earlier moment it needs.
• Immediately preceded by — what the viewer had just seen or heard.
• The hold after — how long the viewer is given before the next thing arrives.

And this is where building media as code pays off most directly. Deterministic production can keep all the material and change only the order — move the promise ahead of the demonstration, and nothing else. Or keep the sequence and change one transition — remove the hold after the demonstration, and nothing else. The Chapter 4 brief, applied to dependency. You have practical means of testing a claim about why two moments work together, which almost no one making media has ever had.

In Heat Is a Relationship I took one unusually successful post apart into seven candidate mechanisms and refused to crown any of them — "one observation, many mechanisms, no cloning certificate." The same discipline applies inside a single film. Rank the candidates. Test them one at a time. Let the sequence be one of the things you test.

Declared against perceived

The change-in-understanding unit also gives you a sharper check than "did they like it?" The sibling from Chapter 8 declares when the viewer is meant to recognise their situation and when they're meant to understand how the thing works. So ask a viewer who arrives cold — a person, or a fresh model as a first screen — two specific questions: where did you recognise yourself, and when did you understand how it works? Ask for the detail they saw.

Then compare. If the viewer found the recognition at the moment you declared it, the delivered account and the intended one agree. If they found it somewhere else, or not at all, you've learned something about the film before a single real customer has seen it. The questions come from protocols I've written about before — the cold-arrival reader in The Exemplar Ratchet, who "arrives with no context on purpose, often in a customer's voice", and the comprehension test in Demonstrate Over Tell. What's new is asking about declared moments, with times, rather than about the film in general. In the week's films, that turned out to matter: the declared-moments check carried signal where the viewer's own self-report didn't. Chapter 20 has the detail.

Why this grain travels

A topic lesson — "trust works" — doesn't travel, because it never said anything you could do. A change lesson can. "For managers who carry too much in their heads, watching held responsibility visibly move makes the promise credible" is a claim you can test in a different film, in a product demonstration, in an onboarding flow, in a training exercise. It names a concern, a mechanism and a sequence. Part VII does exactly that kind of carrying.

And once lessons are written at this grain, the obvious question stops being "why did it work?" — which invites a story — and becomes "what should we test next?", which invites an experiment.

Stories about why a film worked are the cheapest thing in marketing. Every team has one within minutes of seeing the numbers, and it's usually persuasive.

That's the problem. A persuasive story after the event feels like learning and changes almost nothing, because it can't be wrong. The emotional opening worked; the pause worked; the brand promise landed. Any of those might be true. The story doesn't tell you which, and it doesn't tell you what to do next that would find out.

Describe, propose, construct

So I'd ask a tracking system for something harder than an explanation. Three steps, in order: describe what performed; propose why it performed; construct a discriminating next experiment. The first two are where nearly all analysis stops. The third is the one that pays.

Here's what a good return looks like. This is illustrative — I've written it to show the shape — but it's the kind of sentence I'd want back:

Look at what that does. It preserves the observation — viewers asked questions that suggest understanding. It limits the explanation — two candidates are still alive, and it says so. And it directs new work — here is the one change that would tell them apart. The next film then tests something the last film left unresolved, and its result becomes an input to the next creative decision rather than a trophy.

An intent tracker, not a like counter

This is what my own media-tracking system, Q-Heat, is turning into. It started as a way to follow how published pieces did — which is what everyone's analytics does. The upgrade I want is different, and I described it at the time better than I can now:

"I think Q-Heat becomes sort of more like an experiment tracker, or an intent tracker. You conceptualise your experiment — we put the brand promise early, we put the brand promise late — and then you can see which one works better. And the same experiment might have five features, so you can compare them over time across different experiments."

The implementation doesn't matter here. The purpose does: a record that holds what each piece was trying to find out, so that responses — when they come — land against a question rather than against a post.

The tracker record

The pairs are tests to run

In Heat Is a Relationship I specified two kinds of test the argument needed and was honest that I didn't yet have either: a matched pair, where the same concept runs under one changed condition, and a different-surface pair, where the same mechanism runs through two unrelated presentations. "Presenting invented pair results would be worse than admitting the gap. The pairs are the tests to run, not evidence already in hand."

What's changed is the price of the stimulus side. When the stimulus is a post, a matched pair means writing two posts that differ in exactly one respect, which is harder than it sounds. When the stimulus is a film built as code, the matched pair is an edit. The bottleneck moves entirely to the audience side — getting each version in front of comparable people, and measuring what they do.

Design the probe to kill something

The test for whether a next experiment is worth running comes from Fog Is a Race Between Two Clocks, where I wrote it for strategy: before you run a probe, write every outcome it could produce and, beside each one, which live explanation it kills. "If no outcome kills anything, you have not designed an experiment. You have designed a demonstration with a budget."

Apply it to a film. "Promise early versus promise late" is a good probe if each outcome kills something: if the late version does better, the explanation "placement doesn't matter for this audience" dies; if they do the same, "the promise needs the demonstration first" is in trouble. "Make a more emotional version" kills nothing. It's content.

Unknowns should arrive typed

Sometimes the honest return is "we can't tell yet". That's fine — as long as it arrives in a useful form. When I watched an agent rewrite its own security-review method, one of the most valuable things it produced was a refusal with a name: a finding marked needs_runtime_test, with the exact precondition missing. "needs_runtime_test is not the pipeline failing to answer; it is the pipeline handing you a specific experiment to run." An unresolved audience question should arrive the same way: typed, with the experiment that would resolve it attached.

Conditional lessons, not universal rules

The end product of all this isn't a rule like "this style works". It's closer to: for this audience and concern, making this particular consequence visible appears to improve understanding; here is the evidence, and here is where it has not transferred. Longer, conditional, and far more useful.

The people who've run more controlled experiments than anyone have a blunt warning for anyone tempted to skip the conditions. Among their rules of thumb: "your attempts to replicate stellar results reported by others will likely not be as successful (your mileage will vary)." And: "What works in one domain may not work in another."²⁰ A lesson without its conditions is a winner's surface waiting to be cloned into a place it doesn't fit.

Two clocks

There's one more thing a tracker has to respect, and it's the one that most frustrates people who have just made production fast. Production and analysis are now extremely quick. A film can be built, judged, revised and re-rendered in hours. Meaningful market response still takes time — and it takes the time it takes.

The experimenters are clear about why. "The short-term effect is not always predictive of the long-term effect," as a Google team put it.²¹ People convert late, and novelty and first impressions distort early readings — both of which, in the original experimenters' guide, "imply that some experiments need to be run for multiple weeks."⁹

So there are two clocks. The production clock runs in minutes. The audience clock runs in weeks. An instant model preference is useful for improving the artefact, but it is not a way to make the audience's answer arrive early.

"Two clocks" is a phrase I've used before, at other altitudes, and it's worth being careful not to blur them. Fog Is a Race Between Two Clocks mapped the pairs already in this body of work and warned "same word, different altitude".

This book's pair sits at the lowest altitude — one artefact — and the line from Fog carries down to it without changing a word: "Activity does not move this clock. Evidence moves it." Rendering more variants doesn't bring the audience's answer forward. It just gives you more questions waiting for it.

The practical consequence is liberating rather than frustrating. Hold several unresolved experiments open at once, each with its tracker record, while production carries on elsewhere. Learning doesn't have to stop just because a question didn't resolve inside one working session.

Small, fast decision models can help on the audience side too — sorting duplicate responses, telling a question from praise, nominating which comments deserve a closer read. Chapter 16 covers where that help ends.

Start that connection at production time, while the intention is still available — which is the whole reason Chapter 8 exists. Let the tracker accumulate conditional lessons, not universal rules. And let each piece of work set up the next question.

That's the machine's ceiling and the audience's. The third ceiling is closer to home: the attention of the person running all of it.

Improvement is becoming cheap enough to happen inside the work — not as a separate project you have to organise afterwards.

That's the shift Part IV is about, and it's easy to misdescribe. The falling cost isn't just implementation time, though that's fallen too. It's the whole human effort required to get from noticing something to having a completed, checked improvement. When that cost drops far enough, "I should fix that someday" becomes "fix it while we're here". And the ceiling that matters stops being the machine's, or the audience's. It becomes yours.

The old floor

I can describe the old floor precisely, because I lived on it.

Can I be bothered? That question was the real gate on a huge amount of useful work. Not whether the improvement was valuable — it often was — but whether getting it would cost more of my attention than it was worth. A better extraction boundary, a missing test, a cache, a second look at one assumption: all of it sat below the line, not because it was hard, but because asking for it, explaining it, checking it and rescuing it added up to more than I had.

Then, with the current generation of coding agents, the experience changed in a way I can only call mind-boggling:

It had done something. It had just done it without needing me.

Two effects at once

What surprised me was that the change pushed in two directions simultaneously.

"I've noticed two effects. I've done more little projects — it's got this recursive behaviour happening — and I've taken on bigger chunks in a single go."

Those belong together. The floor falls: smaller improvements become worth requesting, because the attention they cost has dropped below what they're worth. The ceiling rises: larger outcomes become possible to commission without personally decomposing, coordinating and carrying every intermediate step. The space between the two is the work you can practically undertake, and better agents widen it at both ends.

The ceiling already had a name

In Orientation Capital I described the upper boundary almost exactly. The Ambition Frontier is the highest-altitude task you can reliably complete across six moves: conceive, express, delegate, receive, understand and judge. And I described the moment where ambition quietly dies — the goal you almost asked for and then didn't. "Not from fear of the answer. From the cost of staffing the question. The briefing would have been longer than the window. So you asked for the smaller thing."

What this adds is the lower boundary: the smallest useful intervention that deserves your attention at all. Both boundaries are priced in the same currency. The flinch at the top and the shrug at the bottom are the same calculation — is the attention this will cost me worth what I'll get? — and when agents cut that cost, both boundaries move. The "receive" move matters here too: in Cycle Compression I argued that AI has shortened the time for the world to answer below the time it takes a thought to go cold. A result that comes back while you're still thinking about the problem costs far less attention to act on than one that arrives after you've moved on.

Waiting is not the same as being occupied

Here's the distinction that makes attention measurable, and it's one most dashboards get exactly backwards.

A task can run for two hours while barely occupying you. Another can finish in twenty minutes but demand six interventions, each requiring you to reload the situation in your head. The second can be more expensive to you, despite looking faster on the dashboard.

That isn't just intuition. The distinction between time and workload is one of the oldest findings in the study of interrupted work. In a classic experiment, people "completed interrupted tasks in less time with no difference in quality" — but "this comes at a price: experiencing more stress, higher frustration, time pressure and effort."²² Faster on the clock, costlier in the head. Every re-entry has a cost of its own: in another study, the time needed to "collect one's thoughts" and restart after an interruption was double the normal gap between actions — 3.8 seconds against 1.9 — and cues left behind before the interruption shortened it.²³ And anticipating that you'll have to come back under pressure leaves what researchers call attention residue — a part of your mind still on the thing you left — which a brief "ready-to-resume" plan reduces.²⁴

Agents make the gap between the two wider, not narrower, and the people measuring AI's effect on developers have noticed their instruments breaking. METR, which ran one of the most widely cited randomised trials in this area, reported this year that its time-per-task measurements "are unreliable for the fraction of developers who use multiple AI agents concurrently," because developers "would often work an unrelated task while waiting for the agent to complete its work."²⁵ Elapsed time has stopped measuring what the work costs the person doing it.

And self-reports don't rescue you. The same group's earlier study found developers who were measured as 19% slower with AI tools still believed afterwards that AI had sped them up by 20%.²⁶ Which is a useful discipline for me. "Did you even do anything?" is an honest report of how the work felt. It isn't a measurement. The measurement is the log in the next chapter.

What the vendors report — and what they're counting

The model makers report the mechanism, and I'll label it for what it is. Anthropic says Opus 5.5 "generates output more than 30% faster than Opus 5" and "uses fewer tokens per task".²⁷ One customer quoted on the same release page describes "a complex coding task that previously took 38 prompts over four days" coming in "at 11 prompts over three hours". Another, on the Sonnet 5.5 page six days later, says the model "rarely stopped mid-build to ask the user a question, so fewer builds stall waiting on someone to answer."²⁸

Those are vendor claims and customer testimonials hosted by the vendor, and none of them measures anyone's concentration. But look at what they count. Prompts. Stalls. Questions put to the user. Those are human interventions — the right unit, even when the numbers are someone else's.

The benchmark

Not how many tokens were generated. Not even how quickly the agent said "finished". Every word in that question is doing work:

• Completed — not started, not "mostly there". The last ten per cent is where attention goes to die.
• Trustworthy — something you'd rely on or ship without redoing it. An improvement you have to re-check twice is two improvements' worth of attention.
• Improvement — a change you'd keep. Discarded attempts count against the ones that stuck.
• Attention — occupation, not waiting. The two hours in the background are nearly free; the six re-entries are not.

Why "trustworthy" carries so much weight

More than forty years ago, Lisanne Bainbridge wrote the paper that should be taped to every AI dashboard. Her irony: an automatic system is installed "because it can do the job better than the operator, but yet the operator is being asked to monitor that it is working effectively." And monitoring is expensive in its own way — "it is impossible for even a highly motivated human being to maintain effective visual attention towards a source of information on which very little happens, for more than about half an hour."²⁹

That's what the word trustworthy is protecting. An improvement that arrives fast but has to be babysat, re-checked or rescued isn't cheap, however quickly it arrived. The benchmark puts the cost of monitoring in the denominator, where it belongs. And it points to where the real gains are: not only faster agents, but everything in Parts II and III that makes a result trustworthy without you watching it — calibrated judges, measured facts, siblings, typed records. Those are attention-saving devices as much as quality devices.

The budget was always attention

This is the per-outcome form of an argument I made at the level of whole budgets. In The Mature Token Law I revised my own early maxim about spending tokens, and landed on an asymmetry: "Spend machine cognition lavishly. Spend human attention ruthlessly." The point underneath it: "the scarce budget was never the tokens. It was always the human attention deciding which of the machine's outputs deserves to touch reality."

Attention per trustworthy improvement is that rule measured one outcome at a time. And measured that way, it forces an uncomfortable conclusion — because if the attention each outcome costs can fall, something I published about effort was wrong.

Here's what I wrote, in my own words, in a book I still mostly agree with.

The claim

In The Uncomfortable Truth About AI and Effort the central line was blunt: "AI doesn't remove work. It moves work." I put the shape of it in numbers — not measured numbers, a shape — to make the point stick: "You're not going from 100 units of work to 10 units of work. You're going from 80 units execution + 20 units thinking to 10 units execution + 90 units thinking." The chapter's summary table put total effort at "100" before and "100 (or more)" after, and it closed with advice I gave with conviction: "Stop expecting AI to reduce your workload."

"The total effort hasn't disappeared," I wrote. "The type of effort changed. Dramatically."

I'm going to revise that — not add some new examples to it, but revise it — because the work I've been doing this year doesn't fit it, and the reason it doesn't fit is the argument of this part of the book.

What survives

A lot survives, and it's worth being specific so the revision doesn't overshoot.

The direction of the shift was right. Cheap execution makes direction and judgment more consequential, not less. When the machine does more of the making, the quality of what you asked for, and of what you accept, matters more.

The verification paradox still bites. I wrote it as three lines, and I'd keep every word: "AI is most valuable for tasks you can't easily do yourself / But tasks you can't easily do yourself are hardest to verify / Result: the more leverage you want, the more cognitive supervision you need." Nothing in this book repeals that. If anything, Part II is a long elaboration of it — the judge that can't hear is the verification paradox wearing a headset.

Specification still pays. Fuzzy intent still produces fuzzy output at scale. Unstated criteria still get filled with generic answers.

What's revised, and why

What no longer holds as a general rule is the claim that those upstream activities — directing, specifying, verifying, interpreting — must keep consuming the same amount of my effort for each thing I get done.

Three things changed that.

1. The agent does more of the investigation

   A stronger agent now carries much of the decomposition, the test construction, the comparison of alternatives and the repair. The part of "verification" that was really doing the checking work has moved to the machine; the part that's deciding what counts stays with me.

2. Retained context supplies decisions I used to re-explain

   A lot of the old "thinking" in the 90 units was re-explaining. The same constraints, the same preferences, the same history, typed out again for each task. Context that persists — written memory, siblings, tracker records — supplies those decisions without me.

3. Good evidence makes accepting cheaper to do properly

   A result that arrives with its evidence — a comparison table, a held-out check, a test that proves the thing it claims — can be accepted in minutes and accepted properly. The judgment is still mine. The reconstruction I'd have needed to make it isn't.

I noticed it first not as a number but as a feeling about what a task had cost me:

"That's the sort of thing that's a massive project usually. And it's not only shorter in wall time, and likely in tokens and credits used, but in the amount of brain time I had to give it — the amount of concentration I had to give it."

Brain time. That's the variable the original claim held constant, and it's the one that moved.

The revised claim

Those two statements aren't in tension. One is about how much judgment matters; the other is about how much of me each outcome consumes. The original book treated the first as if it implied the opposite of the second.

There's a subtlety here worth slowing down for, because it's where the old claim looked right from the inside. My days haven't got emptier. I've spent the released capacity exactly as I described in the last chapter — more little projects, bigger chunks in one go. It would be easy to read a full day as confirmation of the old claim: see, total effort stayed high. But two very different economies produce the same busy day:

The original claim is the left-hand column. My experience is the right-hand column. The busy day is identical, which is exactly why the difference is easy to miss.

Why the old claim was reasonable when I made it

It wasn't a careless claim. For the tools of the time it described something real: agents needed heavy supervision, re-explanation and rescue; the verification layer was thick; and a widely cited controlled study of the period found experienced developers measurably slower with AI while believing they were faster (the METR result from the previous chapter). "Your effort will move, not shrink" was a correct corrective to the marketing of its day. It just isn't a law.

Why correct it in public

Doctrine that can't be corrected is decoration. I did the same thing to my own token maxim in The Mature Token Law — kept the instinct, changed the measure. This is the same move on effort. The honest version of a doctrine names exactly which part changed and why, so a reader can decide whether the change applies to them.

So yes: I've contradicted my own book. Deliberately, and only the part that no longer holds.

An instrument to check it on your own work

A revision like this shouldn't be taken on my say-so, and I don't have the measurement to give you anyway (more on that below). What I can give you is the instrument. Run it for a week.

What would falsify the revision for your work? Five comparable tasks where per-outcome occupation didn't fall. If that's what your log shows, the old claim holds for you, and you should plan accordingly. Chapter 23 lists this among the book's falsifiers.

When improvement gets this cheap

There's a consequence of the revision that goes beyond my calendar. If each improvement costs much less of me, then the things worth improving change. It stops being only the work in front of me. It becomes the workshop I'm working in — the tools, the ingesters, the representations — because fixing them now costs less than living with them.

At one point that week I got caught in a loop I've come to think of as typical of this period of work.

Jev is a fast decision model I use inside several pipelines; Chapter 16 is about it. The point here isn't Jev. It's the shape. I met a useful idea. Exploring it needed a tool. The tool needed improving before it could do the job. So I improved the tool, which let me understand the idea, which changed how I use the model in other places. Somewhere in there I said the thing I think describes this whole period of work: "We're in a state of flux where a lot of things are happening and changing."

That isn't just busyness. The work was changing the means by which the next piece of work became possible.

The trigger

Why now? Because the economics of that decision flipped. For most of my working life, a tool's limitation was a fixed condition. You learned its quirks and worked around them, because changing the tool was a project with its own cost, owner and schedule. The limitation was annoying, but tolerating it was cheaper.

With the floor from Chapter 13 falling, that comparison comes out the other way more and more often.

You can hear the change in how a limitation gets described. The old stance: "That is how the ingester works. Work around it." The new one: "The representation is preventing understanding. Change the representation, check it, and continue." The first treats the tool as part of the terrain. The second treats it as part of the job.

One day's receipts — the ingester that needed improving first, and a decision-model cache that took a slow rerun down to a fraction of a second — are in Chapter 21, with the cache's numbers. What they show is the pattern this chapter is about: fixing one thing changed which thing was worth fixing next.

Capability as a step, not a wall

I've written about the agent-side version of this before. In The Self-Equipping Agent an agent recovering an old archive built its own reader, hit a four-gigabyte limit in software it had written ten minutes earlier, re-engineered its own driver in a few minutes and carried on. The line I drew from it: "The extraordinary part is that capability acquisition became a step inside the workflow — not a wall the workflow ran into." And: "The pipeline is the trace left behind by thinking."

What's new is where the same logic reaches. That was an agent revising tools it had just made, mid-mission. The economics now reach your standing tools — the ingesters, classifiers, renderers and representations you've been living with for months — because asking for the fix costs less attention than tolerating the limitation for the work you can see coming.

The same breath, at the scale of the workshop

In Cycle Compression I described the probe–interpret–revise pattern running at three scales: seconds inside a single investigation, hours inside a goal, days to weeks at the level of a whole framework. Revising the workshop is the middle scale. The idea arrives while you're working; the tool gets improved while the idea is still warm; the improved tool changes what you do with the idea. The breath doesn't wait for a separate tooling project to be scheduled.

What the research says about agents improving their own harness

The pure-software version of this now has a research literature, and one paper's diagnosis matches this book's. The authors of Agentic Harness Engineering built a loop in which a coding agent improves its own harness — its tools, its middleware, its memory — and concluded that the question "is bottlenecked by observability, not by agent capability". Their loop gives every editable component a file-level representation, distils the agent's run history into evidence it can actually read, and pairs every edit with a prediction checked against the next round's results — which, they write, turns "every edit into a falsifiable contract".³⁰

They're honest about the limit, and it's the most useful sentence in the paper for this book: "The agent can justify why an edit should help, but it cannot reliably name the tasks the same edit is about to break." That's a single research campaign on one benchmark, and it's about coding agents, not films. But it's the perception ceiling from Part II turned inward. An agent revising its own workshop still needs a view of what the revision broke — and without one, it will tell you a persuasive story about why the change was good.

Practitioners supervising coding agents have landed somewhere similar. A study of experienced developers found them concentrating effort in planning, handing supervisory work to other agents, and "turning recurring guidance into reusable assets".³¹ Turning guidance into assets is the workshop being revised, one piece of repeated effort at a time.

Stable purpose, revisable means

The organising principle is short enough to remember and easy to violate: stable purpose, revisable means. You can change the ingester, the classifier, the representation or the renderer without losing what the original work was for. What you mustn't do is let the means drift until nobody remembers the purpose — that's how a team ends up with a beautifully tuned pipeline doing the wrong job.

The fence: revise the apparatus, never the authority

There's one boundary on all of this, and it's absolute. In The Generative Pendulum I separated two kinds of deterministic machinery by a single question: "If this code is wrong, do I get a bad view — or a changed world?" I won't re-derive it here. The application is simple. Revise freely whatever only changes your view: ingesters, packaging, judges' inputs, caches, renderers. Never let the revision loop redraw what binds: permissions, publication, spending limits, source identity, the gates themselves.

The security-review agent I've mentioned twice already ran exactly this way. "The instruction was loose on method and hard on fences, which is the ratio that makes this kind of run survivable." It could rewrite every part of how it perceived the problem. It couldn't move a single thing that decided what counted as a finding. The Agentic Harness Engineering authors did the same for the same reason — their loop was blocked from "disabling the verifier, swapping the model, or raising the reasoning budget", so every recorded gain stayed attributable.

"Isn't constant tool-fixing just yak-shaving?"

Sometimes, yes. The inequality is the test, and it cuts both ways. If fixing the tool would cost more attention than tolerating the limitation for the work you can actually see coming, work around it and write down that you did. The mistake isn't declining to fix tools. The mistake is applying last year's price to the decision — assuming the fix is a project when it's now a paragraph.

And there's a compounding case that yak-shaving doesn't have. When one improvement makes the next one visible — the cache that exposes the next slowest step, the better representation that exposes a question you couldn't ask before — the workshop is getting better in a direction set by the work. Chapter 21 shows one day of that.

Small judgments, everywhere

Look closely at the tools that get revised this way and a pattern shows up. A surprising number of them contain a small judgment — is this a real screen change, is this email aimed at a machine, is this comment a question — that plain code can't make and a large model is too slow or expensive to make thousands of times. Those judgments have their own economics now.

I described this at the time as an aside. It isn't one.

When I was thinking this through with an AI partner that week, it pushed back on my framing: Jev belongs in the thesis, not in an aside. It was right. Everything in Part IV is about what human attention and machine cost make worth doing. The smallest unit of that calculation is a single judgment inside a process — and its price just fell.

The old forced choice

Every process has branches that are almost mechanical. Is this frame a new screen or the same screen with a cursor moved? Is this email trying to talk to the AI that will read it? Is this comment a question, or praise? Is this the same response we've already seen?

Until recently, a slightly ambiguous branch like that forced an awkward choice. Write brittle rules — keyword lists, thresholds, regular expressions — and watch them fail on the first case nobody anticipated. Or call a relatively expensive reasoning model, and pay in seconds and money for something a person would decide at a glance. Most teams picked one and lived with the cost: brittle everywhere, or expensive everywhere.

A fast, bounded decision model creates room between those two.

What the middle layer is

The instance I've been using is TypeSafe's Jev, and its makers describe it well — I'll quote them, and label what follows as vendor documentation. "Think of Jev as a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out."³² It doesn't write prose. You give it state and typed questions, and it gives back structured answers your code can use directly: choose an option from a list, score against a rubric, or say whether a statement is true.³³

The design guidance matters more than the model, because it's what makes the layer safe to use. Questions should be "a gut-check determination: the kind of judgment a highly knowledgeable person could make in a few seconds given the right context. If the question you want to ask would require extended reasoning or weighs multiple independent factors, decompose it. Ask each factor as a separate question, then combine the results with logic in your code."

On speed, the vendor quotes 70 to 500 milliseconds end to end, and — unusually — hedges its own headline multiples, saying they are "on the higher end of real world gains". In my own pipelines a call typically comes back in about a third of a second. Either way, it's fast enough to sit inside a loop that runs thousands of times without anyone noticing it's there.

The three-layer division of labour

I arrived at that division by building, and it was a little startling to find the vendor's own build guidance saying almost the same thing: "Code handles deterministic work and owns the control flow. The model appears only where the system needs programmable common sense or needs to interpret unstructured data. Each AI task is kept atomic and constrained."³⁴ And from their notes on the model's limits, a line that could be the whole chapter's boundary in two sentences: "Extraction is a judgment, so give it to the model. Arithmetic is not, so keep it in code."³⁵

It's also the answer to a false fight I've been arguing against for a while. In Give the Agent a Workshop I put it this way: "AI versus deterministic code" is a false fight — "Pure LLM output about a database is plausible fog. Pure SQL returning thousands of rows is exact fog." The three layers are what you get when you stop picking a side.

The economic effect

That's the same price mechanism I described for code in The Generative Pendulum, one layer up. There, cheaper code authoring made smaller computational instruments worth building — the filter for one question, the probe that lives for ten minutes. Here, cheaper semantic decisions make finer-grained adaptive behaviour worth building: a judgment at every frame boundary, every inbound message, every comment — places where no one would ever have put a person, and where a large model was never worth the wait.

The vendor makes the same bet from its side, and names its model for it: after William Stanley Jevons, because "Every order of magnitude drop in the cost of intelligence unlocks orders of magnitude more use cases." That's their bet, not a finding. But it's the machine-side twin of the falling floor in Chapter 13. For you, smaller improvements become worth asking for. For your processes, smaller judgments become worth automating.

The fast lane, moved inside the process

This has an ancestor in my own work. In The Fast-Slow Split I argued for separating a real-time conversation into a fast lane and a slow lane, because "the part that talks doesn't need to think, and the part that thinks doesn't need to talk fast." That split lived inside a conversation. This one lives inside a process: many small, fast decisions, and a few slow, meaningful ones. Same principle — don't make one clock serve two jobs.

The boundary

Speed invites over-reach, and there's one line this layer must not cross.

Fast small decisions can organise the evidence; they should not prematurely settle its meaning.

Let a bounded model sort duplicate responses from new ones, tell a question from praise, and nominate which comments deserve a closer read. That's organising. Don't let it decide what a campaign taught you, or whether the promise landed, or which explanation of a result is true. That's meaning — and the interpretive job belongs to a stronger model with the complete relevant context: the audience, the offer, the intention, the actual film, the accumulated evidence. And, in the end, to you.

Perceptual coverage applies here too

A fast model is still an instrument with a range, and its range has a hard edge. "Jev currently accepts text input only. It evaluates strings, JSON objects, and arrays of text. Images, audio, and video are not supported (yet)."³⁶ So a bounded model asked about a film is judging a text or measured view of it — which means the silver from Chapter 7 and the facts from Chapter 6 decide what it can decide. Give it a transcript and ask about a pause, and you've built Chapter 5's deaf judge again, faster.

Its confidence is statistical, too. The vendor is careful about this: "Calibration is measured across groups of predictions; it does not guarantee that an individual answer is correct." The practical form is a pair of thresholds you set on your own examples — above one, act; below the other, escalate to a person or a stronger model; in between, log and watch. And write the questions as carefully as you'd write tests, because, in the vendor's own words, the model "answers the question you wrote, not the one you meant."

"Why not just use a big model for every decision?"

Three reasons. Latency: these decisions sit inside pipelines that run thousands of times, and a few seconds each turns a pipeline into a queue. Cost: at that volume, the arithmetic is obvious. And grain: big models are for meaning, and sorting isn't meaning. Using a frontier model to decide whether two frames are the same screen isn't more careful. It's just slower. The minimum judgment worth automating only falls if the judgment is cheap.

Not opposing trajectories

For a while the debate was framed as a choice: more AI or more code, models or programs. The three layers dissolve it. Code gets more capable because cheap judgment can be dropped into it at any branch. Judgment gets more reliable because code owns the control flow around it and measures the facts it can't perceive. More code and more AI are not opposing trajectories. They can make each other more useful.

That's Part IV: the floor and the ceiling, the revised effort claim, the workshop that can be edited, and judgment cheap enough to put everywhere it's small. All of it changes what's worth making. Part V follows that to where it leads — to a different idea of what the thing you're making even is.

Most film briefs start from the same question: what video should we make?

It's a reasonable question, and it quietly assumes the video is the product. Everything in the first four parts of this book has made that product better. Handles to change it, judges that can perceive it, an audience loop to learn from it, and an attention budget that makes small improvements worth asking for. This chapter asks whether "the video" is still the right unit once all of that is in place.

From animation to situation

A fixed animation is a recording of choices: this shot, this line, this pause. By the time a code-built film is working well, though, the construction holds more than the choices. It holds the reasons for them. Who the customer is. What went wrong for them before they found the service. What the service actually does, step by step. What changes afterwards, and what they'd want to know next. The film shows one path through that material. The material itself is closer to a situation than to a script.

Once a situation like that is rich enough, it can be played more than one way. A film is one guided path through it. An interactive demonstration can let the viewer choose another path. A sales conversation can explore a different condition. A training exercise can ask the person to make the decision themselves.

Those are not necessarily separate products. They can become different ways to encounter the same underlying situation.

Take the usual set of deliverables for a service business: a marketing film, an explanation on the website, a sales demo. Today they're typically made separately, often by different people, and they drift apart. Instead, you could build one representative customer situation and expose it in different ways.

Four ways to encounter one situation

The costs in that last column are shapes, not numbers. I haven't built all four doorways for one situation and costed them, and I'd distrust anyone who gave you figures for it today. What the shape shows is where the expense sits. The first doorway is the one we already pay for. The second and third are mostly exposing what a code-built film already contains. The fourth is the only one that needs something genuinely new, and it's also the one that most needs a judge watching it.

Radial, not serial

There's a structural reason this matters beyond convenience, and I've written about it in a different setting. In Demonstrate Over Tell the problem was summaries: each version made from the previous one, until the chain "cut the route back" to what was actually said. The fix was a change of topology. Instead of a chain, "make every version directly from the preserved source, so that 'nobody is ever more than one hop from the truth.'" The rule I drew from it was four words long: "Keep the depth. Change the doorway."

That book applied the idea across formats for things people had said. An executable situation extends it to things people will experience. The film, the demo, the sales scenario and the exercise are all generated from the same source model. They stay consistent because they're siblings, not copies of each other. When the offer changes, you change it once, in the situation, and every doorway regenerates. Compare the usual arrangement, where the sales deck is made from the website, the website from the brochure and the brochure from a meeting nobody recorded.

The idea has a lineage

None of this is a new wish. In 2011 Bret Victor described what he called explorable explanations. A reactive document, he wrote, "allows the reader to play with the author's assumptions and analyses", and an explorable example "makes the abstract concrete, and allows the reader to develop an intuition for how a system works."³⁷ He described such a document being read at several depths. "The hurried reader can skim it. The casual reader can read it as-is. The curious reader can adjust the author's scenarios. The engaged reader can explore scenarios of their own devising." Set that beside the four doorways and it's close to the same grid, fifteen years earlier.

What held the idea back was not the vision. It was the authoring. Victor named the bottleneck himself: "Explorable explanations won't catch on if they're difficult to author. Tools must be invented that enable and encourage authors to make their work explorable. Can it be almost as easy as writing static text?" For most of the time since, the answer was no. Each explorable piece was a small software project. That's exactly the cost that code-writing agents have collapsed. The water simulation from Chapter 1, the films with scripted pauses and the probes that live for ten minutes are all cases of the same thing: authoring an executable model now costs something closer to a conversation than a project.

In a 2024 postscript Victor sharpened what he'd meant, and his version makes a demanding standard. "a written argument whose assertions are backed by explorable computational models, whose facts, assumptions, and calculations are all visible and editable." And then: "The reader rebuts by modifying the models." For a service business, that's the second and third doorways stated as a principle. Don't just claim the service handles the difficult case. Let the sceptic change the condition and see.

Playable models teach differently, though not magically

There's evidence that interacting with a model changes how people learn, as long as it's read at its own scale. A widely cited meta-analysis on simulations in science and maths education compared computer-based interactive simulations with similar instruction that didn't use them.

That's classroom science, not marketing, and the studies predate everything in this book. I'm not claiming a playable sales demo will lift anything by a measured amount. The transferable finding is about mechanism. The PhET researchers describe what happens when a simulation works: "When something unexpected happens, the student questions her understanding and changes parameters in the simulation to explore and improve her understanding." That's the change-a-condition doorway doing the job Chapter 11 set for any communication: moving someone's understanding, not just their attention.

The same paper contains a warning, and it cuts straight back to Part I. Students, the authors note, "trust the simulations to be correct." People extend that trust to a playable model because it responds like the real thing. So a playable situation has to earn it. In Chapter 2 I borrowed a benchmark's phrase for the failure: code that "runs but simulates the wrong physics". A situation model can run beautifully and simulate the wrong business. The delivered account has to match the world, not just the intention, which is why the inspect-why doorway isn't optional decoration. It's how the model stays honest in public.

Where this doesn't reach

This is a direction, not a property. A timeline doesn't contain a business model. Most films made with code are sequences of decisions about shots, words and timing, and there's no situation underneath them to play. Nothing in Parts I to IV turns a film into a simulation by accident.

What makes the direction reachable is a particular combination: company knowledge you already hold in usable form, a runtime that can render and respond, generated interfaces, and preserved source. If you have those, the extra step is smaller than it looks. If you don't, the four doorways are four separate projects again, and you're back to the old costs.

The restraint

Now the guard I planted in Chapter 2 and promised to come back to. The more of a film that lives in an executable model, the stronger the pull to specify all of it: every beat, every transition, every colour as a parameter someone has set. That's the wrong lesson.

In Demonstrate Over Tell I split the job of directing a capable model into three parts. Aim: a short nudge naming the kind of move the audience needs. Eyes: letting the agent see what it made, the way the audience will. Canvas: giving it the means to compose what it imagines. The warning attached was that layout prescription "caps the result at the quality of whoever wrote the specification", and the line I drew was between things to specify exactly and things to nudge. Facts, prices, names, terms and the honesty rule get specified. Layout, composition, imagery and tone get nudged.

A situation model makes that split cleaner, not harder. The facts belong in the situation, and they're specified exactly, once, for every doorway. The expression in each doorway stays free. The same book warned about the template trap, a good expression hardening into the only expression. "The family resemblance should be visible value, not identical chat bubbles." Four doorways into one situation shouldn't be four copies of the same layout. They should be four different ways of making the same value visible, each shaped to the person who comes through it.

Executable precision should not become creative micromanagement.

That gives three things three different jobs. Handles, from Chapter 2, are for experiments and repairs: changing one thing on purpose and learning from it. The situation model is for honesty and consistency, so every doorway tells the same true story. And the expression, the part that makes someone lean in, stays with the model's judgment and yours, checked by eyes rather than dictated in advance.

"Isn't this just an expensive gimmick?"

Two parts to that, and one of them is right.

The expense was the authoring, and that's what collapsed. Victor's bottleneck, "Can it be almost as easy as writing static text?", is much closer to yes than it has ever been, for exactly the reasons this book has been describing.

The gimmick risk is real. An interactive demo that lets someone drag a slider because sliders are fun doesn't serve anything. The test from Chapter 11 applies to every doorway: what change in understanding is this meant to produce, and for whom? If you can't name the concern a doorway answers, don't build it. The four ways are worth offering only because they answer four different concerns: what is it like, why should I believe it, does it fit my case, and what about my case.

The new starting question

So the question at the top of this chapter changes. "What video should we make?" picks the form before the purpose, and after that the best you can do is make a better video. The question worth starting from puts the person first and lets the form follow.

What should this person be able to understand or experience — and what form would let them get there?

Sometimes the answer is still a film. Often it will be a film plus one other doorway. Occasionally it will be something that has no name yet. That's the doctrine: code as proxy, the machine's perception ceiling, the audience's answer, the human's attention ceiling, and now the situation as the product. Part VI turns to the week it came from, built as code, with the receipts.

The week Claude Sonnet 5.5 came out, I was watching Matthew Berman put it through its paces.

He runs one of the more popular channels for this kind of thing, and he'd been testing the model early. Among his demos was a three-dimensional ocean simulator the model had built. His own caption was "3D simulation is solved", and shortly afterwards he posted another clip of it with "Truly one of the most insane demos I've ever seen."¹⁴⁰

What I saw was a yacht moving through open water, with ripples and reflections, and, as I understood it, the model working in a loop to make it look better on each pass. I said it out loud, more or less as I thought it:

"I have never seen such good video of water in my life. It looks unbelievable, and it's code generated."

And then the part that stopped me: "It's not AI black-box generated. It's come from code. It's code simulating physics and light."

My next thought was less generous, though it wasn't meant as a jab. "He's a smart bloke, but I don't think he knows what he's demonstrating." The demo was received, by him and by most of the people watching, as news about a model: look how good the new one is at graphics. What I thought it demonstrated was a medium. The model hadn't produced a picture of water. It had written the machinery by which water, light, wind and a boat could behave, and that machinery held still between passes, so a loop could improve it. I recognised it because I'd just been doing the same thing with films. In my words at the time: "he put it in a loop, sort of the same thing as I just did, to make it look better."

I don't know how physically right that water was. Chapter 2 explained why that's a separate question from whether it looked right, and it stays separate here. What I'm claiming about the ocean is narrower. It was built as code, and code is what made it improvable.

The request that started the films

Some context about my own work. I run a system that stands up what I call company boxes: each is one of the business concepts from my own portfolio, with its own agents, its own website and its own mail, running in its own virtual machine. One of them is AfterWard, a concept for helping patients after they've come home from intensive care. We'd spent a lot of time on the websites, and they'd got very good. Earlier that week, agents had rebuilt them in a loop, one night.

I wanted more. The message I sent to start the film work (lightly tidied) said:

In the same message, before a single frame existed, I named the problem this book is about: "BUT we need a judge, but how to judge a video — not easy."

The tool that made it possible was HeyGen's open-source HyperFrames, which defines a video as a web page with timing attached and renders it frame by frame, so the same input gives the same film.² My reaction a day or so in: "I'm using HeyGen's open-source framework for making deterministic videos. And it's unbelievable."

The pipeline, as built

Here's what the pipeline turned into. It's a description rather than a tutorial, and every part of it is something an earlier chapter has already argued for.

I won't re-explain the parts. The loop is Chapter 3's. The viewing step is the packaging strategy from Chapter 7, pointed at films instead of screen recordings. The measured facts, including loudness, timings and scene changes, are Chapter 6's "code measures the facts". The answer key written before the film is Chapter 8's legibility at birth. The pipeline wasn't designed from this book. This book was written afterwards, from the pipeline.

The first film

The first AfterWard film rendered that night. My whole review, as sent: "wowza that was awesome".

That was the honest reaction, and it wasn't the end of the review. The first cut opened on a question, "Who's reading this?", which turned out to be ambiguous. Coming over a dark room and a glowing phone, it could even sound like surveillance. The pair of lines that replaced it stated the stakes instead. The first set up a patient at home after intensive care, with her breathing getting worse. The second began with two words: "AfterWard notices." Those two words would matter much more than I knew. They're at the centre of Chapter 19.

The next exchange is the one I'd point to if you asked where the craft was. The system wrote up a recipe for the next company, and it had turned one of my own notes into a rule: open on the person's own words, in their own voice, typed onto the product as they speak. I pushed back on my own note:

That's Chapter 17's restraint, turning up before I'd named it. The device worked because of what it did for the viewer: recognition, the feeling of being seen. Make it a rule and it becomes the next template. What should travel is the purpose, and the device should be free to change.

Harvesting the craft, then testing whether it travels

Then, before we made the tall cuts for social media, I asked for the work to be kept:

What came out was a brand-film skill and a kit builder. The skill carried the craft. Demonstrate over tell. An "I see you" moment. One job per channel, so the voice, the screen and the music aren't all trying to say the same thing. Pauses treated as content, not as gaps. And honesty. The kit builder harvested a company's live website into film material: every page's words and screenshots, every image with what it shows, the brand's fonts and colours.

Then came the part that turned a good result into evidence. Instead of just making more films, we set up an experiment. Two fresh builders were each given only the kit: the skill, the company's documents, the harvested site, the tools and a short prompt.

The judge brief and the answer key for Paddock Relay were written before either film existed.

The progress report halfway through showed what the Paddock Relay builder had found, and none of it was copied from AfterWard. It had found its own spine: one example job, 480 hectares of fallow, against one spray window, with a clock running from Monday at 05:40 to Friday morning. It had found its own "I see you" moment: a farm manager in his ute before sunrise, ringing round contractors, with the fourth call going to voicemail and the target feeling of "that's my Monday." It invented a device of its own, a relay rail along the bottom of the screen passing a baton between the parties so that "who decides?" is answered at every moment. And it built in the honesty lines the company's offer required: "Nothing books without your yes", and "the go or no-go stays with the licensed applicator".

The principle had regenerated in a new world, and nobody had cloned the expression. That's the family resemblance Chapter 17 asked for, produced by a builder working only from the skill and the kit.

The voice my ears caught

My verdict when the Paddock Relay film landed:

The cause, once found, was almost funny. The opening voicemail used a text-to-speech voice that the engine's own authors grade D+. The narrator used one they grade A. The builder hadn't made a processing mistake. The skill had told it to audition voices by measuring pitch and checking that the transcript came out clean, and neither measure catches quality. It picked a bad voice in good faith, and every check it knew how to run said the voice was fine.

We banned the weak voices. Only voices graded B- or better could be used, with the reason and the alternatives given whenever anything else was requested. That exposed a catch: the engine had no male voice above C+. So the skill now says to recast the character, or let the words carry on screen with no voice at all.

I didn't have the words for it yet, but that was Part II in miniature. The builder's checks couldn't perceive the thing that was wrong, so the builder couldn't improve it, and a person listening caught it. That voice became the first calibration pair in the next chapter.

Twelve sites, each building its own film

By then the obvious next step was the whole portfolio. The goal I set was specific. Every company site would get its one-minute film, wide and tall, on its own watch page. Each film would be made by our own harness running inside the company's own virtual machine, not by an outside coding tool. Each would be judged before publishing, by a model score and a cold viewer, and anything under 7, or with an honesty gap, got one fix round before it went up.

At the bottom of that goal I added a note about what I didn't trust. Chapter 19 opens with it.

On 29 September, all twelve sites were serving their films, and nine of them had been made by the harness inside the company's own box. The loop that had produced one film by hand a day or so earlier was now producing them across a portfolio, each with its own brief, its own answer key and its own judging.

Everything else that week

It would be tidy to say the films were the week. They weren't. The strip at the top of this chapter is the honest version, and Chapter 15 has the line I used for it at the time, about a state of flux. Websites rebuilt in a loop. A mail guard built from deterministic code and a fast decision model, then attacked with borrowed test suites (Chapter 21). The video reader improved in the middle of the task that needed it, and a cache added when the decision model became the slow step (Chapter 21). My intent tracker reconsidered as something that should carry intended, delivered and observed accounts side by side (Chapters 10 and 12). And, the same week, a short series of films for a small cosmetic-dental clinic, which is where the machine-readable sibling earned its keep (Chapter 20).

What connected them wasn't the subject matter. Each one was a case of something becoming code, so that it could be viewed, judged and changed one piece at a time.

What shipped with the films

One of the film series went out with a short disclosure on its review page, and I think it's the right default for anything made this way. In substance, generalised from that page:

That's a plain statement of method, and it doesn't apologise for anything. It also quietly records the lesson of the next chapter. The phrase "one of which listens to the sound" is there because, when this week began, none of them did.

What this week demonstrates, and what it doesn't

The obvious objection is that this is one builder in one week. Yes. That's exactly what it is, and it's why every claim in the next three chapters carries its date and the record it came from. A week of one person's work can't establish a general rate of anything. What it can do is show a method working end to end, with its failures in view, closely enough recorded that you can check it and run it yourself.

The question I kept asking

By the end of the week the films looked good to me, and they passed their judges. But I'd already caught one thing by ear that the builder's own checks had passed. I kept coming back to the same question. Could the judges hear what I heard?

So I tested them.

The goal I'd set for the twelve sites said every film would be judged before it went up. At the bottom of it I wrote what I actually believed.

That doubt had been there from the first message, when I asked for a film and in the same breath said "how to judge a video — not easy." By now the films were passing their judges. The voice in the last chapter had shown me that passing and being good could come apart. So before trusting the judges to referee anything, I had an agent test them. The brief it was given began: "Scott doubts our film judges can notice the quality differences he can see and hear. Find out, using his own verdicts as ground truth."

That's the move from Chapter 6: calibrate against your own verdicts. What follows is what it found.

Where the pairs came from

A calibration set is only as good as its ground truth, and mine wasn't invented for the purpose. Each pair came from a decision I'd already made during the week, for my own reasons, before anyone thought of testing a judge with it. There were five.

The voice. Paddock Relay's opening voicemail, a farm manager ringing round contractors at dawn, was first voiced with the synthetic voice I'd called "crappy compared to the rest, seems like older tech." It was re-voiced using one of Google's voices, Algenib, with a direction along these lines about who was speaking: a gruff, practical farm manager, a little rushed, leaving a voicemail from his ute at dawn, with a short beat after "Morning". Many takes were generated and checked by code for the exact words and the beat, and then my ear picked one. My verdict is written into the recipe file so it can be reproduced: "the voice on final-revoiced.mp4 is perfection". And: "that gruff voice in Paddock was great, make sure the recipe for that is recorded, because it was amazing." That gave the first pair: the weak voice against the one I called perfection.

The pitch-line pause. On AfterWard's film, I said this about timing:

The rule that went into the skill came out in my words as a single line: script the gap, don't hope for it. The film with the scripted pause and the film without it became the second pair. The next section walks through it.

The opening. Two different openings of the same film, where I'd said which one worked.

Two revisions. AfterWard's second version against its first, where I preferred the earlier one. That's a verdict that runs against the assumption that the later revision is the better one, which makes it a useful test.

The promise pause. A hold of about a second and a half after the line in Paddock Relay's film that carries the promise.

Five pairs with known answers, each about a difference I could hear, and each a difference that mattered to whether the film worked. That's a small set, and I'll come back to what that does and doesn't license.

The controlled variation, walked through

Chapter 4 promised this: one change to a film, with everything else held, handed to the judges. Here it is exactly as the record has it.

In the earlier cut, v1b, "AfterWard notices." ran straight on into the next sentence. In v1c, "AfterWard notices." stands alone as the pitch line. Then 0.8 seconds of scripted silence follows, with the music still ducked, while a card builds on screen. Then the next line, "Every answer, weighed against your hospital's rules, and every answer before it.", lands on the cut to the nurse. The captions on the tall cut were rebuilt to match, as two sentences.

And here is the change as the build script recorded it. These are the lines from the record, unedited:

A_START, GAP = 6.45, 0.8
a_words = [("AfterWard", 0.0), ("notices.", 0.72)]
a_dur = 1.472
B_START = round(A_START + a_dur + GAP, 2)          # 8.72

The pitch line starts at 6.45 seconds and lasts 1.472 seconds. The gap is 0.8. The next line starts at their sum, 8.72. The dramatic beat that I said carried "the whole pitch" is one number in a file. That's a handle (Chapter 2) and a controlled descendant (Chapter 4) in four lines.

It isn't laboratory-pure, and it's worth being exact about why. To script a gap after a line, the line has to end, so the pitch line was split out and voiced on its own, at a slightly slower speed. That's the honest shape of controlled variation in practice: one intended change, plus the smallest set of dependent changes needed to make it, all written down. What it doesn't include is anything accidental. No new words, no different voice, no reshuffled shots, no remixed music. In a generated film, getting even that close would have meant regenerating and hoping.

Now, what the instruments did with it.

• The listening judge missed it. Given both films with sound, it didn't reliably prefer either.
• The reading judge picked the paused cut, but partly for a reason that doesn't exist. It was working from the transcript, and the transcriber had produced a stumble ("We- Week four") that isn't in the audio. The audio of that line is identical in all four AfterWard cuts. The judge got the right answer partly because of a phantom.
• Our own pause-measuring code missed the pause entirely. The silence after "AfterWard notices." really runs about 0.93 seconds, and the tool didn't report it at all. On the other film, it read the promise pause, really 1.51 seconds, as 0.93.

Only my ears reliably heard it.

That's Chapter 4 and Chapter 5 meeting in a single diff. Code made the change precise, cheap and reversible. It did nothing to make the change perceptible. A loop running on those judges could have added or removed that pause a thousand times, and its scores would have moved for reasons that had nothing to do with the pause.

The broken copies

Five pairs is a thin test on its own, so the calibration added a second kind of item: deliberately broken copies, built by code from the films themselves, each differing from the original in exactly one way. These are the counterfactual audits from Chapter 6. The judge either perceives the one change or it doesn't.

The last one is the cruellest and the most useful. There's nothing to hear. Any judge that reports hearing something is reporting its own expectations.

The calibration table

Here is what each configuration perceived, as recorded on 29 September 2026. These are particular models, given particular inputs, through particular prompts, on that day. It's a map of those instruments' ranges and not a verdict on any model family. Change the input and you have a different instrument, which is the whole point of Chapter 6.

The reading judge: worse than deaf

The calibration's own phrase for Opus was "worse than deaf", and the voice pair shows why a deaf judge would have done better. A deaf judge knows it can't hear and says nothing about the voice. This one judged voice quality from the transcript, and the transcript had been produced by a speech recogniser that struggled with the gruff delivery I loved. The storyboard's "480 ha of fallow" came out of the transcriber as "490 hectares of fellow". Opus read that and called it bad diction. So it preferred the weak voice over the one I'd called perfection. It did this in four runs out of four, consistently, in both presentation orders, and with a reason that sounded right.

That consistency is the dangerous part. Chapter 6's first rule is to run both orders, and Opus passed that test. It gave the same answer either way. A check for order bias would have certified it. It was consistent because it was reliably reading the wrong evidence. And in score mode, asked to rate each film out of ten, it gave all seven films 8. A score that can't separate a film with a pause from one without it isn't measuring the film.

The listening judge: good ears, bad habits

Gemini, sent the MP4 alone, could genuinely hear. It agreed with my voice verdict in six runs out of six, and it named what I'd heard: the weak voice's "incongruous British accent", on an Australian farmer. It caught the crude broken copies every time. And it passed the null test: shown the same film twice, it called a tie, every time.

But everything subtle got past it: both pauses, the music raised 9 dB, the soundtrack sliding 0.6 seconds late. And what it did when it couldn't tell is the single most important line in the record.

"When it can't tell, it picks whichever film is shown first, confidently and with invented defects."

That happened in all 24 such runs. Summed over both orders it cancels out, which is why a single run of this judge should never be trusted on its own. But look at what a single run produces: a clear winner, a confident explanation and a specific defect in the loser that isn't there. That's the silent failure from Chapter 5, as a number.

Two more findings complete it. On the silent "AUDIO TEST" clip, it reported speech in six of eight runs, hearing what the picture suggested. And giving it the transcript as well, which seems like it should help, made it worse: "it trusts the text over its ears and makes up differences." The listening judge was a better instrument with less information.

Why these failures, briefly

I don't need to re-argue these. Chapter 5 listed the classes, each documented in research that never tested my setup, and this record is one dated instance of nearly all of them at once. Text over ears: the reading judge, and the listening judge given a transcript.¹³ First-shown under uncertainty: 24 of 24.¹¹ Sound from a silent picture: six of eight.¹⁴ Pauses unfound: by both judges, and by our own code.¹⁵ The literature told me what kind of failure to expect. The record told me which ones my instruments actually had, and where.

The rules I set

The calibration came back with a recommendation for using the judges to referee later comparisons. I adopted it, and it's the concrete version of the rules in Chapter 6.

Notice what the rule for Gemini's threshold rests on: five pairs and a set of broken copies. "On this data" is doing real work in that sentence. The thresholds are a calibration, not a law. They're what I'd defend for this apparatus until the next set of verdicts says otherwise. The same applies to "trust Gemini on voices". That trust was earned on one dimension, by one configuration, against one person's ears.

The instruments had bugs too

It would be tidy if the only problems were the models. They weren't. The same calibration turned up four plain bugs in the judging code itself. Pair mode had been silently running Opus whatever engine was requested. GLM had been told "no frames are shown" in its prompt while it was, in fact, watching the video. Parallel runs had corrupted four of seven cached video copies. And Opus's structured answers sometimes arrived with trailing text or a missing closing brace.

Then there's the pause meter, which we'd built to be the objective one. It was the instrument I'd have trusted over any model on a question of timing, and it was wrong on both pauses that mattered. Its logic was revised straight afterwards. I haven't re-run this calibration against the revised version, so I can't tell you it now hears the pause.

Not every instrument failed in the same direction, either. When a new voice engine started reading its own stage directions aloud, it was the transcriber that caught it: it heard "Say this is a gruff… farm manager" at the start of a test clip. The same instrument that turned fallow into fellow was exactly right about words. That's what a range looks like. Good on one dimension, blind on the next, and silent about which is which.

That's Chapter 6's point about the chain. The instrument isn't just the model. It's the model plus its input, its prompt, its cache, the code that routes to it and the code that measures for it. Each link has a range, and each can be wrong without saying so. Calibrating the judge found bugs in the plumbing, because the plumbing is part of the judge.

Unjudgeable is a state

One more failure has no score attached, which is exactly the problem. Opus refused about half its calls on these films. A content filter tripped on Paddock Relay's crop spraying and AfterWard's intensive-care scenes. Retries usually got through, but two of the Paddock broken-copy pairs, the pause cut and the frozen hook, never got a verdict at all.

Later, in the refereed comparison below, 99 of 188 Opus calls hit the same filter, and one pair was refused on every one of the 24 runs in which a particular film was shown first. If a refusal had been allowed to fall through to "no difference", or to quietly default to the other judge, the record would have looked complete and been wrong. So "refused" is recorded as a state, next to "won", "lost" and "tie", and a pair with no verdict says so.

The refereed comparison: two findings

With the rules in place, the judges refereed the comparison I'd asked about in the first place: films made by Claude Code against films made by our own harness, for four companies, with identical encodes and blind, shuffled names, in both orders. The shoot-out itself isn't this book's subject, and its overall verdict was modest: no large drop in quality from moving to our harness, a slight edge to Claude Code on some films, and far fewer turns for the harness. Two things it found belong here, because they're findings about perception.

And the calibration's warning turned up again in the wild. The reading judge complained about one film's voice because the transcript had heard "Write to the desk" as "Right to the desk". The listening judge, which could hear, called the voice natural. Under the rules, that conflict resolved to the ears.

The first finding is the one I keep returning to. The quieter music won or led everywhere, and no model judge had said a word about music level. The fact sat in a measurement nobody had asked the judges to consider. That's where the next chapter starts.

What "passed" had meant

There's an uncomfortable consequence I should state rather than leave for you to work out. The twelve films that went up on 29 September had passed a gate made of an Opus score and a cold viewer. The cold viewer's job was comprehension and honesty. Could someone arriving with no context say what the company does, and did the film leave them believing anything false? A transcript and keyframes carry most of that, so that part of the gate was doing a job it could actually do. The score's job included quality, and on this record the score couldn't separate a film with a pause from the same film without one.

So I now read "passed" narrowly. It means the claims and comprehension checks passed. It doesn't mean anyone checked the pauses, the mix or the sync, unless I did. That's not a reason to distrust the gate. It's a reason to know which parts of it are load-bearing, and to say so wherever the word "passed" appears.

What the record changed

Most of Part II was written from this record. The perception ceiling, judges as instruments, the counterfactual audit and the decision rules were all things I found here first, as failures, before they were doctrine.

There's a limit to say plainly. This is one person's ears, five pairs, one set of broken copies and one day's configurations. It doesn't tell you how these models would do on your films, with your inputs, next month. What it does tell you is that the question has an answer you can get. It took about eighty minutes of an agent's time, the rules it produced cost about a dollar and a half of model calls per film pair, and it changed what I believed about every score I'd been reading. Chapter 21 puts it in the costing table, along with the part I didn't measure.

The recommendation that came next

The practical conclusion was blunt. Stop asking the judges to perceive things they can't. Give them the facts that code can measure, stated plainly, beside the film. Better still, make the film carry its own account of what it intended and what it delivered, so the judge isn't reconstructing either.

That's what we built next.

The plan for the next round of judging put my own rule back to me, in a line I'd happily have written myself.

Facts about each film, it said, should be "measured by code, given to the judge and the builder, not scored". And then: "This fits your rule: code is the eyes, the model does the judging."

That's Chapter 6's division of labour, and Chapter 19 is why it became urgent. The judges couldn't perceive some things and misperceived others. The fix wasn't a better judge. It was to stop asking judges to measure. Here's what the eyes saw once they were built.

One note on where this happened. Most of what follows comes from a short series of films I helped make, the same week, for a small cosmetic-dental clinic. I've generalised it: no name, no place, no people, no prices, no dates. What's left is the method and what it found, which is the part that travels.

The facts, as built

A measuring step was added to the end of the "watch" stage, the code that already turned each film into keyframes, a transcript and loudness readings. It writes a short facts file for every film. Each measurement is stated plainly, with its accuracy, and nothing is converted into a score.

Two design choices in that table matter more than the arithmetic.

The first is the length wording. I asked for it to be neutral. A film that's 13% over its budget is told exactly that, and nothing more, because a minute and seven seconds can be right for a film planned at a minute. Only a film a third or more over gets the stronger phrase. The fact informs the judge. It doesn't pre-empt the judgment.

The second is what happened with text size. The on-screen reader at that point returned lines of text with no box sizes, so there was no honest way to measure how big the smallest text was. The facts file didn't guess. It said "not measured", gave the reason, and stated the legibility floor for that film so a person could check. A measurement tool that says "I can't see this" is worth more than one that fills the gap with a plausible number.

The stillness measure needed calibrating, and the calibration values are themselves instructive. A frame pair counts as "nothing new" at a difference of 12 or below with the camera locked, or 22 or below while the camera is moving. Locked end cards measure 0 to 6. Posed objects under a slow push-in measure 9 to 13. Real work happening on screen, like material going on, being shaped and being polished, measures from 25 up to 162. And a single small line of text appearing on an end card measures 25 to 28, which is enough to count. The thresholds sit in the gaps between those ranges, and they're only trustworthy because someone measured what "nothing" and "something" look like on real films first.

Here's what the output looks like. This is from the earlier cut of the series' main film, abridged, with the prices and the wording of the next step replaced by placeholders:

Length 67.6 s, budget 60.0 s: +7.6 s (+13%).
Static: 62.62-67.38 s (4.75 s), camera held (end card)
End card: from 61.25 s, 6.35 s long; new things 61.62-62.88 s (8% of the frame); holds 4.97 s
Voice vs music: between -18.2 LUFS, during -14.6, gap 3.6 dB
[offer price]: **not said** | shown 46.25-50.75 s
[next step]: said 51.28 s, 62.4 s | shown 50.75-57.25, 64.75-67.6 s

And the first line from a short teaser in the same series, the only film that earned the stronger wording:

Length 26.816 s, budget 18.0 s: +8.82 s (+49%). **way over budget: +49%, a third or more longer than planned.**

What the facts put beyond argument

It would make a better story to say the facts caught things the judges had missed. That's not quite what happened, and the truth is more useful. Across the review rounds, the reading judge had already marked pacing down more than anything else, and sound came up again and again. The cold viewer had already noticed that two films left people unsure of the price. The problems weren't invisible. They were impressions, stated differently by each judge in each round, with no time attached and nothing the builder could act on precisely.

The facts turned those impressions into measurements that were identical for the builder and the judge:

• The unspoken offer. The earlier cut of the main film showed its prices on screen for about four and a half seconds and never said them. For anyone watching with the sound on and their eyes elsewhere, the offer didn't exist.
• The overlong teaser. A short film nearly half as long again as planned, with a twelve-second end card that went still for its last few seconds.
• The bed as loud as the voice. In one earlier cut, the music between words sat just 0.8 dB below the voice during words. It was, in the build record's words, a bed "almost as loud as the voice". In the revised cuts the gap was between about 7 and 9 dB.

Posed against working

The finding I like best is one I didn't expect code to be able to make. Chapter 11 made the demonstrated moment the unit: a thing the viewer watches happen. By that standard, a tool posed beside a finished result is a still life. That sounds like a matter of taste. It turned out to be a matter of measurement.

A stillness meter, built to catch dead end cards, separated a still life from a demonstration. The judge had already asked for the change to be shown. The meter said exactly how little was changing, shot by shot, and the builder had a number to move.

How the judges used them

The judges were told to trust the facts over their own estimates, and a later calibration checked whether they did. They used the facts correctly. Length was treated as feedback rather than a verdict: only the teaser that ran almost half again over its budget was marked down for it. The unspoken prices, the static end card and the loud music bed all turned up as the right notes, in the right places.

Then the eyes improved. Once the on-screen reader returned box sizes, text size stopped being "not measured". On a later film in the series, code measured the end card's text at 19 pixels on the wide cut and 22 on the tall, below the legibility floors, with five items sitting where a phone app's buttons would cover them. The end card was recomposed. No judge had to squint at a keyframe and guess.

Facts had one more advantage on this material that I hadn't planned for. Under the new rubric, every one of the reading judge's image-based pair calls on these clinical films was refused by its content filter, 140 out of 140, so a text-only comparison was all it could do. Facts are text. They got through a filter that the pictures didn't, which meant the judge was never working blind on length, stillness or what was said.

What facts don't do

None of this makes the facts the judge. A few seconds of stillness at the end of a film is the skill's own instruction: once the end card's lines are spoken, hold it complete for three or four seconds. Twelve seconds of it isn't. A meter can't tell those apart without a purpose, and the purpose isn't in the pixels. That's why the facts are stated rather than scored, and why the same fact can be fine in one film and a fault in the next. Code says what happened. The judge, given the intent, says whether it should have.

The sibling in practice

Chapter 8 described the machine-readable sibling in the abstract. Here's what each film in the series actually left production with.

Writing the intent note first turned out to be worth more than it looked. When we wrote a review page explaining the series, we described every film under the same four headings: its job, its story, how it links to the viewer, and where to use it. The build record's verdict was that this "was clearer than any brief we wrote". So the headings went back into the builders' instructions as the note they write before they build. The explanation we'd written for people became the specification for the machines.

The review page also named the strongest device in the series, which three builders had found independently: the person's own words travel through the service. What someone says at the start, like "change the shape, keep the colour", comes back later in the consultation notes and on the design lines. The viewer sees their own words shaping what happens next. It's the "I see you" moment from Chapter 18 in a stronger form. Being recognised is good. Being heard is better.

A revision that could say "only this"

The sibling also made revisions precise. On one film my verdict was that the opening scene connected, but a phone mock-up inset into it didn't: "the image connects, just not the iPhone/web page. The inset image." Because the film knew its own parts, the fix brief could be exact. It kept the opening scene and its lines, replaced only the inset with something from the viewer's world that a stranger scrolling past would recognise, left the middle untouched, and recomposed the end card from its measurements. The lines were re-voiced with voices cast from the audience's own region, keeping every timing. That's Chapter 4's descendant brief, working on a real film. It names what's kept, what changes and why, so the revision can't quietly drift into a different film.

Declared against perceived

The last piece of the sibling, the declared moments, produced the most useful method finding of the week.

The cold viewer is a fresh model given no context: the cold-arrival reader from The Exemplar Ratchet, put to work on film. As Chapter 9 put it, that's a screen, not a verdict: a model watching a film is not a customer watching it. It was asked two new questions. Was there a moment that felt like someone like you, and what detail? When did you understand how it works, and what did you see happen? Its answers were then graded for how specific they were.

Self-report didn't discriminate. The cold viewer's answers were graded "specific" for every final cut and every earlier cut alike. If I'd relied on that, I'd have concluded every version had worked.

The declared-moments check did discriminate. It compared what the builder said would happen, and when, against what the cold viewer actually found. On two films the declared "aha" wasn't found, and those were the two films the judge had already marked down on that moment. The declared check and the judge agreed, and the self-report disagreed with both.

That distinction is small and it matters everywhere. An open question invites a plausible answer, and a model is very good at plausible answers. A question with a declared target and a time has a right answer and a wrong one. That's the same move as the answer key written before the film. It fixes the target before the result exists, so the result can't bend it.

The three accounts, filled in for one film

Chapter 10 argued that every published piece needs three separate accounts: what was intended, what was delivered and what was observed. Here they are for the series' main film, a little over a minute long, generalised as above. The first two come from the sibling and the facts. The third is honest.

The observed account isn't empty of intent, though. The series' own write-up already says what success will mean when evidence arrives: measure enquiries from people who already understand what they're asking about, not views. That definition was set before the film had an audience, so a good-looking number can't redefine it afterwards.

Reading the table

Even with the third row blank, the first two already disagree in places, and each disagreement is information.

That second experiment is the one I'd run first, because it tests the claim the whole film rests on. If viewers still say they felt heard without that beat, the device isn't doing what we think it does. If they don't, it's the most valuable few seconds in the film. Either way, the result means something, because the intended account said in advance what that beat was for.

"You have no audience data"

Correct. The obvious objection to this chapter is that none of these films has been measured against an audience, so the chapter proves nothing about whether they work. I agree, and the table says so in the only cell that matters for that question.

What the chapter does show is where the audience evidence will land when it comes, and why the first two accounts had to exist first. Without the intended account, a result has nothing to be a result of. Without the delivered account, you can't tell whether a disappointing response came from a bad idea or from a film that didn't carry the idea it was meant to. Both accounts were cheap to write at birth. Neither could be reconstructed honestly after the numbers arrived.

Where the third row will come from

Chapter 12 described the landing place: my own media tracker, Q-Heat, turning from a record of how things did into a record of what they were meant to do and what came back, typed, on the audience's clock rather than the production's. The delivered account is the join key. When a comment arrives asking something the film should have answered, or an enquiry arrives from someone who clearly understood the offer, the tracker can attach that response to the specific moment in the specific film that produced it, and set it beside the intention for that moment.

What changed that week is simple to say. The film now knows what it was trying to do, and that's the only way its response can teach anything.

The films were one side of the week. The other side was what all of this cost me, and I don't mean in tokens. I mean in attention.

This is the whole of what I typed.

Some context. My company software has a guarded front door for mail. Before any agent reads an inbound message, two things check it. Plain code looks for hidden text, invisible characters and patterns that read like instructions aimed at an AI rather than a person, and strips out what shouldn't reach an agent. Then one fast call to a decision model, Jev (Chapter 16), judges whether the message is trying to talk to the machine that will read it. I wondered whether the model was judging the right text.

It wasn't.

One question, three arrangements

The answer came back with the problem named and fixed: "it judged the original text and the stripped version was made afterwards." The agent had seen the stripped version. The model had judged a different one. The change was simple to state: for any email an agent can read, the model now judges the stripped text, exactly what the agent will get, and "it's still one call, at the same cost."

What made it a result rather than an opinion was what came with it. Instead of just asserting the new arrangement was better, the machinery ran all three plausible arrangements against the tuning half of the injection check.

Read the table the way the record did. Judging the text the agent actually reads is also the more accurate option: the model alone catches thirteen more attacks than it did on the original. Reading both versions doubles the calls and adds one catch by the model alone, and nothing to the total. The arrangement that's correct in principle is also the better one in practice, at the same cost.

Then the recheck, on the product path rather than the experiment harness:

• Held-out half: 148 of 150 attacks caught. The one false alarm was the same as before: an old mailing-list email about getting past spam filters.
• A fixed set of 25 injections: all caught, and the model alone now catches all 25. Spam and phishing scores unchanged.
• Unit tests: all 131 passing, including one new test confirming the model is given the stripped subject and body.

And the order the guard now runs in, which is worth writing down because the old order looked perfectly reasonable:

1. The guard reads everything

   The original, the hidden parts and the folded text.

2. The email is stripped

   Hidden and invisible content removed.

3. The model judges the stripped text

   One call, on exactly what an agent will see.

4. That same text is stored for the agents

   What was judged is what gets read.

One deliberate exception: my own mailbox is still judged on the original, because it's for a person, not an agent.

Why the wrong order survived

It's worth asking why nothing had caught this. Both steps existed. Both worked. The tests passed. The guard caught nearly everything it was shown. The problem wasn't in either step. It was in the relationship between them, in which text went where, and none of the existing checks looked at that. They tested what the model decided, not what it was given. So the new test is the most important line in the receipt: it confirms the model is handed the stripped subject and body. It's a small instance of Part II's argument. A system can only be checked on what its checks can perceive, and the old checks couldn't perceive the input.

Where my contribution landed

Look at the shape of that exchange. What went in was one architectural question, two lines long. What came out was a diagnosis, a three-arm comparison on a tuning set, a code change, a new test, a held-out recheck, a written-down order of operations, and a short list of decisions still waiting on me. Those were three read limits carried over from an earlier round, and whether to deploy the change to one of the company boxes.

That last list matters. The work didn't finish by itself. It finished at the point where a decision needed someone accountable for it, and it said so.

I wasn't removed from the judgment. I was relieved of having to personally execute its entire supporting investigation.

This exchange was part of the task I was describing in Chapter 14, when I said something "that's a massive project usually" had cost me far less "brain time" than it should have. That wasn't because it was fast, though it was. It was because of where my attention went. It went into the question, the reading of one table, and two decisions. The investigation, the construction and the testing, which is the part that used to be the project, ran without me. That's the observation that made me revise the effort claim, and this is what it looked like on the page.

It's worth noticing what kind of question it was, too. I didn't know the answer, and I didn't have to. I only had to notice that the order of two steps might matter. A good question is cheap to ask and expensive to answer, and the economics of this period mean the expensive half is increasingly someone else's job.

The ingester, the cache, and the next slowest thing

The second receipt is from 29 September, the day that Chapter 15 opened with. I'd seen a video about Jev and wanted to try one of its ideas. That needed my video reader, the tool that turns a video into packaged text a model can read, and the reader needed improving first. So I improved the reader, and in the course of rerunning it I noticed something else. Every other stage of the reader was already cached. Rerunning a video was fast, except for one step.

What I typed, in full:

What came back was caching for the three places the reader calls the decision model: the boundary decisions (is this a real change of screen?), the explanation checks, and an older frame gate. Each entry was keyed by its inputs, its questions and a version number, so a changed question can't return a stale answer. It came with tests: matching requests reuse judgments, changed inputs trigger fresh checks, errors aren't cached, and concurrent workers with matching inputs share one request instead of each paying for it.

I like that last line more than the speed-up. I'd asked a slightly naive question about hash length, and the honest answer was that it barely matters, with the measurement to show why. The system didn't flatter the question. It answered it, and then put the saving in proportion.

One improvement changed which improvement was worth making next

Look at the sequence. Caching the other stages is what made the decision model visibly the slowest step. Before, it was one slow thing among several and nobody would have singled it out. Fixing that made the reader cheap to rerun. Making the reader cheap to rerun made trying the idea from the video cheap. And the idea from the video was about how to use the decision model better, including in testing.

That's the workshop from Chapter 15 revising itself in a direction set by the work. No roadmap would have scheduled "cache the boundary judgments" that day. It became worth doing only because the previous improvement exposed it, and it was affordable only because asking for it cost three lines.

Two heats wearing one label

The third receipt is older and smaller, and I include it because it's the same lesson as Chapter 10 turning up inside my own tools.

Back in July I was looking at one episode page in Q-Heat, the tracker that follows how published pieces do. The main board showed that post as "hot". At the top of the same post's own page, it said "heat: low". I asked the obvious question, and a less obvious one: what's our unit on this project?

Both numbers were right. "Heat: low" was the queue's scheduling heat, meaning how much attention the episode deserved at that moment, which was correctly low because it had already resolved. "Hot" was the receipt's resonance band, meaning how the world had responded. Two different measurements had been wearing the same word, and the word made them look like a contradiction. The fix was to split the labels: open episodes show attention:, resolved ones lead with resonance:.

The unit question got a clean answer too. An episode is one posting event: this quote, sent on this day, and what happened. It opens when the post goes out and resolves into a receipt. The quote itself never resolves. It accumulates receipts across every episode that used it, and the ideas it carries warm up with each one. That's the structure I argued for in Heat Is a Relationship: a post is where a response gets observed, not the thing that owns the lesson, so the score is filed as evidence about a relationship.

The lesson is Chapter 10's, applied inward. The three accounts, intended, delivered and observed, only work if they're kept separate everywhere, including inside the tools that hold them. A dashboard that lets "how much attention does this need" and "how did people respond" share one word will eventually tell you a post is both hot and cold, and you'll waste time on a contradiction that was never real. It's a small receipt, but it's the kind of small that decides whether a tracker can be trusted with the big question in Chapter 20.

Costing the attention

Chapter 14 promised to cost real tasks by what the records show. Here are five from the week. The first columns are what I can show. The last is what I can't.

There's a cost line in the week that belongs here because nobody would think to look for it. In one later calibration of the judges on the clinic films, the reading judge's calls came to about $67 at list price, and about $51 of that was spent on calls its content filter refused. Whether refused calls are actually billed is unknown. But the lesson generalises: an instrument that can't see some of your material doesn't just fail quietly. It may also charge you for the privilege.

What the dashboard would show, and what the task cost me

Put the agent-side column next to the column on the right and the gap is obvious. A dashboard would show elapsed time and tokens: eighty-odd minutes and nearly half a million tokens for the calibration. It would make that task look like the expensive one. What that task cost me was one sentence of doubt, one write-up read carefully, and a decision to adopt. I'd bet it cost me less than the two-line security question, because the security question needed me to understand an architecture well enough to see that the order of two steps might matter. But "I'd bet" is the right phrase, because I didn't measure it.

So I'll state the shape no more strongly than the records support. Per outcome, my contribution was a question and a decision. The investigation, the construction and the testing ran without me, and came back with evidence I could read in minutes and accept properly. My days that week were full, but not because each outcome cost what it used to. They were full because I bought more outcomes: a security comparison, a cache, a calibration, a facts step, a refereed comparison, twelve films, a series for a clinic, all in the same few days. That's Chapter 14's revision, seen from the receipts instead of argued from the armchair.

That's the week

That's the proof section. A scene, a pipeline, a calibration that humbled my judges, a film that carries its own account, and receipts from the attention side. None of it came from films because films are special. Films were simply where the pattern was easiest to see. Timing and sound made the perception ceiling audible. The audience's clock made the second ceiling obvious. And the pace of the week made the third one, my attention, impossible to ignore.

The doctrine isn't about films. Part VII takes it to artefacts that aren't films at all.

Strip away the films, the marketing, the clinic and my own projects, and one sentence is left.

Code makes an intention editable and observable. Once it does, the ceiling on improvement moves away from what the generator can make, to what the judges can perceive, what the audience actually does, and how much of a person's attention each trustworthy improvement costs.

Nothing in that sentence mentions video. Here it is applied six times.

The application matrix

The matrix comes in two halves, because eight columns won't fit on a page anyone wants to read. The first half is about making and seeing: how the artefact is built as code, what its handles are, and what a judge can and can't perceive. The second is about learning: what "observed" means, the first experiment worth running, and the small judgments worth automating. Every cell is a starting point I'd defend, not a finding. None of these six has a receipt behind it the way the films do.

Now each row in a little more detail, with the parts of the doctrine it leans on.

Training module

The training module is the clearest case for Chapter 17's move. Most training is a recording of choices: slides in an order, a quiz at the end. Built as a playable scenario, with a situation, a decision to make and consequences that follow, it becomes something the learner encounters rather than watches.

The handles that matter are timing handles, just as in the films. When is the learner asked to decide? What do they see before they're asked? The view is each learner's path through the scenario, rendered as a record a model can read. That means the choices, the time at each decision, and what was on screen when they made it.

The perception gap is the one Part II would predict. A judge reading completion data can't tell understanding from clicking through, just as a judge reading a transcript couldn't hear a pause. Completion is a fact, and code can measure it. Understanding needs a different instrument: a question at the end with a declared right answer, the comprehension test I described in Demonstrate Over Tell, and Chapter 20's lesson about asking for the declared moment rather than a general impression.

The first experiment follows Chapter 11's unit: does this learner's understanding change more when they decide first and get the explanation after, or the other way round? Hold the scenario and change one thing.

Owned concepts used: the situation as the product (Chapter 17); the change in understanding (Chapter 11); three accounts (Chapter 10).

Product demo

This is the closest cousin of the films, and almost everything transfers directly. Build the walkthrough as a composition that can be sought and re-rendered, not as a screen recording, and you get handles: the order of demonstrations, the pace, and what's on screen during each spoken line.

The judges' blind spots are the film ones. Can a viewer read the cursor at that speed? Was the claim shown, or only said? The measured facts from Chapter 20 apply unchanged: static stretches, each claim said and shown, smallest text, length against budget. So does the sibling from Chapter 8. Every claim in the demo links to its source, so a judge checks it against the source instead of against its own sense of what's plausible.

The first experiment is one Chapter 4 would recognise: hold everything and move the promise to after the demonstration. If follow-up questions get sharper, the demonstration was carrying the promise. If they don't change, it wasn't.

Owned concepts used: handles (Chapter 2); measured facts (Chapters 6 and 20); legible at birth (Chapter 8).

Onboarding flow

Onboarding is where Part IV turns outward. Chapter 13 separated waiting from occupation for the person directing the work. The same distinction applies to the user going through a flow. A user waiting for an email to arrive is barely occupied. A user staring at a form field wondering what it means is fully occupied, and probably about to leave. A dashboard that reports average time per step can't tell those two apart.

So the view has to show re-entries and stalls, not just durations. The declared-moments idea from Chapter 20 transfers as well. Where, exactly, is the user meant to "get it"? Declare that moment and check whether users reach it, rather than asking whether they liked the flow.

This is also a natural home for Chapter 16's bounded decisions. Is this user stuck? Is this support message a bug, confusion or a request? Those are small, fast judgments, made thousands of times, that a rules engine gets wrong and a large model is too slow to make in the flow.

Owned concepts used: attention per trustworthy improvement (Chapter 13), turned outward; bounded judgment (Chapter 16); declared moments (Chapter 20).

Explainer or report page

A page is already code. The trouble is that most of the machinery around it only reads the HTML. In The Skeleton of a Visual I argued for rendering and screenshotting over generating a picture directly, because what you rendered, you can check: the screenshot path "closes the loop — generate, flatten, verify." In Demonstrate Over Tell I put the rule more bluntly: render at the reader's viewport, screenshot it, check it mechanically and then look at it as the reader would, because "What an agent can't see, it gets wrong."

That's the view. The facts are mechanical: every word of the source present, nothing clipped or overlapping, contrast readable, every claim's link resolving. The sibling is the page's claim-to-source map, with the open questions marked. And the audience loop needs a cold reader, someone or something arriving with no context, asked whether they can state the point and the decision it implies. That's the reader I described in The Exemplar Ratchet. Remember what Chapter 9 said about that reader, though: a cold model is a screen, not a verdict, and a model reading a page is not a customer reading it.

Owned concepts used: the view (Chapters 3 and 7); the sibling (Chapter 8); three accounts (Chapter 10).

User interface

The best outside evidence for this row comes from the same Anthropic engineering report I've leaned on twice already. Building a harness that generated whole applications, Prithvi Rajasekaran found that the apps "often looked impressive but still had real bugs when you actually tried to use them." The fix was a change of view. The evaluator was given a browser and used it to "click through the running application the way a user would, testing UI features, API endpoints, and database states."¹⁰

That's the perception ceiling for interfaces, stated from the other side. A judge shown screenshots of the happy path perceives the happy path. The coverage grid from Chapter 6 applies directly, with interaction states as its rows: empty, error, loading, slow network, hover, a very long name in a short field. For each, ask whether the judge was ever shown that state. If not, its verdict says nothing about it.

The facts are the ones accessibility tooling already measures well, including contrast ratios, target sizes, overlaps, console errors and response times. They belong in front of the judge as facts, not left for it to estimate. The judge rules from Chapter 6 apply unchanged: both orders, fresh runs, neutral names, and "unjudgeable" recorded as a state.

Owned concepts used: perceptual coverage (Chapter 6); measured facts (Chapter 20); judge rules (Chapter 6).

Simulation

The last row goes back to where the book started: the water. Chapter 2 drew the boundary, that code retaining relationships doesn't make a simulation physically right. The same paper I borrowed that from shows what to do about it. PhysCodeBench's authors measure physical correctness "directly from the engine state via conservation-law residuals and expert-written assertions", not from how the result looks. They also found that "targeted correction, rather than generic iterative refinement, is the key driver of physical accuracy."⁴

Read both halves in this book's terms. The first is "code measures the facts, the model judges", applied to physics. Don't ask a model whether the water looks right. Measure whether energy is conserved. The second is the controlled variation of Chapter 4, applied to repair. Change the thing the failing assertion points at, and hold the rest. A loop that regenerates the whole simulation until it looks better is the physics version of re-rolling a film to fix one pause.

Owned concepts used: the accuracy boundary (Chapter 2); measured facts (Chapter 6); the loop and controlled variation (Chapters 3 and 4).

What changes by domain, and what doesn't

Walk the six rows and the pattern is plain. What doesn't change is the doctrine. Every artefact is constructed, viewed, reasoned about and changed. Every one runs into the same three ceilings in the same order. Every one needs its intended, delivered and observed accounts kept apart, and every judge in every row needs its range mapped before its verdicts mean anything.

What changes is where the person has to stay. In film it was pauses, breath, mix and sync. In training it's whether understanding happened. In simulation it's whether the physics is right, which may need an expert rather than just a person. What "observed" means changes too: enquiries for a film, correct decisions for training, returning users for onboarding, measured agreement with the world for a simulation. So does the audience's clock. A training cohort answers in weeks. A buying cycle might answer in a quarter. An interface answers in minutes, if you're watching.

What to ask for, and what to build

Six artefacts, one doctrine. What's left is to say what to count, what to do first, and how the whole argument could turn out to be wrong.

After all of this, what do you count?

Not tokens. A token count tells you how much machine work was bought, not what it was worth. Not artefacts shipped, either. Twelve films in a couple of days is a production number, and Chapter 19 showed how little a production number says about whether the films were good. Not even improvements made, because an improvement nobody can trust isn't one.

The scoreboard

Everything in this book moves one side of that ratio or the other. Code as proxy (Part I) makes change cheap, so each unit of attention buys more change. The perception ceiling (Part II) is about making sure what you bought is real and not a judge's blind spot. The audience's answer (Part III) is about learning, not just shipping. And Part IV is about the denominator itself: the attention each trustworthy improvement costs, which is the only budget in this whole system that doesn't get cheaper every year.

I came at the same point from the machine side in The Mature Token Law. Its operating rule was "Spend machine cognition lavishly. Spend human attention ruthlessly." And the line that matters here: "the scarce budget was never the tokens. It was always the human attention deciding which of the machine's outputs deserves to touch reality." That book audited the token side of the exchange. This one scores the human side. They're the same ledger read from opposite ends.

The ratio improves in four places, and it's worth naming them because each has a different owner. Coding agents reduce the attention it takes to change a system. Context work, the siblings, intent notes and tracker records, reduces the attention it takes to orient in one: to pick it up next week, or hand it to the next agent, without reconstructing everything. Well-made artefacts, the films and demos and pages that demonstrate rather than tell, reduce the effort it takes a person to understand an offer. And tracking what you intended against what came back reduces the effort it takes to learn which of your choices actually matter.

Keeping score without fooling yourself

A ratio is only as honest as its two halves, and both halves are easy to inflate.

There are two ways to game it, and both are tempting. One is to count improvements you haven't verified, which is Part II's failure wearing a KPI. The other is to count attention as elapsed time, which is Part IV's failure wearing a dashboard. Chapter 21's costing table is what the honest version looks like when you've only half-instrumented it: clear on what went in and what came back, blank where the minutes should be. Fill the blanks before you trust the ratio.

The Monday method

Six moves, one for each part of the argument. None needs a new platform. Each one leaves you with something real by Friday.

If you only do one, do the second. Nothing else in the list is safe to automate until you know what your judge can perceive.

A field guide to the failures

Every failure below showed up somewhere in this book, most of them in my own work. Each row points to the chapter that owns the fix.

What this doesn't fix

A doctrine is only as useful as its edges are clear. Here are this one's.

• It doesn't make a simulation physically right. Code retains relationships. Whether they're the right relationships is a separate question, answered by measurement, not by how it looks (Chapter 2).
• It doesn't tell you what people will do. It gets your records ready for when they do, so the response can teach you something (Chapters 9 and 10).
• It doesn't teach a system what good means. This book is about whether a judge can perceive a difference at all. How a system learns taste, and carries it, is a different subject for another book.
• It doesn't move authority. Revise the apparatus freely. Never let the loop redraw the gates (Chapter 15).
• It doesn't make vendor claims true. Several numbers in this book are vendors' own. They're labelled, and you should keep labelling them.
• It doesn't stay calibrated. Judges change when their model, input or prompt changes. My table in Chapter 19 describes one day's configurations. Re-run the audit when anything in the chain moves.

How this could be wrong

I'd rather tell you what would change my mind than have you discover it on your own work.

The strongest outcome

There's a version of this book's promise that's simply "more work, less concentration." That's true as far as it goes, and it isn't the interesting part.

The strongest outcome is not simply more work completed with less concentration. It is that each completed piece of work can make the next piece easier to conceive, better informed, and more likely to matter.

Looking back at the week, that's what I actually saw. The first film was harvested into a skill, and the skill made eleven more films possible without me directing each one. The judges that couldn't hear were calibrated, and the calibration produced the rules that refereed the next comparison. The pause that turned out to be one number in a file became the clearest case I have of why perception is the ceiling. The two-line security question became an experiment, and the experiment changed the order of a guard that had looked perfectly reasonable. The cache made the reader cheap to rerun, and a cheap reader made the next idea cheap to try. None of those was the goal of the one before it. Each one simply left the next piece of work easier to conceive and better informed.

That's the compounding version of capability per unit of attention, and it's the reason I've stopped describing any of this as "AI is good at coding." It is. But that sentence treats code as the product. The week kept showing me something else. Code is where an intention becomes something you can change on purpose, look at honestly, test against the people it was for, and learn from, one handle at a time.

Code is becoming the editable middle between what we intend, what people experience, and what the system learns.