The Chairman, Not the Judge: "Humans Retain Judgment" Is Already Wrong

There's a comforting slogan doing the rounds — AI does the work, humans keep the judgment. At my own desk it's already false: the AI generates the options and the critiques. So what, exactly, is left for me to do?

There's a sentence you'll find under almost every thoughtful take on AI and work, offered as reassurance: humans retain the taste and the judgment; AI does the work; keep a human in the loop. It's meant to calm people down, and it does. It's also, at my own desk, already wrong — and I want to be precise about why, because the mistake it makes is the same mistake that leaves smart people defending the wrong part of their own job.

The slogan pictures judgment as an atomic faculty: a gavel the human bangs at the end, after the machine has laid the evidence on the bench. AI drafts; you decide. AI proposes; you dispose. It's a clean division of labour, and it would be lovely if it described anything I actually do.

What judgment actually looks like at my desk

Here's the honest version. When I'm working on something that matters — even a piece of work purely for myself, with no one to answer to — I don't sit and judge. I might start with a good idea. But then I turn to the AI and I ask: what do you think of this? Can you enhance it? Where's the weakness? Give me a few other ways to approach it. Do some web research on it. Look up my own IP wiki — what have I actually thought about this before, in some conversation eighteen months ago I've half-forgotten? Pull the bit of code we wrote on it last month.

By the time I "decide," I've heard a dozen different opinions — the model reading the open web, reading my own accumulated material, reading the conversation history, reading last month's code. Twelve voices, weighted and summed. And then, yes, I look at whether they've surfaced a better idea, or simply validated the one I walked in with.

Sit with that for a second, because it dismantles the slogan from the inside. If AI generated the options, hunted the weaknesses, gathered the evidence, and red-teamed the thing — then what part of "judgment," exactly, did the human retain? Most of what the slogan waves at as a single human faculty has already been decomposed into distinct processes, and nearly all of those processes have already moved to the machine.

By the time I decide, most of what the slogan calls "judgment" has already happened — in the panel, not in my head.

It's worth laying out plainly, because once you see the decomposition you can't unsee it. Take "judgment" apart into the things you actually do when you make a considered call, and mark who runs each one now:

The top half of that table used to be "judgment." It was expensive, it lived in your head, and having a lot of it cached there was the whole game. It has quietly emptied out into the machine. Which raises the question the slogan never asks and the reader always feels: if all of that is delegable, what's my job?

You're the chairman, not the judge

The answer is a change of title, not a demotion. You're not the judge who scores the case and bangs the gavel. You're the chairman who convenes the panel. And chairing is real work with real moves:

• You decide to convene at all. Some calls deserve a twelve-seat panel; most deserve a shrug and a decision. Knowing which is which is judgment the panel can't make for you — it isn't running yet.
• You choose the questions. "Enhance this" and "where's the weakness" are different meetings that produce different panels. The questions you ask determine what you're allowed to hear.
• You choose the axes. Web, wiki, conversation history, last month's code — each is a different witness. Leave one out and you've quietly narrowed the evidence.
• You weight the voices. Twelve opinions arrive; they don't arrive pre-ranked. Deciding that the historical evidence outweighs the clever new reframing is chairing, not judging.
• You decide when it stops. A panel will deliberate forever if you let it. Calling the vote — declaring "enough, we go" — is the most under-rated chair move there is.

This is, transparently, my own architecture run on myself. The twelve opinions are a scout phase — read-only exploration, no authority to commit anything. I'm the senior who reads the whole transcript and emits the one mutation: the actual decision. It's the same split I've written about for AI agents, where a cheap scout ranges over the ground and freezes a transcript, and a more careful senior inherits it and makes exactly one governed call. The human chairing a panel of AI voices is that pattern with a person in the senior seat.

None of this is about how the panel runs — how you structure the disagreement, force the rebuttal round, make it argue with itself instead of just agreeing in parallel. That's a whole discipline of its own, and I've written it up separately as the AI Think Tank. This piece is about the other side of the table: not how the panel deliberates, but what remains yours once it does.

And here's the thing that should un-panic anyone reading this as a story of loss. This is how good judgment always worked. A decent CEO never judged alone either — they convened advisors, invited dissent, weighed it, and decided. What changed isn't that AI took judgment away from humans. It's that convening got cheap.

AI didn't take judgment from humans. It revealed that judgment was never one thing — and made the convening cheap enough that one person can chair a twelve-seat panel before lunch.

The two things that can't move

So if the whole top half of the table is delegable process, is the human just a convener now — a meeting organiser for robots? No. Underneath the chairing there are exactly two things that can't be handed to the panel, and it's worth being precise about why exactly two, because the argument is the whole point.

The first is the North Star — what "good" even means for this piece of work. A panel is an evaluation function: it scores options against a criterion. But it cannot supply the criterion it is being scored against, any more than a ruler can tell you what length you're trying to hit. What counts as a good outcome here — the taste, the standard, the thing you're actually optimising for — is constitutively yours. The moment you let the panel invent the North Star, you're no longer being advised; you're being told what to want. Delegate that and there's nothing left to chair.

The second is accountability — you're the one who ships it and eats the consequence. This one can't move for a reason more structural than duty: the AI has nothing on the line. No job, no reputation, no shame, no future. It cannot be accountable because accountability means a bad outcome costs you something, and nothing costs the model anything. The panel can carry the analysis; it cannot carry the risk. Whoever bears the downside is, by definition, the decision-maker — and that's still you.

Notice what these two have in common: they're precisely the parts a panel can't generate about itself. The criterion it's judged against, and the body that bears its outcome. Everything between those two poles — the generating, critiquing, gathering, comparing — turned out to be delegable process. The slogan wasn't wrong that something irreducibly human remains. It was wrong about the size and the location. It's not a gavel at the end. It's a star overhead and a name on the line.

The trap you set for yourself

Now the honest flag, because there's a failure mode built into everything I've just described, and I walked right into naming it myself. When I described my own practice above, listen to how it ended: I hear the twelve opinions, and then I see whether they've surfaced a better idea — "or validate the idea I've already got."

Read that last clause again, because it's the whole danger. A panel I convene, whose questions I choose, whose axes I select, will validate almost anything I bring it — if I let it. That isn't judgment distributed across twelve advisors. That's preference laundered through twelve rubber stamps. I get to feel rigorous — look at all these perspectives I consulted! — while doing exactly what I'd already decided to do, now wearing a committee's signature.

This is the same bug in a different body. Each of the twelve opinions is really a nudge — a small weighted signal, none of them commanding, summing into a decision the way a dozen king-safety terms sum into a chess engine's evaluation. That composition is robust — but only if the terms can score in both directions. An evaluation whose features only ever come back positive isn't an evaluation.

An eval function whose features only ever score positive isn't an eval. It's a cheer squad.

The everyday discipline that keeps it honest is small and boring: ask "where is this weakest?" exactly as insistently as you ask "make this better." Convene the critic with the same authority as the booster. If your panel only ever has seats for enthusiasm, you've built a cheer squad and called it governance.

The one guard: a stand-pat that has to win sometimes

The discipline of asking for weaknesses helps, but discipline is a personal virtue and personal virtues fail under pressure — especially the pressure of an idea you're already in love with. What you want instead is something structural, something that doesn't rely on you being good that day.

It's a guard I first learned from a chess engine that had to be forced to consider doing nothing, and it ports exactly to a panel of AI advisors. Somewhere among the twelve voices there has to be a live, first-class path to "don't do this project at all" — a scored option to stand pat, to keep the position you've got and make no move. And it can't be a token seat. It has to actually win sometimes. If the kill-path never wins, it isn't a guard; it's set dressing.

Which gives you a test you can run on yourself this week, and it's brutally simple:

When did your panel last kill an idea you liked? If you can't remember, that's the eval staying suspiciously calm — and you know what comes after that.

If you want to make it concrete rather than rhetorical, keep a chairman's log for one week. It doesn't need to be elaborate — one line per real decision:

• The call — what were you deciding?
• Did you convene? — or did you just do the thing and skip the panel?
• Which axes did you consult? — web, wiki, history, code, a human, or only the model's first reply?
• Was a kill-path live? — was "don't do this at all" actually on the table as a scored option?
• What got killed or changed? — did the panel move you off your opening position, even once?

I won't hand you a number for how that week is "supposed" to go, because the number that matters is the one you'll find in your own log — and the shape is what's diagnostic, not any benchmark I could invent. If the last column is empty all week — nothing killed, nothing meaningfully changed, every panel a warm bath — you haven't been chairing a decision process. You've been running a ratification ceremony. The convening was real; the deliberation was theatre. And the confident "yes" it produced is worth exactly what a yes is worth from something that was never able to say no.

A note on the other side of the table: hiring

One corollary, and then I'll stop, because it belongs to a different article. If judgment and accuracy are genuinely different layers — if a developer needs to be accurate and an architect needs good judgment — then evaluating a judgment role with deterministic, one-right-answer recall questions is a category error at the assessment layer. It tests the delegable layer and skips the layer that's still the human. That's the whole argument of a separate piece on hiring for the AI era — show me your systems, not your memorised trivia. The connection is exact: in both rooms, the mistake is testing or defending the part that decomposed, while the part that's still yours goes unexamined.

So, chair it

The comfort of "humans retain judgment" is that it asks nothing of you: keep being a human, keep your seat at the end of the table, the gavel is safe. The truth is more demanding and more interesting. Most of what you were calling judgment has already moved to the panel. What's left is to chair it well — to convene deliberately, question sharply, weight honestly, and stop decisively — while holding the only two things a panel can never hold for you: the star it's aiming at, and the name that signs for the result. Do that, and put a real kill-path in the room, and let it win when it's right. Do that, and you're not a human clinging to the last job the machine hasn't taken. You're the chairman — which was always the job worth having.