Retrieval Architecture · AI Systems Design

The Nudge Doctrine: Small Signals, One Judge

Demote every signal to a nudge, hand them all to one judge, and the retrieval-tuning nightmare disappears. It sounds like a trick. It’s a doctrine — and I first proved it on a chessboard in 2003, with receipts.

Scott Farrell, LeverageAI·AI systems & knowledge architecture·~13 min read

If you are building hybrid retrieval right now — RAG next to grep next to some structure or graph signal — you have probably hit the wall everyone hits. How do I weight these? What threshold? What chunk size? You start building an eval harness to tune the combination, and the harness quietly becomes the project.

Here is the uncomfortable claim I want to defend: most of that tuning burden is self-inflicted. It exists because you are treating each fuzzy signal as an answer. The moment you stop — the moment every fuzzy signal is demoted to a small, advisory hint handed to exactly one judge — the tuning nightmare mostly evaporates. Not because you tuned it better. Because you gave it a job that doesn’t require tuning.

I call the small advisory signal a nudge, and it is probably my most well-used algorithm over twenty years of building things. This is the doctrine it adds up to.

The move: demote the oracle to a prior

In my wiki’s search, when you ask a question like “how do I start an AI project,” a language model answers it — mostly by looking at the index and the one-line description of every page. Recently I wanted to add a vector search into the mix. I did not put it in the toolbelt the agent uses to navigate; that would hand it a second navigator with its own out-of-distribution instincts, muddying the model that is walking the map. Instead I put it in the back end and demoted it. It runs the same query, and I take the top five hits by count and score and pass them to the model as a hint: “you might want to include these.” Not the chunks. Not the text. Just pointers and a scent.

That single design decision — chunks would give RAG a vote; a bare pointer gives it a whisper — is the whole thing in miniature. A bad hint costs a wasted glance. A wrong-but-confident chunk poisons the context. Hand the model an oracle and it inherits the oracle’s mistakes; hand it a prior and the worst case is nothing.

Deterministic code gives ground truth, nudges give priors, and exactly one LLM gives judgment. Nothing in the chorus gets to decide — they only get to bias.

This is a generalisation of a move I’ve made before: demoting RAG from an architecture to a single retrieval axis — a sensor beside wikilink walk, time, entity, and grep — that returns shaped results and never raw chunks.⁸ The Nudge Doctrine widens the demotion to every fuzzy signal, and it adds the design rules that make a chorus of them stable.

Why a chorus of weak signals is robust

Here is the part that “it’s robust” doesn’t quite reach. A classical chess evaluation function is a linear sum of weak features — a dozen king-safety terms, pawn-structure terms, mobility, each nearly worthless alone, strong only in aggregate.¹ What that shape buys you isn’t just modularity. It’s robustness to being wrong about the weights. If no single term can dominate, then getting any individual weight off by a factor of two doesn’t break the evaluation — it perturbs it. The system degrades gracefully under misspecification. This is the same reason an ensemble of weak, diverse learners is more robust than any single strong one: error in a component perturbs the aggregate rather than commanding it.⁵

That is exactly why you can get away with low tuning. It isn’t that the system magically needs none; it’s that it is built out of nudges, and an ensemble of nudges is forgiving in a way a single authority never is. The RAG-as-oracle crowd tunes obsessively because their one component has to be right. Spread the judgment across many small terms and none of them has to be.

The falsifiable claimRAG’s whole tuning nightmare — chunk size, thresholds, metadata — mostly exists because people use it as the answer. The moment it’s only a hint, it just has to be directionally useful, and the tuning burden mostly evaporates. A prior needs direction, not calibration.

And there is the economic asymmetry that makes the whole architecture safe: a wrong authority is expensive; a wrong prior is free to ignore. That is not a slogan. It’s the reason to demote.

Three rules that keep a nudge a nudge

“As long as it stays a nudge, the exact magnitude doesn’t matter” is what I used to say. It’s true in the middle and false at the tails, and building things taught me the sharper version. Three rules.

1. Frequency — it must fire often, and cheap

In chess you are bound by CPU cycles. If you code a brilliant if statement that fires one board position in a million, the problem is the other 999,999 times: it just slowed you down. In a loop run billions of times, that’s fatal. So the rule was: keep the terms cheap and firing often.

In an LLM system you’re not cycle-bound, so it’s tempting to think the frequency rule retires. It doesn’t. The scarce resource just moved.

My RAG hint passes the chess test cleanly: cheap, fires on every search, small footprint. The trap to watch for isn’t slow code anymore — it’s the seductive rare feature that quietly eats the attention budget on every pass to earn its keep once in a blue moon.

2. Weight-class — advisory, not magnitude-free

Too big and the nudge stops being a nudge. King-safety that can override being a queen up has become a verdict — and now you’re back to a single authority that has to be right. Too small and it never tips even a close comparison, so it’s dead weight that only costs you. The real rule is narrower and more useful than “magnitude doesn’t matter”: a nudge has to stay in the same weight class as its peers — right to within an order of magnitude, not right precisely. That’s a much easier target to hit, which is the whole point.

3. Ripple — it never decides; the process amplifies the tip

A nudge that never decides sounds useless. It isn’t — it’s amplified. In chess a 0.1-pawn delta only matters because the minimax search propagates it up the tree to flip the move choice at the root.² The nudge tips a comparison; the search carries the tip. The RAG hint does the identical thing one layer up: it doesn’t answer, it changes which page the model chooses to walk, and that branch choice ripples into the synthesis.

Bias a branch point, and let the bigger process carry it or damp it. That’s the same mechanism, and the same reason it’s safe.

The word is precise, by the way. A nudge in the behavioural-economics sense alters a choice “without forbidding any options or significantly changing incentives.”⁴ That’s exactly the retrieval move: bias the choice, never remove one, never force one.

The worked example: one session, three retrievers

Recently I watched all of this run end to end in a single session — someone using my stack to reconstruct the history of a chess engine I wrote two decades ago. Every load-bearing find came from the RAG hint, not the grep. The whole foundation — the email threads, an old internet-chess game log, the perft benchmark numbers from a Java rewrite — surfaced from the hint after the first grep returned zero. The single best record in the session, an old email from a chess-programming friend warning me about a specific bug, came out of a hint when a regex against the right archive returned nothing at all.

Why it worked is the good part. The grep pattern was some variation of stand.?pat|forced to capture|have to capture. Those words do not exist in the 2003 corpus. My friend and I never once wrote “stand pat”; the concept lived under entirely different vocabulary. No regex I could write would have found that email, because writing the right regex required already knowing the phrase — which is what I was searching for. The grep couldn’t hit it in principle.

That is RAG’s one structural advantage over a plain full-text index, and it’s decisive in exactly one place: when vocabulary drifts and the searcher can’t guess the words. Here it had drifted twenty years, across two people’s private idiom. So RAG-as-a-hint isn’t a hedge bolted on — it’s aimed straight at grep’s blind spot.

And the grep was decisive precisely where you’d expect. A pattern like CCT[ ]?[0-9] returned 37 clean hits and pinned the exact reference in one shot, because that’s a known exact token. Meanwhile a deliberately sloppy pattern blew out to more than ten thousand results — and the tool didn’t hand me ten thousand useless one-liners. It told me the count, gave me a year histogram, and showed me the top senders were spam. That’s the wc -l reflex encoded: when you search too broadly, the count is the answer, and it’s the fastest way to learn your search was bad.

RAG did the needle. Grep did the exact token. Facets did the “you’re asking wrong.” One more thing I noticed, and I’ll be honest that I’m not sure whether it’s designed or emergent: every time the grep returned zero, five hints appeared; when the pattern blew out to facets, the hints went quiet. That’s exactly how a good nudge should behave — loud when nothing else is speaking, quiet when the deterministic layer already has the answer.

Where the doctrine was born — and the half I’d missed

I learned all of this the first time on a Negamax chess engine I wrote in the mid-2000s. Evaluating a static board — king safety, X-ray attacks, pawn structure — you check maybe a dozen things and score them all pretty small. Twelve nudges either way. There’s some tuning, but what I eventually worked out is that as long as each stays a nudge, the exact magnitude doesn’t much matter. The receipts are real: a quiescence search I was proud of, hash tables, principal-variation search with fail-high detection held at the root, a Java rewrite clocking around 1.8 million moves per second on perft and 100–150k nodes per second in the middlegame, benchmarked against Crafty.⁶ I could watch an opponent engine’s score run hot in the tactics and read that it was cranking its king-safety term, while mine stayed dead steady.

But I credited that steadiness to the wrong thing when I first told the story. It wasn’t the eval. It was the quiescence search. A sum of small positional nudges is only valid on a quiet board. Apply king-safety terms to a position mid-capture and they stop being nudges and become noise, because the material about to swing dominates everything. Quiescence search is what guarantees the board is genuinely static before the nudge-sum is allowed to speak.³ I’d said it myself without flagging it: “a static board that’s known to be static.” The qsearch is the “known.”

The nudge rule was never just “keep the terms small.” It was always “keep them small, and only apply them once the unit is settled.”

And that maps onto a knowledge system exactly. Semantic closure is your quiescence. You don’t run the significance-and-edge nudges on a raw record any more than you’d score king safety with a queen hanging — you close the branch first, so the unit is settled, and only then let the small judgments sum. Don’t evaluate until the thing stops moving.

The same move, all over the stack

Once you see it, the nudge is everywhere. Ranking a skeleton by a graph centrality score is a nudge. Perturbing a file’s importance by how often you talked about it in the surrounding conversations is a nudge — a prior, not a verdict, and it only works if you let the signal perturb the ranking rather than command it.⁹ The RAG hint is a nudge. Even the rule that a claim must carry a pointer is a nudge on the writer. What they share is a strict division of labour: deterministic code decides truth, many weak priors bias, and exactly one model judges.

It composes cleanly with the discipline I’ve written about before — use AI for judgment and deterministic code for ground truth, and be willing to move a component either way when the evidence says so.¹⁰ The nudge layer is the “many priors” that feed the one judge. And it rhymes with a rule I use for every AI sub-tool: return a claim attached to a verbatim exhibit and a pointer, plus a confession of what you couldn’t verify — a witness you can check, not an oracle you must trust.¹¹ A nudge is the retrieval-layer version of that humility.

There’s a lovely detail that closes the loop. My wiki phrases its hint to the model as: “advisory — verify before citing, ignore if irrelevant.” The hint arrives carrying explicit permission to decline it. It’s a right-to-decline built into the signal itself — the subject of a separate piece I’m writing — and it’s exactly why a wrong prior is free to ignore: the prompt says so.

The honest part: under-tuning is a choice, not a free lunch

I have to be straight about the cost, because chess knows it too. Engines got a lot stronger once people stopped hand-nudging and started fitting those weights against real game outcomes — Texel tuning,⁶ then neural evaluations that learn the whole function outright.⁷ So “leave the magnitudes rough” is the right call for robustness and the wrong call for peak strength.

Which one you want depends on whether you’re in a rating war. If you are optimising one fixed distribution to the last point, tune — a fitted weight will beat a rough one every time on that distribution. But a fitted weight is fitted to the distribution you fitted it on. If your corpus is “everything” — a wildly heterogeneous knowledge base you need to be excellent on across the board, on the first pass, with zero per-domain tuning — then deliberately under-tuning the nudges is correct. You’re trading twenty rating points you don’t need for generalisation you can’t live without.

The takeawayBuild the stack so deterministic code owns ground truth, every fuzzy signal is a ranked hint that never answers, and exactly one model judges. Keep each nudge frequent, in the same weight class as its peers, and free to be amplified rather than obeyed. Then leave the weights rough — on purpose — and you get a system that’s unreasonably good on the first pass across a corpus you never tuned for.

That’s the doctrine. Small signals, one judge. It cost me months to learn the first time, on a chessboard, from a friendly email I didn’t recognise as load-bearing. Twenty years later it’s the quiet architecture underneath a knowledge system that needs almost no tuning — and the reason it needs almost no tuning is that nothing in the chorus was ever allowed to decide.

This is the field-note version. The full ebook works through the three rules, the settled-unit rule, and the worked example in detail, with the design checklist for building a tuning-free retrieval stack of your own.