Knowledge Architecture · Agentic Wikis

File Back the Walk

Queries That Leave the Map Better Than They Found It

Scott Farrell

LeverageAI — leverageai.com.au

July 2026

After Reading This Ebook, You Will:

  • Treat a query as two harvestable assets — a filable answer and a mineable path
  • File syntheses as typed derived cache that can never outrank its sources
  • Run a zero-model telemetry pass that finds missing edges, dead ends and cold pages
  • Tell walk telemetry apart from path testing, dispute tickets and tracing — and ship all four in the right order
  • Stand up the whole loop in thirty days, starting with a retention setting

TL;DR

01
Part I · Two Assets, One Query

The Expensive Part Is the Part You Bin

Somewhere in another tab, an assistant just threw away the most expensive thing it produced today.

Right now, in a chat window you probably have open in another tab, an assistant is doing something quietly wasteful on your behalf.

You ask a question. It fires off searches, reads across a dozen sources, assembles a genuinely good research package in its working memory — and then, when it answers, throws almost all of it away. By the next turn the durable state is the conversation text, so those fetched pages have compressed down to two or three citations and a paragraph. Ask a follow-up and it does the whole expensive gather again from scratch.

Scott put it plainly, mid-conversation, to the model that was doing it to him at the time:

You do a web search, it becomes part of your tool calls for that turn — and then you throw it all away. Ask again next turn and the whole research package is gone, because it was too big to keep.

Sit with the accounting for a second. The reading was the costly part of the turn. The reading is exactly the part that gets binned.

One turn, two very different inventories

What the turn produced
  • • A dozen sources opened and read
  • • A judgement about which of them were relevant
  • • Several lines of enquiry started and dropped
  • • Sources checked and found irrelevant
  • • A synthesis across all of it
What survived the turn
  • • Three citations
  • • A paragraph

Everything above the fold is regenerable only by paying for it again.

So why doesn’t the assistant just keep it?

Because it can’t. Not in that architecture. The durable state of a chat application is the conversation text, which means anything that has to survive into the next turn must fit back into the conversation text. A hundred-page research package is precisely the class of thing that doesn’t fit. So it gets compressed into citations, and the citations are what survive.

That’s not sloppiness. It’s a decision about which artefact is durable, made once, at the level of the product, and inherited by every turn thereafter.

Andrej Karpathy noticed the same thing and did something about half of it. In his LLM Wiki gist, the Query operation ends with a line most readers skim straight past: “good answers can be filed back into the wiki as new pages. A comparison you asked for, an analysis, a connection you discovered — these are valuable and shouldn’t disappear into chat history.”1

One sentence, and it changes the accounting. Karpathy’s move flips which artifact is durable: conversations are ephemeral, the map persists, so exploration compounds instead of evaporating.

It took, too. Within weeks the community had turned the paragraph into a directory convention: the most-installed open implementation ships a literal syntheses/ folder, described in its own documentation as “query answers filed back as wiki pages.”2 Which is real progress, and also worth noticing carefully: a folder is a location. It is not a status. We’ll come back to what that costs.

He files the answer. Nobody files the path.

Here is the gap this book exists to fill.

A serious query against a navigable knowledge graph produces two things, not one. The first is the synthesised answer — the compiled claim, the thing you asked for. That’s what Karpathy files. The second is the path: the actual record of where the agent went. Which page it entered from. Which edge it followed. Which region it wandered into and backed out of without using a word of it. Which page it needed and could only find by broad search because no edge pointed there.

Almost nobody keeps the path. And the path turns out to be the more informative of the two.

Key Insight

A query isn’t consumption. It’s production — and a system that discards both of its products is paying to explore and then throwing the exploration away.

The wall this sits behind

None of this is a quirk of chat interfaces. It’s the general condition, and it has a structural cause.

In an ordinary retrieval pipeline, writing and reading are two different machines. Ingestion is the dumb half by design — chunk the document, embed the chunks, store the vectors, move on. Nothing at that stage ever asks what the document means or how it relates to the thousand documents already there. All the thinking is deferred to query time, when someone finally asks a question and the system scrambles to re-derive, on the spot, relationships that were never built.

That wall has a cost most people have felt without diagnosing. Your corpus grows and never gets smarter. Double the documents and you double the noise, not the intelligence. It accumulates. It doesn’t compound.

And the reason it can’t compound is the same reason the chat app bins the research package: the read side has no write path. Everything the system learns while answering you is, architecturally, temporary — not because anyone decided the learning was worthless, but because there was nowhere for it to go.

The wider agent-memory conversation arrived at exactly this point in 2026 from the opposite direction. As one practitioner survey of the field put it: “Retrieval is still the read path. The novelty is the write path.”3 Correct. And the write path has two lanes, not one.

What this book does with that

Four movements, and they can be read in order or raided for the artefact you need.

The answer, typed so it can’t rot the map

A filed synthesis is cache, not source. It gets a type, a rank below its sources, edges to what supported it, an invalidation rule, and first place in the compaction queue. And most walks shouldn’t be filed at all — there’s a rule for that too.

The path, mined by code

Keep every walk. Then run a pass over the pile that spends no model tokens at all and returns three things: edges your map is missing, roads that lead nowhere, and pages nobody ever reaches.

One query, all the way through

A single hard question narrated at the level of pages and edges — the walk, the filed page, the stored path, and the three work items that fall out of it. Then a month of them.

Boundaries, and the build

Where none of this applies, what isn’t being claimed, and the shortest path from “we delete our transcripts” to “our map improves from being used.” It’s about thirty days, and week one is a retention setting.

The claim underneath all four is simple enough to state now and unreasonable enough that the rest of the book has to earn it: your system already produces the most valuable maintenance signal it will ever have, and it deletes it on a timer.

Key Takeaways

  • The research package is the expensive part of a turn, and it is the part every chat architecture discards.
  • Karpathy’s write-back flips which artefact is durable — but it files the answer only.
  • A query produces two assets at different resolutions: the compiled answer and the raw path.
  • Corpora accumulate rather than compound because the read side has no write path.

The obvious objection to all of this is that filing answers back means building a second system, with its own quality bar and its own governance. It doesn’t. The machine already exists — you built it for ingestion — and that’s the next chapter.

02
Part I · Two Assets, One Query

A Query Is a Write in Disguise

The machine that files answers back already exists. You built it for ingestion.

The reason most teams never ship query-time write-back isn’t that they disagree with it. It’s that it looks like a second system — a whole new write path, with its own quality bar, its own governance, its own catalogue of ways to go wrong.

It isn’t. It’s the system you already have, holding one extra tool.

One toolbelt, three North Stars

Watch what actually happens when you build an agentic wiki. You write the ingestion agent first, because that’s the hard one. Then you write the query agent and discover it’s almost the same code. Then you write the janitor and discover it’s almost the same code again.

Scott’s version, from the working session that produced this book:

What I learned is that ingest and query time are almost the same code — and it’s almost the same code as the reviewer/gardener later.

That is not a coincidence to be tidied away. It’s a design principle worth enforcing on purpose. All three roles share the same navigation kit — the map, page reads, on-demand skeletons, grep, source reads, and the search tools. What differs between them is small, precise, and entirely declarative.

Query — read-only, interactive

North Star: answer from the map; descend the ladder only where the question warrants it.

Write access: none. Loop: interactive, one question at a time.

Ingest — writes, batch

North Star: file this source into the worldview; capture what it means and how it connects.

Write access: the write tool. Loop: batch, one source at a time.

Janitor — writes, cron

North Star: compact the graph; merge redundancy, retire cold pages, mend broken edges.

Write access: the write tool. Loop: cron, over the whole graph on a schedule.

Three roles. One machine. Which produces the observation this whole book rests on:

Key Insight

A query that files back is just an ingestion whose source is the system’s own exploration.

The query agent’s finaliser, handed the write tool, is the ingestion finaliser. Same navigation, same handoff, same terminal mutation. The only difference is where the material came from: instead of a document arriving from outside, the source is a walk the system just performed on itself.

Why this is a correctness property, not a convenience

Sharing the toolbelt saves build effort, obviously. But there’s a deeper reason to enforce it, and it’s the part most teams miss.

Think about what filing actually requires. To place a page well, the writer has to answer a question about the future: where would a reader look for this? If the writer navigates the graph through different machinery than the reader will, it is guessing. Give it the reader’s own tools and the question stops being a guess — it gets answered empirically, by actually looking, with the reader’s eyes.

The ingestion agent must experience the wiki through exactly the same eyes as the future query agent, or it will file things where no reader will ever look.

Now notice what query-time write-back inherits for free. The agent doing the filing didn’t have to simulate the reader. It was the reader, thirty seconds ago, working a real question. It knows exactly which entry point failed to lead here, because it tried that entry point. There is no version of the writer/reader symmetry problem more solved than this one.

The same shared machinery collapses your testing problem too — one harness covers all three roles because they walk the same paths. That gets its own treatment later, along with the boundary between grading a walk and mining a pile of them.

Silent competence

The shared toolbelt gets you the mechanism. What makes it matter is a discipline that costs nothing and almost nobody enforces.

Picture a walker descending the ladder on a real question. It reads the page. The page says one thing. The agent, being thorough, descends past the map to the underlying source — and the source says something else. The page has drifted.

Two moves are available. It can quietly work around the discrepancy, trust the source, and give you a correct answer. Or it can give you a correct answer and log a work item.

The first move is what almost every system does, and it deserves a name: silent competence. It feels like good behaviour. The user got the right answer. Nothing broke. And it is the single most expensive habit in the system, for one reason: it is invisible and it recurs. That drifted page will be hit again this afternoon, by a different agent, which will also work around it silently. And again next week. The defect has a permanent tenancy and no ticket.

Pitfall

An agent that answers around a defect is hiding a defect. Correct output is not the same as healthy system, and an evaluation suite that only grades answers will rate silent competence as a pass — forever.

Flip the default and something structural changes. Every cache miss becomes both a measurement and a work item. Query-time failures become the janitor’s queue. Which means querying the system improves the system — the write-back loop that makes the whole thing compound rather than decay.

Be precise about what “escalate” means, because the word suggests something noisier than it is. It is not an alert. It is not a human interrupt. It is a queued candidate carrying evidence pointers back to the walk that produced it, disposed of later, in batch, by something with write authority. Nobody is paged. The queue is the point, and it gets a chapter of its own.

One write, not a conversation of them

There’s a mechanical constraint on how the write actually lands, and it’s worth stating now because it shapes everything downstream.

The mutation must be one terminal structured document, not a sequence of small edits. Scott found this the expensive way:

If you make multiple tool calls — add a source, then an edge, then a claim — it costs a fortune, because it reprocesses the whole conversation every time. One call, one complex JSON document, and Opus handles it easily.

The economics are the visible half. The governance is the useful half. A single structured mutation is a lintable artefact: before anything touches the graph you can validate that referenced page ids exist, that edge types are legal, that the claims being updated are real — and reject the whole thing back for one repair round if not. Applied as a single commit, you also get transactionality and replay. The walk transcript and the mutation document together are a complete account of why the graph changed.

Two shapes for the same write

A conversation of small edits
  • • Each call re-reads the whole transcript as prefill
  • • Cost grows with the square of the number of edits
  • • Partial application on failure; no clean rollback
  • • Nothing to lint — the change never exists as one object
One terminal mutation document
  • • One expensive pass over the gathered context
  • • Validated before it lands; rejected as a whole
  • • Applied as a single commit — transactional and revertible
  • • Pairs with the transcript to form a replayable record

So what actually has to be built?

Four things. That’s the whole list.

  1. A write tool the query finaliser can hold. You already wrote it for ingestion.
  2. A derived type in the schema, so a filed answer can never be mistaken for a source.
  3. A triage rule for when to use it, because most walks shouldn’t be filed at all.
  4. A retention decision for the path — which is the half nobody makes deliberately, and the half the second part of this book is about.

Notice what is not on that list: a service, a pipeline, a new agent, a review board, a scoring model. If your write-back design has any of those in it, something has gone wrong upstream.

Key Takeaways

  • Query, ingest and janitor are one machine with three North Stars; only write access and the loop driver differ.
  • The query finaliser with a write tool is the ingestion finaliser — write-back needs no new architecture.
  • Silent competence — answering around a defect — is the failure mode, because it is invisible and it recurs.
  • The write is one terminal structured mutation: cheaper, lintable before it lands, and replayable afterwards.

Two artefacts come out of every walk, and they do not go to the same place. Which layer takes which — and why putting them in the same one breaks both — is next.

03
Part I · Two Assets, One Query

Two Resolutions of One Event

The answer is a claim. The path is evidence. Put them in the same layer and you’ll either rank your evidence or mutate your claims.

When people first hear “keep the answer and keep the path,” they assume these are two views of the same thing — a summary and its detail, a headline and its working. They aren’t. They are two resolutions of one event, with different epistemic status, and that difference decides where each one lives.

You and Karpathy are keeping different resolution layers of the same event: he files the answer (the compiled claim); keep the path too (the raw exploration).

What each one actually is

The answer is a compiled claim. Synthesised, condensed, self-describing, and — if you file it — wired into the map with edges. Scott makes the value case for that artefact better than I can:

You work out a synthetic proxy for the actual work and keep that as the asset. It’s more valuable than the source because it isn’t as big as the source, it’s self-describing, and it describes its relationship to the rest of the wiki.

All true, and it is exactly why the answer is the tempting one to file. It is small, it is dense, and it comes pre-connected.

The path is the raw exploration. Not condensed, not self-describing, and — this is the part that matters — not a claim about the world at all. It is a claim about what happened: these pages were opened, in this order, by this route; this edge was followed; that branch was entered and left without a word of it reaching the answer.

The truth conditions are completely different. An answer can be wrong. A walk can only be mis-recorded. That single distinction is why they can’t share a home.

Where each one goes

Any serious agentic wiki ends up with layers, because “the wiki” must not become one undifferentiated markdown heap. Six of them, with their epistemic status attached:

Layer Belongs there Epistemic status
Bronze territory Original emails, files, code, records Evidence
Semantic twins Closed bundles: identity, claims, edges, pointers Source-grounded interpretation
Canonical worldview Maintained concepts, policies, supersession Current institutional belief
Derived cognition cache Hard-won answers and comparisons — the filed answer lands here Regenerable; never outranks sources
Experience ledger Walks, decisions, action receipts, outcomes — the path lands here Behavioural evidence
Executable apparatus Walkers, boot profiles, policies, tools, gates Runtime behaviour

The first four describe what the system knows. The last two describe how it learns and acts. Autonomy without that separation becomes un-auditable mush.

Why the two rows can’t be merged

Row four is belief with a rank. Row five is record with a timestamp. Merge them and exactly one of two things happens, both bad.

Either you start ranking evidence — which is a category error, because a walk is not more or less true than another walk; it happened or it didn’t. Or you start mutating records, tidying old walks to reflect what you now know, at which point you have destroyed the one property that made the path worth keeping in the first place.

The same event, two artefacts

The answer
  • Is: a compiled claim
  • Claims: something about the world
  • Read by: humans and agents, deliberately
  • Wrong when: the claim is false, or its supports moved
  • When it changes: it is revised, invalidated, or compacted
The path
  • Is: a behavioural record
  • Claims: only that this traversal occurred
  • Read by: code, in aggregate, never voluntarily by a person
  • Wrong when: it was recorded incompletely
  • When it changes: it doesn’t. It is immutable.

Karpathy already built the right home and never used it

Here is the neat part. The prior art already contains the correct layer for a walk; it just never got a walk put in it.

The LLM Wiki has three layers, and the first is defined in exactly the terms you’d want: “Raw sources — your curated collection of source documents. Articles, papers, images, data files. These are immutable — the LLM reads from them but never modifies them. This is your source of truth.”1

A walk transcript satisfies every clause of that definition. It is a document. It is immutable. The system reads it and never modifies it. The only unusual thing about it is that the system produced it rather than a human curating it in — which changes nothing about how it should be stored.

The asymmetry everything else follows from

One sentence predicts the shape of the rest of this book:

Key Insight

The answer is small and gets read. The path is large and gets counted.

Because the answer gets read, it needs a type, a rank and an invalidation rule — humans and agents will encounter it and act on it, and they need to know what they’re holding. That’s Part II, and it’s the Derived-Page Cache.

Because the path gets counted, it needs immutability and a schema — code will aggregate thousands of them, and nobody will ever read one voluntarily. That’s Part III, and it’s Query-Walk Telemetry.

Couldn’t I just keep a summary of the session?

This is the instinct almost everybody has, and it produces a third artefact that is neither claim nor record — the one thing in this chapter with no honest home in the table.

A session summary is too lossy to be evidence and too unsourced to be belief. It keeps the conclusions and discards the material that earned them, which is precisely the material a walk is worth keeping for. There’s a chapter on this later, because it is the single most common reason a system ends up with no telemetry despite having stored something from every session for two years.

Two neighbours, named once

Two adjacent ideas belong to other books, and the fastest way to keep this one focused is to say where they live.

Session transcripts. Coding-session exhaust has the same shape as query-walk exhaust: intent in your own words and dated, alternatives rejected, things planned and never built — all of it structurally invisible in the finished code. Same asset class, different producer. That book owns session exhaust; this one owns query walks.

Decomposition. None of this works unless the graph is made of pages small enough to be walked between — Herbert Simon’s stable intermediate forms, and the reason an interrupted agent survives at all. That argument has its own book and I won’t re-make it. The seam is clean: decomposition gives you the units; this book covers what traversing them leaves behind.

Key Takeaways

  • The answer and the path are different resolutions of one event, not summary and detail.
  • The answer is belief with a rank; the path is record with a timestamp. Different layers, non-negotiably.
  • The immutable raw layer already exists in the prior art — walks belong in it.
  • The answer is small and gets read; the path is large and gets counted. Everything downstream follows.

Part II takes the answer and makes it safe to file. Part III takes the path and makes it useful without anyone ever reading one.

04
Part II · The Answer: Cache That Cannot Rot

Cache, Not Source

Five properties, all load-bearing. Skip one and the graph starts citing its own guesses back to itself.

Type the derived pages, or the wiki eats its own tail.

Here is the mechanism, before the fix, because the fix only makes sense once you can see the rot happening.

A filed synthesis is a set of claims whose provenance is the system’s own reasoning. File one and nothing bad happens; it’s a decent page with decent citations. File a hundred, unmarked, and something quietly changes: later syntheses start drawing on earlier ones. A walk opens a filed answer, finds it useful, leans on it, and produces a new filed answer supported by it. Do that twice and you have a chain whose root is a guess. Nothing in the chain looks wrong. Every link is locally well-sourced. Confidence climbs; evidence doesn’t move at all.

Key Insight

A synthesis is claims whose provenance is wiki-internal — it’s cache, not source.

The failure state has a name, and it comes from inside the industry rather than from us. A memory vendor, writing about its own product category, describes it exactly: a highly-retrieved memory “is accurate until they change jobs, at which point it becomes confidently wrong.”4 Confidently wrong is what an untyped filed answer becomes. Not immediately. Eventually, invisibly, and with excellent formatting.

The five properties

A filed synthesis carries a derived type, ranks below source-backed claims, keeps edges to the pages that supported it so lint can invalidate it when those supports change, and is the janitor’s first candidate for compaction. That’s the entire safety mechanism in one sentence. Unpacked, it’s five properties, and the useful way to hold them is by what breaks when each is missing.

Property What it means What breaks without it
Typed derived Visible in frontmatter and in every search result that returns it Nobody — human or agent — can tell a guess from a source, so nothing downstream can treat it differently
Ranked below sources On conflict the source wins, automatically, with no adjudication Two claims disagree and the system picks the newest, most confident or best-written one — which is the synthesis
Edged to its supports Every page the walk actually leaned on gets a support edge No invalidation handle exists, and the other four properties become decoration
Invalidated on support change The lint pass can reach it when the ground shifts underneath Stale writes outlive the context that produced them3
First to compact It is the cheapest thing in the graph to throw away The graph grows monotonically, and compaction has to make hard calls about real knowledge instead of easy ones about cache

Support edges are the keystone

If you implement one of the five properly, make it this one — because the other four quietly depend on it.

Without support edges, “invalidated when supports change” is unimplementable: there is nothing to watch. “Ranked below sources” becomes unenforceable in practice, because you can’t verify what a page was derived from. And “regenerate first” has no recipe, because regeneration needs to know what to walk.

Job 1 — Provenance

It answers “why does the system believe this?” with a list of pages rather than a shrug. That’s the difference between a citable synthesis and an assertion in a nice typeface.

Job 2 — Invalidation trigger

When a supporting page is edited, superseded or deleted, the edge is how the lint pass finds everything that leaned on it. Staleness propagates along the edges you drew, or it doesn’t propagate at all.

Job 3 — Regeneration input

When you decide to rebuild rather than repair, the support set is the walk you re-run. A derived page without one has to be re-derived from scratch, which is the same as not having had it.

Three jobs, one edge. It is the cheapest structure in this book and the one most implementations skip, because at the moment of filing it feels like bookkeeping.

You already have the invalidation machinery

None of this requires a new subsystem. A lint pass over an agentic wiki already health-checks for “contradictions between pages, stale claims that newer sources have superseded, orphan pages with no inbound links, important concepts mentioned but lacking their own page, missing cross-references, data gaps that could be filled with a web search.”1

Support edges are simply what lets that existing machinery reach a derived page. You are not building an invalidation system. You are giving the one you already run a handle to grab.

Independent convergence, from the outside

It is worth noticing that the sharpest critic in Karpathy’s own comment thread arrived at the same conclusion from the opposite direction. Reviewing what a serious implementation would need, the recommendation was “a citation model with footnotes, a strict type taxonomy (topics, projects, syntheses, questions), graph topology constraints, and explicit handling for contradictions.”5

Read that taxonomy again: syntheses is on it, as its own type, alongside topics and projects. That’s this argument, reached by someone who wasn’t looking for it. Anyone who runs the pattern seriously ends up reinventing classical knowledge-management discipline. You may as well start there.

If the synthesis is more useful, why does it rank lower?

This is the tension in the chapter and it deserves a straight answer rather than a shuffle.

Scott is right that the compiled proxy is more valuable than the source: smaller, self-describing, wired into the map. The type says it must rank below the source. Both are true, because utility and authority are different axes.

Utility is about how often the thing saves you work. Authority is about who wins when two claims disagree. A cache is more useful than the origin server on every hit — that’s the entire reason it exists — and less authoritative on every miss. Nobody finds that paradoxical in a CDN. It shouldn’t be paradoxical in a knowledge graph either.

A derived page is not knowledge you have acquired. It is work you have already paid for and would rather not pay for twice.

Which is the definition of a cache. And caches are allowed to be thrown away — a fact that makes the compaction rules later in this part feel obvious rather than harsh.

What it looks like on disk

A shape, not a specification to copy verbatim. The fields that matter are the ones that make the five properties real.

---
id: derived.escalation-cost-evidence
type: derived
rank: below-source
created: 2026-07-14
derived_from_walk: walk_2026-07-14_0931
supports:
  - concept.query-improves-system
  - concept.context-arbitrage
  - source.vendor-pricing-note      # bronze pointer
expiry_posture: supports            # stale when any support changes
last_verified: 2026-07-14
---

One field there is doing more work than it looks: derived_from_walk. That is the join to the second half of this book. The answer points at the path that produced it, and the path — as we’ll see — points at everything the answer left out.

Key Takeaways

  • A filed synthesis is cache, not source — its provenance is the system’s own reasoning.
  • Five properties: typed, ranked below sources, edged to supports, invalidated on support change, first to compact.
  • Support edges do three jobs at once — provenance, invalidation, regeneration. Skip them and the rest is decoration.
  • Utility and authority are different axes. A cache is more useful and less authoritative, simultaneously and permanently.

The type makes filing safe. It does not make filing wise. Most walks shouldn’t be filed at all, and the rule that decides which ones is next.

05
Part II · The Answer: Cache That Cannot Rot

Not Every Walk Deserves Filing

File on evidence of difficulty, not on how good the answer felt.

Most people who start filing answers back eventually stop. The reason is almost never that the type was wrong. It’s that they filed all of them.

The critics predicted where that ends. Reviewing the pattern in Karpathy’s comment thread, the objection was that it “collapses past ~1,000 files.”5 A thousand isn’t a mystical threshold; it’s roughly where a graph stops fitting in anyone’s head, and you reach it fast when derived-page count grows linearly with query volume.

The type from the last chapter makes filing safe. Triage is what keeps it sub-linear.

The rule

Not every query deserves filing: the triage signal is whether the scout had to do genuine multi-hop work. If the walker traversed five pages to connect something, the map lacked a shortcut — file it. If it was a one-page lookup, discard.

Now the reframe that makes it operational, and it is the hinge of the chapter:

Key Insight

You are not judging whether the answer was good. You are counting how hard the map made it.

That distinction is worth more than it sounds. Answer quality is a property of the model, and it shifts every time you change models. Map difficulty is a property of the graph, and it is stable across models. Triage on the first and your filing policy needs recalibrating with every release. Triage on the second and it holds.

Five signals, all already in the walk log

None of these require an extra call, a judge model, or a rubric someone has to maintain. They are counts over structure the system already recorded.

The difficulty signals

Hops before the first useful page

How far the natural entry point sat from the material that answered the question. High values mean the map’s front door doesn’t open onto this topic.

Distinct clusters joined

A join across two regions is the signature of a synthesis worth keeping. A long walk within one region is usually just reading.

Reaches past the map to raw source

A comprehension-cache miss. The compiled layer didn’t hold it, so something had to go to the territory.

Backtracks

The walker had to try and reject. Expensive, and direct evidence the route wasn’t obvious from the map.

Did an edge exist for the join?

The strongest single signal. If the anchor pages were already linked, the map did its job and the answer is a rendering, not a discovery.

Read them together and they answer one question: how much of this walk was the graph’s fault?

Pitfall

Never file on user satisfaction. Not thumbs-up, not “the user copied the answer”, not a rating.

Satisfaction measures the answer. Filing should measure the map. A brilliant answer to a one-page lookup is evidence that the map worked — not that it needs another page. This is the single most likely wrong implementation, and it’s dangerous precisely because it feels responsible.

Three calls, with the counts that decided them

✓ File

Six pages across three clusters. Two backtracks. One raw-source reach. No edge between the two anchor pages.

The map lacked a shortcut and the join is genuinely expensive to reconstruct. Somebody will need this again and shouldn’t have to earn it twice.

× Discard

One page. Zero backtracks. The canonical page reached directly from the map.

The map worked. Filing this adds a node that competes for attention with the page that already answers the question — and now there are two places to keep current.

? Borderline — default to discard

Four pages, one cluster, every edge already existed, no raw reach.

The walk was long but the map was fine — the length came from the question’s breadth, not the graph’s gaps. When the triage is ambiguous, don’t add citizenship. The default matters more than the individual call.

Those counts are the shape of a triage decision, not a published threshold. Where your own numbers fall depends on the density of your graph and the breadth of your questions, and anyone who hands you a universal cut-off is selling something.

What if the answer was brilliant but the walk was trivial?

Then you have a well-functioning map and a good model, and you should be pleased rather than acquisitive. Discard it.

The instinct to keep brilliant answers is an instinct about the answer, and it is exactly the instinct the pitfall box warns against. A synthesis that a well-connected map produces in one hop will be produced again, in one hop, at negligible cost, by any future walk. It has no scarcity. Filing it converts a cheap, always-current derivation into a permanent maintenance obligation.

Discard means discard the page — never the path

This is the seam into the second half of the book and it is worth stating flatly.

Filing is conditional. Mining is not. Whatever the triage decides about the answer page, the walk transcript goes to telemetry regardless — every walk, filed or discarded, successful or hopeless. Harvest the paths deterministically even when you don’t file the answer.

The asymmetry is deliberate. An answer page is expensive to keep: map cost, walk cost, janitor cost, permanently. A stored path costs storage and nothing else, and it is the only artefact that can tell you where the map is wrong.

What discarding actually costs

Somebody will re-ask a discarded question. When they do, they’ll get a one-page lookup again — which is cheap, and which is the entire point. The cost of discarding a trivial answer is trivial by construction.

Compare that with the cost of keeping it, which is permanent and compounds: another line in the map every agent reads, another corridor every walker may burn attention on, another page the janitor lints forever. The asymmetry is so lopsided that the default should feel almost harsh, and it should still be discard.

One more use for the same counts

The triage signals are a maintenance signal wearing a different hat.

A question class that keeps producing high-difficulty walks isn’t just producing filable answers. It’s telling you the map is structurally thin in that region — that the same shortcut is missing every time, and filing a page each time is treating the symptom. That observation belongs in the queue, and it gets picked up when we get to a month’s worth of walks rather than one.

Key Takeaways

  • Filing on difficulty keeps derived pages sub-linear in query volume; filing on enthusiasm produces exactly the collapse the critics predicted.
  • Triage counts structure — hops, clusters, raw reaches, backtracks, edge presence. No judge model required.
  • Never file on satisfaction. It measures the answer; filing should measure the map.
  • When the triage is ambiguous, discard. The default carries more weight than any individual call.
  • Discard the answer page, never the path. Filing is conditional; mining is unconditional.

The type makes filing safe; the rubric makes it sparing. Both can be in place and the thing can still rot — which is what the next chapter is about.

06
Part II · The Answer: Cache That Cannot Rot

How a Wiki Eats Its Own Tail

Three ways a filed answer poisons the map — and the review that keeps promotion honest.

Karpathy’s gist attracted the objection within days, and it deserves to be read before anything is said in reply.

Commenters argued that the pattern “collapses past ~1,000 files”, that “LLM-only maintenance leads to compounding hallucinations”, and that “calling a folder of AI-written Markdown a ‘wiki’ is a category error since no humans collaborate on it.”5 These are comment-thread positions reported by a secondary source, not Karpathy’s own view — but they are the objections anyone shipping this will meet, and one of them is correct as stated.

Take them in order. The thousand-file point was answered in the last chapter with triage: the number is what you get when derived pages grow linearly with query volume, and they shouldn’t. The category-error point is a definitional argument that this book doesn’t need to win — call it a graph, call it a compiled layer, the mechanism is unaffected.

The middle one is the real risk. It earns the chapter.

Failure mode 1 — Self-citation drift

The mechanism. A derived page appears in the support set of another derived page. Once, that’s arguably fine. Twice, and you have a chain whose root is a guess. Nothing in the chain is flagged, because each link looks locally well-sourced — it cites pages, and those pages cite pages.

The diagnosis. Entirely deterministic and almost free. Walk the support edges of every derived page; if any support is itself typed derived, flag it. That check runs in milliseconds and can sit on the write path.

The policy. Two options. Forbid derived-on-derived outright, or permit it with a depth cap and a visibly rendered chain. Start by forbidding it, for a reason that has nothing to do with purity: the failure is silent, and the restriction costs almost nothing. If a walk genuinely needs a filed synthesis as input, it can walk that synthesis’s own supports instead and cite those — which is one extra hop and produces a page with real provenance.

Pitfall

A derived page supported by a derived page is a claim with no floor. The check is three lines of graph traversal. Run it on write, not on a schedule — by the time a nightly job finds the second link, a third has been written.

Failure mode 2 — The zombie lens

This one has a worked example in the canon, and it’s better than any hypothetical:

A one-off partnership review produces a sharp competitive narrative. Someone files it into the main wiki without type or expiry. Six months later agents treat it as current institutional belief. The partnership died; the narrative did not.

The mechanism. Task-local interpretation is entirely legitimate for its task and entirely illegitimate the moment the task ends. The type says what a page is. Only an expiry posture says how long it’s for.

The fix. Every derived page carries an expiry posture, and there are only three honest kinds:

  • Tied to supports — stale when any supporting page changes. The default, and the right one for most syntheses.
  • Tied to a date — a claim that was true as at a point in time and should be re-verified rather than trusted.
  • Tied to a task world — dies when the deal room closes, the engagement ends, the review completes.

Note what isn’t on the list: a TTL in days, chosen by someone who had no basis for choosing it. A number pretending to precision is worse than a declared basis, because it looks like a decision and isn’t one.

The cultural half. There is an incentive problem underneath the technical one, and it’s worth naming: teams rewarded for “everything in the knowledge base” will over-persist; teams rewarded only for speed will under-persist and re-pay comprehension forever. Both failure modes look like diligence from the inside. The answer is roles, stated in the operating model: integrators own fabric quality; task owners own temporary rooms; reviewers own promotion. Without those, the boundary is a slogan and the zombie wins by default.

Failure mode 3 — Hallucinated consolidation

This one is caused by a rule from Chapter 4, which is why it belongs here rather than in someone else’s book.

The mechanism. Derived pages are compacted first. A janitor compacting under a loose North Star can merge two genuinely distinct ideas because they happened to be old and adjacent — and derived pages, being both regenerable and low-ranked, are exactly the material a compaction pass reaches for. It is a real failure mode, not a hypothetical, and it is the question any honest practitioner asks first.

The mitigations are ownership properties rather than clever algorithms: chronological legibility so the merge is visible as a change in time; a human-auditable diff; instant revert; and a periodic lint pass before you trust the map unattended.

Self-maintaining does not mean unsupervised. It means the supervision is cheap and legible.

You are not watching every consolidation. You are reading a diff when something looks off, and running lint before the map goes unattended. That is a genuinely different activity from reviewing every write, and pretending the janitor is risk-free would be the dishonest move — it’s also the move that gets someone burned.

Promotion is a deliberate act

Everything so far has been about keeping derived pages in their place. The other half is what happens when one of them earns its way out.

When a task world produces something durable — a missing edge repeatedly walked, a hard comparison worth keeping, a lens implication that survived review — promotion to the canonical layer is a deliberate act, not an automatic one triggered by usage.

The trigger is not popularity. Two conditions, both required:

Promotion criteria

1. Recurrence

The same derived conclusion has been reached or reused across multiple independent walks, from different intents. One walk liking it is not evidence of anything.

2. Survival

It has been through at least one support change without needing to be rewritten. A page that survives its supports moving is no longer merely cache — it has earned a claim of its own.

The reviewer’s question is not “is this true?” — the walk already established that. It is: is this stable, and does it belong to the institution rather than to the task that produced it?

Survival is the criterion people skip, and it is the one doing the work. Anything can look canonical on the day it’s written. A synthesis that still reads correctly after the pages beneath it have moved is describing a relationship rather than a moment.

And when it doesn’t survive

You throw it away and re-derive it, without ceremony. Derived pages are nuke-and-regenerate material by definition — regenerable from the map, kept only while they’re earning their storage.

That posture is what keeps the compaction rule from feeling brutal. You’re not deleting knowledge. You’re evicting a cache entry whose recipe you still have.

Myth vs Reality

✗ Myth
  • • Filing answers back rots the wiki.
  • • A folder called syntheses/ is enough separation.
  • • If the janitor is good, supervision is optional.
  • • Popular derived pages should become canon.
✓ Reality
  • Untyped filing rots the wiki. The type is the mechanism, not a label.
  • • A folder is a location. Rank, supports and invalidation are status.
  • • Self-maintaining means the supervision is cheap and legible, not absent.
  • • Promotion needs recurrence and survival of a support change.

Where this book concedes ground

The same criticism that raised compounding hallucination also observes that the pattern “works less well for situations requiring high factual precision or audit trails”, and recommends that for regulated material “a hybrid approach — keeping the raw sources alongside a well-structured wiki — is safer than relying on the compiled layer alone.”5

That is correct, and this book agrees with it without qualification. It is precisely why the layer table in Chapter 3 exists and why bronze never goes away. In audit-bound domains the ranking rule stops being hygiene and becomes a control — the full treatment is in the boundaries chapter.

Key Takeaways

  • The critics are right about the untyped version. Conceding that is what makes the typed version credible.
  • Self-citation drift is silent and deterministically detectable — check it on write.
  • Type says what a page is; expiry posture says how long it’s for. A page with one and not the other is honestly labelled and still wrong.
  • Promotion requires recurrence and survival of a support change. Popularity is not evidence.
  • Self-maintaining does not mean unsupervised — it means the supervision is cheap and legible.

That is the answer half of the book: safe to file, sparing about what gets filed, honest about what goes wrong. Everything so far has been about the artefact everyone reaches for first. The rest of the book is about the one nobody keeps.

07
Part III · The Path: Telemetry From Traffic

Keep the Path, Not the Prose

Cheap to store, impossible to reconstruct — the exact profile of a thing you should never delete.

Harvest the paths deterministically even when you don’t file the answer.

Every system that keeps anything about a session keeps a summary. It is the obvious move, it feels like good hygiene, and it is the reason most organisations running agentic systems have no telemetry despite having stored something from every session for two years.

The problem is what a summary is for. A summary preserves conclusions. The value in a walk is almost entirely in the material that isn’t a conclusion.

Do not trim the history. Compile the state.

The sharpest version of this argument comes from the context-management side of the canon, where the same instinct causes the same damage: “Summarise the oldest messages” keeps the prose and destroys the trace. Remember what was in that field: rejected branches, preserved contradictions, known absences, the specific reason a promising path was abandoned. A summary keeps the conclusions and throws away the reasoning that earned them.

Spelled out, the walk transcript holds four things a summary deletes:

The senior inherits the entire epistemic trail: not just what the junior found, but what it checked and found irrelevant, where it hesitated, which edges it followed and abandoned.

What it checked and found irrelevant. Where it hesitated. Which edges it followed. Which it abandoned. Every one of those turns out to be a telemetry signal in the next chapter, and none of them survives compression into prose.

What a walk record has to contain

For the deterministic pass to have anything to count, the record needs eight things. Most agent frameworks capture three of them by default.

The walk record

Intent

The parent question, held verbatim. Everything else is only interpretable against it.

Ordered page opens

Page id, timestamp, and how it was reached: from the map, via an edge, or via broad search. The reach mode is what distinguishes a graph walk from a list of page views.

Edges followed

Source, target, and edge type. Without the type you can count movement but not characterise it.

Branches entered and abandoned

The branch root, and the critical flag: reached_answer. Did any content from this branch make it into the synthesis? Without that boolean you are counting reading, not rejection.

Reaches past the map

Skeleton regeneration, grep, source read — each with its target and its rung on the ladder.

Searches issued

The query strings, and whether the result was used. A broad search where an edge should exist is one of the most informative events in the whole record.

The stopping decision

Where it stopped and, if the agent stated one, why. Premature stops and exhaustion look identical without it.

The answer’s support set

The pages the synthesis actually leaned on — a strict subset of the pages opened. If the answer gets filed, this becomes its support edges.

Every rung leaves a different mark

The record only makes sense against the shape of a walk, which is a descent through resolutions rather than a flat sequence of reads.

The read-side ladder, and what each rung records

L0 — the map

One screen, pure information scent. Records the entry hypothesis.

L1 — pages

Claims and edges, the deliberately blurry layer. Records opens, edges, branches, abandonment.

L2 — skeleton

Structure without bodies, regenerated on demand. Records a materialisation.

L3 — grep

Pinpoint verification. Records a reach past the map.

L4 — source

Full-resolution ground truth. Records the deepest reach, and the strongest coverage signal.

Cost rises as you descend; the agent’s judgement is deciding how far down each question deserves to go. “Reached past the map” only means something if you know where the map ended.

Why keeping it is affordable now

Two facts, and they compound.

First, append-only transcripts are the cache-optimal shape. Every turn appends; nothing mutates the prefix; the model re-reads a prefix it has already seen. That is exactly the access pattern prefix caching was built for — which quietly converts the “write every tool result into the conversation” design from an extravagance into the economical structure.

Second, cache reads are documented at roughly a tenth of standard input pricing, so re-reading an unchanged prefix costs a fraction of processing it fresh.6

The number that changed the design

~10×

cheaper to re-read a cached prefix than to process it fresh, per vendor documentation

0

model tokens spent by the pass that mines the finished transcript — it is deterministic code

Scott’s own production observation sits alongside that, and should be read as exactly what it is — an operator reporting his own system rather than a benchmark: with automatic prefix caching, an agentic turn through a cheap cached model settles at a roughly flat cost of around three-hundredths of a cent, regardless of how long the transcript has grown. The economics of walk-keeping stopped being a debate somewhere around then.

Pitfall

Cached is not free of attention cost. As the same source notes: cached prompt prefixes still occupy the context window — caching changes what you pay for those tokens, not whether they count. Keeping the path is cheap; keeping it resident is not. The finished transcript goes to cold storage; only the active fragment stays in the window.

The retention argument, in one asymmetry

A walk transcript is text. Text is the cheapest thing in the building.

Set against that: it is impossible to reconstruct after the fact. The branch the agent didn’t take leaves no trace anywhere else in the system. Not in the answer, not in the graph, not in the logs. If you didn’t record it, the information is simply gone, and no amount of re-running the query later will recover what the map looked like on the day.

Key Insight

Cheap to store and impossible to reconstruct is the exact profile of a thing you never delete.

“But we already log queries”

Almost certainly you do, and almost certainly it isn’t this.

Most systems log operations — latency, tokens, status, maybe the query string. That’s an ops artefact, and it cannot answer any of the three questions the next chapter asks, because it records that a call happened rather than where the agent went. Even the well-designed community implementations are explicit about this: their log.md is “an append-only record of every operation.”2 Operations, not routes.

Ops log vs walk record

An ops log knows
  • • That a query happened, and when
  • • How long it took and what it cost
  • • Which tools were called, how many times
  • • Whether it errored
A walk record knows
  • • Which two pages were opened in the same walk
  • • Which edge was followed, and of what type
  • • Which branch was entered and contributed nothing
  • • Which page was needed and reached only by search

The test: if your existing telemetry can’t tell you which two pages were opened in the same walk, it isn’t this.

And once stored, the record is immutable and unranked — row five of the layer table, behavioural evidence. A walk is not more or less true than another walk. It happened.

Key Takeaways

  • Do not trim the history; compile the state. A summary keeps conclusions and destroys the trace.
  • Eight fields make a walk record mineable — and edge type, abandonment-with-flag, and reach rung are the three most systems miss.
  • Append-only transcripts are the cache-optimal shape, and cache reads cost about a tenth of fresh input.
  • Cached still occupies attention. Store cold; keep resident only what the next step needs.
  • An ops log records operations. A walk record records routes. Only one of them can be mined.

You now have a pile of transcripts nobody will ever read. Good — nobody should. The next chapter reads them with code.

08
Part III · The Path: Telemetry From Traffic

Three Signals, Zero Models

The most useful maintenance pass in the system spends no model tokens at all.

Everything else in an agentic wiki is priced in tokens. Ingestion is tokens. Query is tokens. The janitor’s consolidation pass is tokens. This one is priced in nothing, because it isn’t inference. It’s counting.

Here is the whole doctrine in one sentence, and then the specification:

Page-pairs repeatedly traversed together are missing-edge candidates; regions the scout entered and backed out of are dead ends worth recording so future walks don’t repeat them; pages never visited across many walks are cold-page candidates for the janitor.

Signal 1 — Co-traversal → missing-edge candidate

Definition: two pages opened within the same walk, across many walks, with no edge between them in either direction.

The ticket carries: the pair, the count, the distinct walks and distinct intents that produced it — and the proposed edge type left blank. The pass proposes the link, never the meaning.

Threshold: a shape, not a number. Enough distinct walks that coincidence is implausible, and enough distinct intents that it isn’t one question asked repeatedly.

Signal 2 — Abandonment → dead-end candidate

Definition: a branch entered and left where no content from it reached the answer. That second clause is the whole definition — without it you are counting reading, not rejection.

Repetition required: one walk abandoning a branch is a walk having a bad day. Several walks, from different intents, abandoning the same branch is a property of the map.

The ticket carries: the branch root, the intents that entered it, and where the record supports it, the shape of the disappointment.

Signal 3 — Non-visitation → cold-page candidate

Definition: pages never reached across a large sample of walks. This is behavioural coldness, which is strictly stronger than structural orphan-detection.

A lint pass finds “orphan pages with no inbound links” by topology.1 Telemetry finds them by traffic. A page can have six inbound edges and never be walked.

The output is an ordering, not a verdict: rename → relink → demote → compact.

Pitfall — hub co-traversal

Two pages can co-occur constantly because both are hubs everything passes through. An uncorrected count will confidently rank your index page against every page you own. Weight against each page’s overall visit frequency, the way any co-occurrence analysis does. If your top-twenty list is all hubs, this is why — and it’s a bug in the pass, not a verdict on the signal.

The fourth signal, deliberately kept outside

There is a fourth thing the walk logs will happily tell you, and it belongs adjacent to the queue rather than inside it.

Every reach past the map to raw source is a comprehension-cache miss. A tool call is a cache miss, and the wiki is a comprehension cache. Early in an ingestion run the map is thin and the misses are constant; as it fills in, the same questions start being answered from the map and the tool calls fall away — not because the model got smarter, but because the map got more complete. Plot the rate over comparable work and you have a learning curve you can literally chart, from ordinary operation, with no evaluation set at all.

So why isn’t it a fourth ticket type? Because it measures rather than locates. Coverage tells you how complete the map is. The other three tell you what specifically to change. Mix a measurement into a work queue and you generate tickets nobody can action — “coverage is 71%” is not a task. Keep them adjacent and separate.

(And note the honesty that comes with the metric: the curve’s shape is the claim. Nobody should be publishing axis values for it, including us.)

Why the pass must be model-free

Two independent arguments, and they arrive at the same place.

It’s a prior, not an authority

The mining pass emits candidates that bias the janitor’s attention. It does not decide anything. That distinction is load-bearing, and the reason is an asymmetry of cost: a wrong authority is expensive; a wrong prior is free to ignore. An authority’s error propagates into the graph and has to be caught and unwound. A prior’s error costs a glance.

Put a model in the pass and you have promoted a prior to an authority — and worse, one with a plausible narrative attached. A wrong number you can dismiss. A wrong paragraph argues back.

Under-count is auditable; hallucinated rank is not

The same split runs through the influence work, where the temptation is identical — ask a frontier model how important something is and get a confident paragraph back. The discipline there is exact and it transfers wholesale:

Models invent rankings. Graphs under-count when edges are missing. Under-count is auditable: you can see which objects have no typed link yet. Hallucinated rank is not. Prefer the under-count. Prefer the missing-edge ticket over the confident score.

A pass that misses a missing edge has cost you nothing you had. A pass that invents one has cost you an edge, and the walker will use it.

Where the model does belong

Later, and narrowly. The pass proposes a link. A model proposes the type and the reason — is this supports, contradicts, supersedes, applies? — and a human or a governed senior disposes. Semantic typing first, deterministic consequences second, disposal by something accountable. That ordering is what keeps the whole apparatus auditable.

The whole pass

The “zero models” claim is only credible when you can see how short it is. It’s three loops and a set difference.

# mine_walks.py — runs over stored transcripts, no API calls

walks = load_walks(window_days=30)

# 1 — co-traversal → missing edges
pair_counts, page_freq = Counter(), Counter()
for w in walks:
    pages = set(w.pages_opened)
    page_freq.update(pages)
    pair_counts.update(combinations(sorted(pages), 2))

missing = [
    (a, b, n, lift(n, page_freq[a], page_freq[b], len(walks)))
    for (a, b), n in pair_counts.items()
    if not graph.has_edge(a, b) and n >= MIN_WALKS
       and distinct_intents(walks, a, b) >= MIN_INTENTS
]

# 2 — abandonment → dead ends
dead = Counter(
    b.root for w in walks for b in w.branches
    if not b.reached_answer
)
dead_ends = [(root, n) for root, n in dead.items() if n >= MIN_REPEATS]

# 3 — non-visitation → cold pages
visited = {p for w in walks for p in w.pages_opened}
cold    = graph.all_pages() - visited

emit_candidates(missing, dead_ends, cold)   # queue, not mutations

Three counters, a lift correction, and a set difference. No inference anywhere. It runs on a laptop against a month of transcripts, and it is the highest-leverage code in the system.

What comes out

A queue of candidates, each carrying four things: the signal type, evidence pointers to the walks that produced it, a count, and a proposed action class. No mutations. No scores.

Candidate vs mutation

✓ What the pass emits
  • • “These two pages met 9 times across 6 intents; no edge.”
  • • Pointers to the nine walks
  • • An action class: propose edge
  • • Nothing written to the graph
✗ What it must never emit
  • • A typed edge, written directly
  • • A priority score with hidden weights
  • • A natural-language rationale it inferred
  • • A candidate with no evidence pointer

Discard any candidate with no evidence pointer on sight. The pointer is what makes it auditable, and auditability is the only advantage this has over a model’s opinion.

That’s a cheap deterministic pass over stored scout transcripts feeding the janitor’s work queue — the map improves from being used, not just from being fed.

Key Takeaways

  • Three signals: co-traversal → missing edge; abandonment → dead end; non-visitation → cold page.
  • Correct co-traversal for hub frequency or the pass will confidently rank your index page.
  • Comprehension-cache misses measure coverage. Keep them adjacent to the queue, not inside it.
  • The pass emits candidates with evidence pointers — never mutations, never scores.
  • Put a model in the mining pass and you replace an auditable under-count with a plausible narrative.

Three signals, specified. Two of them deserve a chapter each — starting with the one that has independent evidence of the pain, filed by developers who have never heard of any of this.

09
Part III · The Path: Telemetry From Traffic

The Dead End Is the Asset

Developers are hand-writing dead-end logs because nothing derives them. Something should.

The best evidence that this signal matters isn’t ours. It’s a feature request on a public repository, filed by a developer with no interest in knowledge graphs who arrived at the same conclusion through sheer irritation.

Backtrack patterns are invisible — When we try approach X, hit a wall, and pivot to Y, that hard-won knowledge isn’t captured anywhere. Future sessions (or future developers) may repeat the same dead-end exploration.
— anthropics/claude-code issue #15222, “[FEATURE] Decision History Tracking with DECISIONS.md”7

The same issue names the second half of the loss: “Decision rationale disappears — Git tracks what changed, but not why this approach instead of that one. When Claude presents options A, B, and C, and we choose B, the reasoning for rejecting A and C evaporates.” And the cost, in the requester’s own words: “I might waste hours re-exploring OAuth only to hit the same wall.”

The proposal is a hand-written DECISIONS.md with status states — ACTIVE, REJECTED, BACKTRACKED, EXPLORING.

Right instinct, backwards implementation

A hand-authored log is a discipline tax levied at the exact moment of maximum frustration and minimum patience. You have just wasted three hours on an approach that didn’t work. Now write it up carefully, in a file, with a status state, for the benefit of a future self who may not exist.

It will be written for the first three dead ends and never again. Every team that has ever tried an architecture decision record knows this shape.

Meanwhile the system already knows. It entered the branch. It read the pages. It used none of them. That fact is sitting in the walk record, correctly typed, waiting for somebody to count it. Nobody was asking.

Two ways to keep a dead-end log

Hand-written
  • Who writes it: a person, deliberately
  • When: immediately after failing at something
  • What it costs: attention, at the worst possible moment
  • Under pressure: the first thing dropped
  • Coverage: whatever somebody remembered
Derived
  • Who writes it: a counting pass over stored walks
  • When: on a schedule, retrospectively
  • What it costs: nothing anyone notices
  • Under pressure: unaffected — nobody is doing anything
  • Coverage: every walk, including the ones nobody would have written up
Dead ends are informative, and summaries delete them.

Eight words, and they justify the entire retention argument from the last chapter. This is where keeping the raw path instead of a summary of it pays.

Why rejection generalises better than conclusion

Here is the claim worth arguing for, because it’s the counter-intuitive one.

An answer is specific to its question. A missing edge is specific to a pair of pages. But a ruled-out region is reusable by every future walk whose intent points that way, regardless of what the question actually was. Three different agents chasing three unrelated questions can all be spared the same wasted descent by one record.

Key Insight

Rejection generalises better than conclusion — which makes the negative space the most reusable thing a walk produces, and the first thing every summary destroys.

What a dead-end record contains

Five fields

The branch root

The page or region entered. Not the individual pages read inside it — the entry point, because that’s what a future walker will be standing at.

The intents that entered it

Plural, always. One intent is a walk having a bad day. Several unrelated intents is a property of the map.

The shape of the disappointment

Where the record supports inferring it: empty region (nothing there), stale page (contradicted by a source read afterwards), wrong grain (right topic, useless resolution). Three shapes, three different fixes.

The date

The field everyone skips, and the one the next section is about.

The walks

Evidence pointers. A dead-end record without them is a rumour.

What happens when someone finally fills the empty region?

This is the non-obvious half of the chapter and the paragraph to read twice.

A dead-end record is a claim about the map at a date. The region was empty in March. In April somebody files three pages into it. The record is now actively harmful — it steers walkers away from exactly the material they need, and it does so invisibly, which is strictly worse than a stale page a walker can read and doubt.

So a dead end is a derived claim and gets the same treatment as any other derived claim: support edges to the region it describes, invalidation when that region changes, first in line for compaction. The moment any page under the branch root is created or materially edited, the dead-end record is flagged.

Pitfall

A permanent dead end is a new kind of stale — and the most confident liar in the system. Everything else in the graph can be read and questioned. A dead-end record works by preventing reading. If it isn’t dated and invalidated, it will eventually be wrong in a way nobody can see.

How a walker should use it

Not as a wall. As a prior.

A wrong authority is expensive; a wrong prior is free to ignore. The dead-end record enters the walker’s context as a sentence, not a rule: three previous walks entered this region and used nothing from it, as at March. A walker with a good reason still goes. And when it goes and finds something, that visit is the invalidation event — the record updates itself out of existence by being disproved, which is the only self-correcting mechanism in this chapter.

The organisational version

The same instinct at institutional scale is deprecated-but-visible knowledge: the policy that was superseded but still binds old contracts, the approach tried and abandoned with the reason attached. Scott’s framing of what a real knowledge substrate has to carry:

It can hold competing facts. It can hold deprecated facts. It can hold solution paths — all the things agentic AI needs.

A prompt can only assert. A graph can carry status. Deprecated-but-visible alone is worth the architecture, because it’s what stops every new agent — and every new hire — re-litigating settled failures.

The honest limit

This is the easiest of the three signals to implement badly, precisely because a single abandoned branch is so easy to detect and so nearly meaningless.

A dead end recorded from one weak walk is a lie about the territory dressed as evidence. The floor is repetition across distinct intents, and the record should carry the count so a reader can discount it. If your dead-end log is longer than your missing-edge queue, your repetition floor is too low.

What it gives back

Two things, and neither of them has a number attached because nobody has measured one.

The first is direct: future walks stop re-earning rulings that were already earned. The second is more interesting. A cluster of dead ends around one region isn’t a navigation problem at all — it’s a coverage problem. Nobody found anything there because there is nothing there. That routes to ingestion, not to the janitor, and it is one of the few signals in the whole system that tells you what to go and read next.

Key Takeaways

  • Developers already feel this pain and reach for hand-written logs — which fail in exactly the conditions that created them.
  • Rejection generalises better than conclusion; the ruled-out region helps every future walk pointing that way.
  • A dead-end record needs a date and an invalidation rule, or it becomes the most confident liar in the system.
  • Use it as a prior, not a wall. A walker that goes anyway and finds something is the invalidation event.
  • Clustered dead ends are a coverage ticket for ingestion, not a navigation ticket for the janitor.

Abandonment tells you where the map is misleading. Non-visitation tells you where it is merely absent from everyone’s attention — a subtler problem, because the honest first response is almost never deletion.

10
Part III · The Path: Telemetry From Traffic

Cold Pages and the Cost of Citizenship

A page is never free. But cold is a naming problem before it is a deletion problem.

Borges tells it in a single paragraph. In that Empire, the Art of Cartography attained such perfection that the Cartographers Guild “struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it.” The following generations, less fond of cartography, “saw that that vast Map was Useless” — and abandoned it to the deserts.8

The engineer’s version of the same worry, mid-build: once you put every file in your file system into the wiki, the wiki is ginormous. A page per file is the Borges map — coextensive with the territory, and therefore useless. The graph must hold the grain of meaning, not the grain of storage.

Why a page is never free

The distinction that matters isn’t which database you chose. It’s graph citizenship — whether a node participates in navigation at all. A citizen appears in the map and in edge lists, and it therefore imposes three costs that a stored row never pays.

Map cost

Every page widens the overview that every agent reads first — and the map has to stay one screen of useful. This is the cost paid on every single walk, whether the page is relevant or not.

Walk cost

Every edge is a corridor an agent may walk. Dead corridors burn attention on every traversal that considers them — and the walker doesn’t know it’s dead until it has been down there.

Janitor cost

Every page is maintenance surface: lint, dedupe, contradiction checks, forever. The cost doesn’t stop when the page stops being useful.

That’s why cold pages matter at all. They aren’t merely useless — they are taxing. And the tax is levied on every walk anyone ever makes. (This is developed properly in BI for Soft Data, which mints the citizenship distinction and works through the bronze/silver/gold tiering underneath it.)

The ordering rule

Now the part most implementations get wrong, and the reason is that the fix is boring.

The doctrine on non-visitation is precise about causes, and the order it lists them in is not accidental: pages never reached suggest poor naming, missing inbound links, or dead weight. Naming first. Dead weight last.

What to do with a cold page, in order

1. Rename

The page is fine; nobody can tell what it is from its line in the map. Cheapest fix, trivially reversible, and it resolves a surprising share of cold pages.

2. Relink

The page is fine and well-named; no edge puts it in anyone’s path. Add the edge from where a reader would actually be standing.

3. Demote

The meaning doesn’t recur. It can live outside navigation and be materialised on demand — a stored row plus a pointer stub on its old slug.

4. Compact

Merge into a neighbour, leaving a redirect. Last, because it’s the only step that loses a distinction.

Each step is cheaper to reverse than the next. Run them in order and you never destroy something you merely failed to signpost.

Structural orphan is not behavioural cold

A lint pass finds pages with no inbound links. Telemetry finds pages nobody walks. The overlap is partial, and the difference is the interesting part.

Key Insight

A page with six inbound edges that nobody ever traverses is invisible to lint and glaring in telemetry — and it almost always has a naming problem, not a linking problem.

Which is exactly why the ordering above starts where it does. The structural instrument can only ever recommend linking, because links are all it can see. The behavioural instrument can tell the difference between “unreachable” and “reachable and never chosen” — and those need opposite fixes.

Promote by use, not by prediction

The same signal runs in the other direction, and this is where it earns its keep.

The wrong answer is to predict at ingestion which material “seems important”. Prediction is exactly the hand-authored structural judgement that fossilised every knowledge graveyard anyone has inherited. The right answer is to let traffic decide — and every query walk is already a stored transcript, which means the evidence is already there.

Two decisions, from real telemetry

↑ A promotion

A stored summary was materialised in 9 of the last 30 query walks — three different agents kept pulling it up mid-walk.

A file whose summary keeps getting materialised during queries is asking to become a page: claims want to point at it. Promote — extract claims, add edges, retain the pointer.

↓ A demotion

A page with zero walks in six months and no inbound edges exercised.

It dissolves to a stored row plus a pointer stub on its old slug. Nothing is lost; it just stops costing attention.

These are the shape of real promotion and demotion decisions — the counts illustrate the reasoning, not a benchmark.

Cache promotion policy, applied to knowledge: the graph stays exactly as large as its traffic justifies — no larger, forever.

The counter-signal, kept honest

Telemetry rewards the roads already travelled. Left alone, that’s calcification with good instrumentation — the minority position quietly compacted into the majority because nobody walked it this quarter.

Three structural guards, and they are structural on purpose — motivational posters don’t survive a cron job:

  • Keep known absences as first-class entries. “Nobody goes there because there’s nothing there” should be recorded, not inferred from silence.
  • Keep contradictions as edges rather than averaging them into bland prose, so a cold minority position can’t be tidied away by a compaction pass.
  • Keep an advisory similarity sensor running as a smoke detector for material the graph does not route toward. It exists precisely to find what the edges can’t.

And the doctrinal framing: the cold-page output is a nomination. A wrong prior is free to ignore; a wrong deletion is not.

Myth vs Reality

✗ Myth
  • • A cold page is dead weight.
  • • Telemetry says delete it.
  • • If it mattered, someone would have walked it.
  • • Cleaning up means removing pages.
✓ Reality
  • • A cold page is usually badly named or badly linked.
  • • Telemetry nominates. The janitor disposes, in order.
  • • Traffic measures scent, not worth.
  • • Every step but the last is reversible — and deletion isn’t on the ladder at all.

Say that last one plainly: demotion keeps a pointer stub, compaction keeps a redirect, and nothing in this ladder is a delete. Deletion should be the only operation in the whole stack that requires a human decision every single time.

The metric this finally makes computable

Conventional knowledge-management dashboards measure footfall: queries run, monthly actives, documents uploaded this month. Those are write-path and door-count metrics. They tell you the library is open. They do not tell you the long tail is circulating.

The read-path metric is the resurrection rate.

Resurrection rate

  • Numerator: eligible documents returned to circulation during the window after a dormancy period — surfaced by the read path and actually used.
  • Denominator: eligible documents that were dormant at window start.
  • Dormancy threshold: zero human opens for a prior period of N years. Start with N = 3 unless your industry cycles faster.
  • Window: typically a quarter. State it on the dashboard.
  • Always show the raw count beside the rate — rates alone hide whether the corpus is tiny.

Companions worth having:

  • First-read count — documents never read by anyone but their author until the read path surfaced them.
  • Cross-department reads — consumption outside the author’s own unit.
  • Double-click events — how often a human follows a citation into the source. Trust being exercised.

Notice the inversion. Traditional dashboards measure what people put in, to prove adoption. This family measures what the system pulls back out, to prove the archive was worth keeping.

And the connection to this chapter is mechanical: cold-page telemetry is the instrument that makes resurrection rate computable at all. You cannot count what came back into circulation unless you were recording what was circulating.

Key Takeaways

  • A page imposes map, walk and janitor costs that a stored row never pays. Citizenship is earned, not granted by existing.
  • Rename → relink → demote → compact. Naming first, dead weight last, deletion not on the ladder.
  • Structural orphans and behavioural cold pages need opposite fixes; only telemetry can tell them apart.
  • Promote by use, demote by neglect — the graph stays exactly as large as its traffic justifies.
  • Resurrection rate is the read-path metric, and cold-page telemetry is what makes it measurable.

Three signals, specified. Before running them, it’s worth knowing what they are not — because all three neighbours are built from the same artefact, and it is entirely possible to construct one of them by accident.

11
Part III · The Path: Telemetry From Traffic

What Telemetry Is Not

Three neighbours read the same artefact for three different questions. Conflate them and you build a dashboard instead of a map.

Every instrument in this chapter is built from the walk record. That’s the problem. A team that sets out to build one of them can end up with a different one entirely and not notice until it stops producing anything actionable.

One artefact, four questions.

Instrument Question it answers Unit of analysis Output
Path testing Was this walk sound? One walk A verdict on the agent
Dispute tickets Where were readers failed? One labelled failure A prioritised repair
Observability / provenance What happened in this execution? One trace An audit record
Walk telemetry What should the map become? Many walks, in aggregate A graph mutation

Neighbour one — path testing

Picture two agents handed the same hard question, both returning the right answer. The first read the canonical page, followed the right edge, checked the exception, and arrived. The second read three loosely-related pages, none of them canonical, and guessed — and its guess happened to match what the model already believed from pre-training. Grading the output, you cannot tell them apart.

Grade the path and the two combine into four outcomes where output-grading could only ever see two.

Path Answer Meaning What to fix
Good Right Real success Keep it; promote as a regression golden path
Good Wrong Reasoning defect Synthesis policy or evidence ranking — not the map
Bad Right Unsupported lucky answer Tool contracts, boot order, edges, canonical reach — the design
Bad Wrong Navigation defect Same design surface, plus genuinely missing content

The third row is the dangerous one: right answer, wrong journey — a guess from model priors that never touched the canonical source. It passes an output test and fails the day the priors don’t hold. Output-only suites celebrate it.

The distinction. Path testing grades one walk’s epistemic soundness, and its verdict is about the agent. Telemetry aggregates many walks, and its verdict is about the map. A perfect path over a bad map still leaves a missing-edge ticket. A bad path over a perfect map still fails the 2×2. Neither instrument can produce the other’s finding.

Neighbour two — dispute tickets

When a user says the answer is wrong, you get something walk telemetry structurally cannot produce: a label, and a forced freeze of the evidence at the moment of failure.

The sibling work draws the boundary itself, and it’s worth quoting doing so:

Passive telemetry says where walkers went. Dispute tickets say where walkers (and readers) were failed.

What a dispute adds: a label, and evidence frozen at the point of complaint. What it lacks: volume, and any coverage at all of the silent majority who were mildly failed and clicked nothing. The two instruments have precisely inverse biases, which is why they belong in the same queue and not in the same signal.

And the failure mode of shipping only one half — quoted against ourselves, because it’s the sharpest thing in the chapter:

“If your organisation only ever ships the parent half — mine successful walks, ignore disputes — you will improve connectivity and still miss the pages users actively reject.”

This book is the parent. That extension is real, it is somebody else’s chapter, and you should ship both.

Neighbour three — observability and execution provenance

The prior art here is mature and should be named as such rather than talked past.

OpenTelemetry’s GenAI semantic conventions standardise spans for model calls, tool calls and agent steps.9 Agent evaluation platforms capture full trajectories — tool selection, arguments, loops, retrieval paths, latency and cost — and the recommended engineering loop is observe production, identify failure, add the trace to a versioned dataset, change something, replay, compare, deploy.10

The academic side has converged on the same premise from a different angle. A 2026 survey of evidence tracing puts it exactly: “Final-answer accuracy alone cannot explain how an output was produced, which evidence supported each claim, whether tool calls were justified, how memory influenced later decisions, or where failures originated” — and defines execution provenance as “the typed graph of an agent execution.”11

The distinction. Provenance reads a trace to audit one execution. Trace tooling reads trajectories to validate a change you already decided on. Telemetry aggregates traces to discover what the map should become — and its output is a graph mutation, not a chart and not a verdict.

The practical consequence is good news: if you already run agent tracing, you probably don’t need new capture infrastructure. You need three walk-specific fields most traces don’t carry — edge type on each traversal, branch abandonment with the reached-answer flag, and the ladder rung of each reach — and a consumer that emits candidates instead of dashboards.

The metric that is not the metric

One warning belongs in every conversation about walk instrumentation: call count is not a scoreboard.

A better agent might initially make more calls, because it recognises deeper investigation is available. A more mature agent with a better map may make fewer broad searches, because it navigates through typed edges instead of guessing search strings. The same number moves in both directions for good reasons, which means it measures nothing.

What to measure on a walk

✓ Do
  • • Unique relevant pages reached
  • • Useful graph distance travelled
  • • Movement through typed edges vs repeated broad search
  • • Redundant or repeated reads
  • • Cross-cluster joins
  • • Backtracking after false starts
  • • Source-anchor rate
  • • Contradictions preserved
✗ Don’t
  • • Raw call count
  • • Wall-clock latency as a quality proxy
  • • Tokens consumed per answer
  • • Any composite score with hidden weights
  • • Anything whose improvement is indistinguishable from the system doing less work

The target is maximum useful territory activated per unit of attention, with enough source anchoring to remain trustworthy. Traffic is not the thing.

Build them in this order

All four instruments should exist eventually. The order matters more than most teams expect.

  1. Capture — the walk record. Nothing else is possible without it, and it is a retention setting rather than a project.
  2. Telemetry — cheapest, fully automatic, no human in the loop, produces candidates from day thirty.
  3. Disputes — needs a UI, a triage owner and someone to close the loop with the complainant.
  4. Path testing — needs a labelled set and sustained judgement. The most valuable and the most expensive.

Build them in reverse and you end up with an evaluation programme, a lot of graded transcripts, and a map that hasn’t changed.

Key Takeaways

  • Four instruments read the same walk record for four different questions; only one of them outputs a graph mutation.
  • Path testing judges the agent on one walk. Telemetry judges the map across many.
  • Disputes and telemetry have inverse biases — label quality versus coverage. Ship both.
  • Trace tooling is prior art to build on. The extension is discovery-oriented use, and the output is a mutation not a chart.
  • Call count is not a metric. Useful territory per unit of attention is.

Enough distinctions. The next chapter runs one hard query all the way through and produces every artefact in this part from a single walk.

12
Part IV · The Loop, Running

One Hard Query, End to End

A single walk, narrated at the level of pages and edges — and everything it leaves behind.

Everything so far has been mechanism. This chapter invents nothing; it runs what’s already been specified, once, on one question, and shows every artefact that falls out.

The setup

The corpus: a compiled wiki over a body of published writing — frameworks, concepts, source chapters. Roughly a thousand pages, typed edges, a map that fits on one screen.

The question:

“When we tell a client their agent should stop silently working around a broken page, what evidence do we have that the alternative is affordable?”

Why it’s hard: the governance half and the economics half live in different regions of the graph, and there is no edge between them. A one-page lookup cannot produce this answer.

The walk

Eight steps, marked by ladder rung

1 · L0 — map

Reads the overview. Forms two entry hypotheses, one per region. Records both.

2 · L1 — governance region

Opens the escalation-versus-silence page from the map. Follows a typed edge to the maintenance-queue page. Good scent. Nothing about cost.

3 · L1 — the attempted bridge

Looks for an edge from maintenance toward anything about pricing or budget. There isn’t one. Falls back to broad search. First tell in the record: a broad search where an edge should exist is the signature of a structural gap.

4 · L1 — economics region

Broad search lands on the caching-and-pricing page. Correct destination, wrong route — reached by search, not by edge. The record notes both facts separately.

5 · Branch entered and abandoned

An adjacent region — vendor comparison — looked promising from its map line and turned out to be a stub. Entered, read, nothing used. reached_answer: false.

6 · L4 — source

The pricing claim matters enough to verify, so the agent descends past the map to the underlying documentation. Recorded as a comprehension-cache miss with its rung.

7 · The escalation moment

The maintenance-queue page asserts something the source contradicts. A claim has drifted. Two moves available — see below.

8 · Stop

Six pages, two clusters, one backtrack, one raw-source reach, one synthesis.

The beat this whole book has been building to

The page says one thing. The source says another. The agent can answer correctly and move on — or it can answer correctly and log a work item.

The wrong move is silently answering around it. The right move is escalating: log a janitor work item — the page contradicts its source, here are the two, here is the walk that found it. Nobody is paged. The queue absorbs it. And the defect now has a ticket instead of a permanent tenancy.

What the walk produced — 1. The triage call

Run the rubric against the actual counts, and show the working:

Signal This walk Reads as
Hops before first useful page2Entry point was reasonable
Distinct clusters joined2A genuine cross-region synthesis
Reaches past the map1The map didn’t hold the figure
Backtracks1The route wasn’t obvious
Edge existed for the joinNoThe map lacked a shortcut

Verdict: file. The join is expensive to reconstruct and the map didn’t help.

2. The derived page

---
id: derived.escalation-affordability
type: derived
rank: below-source
created: 2026-07-26
derived_from_walk: walk_2026-07-26_1412
supports:
  - concept.query-improves-system        # governance side
  - concept.context-arbitrage            # economics side
  - source.vendor-caching-doc            # bronze pointer
expiry_posture: supports
last_verified: 2026-07-26
---

## Claim
Escalating a graph defect instead of answering around it is affordable
because the escalation is a queued candidate, not an interrupt, and the
walk that produced it was already paid for.

## Note
The cost objection assumes escalation means human attention. It doesn't.

One detail is worth stopping on. The support set is not every page the walk opened. It is the pages the synthesis actually leaned on — three of the six. The other three are in the walk record, which is where they belong. Confusing “what I read” with “what I relied on” is how derived pages acquire fake provenance, and it makes invalidation fire on changes that don’t matter.

3. The stored path

walk_2026-07-26_1412
intent: "evidence that escalation instead of silent workaround is affordable"

opens:
  - {page: concept.query-improves-system, via: map,    rung: L1}
  - {page: concept.janitor-queue,         via: edge:supports, rung: L1}
  - {page: concept.context-arbitrage,     via: search, rung: L1}   # ← no edge existed
  - {page: concept.vendor-comparison,     via: edge:related, rung: L1}

branches:
  - {root: concept.vendor-comparison, reached_answer: false}       # ← abandoned

reaches:
  - {target: source.vendor-caching-doc, rung: L4}                  # ← cache miss

searches:
  - {q: "cost of maintenance escalation agent", used: true}

stop: answered
support_set: [query-improves-system, context-arbitrage, vendor-caching-doc]

4. Three tickets

Missing edge

concept.query-improves-systemconcept.context-arbitrage

Co-traversed here and, per the pass, in four other walks this month across five distinct intents. No edge in either direction.

Proposed: link. Type left blank for a model or a human to assign.

Dead-end candidate

concept.vendor-comparison — entered and unused here and in two other walks from different intents.

Note honestly: three is thin. The record carries the count so the janitor can discount it, and the shape reads as wrong grain rather than empty region.

Contradiction

concept.janitor-queue asserts a claim contradicted by source.vendor-caching-doc.

Different feeder, same queue. This one came from the escalation, not from the deterministic pass — which is the first hint that the queue has more than one input.

The next walk

Some weeks later, the same question — or one adjacent to it — after the edge exists and the dead-end record is in place.

Before and after

Before
  • • Six pages opened
  • • One broad search (step 3’s fallback)
  • • One backtrack (the stub region)
  • • One raw-source reach
After
  • • Four pages opened
  • • No broad search — the fallback is now a typed edge
  • • No backtrack — the branch carries a dead-end prior
  • • One raw-source reach (unchanged — the figure still lives in the source, by design)

The causal claim in full: one edge removed one search; one dead-end record removed one branch. That is the entire mechanism, and it is checkable by reading the two walk records side by side. No percentage is claimed and none is needed.

Note what did not improve. The raw-source reach is still there, because the numbers-out rule puts figures in the source deliberately — a stale relationship is still directionally useful; a stale number is just wrong. Telemetry didn’t fix that because there was nothing to fix.

That is the use loop, run once: intent, walk, synthesis and path, derived cache and graph repairs, a better next walk. Nothing in it required a new system, a new agent, or a decision anybody had to be woken up for.

Key Takeaways

  • A broad search where an edge should exist is the clearest structural-gap signal a walk can produce.
  • The escalation moment is the whole difference between an agent that answers correctly and a system that improves.
  • The support set is a strict subset of the pages opened. Everything else belongs in the walk record.
  • One walk produced a filed page, a stored path and three tickets — from two feeders, not one.
  • State the causal mechanism for the improvement, not a percentage.

One walk is an anecdote. The value is in what thirty days of ordinary use produces without anyone writing a single ticket — which is next.

13
Part IV · The Loop, Running

A Month of Walks

One walk is an anecdote. A month of walks is a map of where your map is wrong.

The previous chapter was legible because it was one walk. But one walk can’t tell you anything — a single abandoned branch is noise, a single co-traversal is a coincidence, and a page unvisited once is a page that happened not to come up.

The value is in the aggregate, and the aggregate has a property worth stating plainly: thirty days of ordinary use produces a ranked, evidenced list of defects without anyone being asked to write a single ticket.

What a month produces

By class, with shapes rather than invented counts — because the actual numbers depend entirely on your corpus, your traffic and your thresholds, and anyone showing you a screenshot with figures on it is showing you fiction.

Class Shape Disposal
High-confidence missing edges A small number. Co-traversed across many distinct intents, hub-corrected, no edge either way. Type the edge, add it. Cheap and high value.
Weak missing-edge candidates A long tail. Should sit below the fold and mostly never be actioned. Leave. Let recurrence promote them.
Dead-end candidates A handful with enough repetition across distinct intents to be credible. Record with a date and an invalidation rule.
Cold pages A set. Most of them are naming problems, not deletion candidates. Rename → relink → demote → compact, in order.
Contradictions Logged by escalation during walks, not by the pass. Route to the page owner. Usually the highest-priority row on the board.
Coverage clusters Several dead ends around one region. Nobody found anything because there’s nothing there. Routes to ingestion, not the janitor. Go and read something.

Three feeders, one queue

The queue in that table has more than one source, and the sources have genuinely different biases.

Passive telemetry — unlabelled successful walks

High volume, complete coverage of what actually happened, no label at all. It knows where everybody went. It has no idea whether they were satisfied.

Dispute tickets — labelled failures with frozen evidence

Low volume, high label quality, biased toward users willing to click a button. Cluster them by hub, edge, page and failure class and you get demand-side failure telemetry: navigation clusters, content clusters, coverage clusters, and scattered synthesis outliers.

The janitor’s own lint — supply-side

Contradictions between pages, stale claims superseded by newer sources, orphan pages, concepts mentioned but lacking a page, missing cross-references, data gaps.1 Structural, thorough, and completely blind to whether anybody cared.

The line that makes the combination worth building: passive telemetry says where walkers went; dispute tickets say where walkers were failed. Lint says what the map looks like from the inside. Three views, and none of them is dispensable.

They disagree usefully

The interesting rows in the queue are where two feeders point at the same region for different reasons — or at the same region with opposite conclusions.

A convergence

A cluster of disputes around one hub, and a cluster of dead ends around the same hub. That isn’t two tickets. It’s one coverage hole with two witnesses — readers are being sent somewhere that has nothing in it, and both the walkers and the humans have noticed. Ranks near the top on the strength of the agreement alone.

A genuine disagreement

Lint says a page is an orphan — no inbound links. Telemetry says it is heavily walked. Both are correct: the page is being reached by broad search, not by edge. The fix is an edge, and the disagreement is what revealed it. Acting on either instrument alone would have produced the wrong action — deletion, or nothing.

Ranking without an oracle

The temptation at this point is overwhelming: build a composite priority_score from the signals, weight it, sort by it.

Don’t. A prior needs direction, not calibration — and the moment you fit weights, you have created something that must be right, which means it needs tuning, which means it needs an owner, which means you have rebuilt the fortune cookie with an API.

Order the queue by a small number of legible facts instead:

  • how many distinct intents produced the signal;
  • how many distinct feeders point at it;
  • how recently, and whether the rate is rising;
  • how expensive the workaround was — a broad search is cheap, a raw-source reach is not.

Four columns a human can read and argue with. Many priors, exactly one judge.

Pitfall

A composite priority score turns four auditable facts into one unauditable number, and it will be wrong in a direction nobody can see. If somebody asks “why is this ticket above that one?”, the answer should be a sentence, not a formula.

Significance has a clock

A candidate that keeps recurring is repricing itself. One that stops recurring may have been fixed by something else, or may simply have gone out of season.

That temporal behaviour has its own treatment in the canon — an observation can be noise at first and load-bearing later, which is why an arrival-ordered queue is a polite lie about attention and a market is the better model. For our purposes the implication is narrow: don’t age candidates out on a timer. Let recurrence promote them and let silence demote them, and keep the whole frontier visible rather than draining it FIFO.

Disposal

The queue is candidates. Something has to decide, and something has to write.

The decision is a judgement pass over a batch, with the evidence pointers attached — not a chat, a review. Batching matters: twenty candidates read together produce better calls than twenty read one at a time, because the pattern across them is often the actual finding.

The write is one terminal structured mutation. Same discipline as Chapter 2, same reason:

If you make multiple tool calls — add a source, then an edge, then a claim — it costs a fortune, because it reprocesses the whole conversation every time. One call, one complex JSON document, and Opus handles it easily.

Which also makes it lintable before it lands — referenced page ids exist, edge types are legal, claims being updated are real — and rejectable as a whole for one repair round. And it makes the whole thing replayable: the walk transcripts plus the mutation document are, together, a complete account of why the graph changed. When a consolidation looks wrong six months later, you can read exactly what the system knew and what it decided.

Self-maintaining does not mean unsupervised

Self-maintaining does not mean unsupervised. It means the supervision is cheap and legible. You are not watching every consolidation; you are reviewing a diff when something looks off, and running a lint pass before you trust the map unattended.

Which sets the size of the human ritual. Not a governance programme. Fifteen minutes.

The weekly review — three items, in order

  1. The top five missing-edge candidates. Do you agree those are edges? If you disagree with four of five, your thresholds are wrong and that’s the actual finding.
  2. The newest dead-end records. Is any of them describing a region you just filled? This is the invalidation check, and it is the one that stops the log becoming a liar.
  3. Promotion nominations. Recurrence plus survival of a support change. A human decides; there is no automatic path from derived to canonical.

The anti-pattern: a monthly knowledge-management steering committee. That is how this becomes theatre, and theatre is more expensive than doing nothing because it consumes the credibility you’ll need later.

What the diff buys you commercially

There’s a question every buyer of an AI system asks and never gets a straight answer to: will it learn how we do things?

Under this architecture it becomes answerable with an artefact rather than a promise. The learned state is a browsable object, so “what did it learn?” isn’t a philosophical question about inscrutable weights — it’s a change log. New pages. Merged claims. A superseded policy. Three edges that this month’s traffic said were missing, with the walks that said so attached.

You read the week’s learning the way you’d read a diligent new hire’s notes, correct it where it’s wrong, and watch it compound.

Key Takeaways

  • A month of ordinary use produces a ranked, evidenced defect list without anyone writing a ticket.
  • Three feeders with inverse biases — passive walks, labelled disputes, structural lint. Where they disagree is where the finding is.
  • Rank by legible facts: distinct intents, distinct feeders, recency, workaround cost. Never a composite score.
  • Dispose in batch, write once, lint before it lands, keep the transcripts so the change is replayable.
  • Fifteen minutes a week, three items. Not a steering committee.

The loop runs. Part V points it at three corpora nobody would call a knowledge base — and then at the walker itself.

14
Part V · The Same Move, Elsewhere

The Inbox, the Dev Folder, the Deal Room

Three corpora nobody calls a knowledge base, all producing the same exhaust — and one version with no agents in it at all.

None of this requires a “knowledge base”. It requires a map, a walker, and somebody asking questions repeatedly. Those three conditions are met in a lot of places that have never been described as knowledge management, and in at least one that predates the phrase entirely.

The inbox

Start with the one that sounds least like a graph problem.

Once I gave it a wiki of a few years of emails it became a genius at triage — so much so that it now hardly ever tells me about any of my email, which is roughly the right angle.

Notice the success criterion, because most people have it backwards. Deciding “this isn’t worth interrupting you” is a high-judgement call requiring a deep model of your priorities. The noisy agent was the ignorant one. The quiet agent is the learned one, and silence is the expensive output.

Now ask what the walks say, which is a different question from what the triage decisions say.

Every triage decision is a walk: the agent opens the sender’s page, the project page, maybe a thread page, and forms a view. Aggregate a few thousand of those and the co-traversal signal lights up in a very personal way. Two entity pages opened together, daily, with no edge between them, is a relationship the map hasn’t recorded — a supplier who is also a neighbour, a project that is really two projects, a person who moved companies and is still the same person.

The cold-page signal here is unusually informative, and it’s worth dwelling on. An entity page that no triage decision ever reaches is not dead weight in the way an unused technical page is dead weight. It’s a relationship that has ended.

Key Insight

In a personal graph, a cold page is usually a fact about a life rather than a fault in a map — and the honest action is a date, not a delete.

The dev folder

A hundred project folders, mostly code, some markdown, some templates. The North Star isn’t find this function — it’s where did I have this thought. Project pages, concept pages, technology pages, and janitor-maintained capability hubs that roll up projects, dates and published work.

The ingestion design for that — the census, the skeletoniser, the per-project dossier, chronological ordering by first commit — is a different chapter in a different book. This one starts once the map exists and queries begin.

What the walks say about a dev wiki

Co-traversal

Two projects opened together across many unrelated intents. They share an idea the map never linked — and that link is often the most valuable page in the graph, because it’s the pattern rather than the artefact.

Abandonment

A capability hub with an inviting line in the map that nothing ever uses. The hub is aspirational — the projects underneath don’t actually support the claim it makes.

Non-visitation

The abandoned weekend spike nobody ever reaches. Rename it before you demote it — a startling share of these are badly titled rather than useless, and the title is the only thing a walker sees from the map.

There is a second-order use that turns the whole thing into a publishing pipeline. Clusters with heavy traffic and no canonical page are concepts you have built but never written about — discovered from traffic rather than from memory, which is the only way anyone ever finds them, because by definition you didn’t know you had them.

The deal room

An agent preparing a strategy against a folder of diligence documents, reading under explicit probes: strategic fit, integration risk, workforce implications, customer concentration, regulatory exposure.

Those interpretations may have little value outside this deal. Paying to make them permanent in advance would be waste. So most of the deal room’s derived material should die with the deal — which is the correct default and the opposite of everybody’s instinct.

What survives is the residue the walks identify: a missing edge repeatedly walked, a hard comparison worth keeping, a lens implication that survived review. And promotion is a deliberate act, reviewed, with recurrence and survival as the criteria.

This is where the zombie lens is most likely to appear, because deal rooms produce confident narratives with expiry dates nobody wrote down. But notice the asymmetry that makes the discipline bearable: the task world’s telemetry outlives the task world. The interpretations expire on schedule. The record that three walks kept joining two regions does not — and that record is about your map, not about the deal.

The version with no agents in it

The strongest argument in this chapter has no AI in it at all, which is precisely why it lands with people who have no graph.

A practice owner keeps a Word document. Ten years of it, hundreds of pages, every question a staff member ever asked her and the answer she gave. As an answer book it failed — nobody could find anything in it, and she stayed the bottleneck for a decade.

Read it as data rather than answers and it is something rarer. Ten years of real questions, frequency-weighted: which topics recur monthly, which policy has never once stuck from the manual, which questions arrive with every new face at the front desk, which spike every school holidays.

Most knowledge bases are built supply-side, from what the owner thinks matters. She accidentally recorded the other half.

It is this framework run on paper, for a decade, without the map underneath. Every entry exists because a real person doing the real job actually needed it — needed it badly enough to interrupt the boss. No survey, no workshop, no guessing about what mattered.

And it steers something concrete: the build order. Compile demand-first. Rank by frequency and recency; make the top twenty answers bulletproof; let the long tail stay thin; let the seasonal spikes schedule their own reviews. The first week of the build kills the largest share of the interruptions, because the log already did the voting.

The test

Four corpora, one shape. The generalisation is a three-condition test you can run against your own situation in about a minute:

Does this apply to you?

  1. Is there a map? Even a bad one. Even a folder structure, a wiki nobody maintains, or a document with headings.
  2. Is there a walker? Even a person. The walker doesn’t have to be an agent; it has to leave a trace.
  3. Is somebody asking questions repeatedly? Not once. Repeatedly, over months, in a way that recurs.

Three yeses and the exhaust exists. It is almost certainly being deleted, or never recorded, or recorded as operations rather than routes.

Key Takeaways

  • In an inbox, co-traversal reveals relationships the map never recorded — and a cold entity page is a relationship that ended.
  • In a dev wiki, heavy-traffic clusters with no canonical page are ideas you built and never wrote about.
  • In a deal room, the interpretations expire and the telemetry doesn’t — the record is about your map, not the deal.
  • Most knowledge bases are supply-side. Traffic is the other half, and almost nobody keeps it.
  • Map, walker, recurring questions. Three yeses and you already have the data.

Four corpora, same move. Two more places it goes — and one of them points the loop back at the tool doing the walking.

15
Part V · The Same Move, Elsewhere

Absorbing the Sensor, and Filing Back the Walker

Point the loop at a retrieval sensor and it absorbs. Point it at the walker and it compounds.

The move generalises twice more, and both generalisations are already running in the canon under other names — which is the best available evidence that the pattern is real rather than a house style.

Part A — absorbing the sensor

In a hybrid stack, similarity search isn’t a substrate. It’s a sensor: one retrieval axis beside wikilink walk, time, entity and raw grep, returning typed and ranked results that a judge reads rather than obeys.

Which leaves an unresolved cost. A sensor that only ever reads is a tax — you pay for it on every query and it never makes the next query cheaper. The composition only closes when a load-bearing similarity hit becomes a candidate edge for the janitor: the same query-is-a-write loop, run over a sensor instead of a walk.

Absorption, in three states

Before

Certifications live in a dump-ground store. Pricing relationships between successive certifications never existed as a schema column. Every few months someone asks what discounts applied. A vector pass surfaces an email — a Notes 5 exam discounted 50% because of the Notes 4 certification. Useful. Fragile. No edge. Next quarter, the same discovery tax.

During

A walk marks the hit as load-bearing — the answer depended on it. The shaped result sets candidate_edge: true. Walk telemetry goes to the janitor queue proposing cert:LN4 —qualifies-for-discount→ cert:LN5, with support pointers to the email and the certification record.

After

The edge is filed at the appropriate type and rank. The next query resolves primarily by graph walk. Similarity still fires as a safety net; it is no longer the only way to know. You did not migrate the corpus — you absorbed one recurring rhyme into structure.

The worked absorption example, from the sensor doctrine.

Key Insight

Absorption is the missing-edge signal with a different sensor on the front. The walk noticed a join; the pass proposed an edge; the janitor disposed. Only the origin of the hit changed.

And the metric that goes with it, kept honest by its own author: if the absorption loop is running, the share of answers that depend on similarity should trend down for stable regions of the graph. Measure it if you instrument typed warrants. Do not invent a percentage to look scientific. The architecture earns the curve; the curve does not invent the architecture.

Pitfall

Publishing a “RAG dependence declined 40%” figure you didn’t instrument. If you can’t say which warrant made an answer depend on similarity, you can’t compute the share — and a number nobody can reproduce is worse than no number, because it invites the wrong follow-up question.

The same loop, from the ingest side

There’s a mirror worth naming in a paragraph. Hand the ingest engine the query engine’s navigation toolbelt and ingestion becomes a query: a new package walks the existing corpus to discover where it fits, so edges are found by travel rather than computed by similarity.

Which means ingestion walks are walks. They record opens, edges, abandonment and reaches exactly like query walks do, and nothing in the mining pass has to change to mine them. If you only instrument query traffic you’re leaving half your walks on the floor — and ingestion walks are unusually informative, because a filing agent that couldn’t find where something belongs has just told you the map has a hole in a place nobody has asked about yet.

Part B — filing back the walker

Now the recursion one step deeper, and this one changes what kind of system you think you’re building.

The wiki already contained the frameworks — progressive resolution, the nudge doctrine, the graph-navigation material. Those frameworks informed a retrieval improvement. The improvement was implemented in the wiki search tooling. Future searches over that same body of work now inherit the improved cognition — and those searches produce the next improvement candidates.

The insight compiled into the cognitive apparatus that retrieves the insight.

Call it what it is: File Back the Improvement to the Walker. The prior search did not merely add knowledge pages. It improved how future agents traverse and perceive the graph. The wiki stops being only a repository describing frameworks; the retrieval system gradually becomes an executable expression of them.

That loop belongs to the intent-compiler work rather than to this book. What this book contributes is the feed — because the candidates that improve a walker come out of the same records as the candidates that improve a page, and the walk record can tell them apart:

Two defects that look identical until you read the record

A map defect

Repeated broad search where an edge should be. The agent looked for a route, there wasn’t one, it fell back. Fix the graph.

An apparatus defect

Repeated failure of the search tool to surface a page that does exist and is well linked — found later, by a different route, in the same walk. Fix the walker.

Same signal source, different repair target. Without the reach-mode field from the walk record, these are indistinguishable and both get filed as missing edges — which produces a graph slowly accreting links that paper over a tooling problem.

Three loops, and which one this book owns

Naming the boundaries is part of the doctrine, so here they are as a figure rather than a disclaimer.

Content loop

New source → walk existing corpus → integrate significance → richer corpus → smarter next ingest.

Owned by the ingest-is-a-query work.

Use loop — this book

Intent → walk → synthesis and path → derived cache and graph repairs → better next walk.

Everything in Parts II–IV is the machinery of this one.

Apparatus loop

Frameworks in the wiki inspire better retrieval architecture → the architecture is implemented in the walker → future agents inherit improved cognition → their walks expose the next improvement.

Owned by the intent-compiler work.

The third one is the qualitative phase change. When methods of thinking are compiled into the machinery through which the wiki is subsequently perceived, the insight becomes part of the organ that finds insights. That’s worth saying once, plainly, and then leaving alone — it is a real property and it is not a magic one, and nobody has measured its size.

What this changes about the investment case

Both directions compound, and neither requires rewriting a page.

The use loop makes the map better from traffic you were already paying for. The apparatus loop makes the walker better from the same records. A system with both has two compounding directions instead of one — and the assets involved are the graph and the harness, neither of which any vendor can take away from you when the model market moves.

Key Takeaways

  • A sensor that only reads is a tax. Absorption closes the loop by turning load-bearing hits into candidate edges.
  • Absorption is the missing-edge signal with a different sensor on the front — the mechanism is unchanged.
  • Ingestion walks are walks. Mine them with the same pass, and note that a filing agent that couldn’t place something has found a hole nobody has queried yet.
  • The walk record distinguishes a map defect from an apparatus defect. Without the reach-mode field, both get filed as missing edges.
  • Three loops: content, use, apparatus. This book owns the middle one, and says so.

Three loops, two of them somebody else’s. Before the build, the honest pass: where none of this applies, and what is not being claimed.

16
Part VI · Boundaries and the Build

Where This Doesn’t Apply

The honest pass, before the build — including the number nobody has, and the experiment that would produce it.

A doctrine that only tells you where it wins is marketing, not engineering. Five boundaries, one failure this system can cause, and one thing that isn’t being claimed.

1. Low-reuse corpora

If the same class of question never comes back, filed answers are pure cost. You pay storage, map cost and janitor cost for a page nobody will read again — and telemetry has nothing to generalise from, because every signal in this book needs repetition before it becomes evidence. One co-traversal is a coincidence. One abandonment is a bad day.

The loop pays where work recurs. If your questions genuinely don’t recur, stop after Chapter 7 — keep the walks, because they cost almost nothing and you may be wrong about recurrence — and don’t build the rest.

The exception inside the exception: the dead-end log still pays. Ruling a path out once is reusable even when the answer isn’t, because a corpus with no repeat questions may still have repeat regions.

2. High-precision and audit-bound domains

Where every claim must trace cleanly to a source — regulatory, legal, clinical — the compiled layer is a convenience and never an authority.

The criticism of the pattern reaches the same conclusion from outside: it “works less well for situations requiring high factual precision or audit trails”, and for that material “a hybrid approach — keeping the raw sources alongside a well-structured wiki — is safer than relying on the compiled layer alone.”5

Agreed, without qualification. It’s why bronze never goes away and why the layer separation exists. The difference in these domains is that the ranking rule stops being hygiene and becomes a control — something you can point at during an audit, enforced at retrieval rather than by convention.

3. Closed-world, tool-shaped agents

Where the knowledge lives entirely in APIs — the CI fixer, the form filler, the scheduler — there is no map to improve, and this whole apparatus is answering a question nobody asked.

The precise claim, stated precisely: agents need a substrate wherever difficulty is context-depth wearing an intelligence costume. That is most knowledge work. It is not all agentic work, and pretending otherwise is how a good architecture gets a bad reputation.

4. Sparse graphs

Co-traversal is a statement about absence: these two pages met, and no edge connects them. Absence only means something against a background of edges that do exist.

Below some density, “no edge” is the default condition of every pair in the graph, and the missing-edge signal degenerates into a list of everything anyone ever opened together. In a young graph, run dead-ends and cold-pages first and switch co-traversal on once the map has opinions.

5. Privacy and governance of the walk log

This is the boundary most likely to stop a rollout, and it deserves a design section rather than a caveat — because the answer is genuinely good.

A walk transcript records who asked what, which is a different sensitivity class from the corpus itself. The corpus may be freely readable internally; the record that a named person spent Tuesday afternoon walking the redundancy-policy region is not.

Key Insight

The mining pass needs the shape of the walk, not its content. You can redact the words and keep the entire signal.

Page ids, edges followed, branches abandoned, rung of each reach. Not the query text, not the answer, not the identity of the asker. Which yields a clean split most compliance conversations can accept:

Two records, two retention policies

Structural record
  • • Page ids, edge types, abandonment flags, reach rungs
  • • Pseudonymous — no asker identity, no query text
  • • Retained long; this is the mineable asset
  • • Scoped by corpus
Content record
  • • Query text, answer, full tool payloads
  • • Access-controlled, attributable
  • • Retained short; used for dispute investigation and replay
  • • Deleted on schedule, deliberately rather than by default

One honest second-order risk: aggregate walk structure over a small team can still be re-identifying — if three people use the system and one of them is the only person who ever walks the acquisitions region, pseudonymity is thin. Cohort thresholds are the standard answer, this book doesn’t own that field, and you should ask someone who does.

The failure this system can cause

Traffic-driven maintenance strengthens the roads already travelled. Left alone, that is calcification with excellent instrumentation — the minority position quietly compacted away because nobody walked it this quarter.

The guards are structural rather than motivational, because a motivational poster does not survive a cron job:

  • preserve contradictions as edges rather than averaging them into bland prose;
  • retain minority findings from individual probes;
  • maintain known absences as first-class entries;
  • keep advisory similarity or raw-source search as a smoke detector for material the graph does not route toward;
  • re-ground early pages against a richer later corpus;
  • compare paths across models and boot profiles;
  • keep derived pages subordinate to bronze.
The healthiest living wiki is not one with no disagreement. It is one where disagreement has an address.

What is not claimed

Explicit non-claim

There is no measured magnitude for any of this, and none will be invented.

I can specify the mechanism. I can tell you the three signals fall out of walk logs you are almost certainly already producing. I cannot tell you how much better your map gets, how much shorter your walks become, or what any of it is worth in hours or dollars — because nobody has measured it, including us. Every number in this book is a vendor figure or a specimen count, labelled as such.

That posture is the canon’s standing practice rather than a one-off. The Traversal Dividend gets the same treatment: no measured magnitude, and none invented — a mechanism argument, plus behavioural evidence from one operator’s own system, reported as exactly that.

The experiment that would settle it

So that the claim is falsifiable rather than merely modest, here is the study nobody has run.

The ablation

Hold constant: the model, the prompt, the corpus, and a fixed set of genuinely hard questions.

Procedure: run the set; mine the walks; apply the resulting mutations; wait; run comparable questions again.

Measure: unique relevant pages reached; useful graph distance travelled; proportion of movement through typed edges versus repeated broad search; backtracks; source-anchor rate; and blind judged answer quality.

Do not measure: call count. Wall-clock latency. Tokens per answer.

The sharp prediction: repeat walks over a repaired map should show fewer broad searches and fewer backtracks at equal or better judged quality. If they don’t, the mining pass is producing edges that don’t correspond to how anyone actually navigates — and the honest response is to keep the dead-end log and turn the rest off.

The cheaper falsification, available this week

You don’t need the ablation to find out whether this works for your corpus. Log co-traversed pairs for a week. Look at the top twenty. Ask whether a human agrees those edges are missing.

If the answer is no, the signal is noise for your corpus and you should say so out loud rather than tuning until it agrees with you. That test costs an afternoon, and it is the one worth running first.

The burden moved; it didn’t vanish

The last honest thing to say is about work.

Prompt drama becomes knowledge governance. The difference is that the old work was per-agent calibration that evaporated on every model release, and the new work is stewardship of an asset that compounds and has owners. Same hours, opposite direction.

Be specific about the new hours, because “it compounds” is not a substitute for a resourcing conversation. Somebody reviews the queue. Somebody adjudicates promotions. Somebody owns the pages in their domain and answers when a claim under their name is contested. Somebody decides where to cut when a page sprouts edges to everything.

That is real work with real owners, and it does not disappear because it is now productive. What changes is that it accumulates into an asset rather than evaporating on the next model release.

Key Takeaways

  • Low reuse, high precision, closed-world tasks and sparse graphs are all real boundaries. Name them before you sell the idea.
  • Walk logs are a different sensitivity class from the corpus — but the mining pass needs shape, not words, so redaction costs you nothing.
  • Traffic-driven maintenance can calcify. The guards are structural: contradictions as edges, known absences, a similarity smoke detector.
  • No measured magnitude is claimed. The ablation is specified; nobody has run it.
  • The burden moved from per-agent calibration to knowledge stewardship. Same hours, opposite direction.

Boundaries stated, non-claims stated. What remains is the smallest set of steps that produces signal — and it is a shorter list than anyone expects.

17
Part VI · Boundaries and the Build

Thirty Days to a Map That Learns From Use

Week one is one line of configuration. Everything after that is optional until it isn’t.

Stop deleting walk transcripts.

That is week one, and for most systems it is a retention setting rather than a project. Everything else in this chapter is what you do once you have a month of them — and you cannot start any of it retrospectively, which is the only genuinely urgent thing in this book.

Week 1 — Retain

Build: turn retention on. Then fix the record shape — specifically the three fields most frameworks don’t capture by default: edge type on each traversal, branch abandonment with a reached_answer flag, and the ladder rung of each reach past the map. Without those three you have logs, not walks.

Decide now, not later: the structural / content split and its two retention policies.

Do not build: anything else. The temptation is to design the queue before you have evidence there’s anything in it.

Week 2 — Count

Build: the falsification test. Group walks over thirty days, count page-pairs opened in the same walk, subtract pairs that already have an edge, correct for each page’s overall frequency, print the top twenty.

Then read them yourself. Not a dashboard. A list of twenty pairs, read by a person who knows the corpus.

Do not build: the queue, the UI, the scoring. Not yet.

Week 3 — Type

Build, in this order: add derived to the schema with rank-below-source enforced at retrieval, not by convention; backfill support edges on anything already filed; wire the lint pass to support-change invalidation; add the derived-on-derived check and run it on write; then implement the triage rubric, defaulting to discard.

Only now turn filing on.

Week 4 — Queue

Build: the dead-end pass, with a repetition floor, a date on every record and an invalidation rule tied to its region. The cold-page pass, emitting in the rename → relink → demote → compact ordering. One queue with evidence pointers and no composite score. One weekly human review — fifteen minutes, three items.

If a dispute channel already exists, point it at the same queue. If not, that’s next month’s work.

Week two is the whole bet

Twenty lines of code, and the outcome tells you whether to build the rest.

# the only code you need in week two
walks = load_walks(days=30)

pair_counts, page_freq = Counter(), Counter()
for w in walks:
    pages = set(w.pages_opened)
    page_freq.update(pages)
    pair_counts.update(combinations(sorted(pages), 2))

candidates = sorted(
    ((a, b, n, lift(n, page_freq[a], page_freq[b], len(walks)))
     for (a, b), n in pair_counts.items()
     if not graph.has_edge(a, b)),
    key=lambda r: -r[3]
)[:20]

for a, b, n, score in candidates:
    print(f"{n:3d} walks  {score:5.2f}  {a}  <->  {b}")
What you see in the top twenty What it means What to do
Most obviously should be linked The signal is live for your corpus Continue to week three
All hubs — index, glossary, home Your frequency correction is wrong Fix the lift calculation; re-run
Plausible, but you wouldn’t link them Pairs co-occur for task reasons, not semantic ones Raise the distinct-intent floor; re-run
Genuinely nothing The missing-edge signal is noise here Say so. Keep dead-ends and cold-pages. Don’t build the rest.

That last row is a real outcome and it should be an available one. A method you can’t fail is a method you can’t trust.

Pitfall

Filing before typing. Turn write-back on in week two and by week three you have a few hundred untyped syntheses, no support edges, and no way to reconstruct which pages any of them leaned on. Retrofitting a type onto a thousand untyped pages is far harder than writing the type first — and the ones you can’t reconstruct supports for are simply lost. The ordering isn’t stylistic.

First thing, last thing, never

First: co-traversal

Cheapest to build, needs no schema change, produces a list a human can argue with on day one, and falsifies itself quickly if it isn’t working.

Last: anything with a model in it

Every model in this pipeline is an authority you will have to govern. The pass is a prior. Keep it that way for as long as you can.

Never: a dashboard

Nobody looks at it after week three, and its existence substitutes for the fifteen-minute review that actually changes the graph.

Why the asset appreciates

One reason to do this now rather than eventually, and it isn’t about your current model.

The Traversal Dividend says that a compiled graph gains value when a more agentically capable model can navigate it more deeply and reliably — better driver, and a larger proportion of the road network becomes usable. The content of the graph may be completely unchanged; what expands is what the model can reach.

Telemetry is how you find out which parts of the network were unreachable in the first place — and a repaired edge raises the usable radius for every model, including the ones that haven’t shipped yet. You are not tuning for today’s walker. You are removing obstacles that every future walker would otherwise hit.

Models do not retain your private learning. Architecture does. But better models can harvest more of what the architecture retained.

The close

Go back to the tab from the first chapter. The assistant is still there, still gathering a hundred pages, still binning them at the end of the turn.

Nothing about that is inevitable. It is a retention decision, made once, by default, by somebody who wasn’t thinking about it — and inherited by every turn since. The same decision is sitting, unmade, inside whatever you are building.

Prompt engineering spends effort on the perishable layer; wiki curation spends it on the compounding one.

Two lanes, and both of them run through the same walk. File the hard answers back as typed derived cache that can never outrank its sources. Keep every path, and let twenty lines of counting code tell you which edges are missing, which roads lead nowhere, and which pages nobody can find.

The map improves from being used, not just from being fed.

Do this before you do anything else

Stop deleting walk transcripts. Then write the twenty lines that count co-traversed page-pairs across the last thirty days and print the top twenty with no edge between them.

If that list makes you wince, you have just found the first thing your map has been quietly getting wrong — and you found it for free, out of traffic you were already paying for.

Your system is already producing this data. The only question is whether anything reads it.

Key Takeaways

  • Week one is retention plus three record fields. Anyone planning a quarter has already lost the month of data they needed.
  • Week two is the falsification test, and “genuinely nothing” is an available outcome.
  • Type before you file. Retrofitting supports onto untyped pages loses the ones you can’t reconstruct.
  • First co-traversal, last anything with a model in it, never a dashboard.
  • A repaired edge raises the usable radius for every future walker, including the models that haven’t shipped.
REF
Sources & Evidence

References & Sources

The evidence base behind every claim — primary research, industry analysis, and technical specifications

Research Methodology

This ebook draws on primary research from standards bodies, independent research firms, enterprise technology vendors, and consulting firms. Statistics cited throughout have been cross-referenced against primary sources.

Frameworks and interpretive analysis developed by Scott Farrell / LeverageAI are listed separately below — these represent the practitioner lens through which external research is interpreted, and are not cited inline to avoid self-promotional appearance.

Primary Research & Standards Bodies

Andrej Karpathy (GitHub Gist, created 4 April 2026) — LLM Wiki [1]

Good answers can be filed back into the wiki as new pages so explorations compound instead of disappearing into chat history

https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f

Mem0 (18 July 2026) — AI Agent Memory 2026: Progress Benchmark Report Evaluations [4]

Memory staleness as an open problem: a highly-retrieved memory is accurate until the underlying fact changes, at which point it becomes confidently wrong

https://mem0.ai/blog/state-of-ai-agent-memory-2026

Jorge Luis Borges (trans. Andrew Hurley) — On Exactitude in Science (1946), Collected Fictions [8]

The 1:1 map of the Empire, coextensive with the territory and therefore useless

https://kwarc.info/teaching/TDM/Borges.pdf

Yiqi Wang et al., arXiv:2606.04990 (3 June 2026) — From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents [11]

Final-answer accuracy alone cannot explain how an output was produced; execution provenance is defined as the typed graph of an agent execution

https://arxiv.org/abs/2606.04990

Industry Analysis & Vendor Research

SamurAIGPT (SkillsLLM listing) — llm-wiki-agent [2]

Directory convention including syntheses/ for query answers filed back as wiki pages, and log.md as an append-only record of every operation

https://skillsllm.com/skill/llm-wiki-agent

Micheal Lanham (Medium, 24 April 2026) — Knowledge and Memory Beyond RAG: Why 2026 Agents Need a Write Path, Not Just a Retriever [3]

Retrieval is still the read path; the novelty is the write path, and stale writes outlive the chats that created them

https://medium.com/@Micheal-Lanham/knowledge-and-memory-beyond-rag-why-2026-agents-need-a-write-path-not-just-a-retriever-ae2547b7ffe9

Denser.ai — LLM Wiki: Karpathy's Idea for AI Knowledge Bases [5]

Reports gist-comment criticism (~1,000-file collapse, compounding hallucinations, category error) and the recommendation for a strict type taxonomy including syntheses, citation model and contradiction handling

https://denser.ai/blog/llm-wiki-karpathy-knowledge-base/

Anthropic (developer documentation) — Prompt caching [6]

Cache-read tokens are billed at approximately one-tenth of the base input rate, so re-reading an unchanged prefix costs a fraction of processing it fresh

https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching

anthropics/claude-code — [FEATURE] Decision History Tracking with DECISIONS.md (issue #15222) [7]

Backtrack patterns are invisible; rejected-option reasoning evaporates; proposed status states ACTIVE / REJECTED / BACKTRACKED / EXPLORING

https://github.com/anthropics/claude-code/issues/15222

OpenTelemetry — GenAI semantic conventions / AI agent observability [9]

Standardised spans for model calls, tool calls and agent steps; agent-specific telemetry beyond traditional observability

https://opentelemetry.io/docs/specs/semconv/gen-ai/

LangChain — LangSmith evaluation documentation [10]

Agent evaluation captures the full trajectory of steps, tool calls and reasoning; datasets are built from production traces for replay and comparison

https://docs.langchain.com/langsmith/evaluation

LeverageAI / Scott Farrell — Practitioner Frameworks

The interpretive frameworks, architectural patterns, and practitioner analysis in this ebook were developed through enterprise AI transformation consulting. The articles below are the underlying thinking behind those frameworks. They are listed here for transparency and further exploration — not cited inline, as this is the author's own analytical voice.

Scott Farrell — The Wiki Playbook

A query yields a filable answer and a mineable path; discarding both means paying to explore and binning the exploration

https://leverageai.com.au/wp-content/media/articles/article.php?article=176-the-wiki-playbook

Scott Farrell — Ingest Is a Query: The Self-Hosting Wiki

Ingestion is dumb by design and all intelligence is deferred to query time, which is why a corpus accumulates but never compounds

https://leverageai.com.au/wp-content/media/articles/article.php?article=110-ingest-is-a-query

Scott Farrell — The Scout and the Senior

One explorer toolbelt, many North Stars: query, ingest and janitor differ only by North Star, write access and loop driver

https://leverageai.com.au/wp-content/media/articles/article.php?article=71-the-scout-and-the-senior

Scott Farrell — Executable Worldview

Six layers kept separate, from bronze territory through derived cognition cache to the experience ledger, each with its epistemic status

https://leverageai.com.au/wp-content/media/articles/article.php?article=159-executable-worldview

Scott Farrell — The Code Is the What; The Transcript Is the Why

The transcript records intent, rejected alternatives and unbuilt plans that the repository structurally cannot hold

https://leverageai.com.au/wp-content/media/articles/article.php?article=79-the-code-is-the-what-the-transcript-is-the-why

Scott Farrell — Hora's Watchmaker

Complex systems that survive an interrupting world are built from stable intermediate forms; decomposition is how work outlives the worker

https://leverageai.com.au/wp-content/media/articles/article.php?article=74-horas-watchmaker

Scott Farrell — Intent-Conditioned Task World

Task-local interpretation filed without type or expiry becomes a zombie lens treated as current institutional belief; promotion must be a deliberate act

https://leverageai.com.au/wp-content/media/articles/article.php?article=160-intent-conditioned-task-world

Scott Farrell — The Index Is the Data

Hallucinated consolidation: a janitor compacting under a loose North Star can merge two distinct ideas; the mitigations are chronological legibility, an auditable diff, instant revert and a periodic lint pass

https://leverageai.com.au/wp-content/media/articles/article.php?article=63-the-index-is-the-data

Scott Farrell — The Inference Field

Do not trim the history, compile the state: summarising keeps the conclusions and destroys the trace of rejected branches, contradictions and known absences

https://leverageai.com.au/wp-content/media/articles/article.php?article=177-the-inference-field

Scott Farrell — The Blur Is Load-Bearing

The read-side resolution ladder: map, page, skeleton, grep, source, with cost rising as you descend

https://leverageai.com.au/wp-content/media/articles/article.php?article=75-the-blur-is-load-bearing

Scott Farrell — Context Arbitrage

Compiled context flips intelligence from opex to capex; cheap cached models make exploration-heavy architectures economical

https://leverageai.com.au/wp-content/media/articles/article.php?article=72-context-arbitrage

Scott Farrell — Nudge Doctrine

Many priors, exactly one judge: a prior needs direction not calibration, and a wrong authority is expensive while a wrong prior is free to ignore

https://leverageai.com.au/wp-content/media/articles/article.php?article=100-nudge-doctrine

Scott Farrell — Cascade Ledger

AI decides the meaning and edge type; deterministic software calculates the graph consequences. Prefer the auditable under-count over the confident score

https://leverageai.com.au/wp-content/media/articles/article.php?article=144-cascade-ledger

Scott Farrell — Paid to Write, Never Paid to Read

Resurrection rate as the read-path metric: dormant documents returned to circulation, with first-read count, cross-department reads and double-click events as companions

https://leverageai.com.au/wp-content/media/articles/article.php?article=128-paid-to-write-never-paid-to-read

Scott Farrell — RAG Was Built for Chatbots — Agents Need a Wiki

Path Testing: grade the path independently of the answer; the 2x2 catches the unsupported lucky answer

https://leverageai.com.au/wp-content/media/articles/article.php?article=69-rag-was-built-for-chatbots-agents-need-a-wiki

Scott Farrell — Reflexive Agent Design

The bad-path/right-answer cell is invisible to output-only suites; path-aware suites treat it as a failure of architecture dressed up as a win

https://leverageai.com.au/wp-content/media/articles/article.php?article=146-reflexive-agent-design

Scott Farrell — The "This Answer Is Wrong" Button

Dispute tickets are labelled failure events with frozen evidence; passive telemetry says where walkers went, dispute tickets say where they were failed

https://leverageai.com.au/wp-content/media/articles/article.php?article=124-feedback-button

Scott Farrell — Signal-Case Queue

Significance reprices as-at now; a FIFO queue is a polite lie about attention and the queue behaves more like a market

https://leverageai.com.au/wp-content/media/articles/article.php?article=143-signal-case-queue

Scott Farrell — Capture Was Never the Bottleneck

Ten years of questions read as data is demand-side telemetry: File Back the Walk run on paper, without the map underneath

https://leverageai.com.au/wp-content/media/articles/article.php?article=84-capture-was-never-the-bottleneck

Scott Farrell — RAG Demoted to a Sensor

Similarity search demoted from substrate to one retrieval axis under the wiki-graph, returning typed and ranked results to a single judge

https://leverageai.com.au/wp-content/media/articles/article.php?article=130-rag-demoted-to-a-sensor

Scott Farrell — Intent Compiler

File Back the Improvement to the Walker: the insight compiled into the cognitive apparatus that retrieves the insight

https://leverageai.com.au/wp-content/media/articles/article.php?article=141-intent-compiler

About This Reference List

Compiled July 2026. All URLs verified at time of compilation. Regulatory documents and standards specifications are subject to revision — check primary sources for the most current versions.

Some links to academic papers and vendor research may require free registration. Government and standards body publications are freely accessible.