AI Engineering · Retrieval & Ingestion

Cache the Significance, Not the Description

Generic “chat with your codebase” summaries read flat because they store the one layer worth nothing — the description, which any pass can regenerate for free. The thing worth keeping is the significance: why it mattered, compiled judgment that can’t be recovered once the moment’s context fades. Aim ingestion at the dear thing, not the cheap thing.

Point any generic “chat with your codebase” tool at a real project and read what it writes back. It is almost always flat. This module handles authentication. This service syncs records. This folder contains PHP utilities. All true, all useless — a one-dimensional, black-and-white comment about the code that tells you nothing you couldn’t have guessed from the filenames. I know the shape of that failure well, because my own ingestion pipeline used to produce it, and I spent a good while working out why.

I’ve been building a wiki over my own dev folder — something like a hundred project directories, mostly Python, some of it years old — so that I can ask it where have I implemented voice AI, where have I used this pattern, why did I build this the way I did. The point was never to find a function. It was to get behind the projects: to recover the thinking. And early on, it read exactly like every other summariser. A project would come back described as a couple of boring PHP files, and I’d think: that’s not what that project was. The description was accurate. It just cached the wrong layer.

Prominence is not importance

Here is the thing that took a while to see, stated as bluntly as it deserves: prominence and importance are uncorrelated in real corpora. The files at the top of a folder are usually plumbing — config, entry points, scaffolding. The interesting thing is typically two levels down, in a file that looks incidental, and it is often visible only in the transcript of the work, not in the tree at all. Structure tells you where a file sits. It does not tell you what that file did for the project — and importance is a property of what it did, which is written in the history of the work, not its layout.

This is why the flat summary is not a prompting problem you can fix with a better adjective. It is structural. A naive summariser reads the top of the folder, the top of the file, the top-k most similar chunks — and then stops. The stopping is the problem. Modern retrieval over a large corpus is almost always a graph walk with a budget: start near the query, explore the most promising neighbours, and terminate as soon as you’ve gathered k good-enough results. That termination rule is not incidental — it is the defining behaviour. The nearest-neighbour literature is precise about the consequence: greedy top-k search halts the moment it has found k nodes closer than the current candidate, and relaxing that stopping rule to let the walk continue is guaranteed to return strictly better results.¹ In other words, the deep, load-bearing node is frequently sitting just past the point where a top-k retriever decided it had enough. Prominence is what the retriever reaches before it stops. Importance is often one hop further in.

So a tool built on top-of-tree, top-k retrieval isn’t writing a flat summary because it’s lazy. It is writing a flat summary because the interesting file is, by construction, the one it structurally never reached. Which means the fix isn’t a smarter model reading the same shallow slice harder. The fix is engineering that makes reaching the depth affordable — and then, having reached it, storing the right thing when you get there.

Two layers: the regenerable and the dear

The reason most pipelines waste the depth even when they reach it is that they store the wrong layer. And once you name the two layers cleanly, the whole storage question resolves itself.

Description is regenerable. What a piece of code does can be recomputed from the skeleton — the signatures, the call graph, the structure — any time you like, by any model, on demand. It is the cheapest thing in the pipeline to produce fresh. Which means caching it was always low-value: you are spending storage, and staleness risk, on the one thing you could have regenerated for free the instant you needed it. A cached description is a photograph of something that will still be standing there when you look up.

Significance is compiled judgment. Why this was built, the clever move inside it, how it links to the rest of your work, the day’s context that made it matter — that is expensive to derive and, crucially, impossible to regenerate once the moment’s context has faded. Re-run your summariser over that boring PHP folder next year and you will get a fresh, accurate description and a permanently lost significance, because the conversation that explained why it mattered is no longer in the frame. Three of the four ingredients of significance — intent, cleverness, the day’s context — evaporate the instant the surrounding transcript is gone.

Early passes stored the cheap thing; the tuned pipeline stores the dear thing. That’s the whole storage doctrine in one before/after.

This is the ordinary logic of caching, applied to meaning. A cache earns its keep only on things that are expensive to compute and stable enough to be worth holding — which is exactly why prompt caching gives you roughly a 90% discount on re-reading a large static context you’d otherwise recompute every call.² You cache what is costly to derive, never what is trivial to reproduce. Every engineer knows not to memoise a function that’s cheap to recompute. Ingestion pipelines have been quietly doing the opposite — lovingly caching the description, the one layer that regenerates for free, and letting the significance, the one thing a cache actually exists to protect, evaporate. Flip it. The description is your get_skeleton call, resolved fresh on demand. The significance is what goes in the wiki.

When I finally got this right, the change in what the ingestion wrote was not subtle. The flat black-and-white comment turned, in my own description of it at the time, three- or four-dimensional, with all these colours in it — the intent, the clever bit, why a decision went the way it did. Not because the model got smarter between passes. Because it was finally storing the layer that was worth storing.

The receipt: a boring PHP folder, two levels down

Here is the project that taught me this, because the doctrine is only worth anything if it survives a real example.

In the root, it looked like nothing — a couple of PHP files, the kind of thing you’d glance at and file under “utility, ignore.” Down a couple of levels was the actual story. I’d once downloaded a full Salesforce backup — a pile of zip files, CSV exports of every table in the deployment — because I was debugging a data problem and couldn’t get at the live system. I asked Claude Code to go into the backups and find the data behind the bug. It wrote its own Python to open the zip files, read the CSVs, and run its own joins and queries across them — reconstructing a query layer over a dead export, with no Salesforce access, no credentials, no connector. It found the problem. I’ve been getting client answers out of that little tool ever since.

That is the brilliance the flat summary threw away. Not “this folder contains PHP that queries CSV data” — true, and dead on the page. The significance is: the backup is the API. Full data-layer archaeology over a raw export, zero system access, still producing answers months later. That’s a reusable move, a small piece of engineering judgment worth surfacing the next time I face a locked-down system — and it lived entirely in the transcript, invisible to any pass that read only the code. The waves, the competing file budgets, the deterministic samplers, the join that put the conversation next to the code — all of that machinery exists for one reason: to make reaching this depth affordable. The Salesforce page is the receipt that it works.

Two things made recovering it possible, and both are worth naming because they’re the substrate the whole doctrine rides on. First, the code and its conversations were joined on a natural key — the dev-folder name — so the ingestion could read the why alongside the what. That join is a piece of work in its own right; I’ve written about it separately as the soft join, and its payoff here is exactly the reason to trust what it surfaces: it tells you these conversations created this code. Provenance, not resemblance. Second, once joined, the transcript doubles as an importance signal — the files I talked about most were the load-bearing ones, wherever they sat in the tree, which is the mechanism behind the author’s attention. Between them, they get the pass to the deep file. The storage doctrine is about what it writes down when it arrives.

Enthusiasm inflation, and the lint that stops it

There is a failure mode built into this the moment you succeed, and it is worth wiring the guard in while it’s cheap.

A pass tuned to “find the brilliance” works beautifully on the Salesforce folder because there was genuine brilliance to find. Point the same lens at a mediocre project — a CRUD app that is exactly what it looks like — and it will be tempted to supply brilliance that isn’t there. It will write that a routine bit of plumbing was an “elegant, sophisticated solution,” because that is the register you tuned it to, and now every folder gets a rave review. Enthusiasm inflation is the failure mode of an editorial lens. A wiki where every page insists its project was remarkable is exactly as useless as a wiki where every page is flat — you’ve just swapped grey for uniform gold, and neither tells you where the real work is.

The antidote is already sitting in the doctrine, and it costs almost nothing: a significance claim must carry a pointer. Every superlative has to cite the exact exchange, file, or dated moment it came from, and the janitor treats any superlative without a receipt as lint — flagged, not published. The Salesforce claim passes: it points to the conversation where I gave the instruction and the tool that came out of it, dated, retrievable. A generic “this is an elegant solution” with nothing behind it fails, and gets stripped.

That single line does more work than any tuning. It converts “how impressed should the ingestion be” from a matter of taste into a matter of evidence. An impressed claim that can point at its receipt is a genuine discovery you want in the wiki forever. An impressed claim that can’t is the model doing PR for a folder, and the whole value of the archive depends on never letting that through. The pointer is also what makes the significance usable later: the reader gets the claim and a link to the original artifact, a receipt they can audit with one click. It’s the difference between a wiki that asserts and a wiki that shows its working.

Commit Zero: the first prompt is the North Star, written involuntarily

All of which leads to the single cheapest, highest-value significance claim in the whole pipeline — and the crispest illustration of caching the dear thing instead of the cheap one.

When I was keeping all those archived conversations, I noticed something about the very first message of a project’s very first conversation. It is unreasonably useful. Before any of the grind — before the debugging, the refactors, the thousand small compromises — the first thing you type to the machine is your intent at its purest concentration. What was the influence, why am I doing this, when did this start. It’s all there, in the opening breath, because you haven’t started negotiating with reality yet. And the first response from the AI is just as useful in a different way: it is the first interpretation — the draft spec, the moment your intent became a plan. So I kept both, deliberately.

The first user message of the first conversation is the North Star of the project, written involuntarily, before any grind contaminated it — maximum intent density, zero noise.

Together, those two messages are the project’s commit zero. And they answer, from the author’s own mouth and dated, exactly the question the whole ingestion exists to ask: get behind it — why was this built? That’s worth promoting from a lucky heuristic to a schema. Every project page carries an Origin claim: extracted deterministically — the first exchange is always available, no model judgment required to locate it — always present, with a pointer to the exact exchange. It is the cheapest high-value claim you can mint, because you don’t have to compute the intent at all; the author already stated it, and you’re just filing it with a date attached.

Notice that the Origin claim is the storage doctrine and the lint rule in miniature. It is pure significance — the why, not the what — and it is the one significance claim that can never be enthusiasm inflation, because it isn’t the ingestion’s opinion of the project. It’s a dated quote from the person who built it, with the receipt built in. It is also, quietly, a priority record: here is me describing this idea to a machine, timestamped, before anyone published anything. Compiled judgment you couldn’t regenerate, captured at the one moment it was legible, and kept.

What to write down

So when you build the ingestion pass — over code, over documents, over your own archived conversations — the question is not “how do I summarise this?” The question is which layer am I caching.

Don’t cache the description. It’s the cheap thing: regenerable from the skeleton on demand, worth nothing in storage, and the reason every generic tool reads flat. Cache the significance — the intent, the clever move, the canon-linkage, the dated origin — because that’s the dear thing: compiled judgment, expensive to derive, and gone for good once the moment’s context fades. Aim the pass at why this was built, not what it does. And make it prove every claim it’s proud of, because the same lens that finds real brilliance will manufacture fake brilliance the moment there’s none to find.

Brilliance with a citation is archive. Brilliance without one is marketing. Store the archive. Regenerate the rest.