AI Engineering · Retrieval & Ingestion

The Author’s Attention: Ranking Files by How Often You Talked About Them

Every heuristic for deciding which files matter loses the same case — the load-bearing file five levels deep that looks like junk. But you talked about it forty times. That count is a free importance signal sitting in your own transcripts, if you treat it as a prior, not a verdict.

In a big, messy project — five hundred files, five directory levels deep — you cannot read everything. You cannot summarise everything either; that is just reading with extra steps. So you give an ingesting agent a budget, because a model’s attention is finite and every token you feed it is spent from a fixed account.¹ The budget’s whole job is to decide which files are worth that scarce attention. And to decide, it needs to rank the files by importance before it has read most of them — which is the trap, because the only things you know about a file for free are the two things that don’t reliably tell you it’s important.

You know its type — a .py outranks a .log, a README outranks a lockfile. And you know its depth — closer to the root usually means more central, and something buried five levels down is usually incidental. Both priors are reasonable. Both are structural. And both share one failure mode: sometimes the thing five levels down is the important one. You could reach it, but only by sifting through all the junk around it — and that’s too big. So the budget spends itself on shallow, well-typed plumbing and never reaches the one file the project was actually built around.

I hit this while building a wiki over my own dev folder, and I’d already given myself the tool that fixes it — I just hadn’t noticed. In a previous piece I described joining two years of archived Claude Code conversations to the code they produced, on a natural key: the dev-folder name. That join was built to answer why a project exists. It turns out it also answers which files matter — and that second answer is what this piece is about.

Prominence is not importance

Here is the thing the budget apparatus taught me, stated as bluntly as it deserves: prominence and importance are uncorrelated in real corpora. The files at the top of the folder were plumbing. The interesting thing — a project where I’d once had Claude write its own zip-reading, CSV-joining query tooling over a raw Salesforce backup, and solve a client’s data problem with no system access at all — was two levels down, and mostly visible in the transcript, not the tree. A naive summariser — top of the folder, top of the file, top-k chunks — structurally cannot find that. It will write a one-dimensional, black-and-white comment about the code and move on, and every generic “chat with your codebase” tool does exactly that.

The reason is worth being precise about, because it tells you where to look for the fix. Type and depth are properties of the file’s position in the structure. Importance is a property of what the file did for the project — and that isn’t written in the structure. It’s written in the history of the work: in which files got returned to, argued about, debugged at 1 a.m., and mentioned again three weeks later. Structural signals can’t see history. But I had the history sitting in Postgres.

The second signal: the author’s attention

Once the conversations are joined to the code, a deterministic pass over the transcripts gives you something no static heuristic can match: a mention-count per file. Run through every conversation turn for a project, extract the file paths, and tally them. A file the author discussed forty times is load-bearing regardless of where it lives in the tree. Three hundred shallow files nobody ever spoke of are safely skippable. That’s it. It’s embarrassingly simple, it’s free once the join exists, and it rescues exactly the deep-file case the depth prior loses.

The code tells you what exists; the conversation tells you what mattered. That’s attention-weighted importance ranking — and the join shouldn’t just add context, it should re-rank the traversal.

What makes this trustworthy where similarity isn’t is that it’s a record of a real behaviour, not a resemblance. Nobody mentions a file forty times by accident. The count is a fossil of attention — the author’s own foraging over the project, captured involuntarily, dated, and now queryable. When I actually looked at what a project’s ingestion had produced with this signal switched on, the flat black-and-white comment had turned three- or four-dimensional, with all these colours in it: intent, the clever bit, why a decision went the way it did. Not because the model got smarter. Because it was finally pointed at the files I’d told it mattered — months ago, without knowing I was labelling training data.

But — and this is where most people would ship the nuisance version and conclude the idea doesn’t work — raw mention-count is not yet a usable signal. Two small disciplines are the difference between a signal and a nuisance, and I found both the only place anyone ever finds them: on contact with real data.

Discipline one: mask the structural files

The first thing the raw count does is scream about the wrong files. The loudest file in my tally was CLAUDE.md — the project’s instruction file — followed by the other structural fixtures. Of course they were loud. They get mentioned in nearly every conversation because they’re structural, which is exactly why their mention-count says nothing about which project this is or where its interesting work lives. A file that appears everywhere carries no information about anywhere.

If that logic sounds familiar, it’s because it is the oldest result in information retrieval. In 1972 Karen Spärck Jones formalised the idea that a term’s value is an inverse function of how many documents contain it — a word in every document discriminates nothing, a rare word discriminates a lot.² That single observation is why every text-frequency system since has a stopword list. What I’d built was a frequency signal over a corpus, so of course it needed one too. The CLAUDE.md mask is stopword removal for attention data. Every corpus-level frequency signal ever constructed has needed its stopword list, and you never find that list in theory — you find it the moment you run the count on real data and watch the structural files float to the top.

Discipline two: perturb, don’t command

The second discipline is about restraint, and it’s the one that’s easy to get wrong in the other direction — by trusting the signal too much. Author-attention is informative, but it is also biased, and in a specific, predictable way: you talk most about what was broken, not only about what was important. The file you fought with for a day because a bug hid in it racks up mentions that a quietly perfect, genuinely central file never will. So attention earns a thumb on the scale — never the whole hand.

In the implementation that meant something deliberately modest. The top-attention files get an asterisk in the file list — a marker the agent can see — and the budget gives them a tiny bump, sized to how prominent the file was in the history. Not a re-sort. Not an override of type and depth. A nudge. The attention signal perturbs the ranking; it doesn’t command it. It’s a prior, not a verdict — it shifts the odds for a file that the structural signals were about to skip, and then gets out of the way.

And the bump’s size wasn’t argued into existence — it was tuned. I ran the budget a few times and adjusted the strength down until the change did what I wanted and no more. Sizing a heuristic empirically against outcomes is the only honest way to weight one; a coefficient you picked because it felt right is just a preference wearing a number. Which raises the obvious question the whole idea lives or dies on: how do you know it helped?

The A/B: measuring the right thing

I A/B test everything now, because Claude Code’s ability to stand up a harness makes it nearly free to. Before I’d ever added the attention signal, I’d had it build a parallel evaluation harness that runs on a fresh copy of the real data each time — real inputs, so the results mean something, but duplicated so a run can never mutate the source. Different models, prompts, and ideas all drop into the same rig. So when the attention bump was ready, “is it actually better?” cost one run instead of one argument.

The part that took a beat to get right wasn’t the harness — it was choosing what the eval should measure. The tempting metric is did the rankings change. That’s the wrong one: rankings churning tells you the signal is doing something, not that it’s doing something good. The right metric is one level up: did the budget’s contents improve — did genuinely interesting files that were absent last time now make the cut, and did definite junk fall out?

That last point is the one worth keeping. Because I measured budget-contents rather than ranking-churn, the harness will now score any future budget heuristic the same way. Recency weighting, a stronger or weaker depth prior, a structural signal borrowed from elsewhere — each one stops being a debate and becomes an afternoon’s A/B. The attention bump is a nice win. The scoring rig is the durable one.

The disagreement is its own instrument

There is a second, structural way to rank a codebase’s files, and it’s worth naming because the relationship between the two signals is more useful than either alone. Aider — an open-source coding agent — builds a map of a repository, extracts each file’s signatures and structure, and ranks files with a graph-ranking algorithm over their dependencies, keeping the most important within a token budget.³ It uses a lineage that goes straight back to PageRank: importance flows from how many important things point at you.⁴ (I know Aider does this because I had Claude Code read it — prior-art archaeology on a peer. Steal the ideas, cite the influence, leave the code. It was too specific to their needs anyway.)

That’s the code’s opinion of importance: what the codebase references most. Author-attention is the author’s opinion: what the human returned to most. They usually agree. When they disagree, the disagreement is diagnostic — which turns two rankings into a single instrument:

The off-diagonal cells are the payoff. Heavily-referenced-but-never-discussed is plumbing — real, necessary, and safe to summarise thinly. Much-discussed-but-barely-referenced is an idea that the author cared about but never built — the design that got argued into shape and then abandoned, which a code-only tool erases completely because there’s almost no code to point at. That second cell is the one a wiki explicitly wants, and it only exists because you have two rankings to disagree with each other.

Idea to production, mid-conversation

Here is the part I keep turning over. The attention signal went from a sentence in a design conversation to a confirmed, A/B’d improvement in the same working session. A few turns after someone said “the conversation tells you what mattered,” it was implemented, masked, tuned, and measured — a real ranking improvement with a stopword mask and a coefficient. That’s the cool bit, and it wasn’t typing speed that made it fast. Three things that were already in place did the work.

The data was already captured — file-touches per turn were sitting in Postgres because I’d archived every conversation fastidiously and joined it on a real key long before I knew why. The harness already existed — so “was it better?” was one run, not one meeting. And the change itself swung the cheap direction: the agent could have found hot files by walking the conversation itself every time, but a deterministic hint computed once costs nothing per run, forever, and frees the model’s attention for judgment instead of bookkeeping. Cheap code hinting, the model deciding — the division of labour this kind of system keeps arriving at.

None of the pieces are clever on their own. Counting is not clever. Stopword removal is fifty years old. A tuned coefficient is the plainest thing in statistics. What’s new is the willingness to point that boring machinery at the residue of your own attention — to treat the fact that you talked about a file forty times as data, not exhaust. The code will always tell you what exists. Your conversations, if you kept them, are quietly telling you what mattered. Count them, mask the noise, nudge the budget, and let the file five levels down finally get read.