Picture an afternoon of digital archaeology. You are restoring old files for a personal memorabilia wiki — not for nostalgia alone, but because you have already learned the hard way that corporate agents are dumb without a compiled world, and the same architecture applies at personal scale. Somewhere in the sediment sits a Lotus Notes database. You do not have Notes.

The old path was a sealed tomb:

The new path is shorter and slightly unhinged. A coding agent looks at the .nsf, decides it needs a reader, writes one, and opens the file. Notes is not a ZIP with ideas above its station. HCL’s lineage still documents a native storage model — open the database, search notes, open notes, read typed items.¹ Independent reverse-engineering projects such as libnsfdb exist precisely because people spent years documenting enough of the on-disk format to extract data forensically.²

So no: the agent probably did not invent the entire format from pure mathematics in five minutes. It synthesised a tool from format knowledge, headers, structural clues, and code patterns that already existed in the world. That is almost more interesting. Old-world software archaeology used to stop at the door of every extinct product. Coding agents can now manufacture the archaeological tool at the tomb door.

Format extinction is ceasing to imply information extinction.

Hold that line. It is true and incomplete.

Recoverable is not thinkable

Opening the NSF is a triumph of tool manufacture. It is not yet a worldview. Beside the Notes database sit email archives, coding-agent transcripts, source trees, photo libraries, ChatGPT exports, old project databases, and backups of backups. Each one can be “opened.” Each one still lives in a separate thought episode with a separate query language.

That is the pain this book is written against. Not “AI cannot read my files.” It increasingly can. The pain is: my life will not join itself. The economic cost of understanding multi-decade sediment used to be absurd because every extinct format was a sealed tomb. Format recovery is getting cheap. Joinability is still expensive — unless you change the representation.

Two collisions, one afternoon

The same archaeology session produced a second collision that only looks like trivia until you see the architecture underneath. Looking up rsync leads to Samba, which leads to Andrew “Tridge” Tridgell — and a memory of a tiny computer-chess world in which Tridge, Joel Veness, and a private engine called Chompster once occupied the same improbable map. That chain is not stored in any one product. We will use it later as the proof that homogenisation is doing real work. For now, notice only this: the tool that copies the archive and the social geometry of a life twenty-five years ago are adjacent once something can think across residue.

Previously, every sealed format created a little tomb. Coding agents punch holes in the tombs. What they do not automatically create is a common language for what comes out.

What this book will do

We are not going to sell you a better chunker for each silo. We are not going to re-litigate the entire RAG-versus-wiki war — that ground is covered in The Index Is the Data and related pieces. We are going to run a compiler frame end to end:

• Semantic-closure bundles as the translation units
• The wiki as the intermediate representation
• Agents as the runtime

Corporates need a compiled organisational world so their agents are not dumb. People need a compiled personal world for the same reason. The Gmail agent that went from dumb-and-annoying to brilliantly silent is not a different model — it is the same cheap model against a legible world. When it finally speaks, you take notice, because it probably means it.

The reader question is simple and brutal: how do NSF files, email, photos, and code transcripts become one thinkable surface? The takeaway, which the rest of the book will earn, is a two-stage discipline: bundle each source until its meaning closes, then lower the bundles into a shared IR. Your life will not join itself. Compile it into one language.

Start with the inventory almost nobody wants to admit they have:

These have almost nothing technically in common. Different formats, schemas, access methods, query languages, metadata conventions, vendors, eras, and modalities. If your plan is “put a vector index on each and retrieve harder,” you have not solved joinability. You have parallelised search over incompatible residue.

Microsoft’s GraphRAG team has already said the quiet part about baseline RAG: it struggles to connect the dots when answering requires traversing disparate pieces of information through their shared attributes.³ That is not a complaint about your embedding model. That is a complaint about representation. Relationship-shaped questions need a place where relationships can live before the question arrives.

Lower meaning, keep the bronze

The move is not to force every system onto one vendor protocol. The move is:

Call it what it is: semantic homogenisation. The source remains heterogeneous. The bronze layer stays immutable — originals you can re-read, re-export, re-argue with. What becomes homogeneous is the thinking surface: people, projects, events, claims, significance, time, places, edges, and pointers back to the exhibit.

Once they’re wiki pages, the original source class stops controlling the kinds of connections you can make.

That sentence is the whole chapter if you let it land. A chess results page does not know you are copying a multi-terabyte RAID with rsync in 2026. Your email does not know what Samba is. rsync documentation does not know your social life. After homogenisation, those edges can meet — not because search got magically smarter, but because representation difference collapsed.

What homogenisation is not

Why text

We will not re-open the full modality essay here. One line is enough for this frame: language models reason sharpest on serialised meaning in text, so the IR is text-native claims and edges rather than a zoo of binary blobs. The deeper pendulum between code and judgment is covered in Text Is the Model’s Home Turf. Photos stay photos; the wiki holds the semantic twin. Significance versus description is a neighbouring doctrine.

The model did not become smarter

When a personal agent flips from hopeless to useful after a wiki lands, people reach for the wrong explanation. They upgrade the model. They rewrite the system prompt. Occasionally those help. The structural explanation is cruder and more reliable: the world became legible. The same cheap model against a compiled personal IR can suppress noise, notice genuine change, and earn the right to interrupt. That is not anthropomorphic childhood for robots. It is institutional — or personal — prehistory, so the present event is not the first event in the universe.

Search each corpus harder if you must. Just do not confuse high-recall lookup with homogenisation. Homogenisation is a compile step. Retrieval is a runtime convenience over whatever you have already made thinkable.

A production compiler does not ask every optimisation pass to understand C, C++, Rust, Fortran, and Swift as native dialects. It lowers those source languages into an intermediate representation, then works on the IR. That is why systems like GCC and LLVM can accept many front-ends and emit for many targets: the IR is the shared working form.⁴ Compiler courses treat this as layered discipline — each layer stabilises before the next consumes it.⁵

You are doing the same thing to a life. Or a firm. The vocabulary just sounds less respectable until you draw the diagram.

The diagram (definitive placement)

The archive is source. The wiki is IR. The agent is runtime.

That is the doctrine in twelve words. Everything else in this book is how to feed the front-end honestly and how not to lie about what the IR is for.

Why new agents boot smart

The naive personal-AI stack teaches the world to each agent separately:

• Teach the Gmail agent your life
• Then teach MemoryMate your life
• Then teach the CV agent your career
• Then teach the dev agent your frameworks

That is O(agents × world). It does not scale, and it does not stay consistent. The compiler frame flips it: compile Scott IR — or Acme IR — once. Agents become backends. They differ in tools and authority, not in which planet they think they inhabit.

Offline indexing research is pushing the same direction from another angle: shift cross-document reasoning out of every online multi-hop crawl and into the index itself.⁷ Multi-hop RAG that alternates retrieve-and-reason at every step pays a token and latency tax for rediscovering structure you could have compiled once.⁸

Where RAG sits now

This is not a funeral for RAG. It is a demotion with dignity. Exhaustive corpus recall, single-document lookup, and loss-intolerant forensics still earn a high-recall lane over bronze and staging stores. What no longer makes sense is asking query-time similarity to be your worldview. Relationships conjured fresh from chunks on every request are not the same object as edges you can walk, lint, and disagree with.

Put more bluntly:

• RAG is a corpus tool below the worldview
• The wiki is memory agents can navigate
• The raw archive is not the memory either — it is the territory

A durable agent kernel is not a monolithic prompt file you paste into every session. It is a queryable graph you demand-page through edges, with flat context treated as a disposable build output. That is the Wiki-as-Kernel move applied to personal and corporate residue alike.

System prompts are not a past

We still ship agents into production with a thousand-word sermon and a search tool, then act surprised when they are loud, generic, and slightly wicked. They do not fear job loss. They do not carry three years of corridor context. A system prompt that screams “be commercially sensible” without a world that contains discount practice, open questions, and known absences is costume jewellery.

The IR is where known absence becomes a first-class fact. “No formal discount policy exists” is knowledge. A retrieval miss is not. That distinction is why compiled worlds suppress noise: most events are not surprising once a baseline exists. Silence becomes an earned output of the runtime — a consequence we will touch at the end of the book without turning this piece into a perception essay.

An individual Lotus Notes response might say:

As a storage object, it is perfect. As an ingestion unit for an AI, it is garbage. Second option of what? Agreed with whom? In which decision thread? Against which parent document? The meaning does not live in the row. It lives in the tree.

This is the grain mistake almost everyone makes when they first point models at archives: they hand the model a record because the database exports records. Storage grain is not semantic grain. Confusing the two is how you get confident nonsense about half a conversation.

Grouping before parsing

The accidental discovery on a real Notes corpus was almost embarrassingly structural. Instead of feeding responses one by one, reconstruct deterministically:

Suddenly the model has something that can close. Then you reuse the idea on email. Then on coding transcripts keyed by project folder. Then on photo events. Then on video timelines. Same move, different natural key. We will table all five in Chapter 7. Here we name the doctrine.

Semantic closure: the smallest deterministic bundle whose meaning closes.

That is the correct unit to hand the model. Not the storage record. Not a fixed-size chunk that cut the argument in half. Not a random top-k of similar sentences. The smallest bundle you can assemble with cheap deterministic structure such that a competent reader — human or model — can understand what the object is about.

Industry is groping toward this

You can see cousins of the idea in production RAG tooling. Parent-document retrieval indexes small units for recall and returns parent sections for context.⁹ Microsoft’s chunking guidance warns that fixed-size splits are a poor fit where semantic understanding and precise context matter, because relevant material spanning chunks is easy to lose.¹⁰

Closure is not “use bigger chunks.” Bigger chunks still fail when the natural object is a tree, a thread, or a multi-day project. Closure is deterministic reconstruction of the meaning-unit the medium already had — Notes hierarchy and UNIDs, email threading, folder paths, EXIF and face clusters, timestamps between narration and screen.

The anti-pattern and the pattern

record → AI
record → AI
record → AI
record → AI

reconstruct natural meaning-unit
        ↓
give closed bundle to AI once
        ↓
compile significance and edges

Where a natural key already exists, use it before similarity. Soft data can take SQL-shaped discipline: the dev folder name can prove which conversations created the code, not merely which ones resemble it. That is the Soft Join move — provenance over resemblance.

We think this is the only sane front-end for personal and corporate archive compilation. Clever prompts on wrong units are a tax you will keep paying. Close the meaning first. Then ask the model to think.

You are copying more than a terabyte of life with rsync. The tool is ordinary infrastructure until a name flickers: Tridgell. Andrew “Tridge” Tridgell — rsync algorithm, major Samba work, and, in another branch of the same life, author of the chess engine KnightCap. Somewhere nearby in the old Australian computer-chess map: Joel Veness, and a private Java engine called Chompster. A few people in the world doing that work. One of them lived down the road.

Twenty-five years later you are discussing agentic search and world models while the copy runs on Tridge’s algorithm. That is not a cute LinkedIn anecdote. It is a join.

The chain no product contained

rsync documentation does not know your social life. Your email does not know Samba’s history. A chess results page does not know you are restoring a RAID in 2026. Your autobiographical memory does not hold the exact archived citations. The source systems would never make that join. After semantic homogenisation, the geometry becomes rotatable: person axis, software axis, time axis, subject axis, present-context axis — until edges intersect.

That is not “search got better.” That is post-homogenisation proof.

Cross-time coordination, conversationally

What felt freaky in the moment was not a single web hit. It was coordinating people and artefacts across decades of residue in conversational time: resolve three people, correct a spelling, associate engines, place them in one technical community, connect Tridge forward to rsync/Samba and Joel forward into later research lines, reconstruct the old social geometry. Intelligence agencies have always done versions of entity resolution and record linkage. The new capability is that compiled, legible residue makes the same class of reconstruction available without a team of analysts and a year.

We will not invent statistics about how long the CIA would have taken. The shape-statement is enough: what used to require institutional research capacity can now be attempted, imperfectly but usefully, in a single sitting once the world is representable on one surface. Baseline RAG still struggles when the answer is a path across shared attributes rather than a passage in one chunk.³ Structure helps; knowledge-graph style retrieval has been shown to improve multi-hop question answering relative to pure vector RAG in published evaluations.¹¹

You need the edges

The product instinct, once you feel the join, is to build a chatbot that explains your life back to you in paragraphs. Resist it.

You do not need a whole sentence. You need the edges. The mature MemoryMate instinct is less “memory answerer” and more edge surfacer: watch the cognitive neighbourhood you are standing in, place one high-potential edge in peripheral vision, shut up. Recognition does the rest. That is consistent with a life wiki built for cue-and-recognition rather than lecture.

We are not writing the full MemoryMate product spec here. The architectural claim is enough: once the IR exists, the highest-value personal interface may be sparse, contextual, and almost rude in its brevity. The human brain is still the largest private corpus in the system. One correctly timed edge is a time portal into it.

Why this is the flagship proof

Many AI demos show retrieval of a paragraph you already knew how to find. The Tridge chain is different. It demonstrates a join that is:

• Cross-corpus — tools, biography, public history, private engines
• Cross-time — decades between event and reconstruction
• Unowned by any source system — no vendor schema held the full path
• Useful — it changes how you experience the present moment of copying a disk

If your architecture cannot produce joins of this class, you do not have a compiled world. You have searchable tombs. Searchable tombs are better than sealed tombs. They are not a cognitive IR.

One spam email can look vaguely plausible. The subject is almost right. The sender almost matches a pattern you have seen. The body has just enough professional vocabulary to survive a hurried glance. Hand that singleton to a model with a “is this spam?” prompt and you will get a coin flip dressed as prose.

Now hand the model a closed branch: three hundred similar messages, same morphology, same link shapes, same timing cluster, same reply-to nonsense. The judgment stops being clever. It becomes structural. The branch screams.

Grouping changes the epistemic quality of the evidence before the AI thinks.

That sentence is deeper than batching for throughput. Batching is an efficiency story. Bundle-epistemics is a judgment story. You changed what kind of object the model is allowed to see.

Wrong grain, confident nonsense

Record-level classifiers fail for the same reason record-level Notes ingestion fails: the pattern is not in the atom. A single reply that says “agreed, second option” is not a decision. A single marketing email is not a campaign. A single failed login is not an incident. Humans know this instinctively when they open a folder and feel the shape of the pile. We then design AI pipelines that throw the pile away and feed atoms, because atoms are what the export button produces.

We are deliberately not inventing a precision percentage for your spam filter. The doctrinal point does not need one. At the wrong grain, evidence is weak. At the closed grain, evidence is often overwhelming. If your pipeline cannot represent the branch, no amount of model upgrade will invent the branch from isolated rows.

Same front-end as the compiler

Notice that this is not a new framework. It is semantic closure applied to judgment:

• Notes needed the document tree before meaning closed
• Email needed the thread before intent closed
• Spam needed the cluster before malice closed

The compiler metaphor stays honest here. Translation units that are too small produce IR full of ambiguous claims and weak edges. Translation units that close produce IR where significance is cheaper to extract and junk is cheaper to mark. Grouping is not a prettifier for the staging store. It is part of how truth and rubbish become separable.

What to build instead of hero prompts

If you are designing ingestion for a personal or corporate wiki, put engineering effort into deterministic clustering before generative judgment:

1. Identify the natural key or cheap structural glue (thread IDs, reply chains, near-duplicate hashes, time-dense bursts, shared link hosts).
2. Materialise the closed bundle as an object the model can read once.
3. Ask for significance, classification, or edges against the bundle.
4. Write results into the IR with pointers back to bronze members of the branch.

That pipeline is boring in the way good ETL is boring. It is also how you stop paying frontier-model prices to rediscover that three hundred near-identical messages are the same insult to your attention.

Bundle-epistemics is the unglamorous sibling of the Tridge join. One proves that homogenisation creates new true edges. The other proves that closure creates new true negatives. Both are reasons to stop feeding the model naked records and hoping for wisdom.

By the time you have done Notes trees, email threads, project-keyed transcripts, photo events, and timestamped video, you are no longer collecting tricks. You are rediscovering the same doctrine with different natural keys. Name it once. Carry it everywhere.

The closure table (definitive placement)

You’ve independently applied the same pattern five times. That’s a doctrine now.

Natural keys before resemblance

Wherever soft data already carries a natural key, a deterministic join is free provenance. The folder name does not say “these conversations are about this project.” It says they are this project’s conversations. Similarity can still help when keys are missing. It is the fallback, not the default. That is SQL discipline for soft data — the Soft Join rule applied as the first move of every row in the table above.

Refuse the lazy probabilistic substrate when keys exist and a quiet star schema starts to appear on its own: soft records as facts keyed to dimensions you can maintain. That is not enterprise theatre. That is how you stop re-deriving identity with cosine similarity every night.

What goes into the wiki for each unit

Closure gets you the object. Compilation still has to write something agent-readable. For each closed bundle, the IR typically wants:

• Identity — stable page name; cool identifiers beat brittle paths when you can manage them¹²
• Claims — what appears to be true, with uncertainty marked
• Edges — people, projects, events, policies, prior incidents
• Significance — why this might matter (or explicitly unresolved)
• Pointers — back to bronze members of the bundle

For photos, do not put the JPEG in the wiki. Put the semantic twin: Christmas 2012, who was there, what happened, why it might matter, links to the image files as exhibits. Description is often cheap to regenerate; significance and edges are the expensive residue. Neighbouring work on caching significance rather than description owns that nuance; here it is a one-line boundary.

Portability test

When a sixth source appears — Slack exports, calendar dumps, browser histories, CRM notes — do not start a new religion. Ask:

1. What is the smallest deterministic bundle whose meaning closes?
2. What natural key already exists?
3. What bronze pointers must survive compilation?
4. What edges become legal only after this lands in the shared IR?

If you cannot answer those four, you are not ready to “AI the corpus.” You are ready to do front-end compiler work. That is not a demotion. That is the job.

There is a corny line from the old game Manifest that keeps showing up when people first feel a personal wiki: everything’s connected. Corny, and almost right. The mature version is more precise and more useful:

Everything doesn’t need to be in context. Everything needs to be reachable through meaning.

That is the operating rule of a cognitive IR. You are not stuffing thirty years of life into a context window. You are compiling a world in which agents can walk from a present event to the people, projects, policies, and scars that make the event interpretable.

The two-stage discipline

RAG retrieves. The wiki breathes.

A living personal or corporate wiki is not a static dump of summaries. It expands from the world and recompresses into a worldview:

That loop is why the wiki feels different from RAG. RAG retrieves. The wiki breathes. We have written the retrieval substrate argument elsewhere; here the point is simply that compilation is ongoing work, not a one-time migration fantasy.

Runtime consequence (without stealing the next article)

Once agents inherit a dense past, most present events are not surprising. The Gmail agent that became brilliantly silent is not a personality transplant. It earned a baseline of what is normal, who matters, what is already known, and what genuinely changed. Every unnecessary intervention spends trust. When a router that has been quiet for a month says a mesh point needs a physical operator, you act — because silence was the default, not a bug.

How an agent perceives moment-to-moment — the full right-to-interrupt design — is neighbouring territory. This book only needs the dependency: perception quality is downstream of IR quality. You cannot prompt your way out of an uncompiled world.

Failure modes

What you should be able to say after this book

• Heterogeneous archives become joinable only when meaning is compiled into a common text surface.
• Semantic-closure bundles are the translation units; the wiki is the IR; agents are the runtime.
• Source class must stop controlling which connections are legal.
• Grouping changes evidence quality before the model thinks.
• The joins that prove the architecture are the ones no single system owned.
• Everything need not be in context; everything must be reachable through meaning.

Neighbouring doctrines — ML-class claims about learning substrates, modality conversion mechanics, cache-layer choice, agent perception at runtime — remain neighbouring. This piece owns the compiler frame end to end.

Your life will not join itself. Neither will your company’s soft data. Bundle until meaning closes. Lower into one language. Run the agents. Then, when five quiet edges appear in the corner of the screen while rsync copies the past, you will know the IR is alive.