The thesis
Most content systems treat each piece of content as a string. The string goes into a model, the model returns a new string, the new string replaces the old one. When the user wants a revision, the system regenerates the whole thing. This is wasteful, slow, and uncacheable. It is also the architecture every off-the-shelf "AI writing tool" ships with, which is why those tools cap out at the same plateau of quality and run hot on the token bill.
The shape that breaks past the plateau is to stop treating content as a string and start treating it as an intermediate representation. Compose the document out of typed atomic nodes: a hook node, a section node, a claim node, a citation node, a CTA node. Each atom has a contract (what arguments it takes, what it produces). Each atom is rendered in isolation, optimized in isolation, cached in isolation. The system that emerges looks more like a compiler with optimization passes than a writer with a prompt.
Why string-based content systems lose
The first failure of string-based systems is revision economics. The user changes one paragraph. The model regenerates the whole document because the whole document was the input. The token cost of that revision is the same as the original generation. On a 10,000-word piece, that is 30,000 tokens of completion plus some multiple of that as input context, every time someone fixes a typo. Multiply that by the number of pieces in production and the bill becomes the line item that kills the project.
The second failure is structural drift. Each regeneration is a fresh roll. The model that generates the new version does not necessarily produce the same structure as the previous version. A section that was three paragraphs becomes four. A bullet list becomes prose. The downstream consumers (the headline, the meta description, the social-card excerpt) expect the previous structure, so each regeneration breaks something downstream that has to be patched. The patches are silent corruption.
The third failure is voice instability. Without a typed contract per atom, the prompt has to do all the voice work in one pass. The result is a generation that is plausible at the macro level and mediocre at the micro level. The hook is fine, the close is fine, the four paragraphs in the middle drift in register because the prompt could not enforce per-section register without explicit structure to hang it on. The reader feels the drift even when they cannot name it.
The IR: typed atomic nodes
The intermediate representation is a Pydantic V2 model tree. The root is a Document. The Document has a register, a target audience, a length budget, and an ordered list of Block nodes. A Block is a discriminated union: HookBlock, NarrativeBlock, EvidenceBlock, ClaimBlock, ListBlock, CTABlock, FigureBlock. Each Block has its own typed fields. A NarrativeBlock has a topic, a claim, a target paragraph count, an array of supporting evidence references, and the rendered prose. An EvidenceBlock has a source citation, a quoted span, an optional commentary, and the rendered prose.
The trick is that the rendered prose is a derived field. It is produced by a pass that takes the structured fields and emits the natural-language version. The structured fields are the source of truth. If the user changes the claim of a NarrativeBlock, only that block is re-rendered. The blocks above and below are untouched. The token cost of the revision is the cost of re-rendering one block, which is roughly two to four percent of the cost of the full document. This is the source of the "ninety percent token reduction" line.
The other affordance the IR enables is composability. The same Document can be emitted as a Wiki article, a Field Notes entry, an email, a tweet thread, or a video script, by changing the emit pass while leaving the IR untouched. This is the same trick LLVM plays with backends: one IR, many targets. Same content, six surfaces, no parallel content team to maintain six versions.
The optimization passes
A pass is a function that takes the IR and returns a modified IR. Passes are idempotent: running the same pass twice produces the same result as running it once. Passes are pure: they do not mutate state outside the IR. Passes are composable: the order in which they run is configuration, not code. The system runs a default pipeline of passes in a known order and emits the final IR; the operator can override the pipeline for specific documents or specific surfaces.
The passes I run in production cover four categories. Structure passes ensure the document has the right block sequence for its register and audience (every Default-register Wiki article has a thesis block early, every Casual-register Field Notes entry has a hook block first). Voice passes apply the writing style guide rules at the block level (kill em dashes, kill "not X. it's Y," kill performative announcements). Citation passes verify that every EvidenceBlock has a resolvable citation and that every claim has supporting evidence somewhere in the document. Render passes produce the final natural-language prose from the structured fields.
Each pass logs its work as a LogFire span. When a pass changes a block, the span captures the before-and-after of the block. When a pass rejects a block (an EvidenceBlock with an unresolvable citation, a NarrativeBlock missing its claim), the span captures the rejection reason. The downstream effect is a compile log per document that the operator can read to understand what the system actually did. This is the same affordance a compiler's verbose flag gives you: visible reasoning, not invisible magic.
Caching at the atom level
Because each block is rendered in isolation, each block can be cached. The cache key is a hash of the block's structured fields plus the model version plus the prompt version. If a block's structured fields have not changed and the model and prompt have not changed, the rendered prose is cached and reused. The first generation populates the cache; subsequent runs hit the cache for unchanged blocks and only re-render the blocks that changed.
The cache lives in Convex (per the project's data layer). Cache entries are typed Pydantic models with a TTL and a hit count. The hit count is its own observability signal: a block that is regenerated frequently is a block where the structured fields are unstable, which is a signal that the block's contract is wrong and needs revision. The cache is not just a performance optimization; it is also an instrument that tells the operator where the system is wasting effort.
When the model version or the prompt version changes, the cache invalidates the affected entries. The operator can run a backfill to regenerate the invalidated entries during off-peak hours, then deploy the new version with a warm cache. This is identical to the way a compiler invalidates and rebuilds objects when the compiler version or the source changes. The pattern is borrowed because it works.
When the compiler is overkill
The Content Compiler architecture earns its keep when content volume is high (dozens of pieces per week or more), revisions are frequent (each piece touched five or more times before publish), and the surfaces are multiple (the same content needs to render to Wiki and Field Notes and email and social). Below those thresholds, the engineering investment is more expensive than the token savings it produces. A small content operation with five pieces a month and a single surface should run a string-based system with a good prompt; the compiler is the wrong abstraction at that scale.
The pattern I see most often in client work is operators trying to scale a string-based system past the volume threshold and watching the token bill double every quarter while quality plateaus. The fix is not a bigger model or a smarter prompt. The fix is recognizing that the system has outgrown its abstraction and rebuilding around a typed IR. The rebuild is a serious project (six to eight weeks of structured work) but the per-piece cost drops by an order of magnitude and quality compounds because the voice and structure passes can be improved independently of the content.
What this gets you
A content compiler ships a content engine that scales sublinearly with volume. The hundredth piece is not ten times the cost of the tenth piece. The thousandth piece is not a hundred times the cost. The economics work because the structural work is paid once and amortized across the corpus.
If you are evaluating a content vendor or an internal team, the questions are about the IR, the passes, the cache strategy, the per-block voice contracts, and the multi-surface emit. If they show you a prompt and tell you the secret is the prompt, what they have built is not durable. If they show you a typed schema and walk you through the passes, what they have built will compound.