Part Two: The Mesh Finally Ships

In the first memo we established a habit: when the crowd and the truth diverge, the gap is the opportunity, and the way to find it is to ask the second-level question. Not "is this idea good" but "what does this idea require, who actually pays for it, and what would have to change for the payment to clear?"

Today's subject is an idea that the data world fell in love with, married, and quietly divorced within five years, and which is currently being rebuilt at scale by people who, for the most part, have never heard its name. The idea is the data mesh. The people rebuilding it are agentic engineers. Neither party seems aware of the other.

Here is the claim. Data mesh, and especially its streaming variant, was a correct architecture attached to an impossible payment plan. It demanded a set of disciplines (data as a product, contracts at every boundary, governance as executable code, events as the source of truth) that human organizations have demonstrated, repeatedly and expensively, they will not sustain. The discipline wasn't wrong. The discipline was priced in a currency humans don't carry: permanent, unglamorous, daily consistency.

Agent harnesses change the currency. When your producers and consumers are agents rather than people, those same disciplines stop being aspirations a steering committee nags you about and become the minimum conditions under which the system functions at all. A cold agent picking up a task cannot walk over to Brenda's desk and ask what the column means. Either the artifact it receives is self-describing, contracted, and discoverable, or the agent fails, visibly, immediately, every single time. The mesh disciplines aren't enforced in an agent-native system. They're load-bearing.

The rest of this memo is the engineering survey: what the mesh actually said, the autopsy of why it died in committee, what the streaming variant added and why it fared even worse, and then the principle-by-principle mapping showing that a properly built agent harness is a streaming data mesh whether or not its builder intended one. As before: one field's impossible requirement is the other field's default behavior, and almost nobody is holding both halves at once.

Credit where due, precisely. In 2019 Zhamak Dehghani, then at Thoughtworks, published "How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh," followed by the O'Reilly book in 2022. Her diagnosis was sharp and her prescription was sharper, and both deserve to be stated without the vendor gloss that later buried them.

The diagnosis: centralized data architecture fails at scale for organizational reasons, not technical ones. The data lake and the warehouse put all of an enterprise's data in the custody of one central team that understands none of it. Domain knowledge lives at the edges, with the teams who produce the data; the ability to serve that data lives in the center, with a team that's a bottleneck by construction. Every question routed through the center loses context and gains latency. The architecture mirrors the org chart, the org chart strangles the data, and everyone blames the tooling.

The prescription came as four principles, and the wording matters.

One: domain-oriented decentralized data ownership. The team that owns the business domain owns its data. Not "contributes raw exhaust to a central lake for someone else to clean." Owns it, serves it, answers for it.

Two: data as a product. This is the heart and it's the part everyone underestimated. Each domain's data is treated as a real product with real consumers, which means it carries the properties practitioners later compressed into the DATSIS mnemonic: discoverable (registered, findable in a catalog), addressable (a stable endpoint, not a path someone emails you), trustworthy (quality metrics, SLAs, lineage you can inspect), self-describing (schema and semantics travel with the data; no tribal knowledge required to use it), interoperable (conforms to organization-wide standards so products compose), and secure (policy enforced at the product, not at the perimeter). Every failure in the next section is one of those six properties going unpaid.

Three: self-serve data infrastructure as a platform. Domain teams can't each build their own pipelines, catalogs, and access controls; a platform team provides those as paved roads so that publishing a data product is cheap and consuming one is trivial.

Four: federated computational governance. The word doing the work is computational. Standards are set federally, across domains, but they are enforced by code: policy-as-code, automated checks, machine-verified interoperability. Not a council. Not a quarterly review. Code.

It's the same shape we spent part one mapping: composition over centralization, contracts at the boundaries, a common representation so that M producers and N consumers don't need M×N bespoke integrations. Dehghani was prescribing hexagonal architecture at the scale of an entire company's data estate, with data products as the ports. So what happened?

What happened is the most predictable thing in organizational life: everyone adopted the vocabulary and almost nobody paid the bill. The 2021 through 2024 retrospectives read like variations on one theme, and practitioners eventually coined a name for it that I'll happily borrow: data mesh theater.

The failure decomposes cleanly along the four principles. The architecture didn't fail at a weak point. Each pillar failed independently, in the same way, for the same reason.

Data-as-product became data-as-rename. Treating data as a product means product management: an owner, a roadmap, documentation kept current, quality metrics published, SLAs honored, consumers interviewed. That is a standing cost, paid forever, by engineers whose performance reviews reward shipped features and not maintained metadata. So enterprises did what incentive gradients dictate: they relabeled existing tables "data products," stood up a catalog that became an inventory of stale entries rather than a marketplace of maintained products, and called it transformation. The self-describing property, the one that makes a product usable without walking over to its producer, was the first thing sacrificed, because documentation is the most deferrable work in software and humans defer what's deferrable. Every time.

Federated computational governance became federated governance committees. Strip the word "computational" out of principle four and what remains is a meeting. That is precisely what most enterprises built: councils that defined policies in slide decks, reviewed exceptions by hand, and met monthly to discuss compliance with standards no machine ever checked. Governance, which Dehghani designed as an enabling layer of executable policy, calcified into the bottleneck it was meant to replace, now with extra steps and a Confluence space.

The self-serve platform was rushed or skipped. Building paved roads is expensive, unglamorous, and shows no business value in quarter one, so organizations reorganized the people before building the platform. Domain teams were declared owners of data they had no tooling to serve. The central data team kept building every pipeline, exactly as before, while the org chart claimed otherwise. Hub-and-spoke in reality, mesh on slides.

And Conway's law collected its usual fee. Systems mirror the organizations that build them. An enterprise whose actual communication structure is centralized will reproduce centralization no matter what architecture the deck proclaims, because the architecture that ships is the org chart, not the aspiration. Domain teams without data engineers can't own data products. Central teams with all the engineers won't give them up. The mesh asked organizations to be something they weren't, and organizations, with great reliability, declined.

None of these are refutations of the architecture. Every postmortem says some version of the same sentence: the principles were right and we couldn't sustain the practice. The mesh failed the way a gym membership fails. Not because exercise doesn't work, but because the payment plan was denominated in daily human discipline, which is the single most reliably defaulted currency in the known universe.

Then the streaming community looked at an architecture enterprises couldn't sustain in batch and said: what if we also made it real time?

I'm being unkind to a body of work I genuinely admire, so let me state the substance first. Hubert Dulay and Stephen Mooney's Streaming Data Mesh (O'Reilly, 2023) and the broader Kafka and Flink community transposed Dehghani's logical architecture onto event streams, and the transposition is intellectually superior to the original in nearly every way. The data product stops being a table refreshed nightly and becomes a live stream or a continuously maintained materialized view. The contract stops being a wiki page and becomes a schema in a registry, machine-enforced at produce time, with evolution rules that break the build instead of breaking the consumer. Coordination stops being orchestration by a central brain and becomes choreography: domains publish events, consumers subscribe and react, nobody files a ticket. And the wall between operational and analytical data, the wall the warehouse world treats as geography, starts to dissolve, because a stream is equally consumable by a dashboard, a microservice, or whatever comes next.

If you spent years studying this, as our resident data engineer did, you were studying the right thing. The streaming mesh is what the batch mesh wanted to be when it grew up. Schemas as binding contracts are DATSIS with teeth. Choreography is domain autonomy made mechanical. Events as the substrate means the system records what happened rather than someone's quarterly summary of what happened.

Adoption, however, was even thinner than the batch mesh's, and the reason is arithmetic. The streaming variant kept every organizational cost of the original (data-as-product discipline, federated governance, domain ownership against the grain of Conway's law) and added a technical bill on top: event modeling, schema evolution under compatibility rules, exactly-once semantics, stateful stream processing, backpressure, 24/7 operational SLAs where batch tolerated a failed nightly job with a morning fix. Flink is a powerful tool and a demanding tenant. Most enterprises looked at the sum, used Kafka as a fancy ingestion pipe into the same old central warehouse, and kept the mesh on the conference slide where it was cheaper. The discipline price went up, the discipline budget didn't, and the gap between the abstract architecture and any deployed instance of it widened into the thing you noticed from inside the field: years of literature, almost no practitioners. An architecture mostly made of paper.

So we arrive at the impasse. The streaming data mesh is the correct shape for an organization's information. It requires permanent contract discipline, permanent documentation discipline, permanent event-capture discipline, and governance that executes rather than convenes. Humans have declined to pay for this, twice, at industry scale, in both batch and streaming denominations. A rational observer in 2025 files the whole thing under "right too early" and moves on.

And then, without ceremony, the payer changed.

Watch what happens to each impossible demand the moment the producers and consumers of work stop being people and start being agents.

Start with the deepest one, the property whose absence killed more mesh implementations than any other: self-describing. Why did human organizations never sustain it? Because humans have a workaround. The undocumented dataset, the tribal-knowledge schema, the "ask Brenda, she built it" column: these are survivable in a human organization because humans share an office, a Slack, a lunch table. Institutional memory papers over every gap the documentation leaves. The workaround is so cheap and so constant that the discipline never feels necessary, so it never gets funded, so the catalog rots.

Now hand the same work to a cold agent. An agent spun up in a sandbox to execute one task has no office, no Slack, no Brenda. It has exactly what's in its context window: the issue it was handed, the spec that issue links to, the schemas of the data it touches. There is no tribal knowledge because there is no tribe. If the artifact it receives is not self-describing, the agent doesn't perform slightly worse the way an annoyed human would. It fails, completely, immediately, and at whatever scale you've deployed it. The harness engineering world has a phrase for the discipline this forces: the cold-start contract. If the spec is written correctly, with the right context, skills, constraints, and acceptance criteria attached, the worker needs no mid-task help; and mid-task ambiguity is treated as a spec failure, not an agent failure. That is the data-as-a-product principle, verbatim, wearing different clothes. Design the artifact around the consumer who cannot ask follow-up questions. Dehghani asked enterprises to please do this. The harness makes it the difference between a system that runs and a system that doesn't.

This is the general pattern, and it repeats at every layer: the mesh disciplines were unenforceable for humans because humans have workarounds, and they are mandatory for agents because agents don't. A human tolerates an ambiguous schema; an agent needs the typed shape or it hallucinates one, which is worse. A human remembers what the team decided last sprint; an ephemeral agent is destroyed at task end, so every decision it made is lost unless the system captures it as a durable event. A human can be governed by norms, shame, and the fear of looking sloppy in review; an agent is shameless, so the only governance that exists for it at all is the kind that executes: a hook that blocks, a CI gate that fails, a schema check that rejects. Federated computational governance was the mesh's most ignored word pair. For agents it isn't a principle. It's the only mechanism on the menu.

There's a name for this in the harness literature, born from a postmortem rather than a whiteboard: the mechanical-owner-loop law. The prior generation of harness, the one built on shared markdown files and good intentions, didn't fail on bad ideas. It failed because artifacts were created faster than the loop that keeps them consistent: catalogs frozen on day one while the things they cataloged grew past them, enforcement hooks running in log-only mode that never blocked anything, rule files drifting into three diverging copies. Each artifact looked like progress; in aggregate they formed a layer that claimed more completeness than it had, which is the most dangerous state, because agents trust it. The law that fell out: every artifact class needs a mechanical owner-loop, a CI step, a blocking hook, a merge driver, and never human discipline. Stale catalog, unenforced governance, diverging copies of what should be canonical: it's the same disease. The harness builders caught it for the same reason enterprises didn't: their consumers were agents, so the rot produced loud failures in days instead of quiet failures over years. Short feedback loops are how disciplines actually get adopted. The mesh's feedback loop was a quarterly steering committee. The harness's feedback loop is the next agent run, which is to say, minutes.

Claims this large deserve a mapping you can check line by line. Take the four principles and walk them through a working agent harness, the kind currently being built in production: a strategist model writing specs, an execution fleet pulling issues from a work queue, specialized agents bounded by tech-stack reality, verification gates on every merge. Watch each principle appear, not as an aspiration but as a structural component the system cannot run without.

Domain ownership becomes agent specialization, and the boundary logic is identical. The mesh said: the team closest to the domain owns the domain's data, because context lives at the edge. The harness says: many specialized agents beat one generalist, every time, in every industry; give each agent one purpose, a focused context window, and tell it what to look for and, just as deliberately, what to ignore. A backend agent that owns pipelines and APIs, a frontend agent that owns the component layer, a DevOps agent that owns CI and deploys: these are domain teams. And the boundary criterion is the mature one. Not walls, since overlap is constant and expected, but specific technologies, patterns, and architectures each domain interfaces with, with shared primitives underneath. That's Dehghani's domain decomposition, restated by people optimizing context windows instead of org charts. They landed on the same answer because it's the same problem: context degrades with distance from the work. Cloudflare's AI code-review factory runs seven specialized reviewer agents (security, performance, code quality, documentation, release, compliance) under one coordinator. That is a mesh of review domains, each owning its lane, federated by a fusion layer. Nobody on that team calls it a data mesh.

Data as a product becomes the artifact contract, and DATSIS reads like a harness spec sheet. Go down the six attributes against the artifacts a real harness passes between agents: the four-layer work issue, the linked spec, the typed records the work produces. Discoverable: the artifact registry, the catalog meta-skill that distributes skills and agents across every repo from one canonical source, with a sync check in CI that fails when disk and catalog diverge (the mechanical owner-loop, again, because the first version of that catalog froze on day one exactly like an enterprise data catalog). Addressable: every unit of work is an issue with a stable identifier, linked to a spec at a stable URL, producing a PR on a deterministically named branch. Trustworthy: the verification ladder, where a change isn't done when code exists but when an observability trace shows real data flowing in the deployed system and a screenshot proves the rendered surface, captured by a separate QC agent because the implementer is the suspect, not the witness. SLAs enforced by a gate with teeth, not published on a dashboard nobody reads. Self-describing: the cold-start contract from section V; the issue carries its goal, its boundaries, its acceptance criteria, its failure modes, because its consumer cannot ask. Interoperable: one shared intermediate representation for data shapes across the whole system (typed schema models as the lingua franca, converted at each boundary to the local dialect), which is part one's IR thesis doing mesh duty. Secure: scoped permissions per agent, glob-bounded file access, allowlisted tools, no production credentials in the blast radius of a worker that might do something creative. Six for six. The data product didn't get adopted by the data org. It got reinvented as the only artifact format that survives contact with an agent fleet.

The self-serve platform becomes the harness itself, and this one is almost too literal. The mesh demanded paved roads so domain teams could publish and consume without filing tickets to a central team. The harness IS paved roads; that's what the word means. The shared spine every agent inherits: identity and discipline floors, a research primitive, a work-queue primitive, a memory primitive (query before acting, write an episode after acting), a code-navigation primitive that reads a precomputed AST graph instead of grepping raw files. Stripe's version is instructive because of its scale: warm developer sandboxes that spin up in ten seconds preloaded with code and services, a "tool shed" meta-service that lets agents discover and load from hundreds of tools without drowning in them, and a blueprint engine that interleaves deterministic code with agent steps so neither does the other's job. Look at the tool shed. It's a data product catalog for capabilities. Look at the dev box pool. It's self-serve infrastructure so that spinning up a new producer is cheap. The enterprises that skipped platform investment got mesh-on-slides; the harness builders who skipped it got agents that couldn't complete tasks, found out the same afternoon, and built the platform. Feedback loops, again. The difference was never wisdom. It was how fast the bill arrived.

Federated computational governance becomes hooks, gates, and validators, which is to say: it finally becomes itself. This is the principle the enterprises hollowed out into committees and the one the harness world implements most completely, because it has no alternative. Pre-tool-use hooks that block a forbidden action before it executes. Post-edit linters that run on every file touched. Schema validation at every boundary, because the typed model is the contract and the contract throws. A brief validator wired into CI so that a malformed work order never reaches an agent. PR rules and CODEOWNERS as the merge-time constitution. The QC gate as a closed loop: findings route back to the implementer, the loop runs until clean, and the reviewer's evidence is itself audited, because a governance layer you don't verify is a governance layer you don't have. Federated, because each domain harness carries its own overlays and validators. Computational, because not one element of it convenes a meeting. The mesh's most violated principle turns out to be the easiest one to build when the governed parties are processes instead of people with feelings about being blocked.

Everything in section VI you could build, in principle, as a batch mesh. What forces the streaming variant, the harder and better one, is a property of agents so basic it's easy to miss: they don't persist. The execution agent that shipped this morning's feature is gone by this afternoon. Its context window, its reasoning, its discoveries: destroyed at task end, by design, because ephemerality is what makes fleets cheap and parallel. Which means an agent-native system has a choice between exactly two states: every meaningful occurrence is captured as a durable event at the moment it happens, or the system as a whole knows nothing about its own history.

So the harness becomes event-sourced, not as an architectural preference but as a survival trait. Walk the loop and count the streams. A webhook fires when an issue is assigned: an event, consumed by a runner, which dispatches a worker. That's choreography, the streaming mesh's coordination model; no central brain polls for work, domains react to published events. The worker posts its decisions to the issue's comment thread as it goes: a verbatim, append-only audit stream, deliberately kept so that even an ephemeral agent leaves a permanent record of what it did and how it decided. It writes a distilled episode to the shared memory graph after acting: an event ingested into a bi-temporal knowledge store, where every fact carries the time it was true and the time it was recorded, and traces back to the episode that produced it. Bi-temporality is a luxury feature in enterprise data systems and table stakes here, for the same reason as everything else: an agent reasoning over stale facts with no validity window produces confident decisions against a fiction, and nobody is at the desk to catch it. The PR, the CI run, the QC verdict, the merge transition: events, events, events, each one both a record and a trigger for the next consumer in the chain.

And presiding over it all, the harness world's bluntest law: if it's not in the trace, it didn't happen. Observability spans aren't monitoring in this regime, a thing you check when something smells wrong. They're the epistemology. The QC agent doesn't trust the implementer's claim that the code ran; it trusts the span showing real data moving through the deployed path, because agent self-reports are testimony and events are evidence. The streaming mesh told enterprises that events should be the source of truth and summaries should be derived. Enterprises mostly filed that under "nice someday." A fleet of amnesiac workers makes it the only design that produces a system anyone can trust, and trust, as the harness people keep saying, is the actual product. The streaming data mesh's strangest property is that its most natural habitat turned out to be not the Fortune 500 data estate it was designed for, but a workshop of forgetful robots that cannot function without it.

One more layer, because architecture without economics is poetry, and we're not running a poetry operation.

The mesh's standing costs were never mysterious: documentation kept current, contracts maintained under evolution rules, quality gates run on everything, lineage captured, catalogs synced. Enterprises declined these costs because, performed by senior engineers, they're brutally expensive and invisible in the quarter they're paid. The part of the story that changed most recently and most decisively: those exact line items are what agents are cheapest at. Cloudflare runs a coordinated team of specialized review agents over every merge request, 130,000 reviews a month across 5,000 repositories, at a median cost of one dollar per review. One dollar, for the category of disciplined, repetitive, judgment-adjacent verification work that the mesh's whole payment plan was denominated in. The discipline didn't get cheaper because anyone negotiated. The labor performing it changed species.

And note what the cost-discipline relationship does in an agent fleet: it inverts. For humans, economizing meant skipping the discipline (skip the docs, skip the contract, ship the feature). For agents, the discipline IS the economizing. The same tokenomics that demand a tiered model stack (don't send the dream team to review a typo: trivial diffs get two agents, full pipelines get seven) also demand scoped context, which forces self-describing artifacts; typed contracts, because malformed handoffs burn retries; precomputed code graphs, because re-reading raw files on every task is a token tax; and risk-tiered verification, because review compute should follow blast radius. Every cost-saving move in harness engineering tightens a mesh discipline. Every corner the enterprise cut to save money, the fleet operator un-cuts to save money. Same ledger, opposite gradients, and the gradient is the whole game. You don't get sustained discipline from values statements. You get it when the cheap path and the disciplined path are the same path.

The data mesh was a correct architecture that failed for one reason: every one of its principles is a standing discipline, and human organizations do not sustain standing disciplines whose violation has no immediate cost. The streaming data mesh sharpened the architecture and raised the discipline price, so it was adopted even less. Agent-native infrastructure inverts the failure condition. Agents have no tribal knowledge, so artifacts must be self-describing products. Agents are ephemeral, so all state must be captured as events at the moment of occurrence. Agents are shameless, so governance must be computational or it does not exist. Agents are cheap, so the disciplines that were unaffordable as human labor become the system's least expensive components, and token economics actively reward tightening them. Therefore an agent harness, built strictly to its own requirements and nothing else, converges on domain ownership, data as a product, a self-serve platform, federated computational governance, and an event-sourced substrate. Which is to say: build the harness correctly and you have built a streaming data mesh, whether or not you meant to. The mesh was never wrong. It was waiting for a workforce whose existence depends on it.

For the practitioner, three positions fall out. First: those years studying streaming data mesh weren't a detour through abstract literature; they were an advance copy of the spec your current field is now implementing under other names, which makes you one of very few people holding both vocabularies, and translation between fields that haven't met is among the most durable forms of alpha there is. Second: when you design harness artifacts (specs, issues, episodes, schemas), you can steal with both hands from a decade of mesh literature on data product design, contract evolution, and federated governance, because it's the same problem with better prior art than the harness world knows it has. Third: when a senior architect tells you data mesh is dead, you can agree precisely, the way we agreed about OOP in part one: dead as an enterprise change program sold to committees, alive as the emergent structure of every serious agent fleet currently shipping.

And the through-line from part one completes itself. That memo ended with six communities discovering one shape: pure decisions in the middle, managed state at the edges, messages in between, a common representation so the parts can change without the whole caring. This memo adds the seventh discovery and the macro scale. Kay's objects, Bernhardt's cores, Cockburn's hexagons, Erlang's actors, the game industry's ECS, LLVM's IR, and now Dehghani's mesh: the same architecture, found again, this time spanning an entire organization's information, and finally buildable because the builders are no longer the bottleneck. The compiler thesis goes all the way up. Domains are modules. Data products are the IR at every boundary. The harness is the toolchain. And the build, for the first time since 2019, actually runs.

Be skeptical of convergence stories, including this one; the universe owes us no isomorphisms. But when an architecture fails twice for clearly diagnosed organizational reasons, and a new substrate arrives that dissolves precisely those reasons, and independent practitioners under pure economic pressure keep rebuilding the failed architecture without knowing its name, the second-level conclusion writes itself. The mesh finally ships. It just needed workers who can't survive without it.

Sources worth your time: Zhamak Dehghani, "How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh" (2019) and "Data Mesh" (O'Reilly, 2022), for the four principles and the data product definition from the source. Hubert Dulay and Stephen Mooney, "Streaming Data Mesh" (O'Reilly, 2023), for streams as products and registry-enforced contracts. The post-2021 mesh retrospectives (Monte Carlo, DataHub, the Kafka-community implementation literature) for the documented failure patterns: rebranded lakes, governance-by-committee, skipped platform investment, Conway's law. Cloudflare's engineering blog on their AI code-review system, for the seven-reviewer factory and the dollar-per-review economics. Stripe's engineering blog on minions, for dev-box pools, the blueprint engine, and the tool shed. And part one of this series, for the compiler architecture this memo scales up.