I keep seeing a version of this question. Sometimes it’s a conference talk title, sometimes a LinkedIn post, occasionally a genuine question from someone I respect who is trying to work out whether their twenty years of experience still count for anything.
Is system design changing?
The honest answer is that the question is framed in a way that guarantees a bad answer, because it bundles three separate questions with three different answers. Untangle them and most of the argument dissolves.
- Is the practice of building software changing? Yes, substantially.
- Are the systems we build changing? Yes. There’s a new material with genuinely awkward properties.
- Are the principles of system design changing? Barely. And this is where almost all the noise is.
Below is my attempt at each, plus the two things I do think are genuinely new, which are not the two things the discourse focuses on.
I should say up front where I land, so you can decide whether to keep reading: system design isn’t changing. The system changed, so design has more to do. Same discipline, new material, harder properties.
The practice really is changing
I want to concede this first and properly, because I think the “nothing has changed” camp is as wrong as the “everything has changed” camp, just less annoying about it.
My own development loop is not what it was eighteen months ago. I write specifications more than I write code. The specification is a real artifact now, versioned in the repo, reviewed like code, because it’s the thing that determines output quality. The code review has moved earlier and become more about intent than syntax. I read more code than I write, and I read it more suspiciously.
Three second-order effects I did not expect:
The optimal amount of abstraction went down. This one surprised me. Classical instinct says: notice the duplication, extract the abstraction. But abstraction is a bet that you understand the shape of the future. When regenerating code is nearly free, a wrong abstraction costs more than duplication does, because duplication is trivially deletable and a wrong abstraction is load-bearing everywhere. Sandi Metz’s “duplication is far cheaper than the wrong abstraction” was always true; it just got cheaper to act on.
The optimal amount of specification went up. Correspondingly. The leverage moved from “how do I express this cleanly” to “how do I state precisely what I want and how I’ll know it’s right.” I now write documents whose entire purpose is to constrain a code agent, listing what it must never do, what order to work in, and what to flag rather than fix. That document is the highest-leverage file in the repository.
Verification became the bottleneck. When producing a candidate solution is cheap and confirming it is expensive, the whole economics of the loop inverts. This is not a new observation in computer science, but living it daily is different from knowing it.
None of that is system design. It’s software engineering practice, and it’s changing fast. Worth separating from the rest.
What is genuinely new about the systems
There is one driver, and it is not any of the terms that have been invented for it.
We now routinely place a component in the critical path that is nondeterministic, priced per invocation, degrades with input size in non-obvious ways, and whose capabilities shift under us on someone else’s schedule.
Every one of those four properties is individually familiar. Nondeterminism we know from networks and from concurrency. Per-invocation pricing we know from cloud. Degradation under load we know from every queue ever built. External capability drift we know from every third-party API.
What’s new is the conjunction, and specifically that the component with all four properties is the reasoning core rather than a peripheral. We have decades of practice pushing unreliable things to the edges of a system and keeping the decision-making part deterministic. This inverts that.
A few consequences follow directly.
The cost model moved. Classical design optimises compute and storage, both cheap and predictable. Agent systems are dominated by per-token inference that scales with how much context you resend. In a controlled study I ran recently, output was 2.92% of all tokens consumed across 160 agent runs. You are not paying the model to think. You are paying it to re-read what you already sent it. That single fact reorganises what “efficient” means.
Testing moved from build time to runtime. Classically you test, then deploy, and the deployed artifact behaves as tested. An agent behaves differently on every run, so the check has to live inside the system. Self-verification stops being a quality practice and becomes an architectural component with a latency and cost budget.
The unit of design shifted from pipeline to environment. You increasingly do not specify a sequence of steps. You specify what an agent can perceive and what it can affect, then constrain it. That’s closer to designing an RL environment, or an OS sandbox, than to designing a DAG. The loop itself is often a boring while statement; all the complexity lives in what the loop manages.
That’s a real shift in emphasis. It is not a shift in principle, which brings me to the part I actually want to argue.
What is not changing, and the mapping that shows it
Take the vocabulary that has grown up around this and map it backwards.
| The new term | What it already was |
|---|---|
| Context engineering | Cache eviction policy, plus information retrieval |
| Loop engineering | Control flow with a slow, expensive, unreliable oracle |
| Graph engineering | Dataflow, rediscovered |
| Compaction | Lossy compression with a retention policy |
| Subagent isolation | Bulkheads and process isolation |
| The agent ledger | Event sourcing, write-ahead logging |
| Tool schemas | API contract design |
| Permission gates | Authorization, entirely unchanged |
| Just-in-time retrieval | Lazy loading |
| The harness | The environment boundary from RL, or the OS sandbox |
| Fault injection for agents | Chaos engineering |
| Halting conditions | Termination analysis, plus circuit breakers |
Not one of these required new theory. What they required was rediscovery by a community that largely did not arrive with that background. A great many people building agent systems in 2026 came in through data science, ML research, or product, not through distributed systems. They are hitting problems that operating systems people solved in the 1970s and distributed systems people solved in the 1990s, and inventing fresh names for them, because that is what happens when a field grows faster than its ability to transmit its own prior art.
I want to be careful here, because this reads as sneering and I don’t mean it that way. Rediscovery is not stupidity. The problems genuinely do look different at first contact, and translating “cache eviction” into “what do I drop from this context window” takes real work. But the theory transfers wholesale, and pretending otherwise means teams relearn expensive lessons that were already written down.
The stronger version of this argument
Mapping vocabulary to vocabulary is a bit cheap, because vocabulary is where the hype lives and it’s easy to deflate. The harder test is to take the actual components of an agent system, one at a time, and ask what each already was. I’ve been describing agent harnesses as having ten primitives. Here is that list against classical building blocks.
| # | Harness primitive | What it already was | Fit |
|---|---|---|---|
| 01 | Loop | Control loop, workflow engine, scheduler | Clean |
| 02 | Durable state | Event sourcing, write-ahead log, session store | Identical |
| 03 | Tools | API contract design, RPC interface, service discovery | Clean |
| 04 | Context assembly | Working set and paging, plus IR relevance ranking | One real disanalogy |
| 05 | Memory | Change data capture, materialised views, cache invalidation | Clean |
| 06 | Verification | Runtime assertions, design by contract, the end-to-end argument | Clean |
| 07 | Permissions | RBAC, capability security, four-eyes approval | Identical |
| 08 | Sandbox | Process isolation, containers, least privilege | Identical |
| 09 | Telemetry | Distributed tracing, audit logging, query plan instrumentation | Scope changed |
| 10 | Limits | Admission control, backpressure, circuit breakers, quotas | Identical |
Four of these are not analogous to classical building blocks. They are classical building blocks, unmodified. You would implement durable state, permissions, sandboxing and rate limiting the same way, with the same libraries, and a distributed systems engineer would recognise them without being told the context.
Two resist a clean mapping, and the ways they resist are more instructive than the eight that don’t.
Context assembly has one genuine disanalogy. The paging model fits well at first: a small expensive resident tier, a large cheap backing store, an eviction policy, page faults as retrieval calls, thrashing as context churn. It breaks on one point. In classical paging, keeping more pages resident never makes correctness worse; it only costs memory. Context rot breaks that. Adding more relevant material actively degrades the model’s ability to use any of it.
The closer analogue is from information retrieval, where precision degrades as you expand a result set. That reframing has practical teeth: it means your context budgeter should be evaluated like a retrieval system, on precision at k, rather than on whether everything fit. That is not how most people build them.
Telemetry is where I have to revise a claim I make later in this essay, and I would rather do it here than have you catch it. That’s the next section.
The test that separates a real mapping from a cute one: does the classical failure taxonomy transfer? For limits, permissions, sandbox and state it transfers wholesale. For context it transfers partially (thrashing yes, correctness degradation no). A mapping whose failure modes don’t come with it is decoration.
One caveat on all of this. A mapping being available is not the same as the engineering transferring. “Context management is like paging” is true and nearly useless on its own, because you cannot reuse an LRU implementation. What you reuse is the framing: there is a resident set, a policy, a cost of eviction, and a measurable hit rate. The framing is worth a great deal. The code is worth nothing.
The fallacies of distributed computing are all still fallacies. The network is not reliable, latency is not zero, bandwidth is not infinite, and the topology does change. An agent calling tools over a network is a distributed system with worse latency and a stranger failure mode. Idempotency, backpressure, timeouts, bulkheads, circuit breakers, and graceful degradation are all still exactly what separates a system that survives contact with production from one that does not.
The causal inversion
Here’s the thing I’d most like people to stop doing.
The question usually arrives as: given harness engineering, context engineering, graph engineering and so on, is system design changing? That treats the vocabulary as the driver.
It is not. The vocabulary is a consequence. There is one driver, stated above, and every one of those terms is a downstream effect of it. Harness engineering exists because a stateless reasoner needs an environment. Context engineering exists because attention is a metered resource that degrades. Graph engineering exists because control flow with an unreliable oracle needs explicit structure.
Get the causality backwards and you end up doing what a lot of the discourse does: treating each new term as independent evidence of a paradigm shift, when they are all the same observation refracted. Six words for one phenomenon is not six phenomena.
There’s also a commercial gradient here worth naming without being cynical about it. Every vendor in this space benefits from the layer they own having a distinct name and appearing novel. That is not a conspiracy; it’s just how naming works when there’s money attached. It does mean you should apply a discount to any taxonomy published by someone selling a product that sits inside it, including the ones I find genuinely useful.
Two things I think are actually new
Having spent most of this essay deflating, here is where I think the discontinuity is real. Neither is what the discourse focuses on.
1. Architectural decisions now have an externally set expiry date
Classical architecture assumes your dependencies change on a schedule you can negotiate. Library versions, deprecation notices, migration windows. You get warning, and often a choice about when.
Agent architecture doesn’t work that way, because a substantial fraction of what you build exists purely to compensate for model deficiencies that vendors are actively working to remove.
Every component encodes an assumption about what the model can’t do on its own. You built an explicit planner because the model couldn’t decompose tasks. Then it could, natively and better, and your planner became dead weight that fights the model and adds latency. You built a verifier pass because it wouldn’t check its own work. Then it did, and you’re paying twice for the same check. You built an aggressive summariser because it forgot across turns. Then effective context got longer, and your summariser became the lossy step in an otherwise complete pipeline.
I don’t think classical practice has a good pattern for components whose obsolescence is scheduled by a third party on a six-month cycle. The nearest analogue I can think of is hardware workarounds that outlive the hardware bug, and even those ran on a timescale of years with a public erratum.
The practical version of this is cheap and almost nobody does it: give every component in your system a dated, explicit statement of which model deficiency it compensates for. One comment. “This exists because the model does not reliably do X, added 2026-08.” Then review that list on every model upgrade, and make retirement cheap. The components you cannot delete are the ones that will hurt you.
In a regulated context this doubles as a governance artifact, because “why does this control exist and is it still load-bearing” is precisely the question a validation reviewer asks.
2. What reaches the model is not what your source says reaches it
This is the one I’d put at the centre. It is also the one where I have to state a narrower claim than I first wanted to, because the obvious version of it is wrong.
The wrong version: “source no longer determines behaviour, and that’s new.” It isn’t new at all. A query optimiser picks an execution plan you did not write, based on statistics you did not author. A JIT runs code you did not compile. A feature flag puts behaviour in a remote value rather than your repository. Every one of those broke source-equals-behaviour long before agents did.
And each of them produced exactly the instrumentation I was about to claim as novel. EXPLAIN exists because source stopped determining behaviour. A firewall logs denials, not just allows. Cache metrics record evictions, not just hits. Exclusion logging has decades of prior art and I under-credited it.
The narrower version, which I think holds: what changed is the scope of the nondeterminism.
A query optimiser varies the route to a provably equivalent result set. The plan differs; the answer does not. That is a bounded, checkable form of hidden behaviour, and it is why you can trust an optimiser you cannot read.
A context assembler that drops material from the window changes the answer. Not the route to it. There is no equivalence guarantee, no way to prove the omitted material would not have changed the conclusion, and no bounded set of outcomes.
That is a far larger claim on your trust, and it is what moves event-level exclusion recording from a debugging convenience to a mandatory control.
In a controlled comparison I ran between a hand-rolled Python agent harness and a framework-based one, I found two things I had not gone looking for. The framework appended an instruction of its own to every single one of 1,344 model requests: a sentence of agent-identity boilerplate that nobody wrote, declared, or approved. And when the framework decided to drop something from the context window, nothing anywhere in its API signalled what had been dropped or why; the assembly step returned survivors and the discards were simply gone.
Neither was visible in any source file I controlled. Both were detectable only because my system recorded dispatched bytes rather than intended bytes.
That is not a framework criticism. It’s a property of any layer that assembles requests on your behalf, and most of them do. But it means the behaviour of your system now depends on content that no file in your repository contains, and the only way to know what your system actually did is to have recorded it at the moment it happened.
This elevates runtime provenance from an operational nicety to a first-class architectural concern. Not logging, which most people have. Provenance: can you reconstruct the exact input that produced a given output, from stored events alone, months later? And the harder half, which almost nothing supports: can you show what was withheld from the model and on what basis?
In the study, that was the only axis on which the two harnesses meaningfully differed. Accuracy, cost, and fault recovery were statistically indistinguishable. Observability of the framework’s own decisions was total separation. I initially treated that as a quirk of my metric design. I now think it’s the actual signal.
If you take one design instruction from this essay, take that one. Record what was sent, not what you meant to send. Record what was excluded and why. Everything else in agent architecture is a rediscovery of something you can look up. This one is genuinely new, and the tooling has not caught up to it.
The conflation nobody separates
One last untangling, and I think it explains most of the confusion.
Building software with agents and building software that is agents are two entirely different transitions.
The first changes velocity, review practice, team composition, the economics of abstraction, and what a junior engineer’s week looks like. It is a change in how we produce systems. Its implications are managerial and cultural at least as much as technical.
The second changes what the artifact is, how it fails, what it costs to run, and how you prove what it did. It is a change in what we produce.
They are almost always discussed as one thing, under headings like “AI is changing system design.” They have different evidence, different timescales, different risks, and different people who should care. A CTO worrying about the first should not be reading essays about the second, and vice versa.
Most people asking “is system design changing?” have actually experienced the first, because that’s the one that shows up in your working day, and are asking about the second, because that’s the one that’s written about. That mismatch is why the answers never satisfy.
Where I could be wrong
I want to state my uncertainty properly rather than as a disclaimer at the end.
The historical base rate is unkind to “this changes everything” claims in software. Cloud, microservices, NoSQL, and serverless each resolved to “changes some things a lot, and most things not at all.” A few genuinely did change the discipline: the network, virtualisation, and arguably version control. Most did not. Prior odds favour the deflationary read, which is where I’ve landed, and I should discount my own position for the comfort of it.
Against that: we are roughly two years into serious production use of agent systems. That’s early. Both camps are arguing well ahead of the evidence, including me.
Here’s what would change my mind. If, in two years, the durable patterns in agent architecture turn out to have no classical analogue. Right now I can map nearly all of them back, which is the whole basis of my position. If the field converges on something that isn’t a rediscovery, and my mapping table stops working, then the principles really were changing and I read it wrong.
I’d also flag that my two “genuinely new” items are load-bearing for my own argument in an uncomfortable way. If externally scheduled component decay and the loss of source-equals-behaviour turn out to be minor in practice, then my position collapses into pure deflation, and pure deflation is almost always wrong about something.
So what do you actually do
If you’re building these systems, the deflationary read is good news, because it means your leverage comes from things that are already written down.
- Read the distributed systems literature. Not the agent blogs. The failure modes you’re hitting have names and known treatments. Concretely, per primitive: loop goes to workflow engines and control theory, state to event sourcing and CQRS, tools to API and capability design, context to information retrieval rather than caching, memory to data warehousing and cache invalidation, verification to runtime verification and the end-to-end argument, permissions to capability security, sandbox to OS isolation, telemetry to distributed tracing and query plan instrumentation, limits to admission control and backpressure.
- Design the environment, not the pipeline. Specify what the agent can perceive and affect, then constrain it. The loop should be the least interesting code you own.
- Record dispatched bytes, and record exclusions. This is the one genuinely new requirement. Do it from day one, because retrofitting provenance after an incident is not possible.
- Date your components against the deficiency they answer. Review the list on every model upgrade. Make deletion cheap.
- Prefer structural fixes to instructional ones. A rule in a prompt is a request. A rule in the system is a guarantee, and the only kind you can test.
- Separate the two transitions in your own head. When someone tells you AI is changing system design, ask which one they mean. Usually they haven’t decided.
The discipline is not being replaced. It’s being handed a material that is nondeterministic, metered, drifting, and opaque about its own behaviour, and asked to build dependable things out of it anyway.
That’s not a new discipline. That’s just the job, on a harder day.
Comments
No comments yet.