← all posts

Architecture Is What Happens After Success

A service handling ten thousand requests a second is usually still easy. What's hard is the schema changes, the backfills, the ownership, and the support load that only show up once people actually depend on you.

Most systems don't fail when they get traffic.

They fail when they get successful.

Those sound like the same thing. They aren't, and the gap between them is where most of what we call "architecture" actually lives.

Traffic is the easy scaling problem

A service handling 10 requests per second is easy. A service handling 10,000 requests per second is usually still easy. That's not a brag, it's just where the industry is. We have good answers for raw throughput: add replicas, add a cache, add a queue, scale the boring parts horizontally. The happy path scales about as well as you're willing to pay for.

This is why "can it handle load?" is the wrong question to lead with. Load is the problem we've mostly solved. It's loud, it's measurable, and it shows up on a graph you're already watching. If load were the hard part, architecture would be a solved discipline.

Success arrives with baggage

The hard part is everything else that shows up the moment people actually depend on your system:

  • Schema changes that now have to happen without downtime, against data you can't afford to lose.
  • Backfills that have to run across millions of existing rows while the system stays live.
  • Operational ownership that someone has to actually hold, including at 3am.
  • Compliance requirements that arrive the moment the data becomes worth regulating.
  • Observability, because you can no longer fix what you can't see, and observability is runtime architecture, not a dashboard you add later.
  • Support requests from real users who found the path you never tested.
  • Recovery procedures for the day something goes wrong and "restore from backup" turns out to be a sentence nobody had ever actually run.

None of that has anything to do with requests per second. All of it arrives with success, and none of it was visible on launch day.

The code usually survives. The system around it often doesn't.

This is the part I want to land. When a successful system gets into trouble, the core code is rarely the thing that broke. The request handler still works. The algorithm is still correct.

The code usually survives. The surrounding system often doesn't.

What buckles is everything the code was quietly leaning on: the assumption that schema changes would be rare, that one team would always own this, that the data volume would stay small enough to migrate in an afternoon, that nobody would ever need to audit it. Those assumptions were free on day one and crushingly expensive on day two hundred.

This is the same shape as "it works" not being "it's ready", scaled up in time. A demo hides the failure modes because you're the only user. A successful system hides them because they were always six months out, and six months always arrives.

Architect for month six, not launch day

So the useful version of architecture isn't "will this handle traffic on launch day." It's "what will this system need from us once everyone depends on it, and have we built the seams to give it to them?"

Concretely, that means asking the unglamorous questions early:

  • How do we change the schema once this table has real data in it?
  • Who owns this at 3am, and do they have what they need to operate it?
  • How do we observe it well enough to debug a problem we didn't predict?
  • How do we recover when, not if, something goes wrong?

You don't have to build all of that on day one. You have to leave room for it, so that adding it later is a change instead of a rewrite. That foresight is the actual deliverable. Everything else is just typing.

The one-line version

Architecture isn't about whether your system survives launch day. It's about what happens six months later, when everyone depends on it and all the assumptions you got for free finally come due.