System design · 8 of 20

Designing a system end to end

A method for designing a system: requirements before boxes, back-of-envelope numbers, the API and the access patterns, a worked URL shortener, and the five questions that judge any design.

Updated 2026-09-18

On this page

The previous seven articles are a box of parts. This one is about assembling them: how to go from a one-line description to a design you can defend, and what the arithmetic tells you long before the diagram does. The method is five moves, then one worked design small enough to hold in your head; the twelve case studies that follow apply the same five moves to systems that do not fit in your head.

The map

Read this first when short on time. Every branch is a section below.

Figure 1. The whole article on one page. Every branch is a section below; fold what you know, open what you do not.

Requirements before boxes

Every bad design starts by drawing boxes. The boxes are the last step, because until the requirements are written down there is no way to tell a good box from a bad one. Five moves, in order, and each one constrains the next.

flowchart TD
  R["Step 1 · requirements: what it does, at what scale, how fast"]
  R --> N["Step 2 · numbers: requests per second, bytes per day"]
  N --> A["Step 3 · API and data model: the queries, then the schema"]
  A --> D["Step 4 · components: only what the first three demand"]
  D --> T["Step 5 · trade-offs: the bottleneck and what breaks first"]

Figure 2. The order matters more than the content. Any box drawn before the first three steps are done is decoration.

Functional requirements are verbs: shorten a URL, redirect a short link, show a feed, send a message. Write the exclusions too — no editing, no analytics in version one — because scope removed is design simplified.

Non-functional requirements are the numbers and the promises: how many users, how many requests, how much data, the p99 latency target, how long data is kept, what happens when a region fails, and how consistent each operation must be. These are what actually decide the architecture. A system with a thousand users needs one machine and a database; the same features at a hundred million users need every article in this series.

One ratio dominates all of them: reads to writes. A read-heavy system is solved with caches, replicas and precomputation. A write-heavy system cannot be cached and needs partitioning, batching and a log. Establishing that ratio early tells you which half of the toolbox to open.

In the wild

A social timeline is read-heavy by two or three orders of magnitude, which is why it is precomputed rather than queried.
Metrics and logging are write-heavy: almost everything ingested is never read, so the design optimises ingestion and compresses aggressively.
A payment system is neither: throughput is modest and correctness dominates, so it buys consistency with latency everywhere.

The numbers: estimating before designing

Back-of-envelope arithmetic is not a formality. It is how you find out, in two minutes, whether the data fits in memory, whether one database will do, and where the cost will land. Round everything to powers of ten; the goal is the order of magnitude, not the digits.

Three constants carry most of the work. A day is 86,400 seconds, near enough 10⁵, so a million events a day is about 12 per second and a billion a day is about 12,000 per second. Peak is two to three times average, and capacity is planned for peak. And latency has a hierarchy that has barely changed in a decade.

Operation	Rough time	What it implies
L1 cache reference	1 ns	Free at any scale you will design for
Main memory reference	100 ns	A memory lookup is a million times cheaper than a disk seek
SSD random read	100 µs	A few thousand random reads per second per device
Round trip inside a datacenter	0.5 ms	Ten chained service calls cost 5 ms before any work
Disk seek, spinning	10 ms	Why anything latency-sensitive left spinning disks
Round trip across a continent	150 ms	One cross-region hop can blow a whole latency budget

Storage follows the same shape: bytes per record, times records per day, times retention, times the replication factor. A record of 500 bytes written ten million times a day is 5 GB a day, about 1.8 TB a year, and about 5.5 TB once it is replicated three times. That number alone answers whether this is a single Postgres instance or a sharded cluster.

Writes   10M/day   -> 10M / 86,400  ~= 115/s   -> peak ~350/s
Reads     1B/day   -> 1B  / 86,400  ~= 11.6k/s -> peak ~35k/s
Ratio    100 reads per write        -> cache-first design
Storage  500 B x 10M/day = 5 GB/day -> 1.8 TB/year, 5 TB replicated
Hot set  10% of keys serve 90% of reads -> ~180 GB of a year's data, cacheable

Tip

Do the arithmetic out loud and write each line down. Half of design decisions become obvious once a number exists: 35,000 reads per second rules out a single database, and a 180 GB hot set rules in a cache.

The API and the data model

Before any component appears, write the API and the list of queries it implies. The API is the contract everything else serves, and the queries — not the entities — determine the schema.

A small API for the shortener below is four endpoints, and each one raises a design question:

POST /links          { url, custom_alias?, expires_at? }  -> { short_url }
GET  /{code}                                              -> 302 redirect
GET  /links/{code}/stats                                  -> { clicks, by_day }
DELETE /links/{code}

The create call needs an idempotency key so a retried request does not mint a second code. The stats call is a different workload entirely — aggregation over events, not a point lookup — so it belongs on a different store, as the analytics article argues. Listing endpoints need cursor pagination, because offset pagination re-scans everything it skips and drifts when rows are inserted.

Then list the access patterns literally, as sentences: "get the URL for a code", "list a user's links newest first", "count clicks for a code by day". The list decides the storage model — key-value for a pure lookup, relational for joins and transactions, wide-column for one entity with a huge, ordered child list — and it decides the shard key, which must be the value present in almost every query. Choosing that wrong is the one mistake that cannot be patched later; the storage article covers why.

In the wild

DynamoDB forces this order by design: you cannot query a table without naming the partition key, so the access patterns must exist before the table does.
Stripe's API takes an idempotency key on every mutating call, which is why a network failure during a charge is recoverable by simply retrying.
Slack and GitHub paginate with opaque cursors, so a page of results stays stable while new rows arrive.

Worked design: a URL shortener

Take the numbers above: 10 million new links a day, a billion redirects, a hundred reads per write. The read path must be fast and cheap; the write path is almost idle by comparison.

The interesting question is where the code comes from. Three options, and the trade is coordination against collisions.

Approach	How	Trade-off
Hash the URL	Take the first 7 base62 characters of a hash	No coordination, but collisions must be detected with a read before every write
Global counter	Increment a counter, encode it in base62	Codes are sequential and guessable, and the counter is a bottleneck and a single point of failure
Counter ranges	Each server claims a block of a million ids, then hands them out locally	One coordination call per million links, no collisions; gaps appear when a server dies, which does not matter

Counter ranges win. Seven base62 characters give 62⁷, about 3.5 trillion codes, which at 10 million a day lasts roughly a thousand years. Scramble the encoded value if guessable codes are a concern.

flowchart TD
  U[client] --> E["edge and CDN"]
  E --> S["redirect service, stateless"]
  S --> C["cache: code to URL"]
  C -- "miss, about 2 percent" --> K[["key-value store, sharded by code"]]
  S --> Q[["click events to a log"]]
  Q --> A["analytics pipeline"]

Figure 3. The redirect path is three hops and almost always ends at the cache. Click recording is deliberately off the critical path.

Serving is then mostly caching. Link popularity is extremely skewed — a small fraction of codes take almost all traffic — so a cache holding the hot set answers well over ninety percent of redirects, and the store behind it sees only a trickle. Use a 302 rather than a 301: a 301 is cached by the browser forever, which is faster but means the second click is never seen, and the click count is usually the product.

Recording clicks must not slow the redirect. Emit an event to a log and return the redirect immediately; the counting happens in the pipeline. Expiry is a TTL on the record, and deletion is a tombstone that the cache must be told about.

Watch out

Custom aliases break the clean model: they are user-chosen, must be unique, and must not collide with generated codes. Keep them in the same keyspace with a uniqueness constraint and reserve a prefix or a length that the generator never produces.

That is the whole method on a system small enough to hold in your head: three numbers, one keyspace decision, one cache. Everything after this article is the same five moves applied to systems that are not small — a shortener at full depth, then caches, rate limiters, feeds, chat, maps, files, video, search, and model serving. The moves do not change; only the size of the numbers does.

Judging a design

A design is finished when you can answer five questions about it without hesitating. They are the same five every time.

Where is the bottleneck? One resource limits the system — a single database's write throughput, the fan-out workers, memory for the hot set. If you cannot name it, the numbers were never done.
What breaks first at ten times the traffic? Usually not the thing you optimised. Say which component you would split, cache or shard next, and why that one.
What happens when each piece fails? Every component gets a sentence: the cache dies and the store takes the full load, can it? The fan-out workers stall and feeds go stale, is that visible? A region fails and traffic shifts, does the data follow?
Which operations need strong consistency? Almost never all of them. A balance and a like are not the same thing, and saying so per operation is what separates a considered design from a defensive one.
What does it cost, and in what shape? Storage that grows forever, egress that scales with reads, compute that scales with writes. The shape tells you which requirement to renegotiate first.

Tip

Write the trade-off down next to each choice, not just the choice: "fan-out on write, because reads outnumber writes a thousand to one, at the cost of celebrity posts needing a separate path". A design people can argue with is worth more than one that only states conclusions.

Recap

Order: requirements, numbers, API and data model, components, trade-offs. Boxes drawn before the numbers are decoration.
Functional requirements are verbs and exclusions; non-functional ones are scale, latency, retention, consistency and failure behaviour, and they are what actually decide the architecture.
The read-to-write ratio picks half the toolbox: reads are solved with caches, replicas and precomputation, writes with partitioning, batching and logs.
A day is 86,400 seconds, so a million a day is 12 per second and a billion a day is 12,000. Plan for a peak of two to three times average.
Latency hierarchy: memory 100 ns, SSD 100 µs, datacenter round trip 0.5 ms, cross-continent 150 ms. One cross-region hop can consume an entire budget.
Storage is bytes times rate times retention times replication, and that one number decides between a single instance and a cluster.
Write the API first — idempotency keys on writes, cursor pagination on lists — then list the queries as sentences, because the queries choose the storage model and the shard key.
URL shortener: counter ranges beat hashing and a global counter, seven base62 characters give 3.5 trillion codes, the cache absorbs almost all reads, 302 keeps analytics alive, and clicks go to a log off the critical path.
Judge a design by naming the bottleneck, what breaks at ten times traffic, each component's failure behaviour, which operations truly need strong consistency, and the shape of the cost.

Questions

Try answering each one out loud before opening it. Lead with the one-line answer, then the reasoning, then one detail from a real system.

What do you establish before drawing any component?

The functional requirements as verbs, the non-functional ones as numbers and promises, and the read-to-write ratio — because the ratio decides whether the design is built around caching and precomputation or around partitioning and logs.

Exclusions matter as much as inclusions: scope removed is design simplified.
Latency targets belong at p99, since averages hide the behaviour that users notice.
The same feature list at a thousand users and at a hundred million is two different systems.

If the numbers were never written down, there is no way to say which of two designs is better, only which is more elaborate.

Why write down what the system will not do?

Because every excluded feature removes constraints from the design, and an unstated exclusion is treated as a requirement by whoever reads the design next.

Editing a short link, for example, forces cache invalidation and versioning that a create-only design never needs.
Exclusions make the scope reviewable: someone can disagree with a boundary they can see.
They also date the design, so a later reader knows which requirements it was actually built against.

Most designs that collapse under their own weight do so because nobody wrote down where they stopped.

Estimate the capacity for a service taking 500 million requests a day.

Roughly 5,800 requests per second on average and 12,000 to 17,000 at peak, because a day is 86,400 seconds and peak runs two to three times average.

500M / 86,400 is about 5.8k per second.
If each request needs 1 KB stored, that is 500 GB a day before replication.
At three replicas and 90 days of retention, that is roughly 135 TB, which is a cluster rather than a machine.

Rounding to powers of ten is deliberate: the decision changes at 12,000 per second versus 120, never at 5,800 versus 5,900.

How do you decide between one database and a sharded cluster?

From the two numbers the estimation produces: total stored bytes against what one machine can hold and serve, and peak writes per second against what one primary can commit.

A few terabytes and a few thousand writes per second is still one well-provisioned Postgres with replicas.
Tens of terabytes, or writes beyond what one primary can take, means sharding or a store that shards for you.
Read load alone rarely forces sharding, because replicas and a cache absorb it.

Sharding is the expensive answer, so it is worth proving the single-node option cannot work before choosing it.

Why does the latency hierarchy matter when choosing where to put data?

Because each step down costs about three orders of magnitude — memory at 100 ns, SSD at 100 µs, a cross-continent round trip at 150 ms — so where a byte lives dominates every other optimisation on the path.

Ten chained service calls inside one datacenter cost 5 ms before any work happens.
A single cross-region call can exceed the whole latency budget for a page.
It is why caching is the first tool for reads and why chatty service boundaries are expensive.

Those numbers have barely moved in a decade, because they are set by physics and device behaviour rather than by software.

Why state a latency target at p99 rather than as an average?

Because an average hides the slow tail that users actually experience, and at scale the tail is common: a user whose page makes a hundred calls will almost certainly hit the one percent.

Averages are dragged down by the many fast responses and say nothing about the worst ones.
Fan-out multiplies tail latency, so a p99 per call becomes a near-certain slow page across a hundred calls.
p99 also exposes the queueing and garbage-collection pauses that averages smooth away.

This is why the observability article insists on percentiles as the unit of any latency promise.

Why write the API and access patterns before the schema?

Because the queries decide the storage model and the shard key, and both are extremely expensive to change later; designing entities first produces a schema that has to be worked around by every query.

List the access patterns as sentences, then check that each one is a single efficient operation.
The shard key must appear in nearly every query, or requests fan out to all shards.
The API also pins down idempotency and pagination, which are hard to retrofit.

DynamoDB enforces this order structurally, since a table cannot be queried without naming its partition key.

Why is the shard key the hardest decision to reverse?

Because it determines the physical placement of every row, so changing it means rewriting and moving the entire dataset while the system stays online, usually with a dual-write migration that runs for weeks.

A key missing from a query turns that query into a scatter-gather across every shard.
A key with skew concentrates load on one shard no matter how many you add.
Application code, caches and downstream pipelines all encode assumptions about it.

It is the one decision worth slowing down for, which is why the access patterns are written before the schema.

How would you generate short codes for a URL shortener?

Hand each service instance a block of a million ids from a coordination service, then encode locally in base62; that removes both the collision checks of hashing and the bottleneck of a global counter.

Seven base62 characters give about 3.5 trillion codes, enough for centuries at ten million a day.
Hashing needs a read before every write to detect collisions.
A global counter is a single point of failure and produces guessable, enumerable codes.

Gaps appear whenever an instance dies holding an unused block, which costs nothing given the size of the keyspace.

Would you return a 301 or a 302 for a short link?

302, because a 301 is cached by the browser indefinitely and every subsequent click bypasses the service entirely, which removes the click analytics that are usually the point of the product.

301 is faster for the user and cheaper to serve, so it is right when analytics do not matter.
A 301 also makes a link impossible to retarget or revoke for anyone who has visited it.
Either way the click event goes to a log asynchronously so recording never delays the redirect.

It is a rare case where the slower option is chosen for a product reason rather than a technical one.

The shortener's cache restarts with an empty keyspace. What happens?

Every redirect misses and goes to the key-value store at once, so a store sized for two percent of traffic is suddenly taking all of it, and the cache fills only as fast as requests arrive.

The store must be provisioned for more than its steady-state trickle, or it becomes the outage.
Warm the cache from the most-requested keys before returning the node to service.
Request coalescing stops a thousand simultaneous misses on the same key becoming a thousand reads.

This is the stampede problem from the storage article, and a cold restart is its most predictable trigger.

What five questions should any design be able to answer?

Where the bottleneck is, what breaks first at ten times the traffic, how each component behaves when it fails, which operations genuinely need strong consistency, and what the cost looks like as the system grows.

Not naming a bottleneck means the arithmetic was skipped.
Consistency is chosen per operation: a balance and a like are not the same problem.
The shape of the cost — storage, egress or compute — tells you which requirement to renegotiate first.

Writing the trade-off beside each choice, not just the choice, is what makes a design reviewable rather than merely stated.