Designing YouTube

Video is the only system in this series where the bytes themselves are the problem. A social feed moves kilobytes; this moves roughly a hundred terabits a second, and every one of those bits was produced by a transcoding farm that had to guess, at upload time, which of eight quality levels anyone would ever watch. The two halves of the design barely touch: an enormous batch pipeline that turns one file into hundreds, and a delivery network whose only job is to never fetch the same segment twice.

The map

Read this first when short on time. Every branch is a section below.

What it must do, and at what size

It must accept a video from an unreliable device, turn it into something every screen and network can play, deliver it worldwide at a quality the connection can sustain, and count what happened. It must not, in this article, cover search and recommendations in depth — those are the subject of the analytics and search articles — nor the curated-catalogue problems that the Netflix article takes up.

Ingest     500 hours of video uploaded per minute
           = 30,000 hours/hour ~= 2 GB/hour of source -> ~60 TB/hour, ~1.4 PB/day

Egress     1 billion hours watched/day at an average ~2 Mbps
           1e9 h x 3,600 s x 2 Mb = 7.2e12 Mb/day ~= 900 PB/day
           ~83 Tbps average, ~150-200 Tbps at peak

Ratio      egress is roughly 600x ingest -> delivery is the business
Transcode  ~8 renditions per video, output ~1.5-2x the source bytes
           encoding one hour of video costs hours of CPU per rendition
Tail       a small fraction of videos take most views
           the median video is watched a handful of times

Two facts dominate. Egress is six hundred times ingest, so the delivery network is the product's cost structure and its engineering centre of gravity. And popularity is a power law that nobody can predict at upload time, which means encoding everything at every quality wastes an enormous amount of compute on videos nobody will watch, while encoding nothing in advance makes the first viewer wait.

flowchart TD
  U["creator uploads"] --> IN["upload service, resumable"]
  IN --> OR[["original in cold storage"]]
  IN --> Q[["transcode job graph"]]
  Q --> TR["encoding farm, chunk parallel"]
  TR --> PK["packager: segments and manifests"]
  PK --> ST[["rendition storage, origin"]]
  ST --> RG["regional cache"]
  RG --> ED["edge cache, inside ISPs"]
  ED --> V["viewer, adaptive bitrate client"]
  V --> EV[["playback and view events"]]

Upload

The upload path is short but has to survive the worst client conditions in this series: a phone on mobile data sending several gigabytes.

Uploads are chunked and resumable. The client asks to start an upload, receives a URL and an id, sends fixed-size parts, and can at any point ask how much the server has — so a dropped connection resumes rather than restarts. The video becomes visible only when every part has landed and the server has assembled and verified them, which is the same ask-then-send-then-commit shape as the file sync design, for the same reasons.

Before anything expensive begins, the file is probed: container, codecs, resolution, frame rate, duration, and whether it is actually decodable. Rejecting a malformed file here costs a second; discovering it after scheduling eight encoding jobs costs hours of CPU. A content hash also catches an exact re-upload, which is common enough to be worth handling.

The original is kept forever, in the cheapest available tier. It is the only thing that cannot be regenerated, and it is what allows the entire catalogue to be re-encoded years later when a better codec arrives — which is a real operation that saves a meaningful fraction of global bandwidth each time.

Transcoding

One upload has to become many files, because the devices and networks watching it have nothing in common. A 4K master is useless to a phone on a slow connection, and a 240p version is unwatchable on a television.

The output is a ladder: roughly eight steps from 144p at a few hundred kilobits to 4K at tens of megabits, each with a resolution and a bitrate chosen together, because encoding 1080p at 300 kbps produces something worse than 360p at the same rate. Each rung is usually produced in more than one codec — a universally supported one for compatibility and a newer one that is meaningfully more efficient for the clients that can decode it.

flowchart TD
  S["source file"] --> P["probe and split at keyframes"]
  P --> C1["chunk 1"]
  P --> C2["chunk 2"]
  P --> C3["chunk N"]
  C1 --> E["encoder workers, one rendition per chunk"]
  C2 --> E
  C3 --> E
  E --> J["stitch chunks per rendition"]
  J --> V["verify: duration, drift, quality score"]
  V --> PK["package into segments and manifests"]

The trick that makes this tractable is chunk-parallel encoding. Video is split at keyframes into independent pieces, each encoded separately, and the results are concatenated. Because each chunk starts at a keyframe, the join is seamless. A video that would take four hours to encode serially finishes in minutes given enough workers — and the work is embarrassingly parallel, which is exactly the shape a batch fleet wants.

Scheduling matters as much as the encoding. The jobs form a graph per video, and the order is chosen so that something is watchable as soon as possible: produce a mid-quality rendition first, publish, then fill in the rest. A creator with a large audience gets priority, because their video will be watched within seconds of publication and an unknown creator's may not be watched at all.

Packaging

A rendition is not shipped as one file. It is cut into segments of a few seconds, each beginning with a keyframe so that it can be decoded independently, and described by a manifest listing every rendition and every segment.

That structure is what makes adaptive streaming possible: a player can fetch segment 41 at 720p and segment 42 at 1080p and splice them, because each is self-contained. It is also what makes the CDN's job easy — a segment is an immutable file with a stable URL, so it can be cached forever, which is the single most important property in the delivery path.

The two manifest formats in use say the same thing in different syntax, and the practical answer is to package once into a common segment format and generate both manifests, rather than encoding twice.

Rendition

One quality level of a video: a resolution, a bitrate and a codec. A video has several, and the client switches between them.

Segment

A few seconds of one rendition, starting at a keyframe, stored as an immutable file. The unit of caching and of adaptation.

Manifest

The index a player fetches first: which renditions exist, and the URLs of their segments.

Storage and the long tail

Here is the decision that saves the most money in the entire design, and it comes from the shape of the popularity curve rather than from any clever mechanism.

Most videos are watched very few times. A tiny fraction are watched enormously. And crucially, nobody knows which is which at upload time — popularity is unpredictable, and it changes when a video resurfaces years later. Encoding every video into every rung of the ladder means spending the majority of the encoding budget on renditions that will never be requested.

The answer is lazy transcoding. Produce a small number of renditions eagerly — enough to play on a phone and on a laptop — and generate the expensive high-resolution ones on first demand, accepting that the very first viewer of a 4K rendition waits, or is served a lower quality while it is produced. Once produced, it is cached like anything else. The same reasoning tiers storage: hot renditions on fast storage, cold ones evicted and regenerated from the original if they are ever wanted again.

flowchart TD
  U["video uploaded"] --> E["encode the cheap renditions now"]
  E --> P["publish, playable on phone and laptop"]
  P --> W{Requested at 4K?}
  W -- "no, most videos" --> N["never encoded, costs nothing"]
  W -- "yes" --> L["encode on demand, first viewer waits"]
  L --> C["cached like any other rendition"]

Delivery

Roughly a hundred terabits a second is not served from a datacentre. It is served from thousands of caches placed as close to viewers as possible, and the entire economics of the product is the fraction of bytes that come from the nearest one.

The hierarchy is edge, regional, origin. A player requests a segment from its nearest edge cache; a miss fills from a regional cache, and a miss there fills from origin. Because segments are immutable, every one of those caches can hold them indefinitely with no invalidation logic at all — the hardest problem in caching simply does not exist here.

flowchart TD
  V["viewer"] --> E["edge cache, inside the ISP"]
  E -- "miss" --> R["regional cache"]
  R -- "miss" --> O[["origin storage"]]
  O --> R
  R --> E
  PP["predicted-popular titles"] --> E

Placement has two modes. Push: a video expected to be popular — a major release, a channel with millions of subscribers — is copied to edge caches before anyone asks, so the first viewer in each region already gets a hit. Pull: everything else arrives at the edge because somebody watched it. The push list is a prediction problem, and getting it right matters because a miss at the edge for popular content multiplies upstream traffic by the number of edges.

The most consequential piece of engineering here is not in the software: it is placing caches inside internet service provider networks. A cache in the ISP's own datacentre means the bytes never cross a paid transit link, which is better for the ISP's costs and for the viewer's latency simultaneously. It is why video platforms run hardware programmes rather than simply buying more bandwidth.

Adaptive bitrate

The server does not decide the quality; the player does, one segment at a time. This is the right design because only the client can observe what is actually happening on the last mile, which changes as someone walks between rooms.

flowchart TD
  S["segment finished downloading"] --> M["measure throughput and buffer level"]
  M --> B{Buffer healthy?}
  B -- "low" --> D["step down, protect playback"]
  B -- "comfortable" --> T{Throughput above the next rung?}
  T -- yes --> U["step up one level"]
  T -- no --> K["stay"]
  D --> N["request the next segment"]
  U --> N
  K --> N

Two families of algorithm exist and both have a characteristic failure. Throughput-based selection reacts quickly and oscillates, because one fast segment convinces it to step up and the resulting larger segment convinces it to step back. Buffer-based selection is stable — it simply keeps the buffer full — but is slow to take advantage of a genuinely improved connection. Real players blend them, with hysteresis so that a step up requires sustained evidence and a step down happens immediately.

The most important moment is the start. Time to first frame determines whether a viewer stays, so playback begins at a low rendition and climbs within a few seconds, rather than waiting to measure the connection properly. This is the same asymmetry as everywhere else in this article: a slightly worse picture is always better than a delay, and a stall is the worst outcome available.

Counting views

A view count looks like an integer and is a policy. What counts as a view — thirty seconds, some fraction of the duration, a deliberate play rather than an autoplay — has to be defined before it can be implemented, and the definition drives payouts, so it is also an adversarial problem.

Mechanically, the player sends periodic heartbeats while playing rather than one event on start. Heartbeats give watch time, which is the signal that ranking actually uses, and they make a view a duration rather than a click. The events flow into the same shape of pipeline as everything else in this series: a partitioned log, a stream aggregation into per-video per-hour counters, and a serving store for the dashboard.

Two adjustments make it affordable. Counters are sharded, because a video going viral concentrates tens of thousands of increments a second on one key. And displayed counts are approximate and delayed above a threshold — nobody distinguishes 2.4 million from 2.41 million — while the creator's own analytics are exact and reconciled from the log, after fraud filtering has had time to run.

Live, in one section

Live streaming is the same pipeline with the time removed, and that changes three things.

There is no time to encode ahead: the ladder is produced in real time as the stream arrives, which means encoding must keep up with the input or fall behind permanently, and chunk-parallel tricks no longer help because there is only one chunk at a time. There is a latency budget measured in seconds rather than milliseconds, and the segment length sets the floor — four-second segments mean at least four seconds of delay, which is why low-latency variants deliver partial segments as they are produced rather than waiting for a complete one. And the DVR window — the ability to seek backwards during a live stream — is just the recent segments kept around, which makes it nearly free.

Everything else is identical: segments, manifests, edge caching, adaptive bitrate. The manifest is simply rewritten as new segments appear, and the player polls it.

What breaks first

1. Encoding capacity at upload spikes. Uploads are bursty and encoding is expensive, so a backlog forms and videos publish late. Relieved by lazy encoding, by priority queues that serve large channels first, and by the fact that the work runs happily on preemptible capacity.
2. Edge cache hit ratio. The number that determines the bandwidth bill. It degrades when the catalogue grows faster than the caches, and it is improved by better push prediction rather than by more storage.
3. Peering and transit capacity. Physical, expensive, and negotiated months in advance. It is the reason caches go inside ISP networks.
4. Metadata for the tail. Billions of videos each with manifests, renditions, thumbnails and rights information is a large database problem hiding behind a much more visible bandwidth problem.

The judgement running through the whole design: playback beats everything. Serve a lower rendition, serve from a further cache, serve a slightly stale manifest — but never stall the video, because a stall is the one failure a viewer will not forgive.

Recap