Designing Netflix
Per-title and per-shot encoding, a purpose-built delivery network with proactive fill, the playback control plane, personalization, rights windows and resilience engineering.
On this page
Streaming a catalogue looks like the YouTube problem and is almost its opposite. There are twenty thousand titles rather than billions of videos, nothing arrives unannounced, and every title will be watched by millions of people. Those three facts invert the economics: it becomes worth spending enormous compute on each encode, and it becomes possible to copy the entire evening's viewing into caches inside people's own internet providers the night before.
The map
Read this first when short on time. Every branch is a section below.
What makes a catalogue different
The requirements look similar to any video platform — play a title on any device at the best quality the connection allows — but three properties of a catalogue change every design decision downstream.
Members ~300 million, concentrated in regional evening peaks
Catalogue ~20,000 titles, versus billions of user-uploaded videos
Encodes hundreds of files per title: renditions x codecs x audio x subtitles
Views every title is watched by many thousands to many millions
Key ratios
encoding cost per title amortised over millions of streams -> spend freely
bytes saved per bit cut multiplied by every stream -> worth enormous effort
demand predictability scheduled releases, no viral surprises -> pre-position
Peak shape regional and nightly: the same hours, the same time zones, every dayCompare that with user-generated video, where a video may be watched three times and popularity is unpredictable. There, lazy encoding and pull-based caching are correct. Here, the opposite is correct on both counts: encode exhaustively in advance, because the cost is amortised over millions of views, and push content to the edge before anyone asks, because you know what people will watch tonight.
The peak shape matters too. Traffic is not uniform: it concentrates in the evening, in each time zone, every day. That produces a large daily trough — and the trough is not wasted capacity, it is the window in which the next day's content is distributed.
Encoding the catalogue
When one encode serves millions of streams, every bit saved is multiplied by millions. That justifies spending compute on encoding that would be absurd for a video watched twice.
Per-title encoding was the first big step. A fixed bitrate ladder assumes all content is equally hard to compress, which is plainly false: an animated film with flat colour compresses far better than a grainy handheld night scene. Encoding each title at many candidate settings, measuring the resulting quality, and selecting a ladder tuned to that title delivers the same perceived quality at substantially fewer bits — or better quality at the same bits, which matters most for members on slow connections.
Per-shot encoding pushes it further, because complexity varies within a title: a static dialogue scene and an explosion have nothing in common. Splitting at shot boundaries and choosing settings per shot squeezes out more again, and it composes naturally with the chunk-parallel encoding described in the YouTube article, since shots are already independent units.
None of this is possible without a way to measure quality that is not "bitrate". Bitrate is an input, not a result, and simple signal metrics correlate poorly with what people actually perceive. The answer is a perceptual quality metric — a model trained on human opinion scores that predicts how good a rendition looks to a viewer — which turns encoding into an optimisation problem with a real objective: maximise predicted quality per bit, per title, per shot.
flowchart TD M["master file"] --> SH["split into shots"] SH --> TR["encode each shot at many settings"] TR --> QM["score each with a perceptual quality metric"] QM --> SEL["select the settings that hit the quality target cheapest"] SEL --> LAD["assemble the per-title ladder"] LAD --> PKG["package: segments, audio, subtitles, DRM"]
- VMAF, Netflix's perceptual quality metric, was open sourced and is now widely used across the industry as the objective for encoding decisions.
- A single title may have hundreds of associated files once every rendition, codec, audio language and subtitle track is counted, which makes the packaging step a serious piece of engineering in itself.
- Every codec generation justifies re-encoding the whole catalogue, because the saving applies to all future streams of every title.
The delivery network
A single content type, a single workload, and a predictable catalogue make a general-purpose CDN a poor fit — so the delivery network is built rather than rented, as purpose-made appliances placed inside internet service provider networks.
The decisive advantage is proactive fill. Because the catalogue is known and popularity is predictable per region, the system can decide in advance what each appliance should hold, and copy it during the local quiet hours — typically overnight, when the network is otherwise idle. By the evening peak, the content people are about to watch is already sitting inside their own provider's network.
flowchart TD P["popularity prediction per region"] --> PL["fill plan per appliance"] PL --> FW["fill window, local quiet hours"] FW --> OCA[["appliances inside ISP networks"]] V["member presses play"] --> ST["steering service ranks servers"] ST --> OCA OCA --> V OCA -. "rare miss" .-> OR[["origin"]]
This is worth contrasting explicitly with the pull-based model. A user-video platform cannot pre-position, because it does not know which of its billions of videos will be watched tonight; it fills caches reactively, one miss at a time. A catalogue service knows, so it converts an unpredictable peak-time fetch into a predictable off-peak copy — using capacity that would otherwise be idle, and turning the evening into almost pure cache hits.
The relationship with the provider is symbiotic rather than transactional: the appliance costs the ISP rack space and gives back a large reduction in transit traffic during their busiest hours, plus better quality for their subscribers.
Playback: two planes
Pressing play triggers a sequence of control-plane calls that have nothing to do with video bytes, and it is worth separating them because they fail differently and are scaled differently.
sequenceDiagram autonumber participant C as Client participant P as Playback API participant L as Licence service participant O as Delivery appliance C->>P: play title 41207 on this device P->>P: check entitlement, rights window, device capability P-->>C: manifest plus a ranked list of servers C->>L: request a licence for the content keys L-->>C: licence, bound to this device C->>O: fetch segments, measure throughput O-->>C: segments Note over C,O: on failure, fall back to the next server in the list
The control plane authorises the member, checks the rights window for their country, decides which renditions their device can actually decode, issues a manifest and hands over a licence for the decryption keys. It runs once per playback, so it can afford to be a proper service with real logic.
Steering is the interesting middle: the service returns a ranked list of servers rather than a single address, based on the member's network, the appliances that hold the title and current load. The client then measures and chooses, and can fall back to the next entry when one performs badly. Splitting the decision this way means neither side needs complete information — the service knows about placement and load, the client knows what the connection is actually doing.
Personalization
With twenty thousand titles and a screen that shows perhaps forty, what appears on the home page determines what gets watched. The page is not a catalogue view; it is a computed object, and every part of it is chosen.
Its structure is rows: each row is a theme with a title, and both the choice of rows and their order are personalized, as is the order of titles within each row. That is a two-level ranking problem — pick the rows, then fill them — with a diversity constraint across the whole page so it does not become forty variations of one genre.
The pipeline is the same funnel as the news feed: generate candidates from many sources — recently added, similar to something finished, popular in this country, continue watching — then rank them with a model, then apply diversity and business rules. Candidates and features are largely precomputed, because a home page must render in a couple of hundred milliseconds on a television with a slow processor.
Even the artwork is chosen: the same title is represented by different images for different members, because which image is shown measurably changes whether someone starts watching. That turns a creative asset into a ranked item with its own model.
What ties it together is that none of these decisions are trusted without measurement. Every change is tested, and the metric is long-term retention rather than clicks — an important distinction, because optimising for immediate engagement is exactly how a recommendation system learns to promote things people regret watching.
Rights and availability
The least glamorous part of this design, and the one that surprises people: a title's existence and its availability are different questions.
Content is licensed per territory and per window. The same title may be available in one country and not another, may arrive next month and leave in six, and may have different audio and subtitle entitlements in each place. So the catalogue is not a global list with a filter; availability is evaluated per member, per country, per device, at request time.
Two consequences worth stating. The encodes may exist for a title that a member cannot play, so storage and rights are decoupled and the delivery network may hold content it must not serve in that region. And availability changes daily as windows open and close, which means the home page, search results and recommendations all need availability as a live input rather than a static attribute — one of the highest-volume metadata lookups in the entire system.
Resilience
This is the design's most influential contribution to the industry, and it comes from one assumption: at this many services on this much hardware, something is always broken, so the system must be built to work anyway.
The mechanisms are the ones in the reliability article — timeouts, circuit breakers, bulkheads per dependency, retries with budgets — applied with a specific rule: every dependency has a fallback. If the personalization service is unavailable, the home page renders with a non-personalized ordering. If the "continue watching" service is slow, that row is omitted. The page degrades in pieces rather than failing, because a member who sees a slightly worse page does not notice, and a member who sees an error does.
flowchart TD
H["home page request"] --> P{Personalization up?}
P -- yes --> R["personalized rows"]
P -- no --> G["generic popular rows"]
R --> C{Continue watching up?}
G --> C
C -- yes --> F["include the row"]
C -- no --> S["omit the row"]
F --> PG["render"]
S --> PG
The practice that made this famous is testing it deliberately: terminating instances, injecting latency, and failing whole regions on purpose, in production, during working hours when engineers are watching. The insight is not that failures are fun but that a fallback which is never exercised does not work — it has bit-rotted, or it was never correct, and the only way to know is to use it routinely rather than during an incident.
At the largest scale this becomes regional failover as a drill: the ability to move all traffic out of a region within minutes, practised often enough that it is unremarkable.
The transferable lesson is not "run chaos experiments". It is that untested fallback paths are decorative. Whatever the scale, the fallback you rely on during an incident should be one that runs, and is observed working, on ordinary days.
Quality of experience
The delivery system's success is measured on the client, continuously, by a small set of metrics that every decision is ultimately judged against.
Rebuffer rate — how often playback stalls — is the one that matters most, because a stall is what members actually punish. Time to first frame determines whether a start is abandoned. Average delivered bitrate measures the quality actually achieved rather than the quality available. The three are in tension: pushing bitrate up improves the third and risks the first, which is exactly the trade the adaptive algorithm makes on every segment.
What makes these metrics powerful is the dimensions they are cut by. Reported per provider, per device type, per region and per title, they turn "video is bad tonight" into "this provider's subscribers in this city are rebuffering on 4K streams", which is an actionable statement about an appliance, a peering link or a device's decoder. The client reporting stream is, in effect, a global measurement network for the last mile.
What breaks first
- Peak evening bandwidth. The whole system is shaped around it. It is relieved not by more capacity at the origin but by better hit rates deeper in provider networks, and by codecs that reduce the bits needed per stream.
- The fill window. A big release must be copied to thousands of appliances before its launch, and the quiet hours are finite. Missing the window means serving a launch from origin, which is exactly what the architecture exists to avoid.
- Personalization latency. The home page has a hard budget on slow devices, which forces precomputation and caps how much ranking can happen at request time.
- Rights metadata lookups. Evaluated per member, per country, on every page render and every playback, and changing daily — a very high-volume lookup for a problem that is not technical in origin.
Recap
- A catalogue of twenty thousand titles watched by millions inverts the economics of user video: encode exhaustively in advance, and pre-position content rather than pulling it on demand.
- Per-title encoding tunes the ladder to the content, because a fixed ladder assumes all video is equally hard to compress; per-shot encoding extends it within a title.
- Bitrate is an input, not a result. A perceptual quality metric turns encoding into an optimisation with a real objective: maximum predicted quality per bit.
- Every codec generation justifies re-encoding the whole catalogue, because the saving applies to every future stream of every title.
- The delivery network is purpose-built appliances inside internet providers, because one content type and one workload do not need a general CDN.
- Proactive fill during local quiet hours converts an unpredictable peak-time fetch into a predictable off-peak copy, so the evening is almost pure cache hits.
- That only works because a catalogue is knowable in advance — the opposite of user-generated video, where pulling reactively is correct.
- Playback splits into a control plane that runs once — entitlement, rights, manifest, licence, steering — and a data plane that runs continuously and carries all the bytes.
- Steering returns a ranked list of servers rather than one, so the service contributes placement and load knowledge and the client contributes real measurements.
- The home page is a computed object: personalized rows, personalized ordering within rows, and personalized artwork, built by the same candidate-then-rank funnel as a feed.
- Optimise personalization for long-term retention rather than clicks, or the system learns to promote things people regret watching.
- Rights are per territory and per window, evaluated per member at request time and changing daily, so availability is a live input to every surface.
- Every dependency has a fallback and the page degrades in pieces; a member who sees a worse page does not notice, and one who sees an error does.
- Fallbacks that are never exercised do not work, which is the real argument for deliberately breaking things in production.
- Rebuffer rate, time to first frame and delivered bitrate are the metrics, cut by provider, device, region and title — which turns a vague complaint into a specific, fixable fact.
Questions
Try answering each one out loud before opening it. Lead with the one-line answer, then the reasoning, then one detail from a real system.
How does streaming a catalogue differ from streaming user-uploaded video?
Three properties invert the design: the catalogue is small and known in advance, every title is watched many times, and nothing arrives unannounced — so exhaustive encoding and proactive edge placement become correct, where user video needs lazy encoding and reactive caching.
- Encoding cost is amortised over millions of streams, so spending heavily per title pays for itself.
- Demand predictability means content can be copied to caches before anyone asks for it.
- Peaks are regional and nightly, which creates a daily trough to do that copying in.
Both designs are right for their own content; applying either one to the other's problem produces a bad system.
What is per-title encoding and why does it save bits?
Choosing the bitrate ladder per title rather than using one fixed ladder, because content differs enormously in how hard it is to compress — flat animation and grainy night footage need very different bitrates for the same perceived quality.
- A fixed ladder either wastes bits on easy content or under-serves hard content.
- The selection is made by encoding at many candidate settings and measuring the result.
- Per-shot encoding extends the same idea within a title, since complexity varies scene to scene.
The savings matter most to members on slow connections, who get a better picture at the bitrate they can actually sustain.
Why is bitrate not a measure of quality?
Because bitrate is an input to the encoder, not a property of the result: the same bitrate produces very different perceived quality depending on the content, the codec and the resolution.
- Simple signal-difference metrics correlate poorly with what humans actually notice.
- A perceptual metric trained on human opinion scores gives an objective the encoder can optimise against.
- With such a metric, encoding becomes a search: find the cheapest settings that reach a target quality.
Netflix open sourced VMAF for this purpose, and it is now a common industry objective.
Why build a delivery network instead of using a commercial CDN?
Because the workload is a single content type with predictable demand, which allows a purpose-built appliance placed inside internet providers and filled in advance — optimisations a general-purpose CDN serving arbitrary traffic cannot make.
- The appliance costs the provider rack space and saves them a large amount of peak transit traffic.
- Serving from inside the provider's network shortens the path and improves quality simultaneously.
- At this share of evening internet traffic, the economics justify the hardware programme.
It is a case where the most important engineering decision is a commercial relationship rather than a piece of software.
Explain proactive fill and why it is possible here.
Predicted popularity per region decides what each edge appliance should hold, and the content is copied during local quiet hours — so by the evening peak, what people are about to watch is already inside their provider's network.
- It uses capacity that would otherwise sit idle overnight.
- It converts an unpredictable peak-time origin fetch into a predictable off-peak copy.
- It is only possible because the catalogue is known and releases are scheduled — nothing goes viral unannounced.
A user-video platform cannot do this, which is precisely why it caches reactively instead.
What happens between pressing play and the first frame?
A control-plane call authorises the member, checks the rights window for their country, determines what the device can decode, and returns a manifest plus a ranked list of servers; the client then obtains a licence for the content keys and starts fetching segments.
- The control plane runs once per playback and carries no video, so it scales completely differently from delivery.
- The licence binds decryption to the device and is a separate service with its own availability requirements.
- Time to first frame determines abandonment, so this sequence is aggressively optimised and partly prefetched.
Separating the planes is what allows the byte path to be little more than a cache hierarchy.
Why return a ranked list of servers rather than a single one?
Because neither side has complete information: the service knows which appliances hold the title and how loaded they are, while only the client can measure what its own connection is actually doing.
- The client can fall back to the next entry when a server performs badly, without another round trip.
- It makes the delivery tier tolerant of an appliance failing mid-stream.
- It also lets the client contribute measurements that improve future steering decisions.
This split-responsibility pattern appears wherever a central service and an edge client each hold half the relevant facts.
How is the home page built?
As a two-level ranking problem: choose and order the rows, then choose and order the titles within each row, with a diversity constraint across the page — all built from precomputed candidates because the page must render fast on slow devices.
- Candidate sources include continue-watching, similar to recently finished titles, new arrivals and regional popularity.
- Even the artwork shown for a title is personalized, because the image measurably changes whether someone starts watching.
- The whole page is a computed object, not a catalogue view with a filter.
It is the same candidate-then-rank funnel as a social feed, applied to a much smaller item set with a much richer layout.
Why optimise recommendations for retention rather than clicks?
Because clicks measure whether something looked appealing, not whether it was worth watching — optimising for immediate engagement teaches the system to promote content people regret starting.
- Retention is a long-horizon metric, which makes experiments slower and more expensive to run.
- It requires testing everything and trusting no intuition, because short-term proxies mislead.
- Completion and repeat viewing are intermediate signals that correlate better than clicks.
The cost of the slower feedback loop is exactly what buys a recommender that does not degrade into clickbait.
Why are rights and availability a system design problem?
Because a title is licensed per territory and per time window, so availability must be evaluated per member, per country and per device at request time, and it changes daily as windows open and close.
- The catalogue is therefore not a global list with a filter — it is a different list in every country.
- Encodes may exist and be cached in a region where the title must not be served.
- It becomes one of the highest-volume metadata lookups in the system, on every page render and every playback.
It is a rare constraint that is entirely contractual in origin and entirely technical in consequence.
What does "every dependency has a fallback" mean in practice?
That each service call on a user-facing path has a defined behaviour when it fails or is slow: personalization unavailable renders a generic ordering, a slow row is omitted, and the page degrades in pieces rather than returning an error.
- Timeouts, circuit breakers and per-dependency bulkheads are what make the degradation fast rather than a hang.
- A member who sees a slightly worse page does not notice; one who sees an error does.
- The fallbacks must be cheap and independent, or they fail alongside the thing they replace.
Deciding the fallback in advance, per dependency, is what turns degradation into a design property rather than an accident.
What is the real argument for deliberately breaking things in production?
That an untested fallback does not work: it has drifted, or it was never correct, and an incident is the worst possible time to discover that. Exercising failure routinely is the only way to know the degraded path functions.
- Terminating instances and injecting latency during working hours means engineers are present to observe and fix.
- Regional failover becomes unremarkable precisely because it is practised often.
- The lesson transfers to any scale: the fallback you rely on should run on ordinary days.
It reframes resilience from a property you design once into a behaviour you verify continuously.
Which playback metrics matter, and how are they used?
Rebuffer rate above all, then time to first frame, then average delivered bitrate — reported continuously by clients and cut by provider, device, region and title.
- The three are in tension: pushing bitrate up improves perceived quality and risks stalling.
- Cutting by dimension turns a vague complaint into a statement about a specific appliance, peering link or device decoder.
- The client telemetry stream effectively becomes a global measurement network for last-mile conditions.
Choosing rebuffering as the primary metric encodes the product judgement that a stall is worse than a softer picture.
A major title launches on Friday night. What has to have happened by then?
Every rendition must be encoded, packaged and copied to the appliances in every region during the preceding fill windows, so that the launch is served entirely from cache rather than from origin.
- Missing the fill window means serving a launch from origin, which is exactly the traffic pattern the architecture exists to prevent.
- Quiet hours are finite, so a large catalogue addition has to be scheduled across several nights.
- Predicted regional popularity decides how widely it is placed.
The launch being boring operationally is the measure of whether the pre-positioning worked.
How would you decide whether to adopt a new codec?
By comparing the bits saved across all future streams against the cost of re-encoding the catalogue and the share of devices that can decode it — at this scale a few percent of bitrate is an enormous absolute saving, so adoption is usually worth it once device support is broad.
- The old codec must be retained for devices that cannot decode the new one, so storage grows during the transition.
- Encoding cost is one-off while the bandwidth saving recurs on every stream.
- Quality must be compared with a perceptual metric rather than at matched bitrates.
This is the clearest example of why keeping the master file forever is worth the storage.
What is the single judgement that shapes this design?
Never stall the stream. Everything else — row ordering, artwork choice, recommendation quality, even picture quality — may degrade, because members notice and punish a rebuffer far more than any of them.
- It explains why adaptation starts low and climbs, and why buffer protection outranks bitrate.
- It explains why the home page has fallbacks while playback has redundancy.
- It gives a clear ordering for what to sacrifice under stress, decided in advance rather than during the incident.
Every case study in this series has one such sentence, and finding it is most of what makes a design coherent.