Designing a news feed

A timeline is a list of posts from accounts you follow, which sounds like a database query and is not one. The query would be "the most recent posts from these two hundred accounts, ranked", run five billion times a day, over a corpus where one account has a hundred million followers and most have thirty. Everything in this article follows from a single question: is that list computed when somebody posts, or when somebody looks?

The map

Read this first when short on time. Every branch is a section below.

What it must do, and at what size

It must let a user post, follow and unfollow accounts, read a ranked timeline with infinite scroll, and see their own post the instant it is made. It must not, in this design, support editing a post after publication, threaded conversations, or a strict guarantee that two users see the same feed — a ranked feed is personal by definition, which removes a whole class of consistency requirement.

Users       500M daily active, ~200M with a materialised feed
Posts       100M/day  -> 1,150/s average, ~3,500/s peak
Feed views  5B/day    -> 58,000/s average, ~175,000/s peak
Follows     ~200 on average, power law: median ~100, maximum ~100M followers

Fan-out cost  100M posts x 200 followers = 20 billion feed writes/day
              -> ~231,000/s average, ~700,000/s peak
One celebrity post to 100M followers = 100M writes
              -> 140 seconds of the entire fan-out budget, for one post

Feed entry  post id 8 B + author id 8 B + score 8 B = 24 B
Feed size   400 entries x 24 B ~= 9.6 KB per user
Feed store  200M active users x 9.6 KB ~= 1.9 TB in memory

Three numbers decide the architecture. The read-to-write ratio is about fifty to one on posts but the fan-out multiplies writes by the follower count, so the effective write load is two hundred thousand a second against a read load of sixty thousand — the only system in this series where precomputation makes the write side the expensive one. The feed store is about two terabytes, which fits in memory across a few dozen machines. And one post can cost a hundred million writes, which is the fact that breaks the simple design.

flowchart TD
  P["user posts"] --> PS["post service"]
  PS --> PD[["post store, sharded by post id"]]
  PS --> EV[["fan-out log, partitioned by author"]]
  EV --> FW["fan-out workers"]
  GR[["social graph"]] --> FW
  FW --> FS[["feed store: per-user id lists"]]
  R["user opens the app"] --> FR["feed service"]
  FR --> FS
  FR --> CP["pull posts from large accounts"]
  FR --> RK["ranker"]
  RK --> HY["hydrate: posts, authors, counters"]
  HY --> PD

The central choice: compute on write or on read

Everything else is detail. There are exactly two ways to produce a timeline, and they place the same work at opposite ends.

flowchart TB
  subgraph W["Fan-out on write"]
    direction LR
    W1["post once"] --> W2["append to 200 follower feeds"]
    W2 --> W3["read is one lookup"]
  end
  subgraph R["Fan-out on read"]
    direction LR
    R1["post once"] --> R2["store once"]
    R2 --> R3["read merges 200 authors"]
  end
  W ~~~ R

Fan-out on write materialises a list per user. When someone posts, a worker appends the post id to each follower's feed. Reading is then one range read of a sorted list — a millisecond, regardless of how many accounts the user follows. The cost is that the write amplification is the follower count, and the follower count is a power law.

Fan-out on read stores each post once and merges at read time: fetch the recent posts of each followed account, merge, rank, return. Posting is one write. Reading is two hundred lookups and a merge, on the path a user is waiting on, five billion times a day.

The last row is the whole problem. Pure write fan-out cannot survive a celebrity: a hundred million appends for one post is not just expensive, it is a burst that delays every other user's feed behind it. Pure read fan-out cannot survive a normal view: two hundred store reads and a merge at a hundred and seventy-five thousand views a second is an enormous, constant cost for work that is mostly identical between one view and the next.

The hybrid, and where the line goes

Every production feed is a hybrid: push for ordinary accounts, pull for the few enormous ones, and merge the two lists at read time.

sequenceDiagram
  autonumber
  participant U as Client
  participant F as Feed service
  participant S as Feed store
  participant P as Post store
  U->>F: GET /feed
  F->>S: range read of the materialised feed
  S-->>F: 400 post ids from ordinary accounts
  F->>P: recent posts from the 6 large accounts followed
  P-->>F: about 120 more post ids
  Note over F: merge, rank, filter blocked and seen
  F->>P: batch get the top 20 posts and their authors
  P-->>F: hydrated posts
  F-->>U: one page of feed

The threshold is a tuned number, not a principle. Around a hundred thousand followers is a common setting, and the way to choose it is to compare costs: pushing to F followers costs F writes once, while pulling costs one read per view by each of those followers. An account whose followers collectively view the feed more often than the account posts is cheaper to push; one that posts often to an audience that rarely looks is cheaper to pull. In practice that puts the line where the follower count is large enough that the burst itself is the problem.

The honest cost of the hybrid is two code paths that must agree. A post from a large account must not also arrive by fan-out, or it appears twice; a user who follows both kinds sees a merge that has to deduplicate and rank across two sources; and an account crossing the threshold in either direction needs a careful transition. It is worth it, and it should be stated as a cost rather than hidden.

Storing feeds

A materialised feed is a sorted list of ids — never post content. Storing the text would multiply the corpus by the average follower count and make every edit or deletion a fan-out of its own.

feed:{user_id}   sorted set, score = ranking score or timestamp
  member: post_id (8 B) | author_id (8 B) | score (8 B)  ~= 24 B per entry
  capped at ~400 entries, trimmed on insert

Why 400: the 99th percentile session reads far fewer, and anything
older is served by paging into the source rather than from the feed.

Two decisions keep this affordable. The list is capped, because a feed is a cache of the near future, not an archive — a user who scrolls past the cap falls through to a slower path that queries the source directly, and almost nobody does. And feeds are materialised only for active users: keeping a feed for every account that ever registered would double or triple the memory for users who will never look. A returning user's feed is rebuilt on demand from their follow list, which takes a second and happens once.

The store itself is memory-backed for the hot path with a durable copy behind it. Losing the in-memory feeds is survivable — they can be rebuilt — but rebuilding two hundred million of them at once is not something to do during an incident, which is the argument for persistence here.

The graph and the posts

Two stores sit underneath the feed, and both have a shape worth being deliberate about.

The social graph is stored twice: a following list sharded by the follower, and a followers list sharded by the followed account. The duplication is deliberate, because the two queries are different — "who do I follow" is read on every feed rebuild, and "who follows this account" is read on every post — and neither can be served efficiently from the other's layout. The followers list of a large account is enormous and is paged through by fan-out workers rather than loaded.

The post store is immutable rows sharded by post id, using time-sortable ids so that a range of ids is a range of time and no secondary index on timestamp is needed. Deletion is a tombstone rather than a removal: feeds all over the system hold the id, and the cheapest way to make it disappear is to filter it during hydration.

Write amplification

How many writes one logical action causes. A post with 200 followers has an amplification of 200; a celebrity post, 100 million.

Materialised feed

A precomputed per-user list of post ids, maintained by fan-out and trimmed to a cap.

Hydration

Turning a list of ids into renderable objects by batch-fetching posts, authors, counters and viewer state.

Fan-out workers

Fan-out is asynchronous, and that is a requirement rather than an optimisation: if posting waited for the writes to complete, post latency would be proportional to follower count, and a popular account could not post at all.

flowchart TD
  P["post committed"] --> E[["fan-out log, partitioned by author id"]]
  E --> W["worker pool"]
  W --> G[["page through the followers list"]]
  G --> B["batch of 1,000 follower ids"]
  B --> F[["append to each feed, trim to the cap"]]
  B --> N["next batch until the list ends"]
  N --> G

Three properties fall out of this design. Delivery is at-least-once, so an append must be idempotent — a sorted set keyed by post id is naturally so, which is a good reason to prefer it over a list. Lag shows up as feed staleness rather than as an error, which is exactly the failure mode to want: a backlog delays posts appearing in feeds, and nothing breaks. And the work is prioritisable: when a backlog forms, small accounts should be drained first, because a ten-follower post completes instantly while a ten-million-follower post would otherwise block it for minutes.

Ranking

A chronological feed needs no ranking, and almost no product ships one by default any more. Ranking turns the feed from a merge into a retrieval problem, structured as a funnel because scoring is expensive.

flowchart TD
  C["candidates: ~2,000 posts from feed store and pulls"] --> L["light ranker: cheap features, keep ~300"]
  L --> H["heavy ranker: model call, keep ~50"]
  H --> D["diversity and dedupe: no 3 posts from one author"]
  D --> F["filters: blocked, muted, already seen"]
  F --> P["page of 20 returned"]

The signals that matter are affinity with the author (how often this viewer interacts with them), recency with a decay so that a day-old post cannot outrank a fresh one indefinitely, predicted engagement from a model, and diversity so that one prolific account cannot fill the page. Features are mostly precomputed — an affinity table updated hourly is good enough — with a few real-time ones such as "posted in the last five minutes" or "the viewer just followed this account".

The serving rule that matters most: score a batch, not an item. A model call per post is hundreds of calls per feed view; a single call scoring three hundred candidates is one. The same applies to feature lookups, which should be one batched read per feature store rather than one per candidate.

Ranking also changes the feed store's contract. If the score is computed at read time, the stored list needs only recency ordering and the ranker re-sorts; if the score is computed at write time, the feed can be stored pre-sorted but must be rewritten whenever the model changes. Most systems keep the store chronological and rank on read, because it decouples model deployments from the data.

Hydration, and the calls you must not make

After ranking, the page is twenty ids. Turning them into something renderable needs the post bodies, the authors, the media URLs, the like and reply counts, and whether this viewer has liked or saved each one.

The rule is one batched call per entity type: one multi-get for twenty posts, one for the distinct authors, one for the counters, one for viewer state. That is four or five round trips for the whole page. The failure mode to avoid is the loop that fetches an author per post — a hundred round trips, a tail latency disaster, and the single most common performance bug in feed code.

Hydration should also degrade rather than fail. If the counter service is unavailable, render without counts. If viewer state is slow, render the posts and fill in the like state a moment later. The feed is a composite of many services, and treating any one of them as required makes the feed's availability the product of all of them.

Counters and viewer state

Like and reply counts look trivial and are the most hostile write pattern in the system: a viral post receives tens of thousands of increments a second against a single row.

The standard fix is a sharded counter: split the count into k sub-counters, have each writer increment one at random, and sum them on read. Writes spread across k keys; reads cost k small lookups, which is fine when k is around ten and the result is cached for a second or two.

Beyond a certain size, exactness stops being worth anything: nobody can tell 2.4 million likes from 2.41 million, so counts above a threshold can be rounded, cached for longer, and updated from a stream aggregate rather than transactionally. The one place exactness still matters is the author's own view of their own post, which is a single user and can afford an exact read.

Viewer state — has this user liked, saved or hidden each post — is a different shape: small, per viewer, and needed for exactly the twenty posts on the page. One batched lookup keyed by (viewer, post ids) answers it, and it is a good candidate for a short-lived cache because the same user reloading their feed asks the same question.

Correctness: the cases people forget

A feed is allowed to be stale. It is not allowed to be wrong, and the difference lives in four specific cases.

Read your writes. A user who posts must see their post immediately, but asynchronous fan-out means it may be seconds away. Two fixes, and most systems use both: write to the author's own feed synchronously before returning, and have the client optimistically render the post it just submitted.

Deletes. A deleted post's id remains in millions of feeds, and hunting it down is not feasible. Tombstone the post and filter it during hydration: the feed entry becomes a dangling id that resolves to nothing and is skipped. Feeds then contain a few dead entries, which is why the page is over-fetched slightly before trimming to twenty.

Unfollow. The new expectation is that future posts stop appearing; the past is genuinely ambiguous. The cheap and common behaviour is to leave existing entries in the feed and let them age out, because rewriting a materialised feed on every unfollow is expensive and nobody notices.

Blocks. Unlike unfollow, a block must take effect on existing entries too — seeing a post from someone you blocked is a trust failure rather than a staleness one. This is the case that forces filtering at read time rather than at fan-out time.

What breaks first

1. Fan-out write throughput. Two hundred thousand feed writes a second at average load, and bursty on top. It is the first thing to saturate, and the symptom is feed lag rather than errors. Raising the pull threshold is the fastest relief, because it removes the largest accounts from the write path entirely.
2. Feed store memory. Two terabytes grows with active users and with the cap. Shrinking the cap is a cheap lever with almost no user-visible effect.
3. Ranking cost. Latency and money both. The funnel exists to bound it, and the levers are a smaller candidate set, a cheaper light ranker, and caching a ranked page for the few seconds in which a user is likely to reload.
4. Counter hot keys. Independent of everything else and triggered by a single viral post. Sharded counters and read caching handle it; the failure to plan for it is what makes a viral moment an incident.

At ten times the scale nothing about the shape changes, which is the point: raise the threshold, shrink the cap, add fan-out workers and feed shards, and cache ranked pages. The design's real load-bearing decision was made in the third section, and everything after it is arithmetic.

Recap