Designing Google Drive
Content-defined chunking, deduplication, the metadata service, resumable uploads, the sync protocol, conflicts, sharing and permissions, versions and storage tiers.
On this page
File sync looks like a storage problem and is a metadata problem. Storing bytes durably and cheaply is a solved commodity; what is hard is keeping a mutable tree of billions of files consistent across a laptop that was offline for a week, a phone on a train and a shared folder that four people are editing — and doing it without ever losing a byte or silently overwriting somebody's work.
The map
Read this first when short on time. Every branch is a section below.
What it must do, and at what size
It must store a file durably, make it available on every one of a user's devices, let several people share and edit a folder, keep previous versions, and allow an accidental deletion to be undone. It must not, in this design, provide real-time collaborative editing of a single document — that is a different product built on operational transformation or CRDTs, and folding it in would obscure the file-level problem this article is about.
Users 1 billion registered, ~100 million daily active
Devices ~300 million devices syncing, mostly idle, long-polling
Files ~100 billion, average ~1 MB, heavily skewed
Storage multiple exabytes, growing ~100 TB/day
Metadata 100 billion rows x ~1 KB = ~100 TB of metadata alone
and it is the part that must be transactional
Uploads ~500 million file operations/day -> ~5,800/s average
Chunk size ~4 MB target -> a 1 GB file is ~250 chunks
Dedupe typically saves a third or more of raw bytesThe numbers make the shape obvious. The bytes are enormous but boring: immutable blobs, written once, read occasionally, stored on the cheapest durable medium available. The metadata is a hundred terabytes of small mutable rows that need transactions, ordering and a consistent tree — which is a far harder database problem than the exabytes are a storage problem. Nearly every design decision below is about protecting the metadata service.
flowchart TD C["client: watcher and local database"] --> API["sync API"] API --> MD[["metadata service: tree, versions, ACLs"]] API --> BS["block service"] BS --> OB[["object storage, erasure coded"]] MD --> NT[["change log per user"]] NT --> LP["long-poll notifier"] LP --> C OB --> CDN["edge cache for downloads"] CDN --> C
Chunks and hashes
Files are not stored as files. They are split into chunks, each named by the hash of its contents, and the file becomes a manifest: an ordered list of chunk hashes plus a little metadata.
That indirection buys four things at once. Uploads become resumable, because a connection that dies has still delivered the chunks it delivered. Transfers become parallel. Edits become incremental, because only changed chunks move. And identical data is stored once, because two identical chunks have the same name.
The interesting question is where to cut. Fixed-size chunking — every 4 MB — is trivial and has one bad failure: insert a byte at the start of a file and every subsequent boundary shifts, so every chunk changes and the whole file is re-uploaded. Content-defined chunking cuts at boundaries chosen by the data itself: a rolling hash slides over the file and a boundary is declared wherever the hash matches a pattern. An inserted byte then changes only the chunk containing it, and the boundaries downstream re-synchronise on their own.
Chunk sizes are targeted rather than exact — a minimum, a target around a few megabytes, and a maximum — because content-defined boundaries are probabilistic. Fixed-size chunking remains perfectly reasonable when files are mostly replaced rather than edited in place, which is true of photos and videos and false of documents and virtual machine images.
Deduplication, and why it is a privacy decision
Because a chunk is named by its hash, storing it twice is trivially avoidable. The question is the scope of that deduplication, and it is not purely an efficiency question.
Per-user dedupe is entirely safe and still valuable — the same attachment saved three times, the same folder copied. Global dedupe across all users saves far more, because popular files are genuinely popular, and it introduces a subtle leak: if the client asks "do you already have chunk h?" and the server says yes, the client has learned that somebody else stored that exact content. Uploading a candidate file and observing whether the upload is instant reveals whether that file already exists in the system.
Two defences. Require a proof of possession — a challenge over a random range of the chunk's bytes — so claiming a hash is not enough to obtain the data. Or encrypt per user, which makes cross-user dedupe impossible by construction and is the honest answer when confidentiality outranks cost.
Hash collisions are not a practical risk with a modern cryptographic hash, but truncating one to save bytes reintroduces the birthday bound. If a chunk's name is its identity, keep the full digest: silently serving the wrong chunk is the most catastrophic failure this system has.
The metadata service
This is the heart of the system: a transactional store holding, for each file, its id, name, parent, owner, size, version and chunk list — and, implicitly, the shape of the tree.
The single most important decision is that files are identified by id, not by path. A path is derived by walking parents; it is never stored. Renaming a folder with ten thousand files under it therefore updates one row rather than ten thousand, and a file keeps its identity through any number of moves — which is what lets sync resolve "this was edited on one device and moved on another" correctly instead of producing a duplicate.
files (shard key: drive_id)
file_id uuid, immutable identity
drive_id owning drive or user
parent_id folder, null at the root
name not unique globally, unique within a parent
version monotonic per file
size, mtime
manifest ordered list of chunk hashes
state active | trashed | deleted
changes (shard key: drive_id)
drive_id, seq monotonic per drive -> this is the sync cursor
file_id, type, versionThe tree brings two rules that must be enforced transactionally. A name must be unique within its parent, which is a uniqueness constraint on the pair rather than on the name. And a move must not create a cycle — making a folder its own grandchild would orphan an entire subtree, and it is a real race when two devices move two folders into each other simultaneously. That check has to walk the ancestry inside the transaction, which is one of the few genuinely expensive metadata operations.
Sharding is by drive: a user's own files live together, so nearly every operation is single-shard. The exception that complicates everything is a shared folder, which appears in several drives at once. The workable model is that the folder belongs to exactly one drive and others hold a reference to it, so there is still one authoritative copy of the subtree and one place its permissions live.
Uploading
The upload protocol falls straight out of chunking, and it is worth stating as a sequence because every property of it is deliberate.
sequenceDiagram autonumber participant C as Client participant M as Metadata participant B as Block service C->>C: chunk the file, hash each chunk C->>M: here are 250 chunk hashes, which do you need? M-->>C: I am missing 12 of them C->>B: upload the 12 chunks, in parallel, retryable B-->>C: stored C->>M: commit the manifest for this file id M-->>C: version 7, the file is now visible
Three properties follow. It is resumable without any explicit resume protocol: a client that restarts simply asks which chunks are missing again, and the answer reflects what already arrived. It is idempotent, because chunks are named by content — uploading the same chunk twice is a no-op. And the file appears atomically, because nothing is visible until the manifest commits, which is why the metadata service and the block service can be entirely separate systems with no distributed transaction between them.
The cost is orphaned chunks: data uploaded for a commit that never happened. They are collected by a background sweep that deletes chunks referenced by no manifest, which must be conservative — a chunk uploaded seconds ago may belong to a commit in flight, so the sweep only considers chunks older than some hours.
Sync
Sync is where the difficulty concentrates, because it is a distributed state machine running on hardware you do not control, and because users notice every mistake.
Detecting local change starts with a filesystem watcher and ends with a scan, because watchers miss events during sleep, network drives and permission failures. The client keeps a local database of what it believes the server state is, and compares. Timestamps are not trusted as evidence of change — clocks are wrong, editors preserve them, and copies alter them — so a candidate change is confirmed by hashing.
Receiving remote change uses a cursor. Each drive has a monotonically increasing change sequence; a device says "give me everything after sequence 40,112" and holds a long-poll connection open until something arrives. This makes the protocol trivially resumable after a week offline: the same request returns the accumulated changes.
sequenceDiagram autonumber participant D as Device participant N as Notifier participant M as Metadata D->>N: long poll, cursor 40112 Note over N: connection held open, no traffic while idle M->>N: change committed, seq 40113 N-->>D: wake up D->>M: give me changes after 40112 M-->>D: file moved, file edited, new cursor 40115 D->>D: apply in dependency order, then fetch missing chunks
Applying changes has an ordering requirement that is easy to get wrong: a folder must exist before a file can be placed in it, a move must be applied before a delete that follows it, and a rename that swaps two names needs a temporary. The robust implementation treats the change list as a dependency graph rather than a sequence, and applies what it can, retrying the rest — because the network may deliver a child's change before its parent's.
Conflicts
Two devices edit the same file while both are offline. There is no correct merge, and the product decision is more important than the algorithm.
For opaque files the answer is keep both: the second version to arrive is saved alongside the first with a name marking it as a conflicted copy, and the user decides. This is unsatisfying and it is right — a binary file cannot be merged, and silently discarding one person's work is the one outcome that destroys trust in a sync product permanently.
Detection uses versions rather than timestamps: each file has a monotonic version, and a client that submits an edit based on version 6 when the server is at version 7 has a conflict by definition. This is an optimistic concurrency check, and it is exact, unlike any comparison of clocks between machines.
The subtler conflicts are structural. A file edited on one device and moved on another is not really a conflict — resolving by file id applies both, which is the payoff for never storing paths. Two devices moving the same folder to different parents is a genuine conflict, resolved by a deterministic rule so that every device reaches the same answer independently. And a delete racing an edit is a policy choice: most products resurrect the file, on the grounds that recovering an unwanted file is easier than recovering a deleted one.
- Dropbox rewrote its sync engine around a model where every change is ordered per user and applied as a planned sequence, precisely because ad-hoc ordering produced conflicts nobody could reason about.
- Google Drive identifies files by id rather than path, which is why a file can live in a folder shared with you and keep working when someone moves it.
- Real-time document editing is a separate subsystem entirely, using operational transformation or CRDTs, and the file-level conflict model described here does not apply to it.
Sharing and permissions
Every read checks access, so permission evaluation is on the hottest path in the system and must be a fast, cached lookup rather than a tree walk.
The model is roles — owner, editor, commenter, viewer — granted on a file or a folder, inherited downwards, plus link-based grants with their own scope. Evaluating an inherited permission naively means walking to the root of the tree on every access, which is unacceptable at this rate, so the effective permissions are materialised: stored per file and recomputed asynchronously when a grant changes. A grant on a folder with a million descendants becomes a background job, which is the same fan-out problem as a feed, with the same solution.
Revocation must be fast, which is in tension with caching a decision. The usual compromise is short cache lifetimes — tens of seconds — plus an explicit invalidation broadcast on revoke, accepting a brief window in which a removed collaborator can still read. Any download URL handed to a client must be short-lived and scoped for the same reason.
The hard case worth naming: a folder shared with one group placed inside a folder shared with another. Whether the inner folder's permissions are replaced, unioned or rejected is a product decision with security consequences, and systems that leave it implicit end up with a surprising answer.
Versions, trash and storage tiers
Every commit creates a version. Because versions are manifests over shared chunks, keeping history is cheap — an edited document's new version shares almost all of its chunks with the old one, so the marginal cost is the chunks that changed. Retention is then a policy: keep everything for thirty days, keep named versions forever, prune the rest by age and count.
Deletion is soft: the file moves to trash, remains in quota, and is permanently removed after a timer. Restoring it needs the path it came from, which is one more reason the parent relationship is stored rather than derived from a path string.
Underneath, storage is tiered. Recently touched and actively shared files stay on fast storage; files untouched for months move to colder, cheaper tiers where retrieval costs money and takes longer. Durability comes from erasure coding rather than replication: splitting an object into data and parity fragments spread across failure domains gives better durability than three copies at roughly 1.5 times the bytes instead of three. For exabytes, that ratio is the difference between a viable business and an impossible one.
What breaks first
- Metadata write rate. Every upload, move, rename and permission change is a transaction, and the tree constraints make some of them expensive. Sharding by drive keeps almost everything single-shard, and the operations that span shards — shared folders — are the ones to watch.
- Sync fan-out on shared folders. A change in a folder shared with five hundred people produces five hundred change-log entries and five hundred notifications. Large shared folders behave like a small social network and need the same batching.
- Long-poll connections. Hundreds of millions of idle sockets, with the same reconnect-storm risk as any connection-heavy system.
- Cold retrieval. Cheap until someone restores a decade of archives, at which point the retrieval cost and latency of the cold tier become very visible. Rate limit restores and make the cost model explicit.
Recap
- The bytes are the easy part. A hundred terabytes of small, mutable, transactional metadata is harder than exabytes of immutable blobs, and most design decisions exist to protect the metadata service.
- Files are stored as chunks named by their content hash, with the file itself reduced to a manifest of hashes — which gives resumable uploads, parallel transfer, incremental edits and deduplication in one move.
- Fixed-size chunking breaks on insertion, because every later boundary shifts; content-defined chunking uses a rolling hash so boundaries re-synchronise and one edit costs one chunk.
- Global deduplication saves far more than per-user, and leaks whether a given file already exists. Proof of possession or per-user encryption are the defences.
- Identify files by id and derive paths, so renaming a folder is one row and a file keeps its identity through moves — which is what makes conflict resolution tractable.
- The tree needs transactional invariants: name uniqueness within a parent, and a cycle check on move that must run inside the transaction.
- Upload is ask-then-send-then-commit: the file becomes visible only when the manifest commits, so blocks and metadata need no distributed transaction and orphans are swept later.
- Sync uses a per-drive monotonic cursor with long polling, which makes a device returning after a week use exactly the same code path as one that never left.
- Apply incoming changes as a dependency graph rather than a sequence: parents before children, moves before deletes.
- Detect conflicts with versions, never timestamps, and resolve opaque-file conflicts by keeping both copies — silently discarding work is the failure users never forgive.
- Edit-plus-move is not a conflict when files have stable ids; concurrent moves of one folder are, and need a deterministic rule so every device converges.
- Permissions are inherited but materialised per file, with a grant on a large folder handled as a background fan-out, short cache lifetimes and explicit invalidation on revoke.
- Versions are cheap because they share chunks; deletion is soft and still counts against quota until the timer expires.
- Erasure coding gives better durability than three copies at about 1.5 times the bytes, which at exabyte scale is the difference between viable and not.
- The bottleneck ladder is metadata writes, shared-folder fan-out, long-poll connections, then cold-tier retrieval.
Questions
Try answering each one out loud before opening it. Lead with the one-line answer, then the reasoning, then one detail from a real system.
Why is file sync a metadata problem rather than a storage problem?
Because storing immutable blobs durably and cheaply is a commodity, while keeping a mutable tree of billions of files consistent across devices — with transactions, ordering, uniqueness and permission checks — is a hard database problem.
- A hundred billion files at about a kilobyte of metadata each is roughly a hundred terabytes that must be transactional.
- The blob path has no consistency requirements at all: chunks are written once and never change.
- Almost every design decision in the system exists to keep load off the metadata service.
This is why sync engines get rewritten and object stores rarely do.
Why split files into chunks at all?
Because it makes uploads resumable, transfers parallel, edits incremental and deduplication automatic — four properties from one indirection, with the file reduced to an ordered list of chunk hashes.
- A dropped connection has still delivered the chunks it delivered.
- Only changed chunks are re-uploaded when a file is edited.
- Naming a chunk by its hash makes storing it twice impossible by construction.
The manifest is also what makes a file appear atomically, since nothing is visible until it commits.
What is wrong with fixed-size chunking?
Inserting a byte near the start shifts every subsequent boundary, so every chunk gets a new hash and the entire file is re-uploaded despite only one byte changing.
- Content-defined chunking picks boundaries from the data using a rolling hash, so cut points re-synchronise after an edit.
- Chunk sizes then vary around a target, bounded by a minimum and maximum, because the boundaries are probabilistic.
- Fixed-size is still fine for files that are replaced rather than edited, such as photos and video.
Backup tools such as rsync and restic use the same rolling-hash idea for exactly this reason.
What is the privacy problem with global deduplication?
If the server tells a client it already has a chunk, the client learns that someone else stored that exact content — so uploading a candidate file and seeing whether it completes instantly reveals whether that file exists in the system.
- The defence is a proof of possession: challenge the client on a random byte range rather than trusting the hash alone.
- Per-user encryption removes the leak entirely and removes cross-user savings with it.
- Per-user deduplication is safe and still worthwhile.
It is a good example of an efficiency mechanism that turns out to be a security decision.
Why identify files by id rather than by path?
Because a path is a derived property: with ids, renaming a folder containing ten thousand files updates one row, and a file keeps its identity through any number of moves.
- Sync can then resolve "edited here, moved there" as two compatible changes rather than as a conflict.
- Storing paths would make every rename a mass update and every move ambiguous.
- Paths are reconstructed by walking parents when they need to be displayed.
It is also what lets a shared file keep working for collaborators when the owner reorganises their drive.
What invariants does the metadata service have to enforce transactionally?
Name uniqueness within a parent, and the absence of cycles when a folder is moved — a folder becoming its own ancestor would orphan an entire subtree.
- The cycle check has to walk the ancestry inside the transaction, which makes move one of the few expensive metadata operations.
- It is a genuine race: two devices moving two folders into each other simultaneously can each pass a naive check.
- Uniqueness is on the pair of parent and name, not on the name alone.
These constraints are why the metadata store needs real transactions rather than eventual consistency.
Walk through an upload.
The client chunks and hashes the file locally, asks the metadata service which of those hashes it lacks, uploads only the missing chunks to the block service in parallel, and finally commits the manifest — at which point the file becomes visible.
- It is resumable with no resume protocol: on restart the client simply asks again which chunks are missing.
- It is idempotent, because chunks are named by content and re-uploading one is a no-op.
- Blocks and metadata need no distributed transaction, since visibility is a single metadata commit.
Chunks uploaded for a commit that never happened are swept later, conservatively, so an in-flight commit is never collected.
How does a device find out that something changed?
Each drive has a monotonic change sequence, and a device long-polls with its cursor: "tell me when there is anything after 40112". The notifier wakes it, and the device fetches the changes and advances the cursor.
- The same request works for a device that has been offline for a week, which is what makes the protocol resumable by construction.
- The notifier carries no content, so it stays cheap enough to hold hundreds of millions of idle connections.
- Change entries are per drive, so a shared folder's change fans out into every member's log.
Polling on a timer would be simpler and would cost either latency or a vast number of empty requests.
Why can't the client just trust file modification timestamps?
Because clocks are wrong, some editors preserve timestamps when saving, copies alter them, and filesystem watchers miss events during sleep or on network volumes — so a timestamp is a hint, not evidence.
- A candidate change is confirmed by hashing the file and comparing to the stored manifest.
- The client keeps a local database of believed server state so it can diff rather than guess.
- A periodic full scan backstops the watcher, because the watcher will miss things.
Conflict detection uses version numbers for the same reason: they are exact, and clock comparisons between machines are not.
Two devices edit the same file offline. What happens?
The second one to submit is detected as a conflict — it was based on an older version — and both copies are kept, with one renamed as a conflicted copy for the user to resolve.
- Merging is impossible for opaque files, and silently discarding one side is the failure users never forgive.
- Detection uses monotonic version numbers rather than timestamps, which makes it exact.
- Real-time collaborative editing solves a different problem with operational transformation or CRDTs and does not apply here.
Keeping both is unsatisfying and correct, which is why every major sync product does it.
Which structural changes are genuinely conflicting?
Concurrent moves of the same folder to different parents, and a delete racing an edit. An edit on one device with a move on another is not a conflict at all when files have stable ids — both changes simply apply.
- Concurrent moves need a deterministic resolution rule so every device converges on the same tree independently.
- Delete versus edit is a policy choice, and most products resurrect the file because recovering an unwanted file is easier than recovering a deleted one.
- A rename that swaps two names needs a temporary name to apply safely.
That edit-plus-move is a non-conflict is the clearest payoff of identifying files by id.
How do you check permissions fast enough when every read needs one?
Materialise the effective permissions per file rather than walking to the root of the tree on every access, and recompute them asynchronously when a grant changes.
- Granting access to a folder with a million descendants becomes a background fan-out job, like a feed.
- Decisions are cached for tens of seconds with explicit invalidation on revoke, which leaves a small, deliberate window.
- Download URLs are short-lived and scoped, since they bypass the permission check once issued.
A shared folder placed inside another shared folder is the case to decide explicitly, because the implicit answer is usually surprising.
Why is keeping full version history affordable?
Because a version is a manifest over shared chunks, so a new version of an edited document shares nearly all its chunks with the previous one and costs only what actually changed.
- Retention is then a policy question — thirty days for everything, named versions forever, prune the rest.
- The same property makes trash cheap: a soft-deleted file is a state change, not a copy.
- Quota still counts trashed files, or deletion becomes a way to exceed it.
Content-addressed storage turning history into a near-free feature is the quiet payoff of the chunking decision.
Why erasure coding rather than three replicas?
Because splitting an object into data and parity fragments spread across failure domains gives higher durability than three copies while storing roughly 1.5 times the original bytes instead of three times.
- At exabyte scale the difference between 1.5x and 3x is the difference between a viable business and an impossible one.
- The cost is CPU on reconstruction and higher latency when a fragment is missing.
- Hot data may still be replicated, because reconstruction latency matters more there than bytes do.
Tiering follows the same logic: untouched files move to colder storage where retrieval is slower and cheaper.
A folder shared with 500 people changes. What does that cost?
Five hundred change-log entries and five hundred notifications for a single edit, which makes a large shared folder behave like a small social network with the same fan-out problem.
- Batching changes over a short window before notifying is the main defence.
- Membership should be capped or handled differently past a threshold, exactly as feed systems treat large accounts.
- The same fan-out applies to permission changes on the folder, which must reach every descendant.
It is the single most common reason a sync system that works fine for individuals struggles inside a large organisation.
What would you change to support real-time collaborative editing?
It would not be built on this file model at all: collaborative editing needs a per-document operation stream with operational transformation or CRDTs, not whole-file versions with conflicted copies.
- The unit of change becomes an operation on a data structure rather than a new immutable blob.
- Conflicts are resolved automatically by construction instead of being surfaced to the user.
- The file system still stores periodic snapshots, so the two models coexist rather than merging.
Keeping them separate is why a document in a sync folder and a document in a collaborative editor behave so differently.