Papers I actually recommend

The distributed-systems papers worth your time

Not an academic syllabus — the 26 papers that actually change how you build and how you interview. Each one comes with the production lesson and the edge it gives you in a loop. You don’t need to derive the proofs; read for the idea.

Every paper here is free — each link opens a public PDF or the author’s page, no sign-up. The 12 marked In the video are the core set — the ones I’d start with.

01

Foundations — storage engines & indexing

The two on-disk primitives every database is quietly built from.

ModerateACM Computing Surveys 1979

The Ubiquitous B-Tree

Douglas Comer · Purdue University · 1979

What it is
The survey that made the B-tree the default on-disk index — a shallow, high-fan-out tree that keeps data sorted and keeps lookups, range scans, and inserts all logarithmic, tuned to the reality that disks read in blocks.
The lesson
Nearly every relational database and file system indexes with a B-tree (really a B+ tree) for one reason: it minimizes disk seeks by matching node size to the storage block. Know this and you know why read-optimized stores are shaped the way they are.
The interview edge
The other half of the “B-tree vs LSM” fork every storage question eventually reaches. If you can say when a B-tree’s read-optimized, in-place-update design wins, you’re defending a datastore choice instead of guessing.
Read the paperPDF · carlosproal.com
ModerateActa Informatica 1996

The Log-Structured Merge-Tree (LSM-Tree)

O'Neil, Cheng, Gawlick, O'Neil · UMass Boston · 1996

What it is
The paper that named the log-structured merge-tree: buffer writes in memory, flush them as immutable sorted files, and merge those files in the background — trading read effort for hugely cheaper writes.
The lesson
The write-optimized half of storage. Turning random writes into sequential ones is why Bigtable, Cassandra, and RocksDB absorb enormous write volume — at the cost of compaction and read amplification you have to manage.
The interview edge
The “why is this write-heavy database fast, and what does it cost you” answer. Naming memtables, SSTables, compaction, and the read-amplification tradeoff is a senior storage signal.
Read the paperPDF · author’s site
02

Distributed storage at scale

Google and Amazon build the data plane — store it, replicate it, stay available.

In the videoApproachableSOSP ’03

The Google File System

Ghemawat, Gobioff, Leung · Google · 2003

What it is
A distributed file system for very large data on cheap, failure-prone commodity hardware — a single master for metadata plus many chunkservers, optimized for huge files and append-heavy writes.
The lesson
Design for failure as the normal case, not the exception. Relax the interface (large chunks, append semantics) to make the distributed problem tractable — a masterclass in trading a general API for scalability.
The interview edge
The blueprint behind every “design a distributed blob / file store.” If you can explain the master/chunkserver split, replication, and why appends win, you can reason about S3, HDFS, and Colossus.
Read the paperPDF · research.google
ApproachableIEEE MSST ’10

The Hadoop Distributed File System

Shvachko, Kuang, Radia, Chansler · Yahoo! · 2010

What it is
The open-source retelling of GFS: a NameNode holding the file-system metadata and a fleet of DataNodes holding replicated blocks — the storage layer under the entire Hadoop ecosystem.
The lesson
Read next to GFS, it shows how a research design becomes production infrastructure the whole industry runs — the same master/worker split, replication, and rack-awareness, with the operational details filled in.
The interview edge
The concrete, still-deployed version of the “design a distributed file store” answer. Pairing it with GFS shows you can trace an idea from paper to production, not just recite one.
Read the paperPDF · cs.wisc.edu
In the videoModerateOSDI ’06

Bigtable: A Distributed Storage System for Structured Data

Chang et al. · Google · 2006

What it is
A distributed, sparse, sorted wide-column store built on GFS — memtables, SSTables, and tablets served by tablet servers, coordinated by a master.
The lesson
The LSM-tree + SSTable pattern behind most modern NoSQL (HBase, Cassandra, LevelDB/RocksDB). Understand why write-optimized stores buffer in memory and flush immutable sorted files — and the compaction cost that follows.
The interview edge
Lets you speak precisely about the read/write path of a NoSQL store and when a log-structured design beats a B-tree — a common system-design fork.
Read the paperPDF · research.google
In the videoModerateSOSP ’07

Dynamo: Amazon's Highly Available Key-value Store

DeCandia et al. · Amazon · 2007

What it is
An always-writable key-value store that chooses availability over strong consistency: consistent hashing, replication, quorums (R + W > N), vector clocks, and hinted handoff.
The lesson
The clearest real-world treatment of the availability-vs-consistency tradeoff. Eventual consistency is a design choice with concrete mechanisms — not a hand-wave. The paper behind DynamoDB and Cassandra.
The interview edge
The AP corner of CAP, made concrete. Being able to tune N/R/W and explain conflict resolution separates strong system-design candidates from vague ones.
Read the paperPDF · allthingsdistributed
ModerateSIGOPS OSR ’10

Cassandra: A Decentralized Structured Storage System

Lakshman, Malik · Facebook · 2010

What it is
Facebook's decentralized store, built by grafting Dynamo's availability model — consistent hashing, replication, tunable quorums — onto a Bigtable-style column data model and an LSM storage engine.
The lesson
The clearest example of composition in systems design: two landmark papers become one production database. It’s why Cassandra has no single master and stays writable through failure, and why its data modeling is query-first.
The interview edge
The living synthesis of Dynamo and Bigtable. If you can name which half gives Cassandra availability and which gives it its storage engine, you’ve shown you understand both parents at once.
Read the paperPDF · cs.cornell.edu
ApproachableUSENIX FAST ’23 · keynote

Building and Operating a Pretty Big Storage System (My Adventures in S3)

Andy Warfield · Amazon · 2023

What it is
A FAST ’23 keynote on building and operating Amazon S3 — durability as a first principle, the discipline of running a system at trillions of objects, and why the careful, boring parts are the hard parts.
The lesson
A rare public look at operating storage at true scale from the inside: how you reason about durability, correctness, and constant background verification when “it basically never loses data” has to be literally true.
The interview edge
The senior framing for any object-store or durability question — eleven-nines thinking and the culture of operating a critical system, not just its architecture. Reads in an afternoon and reframes how you talk about reliability.
Read the paperessay · allthingsdistributed
03

Processing the data

From batch, to in-memory, to streaming, to the log as the source of truth.

In the videoApproachableOSDI ’04

MapReduce: Simplified Data Processing on Large Clusters

Dean, Ghemawat · Google · 2004

What it is
A programming model — map, shuffle, reduce — that lets ordinary code run over thousands of machines, with the framework handling partitioning, scheduling, retries, and stragglers.
The lesson
A good abstraction hides distribution. The engineering win isn’t the algorithm; it’s that fault-tolerance and parallelism become the framework’s job, not yours. The ancestor of Hadoop and Spark.
The interview edge
Teaches you to decompose a batch problem into data-parallel stages and to reason about shuffle cost — the exact instinct “process this huge dataset” questions reward.
Read the paperPDF · research.google
ModerateNSDI ’12

Resilient Distributed Datasets (Spark)

Zaharia et al. · UC Berkeley · 2012

What it is
The abstraction that made cluster computing fast: resilient distributed datasets — immutable, partitioned collections that track their lineage, so lost partitions are recomputed instead of checkpointed and data can stay in memory across steps.
The lesson
The leap past MapReduce. Keeping data in memory and rebuilding from lineage on failure is why iterative and interactive workloads got orders of magnitude faster — the shift that ate MapReduce’s lunch.
The interview edge
Explains why modern pipelines aren’t raw MapReduce, and gives you the vocabulary — lineage, in-memory caching, wide vs narrow dependencies, the shuffle — for any large-scale processing question.
Read the paperPDF · usenix.org
DenseVLDB ’15

The Dataflow Model

Akidau et al. · Google · 2015

What it is
Google’s unified model for stream and batch: separate what you compute from when results are emitted, using event-time windows, watermarks, and triggers to handle late and out-of-order data correctly.
The lesson
It dissolves the batch-vs-streaming split — batch is just a bounded stream. Event-time processing, watermarks, and the correctness-vs-latency tradeoff underpin Beam, Flink, and modern streaming everywhere.
The interview edge
The framework for any “process events in real time” question. Reasoning about event time vs processing time, windows, and late data separates a real streaming answer from “I’d use Kafka and figure it out.”
Read the paperPDF · vldb.org
In the videoApproachableNetDB ’11 · essay ’13

Kafka + “The Log: Real-Time Data’s Unifying Abstraction”

Kreps, Narkhede, Rao · LinkedIn · 2011

What it is
A distributed, partitioned, replicated commit log for high-throughput messaging — and Jay Kreps’ companion essay arguing the append-only log is the backbone of all data systems.
The lesson
Reframes integration, streaming, and change-data-capture as one idea: an ordered, replayable log. Once you see systems as logs + materialized views, event-driven architecture stops being mysterious.
The interview edge
Log-based vs queue-based messaging, partitioning for ordering, consumer offsets — the vocabulary for any “design a real-time / event-driven system.”
Read the paperPDF · Microsoft Research mirror
04

Getting machines to agree

Consensus and coordination — the hard problem, made usable.

ModerateCACM 1978

Time, Clocks, and the Ordering of Events in a Distributed System

Leslie Lamport · Massachusetts Computer Associates · 1978

What it is
Lamport’s foundational result: in a distributed system there is no single “now,” so order must come from causality. It defines the happens-before relation and logical clocks that order events without synchronized time.
The lesson
The why under everything else here. Once you accept that you can’t trust wall clocks and must reason about causality, replication, consistency, and consensus stop being magic and start being consequences.
The interview edge
Small paper, enormous leverage. It reframes every “how do two nodes agree on order” moment — and it’s why Spanner treating time as a bounded, measurable thing is such a big deal.
Read the paperPDF · author’s site
In the videoDenseSIGACT News 2001

Paxos Made Simple

Leslie Lamport · Microsoft Research · 2001

What it is
The canonical (and famously subtle) algorithm for getting independent, failure-prone nodes to agree on a single value — proposers, acceptors, and majority quorums.
The lesson
The foundation under Chubby, Spanner, and every replicated state machine. You may never implement raw Paxos, but internalizing “agreement under failures needs overlapping majorities” is core systems literacy.
The interview edge
Builds the mental model for why consensus is hard — so you can reason about split-brain, quorums, and why an even number of nodes is a bad idea.
Read the paperPDF · author’s site
In the videoApproachableUSENIX ATC ’14

In Search of an Understandable Consensus Algorithm (Raft)

Ongaro, Ousterhout · Stanford · 2014

What it is
Consensus redesigned for humans: decomposed into leader election, log replication, and safety — explicitly optimized to be understandable and implementable.
The lesson
The consensus algorithm you will actually meet in production — etcd, Consul, TiKV, CockroachDB. If you learn one consensus protocol well enough to explain on a whiteboard, make it Raft.
The interview edge
The single most interview-useful distributed-systems paper: leader election, log matching, and commit rules are directly askable and directly explainable.
Read the paperPDF · raft.github.io
In the videoModerateOSDI ’06

The Chubby Lock Service for Loosely-Coupled Distributed Systems

Burrows · Google · 2006

What it is
A highly-available lock and small-file (config) service — Paxos packaged as a product. Coarse-grained locks, leader election, and a filesystem-like API used across Google.
The lesson
Consensus is hard, so centralize it once and expose it as a service everything else leans on. Teaches where a lock actually belongs in a distributed system, and the cost of coordination. The ancestor of ZooKeeper and etcd.
The interview edge
When a design needs “exactly one leader” or “who owns this shard,” Chubby is the reference answer for how coordination is done in practice.
Read the paperPDF · research.google
ModerateUSENIX ATC ’10

ZooKeeper: Wait-free Coordination for Internet-scale Systems

Hunt, Konar, Junqueira, Reed · Yahoo! · 2010

What it is
An open, general coordination service — a replicated, in-order hierarchy of small data nodes with watches — giving applications leader election, config, and locks over a simple API, backed by its own atomic-broadcast protocol (Zab).
The lesson
The Chubby idea made open and ubiquitous: centralize coordination once and let everything lean on it. It’s the coordination layer under Kafka, HBase, and countless clusters — the tool teams actually reach for.
The interview edge
When a design needs leader election, service discovery, or distributed locks, ZooKeeper (or etcd) is the concrete answer. Knowing what it guarantees — and never to put it on the hot path — is real operational depth.
Read the paperPDF · usenix.org
05

Distributed databases & global consistency

Transactions, strong consistency, and buying your way across the CAP tradeoff.

In the videoDenseOSDI ’12

Spanner: Google's Globally-Distributed Database

Corbett et al. · Google · 2012

What it is
A globally-distributed, externally-consistent SQL database. Its trick is TrueTime — an API that exposes bounded clock uncertainty so transactions can be globally ordered.
The lesson
Strong consistency across continents is possible — if you turn time itself into a systems primitive and pay for it (commit-wait). Shows that “just use a distributed transaction” has a real, physical cost.
The interview edge
The high-water mark for the consistency-vs-latency tradeoff. Naming TrueTime and external consistency signals genuine distributed-systems depth.
Read the paperPDF · research.google
In the videoModerateSIGMOD ’17

Amazon Aurora: Cloud-Native Relational Databases

Verbitski et al. · Amazon · 2017

What it is
A cloud-native MySQL/Postgres-compatible database that pushes redo logging into a distributed, multi-tenant, self-healing storage layer — “the log is the database.”
The lesson
The modern move: decouple compute from storage and ship only the log, not pages, over the network. Explains why Aurora scales reads and survives failure the way a classic RDBMS can’t.
The interview edge
The current-generation answer to “how would you build a scalable relational database in the cloud” — compute/storage separation is a reusable design lever.
Read the paperPDF · stanford.edu
ModerateOSDI ’10

Large-scale Incremental Processing Using Distributed Transactions (Percolator)

Peng, Dabek · Google · 2010

What it is
Google’s system for incrementally updating a massive dataset — it layers cross-row, cross-table ACID transactions and an observer/notification model on top of Bigtable, so the web index could update continuously instead of by full rebuild.
The lesson
How to add distributed transactions to a store that didn’t have them, using a two-phase commit coordinated through the data itself and a central timestamp oracle. The transaction model TiDB later adopted wholesale.
The interview edge
The reference for “add transactions to a NoSQL store” and for incremental-vs-batch processing. Explaining its two-phase commit signals you understand distributed transactions, not just single-key ops.
Read the paperPDF · usenix.org
06

Serving the read path

Cache, graph, authorization, time-series — the systems that answer reads fast.

ModerateUSENIX ATC ’13

TAO: Facebook's Distributed Data Store for the Social Graph

Bronson et al. · Facebook · 2013

What it is
A read-optimized store for the social graph — objects and associations served from a giant, geographically distributed cache in front of sharded MySQL, tuned for a workload that is overwhelmingly reads.
The lesson
The canonical answer to serving a huge, interconnected, read-heavy dataset: model it as a graph, cache aggressively, and accept carefully-bounded staleness for availability and latency.
The interview edge
The blueprint for “design the news feed / social graph / friends-of-friends.” Objects-and-associations, read-through caching, and the read:write skew are exactly what these rounds probe.
Read the paperPDF · usenix.org
ModerateNSDI ’13

Scaling Memcache at Facebook

Nishtala et al. · Facebook · 2013

What it is
How Facebook turned a simple single-node cache into a distributed caching layer serving billions of requests a second — covering the hard parts: the thundering herd, stale sets, cross-region invalidation, and cache consistency.
The lesson
The real engineering isn’t the cache; it’s keeping a distributed cache correct under failure and replication. Leases, careful invalidation, and “the database is the source of truth, the cache is a hint” are the durable lessons.
The interview edge
Caching is in almost every design, and the hard parts live here — invalidation, stampede protection, read-your-writes across regions. It’s how you move past “just add a cache.”
Read the paperPDF · usenix.org
ModerateUSENIX ATC ’19

Zanzibar: Google's Consistent, Global Authorization System

Pang et al. · Google · 2019

What it is
Google’s global authorization system — the service that answers “can this user do this to this object?” consistently, at scale, for YouTube, Drive, and more — using a relationship-based model and a consistency token to avoid the “new-enemy” problem.
The lesson
Authorization is a distributed-systems problem, not an if-statement. Modeling permissions as a graph of relationships, and making stale reads safe with a consistency token, is a pattern the whole industry is now copying (SpiceDB and others).
The interview edge
Increasingly asked, and few can answer it well. “Design permissions / access control at scale” now has a real reference — relationship tuples, check evaluation, and why consistency matters for authz.
Read the paperPDF · usenix.org
ModerateVLDB ’15

Gorilla: A Fast, Scalable, In-Memory Time Series Database

Pelkonen et al. · Facebook · 2015

What it is
Facebook’s in-memory time-series database, built to make monitoring fast: aggressive delta-of-delta timestamp and XOR float compression pack recent data into memory so queries over billions of points return in milliseconds.
The lesson
A masterclass in fitting the storage engine to the workload. Monitoring data is written constantly and read recently, so Gorilla optimizes ruthlessly for that shape — the ideas behind Prometheus’s and many TSDBs’ compression.
The interview edge
The answer to “design a metrics / monitoring system.” Talking time-series compression, hot in-memory data with a durable tier, and query latency shows you design for a specific workload, not a generic store.
Read the paperPDF · vldb.org
07

Running it in production

The part interviews under-test and production over-tests: keeping it alive.

In the videoApproachableGoogle Tech Report ’10

Dapper, a Large-Scale Distributed Systems Tracing Infrastructure

Sigelman et al. · Google · 2010

What it is
How Google traces a single request as it fans out across hundreds of services — trace context propagation, spans, and low-overhead sampling.
The lesson
You cannot fix latency you cannot see. Dapper is why distributed tracing exists and the direct lineage of Jaeger, Zipkin, and OpenTelemetry — observability is a first-class design concern, not an afterthought.
The interview edge
Add “and here’s how I’d observe it — trace IDs, spans, sampling” to any microservices design and you instantly sound like someone who has operated one.
Read the paperPDF · research.google
In the videoApproachableCACM ’13

The Tail at Scale

Dean, Barroso · Google · 2013

What it is
Why large fan-out services are only as fast as their slowest component, and the techniques — hedged and tied requests, micro-partitioning, good-enough replicas — that tame p99 latency.
The lesson
Averages lie; the tail is the user experience. When one request touches hundreds of servers, rare slowdowns become common — so latency must be engineered, not hoped for. The mental model behind every latency-SLO conversation.
The interview edge
Say “p99, not average,” reason about fan-out amplifying tail latency, and reach for hedged requests, and you sound like someone who has run a service at scale — exactly the senior signal loops reward.
Read the paperPDF · barroso.org

How the canon gets built today

The papers are the foundation. Company engineering blogs are the same ideas deployed a decade later — read them for the why they chose X over Y, not the diagrams. Each one is anchored to a system-design topic you’ll actually be asked about.

The big-tech canon

A dozen deep-dives, each anchored to one interview topic — read for the tradeoff, not the diagram.

Fundamentals, by named principal engineers

The closest thing to a free distributed-systems textbook a company publishes.