Consensus Theory

Recently, I studied a series of influential papers regarding consensus. Consensus under non-Byzantine circumstance is the theory backing up distributed systems such as Chubby (Paxos), etcd (Raft) and Zookeeper (Zab). These algorithms, together with their correctness reasoning are hard to interpret but worth the effort.

1978 Time, Clocks, and the Ordering of Events in a Distributed System
Leslie Lamport

This paper, brought by Leslie Lamport, the Turing award winner in 2013, won ACM SIGOPS Hall of Fame Award (2007) and Dijkstra award (2000). It introduced partial order and global order in distributed environment, which greatly influenced happens-before concept in multi-threaded program. The vector clock presented in the paper also inspired multithreading race detection of Golang race detector, whose paper is at Vector clock, How Developers Use Data Race Detection Tools.

1985 Impossibility of Distributed Consensus with One Faulty Process
Michael J. Fischer, Nancy A. Lynch, Michael S. Paterson

The amazing paper, which won Dijkstra award (2001), asserts no completely correct asynchronous consensus algorithm exists even when only one faulty process is tolerated. Lemma 3 in the paper is mind boggling, better interpreted with help of A Brief Tour of FLP Impossibility.

2001 Paxos Made Simple
Leslie Lamport

This is the paper written by Lamport himself after the famously rejected paper The Part-Time Parliament. The anecdote can be found on Leslie Lamport Writings.
The paper clearly discusses single paxos, that is, deciding a single value from processes but does not explain well on multi-paxos, that is, deciding a series of values.

1996 How to Build a Highly Available System Using Consensus
Butler W. Lampson

This is the paper Lampson, 1992 Turning Award winner, reinterpreted and made Paxos publicity. It explains Paxos from another perspective, perhaps making people earier to undertand.

2007 Paxos Made Live – An Engineering Perspective
Tushar Deepak Chandra, Robert Griesemer, Joshua Redstone

Google Chubby team implemented Chubby using multi-paxos, with which, replicated state machine is made possible. The paper talks about many engineering issues when putting it into practical use. I found another article that well explains multi-paxos in an easier approach by Tencent Weixin team (in Chinese) Weixin PhxPaxos. Yet there is a slide presenting consensus history, Paxos family and replicated state machine,
Distributed Consensus: Making Impossible Possible.

2014 In search of an understandable consensus algorithm
Diego Ongaro, John Ousterhout

Raft, the replicate state machine consensus algorithm, was designed to be undertood with less effort compared to Paxos family. Nowadays, quite a lot consensus systems, most notably etcd, are implemented using raft.

2011 Zab: High-performance broadcast for primary-backup systems
Flavio P. Junqueira, Benjamin C. Reed, and Marco Serafini

Core protocol of Zookeeper

2012 ZooKeeper’s atomic broadcast protocol: Theory and practice
Andr ́e Medeiros

2012 Viewstamped Replication Revisited
Barbara Liskov, James Cowling

Barbara Liskov, Turing award winner in 2008, revisted another replicate state machine consensus algorithm.