Redis Replication Deep-Dive
Replication Basics
Given that Redis is a single-threaded, in-memory key-value datastore, replication can be leveraged in order to provide high-availability. At the core of a replicated Redis configuration, there is a single master alongside one or more replicas (i.e. “leader-follower” / “master-slave” / “primary-replica” pattern). When a client writes to Redis, the request is processed by the master and then propagation to the replica(s) asynchronously. Then, when a client reads from Redis, it can either read from the master or a replica. Thus, replication provides a mechanism to delegate the read load from the master across replicas.
Typically, there is a service Sentinel or Redis Cluster accompanying the Redis instances that monitors the state of the leader and replica(s) to ensure that there is always one active leader and if it goes down, a replica is then promoted as a leader.
Important Notes
- Replication occurs asynchronously
- Replication is non-blocking on the leader
- Replication can block operations on the replica
- Leader can have an arbitrary numbers of replicas
- Generally, replicas are read-only
- A replica can be a leader for other replicas
Types of Replication
Full-Sync Replication
A Full Sync is when the Redis leader forks itself into a separate process which is then responsible for transferring the entire state/data to the replica(s). This is achieved via two mechanisms:
- The forked process writes to disk and then the disk file is transferred over.
- In disk-less mode, the forked process writes over the write to the replicas.
This method is leveraged when bootstrapping a replica or when encountering a partial sync failure.
Partial-Sync Replication
A circular buffer is stored on the leader which holds data while a replica (re)synchronizes with the master and the offset being the writes issued post-sync initiation.