The use of in memory databases as a new data structure layer for storing has increased recently. Many software architectures, especially high concurrent traffic, use an in memory database layer to store the data with various data types and also persistence for caching strategies.
One of the in memory databases, which is popular is Redis. Redis is a key-value data store where the data is stored at the memory level, many people said that it is faster compared to the SQL (Structured Query Language) database. The access of data structure through memory is definitely faster than accessing the data through the disk.
There are also some use cases that can be done through the redis, such as : message queue, leaderboard, and Pub (Publisher) /Sub (Subscriber).
In this blog, we will review how to set up Redis Master-Slave architecture.
How does Redis Replication work?
The Redis replication uses an asynchronous method to transfer the data from master to slaves node. The slaves periodically acknowledge the received data sent by the master node and also the master node can have many slaves. Redis can support cascade replication, so the slave can be connected to another slave.
It is good practice to enable persistence options in the master and slave nodes, otherwise need to avoid restarting automatically for instance. Every redis has a replication ID which consists of a large pseudo random string and the offset value on the master, this replication ID is the key replication process. The offset is incremented for every byte of streaming from master to slaves.On the other side, slaves connect to the master and use PSYNC to send their replication ID and the offset that has been processed in the slave. In this way, the master node will need to send the incremental data that is needed.
Redis Installation
The installation process of Redis is very clear, on the Centos based, the package installation does not come along as the default repository in CentOS operating system, thus we need to install the epel-release before we go through installation. While on the Ubuntu/Debian based, we can add the official redis repository into Ubuntu PPA (Personal Package Archive). In this blog, we will use Ubuntu as a base operating system for Redis installation.
First, we need to add redis repository as below:
vagrant@n1:~$ sudo add-apt-repository ppa:redislabs/redis
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.
It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams.
Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.
More info: https://launchpad.net/~redislabs/+archive/ubuntu/redis
Press [ENTER] to continue or Ctrl-c to cancel adding it.
0% [Working]
Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic InRelease [15.9 kB]
Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main i386 Packages [1,020 B]
Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:7 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 Packages [1,024 B]
Get:8 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [1,006 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [2,129 kB]
Get:10 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main Translation-en [584 B]
Get:11 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,783 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,310 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [421 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [25.8 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [389 kB]
Get:16 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [329 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [52.8 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,738 kB]
Get:19 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [19.2 kB]
Get:20 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [365 kB]
Get:21 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [48.9 kB]
Get:22 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [983 kB]
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,568 kB]
Get:24 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,131 kB]
Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [371 kB]
Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [26.6 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,792 B]
Get:28 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [256 kB]
Fetched 14.2 MB in 29s (492 kB/s)
Reading package lists... DoneAnd then, continue with the update using apt-update command:
vagrant@n1:~$ sudo apt-get update
Hit:1 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic InRelease
Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit:5 http://security.ubuntu.com/ubuntu bionic-security InRelease
Reading package lists... DoneLast thing, install the Redis package using apt package manager:
vagrant@n1:~$ sudo apt-get install redis
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
redis-server redis-tools
Suggested packages:
ruby-redis
The following NEW packages will be installed:
redis redis-server redis-tools
0 upgraded, 3 newly installed, 0 to remove and 31 not upgraded.
Need to get 1,172 kB of archives.
After this operation, 6,716 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 redis-tools amd64 6:6.2.4-1rl1~bionic1 [1,057 kB]
Get:2 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 redis-server amd64 6:6.2.4-1rl1~bionic1 [79.3 kB]
Get:3 http://ppa.launchpad.net/redislabs/redis/ubuntu bionic/main amd64 redis all 6:6.2.4-1rl1~bionic1 [36.2 kB]
Fetched 1,172 kB in 5s (235 kB/s)
Selecting previously unselected package redis-tools.
(Reading database ... 39766 files and directories currently installed.)
Preparing to unpack .../redis-tools_6%3a6.2.4-1rl1~bionic1_amd64.deb ...
Unpacking redis-tools (6:6.2.4-1rl1~bionic1) ...
Selecting previously unselected package redis-server.
Preparing to unpack .../redis-server_6%3a6.2.4-1rl1~bionic1_amd64.deb ...
Unpacking redis-server (6:6.2.4-1rl1~bionic1) ...
Selecting previously unselected package redis.
Preparing to unpack .../redis_6%3a6.2.4-1rl1~bionic1_all.deb ...
Unpacking redis (6:6.2.4-1rl1~bionic1) ...
Setting up redis-tools (6:6.2.4-1rl1~bionic1) ...
Setting up redis-server (6:6.2.4-1rl1~bionic1) ...
Setting up redis (6:6.2.4-1rl1~bionic1) ...
Processing triggers for systemd (237-3ubuntu10.47) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for ureadahead (0.100.0-21) ...
We will see the status of Redis running process through the command :
root@n1:~# systemctl status redis-server
● redis-server.service - Advanced key-value store
Loaded: loaded (/lib/systemd/system/redis-server.service; disabled; vendor preset: enabled)
Active: active (running) since Thu 2021-07-01 07:54:06 UTC; 6min ago
Docs: http://redis.io/documentation,
man:redis-server(1)
Main PID: 2645 (redis-server)
Status: "Ready to accept connections"
Tasks: 5 (limit: 1106)
CGroup: /system.slice/redis-server.service
└─2645 /usr/bin/redis-server 127.0.0.1:6379
Jul 01 07:54:05 n1 systemd[1]: Starting Advanced key-value store...
Jul 01 07:54:06 n1 systemd[1]: Started Advanced key-value store.Repeat the installation process on the other nodes.
Setting up Redis Master and Slave
After the installation is finished, the next part is we need to configure the Redis master and slave node. Redis configuration resides on the /etc/redis/redis.conf path and there are some parameters we need to adjust to make the service can be accessed from the other host. The parameters are:
bind 127.0.0.1 10.10.10.10
masterauth thisreplicationpassword
masteruser replicationuser
protected-mode noThe bind parameter listens to the Redis port in which interface address, while the master user and masterauth is used for authentication between the master and slave.
In the Redis slave nodes, we need to configure the redis master node, master user and auth as below:
bind: 127.0.0.1 10.10.10.11
masterauth thisreplicationpassword
masteruser replicationuser
replicaof 10.10.10.10 6379
After that, we need to restart the redis service and test and check the replication process using the info command as below:
root@n1:~# redis-cli -h 10.10.10.10 -p 6379
10.10.10.10:6379> info
# Replication
role:master
connected_slaves:1
master_failover_state:no-failover
master_replid:32e7ae56532d3308d3874d7b577f699244d69bf1
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0Manual Failover
The failover in Redis is very simple, we just need to run commands in the slave nodes to promote the redis to become an independent service and accept the traffic. Login to the Redis slave node:
root@n1:~# redis-cli -h 10.10.10.11 -p 6379
10.10.10.11:6379> SLAVEOF NO ONE
If we have more than one slave we can set the ip address of the slave for promotion to master.
10.10.10.11:6379> SLAVEOF 192.168.1.152 6379