【Redis】Cluster集群

一、Redis Cluster 工作原理

在引入哨兵机制后，解决了Redis主从架构Master故障时的主从切换问题，保证了Redis服务可用性。但依旧无法解决单机节点出现的写入性能瓶颈（网卡速率、单机内存容量、并发数量）
1、早期为解决单机性能瓶颈问题采用的解决方案：

1、客户端分片：由客户端程序进行读写key的redis节点判断和分配，并且由客户端自行处理读写请求分配、高可用管理及故障转移操作
2、proxy代理模式：引入第三方代理程序，客户端通过连接proxy代理服务器对数据进行读写，由proxy程序进行读写判断分配，并对集群节点进行管理。但导致proxy又出现单点故障风险，并增加了一层数据处理环节
redis3.0 之后官方推出了无中心化的Cluster集群机制，集群中的每个节点单独保持当前节点数据和集群状态信息，每个节点需要和其余全部节点保持连接。
2、Redis Cluster工作特点

1、Redis cluster集群中的各个节点之间（采用ping机制）进行互联
2、集群中的某个节点失效，是有整个集群中超过半数的节点监测失效（sdown），才会真正判定为节点不可用（odown）
3、客户端无需配置连接proxy即可直连redis，应用程序需配置集群的所有redis节点IP
4、redis cluster将集群中所有的节点平均映射到0-16383（16384个）slot槽位上，将数据库的读写全部映射到对应的节点上操作，因此每增加n个集群主节点就能对redis进行n倍扩展，每个主节点将分担16384/n个slot
5、redis cluster预先将16384个槽位分配至集群中的主节点，当接收到对redis的写入请求时，会通过 CRC16(key)%16384 计算出当前key的所属slot编号，从而将数据写入到对应的主节点上，从而解决单机产生的性能瓶颈
3、Redis Cluster 架构设计


Redis Cluster 基本架构


 Redis Cluster 主从架构

为解决Cluster集群中各Master节点的高可用性，可采用Cluster主从架构部署，将部分加入集群的节点设置为Slave角色，从而解决Master节点的高可用性
Cluster可自行实现类似哨兵机制的每组Master-Slave的主从切换，无需额外配置哨兵机制。并且支持每组节点的 1-n 模式（不支持1-n-n）

 Redis Cluster 部署架构设计

  
3、Redis Cluster局限性

1、大多数时候客户端连接性能会“降低”，由于cluster在读写数据时需要先进行solt计算并重定向访问到对应节点上，造成了数据处理“延迟性”。
2、部分命令无法跨节点使用：mget、keys、scan、flush、sinter 等
3、客户端维护复杂：SDK 和 应用自身消耗增加（更多的数据连接池）
4、不支持多数据库：cluster 模式下仅支持使用db0
5、复制仅支持一层：不支持级联复制
6、key事务、Lua 的支持有限：key操作必须在同一个节点上，事务和Lua无法跨节点使用
 
二、Redis Cluster 相关配置详解

1. ################################ REDIS CLUSTER  ###############################

复制代码

1. # Normal Redis instances can't be part of a Redis Cluster; only nodes that are
2. # started as cluster nodes can. In order to start a Redis instance as a
3. # cluster node enable the cluster support uncommenting the following:
4. #
5. # cluster-enabled yes #启用Cluster模式
7. # Every cluster node has a cluster configuration file. This file is not
8. # intended to be edited by hand. It is created and updated by Redis nodes.
9. # Every Redis Cluster node requires a different cluster configuration file.
10. # Make sure that instances running in the same system do not have
11. # overlapping cluster configuration file names.
12. #
13. # cluster-config-file nodes-6379.conf #Cluster节点配置文件名称
15. # Cluster node timeout is the amount of milliseconds a node must be unreachable
16. # for it to be considered in failure state.
17. # Most other internal time limits are multiple of the node timeout.
18. #
19. # cluster-node-timeout 15000 #Cluster节点超时时间，单位：毫秒
21. # A replica of a failing master will avoid to start a failover if its data
22. # looks too old.
23. #
24. # There is no simple way for a replica to actually have an exact measure of
25. # its "data age", so the following two checks are performed:
26. #
27. # 1) If there are multiple replicas able to failover, they exchange messages
28. #    in order to try to give an advantage to the replica with the best
29. #    replication offset (more data from the master processed).
30. #    Replicas will try to get their rank by offset, and apply to the start
31. #    of the failover a delay proportional to their rank.
32. #
33. # 2) Every single replica computes the time of the last interaction with
34. #    its master. This can be the last ping or command received (if the master
35. #    is still in the "connected" state), or the time that elapsed since the
36. #    disconnection with the master (if the replication link is currently down).
37. #    If the last interaction is too old, the replica will not try to failover
38. #    at all.
39. #
40. # The point "2" can be tuned by user. Specifically a replica will not perform
41. # the failover if, since the last interaction with the master, the time
42. # elapsed is greater than:
43. #
44. #   (node-timeout * replica-validity-factor) + repl-ping-replica-period
45. #
46. # So for example if node-timeout is 30 seconds, and the replica-validity-factor
47. # is 10, and assuming a default repl-ping-replica-period of 10 seconds, the
48. # replica will not try to failover if it was not able to talk with the master
49. # for longer than 310 seconds.
50. #
51. # A large replica-validity-factor may allow replicas with too old data to failover
52. # a master, while a too small value may prevent the cluster from being able to
53. # elect a replica at all.
54. #
55. # For maximum availability, it is possible to set the replica-validity-factor
56. # to a value of 0, which means, that replicas will always try to failover the
57. # master regardless of the last time they interacted with the master.
58. # (However they'll always try to apply a delay proportional to their
59. # offset rank).
60. #
61. # Zero is the only value able to guarantee that when all the partitions heal
62. # the cluster will always be able to continue.
63. #
64. # cluster-replica-validity-factor 10<br># 集群副本有效性因子（0表示slave将始终尝试故障转移为master，默认被注释掉）
65. # (node-timeout * replica-validity-factor) + repl-ping-replica-period
66. # 计算时间（秒），如果slave无法与master通信的时间超过了该时间，则slave将不会尝试故障转移
68. # Cluster replicas are able to migrate to orphaned masters, that are masters
69. # that are left without working replicas. This improves the cluster ability
70. # to resist to failures as otherwise an orphaned master can't be failed over
71. # in case of failure if it has no working replicas.
72. #
73. # Replicas migrate to orphaned masters only if there are still at least a
74. # given number of other working replicas for their old master. This number
75. # is the "migration barrier". A migration barrier of 1 means that a replica
76. # will migrate only if there is at least 1 other working replica for its master
77. # and so forth. It usually reflects the number of replicas you want for every
78. # master in your cluster.
79. #
80. # Default is 1 (replicas migrate only if their masters remain with at least
81. # one replica). To disable migration just set it to a very large value.
82. # A value of 0 can be set but is useful only for debugging and dangerous
83. # in production.
84. #
85. # cluster-migration-barrier 1 #集群迁移屏障，仅当master至少有1个其他工作副本时，副本才会迁移
87. # By default Redis Cluster nodes stop accepting queries if they detect there
88. # is at least an hash slot uncovered (no available node is serving it).
89. # This way if the cluster is partially down (for example a range of hash slots
90. # are no longer covered) all the cluster becomes, eventually, unavailable.
91. # It automatically returns available as soon as all the slots are covered again.
92. #
93. # However sometimes you want the subset of the cluster which is working,
94. # to continue to accept queries for the part of the key space that is still
95. # covered. In order to do so, just set the cluster-require-full-coverage
96. # option to no.
97. #
98. # cluster-require-full-coverage yes # 集群需要全覆盖（默认为 yes，被注释掉）
99. # 如果redis集群节点检测到至少有一个未覆盖的哈希槽（没有可用的节点提供服务），则会停止接受查询
100. # 如果集群部分关闭（例如不再覆盖一系列哈希槽），整个集群将不可用
101. # 当所有插槽再次被覆盖，集群将自动返回可用状态
103. # This option, when set to yes, prevents replicas from trying to failover its
104. # master during master failures. However the master can still perform a
105. # manual failover, if forced to do so.
106. #
107. # This is useful in different scenarios, especially in the case of multiple
108. # data center operations, where we want one side to never be promoted if not
109. # in the case of a total DC failure.
110. #
111. # cluster-replica-no-failover no #主节点故障期间不启用故障转移
113. # In order to setup your cluster make sure to read the documentation
114. # available at http://redis.io web site.
116. ########################## CLUSTER DOCKER/NAT support  ########################
118. # In certain deployments, Redis Cluster nodes address discovery fails, because
119. # addresses are NAT-ted or because ports are forwarded (the typical case is
120. # Docker and other containers).
121. #
122. # In order to make Redis Cluster working in such environments, a static
123. # configuration where each node knows its public address is needed. The
124. # following two options are used for this scope, and are:
125. #
126. # * cluster-announce-ip
127. # * cluster-announce-port
128. # * cluster-announce-bus-port
129. #
130. # Each instruct the node about its address, client port, and cluster message
131. # bus port. The information is then published in the header of the bus packets
132. # so that other nodes will be able to correctly map the address of the node
133. # publishing the information.
134. #
135. # If the above options are not used, the normal Redis Cluster auto-detection
136. # will be used instead.
137. #
138. # Note that when remapped, the bus port may not be at the fixed offset of
139. # clients port + 10000, so you can specify any port and bus-port depending
140. # on how they get remapped. If the bus-port is not set, a fixed offset of
141. # 10000 will be used as usually.
142. #
143. # Example:
144. #
145. # cluster-announce-ip 10.1.1.5
146. # cluster-announce-port 6379
147. # cluster-announce-bus-port 6380<br># 配置集群节点的公共地址（默认被注释掉，使用redis群集自动检测）
148. # 节点的ip地址，客户端端口，集群消息总线端口<br>

复制代码

 
三、Redis Cluster部署

0、基础环境准备

所有redis节点采用相同硬件配置，相同密码，清空所有数据

	IP	角色	slot
node1	10.0.0.20	M1	0-5460
node2	10.0.0.21	M2	5461-10922
node3	10.0.0.22	M3	10923-16383
node4	10.0.0.23	集群扩容缩容备用（M4）
node5	10.0.0.30	S1
node6	10.0.0.31	S2
node7	10.0.0.32	S3
node8	10.0.0.33	集群扩容缩容备用（S4）

 
 
 

 

 

 

 

初始化安装6台 redis服务，并启用Cluster模式

1. [root@Redis-Ubuntu1804-node1:~]# apt install redis -y
2. [root@Redis-Ubuntu1804-node1:~]# sed -i.bak -e 's/bind 127.0.0.1/bind 0.0.0.0/' -e '/masterauth/a masterauth redis' -e '/# requirepass/a requirepass redis' -e '/# cluster-enabled yes/a cluster-enabled yes' -e '/# cluster-config-file/a cluster-config-file nodes-6379.conf' -e '/# cluster-require-full-coverage yes/a cluster-require-full-coverage yes' /etc/redis/redis.conf
3. [root@Redis-Ubuntu1804-node1:~]# systemctl restart redis
4. [root@Redis-Ubuntu1804-node1:~]# ss -ntl
5. State             Recv-Q             Send-Q                          Local Address:Port                           Peer Address:Port            
6. LISTEN            0                  128                             127.0.0.53%lo:53                                  0.0.0.0:*               
7. LISTEN            0                  128                                   0.0.0.0:22                                  0.0.0.0:*               
8. LISTEN            0                  128                                 127.0.0.1:6010                                0.0.0.0:*               
9. LISTEN            0                  128                                   0.0.0.0:16379                               0.0.0.0:*               
10. LISTEN            0                  128                                   0.0.0.0:6379                                0.0.0.0:*               
11. LISTEN            0                  128                                      [::]:22                                     [::]:*               
12. LISTEN            0                  128                                     [::1]:6010                                   [::]:*               
13. [root@Redis-Ubuntu1804-node1:~]# ps -ef | grep redis
14. redis      2098      1  0 01:38 ?        00:00:00 /usr/bin/redis-server 0.0.0.0:6379 [cluster]
15. root       2112   1236  0 01:39 pts/0    00:00:00 grep --color=auto redis
16. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1、基本模式手动部署

基本步骤
1、各节点服务器上安装redis服务，启用Cluster配置
2、通过meet 将节点添加至集群中
3、为各Master节点分配槽位 0~16383 ，必须全部分配完成，集群才可用
4、指定各节点主从关系

1.1、查看当前节点状态信息

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes
2. 98bb2740b622645ff51fd07bc994bb22e94feaaa :6379@16379 myself,master - 0 0 0 connected
3. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
1.2、将所有节点加入到当前集群中 

Cluster节点操作相关命令
1、加入节点：将 ip 和 port 所指定的节点添加到集群当中，让它成为集群的一份子。
1）CLUSTER MEET
2、移除节点：
1）CLUSTER FORGET
2)、redis-trib.rb del-node   
3、设置主从节点：
CLUSTER REPLICATE  将当前节点设置为 node_id 指定的节点的从节点。  
4、节点数据备份到硬盘：
CLUSTER SAVECONFIG 将节点的配置文件保存到硬盘里面。 

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.20 6379
2. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.20 6379
3. OK
4. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.21 6379
5. OK
6. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.22 6379
7. OK
8. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.30 6379
9. OK
10. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.31 6379
11. OK
12. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster meet 10.0.0.32 6379
13. OK
14. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes
15. 98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 myself,master - 0 1681926433000 1 connected
16. 9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 master - 0 1681926434205 5 connected
17. 55fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 master - 0 1681926435212 4 connected
18. 6717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 master - 0 1681926433000 3 connected
19. d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681926434000 0 connected
20. bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681926433199 2 connected
21. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
1.3、查看当前集群信息

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster info
2. cluster_state:fail
3. cluster_slots_assigned:0
4. cluster_slots_ok:0
5. cluster_slots_pfail:0
6. cluster_slots_fail:0
7. cluster_known_nodes:6
8. cluster_size:0
9. cluster_current_epoch:5
10. cluster_my_epoch:1
11. cluster_stats_messages_ping_sent:70
12. cluster_stats_messages_pong_sent:89
13. cluster_stats_messages_meet_sent:6
14. cluster_stats_messages_sent:165
15. cluster_stats_messages_ping_received:88
16. cluster_stats_messages_pong_received:76
17. cluster_stats_messages_meet_received:1
18. cluster_stats_messages_received:165
19. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
1.4、为集群节点分配槽位

Cluster 槽位相关命令
CLUSTER ADDSLOTS  [slot ...] 将一个或多个槽（slot）指派（assign）给当前节点。
CLUSTER DELSLOTS  [slot ...] 移除一个或多个槽对当前节点的指派。
CLUSTER FLUSHSLOTS 移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。
CLUSTER SETSLOT  NODE  将槽 slot 指派给 node_id 指定的节点，如果槽已经指派给另一个节点，那么先让另一个节点删除该槽>，然后再进行指派。
CLUSTER SETSLOT  MIGRATING  将本节点的槽 slot 迁移到 node_id 指定的节点中。
CLUSTER SETSLOT  IMPORTING  从 node_id 指定的节点中导入槽 slot 到本节点。
CLUSTER SETSLOT  STABLE 取消对槽 slot 的导入（import）或者迁移（migrate）。 

使用命令 redis-cli -h  -p  -a  --no-auth-warning cluster addslots  进行槽位分配

1. [root@Client-Ubuntu-1804-250:~/script/redis]# cat add_slots.sh
2. #!/bin/bash
3. #
4. #********************************************************************
5. #Author:                janzen
6. #Date:                  2023-04-20
7. #FileName：             add_slots.sh
8. #Description：          The test script
9. #Copyright (C):        2023 All rights reserved
10. #********************************************************************
11. host=$1
12. port=$2
13. start=$3
14. end=$4
15. passwd=redis
17. for slot in `seq $start $end`; do
18.     redis-cli -h $host -p $port -a $passwd --no-auth-warning cluster addslots $slot &&
19.     echo $slot add to node $host:$port
20. done
21. [root@Client-Ubuntu-1804-250:~/script/redis]#
22. [root@Client-Ubuntu-1804-250:~/script/redis]# ./add_slots.sh 10.0.0.20 6379 0 5460 >> addslot.log
23. [root@Client-Ubuntu-1804-250:~/script/redis]# ./add_slots.sh 10.0.0.21 6379 5461 10922 >> addslot.log
24. [root@Client-Ubuntu-1804-250:~/script/redis]# ./add_slots.sh 10.0.0.22 6379 10923 16383 >> addslot.log

复制代码

 

1. ##查看当前集群信息，槽位已完成分配<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes
2. 98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 myself,master - 0 1681929286000 1 connected 0-5460
3. 55fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 master - 0 1681929287000 4 connected
4. 9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 master - 0 1681929287000 5 connected
5. 6717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 master - 0 1681929288247 3 connected
6. d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681929286000 0 connected 5461-10922
7. bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681929289256 2 connected 10923-16383
8. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1.5、创建 Master-Slave 主从关系

使用命令 redis-cli -h  -p  -a  --no-auth-warning cluster replicate  

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 cluster replicate 98bb2740b622645ff51fd07bc994bb22e94feaaa
2. OK
4. ##观察主节点上的replication信息
5. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 info Replication
6. # Replication
7. role:master
8. connected_slaves:1
9. slave0:ip=10.0.0.30,port=6379,state=online,offset=350,lag=1
10. master_replid:6a823070fcbc13b4f4eb7b5d13bf1d2f4625a1a2
11. master_replid2:0000000000000000000000000000000000000000
12. master_repl_offset:350
13. second_repl_offset:-1
14. repl_backlog_active:1
15. repl_backlog_size:1048576
16. repl_backlog_first_byte_offset:1
17. repl_backlog_histlen:350
18. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 cluster nodes
19. 98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 myself,master - 0 1681929989000 1 connected 0-5460
20. 55fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 master - 0 1681929992372 4 connected
21. 9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 master - 0 1681929991000 5 connected
22. 6717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 slave 98bb2740b622645ff51fd07bc994bb22e94feaaa 0 1681929992000 3 connected
23. d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681929989351 0 connected 5461-10922
24. bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681929991364 2 connected 10923-16383
26. ##观察从节点上的replication信息
27. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 info Replication
28. # Replication
29. role:slave
30. master_host:10.0.0.20
31. master_port:6379
32. master_link_status:up
33. master_last_io_seconds_ago:0
34. master_sync_in_progress:0
35. slave_repl_offset:462
36. slave_priority:100
37. slave_read_only:1
38. connected_slaves:0
39. master_replid:6a823070fcbc13b4f4eb7b5d13bf1d2f4625a1a2
40. master_replid2:0000000000000000000000000000000000000000
41. master_repl_offset:462
42. second_repl_offset:-1
43. repl_backlog_active:1
44. repl_backlog_size:1048576
45. repl_backlog_first_byte_offset:1
46. repl_backlog_histlen:462
47. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
 继续完成主从组配置

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.31 cluster replicate d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed
2. OK
3. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.32 cluster replicate bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2
4. OK<br><br>##观察集群节点状态
5. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 cluster nodes
6. 9fe09d53d884c6c99c926a0e7213cf5386987ebf 10.0.0.32:6379@16379 slave bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 0 1681930655876 5 connected
7. 98bb2740b622645ff51fd07bc994bb22e94feaaa 10.0.0.20:6379@16379 master - 0 1681930654870 1 connected 0-5460
8. 55fd3a5d21e026b6503a89f1d88996b2a67c314a 10.0.0.31:6379@16379 slave d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 0 1681930656886 4 connected
9. bdec3c92bcaa00c8ad1850cd477eb0a8f8a28aa2 10.0.0.22:6379@16379 master - 0 1681930655000 2 connected 10923-16383
10. 6717c6b5a6170963ac251a1f14b0c600e917aa26 10.0.0.30:6379@16379 myself,slave 98bb2740b622645ff51fd07bc994bb22e94feaaa 0 1681930654000 3 connected
11. d9c17ded7fb139f9eb7fd41e476bb8e17c8b9bed 10.0.0.21:6379@16379 master - 0 1681930653863 0 connected 5461-10922

复制代码

  
1.6、验证Cluster集群访问

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.20 set node1 10.0.0.20
2. OK
3. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.21 set node2 10.0.0.21
4. OK
5. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.22 set node3 10.0.0.22
6. OK
7. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.30 set node5 10.0.0.24
8. OK
9. [root@Redis-Ubuntu1804-node1:~]#
10. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.31 get node1
11. "10.0.0.20"
12. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.31 get node2
13. "10.0.0.21"
14. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c -a redis -h 10.0.0.21 get node6
15. "10.0.0.31"
16. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 

 

2、基于 Redis 5 Cluster集群部署

--cluster 常用命令

1. [root@Redis-Ubuntu1804-node3:~]# redis-cli -a redis --cluster help
2. Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
3. Cluster Manager Commands:
4.   create         host1:port1 ... hostN:portN　　　　##创建集群
5.                  --cluster-replicas <arg>　　　　##指定每个集群的slave节点数，通常为1
6.   check          host:port　　　　##检查集群信息
7.                  --cluster-search-multiple-owners
8.   info           host:port　　　　##查看集群节点基本信息
9.   fix            host:port　　　　##修复集群
10.                  --cluster-search-multiple-owners
11.   reshard        host:port　　　　##在线热机迁移指定的slots数据
12.                  --cluster-from <arg>
13.                  --cluster-to <arg>
14.                  --cluster-slots <arg>
15.                  --cluster-yes
16.                  --cluster-timeout <arg>
17.                  --cluster-pipeline <arg>
18.                  --cluster-replace
19.   rebalance      host:port　　　　##平衡集群在线节点的slots数量
20.                  --cluster-weight <node1=w1...nodeN=wN>
21.                  --cluster-use-empty-masters
22.                  --cluster-timeout <arg>
23.                  --cluster-simulate
24.                  --cluster-pipeline <arg>
25.                  --cluster-threshold <arg>
26.                  --cluster-replace
27.   add-node       new_host:new_port existing_host:existing_port　　　　##添加新的节点到集群中
28.                  --cluster-slave
29.                  --cluster-master-id <arg>
30.   del-node       host:port node_id　　　　##删除集群中的节点
31.   call           host:port command arg arg .. arg　　　　##在集群上的所有节点执行命令
32.   set-timeout    host:port milliseconds　　　　##设置节点超时时间
33.   import         host:port　　　　##导入外部redis数据到当前集群
34.                  --cluster-from <arg>
35.                  --cluster-copy
36.                  --cluster-replace
37.   help           
39. For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

复制代码

 
2.0、基本环境准备

1. [root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis --version
2. redis-cli 5.0.14
3. [root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis --no-auth-warning cluster nodes
4. e603091426e2ce29629ca51fb5e7dafb2f0c9524 :6379@16379 myself,master - 0 0 0 connected
5. [root@Redis-Ubuntu1804-node4:~]#

复制代码

 
2.1、创建Cluster集群

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --cluster create 10.0.0.20:6379 10.0.0.21:6379 10.0.0.22:6379 10.0.0.30:6379 10.0.0.31:6379 10.0.0.32:6379 --cluster-replicas 1
2. Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
3. >>> Performing hash slots allocation on 6 nodes...
4. Master[0] -> Slots 0 - 5460
5. Master[1] -> Slots 5461 - 10922
6. Master[2] -> Slots 10923 - 16383
7. Adding replica 10.0.0.31:6379 to 10.0.0.20:6379
8. Adding replica 10.0.0.32:6379 to 10.0.0.21:6379
9. Adding replica 10.0.0.30:6379 to 10.0.0.22:6379
10. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
11.    slots:[0-5460] (5461 slots) master
12. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
13.    slots:[5461-10922] (5462 slots) master
14. M: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
15.    slots:[10923-16383] (5461 slots) master
16. S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
17.    replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3
18. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
19.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
20. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
21.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
22. Can I set the above configuration? (type 'yes' to accept): yes
23. >>> Nodes configuration updated
24. >>> Assign a different config epoch to each node
25. >>> Sending CLUSTER MEET messages to join the cluster
26. Waiting for the cluster to join
27. .
28. >>> Performing Cluster Check (using node 10.0.0.20:6379)
29. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
30.    slots:[0-5460] (5461 slots) master
31.    1 additional replica(s)
32. M: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
33.    slots:[10923-16383] (5461 slots) master
34.    1 additional replica(s)
35. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
36.    slots:[5461-10922] (5462 slots) master
37.    1 additional replica(s)
38. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
39.    slots: (0 slots) slave
40.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
41. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
42.    slots: (0 slots) slave
43.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
44. S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
45.    slots: (0 slots) slave
46.    replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3
47. [OK] All nodes agree about slots configuration.
48. >>> Check for open slots...
49. >>> Check slots coverage...
50. [OK] All 16384 slots covered.
51. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
2.2、查看集群状态

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis cluster nodes
2. Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
3. 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master - 0 1681998239529 3 connected 10923-16383
4. a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1681998238520 2 connected 5461-10922
5. 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1681998237000 6 connected
6. 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1681998238000 5 connected
7. 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 slave 07a36af61eb1f887419f2266b2360b4f795bb7a3 0 1681998237510 4 connected
8. 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1681998236000 1 connected 0-5460
9. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis cluster info
10. Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
11. cluster_state:ok
12. cluster_slots_assigned:16384
13. cluster_slots_ok:16384
14. cluster_slots_pfail:0
15. cluster_slots_fail:0
16. cluster_known_nodes:6
17. cluster_size:3
18. cluster_current_epoch:6
19. cluster_my_epoch:1
20. cluster_stats_messages_ping_sent:290
21. cluster_stats_messages_pong_sent:282
22. cluster_stats_messages_sent:572
23. cluster_stats_messages_ping_received:277
24. cluster_stats_messages_pong_received:290
25. cluster_stats_messages_meet_received:5
26. cluster_stats_messages_received:572
27. [root@Redis-Ubuntu1804-node1:~]#
28. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning --cluster info 10.0.0.20:6379
29. 10.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.
30. 10.0.0.22:6379 (07a36af6...) -> 0 keys | 5461 slots | 1 slaves.
31. 10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.
32. [OK] 0 keys in 3 masters.
33. 0.00 keys per slot on average.
34. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning --cluster check 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.
3. 10.0.0.22:6379 (07a36af6...) -> 0 keys | 5461 slots | 1 slaves.
4. 10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.
5. [OK] 0 keys in 3 masters.
6. 0.00 keys per slot on average.
7. >>> Performing Cluster Check (using node 10.0.0.20:6379)
8. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
9.    slots:[0-5460] (5461 slots) master
10.    1 additional replica(s)
11. M: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
12.    slots:[10923-16383] (5461 slots) master
13.    1 additional replica(s)
14. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
15.    slots:[5461-10922] (5462 slots) master
16.    1 additional replica(s)
17. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
18.    slots: (0 slots) slave
19.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
20. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
21.    slots: (0 slots) slave
22.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
23. S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
24.    slots: (0 slots) slave
25.    replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3
26. [OK] All nodes agree about slots configuration.
27. >>> Check for open slots...
28. >>> Check slots coverage...
29. [OK] All 16384 slots covered.

复制代码

 
2.3、验证集群写入

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster nodes
2. 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master - 0 1681998730094 3 connected 10923-16383
3. a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1681998729087 2 connected 5461-10922
4. 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1681998728000 6 connected
5. 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1681998728079 5 connected
6. 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 slave 07a36af61eb1f887419f2266b2360b4f795bb7a3 0 1681998729000 4 connected
7. 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1681998726000 1 connected 0-5460
8. #写入 key1：value1<br>###经过slot计算 key1 不属于当前节点<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key1 value1
9. (error) MOVED 9189 10.0.0.21:6379<br>##计算 key 值所属的slot号
10. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key1
11. (integer) 9189<br>##结合 nodes 信息，切换到对应节点上进行写入
12. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning set key1 value1
13. OK<br>##get命令也只可查看当前节点上的数据
14. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning get key1
15. (error) MOVED 9189 10.0.0.21:6379
16. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning get key1
17. "value1"<br>

复制代码

1. ##可以用 -c 参数通过连接到集群进行数据操作，就无需反复切换节点（key1、key4 不在当前节点所属的slot范围，出现错误提示）
2. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key1
3. (integer) 9189
4. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key2
5. (integer) 4998
6. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key3
7. (integer) 935
8. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning cluster keyslot key4
9. (integer) 13120
10. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning set key1 value1
11. OK
12. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key1 value1
13. (error) MOVED 9189 10.0.0.21:6379
14. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key2 value2
15. OK
16. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key3 value3
17. OK
18. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --no-auth-warning set key4 value4
19. (error) MOVED 13120 10.0.0.22:6379
20. [root@Redis-Ubuntu1804-node1:~]#<br><br>##使用 -c 参数后执行成功<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning set key1 value1<br>OK<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning set key4 value4<br>OK<br>[root@Redis-Ubuntu1804-node1:~]#

复制代码

 
2.4、模拟故障验证集群故障切换

启用脚本模拟客户端数据读写

故障前集群信息状态

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes
2. 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master - 0 1681999895495 3 connected 10923-16383
3. a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1681999896502 2 connected 5461-10922
4. 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1681999894488 6 connected
5. 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1681999895000 5 connected
6. 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 slave 07a36af61eb1f887419f2266b2360b4f795bb7a3 0 1681999892000 4 connected
7. 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1681999893000 1 connected 0-5460
8. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379
9. 10.0.0.20:6379 (6b142fe3...) -> 97 keys | 5461 slots | 1 slaves.
10. 10.0.0.22:6379 (07a36af6...) -> 100 keys | 5461 slots | 1 slaves.
11. 10.0.0.21:6379 (a4b387d8...) -> 92 keys | 5462 slots | 1 slaves.
12. [OK] 289 keys in 3 masters.
13. 0.02 keys per slot on average.
14. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379
15. 10.0.0.20:6379 (6b142fe3...) -> 99 keys | 5461 slots | 1 slaves.
16. 10.0.0.22:6379 (07a36af6...) -> 103 keys | 5461 slots | 1 slaves.
17. 10.0.0.21:6379 (a4b387d8...) -> 94 keys | 5462 slots | 1 slaves.
18. [OK] 296 keys in 3 masters.
19. 0.02 keys per slot on average.
20. >>> Performing Cluster Check (using node 10.0.0.20:6379)
21. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
22.    slots:[0-5460] (5461 slots) master
23.    1 additional replica(s)
24. M: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
25.    slots:[10923-16383] (5461 slots) master
26.    1 additional replica(s)
27. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
28.    slots:[5461-10922] (5462 slots) master
29.    1 additional replica(s)
30. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
31.    slots: (0 slots) slave
32.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
33. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
34.    slots: (0 slots) slave
35.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
36. S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
37.    slots: (0 slots) slave
38.    replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3
39. [OK] All nodes agree about slots configuration.
40. >>> Check for open slots...
41. >>> Check slots coverage...
42. [OK] All 16384 slots covered.
43. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
停用 10.0.0.22 节点

1. [root@Redis-Ubuntu1804-node3:~]# systemctl stop redis_6379.service
2. [root@Redis-Ubuntu1804-node3:~]#

复制代码

 
观察故障期间集群状态及日志

1. ##10.0.0.22节点标记 faild，并提升它的从节点 10.0.0.30 为master<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379
2. Could not connect to Redis at 10.0.0.22:6379: Connection refused
3. *** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.
4. 10.0.0.20:6379 (6b142fe3...) -> 131 keys | 5461 slots | 1 slaves.
5. 10.0.0.21:6379 (a4b387d8...) -> 126 keys | 5462 slots | 1 slaves.
6. [OK] 257 keys in 2 masters.
7. 0.02 keys per slot on average.
8. >>> Performing Cluster Check (using node 10.0.0.20:6379)
9. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
10.    slots:[0-5460] (5461 slots) master
11.    1 additional replica(s)
12. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
13.    slots:[5461-10922] (5462 slots) master
14.    1 additional replica(s)
15. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
16.    slots: (0 slots) slave
17.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
18. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
19.    slots: (0 slots) slave
20.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
21. S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
22.    slots: (0 slots) slave
23.    replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3
24. [OK] All nodes agree about slots configuration.
25. >>> Check for open slots...
26. >>> Check slots coverage...
27. [ERR] Not all 16384 slots are covered by nodes.
29. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379
30. Could not connect to Redis at 10.0.0.22:6379: Connection refused
31. *** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.
32. 10.0.0.20:6379 (6b142fe3...) -> 131 keys | 5461 slots | 1 slaves.
33. 10.0.0.21:6379 (a4b387d8...) -> 126 keys | 5462 slots | 1 slaves.
34. [OK] 257 keys in 2 masters.
35. 0.02 keys per slot on average.
36. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes
37. 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master,fail - 1682000024732 1682000022415 3 disconnected
38. a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682000042000 2 connected 5461-10922
39. 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682000040567 6 connected
40. 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682000042582 5 connected
41. 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682000041575 7 connected 10923-16383
42. 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1682000041000 1 connected 0-5460
43. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. 17519:M 20 Apr 2023 22:14:01.600 * Marking node 07a36af61eb1f887419f2266b2360b4f795bb7a3 as failing (quorum reached).
2. 17519:M 20 Apr 2023 22:14:01.600 # Cluster state changed: fail
3. 17519:M 20 Apr 2023 22:14:02.167 # Failover auth granted to 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 for epoch 7
4. 17519:M 20 Apr 2023 22:14:02.177 # Cluster state changed: ok

复制代码

客户端日志出现了约4s 的服务不可用

 
恢复 10.0.0.22 节点
 
观察集群节点信息及日志

1. ##10.0.0.22节点被添加为slave节点，并将master指向10.0.0.30<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.
4. 10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.
5. [OK] 445 keys in 3 masters.
6. 0.03 keys per slot on average.
7. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379
8. 10.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.
9. 10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.
10. 10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.
11. [OK] 445 keys in 3 masters.
12. 0.03 keys per slot on average.
13. >>> Performing Cluster Check (using node 10.0.0.20:6379)
14. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
15.    slots:[0-5460] (5461 slots) master
16.    1 additional replica(s)
17. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
18.    slots: (0 slots) slave
19.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
20. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
21.    slots:[5461-10922] (5462 slots) master
22.    1 additional replica(s)
23. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
24.    slots: (0 slots) slave
25.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
26. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
27.    slots: (0 slots) slave
28.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
29. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
30.    slots:[10923-16383] (5461 slots) master
31.    1 additional replica(s)
32. [OK] All nodes agree about slots configuration.
33. >>> Check for open slots...
34. >>> Check slots coverage...
35. [OK] All 16384 slots covered.
36. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. ##日志记录清除 10.0.0.22 节点上的faild 标记，并作为slave节点启动17519:M 20 Apr 2023 22:14:01.600 * Marking node 07a36af61eb1f887419f2266b2360b4f795bb7a3 as failing (quorum reached).
2. 17519:M 20 Apr 2023 22:14:01.600 # Cluster state changed: fail
3. 17519:M 20 Apr 2023 22:14:02.167 # Failover auth granted to 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 for epoch 7
4. 17519:M 20 Apr 2023 22:14:02.177 # Cluster state changed: ok17519:M 20 Apr 2023 22:17:23.792 * Clear FAIL state for node 07a36af61eb1f887419f2266b2360b4f795bb7a3: master without slots is reachable again.

复制代码

 
3、基于Redis 3/4 Cluster集群部署

3.0、基本环境准备

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis cluster nodes
2. 1c2fee65ee50cdf66c862d04710672286b305f55 :6379@16379 myself,master - 0 0 0 connected
3. [root@Redis-Ubuntu1804-node1:~]# redis-cli --version
4. redis-cli 4.0.9
5. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
3.1、准备 redis-trib.rb 工具

查找 redis-trib.rb 工具路径，将二进制文件复制到 /usr/bin 目录下

1. [root@Redis-Ubuntu1804-node1:~]# find / -name redis-trib.rb
2. /usr/share/doc/redis-tools/examples/redis-trib.rb
3. [root@Redis-Ubuntu1804-node1:~]# cp /usr/share/doc/redis-tools/examples/redis-trib.rb /usr/bin/

复制代码

 
安装ruby运行环境

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb
2. /usr/bin/env: ‘ruby’: No such file or directory
3. [root@Redis-Ubuntu1804-node1:~]# ruby
5. Command 'ruby' not found, but can be installed with:
7. snap install ruby  # version 3.2.2, or
8. apt  install ruby
10. See 'snap info ruby' for additional versions.
12. [root@Redis-Ubuntu1804-node1:~]# apt-cache madison ruby
13.       ruby |    1:2.5.1 | http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
14. [root@Redis-Ubuntu1804-node1:~]# apt install ruby -y
15. Reading package lists... Done
16. Building dependency tree      
17. Reading state information... Done
18. The following packages were automatically installed and are no longer required:
19.   linux-headers-4.15.0-151 linux-headers-4.15.0-151-generic linux-image-4.15.0-151-generic linux-modules-4.15.0-151-generic
20.   linux-modules-extra-4.15.0-151-generic
21. Use 'apt autoremove' to remove them.
22. The following additional packages will be installed:
23.   fonts-lato javascript-common libjs-jquery libruby2.5 rake ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert ruby-test-unit
24.   ruby2.5 rubygems-integration unzip zip
25. Suggested packages:
26.   apache2 | lighttpd | httpd ri ruby-dev bundler
27. The following NEW packages will be installed:
28.   fonts-lato javascript-common libjs-jquery libruby2.5 rake ruby ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert
29.   ruby-test-unit ruby2.5 rubygems-integration unzip zip
30. 0 upgraded, 15 newly installed, 0 to remove and 62 not upgraded.
31. Need to get 6499 kB of archives.
32. After this operation, 29.0 MB of additional disk space will be used.
33. Get:1 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 fonts-lato all 2.0-2 [2698 kB]
34. Get:2 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 javascript-common all 11 [6066 B]
35. Get:3 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 libjs-jquery all 3.2.1-1 [152 kB]
36. Get:4 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 rubygems-integration all 1.11 [4994 B]
37. Get:5 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 ruby2.5 amd64 2.5.1-1ubuntu1.13 [48.6 kB]
38. Get:6 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby amd64 1:2.5.1 [5712 B]
39. Get:7 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 rake all 12.3.1-1ubuntu0.1 [44.9 kB]
40. Get:8 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-did-you-mean all 1.2.0-2 [9700 B]
41. Get:9 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-minitest all 5.10.3-1 [38.6 kB]
42. Get:10 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-net-telnet all 0.1.1-2 [12.6 kB]
43. Get:11 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-power-assert all 0.3.0-1 [7952 B]
44. Get:12 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 ruby-test-unit all 3.2.5-1 [61.1 kB]
45. Get:13 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libruby2.5 amd64 2.5.1-1ubuntu1.13 [3074 kB]                              
46. Get:14 http://cn.archive.ubuntu.com/ubuntu bionic-updates/main amd64 unzip amd64 6.0-21ubuntu1.2 [168 kB]                                      
47. Get:15 http://cn.archive.ubuntu.com/ubuntu bionic/main amd64 zip amd64 3.0-11build1 [167 kB]                                                   
48. Fetched 6499 kB in 9s (722 kB/s)                                                                                                               
49. Selecting previously unselected package fonts-lato.
50. (Reading database ... 144903 files and directories currently installed.)
51. Preparing to unpack .../00-fonts-lato_2.0-2_all.deb ...
52. Unpacking fonts-lato (2.0-2) ...
53. Selecting previously unselected package javascript-common.
54. Preparing to unpack .../01-javascript-common_11_all.deb ...
55. Unpacking javascript-common (11) ...
56. Selecting previously unselected package libjs-jquery.
57. Preparing to unpack .../02-libjs-jquery_3.2.1-1_all.deb ...
58. Unpacking libjs-jquery (3.2.1-1) ...
59. Selecting previously unselected package rubygems-integration.
60. Preparing to unpack .../03-rubygems-integration_1.11_all.deb ...
61. Unpacking rubygems-integration (1.11) ...
62. Selecting previously unselected package ruby2.5.
63. Preparing to unpack .../04-ruby2.5_2.5.1-1ubuntu1.13_amd64.deb ...
64. Unpacking ruby2.5 (2.5.1-1ubuntu1.13) ...
65. Selecting previously unselected package ruby.
66. Preparing to unpack .../05-ruby_1%3a2.5.1_amd64.deb ...
67. Unpacking ruby (1:2.5.1) ...
68. Selecting previously unselected package rake.
69. Preparing to unpack .../06-rake_12.3.1-1ubuntu0.1_all.deb ...
70. Unpacking rake (12.3.1-1ubuntu0.1) ...
71. Selecting previously unselected package ruby-did-you-mean.
72. Preparing to unpack .../07-ruby-did-you-mean_1.2.0-2_all.deb ...
73. Unpacking ruby-did-you-mean (1.2.0-2) ...
74. Selecting previously unselected package ruby-minitest.
75. Preparing to unpack .../08-ruby-minitest_5.10.3-1_all.deb ...
76. Unpacking ruby-minitest (5.10.3-1) ...
77. Selecting previously unselected package ruby-net-telnet.
78. Preparing to unpack .../09-ruby-net-telnet_0.1.1-2_all.deb ...
79. Unpacking ruby-net-telnet (0.1.1-2) ...
80. Selecting previously unselected package ruby-power-assert.
81. Preparing to unpack .../10-ruby-power-assert_0.3.0-1_all.deb ...
82. Unpacking ruby-power-assert (0.3.0-1) ...
83. Selecting previously unselected package ruby-test-unit.
84. Preparing to unpack .../11-ruby-test-unit_3.2.5-1_all.deb ...
85. Unpacking ruby-test-unit (3.2.5-1) ...
86. Selecting previously unselected package libruby2.5:amd64.
87. Preparing to unpack .../12-libruby2.5_2.5.1-1ubuntu1.13_amd64.deb ...
88. Unpacking libruby2.5:amd64 (2.5.1-1ubuntu1.13) ...
89. Selecting previously unselected package unzip.
90. Preparing to unpack .../13-unzip_6.0-21ubuntu1.2_amd64.deb ...
91. Unpacking unzip (6.0-21ubuntu1.2) ...
92. Selecting previously unselected package zip.
93. Preparing to unpack .../14-zip_3.0-11build1_amd64.deb ...
94. Unpacking zip (3.0-11build1) ...
95. Setting up libjs-jquery (3.2.1-1) ...
96. Setting up unzip (6.0-21ubuntu1.2) ...
97. Setting up zip (3.0-11build1) ...
98. Setting up fonts-lato (2.0-2) ...
99. Setting up ruby-did-you-mean (1.2.0-2) ...
100. Setting up ruby-net-telnet (0.1.1-2) ...
101. Setting up rubygems-integration (1.11) ...
102. Setting up javascript-common (11) ...
103. Setting up ruby-minitest (5.10.3-1) ...
104. Setting up ruby-power-assert (0.3.0-1) ...
105. Setting up ruby2.5 (2.5.1-1ubuntu1.13) ...
106. Setting up ruby (1:2.5.1) ...
107. Setting up ruby-test-unit (3.2.5-1) ...
108. Setting up rake (12.3.1-1ubuntu0.1) ...
109. Setting up libruby2.5:amd64 (2.5.1-1ubuntu1.13) ...
110. Processing triggers for mime-support (3.60ubuntu1) ...
111. Processing triggers for libc-bin (2.27-3ubuntu1.5) ...
112. Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

复制代码

解决 redis-rtib.rb 执行报错

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb
2. Traceback (most recent call last):
3.     2: from /usr/bin/redis-trib.rb:25:in `<main>'
4.     1: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
5. /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- redis (LoadError)

复制代码

 

1. ##使用 gem 工具安装 ruby 的redis模块
2. [root@Redis-Ubuntu1804-node1:~]# gem install redis
3. Fetching: redis-5.0.6.gem (100%)
4. Successfully installed redis-5.0.6
5. Parsing documentation for redis-5.0.6
6. Installing ri documentation for redis-5.0.6
7. Done installing documentation for redis after 0 seconds
8. 1 gem installed<br><br>##离线模式安装 gem<br>[root@Redis-Ubuntu1804-node1:~]# wget https://rubygems.org/downloads/redis-4.0.3.gem<br>[root@Redis-Ubuntu1804-node1:~]# gem install redis-4.0.3.gem

复制代码

redis-trib.rb 常用命令

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb
2. Usage: redis-trib <command> <options> <arguments ...>
4.   create          host1:port1 ... hostN:portN  #创建集群
5.                   --replicas <arg>  #指定每个master节点的副本数，即每组主从的slave节点数，通常为1
6.   check           host:port  #检查集群信息
7.   info            host:port  #查看集群主机信息
8.   fix             host:port  #修复集群
9.                   --timeout <arg>
10.   reshard         host:port  #在线热机迁移指定的slots数据
11.                   --from <arg>
12.                   --to <arg>
13.                   --slots <arg>
14.                   --yes
15.                   --timeout <arg>
16.                   --pipeline <arg>
17.   rebalance       host:port  #平衡集群在线节点的slots数量
18.                   --weight <arg>
19.                   --auto-weights
20.                   --use-empty-masters
21.                   --timeout <arg>
22.                   --simulate
23.                   --pipeline <arg>
24.                   --threshold <arg>
25.   add-node        new_host:new_port existing_host:existing_port  #添加新的节点到集群中
26.                   --slave
27.                   --master-id <arg>
28.   del-node        host:port node_id  #删除集群中的指定节点
29.   set-timeout     host:port milliseconds  #设置节点超时时间
30.   call            host:port command arg arg .. arg  #在集群上所有节点上执行命令
31.   import          host:port  #导入外部redis服务器数据到当前集群
32.                   --from <arg>
33.                   --copy
34.                   --replace
35.   help            (show this help)
37. For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

复制代码

 
3.2、修改redis-trib.rb 添加连接redis密码

1. [root@Redis-Ubuntu1804-node1:~]# sed -i 's/@r = Redis.new.*/@r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60, :password => "'redis'")/' /usr/bin/redis-trib.rb<br>[root@Redis-Ubuntu1804-node1:~]# vim /usr/bin/redis-trib.rb
3. def connect(o={})
4.         return if @r
5.         print "Connecting to node #{self}: " if $verbose
6.         STDOUT.flush
7.         begin
8.             @r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60, :password => 'redis')                                 
9.             @r.ping
10.         rescue
11.             xputs "[ERR] Sorry, can't connect to node #{self}"
12.             exit 1 if o[:abort]
13.             @r = nil
14.         end
15.         xputs "OK" if $verbose
16.     end

复制代码

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 0 slots | 0 slaves.
3. [OK] 0 keys in 1 masters.
4. 0.00 keys per slot on average.
5. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
3.3、创建Cluster集群

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb create --replicas 1 10.0.0.20:6379 10.0.0.21:6379 10.0.0.22:6379 10.0.0.30:6379 10.0.0.31:6379 10.0.0.32:6379
2. >>> Creating cluster
3. >>> Performing hash slots allocation on 6 nodes...
4. Using 3 masters:
5. 10.0.0.20:6379
6. 10.0.0.21:6379
7. 10.0.0.22:6379
8. Adding replica 10.0.0.31:6379 to 10.0.0.20:6379
9. Adding replica 10.0.0.32:6379 to 10.0.0.21:6379
10. Adding replica 10.0.0.30:6379 to 10.0.0.22:6379
11. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
12.    slots:0-5460 (5461 slots) master
13. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
14.    slots:5461-10922 (5462 slots) master
15. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
16.    slots:10923-16383 (5461 slots) master
17. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
18.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
19. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
20.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
21. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
22.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
23. Can I set the above configuration? (type 'yes' to accept): yes
24. >>> Nodes configuration updated
25. >>> Assign a different config epoch to each node
26. >>> Sending CLUSTER MEET messages to join the cluster
27. Waiting for the cluster to join......
28. >>> Performing Cluster Check (using node 10.0.0.20:6379)
29. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
30.    slots:0-5460 (5461 slots) master
31.    1 additional replica(s)
32. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
33.    slots: (0 slots) slave
34.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
35. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
36.    slots:5461-10922 (5462 slots) master
37.    1 additional replica(s)
38. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
39.    slots: (0 slots) slave
40.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
41. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
42.    slots:10923-16383 (5461 slots) master
43.    1 additional replica(s)
44. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
45.    slots: (0 slots) slave
46.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
47. [OK] All nodes agree about slots configuration.
48. >>> Check for open slots...
49. >>> Check slots coverage...
50. [OK] All 16384 slots covered.
51. [root@Redis-Ubuntu1804-node1:~]#

复制代码

集群创建失败处理

1. #在所有节点上手动清空数据，并重置集群设置<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 --no-auth-warning flushall
2. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 --no-auth-warning cluster reset

复制代码

 
3.4、查看集群状态

查看集群信息自动生产的配置文件

1. [root@Redis-Ubuntu1804-node1:~]# cat /etc/redis/redis.conf | grep cluster-conf
2. # cluster-config-file nodes-6379.conf
3. cluster-config-file nodes-6379.conf
4. [root@Redis-Ubuntu1804-node1:~]# find / -name nodes-6379.conf
5. /var/lib/redis/nodes-6379.conf
6. [root@Redis-Ubuntu1804-node1:~]# cat /var/lib/redis/nodes-6379.conf
7. f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379@16379 slave 709b09b96f19209e7f0a1e3cfc5c035678491e4b 0 1681979886000 4 connected
8. 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379@16379 myself,master - 0 1681979886000 1 connected 0-5460
9. 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379@16379 master - 0 1681979888000 2 connected 5461-10922
10. c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379@16379 slave 1c2fee65ee50cdf66c862d04710672286b305f55 0 1681979888959 5 connected
11. 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379@16379 master - 0 1681979887000 3 connected 10923-16383
12. 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379@16379 slave 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 0 1681979888000 6 connected
13. vars currentEpoch 6 lastVoteEpoch 0
14. [root@Redis-Ubuntu1804-node1:~]#

复制代码

查看集群状态信息

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.
3. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.
4. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.
5. [OK] 0 keys in 3 masters.
6. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
9. >>> Performing Cluster Check (using node 10.0.0.20:6379)
10. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
11.    slots:0-5460 (5461 slots) master
12.    1 additional replica(s)
13. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
14.    slots: (0 slots) slave
15.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
16. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
17.    slots:5461-10922 (5462 slots) master
18.    1 additional replica(s)
19. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
20.    slots: (0 slots) slave
21.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
22. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
23.    slots:10923-16383 (5461 slots) master
24.    1 additional replica(s)
25. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
26.    slots: (0 slots) slave
27.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
28. [OK] All nodes agree about slots configuration.
29. >>> Check for open slots...
30. >>> Check slots coverage...
31. [OK] All 16384 slots covered.
34. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 cluster info
35. cluster_state:ok
36. cluster_slots_assigned:16384
37. cluster_slots_ok:16384
38. cluster_slots_pfail:0
39. cluster_slots_fail:0
40. cluster_known_nodes:6
41. cluster_size:3
42. cluster_current_epoch:6
43. cluster_my_epoch:1
44. cluster_stats_messages_ping_sent:1000
45. cluster_stats_messages_pong_sent:1033
46. cluster_stats_messages_sent:2033
47. cluster_stats_messages_ping_received:1028
48. cluster_stats_messages_pong_received:1000
49. cluster_stats_messages_meet_received:5
50. cluster_stats_messages_received:2033
53. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 cluster nodes
54. f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379@16379 slave 709b09b96f19209e7f0a1e3cfc5c035678491e4b 0 1681980883096 4 connected
55. 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379@16379 myself,master - 0 1681980882000 1 connected 0-5460
56. 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379@16379 master - 0 1681980885110 2 connected 5461-10922
57. c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379@16379 slave 1c2fee65ee50cdf66c862d04710672286b305f55 0 1681980883000 5 connected
58. 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379@16379 master - 0 1681980884000 3 connected 10923-16383
59. 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379@16379 slave 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 0 1681980884103 6 connected
61. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 info replication
62. # Replication
63. role:master
64. connected_slaves:1
65. slave0:ip=10.0.0.31,port=6379,state=online,offset=1428,lag=1
66. master_replid:33b67c12391d0787c01305ee41ccba34db5c843b
67. master_replid2:0000000000000000000000000000000000000000
68. master_repl_offset:1428
69. second_repl_offset:-1
70. repl_backlog_active:1
71. repl_backlog_size:1048576
72. repl_backlog_first_byte_offset:1
73. repl_backlog_histlen:1428
75. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.31 info replication
76. # Replication
77. role:slave
78. master_host:10.0.0.20
79. master_port:6379
80. master_link_status:up
81. master_last_io_seconds_ago:6
82. master_sync_in_progress:0
83. slave_repl_offset:1554
84. slave_priority:100
85. slave_read_only:1
86. connected_slaves:0
87. master_replid:33b67c12391d0787c01305ee41ccba34db5c843b
88. master_replid2:0000000000000000000000000000000000000000
89. master_repl_offset:1554
90. second_repl_offset:-1
91. repl_backlog_active:1
92. repl_backlog_size:1048576
93. repl_backlog_first_byte_offset:1
94. repl_backlog_histlen:1554

复制代码

 
3.5、模拟故障验证集群故障切换

使用脚本写入数据，查看集群信息

1. [root@Client-Ubuntu-1804-250:~/script/redis]# ./redis_cluster_newData.py 100
3. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
4. 10.0.0.20:6379 (1c2fee65...) -> 30 keys | 5461 slots | 1 slaves.
5. 10.0.0.21:6379 (337bf2a1...) -> 31 keys | 5462 slots | 1 slaves.
6. 10.0.0.22:6379 (709b09b9...) -> 39 keys | 5461 slots | 1 slaves.
7. [OK] 100 keys in 3 masters.
8. 0.01 keys per slot on average.
10. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 -c get keys11
11. "value11"
12. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 get keys11
13. "value11"
14. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.20 get keys12
15. (error) MOVED 14046 10.0.0.22:6379
16. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -p 6379 -h 10.0.0.22 get keys12
17. "value12"

复制代码

关闭 10.0.0.22 redis服务，模拟故障

1. [root@Redis-Ubuntu1804-node3:~]# systemctl stop redis
2. [root@Redis-Ubuntu1804-node3:~]#
3. ##初始信息
4. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
5. >>> Performing Cluster Check (using node 10.0.0.20:6379)
6. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
7.    slots:0-5460 (5461 slots) master
8.    1 additional replica(s)
9. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
10.    slots: (0 slots) slave
11.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
12. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
13.    slots:5461-10922 (5462 slots) master
14.    1 additional replica(s)
15. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
16.    slots: (0 slots) slave
17.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
18. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
19.    slots:10923-16383 (5461 slots) master
20.    1 additional replica(s)
21. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
22.    slots: (0 slots) slave
23.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
24. [OK] All nodes agree about slots configuration.
25. >>> Check for open slots...
26. >>> Check slots coverage...
27. [OK] All 16384 slots covered.
28. ##故障发生第一时间
29. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
30. [ERR] Sorry, can't connect to node 10.0.0.22:6379
31. *** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 709b09b96f19209e7f0a1e3cfc5c035678491e4b.
32. >>> Performing Cluster Check (using node 10.0.0.20:6379)
33. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
34.    slots:0-5460 (5461 slots) master
35.    1 additional replica(s)
36. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
37.    slots: (0 slots) slave
38.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
39. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
40.    slots:5461-10922 (5462 slots) master
41.    1 additional replica(s)
42. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
43.    slots: (0 slots) slave
44.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
45. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
46.    slots: (0 slots) slave
47.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
48. [OK] All nodes agree about slots configuration.
49. >>> Check for open slots...
50. >>> Check slots coverage...
51. [ERR] Not all 16384 slots are covered by nodes.
52. ##自动提升Msater
53. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
54. >>> Performing Cluster Check (using node 10.0.0.20:6379)
55. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
56.    slots:0-5460 (5461 slots) master
57.    1 additional replica(s)
58. M: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
59.    slots:10923-16383 (5461 slots) master
60.    0 additional replica(s)
61. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
62.    slots:5461-10922 (5462 slots) master
63.    1 additional replica(s)
64. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
65.    slots: (0 slots) slave
66.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
67. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
68.    slots: (0 slots) slave
69.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
70. [OK] All nodes agree about slots configuration.
71. >>> Check for open slots...
72. >>> Check slots coverage...
73. [OK] All 16384 slots covered.
74. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. 1021:M 20 Apr 17:45:33.030 * Marking node 709b09b96f19209e7f0a1e3cfc5c035678491e4b as failing (quorum reached).
2. 1021:M 20 Apr 17:45:33.030 # Cluster state changed: fail
3. 1021:M 20 Apr 17:45:34.067 # Failover auth granted to f9e73307b3146b25b51cec9a861463a616ed322b for epoch 7
4. 1021:M 20 Apr 17:45:34.070 # Cluster state changed: ok

复制代码

重新启动 10.0.0.22 redis服务，将被自动设置为slave节点

1. [root@Redis-Ubuntu1804-node3:~]# systemctl start redis
2. [root@Redis-Ubuntu1804-node3:~]#
5. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
6. >>> Performing Cluster Check (using node 10.0.0.20:6379)
7. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
8.    slots:0-5460 (5461 slots) master
9.    1 additional replica(s)
10. M: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
11.    slots:10923-16383 (5461 slots) master
12.    1 additional replica(s)
13. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
14.    slots:5461-10922 (5462 slots) master
15.    1 additional replica(s)
16. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
17.    slots: (0 slots) slave
18.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
19. S: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
20.    slots: (0 slots) slave
21.    replicates f9e73307b3146b25b51cec9a861463a616ed322b
22. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
23.    slots: (0 slots) slave
24.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
25. [OK] All nodes agree about slots configuration.
26. >>> Check for open slots...
27. >>> Check slots coverage...
28. [OK] All 16384 slots covered.

复制代码

1. 1014:M 20 Apr 17:45:33.030 * Marking node 709b09b96f19209e7f0a1e3cfc5c035678491e4b as failing (quorum reached).
2. 1014:M 20 Apr 17:45:33.030 # Cluster state changed: fail
3. 1014:M 20 Apr 17:45:34.067 # Failover auth granted to f9e73307b3146b25b51cec9a861463a616ed322b for epoch 7
4. 1014:M 20 Apr 17:45:34.070 # Cluster state changed: ok
5. 1014:M 20 Apr 17:51:47.852 * Clear FAIL state for node 709b09b96f19209e7f0a1e3cfc5c035678491e4b: master without slots is reachable again.

复制代码

 
 
4、Cluster集群节点维护

4.1、集群维护之动态扩容

当生产环境集群负载达到上限，需要扩充Master节点以环境集群性能瓶颈，将涉及到集群扩容操作
*通常生产环境下建议集群Master节点数应设置为奇数（3、5、7），以避免出现脑裂现象

 

 
使用集群管理工具的 add-node 命令
  add-node        new_host:new_port existing_host:existing_port  #添加新的节点到集群中                  --slave                  --master-id

4.1.1、Redis5 集群节点扩容 

 
查看当前集群节点信息

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.
4. 10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.
5. [OK] 445 keys in 3 masters.
6. 0.03 keys per slot on average.
7. >>> Performing Cluster Check (using node 10.0.0.20:6379)
8. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
9.    slots:[0-5460] (5461 slots) master
10.    1 additional replica(s)
11. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
12.    slots: (0 slots) slave
13.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
14. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
15.    slots:[5461-10922] (5462 slots) master
16.    1 additional replica(s)
17. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
18.    slots: (0 slots) slave
19.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
20. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
21.    slots: (0 slots) slave
22.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
23. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
24.    slots:[10923-16383] (5461 slots) master
25.    1 additional replica(s)
26. [OK] All nodes agree about slots configuration.
27. >>> Check for open slots...
28. >>> Check slots coverage...
29. [OK] All 16384 slots covered.

复制代码

将节点 10.0.0.23 添加至集群中

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster add-node 10.0.0.23:6379 10.0.0.20:6379
2. >>> Adding node 10.0.0.23:6379 to cluster 10.0.0.20:6379
3. >>> Performing Cluster Check (using node 10.0.0.20:6379)
4. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
5.    slots:[0-5460] (5461 slots) master
6.    1 additional replica(s)
7. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
8.    slots: (0 slots) slave
9.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
10. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
11.    slots:[5461-10922] (5462 slots) master
12.    1 additional replica(s)
13. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
14.    slots: (0 slots) slave
15.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
16. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
17.    slots: (0 slots) slave
18.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
19. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
20.    slots:[10923-16383] (5461 slots) master
21.    1 additional replica(s)
22. [OK] All nodes agree about slots configuration.
23. >>> Check for open slots...
24. >>> Check slots coverage...
25. [OK] All 16384 slots covered.
26. >>> Send CLUSTER MEET to node 10.0.0.23:6379 to make it join the cluster.
27. [OK] New node added correctly.
28. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. ##10.0.0.23 节点已经加入到集群中，但是未分配任何槽位数据，也没有从节点<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.
4. 10.0.0.23:6379 (ac947d4f...) -> 0 keys | 0 slots | 0 slaves.
5. 10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.
6. [OK] 445 keys in 4 masters.
7. 0.03 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 150 keys | 5461 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 141 keys | 5462 slots | 1 slaves.
4. 10.0.0.23:6379 (ac947d4f...) -> 0 keys | 0 slots | 0 slaves.
5. 10.0.0.30:6379 (5c9789e1...) -> 154 keys | 5461 slots | 1 slaves.
6. [OK] 445 keys in 4 masters.
7. 0.03 keys per slot on average.
8. >>> Performing Cluster Check (using node 10.0.0.20:6379)
9. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
10.    slots:[0-5460] (5461 slots) master
11.    1 additional replica(s)
12. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
13.    slots: (0 slots) slave
14.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
15. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
16.    slots:[5461-10922] (5462 slots) master
17.    1 additional replica(s)
18. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
19.    slots: (0 slots) master
20. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
21.    slots: (0 slots) slave
22.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
23. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
24.    slots: (0 slots) slave
25.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
26. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
27.    slots:[10923-16383] (5461 slots) master
28.    1 additional replica(s)
29. [OK] All nodes agree about slots configuration.
30. >>> Check for open slots...
31. >>> Check slots coverage...
32. [OK] All 16384 slots covered.
33. [root@Redis-Ubuntu1804-node1:~]#

复制代码

备份并清空集群所有Master节点上的数据，为新节点分配槽位

 

1.   reshard         host:port  #在线热机迁移指定的slots数据
2.                   --from <arg>
3.                   --to <arg>
4.                   --slots <arg>
5.                   --yes
6.                   --timeout <arg>
7.                   --pipeline <arg>

复制代码

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.20 --no-auth-warning flushall
2. OK
3. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.21 --no-auth-warning flushall
4. OK
5. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.30 --no-auth-warning flushall
6. OK

复制代码

1. ##为新节点分配槽位<br>[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379
2. >>> Performing Cluster Check (using node 10.0.0.20:6379)
3. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
4.    slots:[0-5460] (5461 slots) master
5.    1 additional replica(s)
6. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
7.    slots: (0 slots) slave
8.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
9. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
10.    slots:[5461-10922] (5462 slots) master
11.    1 additional replica(s)
12. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
13.    slots: (0 slots) master
14. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
15.    slots: (0 slots) slave
16.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
17. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
18.    slots: (0 slots) slave
19.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
20. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
21.    slots:[10923-16383] (5461 slots) master
22.    1 additional replica(s)
23. [OK] All nodes agree about slots configuration.
24. >>> Check for open slots...
25. >>> Check slots coverage...
26. [OK] All 16384 slots covered.
27. How many slots do you want to move (from 1 to 16384)? 4096   ##需要进行重新分配的槽位数
28. What is the receiving node ID? ac947d4fc970b79c2364ce305441906b462f8b65  ##接收分配槽位的节点 node id
29. Please enter all the source node IDs.  ##提供重新分配槽位的节点node id
30.   Type 'all' to use all the nodes as source nodes for the hash slots. ##输入all将从拥有槽位的所有节点进行自动分配
31.   Type 'done' once you entered all the source nodes IDs. ##输入提供槽位的节点node id，输入done结束
32. Source node #1: all
33. ...
34. ...
35. ...
36.     Moving slot 12280 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
37.     Moving slot 12281 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
38.     Moving slot 12282 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
39.     Moving slot 12283 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
40.     Moving slot 12284 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
41.     Moving slot 12285 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
42.     Moving slot 12286 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
43.     Moving slot 12287 from 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
44. Do you want to proceed with the proposed reshard plan (yes/no)? yes ##确认槽位分配方案
45. ...
46. ...
47. ...
48. Moving slot 12280 from 10.0.0.30:6379 to 10.0.0.23:6379:
49. Moving slot 12281 from 10.0.0.30:6379 to 10.0.0.23:6379:
50. Moving slot 12282 from 10.0.0.30:6379 to 10.0.0.23:6379:
51. Moving slot 12283 from 10.0.0.30:6379 to 10.0.0.23:6379:
52. Moving slot 12284 from 10.0.0.30:6379 to 10.0.0.23:6379:
53. Moving slot 12285 from 10.0.0.30:6379 to 10.0.0.23:6379:
54. Moving slot 12286 from 10.0.0.30:6379 to 10.0.0.23:6379:
55. Moving slot 12287 from 10.0.0.30:6379 to 10.0.0.23:6379:

复制代码

查看槽位分配最终结果

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 0 keys | 4096 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 0 keys | 4096 slots | 1 slaves.
4. 10.0.0.23:6379 (ac947d4f...) -> 0 keys | 4096 slots | 0 slaves.
5. 10.0.0.30:6379 (5c9789e1...) -> 0 keys | 4096 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
将 10.0.0.33 添加至集群中，并设置为 10.0.0.23 的从节点

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster add-node 10.0.0.33:6379 10.0.0.20:6379 --cluster-slave --cluster-master-id ac947d4fc970b79c2364ce305441906b462f8b65
2. >>> Adding node 10.0.0.33:6379 to cluster 10.0.0.20:6379
3. >>> Performing Cluster Check (using node 10.0.0.20:6379)
4. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
5.    slots:[1365-5460] (4096 slots) master
6.    1 additional replica(s)
7. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
8.    slots: (0 slots) slave
9.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
10. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
11.    slots:[6827-10922] (4096 slots) master
12.    1 additional replica(s)
13. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
14.    slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
15. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
16.    slots: (0 slots) slave
17.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
18. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
19.    slots: (0 slots) slave
20.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
21. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
22.    slots:[12288-16383] (4096 slots) master
23.    1 additional replica(s)
24. [OK] All nodes agree about slots configuration.
25. >>> Check for open slots...
26. >>> Check slots coverage...
27. [OK] All 16384 slots covered.
28. >>> Send CLUSTER MEET to node 10.0.0.33:6379 to make it join the cluster.
29. Waiting for the cluster to join
31. >>> Configure node as replica of 10.0.0.23:6379.
32. [OK] New node added correctly.
33. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
查看集群节点信息

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 0 keys | 4096 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 0 keys | 4096 slots | 1 slaves.
4. 10.0.0.23:6379 (ac947d4f...) -> 0 keys | 4096 slots | 1 slaves.
5. 10.0.0.30:6379 (5c9789e1...) -> 0 keys | 4096 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. >>> Performing Cluster Check (using node 10.0.0.20:6379)
9. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
10.    slots:[1365-5460] (4096 slots) master
11.    1 additional replica(s)
12. S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379
13.    slots: (0 slots) slave
14.    replicates ac947d4fc970b79c2364ce305441906b462f8b65
15. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
16.    slots: (0 slots) slave
17.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
18. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
19.    slots:[6827-10922] (4096 slots) master
20.    1 additional replica(s)
21. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
22.    slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
23.    1 additional replica(s)
24. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
25.    slots: (0 slots) slave
26.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
27. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
28.    slots: (0 slots) slave
29.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
30. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
31.    slots:[12288-16383] (4096 slots) master
32.    1 additional replica(s)
33. [OK] All nodes agree about slots configuration.
34. >>> Check for open slots...
35. >>> Check slots coverage...
36. [OK] All 16384 slots covered.

复制代码

 
4.1.2、Redis 3/4 集群节点扩容

当前集群节点信息

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.
4. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.
5. [OK] 0 keys in 3 masters.
6. 0.00 keys per slot on average.
7. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
8. >>> Performing Cluster Check (using node 10.0.0.20:6379)
9. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
10.    slots:0-5460 (5461 slots) master
11.    1 additional replica(s)
12. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
13.    slots: (0 slots) slave
14.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
15. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
16.    slots:10923-16383 (5461 slots) master
17.    1 additional replica(s)
18. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
19.    slots: (0 slots) slave
20.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
21. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
22.    slots: (0 slots) slave
23.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
24. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
25.    slots:5461-10922 (5462 slots) master
26.    1 additional replica(s)
27. [OK] All nodes agree about slots configuration.
28. >>> Check for open slots...
29. >>> Check slots coverage...
30. [OK] All 16384 slots covered.
31. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
将节点 10.0.0.23 添加至集群中

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb add-node 10.0.0.23:6379 10.0.0.20:6379
2. >>> Adding node 10.0.0.23:6379 to cluster 10.0.0.20:6379
3. >>> Performing Cluster Check (using node 10.0.0.20:6379)
4. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
5.    slots:0-5460 (5461 slots) master
6.    1 additional replica(s)
7. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
8.    slots: (0 slots) slave
9.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
10. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
11.    slots:10923-16383 (5461 slots) master
12.    1 additional replica(s)
13. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
14.    slots: (0 slots) slave
15.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
16. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
17.    slots: (0 slots) slave
18.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
19. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
20.    slots:5461-10922 (5462 slots) master
21.    1 additional replica(s)
22. [OK] All nodes agree about slots configuration.
23. >>> Check for open slots...
24. >>> Check slots coverage...
25. [OK] All 16384 slots covered.
26. >>> Send CLUSTER MEET to node 10.0.0.23:6379 to make it join the cluster.
27. [OK] New node added correctly.
28. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.
4. 10.0.0.23:6379 (0e504f59...) -> 0 keys | 0 slots | 0 slaves.
5. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
9. >>> Performing Cluster Check (using node 10.0.0.20:6379)
10. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
11.    slots:0-5460 (5461 slots) master
12.    1 additional replica(s)
13. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
14.    slots: (0 slots) slave
15.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
16. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
17.    slots:10923-16383 (5461 slots) master
18.    1 additional replica(s)
19. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
20.    slots: (0 slots) slave
21.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
22. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
23.    slots: (0 slots) master
24.    0 additional replica(s)
25. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
26.    slots: (0 slots) slave
27.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
28. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
29.    slots:5461-10922 (5462 slots) master
30.    1 additional replica(s)
31. [OK] All nodes agree about slots configuration.
32. >>> Check for open slots...
33. >>> Check slots coverage...
34. [OK] All 16384 slots covered.
35. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
备份并清空集群所有Master节点上的数据，为新节点分配槽位

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb call 10.0.0.20:6379 flushall
2. >>> Calling FLUSHALL
3. 10.0.0.20:6379: OK
4. 10.0.0.31:6379: READONLY You can't write against a read only slave.
5. 10.0.0.22:6379: OK
6. 10.0.0.32:6379: READONLY You can't write against a read only slave.
7. 10.0.0.23:6379: OK
8. 10.0.0.30:6379: READONLY You can't write against a read only slave.
9. 10.0.0.21:6379: OK
10. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379
2. >>> Performing Cluster Check (using node 10.0.0.20:6379)
3. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
4.    slots:0-5460 (5461 slots) master
5.    1 additional replica(s)
6. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
7.    slots: (0 slots) slave
8.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
9. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
10.    slots:10923-16383 (5461 slots) master
11.    1 additional replica(s)
12. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
13.    slots: (0 slots) slave
14.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
15. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
16.    slots: (0 slots) master
17.    0 additional replica(s)
18. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
19.    slots: (0 slots) slave
20.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
21. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
22.    slots:5461-10922 (5462 slots) master
23.    1 additional replica(s)
24. [OK] All nodes agree about slots configuration.
25. >>> Check for open slots...
26. >>> Check slots coverage...
27. [OK] All 16384 slots covered.
28. How many slots do you want to move (from 1 to 16384)? 4096
29. What is the receiving node ID? 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
30. Please enter all the source node IDs.
31.   Type 'all' to use all the nodes as source nodes for the hash slots.
32.   Type 'done' once you entered all the source nodes IDs.
33. Source node #1:all
34. Do you want to proceed with the proposed reshard plan (yes/no)? yes
35. ...
36. ...
37. ...
38. Moving slot 12281 from 10.0.0.22:6379 to 10.0.0.23:6379:
39. Moving slot 12282 from 10.0.0.22:6379 to 10.0.0.23:6379:
40. Moving slot 12283 from 10.0.0.22:6379 to 10.0.0.23:6379:
41. Moving slot 12284 from 10.0.0.22:6379 to 10.0.0.23:6379:
42. Moving slot 12285 from 10.0.0.22:6379 to 10.0.0.23:6379:
43. Moving slot 12286 from 10.0.0.22:6379 to 10.0.0.23:6379:
44. Moving slot 12287 from 10.0.0.22:6379 to 10.0.0.23:6379:

复制代码

 
查看槽位分配最终结果

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 4096 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 4096 slots | 1 slaves.
4. 10.0.0.23:6379 (0e504f59...) -> 0 keys | 4096 slots | 0 slaves.
5. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 4096 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
9. >>> Performing Cluster Check (using node 10.0.0.20:6379)
10. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
11.    slots:1365-5460 (4096 slots) master
12.    1 additional replica(s)
13. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
14.    slots: (0 slots) slave
15.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
16. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
17.    slots:12288-16383 (4096 slots) master
18.    1 additional replica(s)
19. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
20.    slots: (0 slots) slave
21.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
22. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
23.    slots:0-1364,5461-6826,10923-12287 (4096 slots) master
24.    0 additional replica(s)
25. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
26.    slots: (0 slots) slave
27.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
28. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
29.    slots:6827-10922 (4096 slots) master
30.    1 additional replica(s)
31. [OK] All nodes agree about slots configuration.
32. >>> Check for open slots...
33. >>> Check slots coverage...
34. [OK] All 16384 slots covered.
35. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
将 10.0.0.33 添加至集群中，并设置为 10.0.0.23 的从节点

 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb add-node --slave --master-id 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.33:6379 10.0.0.20:6379
2. >>> Adding node 10.0.0.33:6379 to cluster 10.0.0.20:6379
3. >>> Performing Cluster Check (using node 10.0.0.20:6379)
4. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
5.    slots:1365-5460 (4096 slots) master
6.    1 additional replica(s)
7. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
8.    slots: (0 slots) slave
9.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
10. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
11.    slots:12288-16383 (4096 slots) master
12.    1 additional replica(s)
13. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
14.    slots: (0 slots) slave
15.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
16. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
17.    slots:0-1364,5461-6826,10923-12287 (4096 slots) master
18.    0 additional replica(s)
19. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
20.    slots: (0 slots) slave
21.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
22. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
23.    slots:6827-10922 (4096 slots) master
24.    1 additional replica(s)
25. [OK] All nodes agree about slots configuration.
26. >>> Check for open slots...
27. >>> Check slots coverage...
28. [OK] All 16384 slots covered.
29. >>> Send CLUSTER MEET to node 10.0.0.33:6379 to make it join the cluster.
30. Waiting for the cluster to join.
31. >>> Configure node as replica of 10.0.0.23:6379.
32. [OK] New node added correctly.
33. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
查看集群节点信息

 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 4096 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 4096 slots | 1 slaves.
4. 10.0.0.23:6379 (0e504f59...) -> 0 keys | 4096 slots | 1 slaves.
5. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 4096 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
9. >>> Performing Cluster Check (using node 10.0.0.20:6379)
10. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
11.    slots:1365-5460 (4096 slots) master
12.    1 additional replica(s)
13. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
14.    slots: (0 slots) slave
15.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
16. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
17.    slots:12288-16383 (4096 slots) master
18.    1 additional replica(s)
19. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
20.    slots: (0 slots) slave
21.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
22. S: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379
23.    slots: (0 slots) slave
24.    replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
25. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
26.    slots:0-1364,5461-6826,10923-12287 (4096 slots) master
27.    1 additional replica(s)
28. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
29.    slots: (0 slots) slave
30.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
31. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
32.    slots:6827-10922 (4096 slots) master
33.    1 additional replica(s)
34. [OK] All nodes agree about slots configuration.
35. >>> Check for open slots...
36. >>> Check slots coverage...
37. [OK] All 16384 slots covered.

复制代码

 
4.2、集群维护之动态缩容

当生产环境集群中的某台服务器存在故障风险 或 集群性能闲置需要对集群节点进行服务器替换或资源回收操作时，需要涉及到动态缩容操作
下线Master节点10.0.0.23 及对应的 Slave节点 10.0.0.33

 

del-node       host:port node_id 

4.2.1、Redis 5 集群节点缩容

查看当前集群状态

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 0 keys | 4096 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 0 keys | 4096 slots | 1 slaves.
4. 10.0.0.23:6379 (ac947d4f...) -> 0 keys | 4096 slots | 1 slaves.
5. 10.0.0.30:6379 (5c9789e1...) -> 0 keys | 4096 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. >>> Performing Cluster Check (using node 10.0.0.20:6379)
9. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
10.    slots:[1365-5460] (4096 slots) master
11.    1 additional replica(s)
12. S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379
13.    slots: (0 slots) slave
14.    replicates ac947d4fc970b79c2364ce305441906b462f8b65
15. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
16.    slots: (0 slots) slave
17.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
18. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
19.    slots:[6827-10922] (4096 slots) master
20.    1 additional replica(s)
21. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
22.    slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
23.    1 additional replica(s)
24. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
25.    slots: (0 slots) slave
26.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
27. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
28.    slots: (0 slots) slave
29.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
30. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
31.    slots:[12288-16383] (4096 slots) master
32.    1 additional replica(s)
33. [OK] All nodes agree about slots configuration.
34. >>> Check for open slots...
35. >>> Check slots coverage...
36. [OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#

复制代码

 
将删除Master节点的槽位分配至其他可用Master节点

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379
2. >>> Performing Cluster Check (using node 10.0.0.20:6379)
3. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
4.    slots:[1365-5460] (4096 slots) master
5.    1 additional replica(s)
6. S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379
7.    slots: (0 slots) slave
8.    replicates ac947d4fc970b79c2364ce305441906b462f8b65
9. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
10.    slots: (0 slots) slave
11.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
12. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
13.    slots:[6827-10922] (4096 slots) master
14.    1 additional replica(s)
15. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
16.    slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
17.    1 additional replica(s)
18. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
19.    slots: (0 slots) slave
20.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
21. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
22.    slots: (0 slots) slave
23.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
24. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
25.    slots:[12288-16383] (4096 slots) master
26.    1 additional replica(s)
27. [OK] All nodes agree about slots configuration.
28. >>> Check for open slots...
29. >>> Check slots coverage...
30. [OK] All 16384 slots covered.
31. How many slots do you want to move (from 1 to 16384)? 1365
32. What is the receiving node ID? 6b142fe3438c3ea488c34366e5ea5a298f89f518
33. Please enter all the source node IDs.
34.   Type 'all' to use all the nodes as source nodes for the hash slots.
35.   Type 'done' once you entered all the source nodes IDs.
36. Source node #1: ac947d4fc970b79c2364ce305441906b462f8b65
37. Source node #2: done
38. Do you want to proceed with the proposed reshard plan (yes/no)? yes
41. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379
42. >>> Performing Cluster Check (using node 10.0.0.20:6379)
43. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
44.    slots:[0-5460] (5461 slots) master
45.    1 additional replica(s)
46. S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379
47.    slots: (0 slots) slave
48.    replicates ac947d4fc970b79c2364ce305441906b462f8b65
49. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
50.    slots: (0 slots) slave
51.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
52. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
53.    slots:[6827-10922] (4096 slots) master
54.    1 additional replica(s)
55. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
56.    slots:[5461-6826],[10923-12287] (2731 slots) master
57.    1 additional replica(s)
58. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
59.    slots: (0 slots) slave
60.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
61. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
62.    slots: (0 slots) slave
63.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
64. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
65.    slots:[12288-16383] (4096 slots) master
66.    1 additional replica(s)
67. [OK] All nodes agree about slots configuration.
68. >>> Check for open slots...
69. >>> Check slots coverage...
70. [OK] All 16384 slots covered.
71. How many slots do you want to move (from 1 to 16384)? 1366
72. What is the receiving node ID? a4b387d82ac06e5cbf212e48c781e1c27b50320f
73. Please enter all the source node IDs.
74.   Type 'all' to use all the nodes as source nodes for the hash slots.
75.   Type 'done' once you entered all the source nodes IDs.
76. Source node #1: ac947d4fc970b79c2364ce305441906b462f8b65
77. Source node #2: done
78. Do you want to proceed with the proposed reshard plan (yes/no)? yes
81. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster reshard 10.0.0.20:6379
82. >>> Performing Cluster Check (using node 10.0.0.20:6379)
83. M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379
84.    slots:[0-5460] (5461 slots) master
85.    1 additional replica(s)
86. S: 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379
87.    slots: (0 slots) slave
88.    replicates ac947d4fc970b79c2364ce305441906b462f8b65
89. S: 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379
90.    slots: (0 slots) slave
91.    replicates 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
92. M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379
93.    slots:[5461-10922] (5462 slots) master
94.    1 additional replica(s)
95. M: ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379
96.    slots:[10923-12287] (1365 slots) master
97.    1 additional replica(s)
98. S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379
99.    slots: (0 slots) slave
100.    replicates a4b387d82ac06e5cbf212e48c781e1c27b50320f
101. S: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379
102.    slots: (0 slots) slave
103.    replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518
104. M: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379
105.    slots:[12288-16383] (4096 slots) master
106.    1 additional replica(s)
107. [OK] All nodes agree about slots configuration.
108. >>> Check for open slots...
109. >>> Check slots coverage...
110. [OK] All 16384 slots covered.
111. How many slots do you want to move (from 1 to 16384)? 1365
112. What is the receiving node ID? 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34
113. Please enter all the source node IDs.
114.   Type 'all' to use all the nodes as source nodes for the hash slots.
115.   Type 'done' once you entered all the source nodes IDs.
116. Source node #1: ac947d4fc970b79c2364ce305441906b462f8b65
117. Source node #2: done
118. Do you want to proceed with the proposed reshard plan (yes/no)? yes

复制代码

查看当前槽位分配结果

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.
4. 10.0.0.23:6379 (ac947d4f...) -> 0 keys | 0 slots | 0 slaves.
5. 10.0.0.30:6379 (5c9789e1...) -> 0 keys | 5461 slots | 2 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
删除下线节点

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes
2. 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007712000 15 connected
3. 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007713000 15 connected
4. a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682007713079 14 connected 5461-10922
5. ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379@16379 master - 0 1682007710000 12 connected
6. 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682007711000 14 connected
7. 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682007712000 13 connected
8. 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682007714086 15 connected 10923-16383
9. 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1682007709000 13 connected 0-5460
10. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster del-node 10.0.0.20:6379 ac947d4fc970b79c2364ce305441906b462f8b65
11. >>> Removing node ac947d4fc970b79c2364ce305441906b462f8b65 from cluster 10.0.0.20:6379
12. >>> Sending CLUSTER FORGET messages to the cluster...
13. >>> SHUTDOWN the node.
14. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster del-node 10.0.0.20:6379 829eef61e20def1a36e02a99f1650c9267be2108
15. >>> Removing node 829eef61e20def1a36e02a99f1650c9267be2108 from cluster 10.0.0.20:6379
16. >>> Sending CLUSTER FORGET messages to the cluster...
17. >>> SHUTDOWN the node.

复制代码

 
查看缩容后的集群状态

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes
2. 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682008285000 15 connected
3. a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682008285000 14 connected 5461-10922
4. 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682008286181 14 connected
5. 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682008283157 13 connected
6. 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682008285173 15 connected 10923-16383
7. 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1682008283000 13 connected 0-5460
9. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379
10. 10.0.0.20:6379 (6b142fe3...) -> 0 keys | 5461 slots | 1 slaves.
11. 10.0.0.21:6379 (a4b387d8...) -> 0 keys | 5462 slots | 1 slaves.
12. 10.0.0.30:6379 (5c9789e1...) -> 0 keys | 5461 slots | 1 slaves.
13. [OK] 0 keys in 3 masters.
14. 0.00 keys per slot on average.
15. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 清理下线节点node配置

被移除集群的redis节点进程将被自动关闭，需要先清理原node配置后，重新启动服务，否则节点将误以为自身仍在集群中

1. [root@Redis-Ubuntu1804-node4:~]# cat /app/redis/data/nodes-6379.conf
2. 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 master - 0 1682007475000 13 connected 0-5460
3. 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682007476000 14 connected
4. 829eef61e20def1a36e02a99f1650c9267be2108 10.0.0.33:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007474000 15 connected
5. ac947d4fc970b79c2364ce305441906b462f8b65 10.0.0.23:6379@16379 myself,master - 0 1682007470000 12 connected
6. a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682007475000 14 connected 5461-10922
7. 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682007477017 15 connected 10923-16383
8. 07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 slave 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 0 1682007476000 15 connected
9. 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682007476010 13 connected
10. vars currentEpoch 15 lastVoteEpoch 0
11. [root@Redis-Ubuntu1804-node4:~]# rm -rf /app/redis/data/nodes-6379.conf

复制代码

 
 
4.2.2、Redis 3/4 集群节点缩容

查看当前集群状态

 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 4096 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 4096 slots | 1 slaves.
4. 10.0.0.23:6379 (0e504f59...) -> 0 keys | 4096 slots | 1 slaves.
5. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 4096 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
9. >>> Performing Cluster Check (using node 10.0.0.20:6379)
10. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
11.    slots:1365-5460 (4096 slots) master
12.    1 additional replica(s)
13. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
14.    slots: (0 slots) slave
15.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
16. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
17.    slots:12288-16383 (4096 slots) master
18.    1 additional replica(s)
19. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
20.    slots: (0 slots) slave
21.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
22. S: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379
23.    slots: (0 slots) slave
24.    replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
25. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
26.    slots:0-1364,5461-6826,10923-12287 (4096 slots) master
27.    1 additional replica(s)
28. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
29.    slots: (0 slots) slave
30.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
31. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
32.    slots:6827-10922 (4096 slots) master
33.    1 additional replica(s)
34. [OK] All nodes agree about slots configuration.
35. >>> Check for open slots...
36. >>> Check slots coverage...
37. [OK] All 16384 slots covered.[root@Redis-Ubuntu1804-node1:~]#

复制代码

 
将删除Master节点的槽位分配至其他可用Master节点 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379
2. >>> Performing Cluster Check (using node 10.0.0.20:6379)
3. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
4.    slots:1365-5460 (4096 slots) master
5.    1 additional replica(s)
6. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
7.    slots: (0 slots) slave
8.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
9. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
10.    slots:12288-16383 (4096 slots) master
11.    1 additional replica(s)
12. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
13.    slots: (0 slots) slave
14.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
15. S: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379
16.    slots: (0 slots) slave
17.    replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
18. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
19.    slots:0-1364,5461-6826,10923-12287 (4096 slots) master
20.    1 additional replica(s)
21. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
22.    slots: (0 slots) slave
23.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
24. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
25.    slots:6827-10922 (4096 slots) master
26.    1 additional replica(s)
27. [OK] All nodes agree about slots configuration.
28. >>> Check for open slots...
29. >>> Check slots coverage...
30. [OK] All 16384 slots covered.
31. How many slots do you want to move (from 1 to 16384)? 1365
32. What is the receiving node ID? 1c2fee65ee50cdf66c862d04710672286b305f55
33. Please enter all the source node IDs.
34.   Type 'all' to use all the nodes as source nodes for the hash slots.
35.   Type 'done' once you entered all the source nodes IDs.
36. Source node #1:0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
37. Source node #2:done
38. Do you want to proceed with the proposed reshard plan (yes/no)? yes
41. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379
42. >>> Performing Cluster Check (using node 10.0.0.20:6379)
43. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
44.    slots:0-5460 (5461 slots) master
45.    1 additional replica(s)
46. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
47.    slots: (0 slots) slave
48.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
49. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
50.    slots:12288-16383 (4096 slots) master
51.    1 additional replica(s)
52. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
53.    slots: (0 slots) slave
54.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
55. S: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379
56.    slots: (0 slots) slave
57.    replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
58. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
59.    slots:5461-6826,10923-12287 (2731 slots) master
60.    1 additional replica(s)
61. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
62.    slots: (0 slots) slave
63.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
64. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
65.    slots:6827-10922 (4096 slots) master
66.    1 additional replica(s)
67. [OK] All nodes agree about slots configuration.
68. >>> Check for open slots...
69. >>> Check slots coverage...
70. [OK] All 16384 slots covered.
71. How many slots do you want to move (from 1 to 16384)? 1366
72. What is the receiving node ID? 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
73. Please enter all the source node IDs.
74.   Type 'all' to use all the nodes as source nodes for the hash slots.
75.   Type 'done' once you entered all the source nodes IDs.
76. Source node #1:0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
77. Source node #2:done
78. Do you want to proceed with the proposed reshard plan (yes/no)? yes
80. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb reshard 10.0.0.20:6379
81. >>> Performing Cluster Check (using node 10.0.0.20:6379)
82. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
83.    slots:0-5460 (5461 slots) master
84.    1 additional replica(s)
85. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
86.    slots: (0 slots) slave
87.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
88. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
89.    slots:12288-16383 (4096 slots) master
90.    1 additional replica(s)
91. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
92.    slots: (0 slots) slave
93.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
94. S: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379
95.    slots: (0 slots) slave
96.    replicates 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
97. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
98.    slots:10923-12287 (1365 slots) master
99.    1 additional replica(s)
100. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
101.    slots: (0 slots) slave
102.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
103. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
104.    slots:5461-10922 (5462 slots) master
105.    1 additional replica(s)
106. [OK] All nodes agree about slots configuration.
107. >>> Check for open slots...
108. >>> Check slots coverage...
109. [OK] All 16384 slots covered.
110. How many slots do you want to move (from 1 to 16384)? 1365
111. What is the receiving node ID? 709b09b96f19209e7f0a1e3cfc5c035678491e4b
112. Please enter all the source node IDs.
113.   Type 'all' to use all the nodes as source nodes for the hash slots.
114.   Type 'done' once you entered all the source nodes IDs.
115. Source node #1:0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
116. Source node #2:done
117. Do you want to proceed with the proposed reshard plan (yes/no)? yes

复制代码

 
查看当前槽位分配结果

 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 2 slaves.
4. 10.0.0.23:6379 (0e504f59...) -> 0 keys | 0 slots | 0 slaves.
5. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.
6. [OK] 0 keys in 4 masters.
7. 0.00 keys per slot on average.
8. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
9. >>> Performing Cluster Check (using node 10.0.0.20:6379)
10. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
11.    slots:0-5460 (5461 slots) master
12.    1 additional replica(s)
13. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
14.    slots: (0 slots) slave
15.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
16. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
17.    slots:10923-16383 (5461 slots) master
18.    2 additional replica(s)
19. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
20.    slots: (0 slots) slave
21.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
22. S: 6e761ee02c9c17838e4643b9030ddfe327f1c742 10.0.0.33:6379
23.    slots: (0 slots) slave
24.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
25. M: 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a 10.0.0.23:6379
26.    slots: (0 slots) master
27.    0 additional replica(s)
28. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
29.    slots: (0 slots) slave
30.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
31. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
32.    slots:5461-10922 (5462 slots) master
33.    1 additional replica(s)
34. [OK] All nodes agree about slots configuration.
35. >>> Check for open slots...
36. >>> Check slots coverage...
37. [OK] All 16384 slots covered.
38. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
删除下线节点

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb del-node 10.0.0.20:6379 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a
2. >>> Removing node 0e504f596a5e8b55d0a38f5eacd3bd4b5c8f462a from cluster 10.0.0.20:6379
3. >>> Sending CLUSTER FORGET messages to the cluster...
4. >>> SHUTDOWN the node.
5. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb del-node 10.0.0.20:6379 6e761ee02c9c17838e4643b9030ddfe327f1c742
6. >>> Removing node 6e761ee02c9c17838e4643b9030ddfe327f1c742 from cluster 10.0.0.20:6379
7. >>> Sending CLUSTER FORGET messages to the cluster...
8. >>> SHUTDOWN the node.
9. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
查看缩容后的集群状态 

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.
4. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.
5. [OK] 0 keys in 3 masters.
6. 0.00 keys per slot on average.
7. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb check 10.0.0.20:6379
8. >>> Performing Cluster Check (using node 10.0.0.20:6379)
9. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
10.    slots:0-5460 (5461 slots) master
11.    1 additional replica(s)
12. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
13.    slots: (0 slots) slave
14.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
15. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
16.    slots:10923-16383 (5461 slots) master
17.    1 additional replica(s)
18. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
19.    slots: (0 slots) slave
20.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
21. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
22.    slots: (0 slots) slave
23.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
24. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
25.    slots:5461-10922 (5462 slots) master
26.    1 additional replica(s)
27. [OK] All nodes agree about slots configuration.
28. >>> Check for open slots...
29. >>> Check slots coverage...
30. [OK] All 16384 slots covered.

复制代码

 
4.3、集群维护之数据导入

当企业Redis服务由 原先的单机运行或主从架构 升级为Cluster模式后，需要将原数据迁移至当前Cluster集群中，需要使用集群数据导入
从历史Redis服务器node8 10.0.0.33 上将数据导入至当前集群中

1. [root@Redis-Ubuntu1804-node8:~]# redis-cli -a redis --no-auth-warning info keyspace
2. # Keyspace
3. db0:keys=20000,expires=0,avg_ttl=0[root@Redis-Ubuntu1804-node8:~]# redis-cli -a redis --no-auth-warning get key_2931
4. "value-2931"
5. [root@Redis-Ubuntu1804-node8:~]#

复制代码

  
*集群导入数据需要确保新导入数据不能与Redis Cluster已有数据出现重复的key名称，否则将导致导入失败或中断（建议在导入数据前清空当前集群数据）

 

  import         host:port
                 --cluster-from   ##外部redis的连接地址
                 --cluster-copy ##单独使用时，导入数据与当前集群中key值不可冲突
                 --cluster-replace  ##强制替换key值

 
4.3.1、在 Redis 5 上执行数据导入

清理集群和外部redis所有节点密码

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster call 10.0.0.20:6379 config set requirepass ''
2. >>> Calling config set requirepass
3. 10.0.0.20:6379: OK
4. 10.0.0.22:6379: OK
5. 10.0.0.21:6379: OK
6. 10.0.0.32:6379: OK
7. 10.0.0.31:6379: OK
8. 10.0.0.30:6379: OK
9. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.33 --no-auth-warning config set requirepass ''
10. OK

复制代码

 
执行数据导入

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis --cluster info 10.0.0.20:6379
2. Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
3. 10.0.0.20:6379 (6b142fe3...) -> 1 keys | 5461 slots | 1 slaves.
4. 10.0.0.21:6379 (a4b387d8...) -> 1 keys | 5462 slots | 1 slaves.
5. 10.0.0.30:6379 (5c9789e1...) -> 2 keys | 5461 slots | 1 slaves.
6. [OK] 4 keys in 3 masters.
7. 0.00 keys per slot on average.

复制代码

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli --cluster import 10.0.0.20:6379 --cluster-from 10.0.0.33:6379 --cluster-copy --cluster-replace ... ... ... Migrating key_7867 to 10.0.0.20:6379: OK Migrating key_6810 to 10.0.0.20:6379: OK Migrating key_1232 to 10.0.0.21:6379: OK Migrating key_10171 to 10.0.0.21:6379: OK Migrating key_15561 to 10.0.0.30:6379: OK Migrating key_13723 to 10.0.0.21:6379: OK Migrating key_18761 to 10.0.0.21:6379: OK Migrating key_14471 to 10.0.0.20:6379: OK Migrating key_12496 to 10.0.0.30:6379: OK Migrating key_10559 to 10.0.0.21:6379: OK Migrating key_15282 to 10.0.0.21:6379: OK Migrating key_19021 to 10.0.0.21:6379: OK Migrating key_12831 to 10.0.0.21:6379: OK Migrating key_13260 to 10.0.0.20:6379: OK Migrating key_15767 to 10.0.0.30:6379: OK

复制代码

 
查看数据

1. [root@Redis-Ubuntu1804-node1:~]# redis-cli --cluster info 10.0.0.20:6379
2. 10.0.0.20:6379 (6b142fe3...) -> 6662 keys | 5461 slots | 1 slaves.
3. 10.0.0.21:6379 (a4b387d8...) -> 6666 keys | 5462 slots | 1 slaves.
4. 10.0.0.30:6379 (5c9789e1...) -> 6676 keys | 5461 slots | 1 slaves.
5. [OK] 20004 keys in 3 masters.
6. 1.22 keys per slot on average.
7. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning get key_2981
8. "value-2981"
9. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
4.3.2、在 Redis 3/4 上执行数据导入

清理集群和外部redis所有节点密码

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb call 10.0.0.20:6379 config set requirepass ''
2. >>> Calling CONFIG set requirepass
3. 10.0.0.20:6379: OK
4. 10.0.0.31:6379: OK
5. 10.0.0.22:6379: OK
6. 10.0.0.32:6379: OK
7. 10.0.0.30:6379: OK
8. 10.0.0.21:6379: OK
9. [root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -h 10.0.0.33 config set requirepass ''
10. OK
11. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
12. [ERR] Sorry, can't connect to node 10.0.0.20:6379
13. <br>##清理 redis-trib.rb 工具配置的密码<br>[root@Redis-Ubuntu1804-node1:~]# sed -i 's/@r = Redis.new.*/@r = Redis.new(:host => @info[:host], :port => @info[:port], :timeout => 60)/' /usr/bin/redis-trib.rb
14. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
15. 10.0.0.20:6379 (1c2fee65...) -> 0 keys | 5461 slots | 1 slaves.
16. 10.0.0.22:6379 (709b09b9...) -> 0 keys | 5461 slots | 1 slaves.
17. 10.0.0.21:6379 (337bf2a1...) -> 0 keys | 5462 slots | 1 slaves.
18. [OK] 0 keys in 3 masters.
19. 0.00 keys per slot on average.
20. [root@Redis-Ubuntu1804-node1:~]# <br>

复制代码

 
执行数据导入

由于gem redis 版本与客户端版本不一致出现数据导入错误 Unsupported command argument type: NilClass (TypeError)

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb import --from 10.0.0.33:6379 --copy --replace 10.0.0.20:6379
2. >>> Importing data from 10.0.0.33:6379 to cluster
3. >>> Performing Cluster Check (using node 10.0.0.20:6379)
4. M: 1c2fee65ee50cdf66c862d04710672286b305f55 10.0.0.20:6379
5.    slots:0-5460 (5461 slots) master
6.    1 additional replica(s)
7. S: c567eea1d3fda7b37c42b741bc32d14385bbe872 10.0.0.31:6379
8.    slots: (0 slots) slave
9.    replicates 1c2fee65ee50cdf66c862d04710672286b305f55
10. M: 709b09b96f19209e7f0a1e3cfc5c035678491e4b 10.0.0.22:6379
11.    slots:10923-16383 (5461 slots) master
12.    1 additional replica(s)
13. S: 0428f9a71c369004c0ca01c0a403f9c2ee5fa41c 10.0.0.32:6379
14.    slots: (0 slots) slave
15.    replicates 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e
16. S: f9e73307b3146b25b51cec9a861463a616ed322b 10.0.0.30:6379
17.    slots: (0 slots) slave
18.    replicates 709b09b96f19209e7f0a1e3cfc5c035678491e4b
19. M: 337bf2a15dd9b0ae8caaee08c5ec41beb754b78e 10.0.0.21:6379
20.    slots:5461-10922 (5462 slots) master
21.    1 additional replica(s)
22. [OK] All nodes agree about slots configuration.
23. >>> Check for open slots...
24. >>> Check slots coverage...
25. [OK] All 16384 slots covered.
26. >>> Connecting to the source Redis instance
27. *** Importing 20000 keys from DB 0
28. Traceback (most recent call last):
29.     11: from /usr/bin/redis-trib.rb:1830:in `<main>'
30.     10: from /usr/bin/redis-trib.rb:1604:in `import_cluster_cmd'
31.      9: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis/commands/keys.rb:26:in `scan'
32.      8: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis/commands/keys.rb:433:in `_scan'
33.      7: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis.rb:166:in `send_command'
34.      6: from /usr/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
35.      5: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis.rb:167:in `block in send_command'
36.      4: from /var/lib/gems/2.5.0/gems/redis-5.0.6/lib/redis/client.rb:73:in `call_v'
37.      3: from /var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client.rb:223:in `call_v'
38.      2: from /var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client/command_builder.rb:68:in `generate'
39.      1: from /var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client/command_builder.rb:68:in `map!'
40. /var/lib/gems/2.5.0/gems/redis-client-0.14.1/lib/redis_client/command_builder.rb:75:in `block in generate': Unsupported command argument type: NilClass (TypeError)

复制代码

gem卸载当前版本 redis工具，重新安装4.0版本gem

1. [root@Redis-Ubuntu1804-node1:~]# gem uninstall redis
2. Successfully uninstalled redis-5.0.6
3. [root@Redis-Ubuntu1804-node1:~]# wget https://rubygems.org/downloads/redis-4.0.3.gem
4. --2023-04-21 03:36:14--  https://rubygems.org/downloads/redis-4.0.3.gem
5. Resolving rubygems.org (rubygems.org)... 151.101.1.227, 151.101.65.227, 151.101.129.227, ...
6. Connecting to rubygems.org (rubygems.org)|151.101.1.227|:443... connected.
7. HTTP request sent, awaiting response... 200 OK
8. Length: 116224 (114K) [application/octet-stream]
9. Saving to: ‘redis-4.0.3.gem’
11. redis-4.0.3.gem                     100%[===================================================================>] 113.50K  --.-KB/s    in 0.1s   
13. 2023-04-21 03:36:14 (1.11 MB/s) - ‘redis-4.0.3.gem’ saved [116224/116224]
15. [root@Redis-Ubuntu1804-node1:~]# gem install redis-4.0.3.gem
16. Successfully installed redis-4.0.3
17. Parsing documentation for redis-4.0.3
18. Installing ri documentation for redis-4.0.3
19. Done installing documentation for redis after 0 seconds
20. 1 gem installed
21. [root@Redis-Ubuntu1804-node1:~]#

复制代码

由于gem版本过高导致出现 ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)

1. Migrating key_2307 to 10.0.0.22:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
2. Migrating key_5080 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
3. Migrating key_15899 to 10.0.0.21:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
4. Migrating key_6538 to 10.0.0.22:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
5. Migrating key_1665 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
6. Migrating key_14837 to 10.0.0.21:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
7. Migrating key_1757 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
8. Migrating key_13190 to 10.0.0.20:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)

复制代码

卸载当前版本，安装gem-redis 3.5 版本

1. [root@Redis-Ubuntu1804-node1:~]# gem uninstall redis
2. Successfully uninstalled redis-4.0.3
3. [root@Redis-Ubuntu1804-node1:~]# wget https://rubygems.org/downloads/redis-3.3.5.gem
4. --2023-04-21 03:46:33--  https://rubygems.org/downloads/redis-3.3.5.gem
5. Resolving rubygems.org (rubygems.org)... 151.101.193.227, 151.101.129.227, 151.101.65.227, ...
6. Connecting to rubygems.org (rubygems.org)|151.101.193.227|:443... connected.
7. HTTP request sent, awaiting response... 200 OK
8. Length: 92672 (90K) [application/octet-stream]
9. Saving to: ‘redis-3.3.5.gem’
11. redis-3.3.5.gem                     100%[===================================================================>]  90.50K   497KB/s    in 0.2s   
13. 2023-04-21 03:46:34 (497 KB/s) - ‘redis-3.3.5.gem’ saved [92672/92672]
15. [root@Redis-Ubuntu1804-node1:~]# gem install redis-3.3.5.gem
16. Successfully installed redis-3.3.5
17. Parsing documentation for redis-3.3.5
18. Installing ri documentation for redis-3.3.5
19. Done installing documentation for redis after 0 seconds
20. 1 gem installed

复制代码

 
执行数据导入

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb import --from 10.0.0.33:6379 --copy --replace 10.0.0.20:6379
2. ...
3. ...
4. ...
5. Migrating key_19321 to 10.0.0.20:6379: OK
6. Migrating key_3394 to 10.0.0.20:6379: OK
7. Migrating key_11273 to 10.0.0.22:6379: OK
8. Migrating key_15899 to 10.0.0.21:6379: OK
9. Migrating key_14837 to 10.0.0.21:6379: OK
10. Migrating key_1665 to 10.0.0.20:6379: OK
11. Migrating key_5080 to 10.0.0.20:6379: OK
12. Migrating key_13190 to 10.0.0.20:6379: OK
13. Migrating key_1757 to 10.0.0.20:6379: OK
14. Migrating key_6538 to 10.0.0.22:6379: OK
15. Migrating key_2307 to 10.0.0.22:6379: OK
16. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
查看数据

1. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb info 10.0.0.20:6379
2. 10.0.0.20:6379 (1c2fee65...) -> 6661 keys | 5461 slots | 1 slaves.
3. 10.0.0.22:6379 (709b09b9...) -> 6674 keys | 5461 slots | 1 slaves.
4. 10.0.0.21:6379 (337bf2a1...) -> 6665 keys | 5462 slots | 1 slaves.
5. [OK] 20000 keys in 3 masters.
6. 1.22 keys per slot on average.
7. [root@Redis-Ubuntu1804-node1:~]# redis-cli -c get key_12332
8. "value-12332"
9. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
4.4、集群偏斜

Redis Cluser 多节点在运行一段时间以后，会出现倾斜现象，某个节点数据偏多，内存消耗更大，或者接收到的用户请求更多，发生偏移现象的可能性，如下：

• 节点和槽位分配不均
  
• 不同槽位对应的键值数量差异较大
  
• 包含bigkey（建议少用）
  
• 内存相关配置不一致
  
• 热点数据不均衡：一致性不高时，可使用本地缓存或者MQ
  

1. #查看指定槽位中的键值数
2. [root@Redis-Ubuntu1804-node1:~]# redis-cli cluster countkeysinslot 0
3. (integer) 3
4. [root@Redis-Ubuntu1804-node1:~]# redis-cli cluster countkeysinslot 16383
5. (integer) 0
6. [root@Redis-Ubuntu1804-node1:~]# redis-cli cluster countkeysinslot 1
7. (integer) 2
8. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. #槽位自动平衡分布
2. [root@Redis-Ubuntu1804-node1:~]# redis-trib.rb rebalance 10.0.0.20:6379
3. >>> Performing Cluster Check (using node 10.0.0.20:6379)
4. [OK] All nodes agree about slots configuration.
5. >>> Check for open slots...
6. >>> Check slots coverage...
7. [OK] All 16384 slots covered.
8. *** No rebalancing needed! All nodes are within the 2.0% threshold.
9. [root@Redis-Ubuntu1804-node1:~]#

复制代码

1. #查找bigkeys,建议在slave上执行
2. [root@Redis-Ubuntu1804-node1:~]# redis-cli --bigkeys
4. # Scanning the entire keyspace to find biggest keys as well as
5. # average sizes per key type.  You can use -i 0.1 to sleep 0.1 sec
6. # per 100 SCAN commands (not usually needed).
8. [00.00%] Biggest string found so far 'key_13312' with 11 bytes
10. -------- summary -------
12. Sampled 6661 keys in the keyspace!
13. Total key length in bytes is 56245 (avg len 8.44)
15. Biggest string found 'key_13312' has 11 bytes
17. 6661 strings with 69567 bytes (100.00% of keys, avg size 10.44)
18. 0 lists with 0 items (00.00% of keys, avg size 0.00)
19. 0 sets with 0 members (00.00% of keys, avg size 0.00)
20. 0 hashs with 0 fields (00.00% of keys, avg size 0.00)
21. 0 zsets with 0 members (00.00% of keys, avg size 0.00)
22. 0 streams with 0 entries (00.00% of keys, avg size 0.00)
23. [root@Redis-Ubuntu1804-node1:~]#

复制代码

 
四、集群操作自动化

1、初始环境自动化配置脚本

-shell Ubuntu编译安装

1. [root@Client-Ubuntu-1804-250:~/script/redis]# cat redis_install.sh
2. #!/bin/bash
3. #
4. #********************************************************************
5. #Author:                janzen
6. #Date:                  2023-04-13
7. #FileName：             redis_install.sh
8. #Description：          Install Redis
9. #Copyright (C):        2023 All rights reserved
10. #********************************************************************
12. DIR='/app/redis'
13. DIR_S='\/app\/redis'
14. PASSWD='redis'
15. PORT=()
16. echo -e "启用架构：\n 1、single node \n 2、Replicae_node \n 3、Cluster_node\n"
17. read -p "选择节点类型：" node_type
18. read -p "创建的副本数：" num
19. case $num in
20.         0)
21.                 echo 不可创建0副本
22.                 exit
23.         ;;
24.         [1-9]*)
25.                 echo 创建$num副本
26.         ;;
27.         *)
28.                 echo $num 不是数字
29.                 exit
30.         ;;
31. esac
32. for i in $(seq $num); do
33.         rds_port=`echo $i-1+6379|bc`
34.         PORT+=($rds_port)
35. done
37. wget https://download.redis.io/releases/redis-5.0.14.tar.gz
38. tar xf redis-5.0.14.tar.gz
39. useradd redis -s /sbin/nologin
40. mkdir $DIR/{etc,log,data,run} -p
41. chown redis.redis $DIR/ -R
42. apt-get install build-essential tree -y
43. cd redis-5.0.14
44. make PREFIX=$DIR install
45. ln -s $DIR/bin/* /usr/bin/
48. echo net.core.somaxconn=1024 >> /etc/sysctl.conf
49. echo vm.overcommit_memory=1 >> /etc/sysctl.conf
50. sysctl -p
51. echo never > /sys/kernel/mm/transparent_hugepage/enabled
52. for i in ${PORT[*]};do
53.     case $node_type in
54.         1)
55.             sed -e 's/^bind 127.0.0.1/bind 0.0.0.0/' -e 's/^port 6379/port '"$i"'/' -e 's/^pidfile \/var\/run\/redis_6379.pid/pidfile '"$DIR_S"'\/run\/redis_'"$i"'.pid/' -e 's/^logfile ""/logfile "'"$DIR_S"'\/log\/redis_'"$i"'.log"/g' -e 's/^always-show-logo yes/always-show-logo no/g' -e 's/^appendonly no/appendonly yes/' -e 's/^appendfilename "appendonly.aof"/appendfilename "appendonly_'"$i"'.aof"/g' -e 's/^dbfilename dump.rdb/dbfilename dump_'"$i"'.rdb/g' -e 's/^dir \.\//dir '"$DIR_S"'\/data/g' -e 's/^# masterauth <master-password>$/masterauth '"$PASSWD"'/' -e 's/^# requirepass foobared/requirepass '"$PASSWD"'/' redis.conf > $DIR/etc/redis_$i.conf
56.             ;;
57.         2)
58.             read -p "node $i Master：<IP> <port>: " Master
59.             sed -e 's/^bind 127.0.0.1/bind 0.0.0.0/' -e 's/^port 6379/port '"$i"'/' -e 's/^pidfile \/var\/run\/redis_6379.pid/pidfile '"$DIR_S"'\/run\/redis_'"$i"'.pid/' -e 's/^logfile ""/logfile "'"$DIR_S"'\/log\/redis_'"$i"'.log"/g' -e 's/^always-show-logo yes/always-show-logo no/g' -e 's/^appendonly no/appendonly yes/' -e 's/^appendfilename "appendonly.aof"/appendfilename "appendonly_'"$i"'.aof"/g' -e 's/^dbfilename dump.rdb/dbfilename dump_'"$i"'.rdb/g' -e 's/^dir \.\//dir '"$DIR_S"'\/data/g' -e 's/^# masterauth <master-password>$/masterauth '"$PASSWD"'/' -e 's/^# requirepass foobared/requirepass '"$PASSWD"'/' -e '/# replicaof/a replicaof '"$Master"'' redis.conf > $DIR/etc/redis_$i.conf
60.             ;;
61.         3)
62.             sed -e 's/^bind 127.0.0.1/bind 0.0.0.0/' -e 's/^port 6379/port '"$i"'/' -e 's/^pidfile \/var\/run\/redis_6379.pid/pidfile '"$DIR_S"'\/run\/redis_'"$i"'.pid/' -e 's/^logfile ""/logfile "'"$DIR_S"'\/log\/redis_'"$i"'.log"/g' -e 's/^always-show-logo yes/always-show-logo no/g' -e 's/^appendonly no/appendonly yes/' -e 's/^appendfilename "appendonly.aof"/appendfilename "appendonly_'"$i"'.aof"/g' -e 's/^dbfilename dump.rdb/dbfilename dump_'"$i"'.rdb/g' -e 's/^dir \.\//dir '"$DIR_S"'\/data/g' -e 's/^# masterauth <master-password>$/masterauth '"$PASSWD"'/' -e 's/^# requirepass foobared/requirepass '"$PASSWD"'/' -e '/# cluster-enabled yes/a cluster-enabled yes' -e '/# cluster-config-file/a cluster-config-file nodes-'"$i"'.conf' -e '/# cluster-require-full-coverage yes/a cluster-require-full-coverage yes' redis.conf > $DIR/etc/redis_$i.conf
63.             ;;
64.         *)
65.             exit
66.             ;;
67.     esac
68.     cat > /etc/systemd/system/redis_$i.service << EOF
69. [Unit]
70. Description=Redis persistent key-value database
71. After=network.target
72. [Service]
73. ExecStart=$DIR/bin/redis-server $DIR/etc/redis_$i.conf --supervised systemd
74. ExecStop=/bin/kill -s QUIT $MAINPID
75. Type=notify
76. User=redis
77. Group=redis
78. RuntimeDirectory=redis
79. RuntimeDirectoryMode=0755
80. LimitNOFILE=65536
81. [Install]
82. WantedBy=multi-user.target
83. EOF
84.     chown redis.redis $DIR/ -R
85.     systemctl daemon-reload
86.     systemctl enable --now redis_$i.service
87.     redis-cli -a $PASSWD -P $I --no-auth-warning info keyspace
88. done
90. cat > /etc/init.d/disable_transparent_hugepage << EOF
91. #!/bin/bash
92. echo never > /sys/kernel/mm/transparent_hugepage/enabled
93. EOF
94. chmod 755 /etc/init.d/disable_transparent_hugepage
95. cd /etc/rcS.d/
96. ln -s ../init.d/disable_transparent_hugepage disable_transparent_hugepage
98. ss -ntl
99. ps -ef | grep redis-server
100. # reboot

复制代码

-shell Ubuntu 二进制包安装

1. #!/bin/bash
2. #
3. #********************************************************************
4. #Author:                janzen
5. #Date:                  2023-04-20
6. #FileName：             redis_install_apt.sh
7. #Description：          The test script
8. #Copyright (C):        2023 All rights reserved
9. #********************************************************************
10. apt install redis -y
11. sed -i.bak -e 's/bind 127.0.0.1/bind 0.0.0.0/' -e '/masterauth/a masterauth redis' -e '/# requirepass/a requirepass redis' -e '/# cluster-enabled yes/a cluster-enabled yes' -e '/# cluster-config-file/a cluster-config-file nodes-6379.conf' -e '/# cluster-require-full-coverage yes/a cluster-require-full-coverage yes' /etc/redis/redis.conf
12. systemctl restart redis
13. ss -ntl

复制代码

 
2、批量分配slot槽位

 

1. [root@Client-Ubuntu-1804-250:~/script/redis]# cat add_slots.sh
2. #!/bin/bash
3. #
4. #********************************************************************
5. #Author:                janzen
6. #Date:                  2023-04-20
7. #FileName：             add_slots.sh
8. #Description：          The test script
9. #Copyright (C):        2023 All rights reserved
10. #********************************************************************
11. host=$1
12. port=$2
13. start=$3
14. end=$4
15. passwd=redis
17. for slot in `seq $start $end`; do
18.     redis-cli -h $host -p $port -a $passwd --no-auth-warning cluster addslots $slot &&
19.     echo $slot add to node $host:$port
20. done

复制代码

 
3、python脚本实现自动创建数据

pip 安装 redis-py-cluster 模块

1. [root@Client-Ubuntu-1804-250:~/script/redis]# pip3 install redis-py-cluster
2. Collecting redis-py-cluster
3.   Downloading https://files.pythonhosted.org/packages/b2/96/153bbcf5dee29b52b2674e77a87ce864d381f72151737317529b7de4f337/redis_py_cluster-2.1.3-py2.py3-none-any.whl (42kB)
4.     100% |████████████████████████████████| 51kB 403kB/s
5. Collecting redis<4.0.0,>=3.0.0 (from redis-py-cluster)
6.   Downloading https://files.pythonhosted.org/packages/a7/7c/24fb0511df653cf1a5d938d8f5d19802a88cef255706fdda242ff97e91b7/redis-3.5.3-py2.py3-none-any.whl (72kB)
7.     100% |████████████████████████████████| 81kB 1.5MB/s
8. Installing collected packages: redis, redis-py-cluster
9.   Found existing installation: redis 2.10.6
10.     Not uninstalling redis at /usr/lib/python3/dist-packages, outside environment /usr
11. Successfully installed redis-3.5.3 redis-py-cluster-2.1.3
12. <br>

复制代码

python脚本连接集群创建数据

1. #!/usr/bin/python3
2. # -*- coding: UTF-8 -*-
3. #********************************************************************
4. #Author:                janzen
5. #Date:                  2023-04-20
6. #FileName：             redis_cluster_newData.py
7. #Description：          The python3 script
8. #Copyright (C):        2023 All rights reserved
9. #********************************************************************
10. from rediscluster import RedisCluster
11. import sys
12. num=int(sys.argv[1])
14. passwd='redis'
16. startup_nodes = [
17.     {"host":"10.0.0.20","port":"6379"},
18.     {"host":"10.0.0.21","port":"6379"},
19.     {"host":"10.0.0.22","port":"6379"},
20.     {"host":"10.0.0.30","port":"6379"},
21.     {"host":"10.0.0.31","port":"6379"},
22.     {"host":"10.0.0.32","port":"6379"}
23.     ]
25. redis_conn = RedisCluster(startup_nodes = startup_nodes,password = passwd,decode_responses=True)
27. for i in range(0,num):
28.     redis_conn.set("keys%s" % i,"value%s" % i)
29.     print("keys%s:" % i,redis_conn.get("keys%s" % i))

复制代码

1. #!/usr/bin/python3
2. # -*- coding: UTF-8 -*-
3. #********************************************************************
4. #Author:                janzen
5. #Date:                  2023-04-20
6. #FileName：             redis_cluster_newData.py
7. #Description：          The python3 script
8. #Copyright (C):        2023 All rights reserved
9. #********************************************************************
10. from rediscluster import RedisCluster
11. import sys
12. num=int(sys.argv[1])
14. passwd='redis'
16. startup_nodes = [
17.     {"host":"10.0.0.20","port":"6379"},
18.     {"host":"10.0.0.21","port":"6379"},
19.     {"host":"10.0.0.22","port":"6379"},
20.     {"host":"10.0.0.30","port":"6379"},
21.     {"host":"10.0.0.31","port":"6379"},
22.     {"host":"10.0.0.32","port":"6379"}
23.     ]
25. redis_conn = RedisCluster(startup_nodes = startup_nodes,password = passwd,decode_responses=True)
28. while 1:
29.         time.sleep(1)
30.         try:
31.         redis_conn.set("keys%s" % num,"value%s" % num)
32.         print("set data %s:%s" % (num,redis_conn.get("keys%s" % num)))
33.         except Exception as err:
34.                 print(err)
35.                 continue
36.         num+=1

复制代码

python脚本连接集群读取数据

1. #!/usr/bin/python3
2. # -*- coding: UTF-8 -*-
3. #********************************************************************
4. #Author:                janzen
5. #Date:                  2023-04-20
6. #FileName：             redis_cluster_newData.py
7. #Description：          The python3 script
8. #Copyright (C):        2023 All rights reserved
9. #********************************************************************
10. from rediscluster import RedisCluster
11. import sys,time
12. num=int(sys.argv[1])
14. passwd='redis'
16. startup_nodes = [
17.     {"host":"10.0.0.20","port":"6379"},
18.     {"host":"10.0.0.21","port":"6379"},
19.     {"host":"10.0.0.22","port":"6379"},
20.     {"host":"10.0.0.30","port":"6379"},
21.     {"host":"10.0.0.31","port":"6379"},
22.     {"host":"10.0.0.32","port":"6379"}
23.     ]
25. redis_conn = RedisCluster(startup_nodes = startup_nodes,password = passwd,decode_responses=True)
27. while 1:
28.     time.sleep(1)
29.     try:
30.         print("get data %s:%s" % (str(num),redis_conn.get("keys"+str(num))))
31.     except Exception as err:
32.         print(err)
33.         continue

复制代码

 
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster check 10.0.0.20:6379Could not connect to Redis at 10.0.0.22:6379: Connection refused*** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.10.0.0.20:6379 (6b142fe3...) -> 131 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 126 keys | 5462 slots | 1 slaves.[OK] 257 keys in 2 masters.0.02 keys per slot on average.>>> Performing Cluster Check (using node 10.0.0.20:6379)M: 6b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379   slots:[0-5460] (5461 slots) master   1 additional replica(s)M: a4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379   slots:[5461-10922] (5462 slots) master   1 additional replica(s)S: 6a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379   slots: (0 slots) slave   replicates a4b387d82ac06e5cbf212e48c781e1c27b50320fS: 60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379   slots: (0 slots) slave   replicates 6b142fe3438c3ea488c34366e5ea5a298f89f518S: 5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379   slots: (0 slots) slave   replicates 07a36af61eb1f887419f2266b2360b4f795bb7a3[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[ERR] Not all 16384 slots are covered by nodes.
[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning --cluster info 10.0.0.20:6379Could not connect to Redis at 10.0.0.22:6379: Connection refused*** WARNING: 10.0.0.30:6379 claims to be slave of unknown node ID 07a36af61eb1f887419f2266b2360b4f795bb7a3.10.0.0.20:6379 (6b142fe3...) -> 131 keys | 5461 slots | 1 slaves.10.0.0.21:6379 (a4b387d8...) -> 126 keys | 5462 slots | 1 slaves.[OK] 257 keys in 2 masters.0.02 keys per slot on average.[root@Redis-Ubuntu1804-node1:~]# redis-cli -a redis -c --no-auth-warning cluster nodes07a36af61eb1f887419f2266b2360b4f795bb7a3 10.0.0.22:6379@16379 master,fail - 1682000024732 1682000022415 3 disconnecteda4b387d82ac06e5cbf212e48c781e1c27b50320f 10.0.0.21:6379@16379 master - 0 1682000042000 2 connected 5461-109226a9dd64b6bf1a7eb336ec9d320d86ae7450d5e2b 10.0.0.32:6379@16379 slave a4b387d82ac06e5cbf212e48c781e1c27b50320f 0 1682000040567 6 connected60aec73a6b0d3929cda3a284bb513898d2e73657 10.0.0.31:6379@16379 slave 6b142fe3438c3ea488c34366e5ea5a298f89f518 0 1682000042582 5 connected5c9789e105f8d8a0f1dc5ac2a6d51831482fbd34 10.0.0.30:6379@16379 master - 0 1682000041575 7 connected 10923-163836b142fe3438c3ea488c34366e5ea5a298f89f518 10.0.0.20:6379@16379 myself,master - 0 1682000041000 1 connected 0-5460[root@Redis-Ubuntu1804-node1:~]#  

[root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis --versionredis-cli 5.0.14[root@Redis-Ubuntu1804-node4:~]# redis-cli -a redis --no-auth-warning cluster nodese603091426e2ce29629ca51fb5e7dafb2f0c9524 :6379@16379 myself,master - 0 0 0 connected[root@Redis-Ubuntu1804-node4:~]#
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！