CentOS6.5搭建Redis-v4.0.8集群

环境说明

  • CentOS release 6.5 (Final)
  • 三台主机:10.10.10.44 10.10.10.45 10.10.10.46

集群搭建

安装

$ wget http://download.redis.io/releases/redis-4.0.8.tar.gz
$ tar xzf redis-4.0.8.tar.gz
$ cd redis-4.0.8
$ make

测试安装结果

#二进制文件是编译完成后在src目录下,通过下面的命令启动Redis服务
$ src/redis-server

#你可以使用内置的客户端命令redis-cli进行使用
$ src/redis-cli
redis> set foo bar
OK
redis> get foo
"bar"

创建节点

3台机器每台机器都启动3个节点,需对节点进行配置。

  1. 分别在10.10.10.44 10.10.10.45 10.10.10.46 主机下的redis-4.0.8 目录下创建redis-cluster 目录

    $ mkdir redis-cluster
  2. redis-cluster 目录下,创建三个名为7000 7001 7002

    $ cd redis-cluster
    $ mkdir 7000 7001 7002
  3. 分别创建redis.conf 文件,进行节点配置(关键点)

    $ vi 7000/redis.conf 
    $ vi 7001/redis.conf
    $ vi 7002/redis.conf

    redis.conf 配置

    #端口7000,7001,7002
    port 7000

    #默认ip为127.0.0.1,需要改为其他节点机器可访问的ip,否则创建集群时无法访问对应的端口,无法创建集群
    bind 10.10.10.45

    #redis后台运行
    daemonize yes

    #pidfile文件对应7000,7001,7002
    #pidfile /var/run/redis_7000.pid

    #开启集群
    cluster-enabled yes

    #集群的配置,配置文件首次启动自动生成 7000,7001,7002
    cluster-config-file nodes_7000.conf

    #请求超时,默认15秒,可自行设置
    cluster-node-timeout 5000

    #aof日志开启,有需要就开启,它会每次写操作都记录一条日志
    appendonly yes

    redis.conf 配置中,注意bind cluster-config-file 字段配置。配置为对应的值。

  4. 检查配置

    • 三台机器redis-cluster目录下7000、7001、7002目录下各有一份redis.conf 配置文件。
    • bind 字段配置为对应机器的IP地址
    • cluster-config-file 字段配置为对应节点值。如nodes_7000.confnodes_7001.conf

启动节点

在三台主机下分别执行命令

$ for((i=0;i<=2;i++)); do /home/redis/redis-4.0.8/src/redis-server /home/redis/redis-4.0.8/redis-cluster/700$i/redis.conf; done

检查各 Redis 各个节点启动情况

$ ps -ef | grep redis           //redis是否启动成功
$ netstat -tnlp | grep redis //监听redis端口

创建&启动集群

上面三台机都各自启动了3个节点后,我们开始创建和启动集群。创建和启动集群只需在其中任意一台中就可以了,下面步骤我都是在10.10.10.45 这台机上操作。

安装Ruby

创建集群依赖于Ruby,新版Redis需要Ruby的版本为2.2.2以上,所以需要安装个高版本的Ruby。

我采用的是RVM(Ruby Version Manager) 安装方式,比较简单方便。

先安装RVM 再安装Ruby,再安装gem。

$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

$ \curl -sSL https://get.rvm.io | bash -s stable

$ source ~/.rvm/scripts/rvm

$ rvm install ruby

$ rvm -v
rvm 1.29.3 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]

$ ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

$ rvm rubygems current
Rubygems 2.6.14 already available in installed ruby, skipping installation, use --force to reinstall.

$ gem install redis
Fetching: redis-4.0.1.gem (100%)
Successfully installed redis-4.0.1

创建&启动

使用官方提供的redis-trib.rb 工具,在src 目录中。

$ src/redis-trib.rb create --replicas 1 10.10.10.44:7000 10.10.10.44:7001 10.10.10.44:7002 10.10.10.45:7000 10.10.10.45:7001 10.10.10.45:7002 10.10.10.46:7000 10.10.10.46:7001 10.10.10.46:7002
  • 执行后
$ src/redis-trib.rb create --replicas 1 10.10.10.44:7000 10.10.10.44:7001 10.10.10.44:7002 10.10.10.45:7000 10.10.10.45:7001 10.10.10.45:7002 10.10.10.46:7000 10.10.10.46:7001 10.10.10.46:7002
>>> Creating cluster
>>> Performing hash slots allocation on 9 nodes...
Using 4 masters:
10.10.10.44:7000
10.10.10.45:7000
10.10.10.46:7000
10.10.10.44:7001
Adding replica 10.10.10.46:7001 to 10.10.10.44:7000
Adding replica 10.10.10.44:7002 to 10.10.10.45:7000
Adding replica 10.10.10.45:7002 to 10.10.10.46:7000
Adding replica 10.10.10.46:7002 to 10.10.10.44:7001
Adding replica 10.10.10.45:7001 to 10.10.10.44:7000
M: 1cc8f932f2546b38055652772c2e47330f132c9e 10.10.10.44:7000
slots:0-4095 (4096 slots) master
M: aade4eeae71e65901c443f196539b09d5cece2ba 10.10.10.44:7001
slots:12288-16383 (4096 slots) master
S: 0d2acc3b0ca9c090e02fd08c96903f3dddd82a9e 10.10.10.44:7002
replicates cb029a8dae638d41fa00f10d0d2c7433158e5e1d
M: cb029a8dae638d41fa00f10d0d2c7433158e5e1d 10.10.10.45:7000
slots:4096-8191 (4096 slots) master
S: 8d9df95a91468692c7519ed2d9814fa1dc670379 10.10.10.45:7001
replicates 1cc8f932f2546b38055652772c2e47330f132c9e
S: 44e1d791a5620d602047069e66893ee88df86808 10.10.10.45:7002
replicates 6ab31a860271d1ea47e73588512be39d808f0dff
M: 6ab31a860271d1ea47e73588512be39d808f0dff 10.10.10.46:7000
slots:8192-12287 (4096 slots) master
S: 69a0036ca74e37b326a98df554bffcbb87eaea73 10.10.10.46:7001
replicates 1cc8f932f2546b38055652772c2e47330f132c9e
S: 5bae9f7939b33756e9ff3dcd64d8d14d37e4a278 10.10.10.46:7002
replicates aade4eeae71e65901c443f196539b09d5cece2ba
Can I set the above configuration? (type 'yes' to accept): yes
  • 确认集群创建配置,输入yes ,开始启动。
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 10.10.10.44:7000)
M: 1cc8f932f2546b38055652772c2e47330f132c9e 10.10.10.44:7000
slots:0-4095 (4096 slots) master
2 additional replica(s)
S: 69a0036ca74e37b326a98df554bffcbb87eaea73 10.10.10.46:7001
slots: (0 slots) slave
replicates 1cc8f932f2546b38055652772c2e47330f132c9e
S: 5bae9f7939b33756e9ff3dcd64d8d14d37e4a278 10.10.10.46:7002
slots: (0 slots) slave
replicates aade4eeae71e65901c443f196539b09d5cece2ba
S: 0d2acc3b0ca9c090e02fd08c96903f3dddd82a9e 10.10.10.44:7002
slots: (0 slots) slave
replicates cb029a8dae638d41fa00f10d0d2c7433158e5e1d
M: aade4eeae71e65901c443f196539b09d5cece2ba 10.10.10.44:7001
slots:12288-16383 (4096 slots) master
1 additional replica(s)
M: 6ab31a860271d1ea47e73588512be39d808f0dff 10.10.10.46:7000
slots:8192-12287 (4096 slots) master
1 additional replica(s)
S: 44e1d791a5620d602047069e66893ee88df86808 10.10.10.45:7002
slots: (0 slots) slave
replicates 6ab31a860271d1ea47e73588512be39d808f0dff
S: 8d9df95a91468692c7519ed2d9814fa1dc670379 10.10.10.45:7001
slots: (0 slots) slave
replicates 1cc8f932f2546b38055652772c2e47330f132c9e
M: cb029a8dae638d41fa00f10d0d2c7433158e5e1d 10.10.10.45:7000
slots:4096-8191 (4096 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
  • 启动成功

注意防火墙问题

节点间通信,需要注意防火墙,可以选择关闭或者添加防火墙规则。

$ sudo vim /etc/sysconfig/iptables
$ sudo service iptables restart

我添加的防火墙规则(每台机都添加):

-A INPUT -m state –state NEW -m tcp -p tcp –dport 7000 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 7001 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 7002 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 17000 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 17001 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 17002 -j ACCEPT

集群验证

检查集群状态

$ src/redis-trib.rb check 10.10.10.45:7000
>>> Performing Cluster Check (using node 10.10.10.45:7000)
M: cb029a8dae638d41fa00f10d0d2c7433158e5e1d 10.10.10.45:7000
slots:4096-8191 (4096 slots) master
1 additional replica(s)
S: 5bae9f7939b33756e9ff3dcd64d8d14d37e4a278 10.10.10.46:7002
slots: (0 slots) slave
replicates aade4eeae71e65901c443f196539b09d5cece2ba
S: 69a0036ca74e37b326a98df554bffcbb87eaea73 10.10.10.46:7001
slots: (0 slots) slave
replicates 1cc8f932f2546b38055652772c2e47330f132c9e
M: 1cc8f932f2546b38055652772c2e47330f132c9e 10.10.10.44:7000
slots:0-4095 (4096 slots) master
2 additional replica(s)
S: 8d9df95a91468692c7519ed2d9814fa1dc670379 10.10.10.45:7001
slots: (0 slots) slave
replicates 1cc8f932f2546b38055652772c2e47330f132c9e
M: aade4eeae71e65901c443f196539b09d5cece2ba 10.10.10.44:7001
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: 44e1d791a5620d602047069e66893ee88df86808 10.10.10.45:7002
slots: (0 slots) slave
replicates 6ab31a860271d1ea47e73588512be39d808f0dff
M: 6ab31a860271d1ea47e73588512be39d808f0dff 10.10.10.46:7000
slots:8192-12287 (4096 slots) master
1 additional replica(s)
S: 0d2acc3b0ca9c090e02fd08c96903f3dddd82a9e 10.10.10.44:7002
slots: (0 slots) slave
replicates cb029a8dae638d41fa00f10d0d2c7433158e5e1d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

打印集群信息

$ 10.10.10.45:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:9
cluster_size:4
cluster_current_epoch:9
cluster_my_epoch:4
cluster_stats_messages_ping_sent:222458
cluster_stats_messages_pong_sent:221110
cluster_stats_messages_meet_sent:4
cluster_stats_messages_sent:443572
cluster_stats_messages_ping_received:221106
cluster_stats_messages_pong_received:222462
cluster_stats_messages_meet_received:4
cluster_stats_messages_received:443572

列出集群节点

$ 10.10.10.45:7000> cluster nodes
5bae9f7939b33756e9ff3dcd64d8d14d37e4a278 10.10.10.46:7002@17002 slave aade4eeae71e65901c443f196539b09d5cece2ba 0 1519787306481 9 connected
69a0036ca74e37b326a98df554bffcbb87eaea73 10.10.10.46:7001@17001 slave 1cc8f932f2546b38055652772c2e47330f132c9e 0 1519787307081 8 connected
1cc8f932f2546b38055652772c2e47330f132c9e 10.10.10.44:7000@17000 master - 0 1519787306481 1 connected 0-4095
8d9df95a91468692c7519ed2d9814fa1dc670379 10.10.10.45:7001@17001 slave 1cc8f932f2546b38055652772c2e47330f132c9e 0 1519787306000 5 connected
aade4eeae71e65901c443f196539b09d5cece2ba 10.10.10.44:7001@17001 master - 0 1519787305478 2 connected 12288-16383
44e1d791a5620d602047069e66893ee88df86808 10.10.10.45:7002@17002 slave 6ab31a860271d1ea47e73588512be39d808f0dff 0 1519787306581 7 connected
6ab31a860271d1ea47e73588512be39d808f0dff 10.10.10.46:7000@17000 master - 0 1519787306581 7 connected 8192-12287
0d2acc3b0ca9c090e02fd08c96903f3dddd82a9e 10.10.10.44:7002@17002 slave cb029a8dae638d41fa00f10d0d2c7433158e5e1d 0 1519787307482 4 connected
cb029a8dae638d41fa00f10d0d2c7433158e5e1d 10.10.10.45:7000@17000 myself,master - 0 1519787304000 4 connected 4096-8191

连接集群

$ src/redis-cli -h 10.10.10.46 -c -p 7000

10.10.10.46:7000> set name liutf
-> Redirected to slot [5798] located at 10.10.10.45:7000
OK
10.10.10.45:7000> get name
"liutf"
  • 10.10.10.45机器上通过集群方式-c 连接到46 节点。
  • 在46中setkey-name 的值为liutf,key name 存储到了slot [5798] located at 10.10.10.45:7000 。每个key通过CRC16校验后对16384取模来决定放置哪个槽。
分 享