安装Zookeeper和Kafka集群

安装Zookeeper和Kafka集群

本文介绍如何安装Zookeeper和Kafka集群。为了方便，介绍的是在一台服务器上的安装，实际应该安装在多台服务器上，但步骤是一样的。
安装Zookeeper集群

下载安装包

从官网上下载安装包：

1. curl https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz -o apache-zookeeper-3.7.1-bin.tar.gz

复制代码

解压：

1. tar xvf apache-zookeeper-3.7.1-bin.tar.gz

复制代码

配置

创建目录 zk1，然后添加如下配置：
zk1/myid:

1. 1

复制代码

zk1/zk.config:

1. tickTime=2000
2. initLimit=10
3. syncLimit=5
4. dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk1
5. clientPort=2181
7. server.1=127.0.0.1:2888:3888
8. server.2=127.0.0.1:2889:3889
9. server.3=127.0.0.1:2890:3890

复制代码

对于zk2和zk3也重复同样的步骤，并修改相应的配置：
zk2/myid:

1. 2

复制代码

zk2/zk.config:

1. tickTime=2000
2. initLimit=10
3. syncLimit=5
4. dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk2
5. clientPort=2182
7. server.1=127.0.0.1:2888:3888
8. server.2=127.0.0.1:2889:3889
9. server.3=127.0.0.1:2890:3890

复制代码

zk3/myid:

1. 3

复制代码

zk3/zk.config:

1. tickTime=2000
2. initLimit=10
3. syncLimit=5
4. dataDir=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/zookeeper/zk3
5. clientPort=2183
7. server.1=127.0.0.1:2888:3888
8. server.2=127.0.0.1:2889:3889
9. server.3=127.0.0.1:2890:3890

复制代码

启动集群

启动三个服务如下：

1. $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk1/zk.config
2. ZooKeeper JMX enabled by default
3. Using config: ./zk1/zk.config
4. Starting zookeeper ... STARTED
6. $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk2/zk.config
7. ZooKeeper JMX enabled by default
8. Using config: ./zk2/zk.config
9. Starting zookeeper ... STARTED
11. $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh start ./zk3/zk.config
12. ZooKeeper JMX enabled by default
13. Using config: ./zk3/zk.config
14. Starting zookeeper ... STARTED

复制代码

查看状态

通过status命令查看：

1. $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk1/zk.config
2. ZooKeeper JMX enabled by default
3. Using config: ./zk1/zk.config
4. Client port found: 2181. Client address: localhost. Client SSL: false.
5. Mode: follower
8. $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk2/zk.config
9. ZooKeeper JMX enabled by default
10. Using config: ./zk2/zk.config
11. Client port found: 2182. Client address: localhost. Client SSL: false.
12. Mode: leader
15. $ ./apache-zookeeper-3.7.1-bin/bin/zkServer.sh status ./zk3/zk.config
16. ZooKeeper JMX enabled by default
17. Using config: ./zk3/zk.config
18. Client port found: 2183. Client address: localhost. Client SSL: false.
19. Mode: follower

复制代码

连接其中一个服务并添加数据：

1. $ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2181
3. [zk: localhost:2181(CONNECTED) 0] create /pkslow
4. Created /pkslow
5. [zk: localhost:2181(CONNECTED) 1] create /pkslow/website www.pkslow.com
6. Created /pkslow/website

复制代码

连接另外一个服务，并查看数据，发现与之前创建的是一样的：

1. $ ./apache-zookeeper-3.7.1-bin/bin/zkCli.sh -server localhost:2182
3. [zk: localhost:2182(CONNECTED) 1] get /pkslow/website
4. www.pkslow.com

复制代码

目录结构如下：
安装Kafka集群

下载安装包

通过官网下载如下：

1. curl https://downloads.apache.org/kafka/3.4.0/kafka_2.13-3.4.0.tgz -o kafka_2.13-3.4.0.tgz

复制代码

解压安装包：

1. tar -xzf kafka_2.13-3.4.0.tgz

复制代码

配置

Broker 1的配置如下：

1. broker.id=1
2. port=9091
3. listeners=PLAINTEXT://:9091
4. zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
5. log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka1/kafka-logs

复制代码

Broker 2的配置如下：

1. broker.id=2
2. port=9092
3. listeners=PLAINTEXT://:9092
4. zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
5. log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka2/kafka-logs

复制代码

Broker 3的配置如下：

1. broker.id=3
2. port=9093
3. listeners=PLAINTEXT://:9093
4. zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
5. log.dirs=/Users/larry/IdeaProjects/pkslow-samples/other/install-kafka-cluster/src/main/kafka/kafka3/kafka-logs

复制代码

目录结构如下：

启动集群

启动kafka服务如下：

1. ./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka1/server.properties
2. ./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka2/server.properties
3. ./kafka_2.13-3.4.0/bin/kafka-server-start.sh ./kafka3/server.properties

复制代码

检查如测试

创建topic：

1. $ kafka_2.13-3.4.0/bin/kafka-topics.sh --create --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --partitions 3 --replication-factor 3
2. Created topic pkslow-topic.

复制代码

列出topic：

1. $ kafka_2.13-3.4.0/bin/kafka-topics.sh --list --bootstrap-server localhost:9091,localhost:9092,localhost:9093
2. pkslow-topic

复制代码

查看topic：

1. $ kafka_2.13-3.4.0/bin/kafka-topics.sh --describe --topic pkslow-topic --bootstrap-server localhost:9091,localhost:9092,localhost:9093
2. Topic: pkslow-topic     TopicId: 7CLy7iZeRvm8rCrn8Dw_mA PartitionCount: 3       ReplicationFactor: 3    Configs:
3.         Topic: pkslow-topic     Partition: 0    Leader: 3       Replicas: 3,1,2 Isr: 3,1,2
4.         Topic: pkslow-topic     Partition: 1    Leader: 1       Replicas: 1,2,3 Isr: 1,2,3
5.         Topic: pkslow-topic     Partition: 2    Leader: 2       Replicas: 2,3,1 Isr: 2,3,1

复制代码

生产者发消息到brokers：

1. $ kafka_2.13-3.4.0/bin/kafka-console-producer.sh --broker-list localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic
2. >My name is Larry Deng.
3. >My website is www.pkslow.com.
4. >

复制代码

消费者从brokers收消息：

1. $ kafka_2.13-3.4.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9091,localhost:9092,localhost:9093 --topic pkslow-topic --from-beginning
2. My name is Larry Deng.
3. My website is www.pkslow.com.

复制代码

代码

配置可以参考 GitHub pkslow-samples

免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！