Kafka Cluster

Set up a Kafka Cluster in Local with multiple Kafka Brokers

In this example, we are going to set up a Kafka Cluster in Local with 3 Kafka Brokers.

Multiple Brokers in Kafka

Setting up a cluster (configuration)

Creating new Broker-1

Do the following changes in the file.

Creating new Broker-2

Starting up these 2 Kafka brokers

.\bin\windows\kafka-server-start.bat .\config\server-1.properties
.\bin\windows\kafka-server-start.bat .\config\server-2.properties

Kafka Cluster

Running 3 brokers simultaneously.

Creating new Topic

.\bin\windows\kafka-topics.bat --create --topic test-topic-replicated -zookeeper localhost:2181 --replication-factor 3 --partitions 3
a new topic is created

Produce the messages using console producer.

.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test-topic-replicated
message sent: Hi

Instantiate a new Consumer to receive the messages.

.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test-topic-replicated --from-beginning
message received: Hi

Log directories

Software Engineer at HCL | Technical Content Writing | Follow me on LinkedIn https://www.linkedin.com/in/sagarkudu/