Sending and Receiving messages

How to Publish and Subscribe messages (without key) using Kafka?

Let’s learn how key solves the problem of messages to random partitions.

Before starting this tutorial, please create a Topic first. Learn How to create a Topic in Kafka.

Without key

• The partitioner first checks whether the key is present as part of the message or not. In this example, we are not sending any key, so in this case, the partitioner will use the Round Robin algorithm to send the messages.

• In the above approach, there is no guarantee the consumer will able to read the messages in the same order because the consumer pulls the messages from all the partitions at the same time.

Sending and Receiving messages

• make sure you in c:\kafka directory (root)

How to instantiate a Console Producer?

Step 1: let's start our Kafka producer

• open new cmd to send messages

.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic test-topic

→ Type your message in a single line

Producer — Creating and sending messages to consumers.

→ Now we have sent messages till now, now Kafka consumer is waiting to consume these messages.

How to instantiate a Console Consumer?

Step 2: let's start our Kafka consumer

• open new cmd to receive messages

.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test-topic --from-beginning

→ Type your message in a single line e.g

Consumer- Receiving the messages sent by the Producer.

→ Now we have received a message till now, which is sent by Kafka Producer and it will continuously wait in queue for the new message from the producer.

Note: The --from-beginning is very important, and if not used-

• it will be not able to read older which was sent by the Producer i.e it is going to read only those messages which are just sent.
• If there are multiple partitions say --partitions 4then the order will not be maintained and messages will go into other partitions.
• So in order to post messages in the same order then make sure you are posting to the same partition, so for getting the same order, we need to pass the same key, then messages will be retrieved in the same order.

Note: you can also check for the server console for logs or what is activities going here, so you see what is clean-up policy, what are the segments available, how many partitions are there, data retention time period, etc.

Kafka Tutorials

Next →

How to Publish and Subscribe messages (with key) using Kafka?

Previous ←

2. How to create Topics with Partitions & Replication factor using Kafka?