Before starting this tutorial, please create a Topic first. Learn How to create a Topic in Kafka.
— Yes, it is possible but not a good idea to create a topic in this way.
kafka-console-producer.sh --bootstrap-server 127.0.0.1:9092 --topic new_topic
→ Think KafkaTemplate as a JDBCTemplate for database interactions.
KafkaTemplate wraps a producer and provides convenient methods to send data to Kafka topics. The following listing shows the relevant methods from
— If you look below it has many different overloaded versions of
KafkaTemplate with send method.
KafkaTemplate.send() is going to send the message to Kafka. But in reality, it goes through different layers before the message is sent to Kafka.
— The very first layer is the serializer. Any records sent to Kafka need to be serialized to bytes.
There are two serialization techniques that…
group-idis mandatory, it plays a major role when it comes to scalable message consumption.
group-idplays a major role when it comes to scalable message consumption.
let’s consider we have a topic
test-topic and 4 partitions. Now we have a consumer-ready with group 1. …
We have a zookeeper and Kafka cluster. In this example, we have a cluster with 3 brokers.
Here we have a Kafka cluster and a representation of how the topic is distributed across the cluster and we have some records present in the file system.
As we all know clients are producers and consumers always talk to the leader to retrieve data from the partition.