Kafka
What is Kafka? Explain Kafka in detail.
Apache Kafka is an event streaming platform.
2 min readApr 7, 2021
- Kafka is open-source software that provides a framework for storing, reading, and analyzing streaming data.
- Kafka was originally created at LinkedIn, where it played a part in analyzing the connections between their millions of professional users in order to build networks between people.
How Kafka works?
Apache Kafka is a publish-subscribe-based durable messaging system. A messaging system sends messages between processes, applications, and servers.
Figure: This example topic has four partitions P1–P4.
- Two different producer clients are publishing, independently from each other, new events to the topic by writing events over the network to the topic’s partitions.
- Events with the same key (denoted by their color in the figure) are written to the same partition.
Note that both producers can write to the same partition if appropriate.
Uses of Kafka
- Kafka is used for real-time streams of data, to collect big data, or to do real-time analysis (or both).
- Kafka is used with in-memory microservices to provide durability and it can be used to feed events to CEP (complex event streaming systems) and IoT/IFTTT-style automation systems.
