/
Kafka

Kafka

docker pull bashj79/kafka-kraft docker run -p 9092:9092 -d --name myKafka bashj79/kafka-kraft

Create a topic

docker exec -it myKafka /bin/bash cd /opt/kafka/bin

create topic 'my-first-topic'

sh kafka-topics.sh --bootstrap-server localhost:9092 --create --topic my-first-topic --partitions 1 --replication-factor 1

list topics

send messages to the topic

Hello World
The weather is fine
I love Kafka

Using the Offset Explorer

The Offset Explorer (formerly: Kafka Tool) is a GUI application for managing Kafka. We can download and install it quickly. Then, we create a connection and specify the host and port of the Kafka broker

image-20240704-085158.png

You will see

image-20240704-085221.png

Using UI for Apache Kafka (Kafka UI) - my preferred option

The UI for Apache Kafka (Kafka UI) is a web UI, implemented with Spring Boot and React, and provided as a Docker container for a simple installation with the following command:

We can then open the UI in the browser using http://localhost:8080 and define a cluster, as this picture shows:

Because the Kafka broker runs in a different container than the Kafka UI’s backend, it will not have access to localhost:9092. We could instead address the host system using host.docker.internal:9092, but this is just the bootstrapping URL.

Unfortunately, Kafka itself will return a response that leads to a redirection to localhost:9092 again, which won’t work. If we do not want to configure Kafka (because this would break with the other clients then), we need to create a port forwarding from the Kafka UI’s container port 9092 to the host systems port 9092. The following sketch illustrates the connections:

We can set up this container-internal port forwarding, e.g. using socat. We have to install it within the container (Alpine Linux), so we need to connect to the container’s bash with root permissions. So we need these commands, beginning within the host system’s command line:

Connect to the container's bash (find out the name with 'docker ps')

Now, we are connected to the container's bash.

Let's install 'socat'

Use socat to create the port forwarding

This will lead to a running process that we don't kill as long as the container is running

kafka_producer app

Commands

Related content