Docker compose Kafka, Zookeeper and Kafka manager

Gist inspired by this one intending to be an updated version of it : https://gist.github.com/17twenty/d619f922ab209f750630824f7c6836e3

Install

• Create the environment variable that contains our host name (IP address) :

$ export EXPOSED_HOSTNAME=$(ip addr | grep 'state UP' -A2 | tail -n1 | awk '{print $2}' | cut -f1  -d'/')

• Create the folders :

$ mkdir -p zookeeper/{data,logs}

Run

• Execute the docker compose :

$ docker-compose -f docker-compose.yml up -d

• Access Kafka Manager : http://localhost:9000/.

• Add a new cluster.

• Name it as you like (Localtest for example).

• IP : localhost:2181

• Kafka version (corresponding here to the Kafka version inside the image : wurstmeister/kafka:1.0.0). But as I write this Gist, the latest Kafka version available in Kafka Manager is 0.11.0.0 so I select this one, but it's sufficiently compatible with the 1.0.0 version of Kafka according to this topic: yahoo/CMAK#451

• Tick Enable JMX Polling in order to see the metrics of the topics

• Tick Poll consumer information to know the consumer of a topic (it may not work)

• Tick Enable Active OffsetCache to see the offsets

And then save it.

✨ This is done ! ✨

Uninstall

• Stop the containers and remove them :

$ docker-compose -f docker-compose.yml stop && docker-compose -f docker-compose.yml rm -vf

Tips

• Remove content of Docker volumes (we need to do this trick to remove hidden and non-hidden files) :

$ rm -rf zookeeper/{data,logs} && mkdir -p zookeeper/{data,logs}