Last active
January 21, 2024 12:10
-
-
Save abacaphiliac/f0553548f9c577214d16290c2e751071 to your computer and use it in GitHub Desktop.
Revisions
-
abacaphiliac revised this gist
Jan 17, 2017 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -46,5 +46,5 @@ This command will start an unnamed instance of `spotify/kafka` linked to the `ka docker run -it --rm --link kafka spotify/kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic test --from-beginning ``` # Send messages Send some newline-delimited messages in the Producer terminal window. The messages appear in the Consumer terminal window. -
abacaphiliac revised this gist
Jan 17, 2017 . 1 changed file with 4 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -44,4 +44,7 @@ docker run -it --rm --link kafka spotify/kafka /opt/kafka_2.11-0.10.1.0/bin/kafk This command will start an unnamed instance of `spotify/kafka` linked to the `kafka` service, start a consumer, display existing messages from the `test` topic, and wait for new messages until you quit (which destroys the container): ``` docker run -it --rm --link kafka spotify/kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic test --from-beginning ``` # Send some messages Send some newline-delimited messages in the Producer terminal window. The messages appear in the Consumer terminal window. -
abacaphiliac revised this gist
Jan 17, 2017 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -34,13 +34,13 @@ output: test ``` # Start a producer (in a new terminal window) This command will run an unnamed instance of `spotify/kafka` linked to the `kafka` service, start a producer, and wait for newline-delimited input until you quit (which destroys the container): ``` docker run -it --rm --link kafka spotify/kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-console-producer.sh --broker-list kafka:9092 --topic test ``` # Start a consumer (in a new terminal window) This command will start an unnamed instance of `spotify/kafka` linked to the `kafka` service, start a consumer, display existing messages from the `test` topic, and wait for new messages until you quit (which destroys the container): ``` docker run -it --rm --link kafka spotify/kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic test --from-beginning -
abacaphiliac revised this gist
Jan 17, 2017 . 1 changed file with 5 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -7,11 +7,12 @@ docker run -d -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=kafka --env ADVERT [Why Spotify?](https://github.com/spotify/docker-kafka#why) `ADVERTISTED_HOST` was set to `kafka`, which will allow other containers to be able to run Producers and Consumers. Setting `ADVERTISED_HOST` to `localhost`, `127.0.0.1`, or `0.0.0.0` will work great only if Producers and Consumers are started within the `kafka` container itself, or if you are using DockerForMac (like me) and you want to run Producers and Consumers from OSX. These are far less interesting use cases though, so we'll start Producers and Consumers from other containers. We need to use an IP address or hostname in order for the `kafka` service to be reachable from another container. IP address is not known before the container is started, so we have to choose a hostname, and I chose `kafka` in this example. # Create a topic ``` -
abacaphiliac revised this gist
Jan 17, 2017 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -8,7 +8,7 @@ docker run -d -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=kafka --env ADVERT [Why Spotify?](https://github.com/spotify/docker-kafka#why) `ADVERTISTED_HOST` was set to `kafka`, which will allow other containers to be able to run Producers and/or Consumers. Setting it to `localhost`, `127.0.0.1`, or `0.0.0.0` will work great only if Producers and Consumers are started within the `kafka` container itself, or if you are using DockerForMac (like me) and you want to run Producers and Consumers from OSX. These are far less interesting use cases though, so we'll start Producers and Consumers from other containers. In order for the `kafka` service to be reachable from another container, we need to use an IP address or hostname. Since we do not know the IP address before the container is started, we have to choose a hostname, and I chose `kafka` in this example. -
abacaphiliac revised this gist
Jan 17, 2017 . 1 changed file with 2 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -10,6 +10,8 @@ docker run -d -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=kafka --env ADVERT `ADVERTISTED_HOST` was set to `kafka`, which will allow other containers to be able to run Producers and/or Consumers. Setting it to `localhost`, `127.0.0.1`, or `0.0.0.0` will work great if Producers and Consumers are started within the `kafka` container itself, or if you are using DockerForMac (like me) and you want to run Producers and Consumers from OSX. These are far less interesting use cases though, so we'll start Producers and Consumers from other containers. In order for the `kafka` service to be reachable from another container, we need to use an IP address or hostname. Since we do not know the IP address before the container is started, we have to choose a hostname, and I chose `kafka` in this example. # Create a topic ``` -
abacaphiliac revised this gist
Jan 17, 2017 . 1 changed file with 2 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,6 +5,8 @@ docker pull spotify/kafka docker run -d -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=kafka --env ADVERTISED_PORT=9092 --name kafka spotify/kafka ``` [Why Spotify?](https://github.com/spotify/docker-kafka#why) `ADVERTISTED_HOST` was set to `kafka`, which will allow other containers to be able to run Producers and/or Consumers. Setting it to `localhost`, `127.0.0.1`, or `0.0.0.0` will work great if Producers and Consumers are started within the `kafka` container itself, or if you are using DockerForMac (like me) and you want to run Producers and Consumers from OSX. These are far less interesting use cases though, so we'll start Producers and Consumers from other containers. @@ -29,8 +31,6 @@ output: test ``` # Start a producer (in another container) This command will run an unnamed instance of `spotify/kafka` linked to the `kafka` service, start a producer, and wait for newline-delimited input until you quit (which destroys the container): ``` -
abacaphiliac created this gist
Jan 17, 2017 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,44 @@ # Start Kafka service The following commands will start a container with Kafka and Zookeeper running on mapped ports 2181 (Zookeeper) and 9092 (Kafka). ``` docker pull spotify/kafka docker run -d -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=kafka --env ADVERTISED_PORT=9092 --name kafka spotify/kafka ``` `ADVERTISTED_HOST` was set to `kafka`, which will allow other containers to be able to run Producers and/or Consumers. Setting it to `localhost`, `127.0.0.1`, or `0.0.0.0` will work great if Producers and Consumers are started within the `kafka` container itself, or if you are using DockerForMac (like me) and you want to run Producers and Consumers from OSX. These are far less interesting use cases though, so we'll start Producers and Consumers from other containers. # Create a topic ``` docker exec kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test ``` output: ``` Created topic "test". ``` # List topics ``` docker exec kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-topics.sh --list --zookeeper localhost:2181 ``` output: ``` test ``` [Why Spotify?](https://github.com/spotify/docker-kafka#why) # Start a producer (in another container) This command will run an unnamed instance of `spotify/kafka` linked to the `kafka` service, start a producer, and wait for newline-delimited input until you quit (which destroys the container): ``` docker run -it --rm --link kafka spotify/kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-console-producer.sh --broker-list kafka:9092 --topic test ``` # Start a consumer (in another container) This command will start an unnamed instance of `spotify/kafka` linked to the `kafka` service, start a consumer, display existing messages from the `test` topic, and wait for new messages until you quit (which destroys the container): ``` docker run -it --rm --link kafka spotify/kafka /opt/kafka_2.11-0.10.1.0/bin/kafka-console-consumer.sh --bootstrap-server kafka:9092 --topic test --from-beginning ```