Skip to content

Instantly share code, notes, and snippets.

@stimata-debug

stimata-debug/docker-for-the-impatient.md

Forked from jaceklaskowski/docker-for-the-impatient.md
Created November 2, 2016 12:50
Show Gist options
  • Save stimata-debug/6434e15d489c9e5d395de51a5d25ea5a to your computer and use it in GitHub Desktop.
Save stimata-debug/6434e15d489c9e5d395de51a5d25ea5a to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @jaceklaskowski jaceklaskowski revised this gist Jul 14, 2015. 1 changed file with 1 addition and 105 deletions.
    106 changes: 1 addition & 105 deletions docker-for-the-impatient.md
    View file
    Original file line number Diff line number Diff line change
    @@ -24,111 +24,7 @@ You can work in the Docker container using:

    ## Docker sample - Easy deployment of Apache Kafka using Docker

    You're going to use the images [wurstmeister/kafka](https://registry.hub.docker.com/u/wurstmeister/kafka/) and [wurstmeister/zookeeper](https://registry.hub.docker.com/u/wurstmeister/zookeeper/).

    For now, you'll need two terminals for each server - Apache Kafka and Apache Zookeeper.

    1. `boot2docker up` to start the tiny Linux core on Mac OS.

    ➜ ~ boot2docker up
    Waiting for VM and Docker daemon to start...
    .o
    Started.
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/ca.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/cert.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/key.pem

    To connect the Docker client to the Docker daemon, please set:
    export DOCKER_TLS_VERIFY=1
    export DOCKER_HOST=tcp://192.168.59.103:2376
    export DOCKER_CERT_PATH=/Users/jacek/.boot2docker/certs/boot2docker-vm

    1. `$(boot2docker shellinit)` to have terminal set up. You should be doing the step in any terminal you open to work with Docker so the `export`s above are set. Any issue with `docker` commands, think about the step.

    1. As an additional check, run `docker ps` to verify the terminal is configured properly for Docker.

    ➜ ~ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

    No containers are running at this time. It will change soon once we start the containers for Kafka and Zookeeper.

    1. Create an account on Docker Hub and execute `docker login` to store the credentials so you don't have to repeat them whenever needed, esp. for `docker pull` to pull images off the hub.

    1. Run `docker pull wurstmeister/zookeeper` to pull the Zookeeper image off Docker Hub. It takes a very long time as the repository is very sluggish. You will see hashes of respective layers printed out on the console. It's expected.

    ➜ ~ docker pull wurstmeister/zookeeper
    Pulling repository wurstmeister/zookeeper
    a3075a3d32da: Download complete
    ...
    840840289a0d: Download complete
    e7381f1a45cf: Download complete
    5a6fc057f418: Download complete
    Status: Downloaded newer image for wurstmeister/zookeeper:latest

    1. `docker pull wurstmeister/kafka` to pull the Kafka image off Docker Hub.

    ➜ ~ docker pull wurstmeister/kafka
    latest: Pulling from wurstmeister/kafka
    428b411c28f0: Pull complete
    ...
    422705fe88c8: Pull complete
    02bb7ca441d8: Pull complete
    0f9a08061516: Pull complete
    24fc32f98556: Already exists
    Digest: sha256:06150c136dcfe6e4fbbf37731a2119ea17a953c75902e52775b5511b3572aa1f
    Status: Downloaded newer image for wurstmeister/kafka:latest

    1. `docker run --rm --name zookeeper -p 2181 -t wurstmeister/zookeeper` in one terminal to boot Zookeeper up.

    Remember `$(boot2docker shellinit)`

    ➜ ~ docker run --rm --name zookeeper -p 2181 -t wurstmeister/zookeeper
    JMX enabled by default
    Using config: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
    2015-07-09 07:33:16,731 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
    ...
    2015-07-09 07:33:16,764 [myid:] - INFO [main:ZooKeeperServer@773] - maxSessionTimeout set to -1
    2015-07-09 07:33:16,778 [myid:] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181

    1. `docker run --rm --name kafka -e HOST_IP=localhost -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_BROKER_ID=1 -e ZK=zk -p 9092 --link zookeeper:zk -t wurstmeister/kafka` in another terminal.

    Remember `$(boot2docker shellinit)`

    ➜ ~ docker run --rm --name kafka -e HOST_IP=localhost -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_BROKER_ID=1 -e ZK=zk -p 9092 --link zookeeper:zk -t wurstmeister/kafka
    [2015-07-09 07:35:30,145] INFO Verifying properties (kafka.utils.VerifiableProperties)
    [2015-07-09 07:35:30,173] INFO Property advertised.port is overridden to 9092 (kafka.utils.VerifiableProperties)
    ...
    [2015-07-09 07:35:30,437] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
    [2015-07-09 07:35:30,439] INFO [Socket Server on Broker 1], Started (kafka.network.SocketServer)
    [2015-07-09 07:35:30,496] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
    [2015-07-09 07:35:30,533] INFO 1 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
    [2015-07-09 07:35:30,613] INFO Registered broker 1 at path /brokers/ids/1 with address 0b34a9927004:9092. (kafka.utils.ZkUtils$)
    [2015-07-09 07:35:30,632] INFO [Kafka Server 1], started (kafka.server.KafkaServer)
    [2015-07-09 07:35:30,682] INFO New leader is 1 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)

    You're now a happy user of Apache Kafka on your computer using Docker. Check the status of the containers using `docker ps`:

    ➜ ~ $(boot2docker shellinit)
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/ca.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/cert.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/key.pem
    ➜ ~ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    0b34a9927004 wurstmeister/kafka "/bin/sh -c start-ka 2 minutes ago Up 2 minutes 0.0.0.0:32769->9092/tcp kafka
    14fd32558b1c wurstmeister/zookeeper "/bin/sh -c '/usr/sb 4 minutes ago Up 4 minutes 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:32768->2181/tcp zookeeper

    Once you're done with your journey into Apache Kafka, `docker stop` the containers using `docker stop kafka zookeeper` (or `docker stop $(docker ps -aq)` if the only running containers are `kafka` and `zookeeper`).

    ➜ ~ docker stop kafka zookeeper
    kafka
    zookeeper

    Running `docker ps` should show no running containers:

    ➜ ~ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

    Stop `boot2docker` daemon using `boot2docker down`.
    This part was published as a blog post [Apache Kafka on Docker](http://blog.jaceklaskowski.pl/2015/07/14/apache-kafka-on-docker.html) as part of a larger project to explore the features of [Apache Kafka](http://kafka.apache.org).

    ## Docker sample - Artifactory on Docker

  2. @jaceklaskowski jaceklaskowski revised this gist Jul 12, 2015. 1 changed file with 55 additions and 0 deletions.
    55 changes: 55 additions & 0 deletions docker-for-the-impatient.md
    View file
    Original file line number Diff line number Diff line change
    @@ -16,6 +16,12 @@ There's Docker Hub to host Docker images (just like GitHub for social coding). P

    where `user` is the owner of the `image` on Docker Hub.

    ## Working inside a Docker container

    You can work in the Docker container using:

    docker exec -it <container name> /bin/bash

    ## Docker sample - Easy deployment of Apache Kafka using Docker

    You're going to use the images [wurstmeister/kafka](https://registry.hub.docker.com/u/wurstmeister/kafka/) and [wurstmeister/zookeeper](https://registry.hub.docker.com/u/wurstmeister/zookeeper/).
    @@ -124,6 +130,54 @@ Running `docker ps` should show no running containers:

    Stop `boot2docker` daemon using `boot2docker down`.

    ## Docker sample - Artifactory on Docker

    There are different ways to run [Artifactory](http://www.jfrog.com/open-source/) on your laptop. You may pull down zip or rpm files and install them on your local workstation. Or you could merely `docker pull` the latest Docker image of Artifactory and give it a go without messing the local workstation up (at least not that much as installing it on your OS directly).

    Consult [Artifactory as a Docker Image](http://www.jfrog.com/confluence/display/RTF/Running+with+Docker).

    1. `boot2docker up` unless you've done it already.

    1. `$(boot2docker shellinit)` to set up the terminal.

    1. Execute `docker pull jfrog-docker-registry.bintray.io/artifactory/artifactory-oss` to download Artifactory OSS's Docker image.

    ➜ ~ docker pull jfrog-docker-registry.bintray.io/artifactory/artifactory-oss
    Pulling repository jfrog-docker-registry.bintray.io/artifactory/artifactory-oss
    22a955eb08d4: Pulling dependent layers
    511136ea3c5a: Download complete
    5b12ef8fd570: Download complete
    ...
    091320beb869: Download complete
    c33fb0470c44: Download complete
    8d466d99a7ce: Download complete
    Status: Downloaded newer image for jfrog-docker-registry.bintray.io/artifactory/artifactory-oss:latest

    1. `docker run --rm --name artifactory -p 8081:8081 -p 80:80 -t jfrog-docker-registry.bintray.io/artifactory/artifactory-oss`

    ➜ development_environment git:(DS-880-ansible) ✗ docker run --rm --name artifactory -t jfrog-docker-registry.bintray.io/artifactory/artifactory-oss
    Starting nginx: [ OK ]
    Starting Artifactory tomcat as user artifactory...
    ...
    ###########################################################
    ### Artifactory successfully started (8.133 seconds) ###
    ###########################################################

    1. `docker ps` to see the Artifactory container's status

    ➜ development_environment git:(DS-880-ansible) ✗ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    4cf495405166 jfrog-docker-registry.bintray.io/artifactory/artifactory-oss "/tmp/runArtifactory About a minute ago Up About a minute 0.0.0.0:80->80/tcp, 0.0.0.0:8081->8081/tcp, 443/tcp artifactory

    1. Enter Artifactory's using http://localhost/artifactory. If you are running on a Windows or Mac systems using `boot2docker` you need to use DOCKER_HOST IP instead of localhost, i.e. `boot2docker ip`.

    1. You may also access the shell of the container using `docker exec -it artifactory /bin/bash`.

    ➜ ~ docker exec -it artifactory /bin/bash
    [root@eb7e727e5b36 /]#

    Please restrain yourself from modifying the container as the idea of Docker containers is that they're immutable once booted up and any changes should rather be done at their definition.

    ## Docker and GitHub

    1. Create `docker-whatever` repo on GitHub with `Dockerfile` and `README.md`.
    @@ -143,3 +197,4 @@ Enter the shell of a given container using `docker run -ti -p 8000:8000 mistio /

    * https://github.com/paimpozhil/docker-mist#docker-mist
    * https://github.com/sequenceiq/docker-enter/blob/master/README.md
    * http://www.jfrog.com/confluence/display/RTF/Running+with+Docker
  3. @jaceklaskowski jaceklaskowski revised this gist Jul 9, 2015. 1 changed file with 143 additions and 2 deletions.
    145 changes: 143 additions & 2 deletions docker-for-the-impatient.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,145 @@
    aka "Let's take some notes about using Docker on Mac OS X to turn Scala applications deployment into a much better experience."
    *aka* "Let's take some notes about using [Docker](https://www.docker.com/) on Mac OS X to turn deployment of Scala applications into a much better experience."

    *DISCLAIMER* The doc is a compilation of different articles and videos found on the Internet. Almost nothing's mine. See CREDITS section, below.
    **DISCLAIMER** The doc is a compilation of different articles and videos found on the Internet. Almost nothing's mine - mostly layout. See CREDITS section below to know who to praise. All mistakes are mine and are not intended. Drop me an email at [[email protected]](mailto:[email protected]) if you spot any errors or just share what you think about the doc.

    The document lives at https://gist.github.com/jaceklaskowski/ca55be80cb76e84ce478

    ## Intro

    I'm on Mac OS X and so you're going to see a lot of setup tweaks for the platform that are not necessarily needed for your environment, esp. Linux one. When you see `boot2docker` and you're on Linux, just disregard the line or even entire paragraph.

    Docker doesn't run directly on OS X and on Windows and that's why you need `boot2docker` that is just a tiny core of Linux to have environment with things required to run Docker - lxc, cgroups and others (highly advanced)

    There's Docker Hub to host Docker images (just like GitHub for social coding). Pick one image and start it off right away using:

    docker run --rm user/image

    where `user` is the owner of the `image` on Docker Hub.

    ## Docker sample - Easy deployment of Apache Kafka using Docker

    You're going to use the images [wurstmeister/kafka](https://registry.hub.docker.com/u/wurstmeister/kafka/) and [wurstmeister/zookeeper](https://registry.hub.docker.com/u/wurstmeister/zookeeper/).

    For now, you'll need two terminals for each server - Apache Kafka and Apache Zookeeper.

    1. `boot2docker up` to start the tiny Linux core on Mac OS.

    ➜ ~ boot2docker up
    Waiting for VM and Docker daemon to start...
    .o
    Started.
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/ca.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/cert.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/key.pem

    To connect the Docker client to the Docker daemon, please set:
    export DOCKER_TLS_VERIFY=1
    export DOCKER_HOST=tcp://192.168.59.103:2376
    export DOCKER_CERT_PATH=/Users/jacek/.boot2docker/certs/boot2docker-vm

    1. `$(boot2docker shellinit)` to have terminal set up. You should be doing the step in any terminal you open to work with Docker so the `export`s above are set. Any issue with `docker` commands, think about the step.

    1. As an additional check, run `docker ps` to verify the terminal is configured properly for Docker.

    ➜ ~ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

    No containers are running at this time. It will change soon once we start the containers for Kafka and Zookeeper.

    1. Create an account on Docker Hub and execute `docker login` to store the credentials so you don't have to repeat them whenever needed, esp. for `docker pull` to pull images off the hub.

    1. Run `docker pull wurstmeister/zookeeper` to pull the Zookeeper image off Docker Hub. It takes a very long time as the repository is very sluggish. You will see hashes of respective layers printed out on the console. It's expected.

    ➜ ~ docker pull wurstmeister/zookeeper
    Pulling repository wurstmeister/zookeeper
    a3075a3d32da: Download complete
    ...
    840840289a0d: Download complete
    e7381f1a45cf: Download complete
    5a6fc057f418: Download complete
    Status: Downloaded newer image for wurstmeister/zookeeper:latest

    1. `docker pull wurstmeister/kafka` to pull the Kafka image off Docker Hub.

    ➜ ~ docker pull wurstmeister/kafka
    latest: Pulling from wurstmeister/kafka
    428b411c28f0: Pull complete
    ...
    422705fe88c8: Pull complete
    02bb7ca441d8: Pull complete
    0f9a08061516: Pull complete
    24fc32f98556: Already exists
    Digest: sha256:06150c136dcfe6e4fbbf37731a2119ea17a953c75902e52775b5511b3572aa1f
    Status: Downloaded newer image for wurstmeister/kafka:latest

    1. `docker run --rm --name zookeeper -p 2181 -t wurstmeister/zookeeper` in one terminal to boot Zookeeper up.

    Remember `$(boot2docker shellinit)`

    ➜ ~ docker run --rm --name zookeeper -p 2181 -t wurstmeister/zookeeper
    JMX enabled by default
    Using config: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
    2015-07-09 07:33:16,731 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper-3.4.6/bin/../conf/zoo.cfg
    ...
    2015-07-09 07:33:16,764 [myid:] - INFO [main:ZooKeeperServer@773] - maxSessionTimeout set to -1
    2015-07-09 07:33:16,778 [myid:] - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181

    1. `docker run --rm --name kafka -e HOST_IP=localhost -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_BROKER_ID=1 -e ZK=zk -p 9092 --link zookeeper:zk -t wurstmeister/kafka` in another terminal.

    Remember `$(boot2docker shellinit)`

    ➜ ~ docker run --rm --name kafka -e HOST_IP=localhost -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_BROKER_ID=1 -e ZK=zk -p 9092 --link zookeeper:zk -t wurstmeister/kafka
    [2015-07-09 07:35:30,145] INFO Verifying properties (kafka.utils.VerifiableProperties)
    [2015-07-09 07:35:30,173] INFO Property advertised.port is overridden to 9092 (kafka.utils.VerifiableProperties)
    ...
    [2015-07-09 07:35:30,437] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
    [2015-07-09 07:35:30,439] INFO [Socket Server on Broker 1], Started (kafka.network.SocketServer)
    [2015-07-09 07:35:30,496] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
    [2015-07-09 07:35:30,533] INFO 1 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
    [2015-07-09 07:35:30,613] INFO Registered broker 1 at path /brokers/ids/1 with address 0b34a9927004:9092. (kafka.utils.ZkUtils$)
    [2015-07-09 07:35:30,632] INFO [Kafka Server 1], started (kafka.server.KafkaServer)
    [2015-07-09 07:35:30,682] INFO New leader is 1 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)

    You're now a happy user of Apache Kafka on your computer using Docker. Check the status of the containers using `docker ps`:

    ➜ ~ $(boot2docker shellinit)
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/ca.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/cert.pem
    Writing /Users/jacek/.boot2docker/certs/boot2docker-vm/key.pem
    ➜ ~ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    0b34a9927004 wurstmeister/kafka "/bin/sh -c start-ka 2 minutes ago Up 2 minutes 0.0.0.0:32769->9092/tcp kafka
    14fd32558b1c wurstmeister/zookeeper "/bin/sh -c '/usr/sb 4 minutes ago Up 4 minutes 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:32768->2181/tcp zookeeper

    Once you're done with your journey into Apache Kafka, `docker stop` the containers using `docker stop kafka zookeeper` (or `docker stop $(docker ps -aq)` if the only running containers are `kafka` and `zookeeper`).

    ➜ ~ docker stop kafka zookeeper
    kafka
    zookeeper

    Running `docker ps` should show no running containers:

    ➜ ~ docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

    Stop `boot2docker` daemon using `boot2docker down`.

    ## Docker and GitHub

    1. Create `docker-whatever` repo on GitHub with `Dockerfile` and `README.md`.
    1. Run `docker run -td -p 8000:8000 paimpozhil/docker-mist` to use the image from Docker Hub.
    1. Use `8000` as if you ran the app on the bare metal of your local workstation.

    If you however want to customize the Docker-based offering, `git clone` the repo and tweak `Dockerfile` that comes with it:

    1. `git clone https://github.com/paimpozhil/docker-mist.git`
    1. `cd docker-mist`
    1. `docker build -t mistio .`
    1. `docker run -td -p 8000:8000 mistio`

    Enter the shell of a given container using `docker run -ti -p 8000:8000 mistio /bin/bash`. Note the `/bin/bash` as the last parameter that gives you the entry point to the running container.

    ## Credits (aka who to praise while only me to blame)

    * https://github.com/paimpozhil/docker-mist#docker-mist
    * https://github.com/sequenceiq/docker-enter/blob/master/README.md
  4. @jaceklaskowski jaceklaskowski created this gist Jul 9, 2015.
    4 changes: 4 additions & 0 deletions docker-for-the-impatient.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,4 @@
    aka "Let's take some notes about using Docker on Mac OS X to turn Scala applications deployment into a much better experience."

    *DISCLAIMER* The doc is a compilation of different articles and videos found on the Internet. Almost nothing's mine. See CREDITS section, below.