Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save viiftw/7cfb5e3857fe234ea6724b1432a8164d to your computer and use it in GitHub Desktop.
Save viiftw/7cfb5e3857fe234ea6724b1432a8164d to your computer and use it in GitHub Desktop.

Revisions

  1. @olljanat olljanat revised this gist Jan 2, 2020. 1 changed file with 6 additions and 4 deletions.
    10 changes: 6 additions & 4 deletions overlay-network-and-static-ips-for-docker-containers.md
    Original file line number Diff line number Diff line change
    @@ -1,9 +1,11 @@
    I don't expect https://github.com/moby/moby/issues/24170 to be implemented on near future so this gist is about looking for least ugly workarounds which are available today and on near future.
    I don't expect [moby/moby#24170](https://github.com/moby/moby/issues/24170) to be solved by Docker team on near future so this gist is about looking for least ugly workarounds which are available today and on near future.

    # What is possible on Docker 19.03
    ## Network
    It is possible to create overlay network with use specified subnet.
    On this example I create network which is size is two C -class networks ( IP range 10.0.0.0 - 10.0.1.255 ) and force Docker to use IPs from second part of it ( 10.0.1.0 - 10.0.1.255 ).
    It is possible to create overlay network with use user specified subnet.
    On this example I create network which is size is two C -class networks ( IP range 10.0.0.0 - 10.0.1.255 )
    and force Docker to use IPs from second part of it ( 10.0.1.0 - 10.0.1.255 ).

    That way I can make sure that IPs 10.0.0.2 - 10.0.0.254 can be specified to containers and they do not collide with IPs which Docker engine assign for service(s)/container(s).
    ```bash
    docker network create --driver overlay --subnet 10.0.0.0/23 --ip-range 10.0.1.0/24 --gateway 10.0.0.1 --attachable test
    @@ -52,7 +54,7 @@ Above solution solves two issues:
    - It allow containers communicate between Docker nodes using overlay network.

    However those containers are not swarm services which means that you lose some control to them.
    Luckily I have already implemented https://github.com/docker/cli/pull/1940 and I'm expecting that it will get merged and released as part of next Docker version.
    Luckily I have already implemented [docker/cli#1940](https://github.com/docker/cli/pull/1940) and I'm expecting that it will get merged and released as part of next Docker version.

    That makes possible to use swarm stack like this:
    ```yaml
  2. @olljanat olljanat created this gist Jan 2, 2020.
    86 changes: 86 additions & 0 deletions overlay-network-and-static-ips-for-docker-containers.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,86 @@
    I don't expect https://github.com/moby/moby/issues/24170 to be implemented on near future so this gist is about looking for least ugly workarounds which are available today and on near future.

    # What is possible on Docker 19.03
    ## Network
    It is possible to create overlay network with use specified subnet.
    On this example I create network which is size is two C -class networks ( IP range 10.0.0.0 - 10.0.1.255 ) and force Docker to use IPs from second part of it ( 10.0.1.0 - 10.0.1.255 ).
    That way I can make sure that IPs 10.0.0.2 - 10.0.0.254 can be specified to containers and they do not collide with IPs which Docker engine assign for service(s)/container(s).
    ```bash
    docker network create --driver overlay --subnet 10.0.0.0/23 --ip-range 10.0.1.0/24 --gateway 10.0.0.1 --attachable test
    ```

    ## Containers
    By default containers IP addresses are controlled by Docker engine and those cannot be changed inside of container.
    Luckily that behavior can be overridden by adding `NET_ADMIN` capability for container.

    ### Custom Docker image
    So first we need create custom docker image which is able set IP for container when it starts.
    That we can do example with Dockerfile like this:
    ```
    FROM ubuntu
    RUN apt-get update \
    && apt-get install -y net-tools iputils-ping
    COPY /start.sh /
    ENTRYPOINT /start.sh
    ```
    and start script like this:
    ```bash
    #!/bin/bash
    if [[ -f "${STATIC_IP}" ]]; then
    echo "Using default IP from Docker"
    else
    echo "Found static IP: ${STATIC_IP} using it"
    ifconfig eth0 ${STATIC_IP} netmask 255.255.255.0 up
    fi
    sleep infinity
    ```
    Then just build that with command:
    ```bash
    docker build . -t static-ip
    ```

    ### Starting container
    Then we can start two containers with commands like these which connects then to overlay network and changes IP addresses inside of the to static ones.
    ```bash
    docker run --name test1 --detach=true --rm --cap-add=NET_ADMIN --network=test -e STATIC_IP=10.0.0.11 static-ip
    docker run --name test2 --detach=true --rm --cap-add=NET_ADMIN --network=test -e STATIC_IP=10.0.0.12 static-ip
    ```

    # What will be possible on next version
    Above solution solves two issues:
    - It sets static IPs for containers.
    - It allow containers communicate between Docker nodes using overlay network.

    However those containers are not swarm services which means that you lose some control to them.
    Luckily I have already implemented https://github.com/docker/cli/pull/1940 and I'm expecting that it will get merged and released as part of next Docker version.

    That makes possible to use swarm stack like this:
    ```yaml
    version: "3.7"
    services:
    test1:
    image: static-ip
    environment:
    STATIC_IP: 10.0.0.11
    networks:
    - test
    cap_add:
    - NET_ADMIN
    test2:
    image: static-ip
    environment:
    STATIC_IP: 10.0.0.12
    networks:
    - test
    cap_add:
    - NET_ADMIN

    networks:
    test:
    external: true
    name: test
    ```
    and deploy it with command:
    ```bash
    docker stack deploy -c static-ip.yml static-ip
    ```