Forked from olljanat/overlay-network-and-static-ips-for-docker-containers.md
Created
March 26, 2024 04:51
-
-
Save viiftw/7cfb5e3857fe234ea6724b1432a8164d to your computer and use it in GitHub Desktop.
Revisions
-
olljanat revised this gist
Jan 2, 2020 . 1 changed file with 6 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,9 +1,11 @@ I don't expect [moby/moby#24170](https://github.com/moby/moby/issues/24170) to be solved by Docker team on near future so this gist is about looking for least ugly workarounds which are available today and on near future. # What is possible on Docker 19.03 ## Network It is possible to create overlay network with use user specified subnet. On this example I create network which is size is two C -class networks ( IP range 10.0.0.0 - 10.0.1.255 ) and force Docker to use IPs from second part of it ( 10.0.1.0 - 10.0.1.255 ). That way I can make sure that IPs 10.0.0.2 - 10.0.0.254 can be specified to containers and they do not collide with IPs which Docker engine assign for service(s)/container(s). ```bash docker network create --driver overlay --subnet 10.0.0.0/23 --ip-range 10.0.1.0/24 --gateway 10.0.0.1 --attachable test @@ -52,7 +54,7 @@ Above solution solves two issues: - It allow containers communicate between Docker nodes using overlay network. However those containers are not swarm services which means that you lose some control to them. Luckily I have already implemented [docker/cli#1940](https://github.com/docker/cli/pull/1940) and I'm expecting that it will get merged and released as part of next Docker version. That makes possible to use swarm stack like this: ```yaml -
olljanat created this gist
Jan 2, 2020 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,86 @@ I don't expect https://github.com/moby/moby/issues/24170 to be implemented on near future so this gist is about looking for least ugly workarounds which are available today and on near future. # What is possible on Docker 19.03 ## Network It is possible to create overlay network with use specified subnet. On this example I create network which is size is two C -class networks ( IP range 10.0.0.0 - 10.0.1.255 ) and force Docker to use IPs from second part of it ( 10.0.1.0 - 10.0.1.255 ). That way I can make sure that IPs 10.0.0.2 - 10.0.0.254 can be specified to containers and they do not collide with IPs which Docker engine assign for service(s)/container(s). ```bash docker network create --driver overlay --subnet 10.0.0.0/23 --ip-range 10.0.1.0/24 --gateway 10.0.0.1 --attachable test ``` ## Containers By default containers IP addresses are controlled by Docker engine and those cannot be changed inside of container. Luckily that behavior can be overridden by adding `NET_ADMIN` capability for container. ### Custom Docker image So first we need create custom docker image which is able set IP for container when it starts. That we can do example with Dockerfile like this: ``` FROM ubuntu RUN apt-get update \ && apt-get install -y net-tools iputils-ping COPY /start.sh / ENTRYPOINT /start.sh ``` and start script like this: ```bash #!/bin/bash if [[ -f "${STATIC_IP}" ]]; then echo "Using default IP from Docker" else echo "Found static IP: ${STATIC_IP} using it" ifconfig eth0 ${STATIC_IP} netmask 255.255.255.0 up fi sleep infinity ``` Then just build that with command: ```bash docker build . -t static-ip ``` ### Starting container Then we can start two containers with commands like these which connects then to overlay network and changes IP addresses inside of the to static ones. ```bash docker run --name test1 --detach=true --rm --cap-add=NET_ADMIN --network=test -e STATIC_IP=10.0.0.11 static-ip docker run --name test2 --detach=true --rm --cap-add=NET_ADMIN --network=test -e STATIC_IP=10.0.0.12 static-ip ``` # What will be possible on next version Above solution solves two issues: - It sets static IPs for containers. - It allow containers communicate between Docker nodes using overlay network. However those containers are not swarm services which means that you lose some control to them. Luckily I have already implemented https://github.com/docker/cli/pull/1940 and I'm expecting that it will get merged and released as part of next Docker version. That makes possible to use swarm stack like this: ```yaml version: "3.7" services: test1: image: static-ip environment: STATIC_IP: 10.0.0.11 networks: - test cap_add: - NET_ADMIN test2: image: static-ip environment: STATIC_IP: 10.0.0.12 networks: - test cap_add: - NET_ADMIN networks: test: external: true name: test ``` and deploy it with command: ```bash docker stack deploy -c static-ip.yml static-ip ```