By Alwyn Botha, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.
This set of tutorials focuses on giving you practical experience on using Docker Compose when working with containers on Alibaba Cloud Elastic Compute Service (ECS).
Part 2 of this series explored several Docker compose configurations. In Part 3, we will explore depends_on, volumes and the important init docker-compose options.
depends_on declares dependencies between services.
depends_on let docker-compose up starts services in dependency order.
The dependent services start first, before the main service starts.
As you will see the dependencies start in random order. Also, docker-compose does not wait for any dependencies to be running before starting the main service.
Let's see that in action so you can understand how it works:
Add the following to your docker-compose.yml using
nano docker-compose.ymlversion: "3.7"
services:
alpine:
image: alpine:3.8
command: sleep 600
depends_on:
- service-2
- service-3
- service-4
container_name: main-container
service-2:
image: alpine:3.8
container_name: service-2
command: sleep 600
service-3:
image: alpine:3.8
container_name: service-3
command: sleep 600
service-4:
image: alpine:3.8
container_name: service-4
command: sleep 600Please note the dependencies are declared in neat number order.
Let's start up this docker-compose file using:
docker-compose up -d -t 0Expected output :
Creating service-3 ... done
Creating service-4 ... done
Creating service-2 ... done
Recreating compose-tuts_alpine_1 ... doneNote that dependencies started in random order. The main service started last - not clearly shown in above output.
If you make small edits to the docker-compose file ( such as sleep time edits ) and redo the docker-compose up you will see those services created in different sequences each time. Try it.
To more clearly see the sequence of events, let us shut down all services an redo the docker-compose up.
docker-compose down -t 0Expected output :
Stopping main-container ... done
Stopping service-4 ... done
Stopping service-3 ... done
Stopping service-2 ... done
Removing main-container ... done
Removing service-4 ... done
Removing service-3 ... done
Removing service-2 ... done
Removing network compose-tuts_defaultNote the main-container gets stopped first, then the dependencies.
Now run
docker-compose up -d -t 0Expected output :
Creating network "compose-tuts_default" with the default driver
Creating service-3 ... done
Creating service-2 ... done
Creating service-4 ... done
Creating main-container ... doneClearly shows main-container created last.
If you need you dependency services to be READY before the main container starts you need to study https://docs.docker.com/compose/startup-order/
volumes are used to mount named volumes.
To make this interesting we will reuse the 4 services from the previous example.
All 4 services will use the named volume: demo-data-volume. It does not exist yet.
Add the following to your docker-compose.yml using
nano docker-compose.ymlversion: "3.7"
services:
alpine:
image: alpine:3.8
command: sleep 600
depends_on:
- service-2
- service-3
- service-4
container_name: main-container
service-2:
image: alpine:3.8
container_name: service-2
command: sleep 600
volumes:
- demo-data-volume:/root/dir1/dir2
service-3:
image: alpine:3.8
container_name: service-3
command: sleep 600
volumes:
- demo-data-volume:/root/dira/dirb
service-4:
image: alpine:3.8
container_name: service-4
command: sleep 600
volumes:
- demo-data-volume:/root/1/2/3/4
volumes:
demo-data-volume:Right at the bottom: that is the top level volumes key ( as named in the Docker documentation ).
Top level keys start at extreme left side of the docker-compose file.
Our top level key defines demo-data-volume - the named volume we want to use.
When we run
docker-compose up -d -t 0demo-data-volume will be created ( if it does not exist ).
Expected output :
Creating network "compose-tuts_default" with the default driver
Creating volume "compose-tuts_demo-data-volume" with default driver
Creating service-2 ... done
Creating service-4 ... done
Creating service-3 ... done
Creating main-container ... doneNote second line: Creating volume "compose-tuts_demo-data-volume" with default driver
Run:
docker volume lsExpected output :
DRIVER VOLUME NAME
local compose-tuts_demo-data-volumeThere is our named volume. Its name is the concatenation of our current directory and the volume name we specified in our docker-compose file.
In our docker-compose file we defined that we want to use this named volume at different mount points in 3 services.
service-2:
volumes:
- demo-data-volume:/root/dir1/dir2
service-3:
volumes:
- demo-data-volume:/root/dira/dirb
service-4:
volumes:
- demo-data-volume:/root/1/2/3/4We are now going to see these 3 services all have access to the same named volume.
Let's enter service-2 and work with our named volume.
docker exec -it service-2 /bin/shExecute commands as shown: ls to show the path exists and echo to create myoutfile. Then exit.
/ # ls /root/dir1
dir2
/ # echo from service 2 > /root/dir1/dir2/myoutfile
/ # exitLet's enter service-3 and investigate our named volume.
docker exec -it service-3 /bin/shEnter commands shown: ls to confirm that our file exists. cat it to show its content.
/ # ls /root/dira/dirb
myoutfile
/ # cat /root/dira/dirb/myoutfile
from service 2
/ # exitNote that with service-3 the volume is mounted at /root/dira/dirb
service-3:
volumes:
- demo-data-volume:/root/dira/dirbNote that with service-4 the volume is mounted at /root/1/2/3/4
service-4:
volumes:
- demo-data-volume:/root/1/2/3/4Enter service-4 to confirm that our file is available there.
docker exec -it service-4 /bin/shExpected output :
/ # cat /root/1/2/3/4/myoutfile
from service 2
/ # exitOur named volume is available for reading and writing inside these 3 containers. It will continue to exist even if we stop these containers.
Other containers, services and swarms are allowed to used this named volume. It is not specifically linked to these 3 services.
During these few minutes you explored probably less than 5% of Docker volume functionality.
There are more than 25 volume plugins listed at https://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins
You can read the official docs at https://docs.docker.com/storage/volumes/ and https://docs.docker.com/storage/
At least now you have actually used named volumes and have some idea of how it works.
Fortunately named volumes are the easiest to understand, most flexible and most frequently used.
restart is applied when starting a service using docker-compose up
It has 4 easy to understand options:
Let's experiment with those:
Add the following to your docker-compose.yml using
nano docker-compose.ymlversion: "3.7"
services:
alpine:
image: alpine:3.8
command: sleep 1Bring up the service:
docker-compose up -d -t 0Check its status:
docker ps -aThe sleep 1 causes the container to exit after just one second.
You will only see one exited container - even if you rerun docker ps -a many times. As expected: restart = no does not restart containers.
Add the following to your docker-compose.yml using
nano docker-compose.ymlversion: "3.7"
services:
alpine:
image: alpine:3.8
command: sleep 1
restart: unless-stoppedPrune previous container - so we have a clean docker ps -a to work from.
docker container prune -f;docker ps -aBring up new container.
docker-compose up -d -t 0Check its status:
docker ps -aThe sleep 1 causes the container to exit after just one second.
If you run this command repeatedly you will see the container continually restarting.
docker container stop your-container-id does not stop it.
docker-compose down removes the container within seconds.
If you use docker-compose up to start up a container, use docker-compose down to take it down.
restart: unless-stopped work as expected.
Add the following to your docker-compose.yml using
nano docker-compose.ymlversion: "3.7"
services:
alpine:
image: alpine:3.8
command: sleep 1
restart: alwaysBring up new container.
docker-compose up -d -t 0Check its status:
docker ps -aThe sleep 1 causes the container to exit after just one second.
If you run this command repeatedly you will see the container continually restarting.
docker container stop your-container-id does not stop it.
docker-compose down removes the container within seconds.If you use docker-compose up to start up a container, use docker-compose down to take it down.
restart: always restarts containers that exit with zero ( success ) exit code.
If you replace command with:
command: sleep 1; exit 1you can test that restart: always restarts containers that exit with non-zero ( failed ) exit code.
You learn new software by reading the docs and then using it.
Reading means zero experience.
Experiments means experience.
So based on what you experienced above, design and execute tests for restart: on-failure
It must test container restart on failure ( obviously )
It must also test if it will restart on successful exits ( exit return code 0 ).
Based on the above texts this should take 5 minutes maximum.
From https://docs.docker.com/compose/compose-file/#init
Run an init inside the container that forwards signals and reaps processes. Set init to true to use the default init.
Having this option set to true is important if you care about having processes inside your container shut down cleanly.
If you run docker container stop the processes inside the container must receive the correct signals forwarded to them. Then the shutdown subroutines for your applications are correctly signaled to shutdown cleanly.
If you have several programs or threads running in your container they must receive the STOP signal correctly managed by some overall manager process: INIT is that process.
You will see the explanation below demonstrated in real life example:
Init first sends SIGTERM ( signal 15 ) to let processes catch that signal to execute their clean shutdown routines. SIGTERM means - you must shut down, but do you clean shutdown routines first.
stop_grace_period = 10 seconds default.stop_grace_period then waits 10 seconds for the container to exit CLEANLY before sending SIGKILL ( signal 9). SIGKILL immediately kills the proces - files not closed properly - no shutdown routines done. SIGKILL means process is running one moment; brains blown out the next.
Only with init: true does this critical important signals propagate properly through your container.
Summary : specify init: true if you care about preventing data corruption.
Important: This option got added in version 3.7 file format. The first line of your docker-compose.yml must read: version: "3.7"
First we will see how no init setting causes STOP to be handled badly.
Then we do it correctly by specifying init: true and observing the difference.
Add the following to your docker-compose.yml using
nano docker-compose.ymlversion: "3.7"
services:
alpine:
image: alpine:3.8Bring up new container ( no init anywhere ).
docker-compose up -d -t 0Check its status and get container id:
docker ps -a
docker exec -it your-container-id /bin/shEnter it and run ps
/ # ps
PID USER TIME COMMAND
1 root 0:00 sleep 600
11 root 0:00 /bin/sh
16 root 0:00 ps
/ # exitNote PID process 1 is not init.
Open a new shell console and run docker events
Back to original shell, if you now run
docker container stop your-container-idyou will see it takes 10 seconds to stop.
IMPORTANT: Look at the console output for docker events
Expected output :
2018-11-12T09:51:46.112995194+02:00 container kill b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=15)
2018-11-12T09:51:56.134707392+02:00 container kill b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=9)
2018-11-12T09:51:56.307015745+02:00 container die b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, exitCode=137, image=alpine:3.8, name=compose-tuts_alpine_1)
2018-11-12T09:51:56.387969293+02:00 network disconnect 9802e2506ea80e7597f0b018450f86fa0c7dcc045a920702cc0e463aacfda84f (container=b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a, name=compose-tuts_default, type=bridge)
2018-11-12T09:51:56.454351535+02:00 container stop b676e3bf51f9f1f99edc531ffa04b0ab164834cb218fe9b78abb65b396782d6a (com.docker.compose.config-hash=c19f3ca3bb22826fee98596d3cb40c4f403f8a51cede518945f5ef37bb989589, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1)First event sends container SIGTERM - signal 15 text right at end of that line.
Docker determines container still does not shut down properly. Container ignores signal 15.
Second line: KILL signal event happens 10 seconds later - signal 9 text right at end of that line.
Docker kills container with signal 9 = KILL.
Summary: no init, no orderly shutdown, KILL had to be used.
Now observe how init handles the stop properly:
Add the following to your docker-compose.yml using
nano docker-compose.ymlversion: "3.7"
services:
alpine:
image: alpine:3.8
command: sleep 600
init: trueBring up new container with init: true
docker-compose up -d -t 0Check its status and get container id:
docker ps -a
docker exec -it your-container-id /bin/shEnter it and run ps
/ # ps
PID USER TIME COMMAND
1 root 0:00 /dev/init -- sleep 600
6 root 0:00 sleep 600
7 root 0:00 /bin/sh
12 root 0:00 ps
/ # exitNote PID 1 runs /dev/init. Init is running PID 1: the only perfect place to manage sending signals to all processes in the container.
Press CTRL-c in the docker events console so that list of events are interrupted.
In this console run docker events again - now we going to observe INIT-managed shutdown events.
Back to original shell, do docker ps -a to get your container id.
If you now run
docker container stop your-container-idyou will see it stops very quickly.
IMPORTANT: Look at the console output for docker events
Expected output :
2018-11-12T09:50:14.892684432+02:00 container kill 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1, signal=15)
2018-11-12T09:50:15.019039627+02:00 container die 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, exitCode=143, image=alpine:3.8, name=compose-tuts_alpine_1)
2018-11-12T09:50:15.104203647+02:00 network disconnect 9802e2506ea80e7597f0b018450f86fa0c7dcc045a920702cc0e463aacfda84f (container=7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509, name=compose-tuts_default, type=bridge)
2018-11-12T09:50:15.149018229+02:00 container stop 7973ccec4849cf8759821d613e1b4870c68d76f396fefcfb1cd1bf1f7a3bd509 (com.docker.compose.config-hash=8d5184879de73d9f624319983822eafca40959cbe9f91929ff4648ace18acc6a, com.docker.compose.container-number=1, com.docker.compose.oneoff=False, com.docker.compose.project=compose-tuts, com.docker.compose.service=alpine, com.docker.compose.version=1.22.0, image=alpine:3.8, name=compose-tuts_alpine_1)First event sends container SIGTERM - signal 15 text right at end of that line.
Docker determines container does shut down properly. Container properly processes signal 15. ( Container processes caught signal 15 and called their clean shutdown routines. )
The container die events happens 100 milliseconds later.
NO signal 9 = KILL sent.
Summary: init, fast orderly shutdown, no KILL used.
In short, I strongly suggesting using init: true. It's easy, fast and saves corrupted data hassles.
Log on to Alibaba Cloud Using Internal Enterprise Accounts with RAM Single Sign-On
2,599 posts | 764 followers
FollowAlibaba Clouder - January 24, 2019
Alibaba Clouder - January 24, 2019
Alibaba Clouder - January 25, 2019
Alibaba Clouder - January 25, 2019
Alibaba Clouder - July 24, 2020
Apache Flink Community China - April 23, 2020
2,599 posts | 764 followers
FollowLearn More
Container Registry
A secure image hosting platform providing containerized image lifecycle management
Learn More
ECS(Elastic Compute Service)
Elastic and secure virtual cloud servers to cater all your cloud hosting needs.
Learn MoreMore Posts by Alibaba Clouder