Why do we need Docker Compose?

Isn't it annoying to start containers one by one like this? Can you write a script to start the corresponding containers in order of dependency?

----Docker compose it's coming

Docker Compose

brief introduction

Compose project is the official open source project of Docker, which is responsible for the rapid arrangement of Docker container clusters. In terms of function, it is very similar to the Heat in OpenStack.

Its code is currently in https://github.com/docker/compose Open source.

Compose is positioned as "defining and running multi container Docker applications". Its predecessor is the open source project Fig.

Through the introduction in the first part, we know that using a Dockerfile template file can make it convenient for users to define a separate application container. However, in daily work, we often encounter the situation that multiple containers need to cooperate with each other to complete a task. For example, to implement a Web project, in addition to the Web service container itself, you often need to add the back-end database service container, and even include the load balancing container.

Compose just meets this need. It allows users to use a separate docker - compose YML template file (YAML format) to define a set of associated application containers as a project.

There are two important concepts in Compose:

• Service: an application container can actually include several container instances running the same image.
• Project: a complete business unit composed of a group of associated application containers, which is located in docker compose Defined in the YML file.

The default management object of Compose is the project, which can conveniently manage the life cycle of a group of containers in the project through subcommands.

The Compose project is written in Python and calls the API provided by Docker service to manage the container. Therefore, as long as the operating platform supports Docker API, composition can be used for orchestration management.

Installation and uninstallation

1.linux

• The installation on Linux is also very simple. You can download the compiled binary directly from the official GitHub Release. For example, download the corresponding binary package directly on the Linux 64 bit system.

$ sudo curl -L https://github.com/docker/compose/releases/download/1.25.5/docker-compose-`uname -s`-`uname -m` 
# This is where the downloaded compose script files are stored - the reason why they are placed in this directory by default is that this directory is a directory where environment variables have been configured, and the script can be executed in any directory
> /usr/local/bin/docker-compose

The download speed of github above will be very slow, and the domestic download link is given below:

sudo curl -L https://download.fastgit.org/docker/compose/releases/download/1.27.4/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

# Grant executable permissions to script files
$ sudo chmod +x /usr/local/bin/docker-compose

2.macos,window

• Compose can be installed through Python's package management tool pip, can also download compiled binary files directly, and can even run directly in the docker container. Docker Desktop for Mac/Windows comes with docker compose binary files, which can be used directly after installing docker.

3.bash command completion

$ curl -L https://raw.githubusercontent.com/docker/compose/1.25.5/contrib/completion/bash/docker-compose > /etc/bash_completion.d/docker-compose

4. Unloading

• If it is installed in binary package mode, delete the binary file.

$ sudo rm /usr/local/bin/docker-compose

5. The test installation is successful

$ docker-compose --version
 docker-compose version 1.25.5, build 4667896b

docker compose usage

# 1. Related concepts

First, several terms are introduced.

• Service: an application container can actually run multiple instances of the same image.
• Project: a complete business unit composed of a set of associated application containers. ∂ a project can be associated by multiple services (containers), and Compose is managed for the project.

# 2. Scene

The most common project is a web site, which should include web applications and caching.

• springboot application
• mysql service
• redis service
• elasticsearch service
• ...

# 3. Docker compose template

version: "3.0"
services:
  mysqldb:
    image: mysql:5.7.19
    container_name: mysql
    ports:
      - "3306:3306"
    volumes:
      - /root/mysql/conf:/etc/mysql/conf.d
      - /root/mysql/logs:/logs
      - /root/mysql/data:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: root
    networks:
      - ems
    depends_on:
      - redis

  redis:
    image: redis:4.0.14
    container_name: redis
    ports:
      - "6379:6379"
    networks:
      - ems
    volumes:
      - /root/redis/data:/data
    command: redis-server
    
networks:
  ems:

# 4. Run a group of containers through compose doc

[root@centos ~]# docker-compose up     							// The front desk starts a set of services
[root@centos ~]# docker-compose up -d  							// Start a group of services in the background

docker compose usage example demonstration

Let's mainly demonstrate how to start three tomcat services, mapping 80818082 respectively

#Represents the version number of the docker compose project
version: "3.0"

services:
  
  tomcat:
    #Delegate to assign a name to the container, similar to the docker run --name parameter. The default name is recommended
    # The default name will be preceded by the file name of the current deployment project, which can distinguish containers under different projects
    #container_name: tomcat01
    #Represents which container to use, similar to the image name specified by docker run
    image: tomcat:8.0
    #Represents the port mapping in the host and container, similar to the docker run -p parameter
    ports:
      - 8080:8080
    
  tomcat01:
    image: tomcat:8.0
    ports:
      - 8081:8080
  
  tomcat02:
    image: tomcat:8.0
    ports:
      - 8082:8080
  
  redis:
    image: redis:5.0.12
    ports:
      - "8379:8379"
  
  mysql:
    image: mysql:5.6
    ports:
      - "3306:3006"
    #Represents starting the specified environment for the current container, similar to docker run -e MYSQL_ROOT_PASSWORD=root
    environment:
      - "MYSQL_ROOT_ENVIRONMENT=root"
    # Represents the assignment of data volumes to the current container and host, similar to docker run -v
    #Note: docker compose must be created before using the absolute path
    volumes:
      #- /root/msyqldatas:/var/lib/mysql
      - mysqlData:/var/lib/mysql
  
volumes:
  mysqlData #Declare data volume alias

Docker compose template file

Template file is the core of using Compose, and involves many instruction keywords. But don't worry. Most of these instructions have similar meanings to the parameters related to docker run.

The default template file name is docker compose YML in YAML format.

version: "3"

services:
  webapp:
    image: examples/web
    ports:
      - "80:80"
    volumes:
      - "/data"

Note that each service must automatically build and generate an image by specifying an image instruction or a build instruction (Dockerfile required).

If the build instruction is used, the options set in Dockerfile (such as CMD, EXPOSE, VOLUME, ENV, etc.) will be automatically obtained without the need to be in docker compose Duplicate settings in YML.

The following describes the usage of each instruction.

build

Specify the path of the folder where the Dockerfile is located (either absolute or relative to the docker-compose.yml file). Compose will use it to automatically build the image and then use it.

version: '3'
services:

  webapp:
    build: ./dir

You can also use the context command to specify the path of the folder where the Dockerfile is located.

Use the dockerfile directive to specify the dockerfile file name.

Use the arg instruction to specify the variables when building the image.

version: '3'
services:

  webapp:
    build:
      context: ./dir
      dockerfile: Dockerfile-alternate
      args:
      #If the environment variable named buildno is used in Dockerfile, the value can be passed in here
        buildno: 1

Using cache_from specifies the cache from which to build the image

build:
  context: .
  cache_from:
    - alpine:latest
    - corp/web_app:3.14

command

Overwrite the command executed by default after the container is started.

command: echo "hello world"

The CMD command in Dockerfile is overwritten

container_name

Specifies the container name. The project name will be used by default_ Service name_ The format of serial number.

container_name: docker-web-container

Note: after specifying the container name, the service cannot scale because Docker does not allow multiple containers with the same name.

depends_on

Solve the problems of container dependency and startup sequence. In the following example, redis db will be started first, and then web will be started

version: '3'

services:
  web:
    build: .
    depends_on:
      - db
      - redis

  redis:
    image: redis

  db:
    image: postgres

Note: the web service will not wait for redis db to be "fully started" before starting.

env_file

Get the environment variable from the file, which can be a separate file path or list.

If the Compose template file is specified by docker Compose - f file, env_ The path of the variable in file is based on the path of the template file.

If there is a conflict between the variable name and the environment instruction, the latter shall prevail according to the Convention.

env_file: .env

env_file:
  - ./common.env
  - ./apps/web.env
  - /opt/secrets.env

Each line in the environment variable file must conform to the format, and the comment line at the # beginning is supported.

# common.env: Set development environment
PROG_ENV=development

env_file file naming, recommended Env, because Hide files when opening files

environment

Set environment variables. You can use both array and dictionary formats.

A variable with only a given name will automatically obtain the value of the corresponding variable on the host running Compose, which can be used to prevent unnecessary data disclosure.

environment:
  RACK_ENV: development
  SESSION_SECRET:

environment:
  - RACK_ENV=development
  - SESSION_SECRET

If true|false, yes|no and other expressions are used in the variable name or value Boolean Words with meanings should be put in quotation marks to avoid YAML automatically parsing some contents into corresponding Boolean semantics. These specific words include

y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF

expose

Exposed ports, but not mapped to the host, are only accessed by connected services.

Only internal ports can be specified as parameters

expose:
 - "3000"
 - "8000"

Generally, this will be written in Dockerfile instead of docker compose

healthcheck

Check the healthy operation of the container through the command.

healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost"]
  interval: 1m30s
  timeout: 10s
  retries: 3

image

Specify as the mirror name or mirror ID. If the image does not exist locally, Compose will try to pull the image.

image: ubuntu
image: orchardup/postgresql
image: a4bc65fd

labels

Add Docker metadata information to the container. For example, you can add auxiliary description information to a container.

labels:
  com.startupteam.description: "webapp for a startup team"
  com.startupteam.department: "devops department"
  com.startupteam.release: "rc3 for v1.0"

docker insepct can view

networks

Configure the network to which the container is connected.

version: "3"
services:

  some-service:
    networks:
     - some-network
     - other-network

networks:
  some-network:
  other-network:

After the bridge is declared, when the docker compose up instruction is executed, first create the bridge, and then add the containers specified under the bridge to the network

The bridge created in this way can no longer use the container name to access another container, but can only use the ip in the container to complete the access

ports

Expose port information.

Use the host port: container port (HOST:CONTAINER) format, or just specify the port of the container (the host will randomly select the port).

ports:
 - "3000"
 - "8000:8000"
 - "49100:22"
 - "127.0.0.1:8001:8001"

Note: when using the HOST:CONTAINER format to map ports, if the container port you use is less than 60 and is not put in quotation marks, you may get wrong results, because YAML will automatically parse the number format of xx:yy as base 60. To avoid this problem, it is recommended that all numeric strings be in string format with quotation marks.

If you add ip before it, you can limit which ip can access the container

secrets

Store sensitive data, such as mysql service password.

version: "3.1"
services:

mysql:
  image: mysql
  environment:
    MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_root_password
  secrets:
    - db_root_password
    - my_other_secret

secrets:
  my_secret:
    file: ./my_secret.txt
  my_other_secret:
    external: true

sysctls

Configure container kernel parameters.

sysctls:
  net.core.somaxconn: 1024
  net.ipv4.tcp_syncookies: 0

sysctls:
  - net.core.somaxconn=1024
  - net.ipv4.tcp_syncookies=0

ulimits

Specifies the ulimits limit value for the container.

For example, the specified maximum number of processes is 65535, the specified number of file handles is 20000 (soft limit, the application can be modified at any time, and cannot exceed the hard limit) and 40000 (system hard limit, which can only be increased by root user).

  ulimits:
    nproc: 65535
    nofile:
      soft: 20000
      hard: 40000

volumes

The path to which the data volume is mounted is set. It can be set as the host path (HOST:CONTAINER) or data volume name (VOLUME:CONTAINER), and the access mode (HOST:CONTAINER:ro) can be set.

The path in this instruction supports relative paths.

volumes:
 - /var/lib/mysql
 - cache/:/tmp/cache
 - ~/configs:/etc/configs/:ro

If the path is a data volume name, the data volume must be configured in the file.

version: "3"

services:
  my_src:
    image: mysql:8.0
    volumes:
    #The way to use an alias must be declared
      - mysql_data:/var/lib/mysql

#Declare alias data volume
volumes:
  mysql_data:

restart

Specifies that the restart policy after the container exits is always restart. This command is very effective to keep the service running all the time. It is recommended to configure it as always or unless stopped in the production environment.

restart: always

Equivalent to: docker run --restart=always

read_only

Mounting the root file system of the container in read-only mode means that the contents of the container cannot be modified.

read_only: true

working_dir

Specify the working directory in the container.

working_dir: /code

This command is recommended to be written in Dockerfile, not docker compose Written in yaml file

Read variable

The Compose template file supports dynamic reading of the system environment variables of the host and the data in the current directory Variables in the env file.

For example, the following Compose file will read the value of the variable ${MONGO_VERSION} from the environment in which it is running and write it to the executed instruction.

version: "3"
services:

db:
  image: "mongo:${MONGO_VERSION}"

If you execute Mongo_ Version = 3.2 docker compose up will start a mongo:3.2 image container; If you execute Mongo_ Version = 2.8 docker compose up will start a mongo:2.8 image container.

If the current directory exists env file from which variables will be read when the docker compose command is executed.

Create a new in the current directory env file and write the following.

# support # Note No
MONGO_VERSION=3.6

For example, when the mysql container starts, you need to specify the environment variable of password. At this time, the mysql container finds docker compose Specified in YML env file, you will go inside to find the variable value you need

Compose command description

Command object and format

For Compose, the object of most commands can be either the project itself or the service or container in the project. If there is no special description, the command object will be the project, which means that all services in the project will be affected by the command.

Execute docker compose [command] - help or docker compose help [command] to view the usage format of a specific command.

The basic format of docker compose command is

docker-compose [-f=<arg>...] [options] [COMMAND] [ARGS...]

Command options

• -f. -- file file specifies the Compose template file to be used. The default is docker Compose YML, which can be specified multiple times.
• -p. -- project name specifies the project name. By default, the name of the directory will be used as the project name.
• --verbose outputs more debugging information.
• -v. -- version print the version and exit.

Command instructions

up

The format is docker compose up [options] [service...].

• This command is very powerful. It will attempt to automatically complete a series of operations, including building an image, (RE) creating a service, starting a service, and associating a service related container.

• All linked services will be started automatically unless they are already running.

• It can be said that most of the time, you can start a project directly through this command.

• By default, all containers started by docker compose up are in the foreground, and the console will print the output information of all containers at the same time, which is convenient for debugging.

• When you stop the command through Ctrl-C, all containers will stop.

• If docker compose up - D is used, all containers will be started and run in the background. This option is generally recommended for production environments.

• By default, if the service container already exists, docker compose up will try to stop the container and recreate it (keep the volume mounted with volumes from) to ensure that the newly started service matches docker compose Latest contents of YML file

Start the whole project
docker-compose up -d
 Start a single service
docker-compose up -d service name

down

Close the whole project
docker-compose down
 Close a service in the project 
docker-compose down service name

• This command will stop the container started by the up command and remove the network

build

The format is docker compose build [options] [service...].

Build (rebuild) the service container in the project.

Once the service container is built, it will carry a tagName. For example, for a DB container in a web project, it may be web_db.

You can run docker compose build at any time in the project directory to rebuild the service.

Options include:

• --Force RM deletes temporary containers during construction.
• --No cache does not use cache during image building (this will lengthen the building process).
• --Pull always tries to get the image of the updated version through pull.

For the whole project
docker-compose build
 For a service
docker-compose build service name

exec

• Enter the specified container.

docker-compose exec service name bash

images

Lists the images contained in the Compose file.

docker-compose images

ps

Format is [doc comps...].

Lists all containers currently in the project.

Options:

• -q only print the ID information of the container.

docker-compose ps [-q] [service name]

restart

The format is docker compose restart [options] [service...].

Restart the service in the project.

Options:

• -t. -- timeout timeout specifies the timeout to stop the container before restarting (the default is 10 seconds).

For the project or a service in the project
docker-compose restart  [-t 20s ] [service name]

rm

The format is docker compose RM [options] [service...].

Delete all (stopped) service containers. It is recommended to execute the docker compose stop command first to stop the container.

Options:

• -f. -- force to force direct deletion, including non stopped containers. Generally, try not to use this option.
• -v delete the data volume mounted on the container

docker-compose rm [-f] [-v] [service name]

start

The format is docker compose start [service...].

Start an existing service container.

docker-compose start [service name]

stop

The format is docker compose stop [options] [service...].

Stop a container that is already running without deleting it. These containers can be started again through docker compose start.

Options:

• -t. -- timeout timeout the timeout when the container is stopped (the default is 10 seconds).

docker-compose stop [-t 20s] [service name]

top

View the processes running in each service container.

docker-compose top [service name]

unpause

The format is docker compose unpause [service...].

Resume services that are suspended.

docker-compose unpause [service name]

pause

The format is docker compose pause [service...].

Pause a service container.

docker-compose pause [service name]

logs

The format is docker compose logs [options] [service...].

View the output of the service container. By default, docker compose will use different colors to distinguish different service outputs. You can turn off color by -- no color.

This command is very useful when debugging problems.

docker-compose logs [-f] [service name]

-f: Real time display

summary

The docker command operates on the container, so the docker is followed by the container name or id.

The docker compose command operates on the project and the services in the project. Therefore, if you do not add a service name after docker compose, you will operate on the services (containers) in the whole project by default. If you add a service name, you will operate on the specific services (containers).