[Docker learning notes iv] in depth understanding of Docker image principle

Posted by johnseito on Sun, 20 Feb 2022 20:25:44 +0100

The previous Blog described in detail how to quickly follow several middleware or servers and simply test and use them. As of the last article, two blogs were used to complete the form and bottom things, that is, how to play Docker. This Blog will talk about some principle level content, which is convenient for further use. There is a theoretical basis to support it. The learning process is similar to this. First use and then learn the theory to guide, and then use it in depth, layer by layer.

Docker image loading principle

What is mirroring? Image is a lightweight and executable independent software package, which is used to package the software running environment and the software developed based on the running environment. It contains all the contents required to run a software, including code, runtime (the dependency of a program running or being executed), library, environment variables and configuration files. In fact, it can also be understood as a file directory.

UnionFS - federated file system

Docker's image is actually composed of a layer by layer file system. This layer of file system is the UnionFS federated file system. The UnionFS file system (UnionFS) is a layered, lightweight and high-performance file system. It supports the modification of the file system as a layer by layer superposition, At the same time, you can mount different directories into a single virtual file system. The Union file system is the foundation of docker image. Images can be inherited through layering. Based on the basic image, various specific application images can be made

Features: multiple file systems can be loaded at the same time, but from the outside, only one file system can be seen. Joint loading will overlay all layers of file systems, so that the final file system will contain all underlying files and directories

A typical Linux file system consists of bootfs and rootfs:

  • bootfs(boot file system) mainly includes bootloader and kernel. Bootloader is mainly used to boot and load the kernel. When the kernel is loaded into memory, bootfs is umount ed, that is, unloaded
    • When loading bootfs, traditional Linux will first set rootfs to read-only, then change rootfs from read-only to read-write after system self-test, and then we can write and read on rootfs.
    • After the self-test is completed, it will not be changed to docs read-boot after the self-test is completed. Instead, one or more read only rootfs are loaded onto the previous read only rootfs layer by using union mount (a mounting mechanism of UnionFS). After loading so many layers of rootfs, it still looks like a file system. In the Docker system, these read-only rootfs of union mount are called Docker images. However, at this time, each layer of rootfs is read only, and we cannot operate it at this time. When we create a container, that is, instantiate the Docker image, the system will allocate a layer of empty read-write rootfs over one or more layers of read-only rootfs
  • rootfs (root file system) contains standard directories and files such as / dev, / proc, / bin, / etc in a typical linux system. For a simplified OS, rootfs can be very small and only need to include the most basic commands, tools and program libraries. Because the underlying layer directly uses the Host kernel, it only needs to provide rootfs itself. It can be seen that for different linux distributions, bootfs are the same, and rootfs will be different. Therefore, different distributions can share bootfs

Remember my previous description in the first Blog [Docker learning notes I] basic concepts and theoretical basis of Docker , the applications in Docker container run directly in the kernel of the host machine. The container has neither its own kernel nor virtual hardware. It is a kernel level virtualization technology.

Mirror layering mechanism

In the design of Docker image, the concept of layer is introduced, that is, every step of user's image making operation will generate a layer, that is, an incremental Directory:

The benefits of layering are obvious: resource sharing. If multiple images are built from the same base image, the host only needs to save a base image on disk and load a base image in memory to serve all containers. And every layer of the image can be shared.

The unified file system technology can integrate different layers into a file system, providing a unified perspective for these layers, which hides the existence of multiple layers. From the perspective of users, there is only one file system.

For example, if you install a tomcat, you will find that the basic image layer does not need to be downloaded repeatedly if it already exists

[root@192 ~]# docker pull tomcat
Using default tag: latest
latest: Pulling from library/tomcat
0e29546d541c: Already exists 
9b829c73b52b: Already exists 
cb5b7ae36172: Already exists 
6494e4811622: Already exists 
668f6fcc5fa5: Already exists 
dc120c3e0290: Already exists 
8f7c0eebb7b1: Already exists 
77b694f83996: Already exists 
0f611256ec3a: Already exists 
4f25def12f23: Already exists 
Digest: sha256:9dee185c3b161cdfede1f5e35e8b56ebc9de88ed3a79526939701f3537a52324
Status: Downloaded newer image for tomcat:latest
docker.io/library/tomcat:latest

You can view the layered loaded image through the following command:

[root@192 ~]# docker image inspect tomcat
[
    {
        "Id": "sha256:fb5657adc892ed15910445588404c798b57f741e9921ff3c1f1abe01dbb56906",
        "RepoTags": [
            "tomcat:latest"
        ],
        "RepoDigests": [
            "tomcat@sha256:9dee185c3b161cdfede1f5e35e8b56ebc9de88ed3a79526939701f3537a52324"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2021-12-22T17:07:13.333084424Z",
        "Container": "de0900b3a6caf902ccdaa1c7871d244e29978119ad8a1cce799cf47f1717b679",
        "ContainerConfig": {
            "Hostname": "de0900b3a6ca",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/tomcat/bin:/usr/local/openjdk-11/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "JAVA_HOME=/usr/local/openjdk-11",
                "LANG=C.UTF-8",
                "JAVA_VERSION=11.0.13",
                "CATALINA_HOME=/usr/local/tomcat",
                "TOMCAT_NATIVE_LIBDIR=/usr/local/tomcat/native-jni-lib",
                "LD_LIBRARY_PATH=/usr/local/tomcat/native-jni-lib",
                "GPG_KEYS=A9C5DF4D22E99998D9875A5110C01C5A2F6059E7",
                "TOMCAT_MAJOR=10",
                "TOMCAT_VERSION=10.0.14",
                "TOMCAT_SHA512=c2d2ad5ed17f7284e3aac5415774a8ef35434f14dbd9a87bc7230d8bfdbe9aa1258b97a59fa5c4030e4c973e4d93d29d20e40b6254347dbb66fae269ff4a61a5"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"catalina.sh\" \"run\"]"
            ],
            "Image": "sha256:6e2683bf6f13f0050833b6807871b4980142835747139a2c2ae91b274787e399",
            "Volumes": null,
            "WorkingDir": "/usr/local/tomcat",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "DockerVersion": "20.10.7",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8080/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/tomcat/bin:/usr/local/openjdk-11/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "JAVA_HOME=/usr/local/openjdk-11",
                "LANG=C.UTF-8",
                "JAVA_VERSION=11.0.13",
                "CATALINA_HOME=/usr/local/tomcat",
                "TOMCAT_NATIVE_LIBDIR=/usr/local/tomcat/native-jni-lib",
                "LD_LIBRARY_PATH=/usr/local/tomcat/native-jni-lib",
                "GPG_KEYS=A9C5DF4D22E99998D9875A5110C01C5A2F6059E7",
                "TOMCAT_MAJOR=10",
                "TOMCAT_VERSION=10.0.14",
                "TOMCAT_SHA512=c2d2ad5ed17f7284e3aac5415774a8ef35434f14dbd9a87bc7230d8bfdbe9aa1258b97a59fa5c4030e4c973e4d93d29d20e40b6254347dbb66fae269ff4a61a5"
            ],
            "Cmd": [
                "catalina.sh",
                "run"
            ],
            "Image": "sha256:6e2683bf6f13f0050833b6807871b4980142835747139a2c2ae91b274787e399",
            "Volumes": null,
            "WorkingDir": "/usr/local/tomcat",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 679618222,
        "VirtualSize": 679618222,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/2fa9b45db352ce43e33bc21cbcbac9570ed998ba2d0e6f2ea1bd849848aa378e/diff:/var/lib/docker/overlay2/4ed31015b0a7955642669dfa51bcc51d41050d08fe3b27a103d088e366a83c85/diff:/var/lib/docker/overlay2/6ec7f2f6d2420d08adcaa7f8f4c63050bd4621bc7c9a39f1042c0d3c1ecaef03/diff:/var/lib/docker/overlay2/24a7055f343fe15c5fe4957e0266a7cdccaf5e09debe2a13224afd16eb4dacb6/diff:/var/lib/docker/overlay2/3a8a1619c0ca510532d8062be2b40e09f59bbb91ea48704da894006c34680d29/diff:/var/lib/docker/overlay2/c2a07a3ad966e9e6bb504c23e3007a491e82f5561c5538ed1d73278612fe2ca3/diff:/var/lib/docker/overlay2/3cfbce5aac5aa34a20a12b86a0d76a2b757eb39f23e2d27c2ce937e041129f6c/diff:/var/lib/docker/overlay2/cc83174f85ca6519fd5f5b439acb26ea36d45bcff37704e1e4e19ef0d747499e/diff:/var/lib/docker/overlay2/e38e7ef258495ff25d8c928367274a6097f2b950527f03a941f4746debb77215/diff",
                "MergedDir": "/var/lib/docker/overlay2/7f19eb1e3fdaed283c2e2c1a2eb865150de7f8ee50ce64e9ebdf6e595695ad75/merged",
                "UpperDir": "/var/lib/docker/overlay2/7f19eb1e3fdaed283c2e2c1a2eb865150de7f8ee50ce64e9ebdf6e595695ad75/diff",
                "WorkDir": "/var/lib/docker/overlay2/7f19eb1e3fdaed283c2e2c1a2eb865150de7f8ee50ce64e9ebdf6e595695ad75/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:11936051f93baf5a4fb090a8fa0999309b8173556f7826598e235e8a82127bce",
                "sha256:31892cc314cb1993ba1b8eb5f3002c4e9f099a9237af0d03d1893c6fcc559aab",
                "sha256:8bf42db0de72f74f4ef0c1d1743f5d54efc3491ee38f4af6d914a6032148b78e",
                "sha256:26a504e63be4c63395f216d70b1b8af52263a5289908df8e96a0e7c840813adc",
                "sha256:f9e18e59a5651609a1503ac17dcfc05856b5bea21e41595828471f02ad56a225",
                "sha256:832e177bb5008934e2f5ed723247c04e1dd220d59a90ce32000b7c22bd9d9b54",
                "sha256:3bb5258f46d2a511ddca2a4ec8f9091d676a116830a7f336815f02c4b34dbb23",
                "sha256:59c516e5b6fafa2e6b63d76492702371ca008ade6e37d931089fe368385041a0",
                "sha256:bd2befca2f7ef51f03b757caab549cc040a36143f3b7e3dab94fb308322f2953",
                "sha256:3e2ed6847c7a081bd90ab8805efcb39a2933a807627eb7a4016728f881430f5f"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]
[root@192 ~]#

The loading process is as follows:

 "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:11936051f93baf5a4fb090a8fa0999309b8173556f7826598e235e8a82127bce",
                "sha256:31892cc314cb1993ba1b8eb5f3002c4e9f099a9237af0d03d1893c6fcc559aab",
                "sha256:8bf42db0de72f74f4ef0c1d1743f5d54efc3491ee38f4af6d914a6032148b78e",
                "sha256:26a504e63be4c63395f216d70b1b8af52263a5289908df8e96a0e7c840813adc",
                "sha256:f9e18e59a5651609a1503ac17dcfc05856b5bea21e41595828471f02ad56a225",
                "sha256:832e177bb5008934e2f5ed723247c04e1dd220d59a90ce32000b7c22bd9d9b54",
                "sha256:3bb5258f46d2a511ddca2a4ec8f9091d676a116830a7f336815f02c4b34dbb23",
                "sha256:59c516e5b6fafa2e6b63d76492702371ca008ade6e37d931089fe368385041a0",
                "sha256:bd2befca2f7ef51f03b757caab549cc040a36143f3b7e3dab94fb308322f2953",
                "sha256:3e2ed6847c7a081bd90ab8805efcb39a2933a807627eb7a4016728f881430f5f"
            ]
        },

Let's understand the following contradictions:

Why does tomcat need 15M for normal installation and 680M for docker installation

This is a tomcat on my machine

This is tomcat installed by Docker

This is because the docker image principle is layered construction. For example, Tomcat is the top layer of Tomcat. Because Tomcat depends on jdk, the next layer is jdk, the basic image is rootfs (Ubuntu, centos), and the lowest layer is bootfs. Because the Tomcat image construction needs to rely on other files, the docker needs 680MB to install Tomcat. You can see the image content of Tomcat:

Why is a centos image in Docker only 200M, while the iso file of a centos operating system needs several G?

Docker image is built in layers, and each layer can be reused. The bootfs of Linux system are basically the same. Therefore, installing docker image in linux series system will reuse the kernel at the bottom of host Linux. Only rootfs and other image layers need to be downloaded, so it is relatively small.

Container startup process

Docker images are read-only. When the container starts, a new writable layer is loaded to the top of the image. This layer is usually called the container layer, and those below the container layer are called the image layer. If users want to modify an image, they can build an image through a read-write container.

Here, the operation of the Container uses the technology used by all drivers - write time replication (CoW). CoW is copy on write, which means to copy only when writing is needed. This is a modification scenario for existing files. For example, if you start multiple containers based on one image, if you allocate a file system with the same image to each Container, it will occupy a lot of disk space. CoW technology allows all containers to share the file system of the image, and all data is read from the image. Only when the file is to be written, the file to be written is copied from the image to its own file system for modification. Therefore, no matter how many containers share the same image, the write operation is performed on the replica copied from the image to their own file system, and the source file of the image will not be modified. If multiple containers operate on the same file, a replica will be generated in the file system of each Container. What each Container modifies is its own replica, which is isolated from each other, They don't affect each other. Using CoW can effectively improve disk utilization

Commit an image

As mentioned above, when starting the container, we can divide the structure into two layers. Read only is used as the image layer and read write is used as the container layer. If we want to make a new image based on the current image, we actually submit read write as a new layer.

use docker commit The command submission container becomes a new version

docker commit -m=""Submitted description"  -a="author" container id Target image name:[TAG]

Because the original downloaded tomcat cannot be started, we put the tomcat made yesterday tomcat-tml Publish as a new image:

[root@192 ~]# docker ps
CONTAINER ID   IMAGE                 COMMAND             CREATED        STATUS          PORTS                                       NAMES
78664208a245   portainer/portainer   "/portainer"        24 hours ago   Up 2 hours      0.0.0.0:8088->9000/tcp, :::8088->9000/tcp   thirsty_gauss
2f59536a92da   tomcat                "catalina.sh run"   26 hours ago   Up 23 seconds   0.0.0.0:3335->8080/tcp, :::3335->8080/tcp   tomcat-tml
[root@192 ~]# docker commit -m="add webapps" -a="Ethan" 2f59536a92da mytomcat:1.0
sha256:251e3ac9aff1efe0651c0dfb0e60ce097ab0e56e7b813b31291300988b6e4668
[root@192 ~]# docker images
REPOSITORY            TAG       IMAGE ID       CREATED         SIZE
mytomcat              1.0       251e3ac9aff1   5 seconds ago   684MB
nginx                 latest    605c77e624dd   7 weeks ago     141MB
tomcat                latest    fb5657adc892   8 weeks ago     680MB
mysql                 latest    3218b38490ce   2 months ago    516MB
hello-world           latest    feb5d9fea6a5   4 months ago    13.3kB
centos                latest    5d0da3dc9764   5 months ago    231MB
portainer/portainer   latest    580c0e4e98b0   11 months ago   79.1MB
elasticsearch         7.6.2     f29a1ee41030   23 months ago   791MB
elasticsearch         latest    5acf0e8da90b   3 years ago     486MB
[root@192 ~]#

You can also see from the panel that the image is finished:

To sum up

Image is a lightweight and executable independent software package, which is used to package the software running environment and the software developed based on the running environment. It contains all the contents required to run a software, including code, runtime (the dependency of a program running or being executed), library, environment variables and configuration files. In fact, it can also be understood as a file directory. This explains why image-based containers can operate independently and in isolation. The file system organization mode, image layering and container write time replication characteristics of UnionFS can also explain in principle why the container can only occupy a small amount of memory when running and why the startup speed can reach the second level.

Topics: Linux Docker

Hot Topics

©2024 Programmer Think