preface
This is a basic article about image and container, although some of it is related to the article written in 18 years Enter the door of Docker and Kubernetes container world There are overlaps, but with my familiarity with containers in recent years, I would like to share some of my knowledge and serve as a technical foreshadowing for my future articles.
What is a mirror image
Before describing what image is, let's take a look at the following example program, which is a simple python program written based on the flash framework.
# file app.py from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return "Hello, World" if __name__ == "__main__": app.run(host='::', port=9080, threaded=True) # file requirements.txt flask
In order to run this program, we first need an operating system (such as ubuntu), then upload the program to a directory of the host (such as / app), then install python 3 and pip programs, then use pip to install the flash module, and finally use python to run this program. The procedures and commands are as follows:
apt-get update apt-get install python3 python3-pip pip install -r /app/requirements.txt python3 /app/app.py
If another program can only run on a specific version (such as 0.8) of the flash module, then running PIP install flash = 0.8 will conflict with the above installed version of the flash. To solve this problem, we can use the container technology to package the program running environment and the program itself, and the packaged things are called Image images.
In order to make images, we need to choose a tool, such as docker, and in this paper, we choose a tool named podman, whose functions can be described by alias docker=podman. On centos 7.6 or above, execute the following command to install:
yum -y install podman
Generally, we write the process or logic of image creation in a file named Dockerfile. For example programs, we add a Dockerfile in the source directory of the host, which contains the following construction logic:
# 1. Select ubuntu operating system, version is bionic(18.04), we will use apt get to install python and pip later FROM docker.io/library/ubuntu:bionic # 2. Specify the working directory, equivalent to the command: MKDIR / APP & & CD / APP WORKDIR /app # 3. Install python using ubuntu package management software apt get RUN apt-get update && apt-get install -y \ python3 \ python3-pip \ && rm -rf /var/lib/apt/lists/* # 4. Copy the python module dependency file to the working directory and execute pip to install the python module from the Alibaba cloud pypi source COPY requirements.txt ./ ENV INDEX_URL https://mirrors.aliyun.com/pypi/simple RUN pip3 install -i $INDEX_URL --no-cache-dir -r requirements.txt # 5. Copy the main program to the working directory COPY app.py ./ # 6. Specify the command to run the container with this image CMD python3 /app/app.py
Then, we execute the following command to package the application into an image. That is to say, the following command executes the instructions in the Dockerfile file to generate an application image (named Hello flask), which contains the python running environment and source code.
podman build -t hello-flask -f Dockerfile .
At this time, the generated image is saved to our host computer. At this time, it is static, as shown below. The total size of this image is 460MB.
$ podman images hello-flask REPOSITORY TAG IMAGE ID CREATED SIZE localhost/hello-flask latest ffe9ef09e05d 6 minutes ago 460 MB
What is a container
Image packages our program running environment and program itself as a whole, which is static. When we run an instance based on image, the running instance is described as a container.
Because the created image already contains the program runtime environment, for example, the example image contains Python and python flash modules, so when running the container, the host computer where the container is located does not need to prepare the runtime environment for the program, we only need to install a container runtime engine on the host computer to run the container, for example, podman is selected in this paper.
As shown below, we run a container (named hello-1) based on the image Hello flag, which can be accessed through port 9080 of the host.
# Start container $ podman run --rm --name hello-1 -p 9080:9080 hello-flask * Serving Flask app "app" (lazy loading) * Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. * Debug mode: off * Running on http://[::]:9080/ (Press CTRL+C to quit) # Access container $ curl localhost:9080 Hello, World
We can run multiple containers based on the same image, as shown below. Once again, we can run a container (named hello-2) based on the image Hello flag, which can be accessed through port 9081 of the host.
$ podman run --rm --name hello-2 -p 9081:9080 hello-flask
Which containers the host runs can be viewed by the following command:
$ podman ps CONTAINER ID IMAGE ... PORTS NAMES 7687848eb0b5 hello-flask:latest ... 0.0.0.0:9081->9080/tcp hello-2 aab353fb7008 hello-flask:latest ... 0.0.0.0:9080->9080/tcp hello-1
Each container is isolated by Linux Namespace, that is to say, the hello-1 container and the hello-2 container are invisible to each other. As shown below, we execute the following command to log in to the container hello-1, and then execute ps-ef to find that there are only a few commands:
$ podman exec -it hello-1 /bin/sh # ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 09:01 ? 00:00:00 /bin/sh -c python3 /app/app.py root 7 1 0 09:01 ? 00:00:00 python3 /app/app.py root 10 0 40 09:21 pts/0 00:00:00 /bin/sh root 16 10 0 09:21 pts/0 00:00:00 ps -ef
As shown above, we can find that the container does not contain the operating system kernel. Through ps, we can find several commands that the container runs. In the previous chapter, when building the image, I mentioned that in the Dockerfile, FROM ubuntu:bionic The instruction selects the Ubuntu system, which is not very correct, but the correct statement is to select a Ubuntu operating system image without kernel.
Because the container is not a virtual machine, the virtual machine is a complete operating system, but the container does not have an operating system kernel. All containers still share the kernel of the host. We can find the commands executed by the container through ps-ef on the host.
$ ps -ef|grep app.py root 3133 3120 0 17:01 ? 00:00:00 /bin/sh -c python3 /app/app.py root 3146 3133 0 17:01 ? 00:00:00 python3 /app/app.py root 14041 14029 0 17:15 ? 00:00:00 /bin/sh -c python3 /app/app.py root 14057 14041 0 17:15 ? 00:00:00 python3 /app/app.py
Usage of image warehouse
In order to distribute the image, we upload the image to the image warehouse through the network, and then as long as the host can access the image warehouse, it can download the image through the image warehouse and quickly deploy the container, which is similar to github. In github, we store the source code, while the image warehouse stores the image.
When building the image, the Dockerfile contains the following from instruction. We specify to obtain the image from the docker hub. This is a public image made by docker company. From the https://hub.docker.com Come on, we.
FROM docker.io/library/ubuntu:bionic
For enterprises, they usually build their own private image warehouse, such as habor,quay But for personal test purposes, we can build a simple private image warehouse based on the registry image, as follows:
mkdir /app/registry cat > /etc/systemd/system/poc-registry.service <<EOF [Unit] Description=Local Docker Mirror registry cache After=network.target [Service] ExecStartPre=-/usr/bin/podman rm -f %p ExecStart=/usr/bin/podman run --name %p \ -v /app/registry:/var/lib/registry:z \ -e REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry \ -p 5000:5000 registry:2 ExecStop=-/usr/bin/podman stop -t 2 %p Restart=on-failure RestartSec=10 [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable poc-registry systemctl restart poc-registry
Suppose that for the deployed image service, we configure the host name as registry.zyl.io , because it does not use SSL encryption, for the podman container engine, we need to add the following information in the following file, and the HTTPS certificate will not be verified when we visit this image warehouse in the future:
# vi /etc/containers/registries.conf ... [[registry]] location = "registry.zyl.io:5000" insecure = true ...
Next, we push the image to this warehouse, but before that we execute the podman tag image name.
podman tag localhost/hello-flask:latest registry.zyl.io:5000/hello-flask:latest podman push registry.zyl.io:5000/hello-flask:latest
Then, we delete the image first, and then use the pull command to download the image.
podman rmi registry.zyl.io:5000/hello-flask:latest podman pull registry.zyl.io:5000/hello-flask:latest
Structure of image
Refer to official docker documentation About storage drivers It can be seen that the image is stacked by read-only layers, and the upper layer is a reference to the next layer, while the container running on the image layer will generate a read-write Container layer on the image layer, and our writes to the containers all take place on the Container layer, while how the layers interact is driven by different storage drivers).
Generally, each instruction in the Dockerfile will generate a read-only image layer, as shown in the official example, which contains four instructions in total:
FROM ubuntu:15.04 COPY . /app RUN make /app CMD python /app/app.py
The following figure is taken from the official document of docker. It shows that the above dockefile constructs four image layers. From top to bottom, the first layer is generated by cmd instruction, the second layer is generated by run instruction, the third layer is generated by copy instruction, and the fourth layer is generated by from instruction. However, the fourth layer in the following figure is a general summary, which includes all layers of basic image.
Next, we use the command to observe the layers contained in the image ubuntu, which shows five image layers:
$ podman history ubuntu:bionic ID CREATED CREATED BY SIZE ... c3c304cb4f22 7 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B <missing> 7 weeks ago /bin/sh -c mkdir -p /run/systemd && echo '... 161B <missing> 7 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /... 847B <missing> 7 weeks ago /bin/sh -c [ -z "$(apt-get indextargets)" ] 35.37kB <missing> 7 weeks ago /bin/sh -c #(nop) ADD file:c3e6bb316dfa6b8... 26.69MB
The Hello flag image built above is based on ubuntu image, which contains 12 layers in total:
$ podman history hello-flask ID CREATED CREATED BY SIZE # CMD python3 /app/app.py ffe9ef09e05d 2 hours ago /bin/sh -c #(nop) CMD python3 /app/app.py 0B # COPY app.py ./ <missing> 2 hours ago /bin/sh -c #(nop) COPY file:e007c2b54ecd4c... 294B # RUN pip3 install -i $INDEX_URL --no-cache-dir -r requirements.txt <missing> 2 hours ago /bin/sh -c pip3 install -i $INDEX_URL --no... 1.291MB # ENV INDEX_URL https://mirrors.aliyun.com/pypi/simple <missing> 2 hours ago /bin/sh -c #(nop) ENV INDEX_URL https://mi... 1.291MB # COPY requirements.txt ./ <missing> 2 hours ago /bin/sh -c #(nop) COPY file:774347764755ea... 179B # RUN apt-get update && ... <missing> 2 hours ago /bin/sh -c apt-get update && apt-get insta... 165.4MB # WORKDIR /app <missing> 2 hours ago /bin/sh -c #(nop) WORKDIR /app 322B # FROM docker.io/library/ubuntu:bionic <missing> 7 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 322B <missing> 7 weeks ago /bin/sh -c mkdir -p /run/systemd && echo '... 185B <missing> 7 weeks ago /bin/sh -c set -xe && echo '#!/bin/sh' > /... 965B <missing> 7 weeks ago /bin/sh -c [ -z "$(apt-get indextargets)" ] 38.94kB <missing> 7 weeks ago /bin/sh -c #(nop) ADD file:c3e6bb316dfa6b8... 27.76MB
Knowing that images are stacked by read-only layers is very useful for building elegant images, I will use a simple example to explain the reason, but for more details, please refer to the official documents Best practices for writing Dockerfiles.
Consider a scenario where there is a temporary file that we delete after processing to avoid taking up space. If the following example is performed on the operating system, the process and result are in line with our expectation: disk space is freed.
# 1. Generate a 50m file for testing dd if=/dev/zero of=test.txt bs=1M count=50 # 2. Handle temporary files. Here we use ls command ls -lh test.txt -rw-r--r-- 1 root root 50M Jun 12 18:49 test.txt # 3. Delete temporary files to avoid occupying disk space rm -f test.txt
According to the above process, we translate to the Dockerfile intact. For each of the above commands, we put it in a RUN instruction separately:
$ podman build -t test -f - . <<EOF FROM docker.io/library/ubuntu:bionic RUN dd if=/dev/zero of=test.txt bs=1M count=50 RUN ls -lh test.txt RUN rm -f test.txt EOF
We expect the built image to be the same as the basic image ubuntu:bionic The size is almost the same, because we finally deleted the file, but the actual result is far from what we expected, and the resulting image is about 50M larger than the basic image.
$ podman images | grep -w ubuntu docker.io/library/ubuntu bionic ... 66.6 MB $ podman images | grep -w test localhost/test latest ... 119 MB $ podman history localhost/test ID CREATED CREATED BY SIZE 719f3ed7b57c 5 minutes ago /bin/sh -c rm -f test.txt 1.536kB <missing> 5 minutes ago /bin/sh -c ls -lh test.txt 1.024kB # RUN dd if=/dev/zero of= test.txt BS = 1m count = 50 generates a 50m read-only image layer <missing> 5 minutes ago /bin/sh -c dd if=/dev/zero of=test.txt bs=...52.43MB ...
When we know that the image is stacked by read-only layers, then the result is acceptable. For similar problems, we can adjust the image construction logic and put it on the same layer to optimize the image size.
$ podman build -t test -f - . <<EOF FROM docker.io/library/ubuntu:bionic RUN dd if=/dev/zero of=test.txt bs=1M count=50 && \ ls -lh test.txt && \ rm -f test.txt EOF
At this point, we can find that the image size is in line with our expectation.
$ podman images | grep -w test localhost/test latest d57331d89d86 9 seconds ago 66.6 MB $ podman history test ID CREATED CREATED BY SIZE d57331d89d86 20 seconds ago /bin/sh -c dd if=/dev/zero of=test.txt bs=... 167B ...
Mirrored layers are reusable
If we run the same build process repeatedly, we can find that subsequent builds will be much faster than before. In the build output, we can find the hint of using cache... Which indicates that the image generated by the new build reuses the layer of the original image, thus speeding up the build speed, but it will also cause problems.
As shown in the following construction logic, it seems that there is no problem. Before we install curl tool, we perform apt get update to update the system source, but later our construction may reuse run apt get update for caching reasons, which may result in that the curl tool installed later may not be up-to-date, which is different from our expectation.
FROM ubuntu:18.04 RUN apt-get update RUN apt-get install -y curl
Official documents Best practices for writing Dockerfiles It's said that run apt get update & & apt get install - y can ensure that the latest software package is installed, which will lead to clearing or invalidation of this layer's cache. However, the test found that the cache is still reused. The final way to solve this problem may be to pass the -- no cache parameter during the construction, which explicitly tells the construction process not to reuse any cache, but causes the construction time to be too long.
The image is stacked by layers, while the upper layer is a reference to the lower layer, and the construction process can reuse the cache to speed up. Consider the following construction logic. First, copy the source code to the image, and then install Python and python modules.
FROM ubuntu COPY app.py ./ COPY requirements.txt ./ RUN apt-get update && apt-get install -y \ python3 \ python3-pip \ && rm -rf /var/lib/apt/lists/* RUN pip3 install -r requirements.txt CMD python3 /app/app.py
The above build logic will cause such a problem, assuming that we have modified app.py Source code, which will lead to COPY app.py The cache of. / layer is invalid, so this layer needs to be rebuilt, and the failure of the lower layer will cause the failure of all the upper caches that depend on this layer. Therefore, all the following instructions cannot use the cache layer. In view of this, we adjust the construction logic to minimize the cache layer failure caused by code modification.
FROM ubuntu RUN apt-get update && apt-get install -y \ python3 \ python3-pip \ && rm -rf /var/lib/apt/lists/* COPY requirements.txt ./ RUN pip3 install -r requirements.txt COPY app.py ./ CMD python3 /app/app.py
Building images with multiple segments
Before introducing multi-stage image building, let's consider how to build the following example as image, which is a hello world program written in c language:
$ mkdir hello-c && cd hello-c $ cat > hello.c <<EOF #include <stdio.h> int main(void) { printf("hello world\n"); } EOF $ cat > Makefile <<EOF all: gcc --static hello.c -o hello EOF
We need the gcc and make commands to compile this program, so we write the following Dockerfile to build the image:
$ cat > Dockerfile <<'EOF' FROM ubuntu:bionic WORKDIR /app RUN apt-get update && apt-get install -y \ build-essential \ libc-dev \ && rm -rf /var/lib/apt/lists/* COPY Makefile ./ COPY hello.c ./ RUN make all CMD ["./hello"] EOF
After performing podman build -t test -f Dockerfile. To build the image, its final size is nearly 300M.
$ podman images|grep test localhost/test latest ... 281 MB
The application image generated above contains the compiling environment. These tools only work when compiling C programs, but the program operation does not depend on the compiling environment. That is to say, we can remove these compiling environments from the final application image generated. In view of this, we can use the Multi stage construction Build the image.
As shown below, we adjust the construction logic. In a Dockerfile, we nest two from instructions. In the first from block, we install the compilation environment and compile the code because GCC is used --Static static compiler, so the final binary program does not depend on any dynamic library on the host, so we copy it to the final image, which we use a system reserved image name scratch , this image does not exist in any image warehouse, but using this image will tell the build process to generate the smallest image structure.
cat > Dockerfile <<'EOF' FROM ubuntu:bionic AS builder WORKDIR /app COPY files/sources.list /etc/apt/sources.list RUN apt-get update && apt-get install -y \ build-essential \ libc-dev \ && rm -rf /var/lib/apt/lists/* COPY Makefile ./ COPY hello.c ./ RUN make all FROM scratch WORKDIR /app COPY --from=builder /app/hello . CMD ["./hello"] EOF
After executing podman build -t test -f Dockerfile. To build the image, its final size is less than 1M, and the image can be run.
$ podman images|grep test localhost/test latest ... 848 kB $ podman run --rm test hello world
How to debug a Pod or container
Most of our containers are deployed in k8s cluster, so I will show you how to Debugging pod in k8s cluster environment Method.
At present, the common methods of debugging pod are to view its log, log in the container, etc., as follows:
kubectl logs <pod_name> kubectl exec <pod_name> -- /bin/sh
However, as shown in the above section, for the size of the container, many debugging tools are not included in the final image, even / bin/sh, or the container is in an abnormal state. At this time, we cannot log in to the container for debugging.
In this case, in K8S 1.18 cluster, the official built-in debugging function in kubectl tool, we can start a temporary debugging container to attach to the pod to be debugged, but it is currently in alpha state, we need to enable this characteristic . Edit the following file to add -- feature gates = ephemeralcontainers = true at the command, and wait for kubelet to restart Kube API server and Kube scheduler automatically.
- /etc/kubernetes/manifests/kube-apiserver.yaml
- /etc/kubernetes/manifests/kube-scheduler.yaml
For testing purposes, we start a pod with a pause image. Note: Here we specify -- restart=Never to prevent the problematic pod from being automatically restarted.
$ podman images|grep pause k8s.gcr.io/pause 3.2 ... 686 kB $ kubectl run ephemeral-demo --image=k8s.gcr.io/pause:3.2 --restart=Never pod/ephemeral-demo created $ kubectl get pod NAME READY STATUS RESTARTS AGE ephemeral-demo 1/1 Running 0 23s
The pause image, like the image we built above, has no shell, so we can't log in to the container:
$ kubectl exec ephemeral-demo -- /bin/sh ... exec failed: container_linux.go:346: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory" command terminated with exit code 1
Start a debug container and attach it to the pod to be debugged. At this time, we will get a shell shell shell. At this time, we can do the debugging task.
$ kubectl alpha debug -it ephemeral-demo --image=ubuntu:bionic --target=ephemeral-demo Defaulting debug container name to debugger-rzwl2. If you don't see a command prompt, try pressing enter. / #
However, when my environment is running with crio container, the above kubectl alpha debug command cannot start the debug container. Maybe, as the official document shows, the container does not support the -- target parameter, even if you follow the Ephemeral Containers — the future of Kubernetes workload debugging This article shows that you can start a temporary pod, but it is in a separate Pid namespace, which is certainly problematic. Finally, we can try to use kubectl-debug Tools debugging container, this article will not describe.
Note: The --target parameter must be supported by the Container Runtime. When not supported, the Ephemeral Container may not be started, or it may be started with an isolated process namespace.
Conclusion
In this paper, the author introduces some basic knowledge and skills of image construction. We know that image is made of read-only layers, so we can adjust the construction logic to optimize the size of the generated image considering its layer structure when building the image. Similarly, we use multi-stage construction to utilize the capabilities provided by different images and optimize the size of the image.
In this chapter, we build images through Dockerfile, which provides us with enough ability to control all the construction details, but in fact, it is too low-level, and users need to master too much knowledge. For example, for R & D, we don't need them to spend on how to build images. In view of this, is there a friendly enough method to generate images? The answer is yes, such as s2i,cnb These methods will be explained by the author in the following article.