The development and operation of Docker container applications are inseparable from reliable image management. Although Docker officially provides a public image warehouse, we deploy the Registry in our private environment in terms of security and efficiency
It is also very necessary. Previously, we introduced Docker private warehouse Registry. Here we introduce the deployment and use of another enterprise Docker image warehouse Harbor. In Kubernetes cluster, we recommend using Harbor warehouse environment.
1, Introduction to Harbor warehouse
During the daily use and management of Docker containers, we gradually found that it is necessary to deploy enterprise private warehouses, which can help you manage some sensitive images of the enterprise. At the same time, due to the download speed of Docker Hub and GFW, we often need to import some images that cannot be downloaded directly into the local private warehouse Harbor is the best choice for deploying enterprise private warehouses. Harbor is an enterprise level Docker registry management project open source by VMware. Harbor mainly provides Dcoker Registry management UI. The functions provided include role-based access control permission management (RBAC), AD/LDAP integration, log audit, management interface, self registration, image replication and Chinese support. Harbor's goal is to help users quickly build an enterprise level Docker registry service. Based on Docker's open source registry, it provides the following additional functions:
-
->Role based access control
-
->Policy based image replication
-
->Mirror vulnerability scanning
-
->Ad / LDAP integration (LDAP/AD support)
-
->Image deletion & garbage collection
-
->Friendly management UI(Graphical user portal)
-
->Audit logging
-
-> RESTful API
-
->Easy deployment
All components of Harbor are deployed in Dcoker, so Harbor can be quickly deployed using Docker Compose. Special note: since Harbor is based on Docker Registry V2, docker must be greater than or equal to 1.10 0 version, Docker Compose must be greater than 1.6 Version 0!
2, Harbor warehouse structure
Each component of harbor is built in the form of Docker container, which can be deployed using Docker Compose. If kubernetes is used in the environment, harbor also provides kubernetes configuration files. Harbor is composed of the following containers: ui(Harbor's core service), log (container running rsyslog for log collection), mysql (database container composed of official mysql image), Nginx (using Nginx as reverse agent), Registry (official Docker registry), adminserver(Harbor's configuration data manager), jobservice(Harbor's task management service) Redis (used to store sessions).
Harbor is an enterprise Registry server used to store and distribute Docker images. The overall architecture is still very clear. The following is an online architecture diagram:
Harbor dependent external components
->Nginx (Proxy proxy layer): nginx front-end Proxy is mainly used to distribute front-end page ui access and image upload and download traffic; Harbor's registry,UI,token and other services uniformly receive requests from browsers and Docker clients through a front-end reverse Proxy, and forward the requests to different back-end services.
->Registry V2: image warehouse, which is responsible for storing image files; The Docker official image warehouse is responsible for storing Docker images and processing docker push/pull commands. Because we need to control users' access, that is, different users have different read and write permissions on Docker image, registry will point to a token service, forcing users to carry a legal token in each docker pull/push request, and registry will decrypt and verify the token through the public key.
->Database (MySQL or Postgresql): provides database services for core services, and is responsible for storing data such as user permissions, audit logs, Docker image grouping information, etc.
Harbor own components
->Core services (admin server): This is the core function of Harbor. It mainly provides the following services:
->UI: provides a graphical interface to help users manage image s on the registry and authorize users.
->Webhook: in order to timely obtain the status changes of the image on the Registry, configure webhook on the Registry and pass the status changes to the UI module.
->Auth service: responsible for issuing tokens to each docker push/pull command according to user permissions If the request initiated by the docker client to the Registry ø ry service does not contain a token, it will be redirected here. After obtaining the token, it will make a request to the Registry again.
->API: provide Harbor RESTful API
->Replication job service: provides image synchronization between multiple Harbor instances.
->log collector: to help monitor Harbor operation, it is responsible for collecting logs of other components for later analysis.
Let's take a closer look at the main components and data flow trend of Harbor:
->Proxy, which is an nginx front-end proxy, is mainly used to distribute front-end page ui access and image upload and download traffic. In the figure above, it is first identified by dark blue;
->The UI provides a web management page, of course, including a front-end page and a back-end API. The underlying layer uses mysql database;
->Registry is the image warehouse, which is responsible for storing image files. After the image is uploaded, notify the ui to create a repository through hook. The above figure is marked by a red line. Of course, the token authentication of registry is also completed through the ui component;
->Adminserver is the configuration management center of the system. It is attached to check the storage consumption. When the ui and jobserver are started, the configuration of adminserver needs to be loaded, which is identified by a gray line;
->Jobdevice is responsible for image replication. It communicates with the registry, pulls the image from one registry, then push es it to another registry, and records the job_log, the above figure is marked by a purple line;
->Log is a log summary component. Logs are summarized together through docker's log driver and identified by light blue lines.
Harbor's misunderstanding
Myth 1: Harbor is responsible for storage container mirroring (if Harbor is an image warehouse, it should be a storage image)
In fact, for image storage, harbor uses the official docker registry service. As for the registry, it can be stored locally or s3. Harbor's function is to provide user permission management, image replication and other functions to improve the efficiency of the registry.
Myth 2: Harbor image replication is storage direct replication (image replication, many people think it should be a direct copy of image layered files)
In fact, Harbor image replication adopts a more general and strategic approach to copy through the docker registry API, which is not easy. This approach shields the cumbersome underlying file operations. It can not only make use of the existing docker registry function without repeating the wheel, but also solve the problems of conflict and consistency.
Harbor deployment
kubernetes is not recommended here because the image warehouse is very important to ensure the simplicity of deployment and maintenance. Therefore, the method of compose is directly used here. There are three official ways to deploy Harbor:
1) Online installation: download the image of Harbor from Docker Hub to install. Since Docker Hub is relatively slow, it is recommended that Docker configure the accelerator.
2) Offline installation: this method should be used when there is no networking with the deployment host. You need to download the offline installation package in advance: Harbor offline installer - Tgz to local
3) OVA installation: this mainly uses vCenter environment
3, Install Harbor
Install system tools plug-ins
yum -y install yum-utils device-mapper-persistent-data lvm2
Install docker compose
curl -L https://github.com/docker/compose/releases/download/1.25.0-rc4/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose ls -l /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose #View version docker-compose -v
Download harbor
wget https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.4.tgz tar zxvf harbor-offline-installer-v1.7.4.tgz cd harbor
Modify profile
vim harbor.cfg hostname=#Modify to your own IP
Start operation
./install.sh #Execute installation script [Step 4]: starting Harbor ... Creating network "harbor_harbor" with the default driver Creating harbor-log ... done Creating registryctl ... done Creating harbor-adminserver ... done Creating registry ... done Creating harbor-db ... done Creating redis ... done Creating harbor-core ... done Creating harbor-portal ... done Creating harbor-jobservice ... done Creating nginx ... done ✔ ----Harbor has been installed and started successfully.---- #After the installation is completed, access the IP or domain name corresponding to the hostname entered in the configuration file #Default user name: admin, password: Harbor12345
harbor start, stop
start-up stop it [root@k8s-node01 harbor]# docker-compose stop Stopping nginx ... done Stopping harbor-portal ... done Stopping harbor-jobservice ... done Stopping harbor-core ... done Stopping redis ... done Stopping harbor-db ... done Stopping harbor-adminserver ... done Stopping registry ... done Stopping registryctl ... done Stopping harbor-log ... done
Using harbor to mirror the warehouse
Add an insecure image warehouse on another docker
[root@k8s-node02 ~]# vim /etc/docker/daemon.json
[root@k8s-node02 ~]# cat /etc/docker/daemon.json
{
"insecure-registries": ["http://192.168.10.101:80"]
#Modify 192.168 10.101 for your own ip
}
Restart docker
systemctl daemon-reload systemctl restart docker
Log in and upload the image
Create a project on the Harbor administration page
[root@k8s-master ~]# docker login -u admin --password Harbor12345 http://192.168.10.101 WARNING! Using --password via the CLI is insecure. Use --password-stdin. WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded #At this time, you have successfully logged in and uploaded the image #Step 1: change the tag Syntax format: docker tag SOURCE_IMAGE[:TAG] 192.168.10.101:80/test-k8s/IMAGE[:TAG] #Where SOURCE_IMAGE[:TAG] is the source image for which the tag needs to be changed #192.168. 10.101:80 is the harbor server address. If no port number is added, the default is https access #test-k8s: the name of the project in the horbor server #IMAGE[:TAG] is the name and tag of the generated image [root@k8s-master ~]# docker tag nginx:1.7.9 192.168.10.101:80/test-k8s/nginx:1.7.9 [root@k8s-master ~]# docker images | grep test-k8s 192.168.10.101:80/test-k8s/nginx 1.7.9 84581e99d807 6 years ago 91.7MB [root@k8s-master ~]# docker push 192.168.10.101:80/test-k8s/nginx:1.7.9 5f70bf18a086: Pushed 4b26ab29a475: Pushed ccb1d68e3fb7: Pushed e387107e2065: Pushed 63bf84221cce: Pushed e02dce553481: Pushed dea2e4984e29: Pushed 1.7.9: digest: sha256:b1f5935eb2e9e2ae89c0b3e2e148c19068d91ca502e857052f14db230443e4c2 size: 3012
After uploading, go to horbor to confirm
At this point, the image has been uploaded
K8s access to private warehouse
By default, you can pull the image of a public warehouse, but if it is a private warehouse, you need to authenticate
Test private warehouse
#Log in to harbor warehouse
[root@k8s-master ~]# docker login -u admin --password Harbor12345 http://192.168.10.101
#After logging in, it will be generated in the current home directory docker/config.json, which is the docker authentication file
[root@k8s-master ~]# cat .docker/config.json
{
"auths": {
"47.100.126.187:80": {
"auth": "YWRtaW46YXB0ZWNoMSE="
}
}
Use this file to create secret
[root@k8s-master ~]# kubectl create secret docker-registry docker-login --docker-username=admin --docker-password=Harbor12345 --docker-server=192.168.10.101:80
#Docker logi is the secret name
#--Docker username = user with habor
#--Docker password = user password for Gabor
#--Docker server = harbor server ip and port
[root@k8s-master ~]# kubectl get secrets | grep docker
docker-login kubernetes.io/dockerconfigjson 1 25m
Test pull image
vim test.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: test
name: test
namespace: default
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: test
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app: test
spec:
containers:
- image: 47.100.126.187:80/test-k8s/nginx:1.7.9 #Private warehouse image
imagePullPolicy: IfNotPresent
name: nginx
imagePullSecrets: #Use secrets
- name: docker-login
#Verify image pull
[root@k8s-master ~]# kubectl apply -f test.yaml
[root@k8s-master ~]# kubectl describe pod test-c8db9dffd-9r9k5
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 8m32s default-scheduler Successfully assigned default/test-c8db9dffd-9r9k5 to minikube
Normal Pulled 8m31s kubelet Container image "192.168.10.101:80/test-k8s/nginx:1.7.9" already present on machine
Normal Created 8m31s kubelet Created container nginx
Normal Started 8m31s kubelet Started container nginx
[root@k8s-master ~]# kubectl get pod | grep test
test-c8db9dffd-9r9k5 1/1 Running 0 9m40s
#At this time, the pod has been running normally