Registry Construction Practice

Posted by illuz1on on Tue, 23 Jul 2019 21:17:41 +0200

1. Building Private registry

1. Regisry Mirror Transfer

Test host ip: 192.168.192.225 (Intranet Machine)
With the help of other machines that can access the public network
Docker search registry and docker save-o. / registry. tar
Copy to 192.168.192.225 machine docker load-i registry.tar to transfer the docker image of registry

Label:
[root@node1 cert]# docker tag $tag localhost/registry:latest of rgistry after import
[root@node1 cert]# mkdir -pv /data/registry/{cert,conf,auth}

2. Creating Certificates

Operating on master 1
[root@master1 cert]# vim registry-csr.json

{
  "CN": "registry",
  "hosts": [
      "127.0.0.1",
      "192.168.192.222",
      "192.168.192.223",
      "192.168.192.224",
      "192.168.192.225",
      "192.168.192.226"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "HangZhou",
      "L": "HangZhou",
      "O": "k8s",
      "OU": "FirstOne"
    }
  ]
}

[root@master1 cert]# cfssl gencert -ca=/etc/kubernetes/cert/ca.pem -ca-key=/etc/kubernetes/cert/ca-key.pem -config=/etc/kubernetes/cert/ca-config.json -profile=kubernetes registry-csr.json | cfssljson -bare registry
Copy the registry*.pem certificate to the 192.168.192.225 node/data/registry/cert directory

3. Configuration file

[root@node1 registry]# vim /data/registry/conf/config.yml 
[root@node1 registry]# cat conf/config.yml 
version: 0.1
log:
  level: info
  fromatter: text
  fields:
    service: registry

storage:
  filesystem:
    rootdirectory: /var/lib/registry
    maxthreads: 100

http:
  addr: 0.0.0.0:888
  headers:
    X-Content-Type-Options: [nosniff]
  tls:
    certificate: /cert/registry.pem
    key: /cert/registry-key.pem

health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

Running test

1. Running registry

[root@node1 registry]# docker run -itd -p 888:888 --privileged -v /data/registry/data:/var/lib/registry  -v /data/registry/cert:/cert -v /data/registry/conf/config.yml:/etc/docker/registry/config.yml --name registry localhost/registry:latest

2. ca Certificate Distribution

[root@master1 docker]# ansible all -i /root/udp/hosts.ini -m shell -a "mkdir /etc/docker/certs.d/192.168.192.225:888/ -pv "
[root@master1 docker]# ansible all -i /root/udp/hosts.ini -m copy -a "src=/etc/kubernetes/cert/ca.pem  dest=/etc/docker/certs.d/192.168.192.225:888/ca.crt"

3. The mirror can be pulled out normally on other nodes.

[root@master1 service]# ansible all -i /root/udp/hosts.ini -m shell -a "docker pull 192.168.192.225:888/pause:latest   "

4. Look at the current image s

[root@node1 conf]# curl -k    https://192.168.192.225:888/v2/_catalog
{"repositories":["addon-resizer","kubernetes-dashboard-amd64","metrics-server-amd64","nginx","pause"]}

3. Adding Authentication

1. Modifying configuration files

[root@node1 registry]# htpasswd  -Bbn Firstone Passwd123 &> /data/registry/auth/htpasswd
[root@node1 registry]# cat /data/registry/auth/htpasswd
Firstone:$2y$05$0CnJRBMCTYcaL8WNi/2dj.cT3q/RekI2EVo.UUoEEqPb2B2G3vWm6

[root@node1 registry]# cat conf/config.yml 
version: 0.1
log:
  level: info
  fromatter: text
  fields:
    service: registry

storage:
  filesystem:
    rootdirectory: /var/lib/registry
    maxthreads: 100

auth:
  htpasswd:
    realm: basic-realm
    path: /auth/htpasswd

http:
  addr: 0.0.0.0:888
  headers:
    X-Content-Type-Options: [nosniff]
  tls:
    certificate: /cert/registry.pem
    key: /cert/registry-key.pem

health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

2. Running registry

[root@node1 registry]# docker run -itd -p 888:888 --privileged -v /data/registry/data:/var/lib/registry -v /data/registry/auth:/auth  -v /data/registry/cert:/cert -v /data/registry/conf/config.yml:/etc/docker/registry/config.yml --name registry localhost/registry:latest

3. Landing Test

[root@node1 192.168.192.225:888]# docker login 192.168.192.225:888 
Username: Firstone
Password: 
Login Succeeded

Recording information after successful landing

[root@node1 192.168.192.225:888]# cat  ~/.docker/config.json
{
    "auths": {
        "127.0.0.1:888": {
            "auth": "Rmlyc3RvbmU6UGFzc3dkMTIz"
        },
        "192.168.192.225:888": {
            "auth": "Rmlyc3RvbmU6UGFzc3dkMTIz"
        }
    }
}

4. Upload Mirror Testing

[root@node1 192.168.192.225:888]# docker images
REPOSITORY                                       TAG                 IMAGE ID            CREATED             SIZE
192.168.192.225:888/nginx                        latest              719cd2e3ed04        5 weeks ago         109MB
192.168.192.225:888/kubernetes-dashboard-amd64   v1.10.1             f9aed6605b81        7 months ago        122MB
192.168.192.225:888/addon-resizer                1.8.4               5ec630648120        8 months ago        38.3MB
192.168.192.225:888/metrics-server-amd64         v0.3.1              61a0c90da56e        10 months ago       40.8MB
localhost/registry                               latest              265eba1842c4        2 years ago         37.6MB
192.168.192.225:888/pause                        latest              f9d5de079539        5 years ago         240kB
[root@node1 192.168.192.225:888]# for i in $(docker images |awk '{print $1":"$2}') ;do docker  push  $i ;done

5. Query Mirror

uri Route:v2/<repoName>/manifests/<tagName> hair GET request
[root@node1 192.168.192.225:888]# curl --user Firstone:Passwd123 --cacert /etc/docker/certs.d/192.168.192.225\:888/ca.crt  https://192.168.192.225:888/v2/nginx/tags/list
{"name":"nginx","tags":["latest"]}
[root@node1 192.168.192.225:888]# curl --user Firstone:Passwd123 --cacert /etc/docker/certs.d/192.168.192.225\:888/ca.crt  https://192.168.192.225:888/v2/addon-resizer/tags/list
{"name":"addon-resizer","tags":["1.8.4"]}

More API usage references: https://docs.docker.com/registry/spec/api/

4. daemon.json Configuration Reference

[root@master1 ~]# cat /etc/docker/daemon.json 
{
    "registry-mirrors": ["192.168.192.225:888"],
    "max-concurrent-downloads": 20,
    "live-restore": true,
    "max-concurrent-uploads": 10,
    "debug": true,
    "log-opts": {
      "max-size": "100m",
      "max-file": "5"
    }
}

Principle introduction:

Encrypted Transport: Symmetric Encryption and Asymmetric Encryption // Actually Symmetric Encrypted Transport is used
Symmetric Encryption: Decryption and encryption use the same secret key, which is insecure. Because plaintext transmission is used in the process of negotiating secret keys
Asymmetric Encryption: Private Key Encryption Public Key Decryption or Public Key Encryption Private Key Decryption
Negotiating secret key process: For security, using asymmetric encryption, symmetric encryption algorithm negotiation process is carried out by using the other party's public key encryption and then transmitted to the other party//asymmetric encryption algorithm.
Secure access to public keys: CA has emerged, using certificates issued by digital certificate issuing agencies to ensure the security of asymmetric encryption process itself
1) client - > access server, server returns its certificate to client (certificate includes certificate issuing authority, validity period, public key, certificate holder, signature, etc.)
2)client searches for the built-in trusted certificate publisher CA in the operating system to compare with the issuer CA in the certificate issued by the server to verify whether the certificate is issued by the legitimate authority.
3) If you can't find it, you think it's not feasible. You find the client to take out the public key of the issuer CA from the operating system, and then decrypt the signature in the certificate sent by the server.
Using the same hash algorithm to calculate the hash value of the certificate sent by the server, and comparing the hash value calculated with the signature in the certificate, the result is consistent and legitimate.
4)clent reads the public key in the certificate for subsequent encryption

Question Record:

1. Error reporting when cleaning up previous registry
[root@node1 ~]# docker rm 6f0d1bcd9f87
Error response from daemon: driver "overlay" failed to remove root filesystem for 6f0d1bcd9f87a62f9b991d18d460c215f49633d16559bb07eca2ed3d1c1742fd: remove /var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/merged: device or resource busy
[root@node1 ~]# grep docker /proc/*/mountinfo | grep ec8a0744de1
/proc/20276/mountinfo:125 110 0:37 / /var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/merged rw,relatime shared:60 - overlay overlay rw,lowerdir=/var/lib/docker/overlay/59fce193b8b2ab730f7c4c556d2ac931c1567e772efb72aafcb29716287bffc2/root,upperdir=/var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/upper,workdir=/var/lib/docker/overlay/ec8a0744de13547e690eb421e968c181acf4c043a94b9643a8867e37ec8217a0/work
[root@node1 ~]# ps -ef |grep 20276
root 19972 18147 0 14:22 pts/0 00:00:00 grep --color=auto 20276
ntp 20276 1 0 Jul18 ? 00:00:00 /usr/sbin/ntpd -u ntp:ntp -g
[root@node1 ~]# service ntpd restart
[root@node1 ~]# docker rm 6f0d1bcd9f87
2. Pull mirror error certificate signed by unknown authority
Solution 1: docker. service ExecStart=/usr/bin/dockerd -- address of insecure-registry image
Solution 2: [root@node 1 192.168.192.234:888] LS
/etc/docker/certs.d/192.168.192.234:888/ca.pem
[root@node1 192.168.192.234:888]# mv ca.pem ca.crt
Note: In the installation process, you can only open https

Reference documents:
https://docs.docker.com/registry/deploying/
https://docs.docker.com/registry/configuration/#list-of-configuration-options
https://deepzz.com/post/secure-docker-registry.html
https://blog.51cto.com/11883699/2160032

Topics: Linux Docker Kubernetes JSON Nginx

Programmer Think