Kubernetes Binary Wire Network Deployment (Example!!!)

Posted by fotakis on Thu, 23 Jan 2020 18:57:54 +0100

This category

1. Official three deployment options
2.Kubernetes Platform Environment Planning
3. Self-signed SSL certificate
4.Etcd database cluster deployment
5.Node Install Docker
6.Flannel Container Cluster Network Deployment
7. Deploy Master Components
8. Deploy Node Components
9. Deploy a test sample
10. Deploy Web UI (Dashboard)
11. Deploy the DNS Resolution Service within the cluster (CoreDNS)

Official three deployment options

  • minikube

    Minikube is a tool that runs quickly locally - a single Kubernetes, used only by users trying Kubemnetes or day-to-day development.Deployment address: htps://kubernetese io/docs/setup/minikube/

  • kubeadm

    Kubeadm is also a tool that exposes kubeadm init and ukubeadm join for the rapid deployment of the Kubermnetes cluster at htpst/:/ubee/es.cs/do/s/cference/scetup tos/kubedm/kubeadm/

  • Binary Package

    It is recommended that you download and distribute the official version of the binary package, manually deploy each component, and form a Kubermetes cluster.Download address: htpts//github.com/kubemetes/kuberetes/teleases

Environment Planning for Kubernetes Platform

  • Single Master Cluster Architecture Diagram

  • Multi-Master Cluster Architecture Diagram

    Self-signed SSL certificate

assembly Certificates used
etcd capem, server.pem, server-key.pem
flannel ca.pem,server.pem, server-key.pem
kube-apiserver ca.pem. server.pem. server-key.pem
kubelet ca.pem, ca-key.pem
kube-proxy ca.pem, kube-proxy pem, kube-proxy-key.pem
kubectl ca.pem, admin.pem, admin-key.pem

Etcd database cluster deployment

Introduction to etcd

Etcd is an open source project initiated by the CoreOS team in June 2013 with the goal of building a highly available distributed key-value database.Within etcd, raft protocol is used as a consistency algorithm, and etcd is based on Go language.

  • As a service discovery system, etcd has the following characteristics:

    Simple: Easy to install, configure, interact with, and use the HTTP API
    Security: Supports SSL certificate validation
    Fast: According to officially provided benchmark data, a single instance supports 2k+read operations per second
    Reliable: Use raft algorithm for distributed system data availability and consistency

Etcd three pillars

  • A highly consistent, highly available service store directory.
    Ralf-based etcd is naturally such a highly consistent, highly available service storage directory.

  • A mechanism for registering the health status of services and health services.
    Users can register services in etcd and configure key TTL for registered services to keep the service's heart beating at regular intervals for the purpose of monitoring health status.

  • A mechanism for finding and connecting services.
    Services registered under the theme specified by etcd can be found under the corresponding theme.To ensure connectivity, we can deploy a proxy-mode etcd on each service machine, which ensures that services accessing the etcd cluster can connect to each other.

Etcd Deployment

  • View cluster status
/opt/etcd/bin/etcdctl \
--a-file=ca.pem -crt-file=server.pem --key-file= server-key.pem \
--endpoints=*https://192.168.0.x:2379.https://192.168.0.x:2379,https://192.168.0x:2379" \
cluster-health

Node Installation Docker

Instance demo

Environment Deployment

Host Software to be installed
master(192.168.142.129/24) kube-apiserver,kube-controller-manager,kube-scheduler,etcd
node01(192.168.142.130/24) kubelet,kube-proxy,docker,flannel,etcd
node02(192.168.142.131/24) kubelet,kube-proxy,docker ,flannel ,etcd

k8s official website address, Click to get oh!


ETCD binary package address, Click to get oh!



Copy the above package into the k8s directory that will be created below centos7

Operations on the master side

mkdir k8s
cd k8s/
mkdir etcd-cert
mv etcd-cert.sh etcd-cert
  • Edit script to download official cfssl package
vim cfssl.sh

curl -L https:#pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
curl -L https:#pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
curl -L https:#pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo

chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
  • Execute scripts to download official cfssl packages
bash cfssl.sh
cfssl Generate Certificate Tool   
cfssljson generates a certificate by passing in a json file
 cfssl-certinfo View Certificate Information
cd etcd-cert/
  • Define a ca certificate
cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"     
        ]  
      } 
    }         
  }
}
EOF
  • Implement Certificate Signature
cat > ca-csr.json <<EOF 
{   
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}
EOF
  • Production certificate, generate ca-key.pem, ca.pem
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
  • Specify communication validation between three etcd nodes
cat > server-csr.json <<EOF
{
    "CN": "etcd",
    "hosts": [
    "192.168.142.129",
    "192.168.142.130",
    "192.168.142.131"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}
EOF
  • Generate ETCD certificate server-key.pem server.pem
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
  • Unzip ETCD Binary Package
tar zxvf etcd-v3.3.10-linux-amd64.tar.gz
  • Configuration File, Command File, Certificate
mkdir /opt/etcd/{cfg,bin,ssl} -p    
mv etcd-v3.3.10-linux-amd64/etcd etcd-v3.3.10-linux-amd64/etcdctl /opt/etcd/bin/
  • Certificate Copy
cp etcd-cert/*.pem /opt/etcd/ssl/
  • Enter a stuck state and wait for other nodes to join
bash etcd.sh etcd01 192.168.142.129 etcd02=https:#192.168.142.130:2380,etcd03=https:#192.168.142.131:2380
  • Open with another session and you will find that the etcd process is already started
ps -ef | grep etcd
  • Copy Certificate to Other Nodes
scp -r /opt/etcd/ root@192.168.142.130:/opt/
scp -r /opt/etcd/ root@192.168.142.131:/opt/
  • Start script to copy other nodes
scp /usr/lib/systemd/system/etcd.service root@192.168.142.130:/usr/lib/systemd/system/
scp /usr/lib/systemd/system/etcd.service root@192.168.142.131:/usr/lib/systemd/system/

Operation on Node 01

  • Modify etcd file
vim /opt/etcd/cfg/etcd
  • Modify Name and Address
#[Member]
ETCD_NAME="etcd02"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https:#192.168.142.130:2380"
ETCD_LISTEN_CLIENT_URLS="https:#192.168.142.130:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https:#192.168.142.130:2380"
ETCD_ADVERTISE_CLIENT_URLS="https:#192.168.142.130:2379"
ETCD_INITIAL_CLUSTER="etcd01=https:#192.168.142.129:2380,etcd02=https:#192.168.142.130:2380,etcd03=https:#192.168.142.131:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
  • Start Services
systemctl start etcd
systemctl status etcd

Operation on Node 02

  • Modify etcd file
vim /opt/etcd/cfg/etcd
  • Modify Name and Address
#[Member]
ETCD_NAME="etcd03"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https:#192.168.142.131:2380"
ETCD_LISTEN_CLIENT_URLS="https:#192.168.142.131:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https:#192.168.142.131:2380"
ETCD_ADVERTISE_CLIENT_URLS="https:#192.168.142.131:2379"
ETCD_INITIAL_CLUSTER="etcd01=https:#192.168.142.129:2380,etcd02=https:#192.168.142.130:2380,etcd03=https:#192.168.142.131:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
  • Start Services
systemctl start etcd
systemctl status etcd

Check cluster state on master side

/opt/etcd/bin/etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https:#192.168.142.129:2379,https:#192.168.142.130:2379,https:#192.168.142.131:2379" cluster-health
member 3eae9a550e2e3ec is healthy: got healthy result from https:#192.168.142.129:2379
member 26cd4dcf17bc5cbd is healthy: got healthy result from https:#192.168.142.130:2379
member 2fcd2df8a9411750 is healthy: got healthy result from https:#192.168.142.131:2379
cluster is healthy

To be continued

Topics: JSON SSL Kubernetes Database