Introduction to Kubernetes using Yunxi database

Posted by phpnewbie25 on Fri, 11 Feb 2022 14:56:58 +0100

Why Kubernetes

Containers are a great way to package and run applications. In a production environment, we need to manage the containers that run our applications and ensure that they don't go down. If one container fails, you need to start another container. If the system handles this behavior, will it be more convenient? This is the value of Kubernetes (hereinafter referred to as k8s), which provides you with a framework that can flexibly run distributed systems. K8s is used to manage containerized workloads and services, facilitating declarative configuration and automation. At the same time, k8s provides you with many functions, such as automatic deployment and rollback, service discovery and load balancing, storage scheduling, self-healing, key and configuration management. Even if there are some complaints about k8s complexity, we still firmly believe that its benefits far outweigh the cost of complexity.

How does Kubernetes maintain application status

Kubernetes was mainly aimed at stateless applications in its early stage (applications that do not manage their own persistent data). This feature was improved until k8s the introduction of persistent volumes and stateful sets. Typically, when the kubernetes container dies, it will be replaced by a new container with a new identity, including a new IP address and host name. However, the StatefulSet function ensures that each Pod has its own stable identity (which can be resolved through DNS), no matter how many times it is restarted. This is very useful for Yunxi database (Yunxi database), because it means that every time we replace or restart the Pod, we don't have to treat it as a new node in the cluster, and we avoid a lot of data replication. This is very important for supporting consensus protocols and distributed transactions of Yunxi database. As a cloud native database, Yunxi database can tolerate the data loss of a single data node. It can detect the missing replica in the cluster and automatically add a new replica. In order to reduce latency, we recommend that you use local disk as storage, although remote storage allows replicas to move without losing data.

Why use Yunxi database on Kubernetes

In most cases, it is more convenient to deploy and maintain Yunxi database on k8s than on physical or virtual machines. This is mainly because Yunxi database is a single executable file, which can provide a general gateway to the database through each node. Each node is fully peer-to-peer. The only difference is which part of the data the node manages. In case of intermittent failure or rolling upgrade, k8s helps the rapid recovery of database cluster. Although k8s brings many conveniences, we still have to weigh the advantages and disadvantages of the flexibility brought by k8s and higher performance through physical or virtual machines. Even though it requires more manual operations to maintain the database on the physical machine or virtual machine, the best performance is better than k8s.

How does Yunxi database run on Kubernetes

1, Create database cluster
1. If you do not use network storage but local storage volume, you need to create three persistent volumes for pod:

$ kubectl apply -f pv0.yaml -f pv1.yaml -f pv2.yaml

Refer to the following persistent volume (pv) yaml files:

apiVersion: v1
kind: PersistentVolume
 name: pv-bini-0
   app: bini
   storage: 10Gi
 volumeMode: Filesystem
 - ReadWriteOnce
 storageClassName: local
 persistentVolumeReclaimPolicy: Retain
   path: /home/inspur/pv/pv0
     - matchExpressions:
       - key:
         operator: In
         values: ['slave1']

2. After preparing Kubernetes environment, you can use yaml file to create StatefulSet to create database cluster:

$ kubectl create -f bini-statefulset.yaml
service/bini-public created
service/bini created
poddisruptionbudget.policy/bini-budget created
statefulset.apps/bini created

3. Since we haven't initialized the cluster yet, the three Pods are in running status:

$ kubectl get pods
bini-0   0/1       Running   0          1m
bini-1   0/1       Running   0          1m
bini-2   0/1       Running   0          1m

4. Check that the local persistent volume declaration and the local persistent volume have been successfully bound:

$ kubectl get persistentvolumeclaims
datadir-bini-0   Bound    pv-bini-1   1Gi        RWO            local          2m
datadir-bini-1   Bound    pv-bini-0   1Gi        RWO            local          2m
datadir-bini-2   Bound    pv-bini-2   1Gi        RWO            local          2m

5. Create yaml file for cluster initialization:

$ kubectl create -f cluster-init.yaml
job.batch/cluster-init created

6. The job used for initialization will soon be in the completed state:

$ kubectl get job cluster-init
cluster-init   1/1           10s        30s

The three pods of the database node will also be in running status:

$ kubectl get pods
NAME                 READY   STATUS      RESTARTS   AGE
cluster-init-bw928   0/1     Completed   0          50s
bini-0               1/1     Running     0          3m23s
bini-1               1/1     Running     0          3m23s
bini-2               1/1     Running     0          3m23s

2, Use the built-in SQL client: 1 Start a temporary interactive pod and start the built-in SQL client:

$ kubectl run bini -it \
--image=bini:release2.0 \
--rm \
--restart=Never \
-- sql \
--insecure \

2. Run some SQL statements:

root@bini-public:26257/defaultdb> create database bank;
root@bini-public:26257/defaultdb> create database bank;
root@bini-public:26257/defaultdb> create table bank.accounts (
name varchar(255),
balance decimal
root@bini-public:26257/defaultdb> insert into bank.accounts values ('Andy',100),('Bob',200),('Chris',300);
root@bini-public:26257/defaultdb> select * from bank.accounts;
 name  | balance  
 Andy  |     100  
 Bob   |     200  
 Chris |     300  
(3 rows)

3. After exiting the SQL shell, this pod will also be deleted

root@bini-public:26257/defaultdb> \q

3, View cluster status through Admin UI

Method 1: set NodePort for Service
Use the NodePort in the Service to map the port of the Admin UI to a port of the local machine and access it in the way of localhost + NodePort.

Method 2: use port forwarding to manually forward the Service in K8s to a port of the local machine

$ kubectl port-forward service/bini-public --address 8080:8080
Forwarding from -> 8080

Management cluster

1, Add node

$ kubectl scale statefulset bini --replicas=4
statefulset.apps/bini scaled

After configuring the new local persistent storage volume, you will see that the fourth pod successfully joins the cluster:

NAME                 READY   STATUS      RESTARTS   AGE
cluster-init-bw928   0/1     Completed   0          1m39s
bini-0               1/1     Running     0          4m12s
bini-1               1/1     Running     0          4m12s
bini-2               1/1     Running     0          4m12s
bini-3               1/1     Running     0          30s

2, Remove node

1. Start a temporary interactive pod and use the bini node status command to obtain the internal ID of the database node:

$ kubectl run bini -it \
--image=bini/release2.0 \
--rm \
--restart=Never \
-- node status \
--insecure \
 id |                   address                   |  build  |         started_at         |         updated_at         | is_available | is_live  
  1 | bini-0.bini.default.svc.cluster.local:26257 | ff04cdd | 2021-02-04 09:34:47.053657 | 2021-02-05 02:45:31.756302 | true         | true    
  2 | bini-2.bini.default.svc.cluster.local:26257 | ff04cdd | 2021-02-04 09:34:47.814316 | 2021-02-05 02:45:32.464036 | true         | true    
  3 | bini-1.bini.default.svc.cluster.local:26257 | ff04cdd | 2021-02-04 09:34:47.077002 | 2021-02-05 02:45:31.756099 | true         | true    
  4 | bini-3.bini.default.svc.cluster.local:26257 | ff04cdd | 2021-02-05 02:01:14.868258 | 2021-02-05 02:45:29.947311 | true         | true    
(4 rows)

2. Note the ID of the node with the highest number in the address (that is, the address containing bini-3 in the previous step), and use the bini node decommission command to disable it:

kubectl run bini -it \
--image=bini:release2.0 \
--rm \
--restart=Never \
-- node decommission <node ID> \
--insecure \

Next, you will see the status of the retired node:

id | is_live | replicas | is_decommissioning | is_draining  
 4 |  true   |       28 |        true        |    false
 After the node stops running completely, you will see the following information:

id | is_live | replicas | is_decommissioning | is_draining  
 4 |  true   |        0 |        true        |    false    
(1 row)
No more data reported on target nodes. Please verify cluster health before removing the nodes.

3. Remove a node from StatefulSet:

$ kubectl scale statefulset bini --replicas=3
statefulset "bini" scaled

3, Rolling upgrade

$ kubectl patch statefulset bini -p '{"spec":{"template":{"spec":{"containers":[{"name":"bini","image":"bini:release2.0"}]}}}}'

4, Delete cluster

Delete all created resources

$ kubectl delete pods,statefulsets,services,persistentvolumeclaims,persistentvolumes,poddisruptionbudget,jobs -l app=bini
pod "bini-0" deleted
pod "bini-1" deleted
pod "bini-2" deleted
pod "bini-3" deleted
service "bini" deleted
service "bini-public" deleted
persistentvolumeclaim "datadir-bini-0" deleted
persistentvolumeclaim "datadir-bini-1" deleted
persistentvolumeclaim "datadir-bini-2" deleted
persistentvolumeclaim "datadir-bini-3" deleted
poddisruptionbudget "bini-budget" deleted
job "cluster-init" deleted

Topics: Database Kubernetes SQL cluster distributed system