Deploying K8s cluster using kubeadm on Ubuntu 20.04 LTS

Posted by Kyori on Tue, 22 Feb 2022 18:25:48 +0100

Deploying K8s cluster using kubeadm on Ubuntu 20.04 LTS

1. Environmental Information

  • MacBook Pro (13-inch, 2020, Four Thunderbolt 3 ports)
    macOS Big Sur Version 11.5.2
    2.3 GHz Quad-Core Intel Core i7
    16 GB Memory
  • VirtualBox 6.1.22 r144080
  • Ubuntu 20.04.2 LTS server

Although this experiment was conducted on Mac, it is still theoretically applicable on Windows hosts, mainly using virtual machine software VirtualBox to build Ubuntu 20.04.2 LTS server.

Finally, if you still find network problems, see if there is a conflict between the VirtualBox virtual network and the Pod network. I switched to Bridged Adapter here because of NAT conflict

1.1 Virtual Machine Network Environment

Each virtual machine is configured with two network cards: Bridged Adapter and Host-Only Adapter

1.1.1Bridged Adapter

Access to the external network using dynamic IP

Host NameIP
master192.168.8.156/24 brd scope global dynamic enp0s3
node02192.168.8.157/24 brd scope global dynamic enp0s3

1.1.2 HostOnly

Network Card Configuration for Fixed IP

Host NameIP

1.1.3 Configure Fixed IP

For example, configure node02 node IP:
sudo vim /etc/netplan/00-installer-config.yaml

# This is the network config written by 'subiquity'
      dhcp4: true
      optional: true
  version: 2

sudo netplan apply
ssh connection disconnected after applying network configuration

More introduction to VirtualBox network mode

1.2 System Configuration

Disable swap

sudo swapoff -a
free -h
              total        used        free      shared  buff/cache   available
Mem:          2.4Gi       153Mi       1.8Gi       1.0Mi       416Mi       2.1Gi
Swap:            0B          0B          0B

Persist configuration, comment out swap line
sudo vim /etc/fstab

# /etc/fstab: static file system information.
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/ubuntu-vg/ubuntu-lv during curtin installation
/dev/disk/by-id/dm-uuid-LVM-L3Gvl91St70eyL2oJuu2BRgb2xWwZbjp67q0YZ6Vhef7M1EeUSWxeRocIegvMdz5 / ext4 defaults 0 0
# /boot was on /dev/sda2 during curtin installation
/dev/disk/by-uuid/418da6e3-3c15-49d3-bbc2-4355999de12e /boot ext4 defaults 0 0
# k8s Close swap
#/swap.img	none	swap	sw	0	0

Modify Time Zone

The default time zone after system installation is UTC, which is eight hours behind current time
To use Beijing Time, you need to change the system time zone to CST

$ date
Tue Feb 22 06:57:53 UTC 2022
$ sudo timedatectl set-timezone Asia/Shanghai
$ date
Tue Feb 22 14:58:11 CST 2022

After adjusting the time zone, rsyslog needs to be restarted if you want the system log timestamp to take effect immediately

$ sudo systemctl restart rsyslog

Adjust Kernel Parameters

If the system does not load br_netfilter module, which needs to be loaded first.
By lsmod | grep br_netfilter to confirm that br_has been loaded Netfilter module.
The default is not loaded. You can install bridge-utils before loading through modprobe.

sudo apt-get install -y bridge-utils
sudo modprobe br_netfilter
lsmod | grep br_netfilter
br_netfilter           28672  0
bridge                176128  1 br_netfilter

At this point lsmod can see br_netfilter

On Ubuntu 20.04 Server, this value is 1. If your system is inconsistent, use the following command to modify it:

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
sudo sysctl --system

2. Formal Installation

2.1 Docker

Install Docker

sudo apt install

Start Docker

sudo systemctl start docker

Start Docker

sudo systemctl enable docker

Docker installed this way is still newer

$ docker --version
Docker version 20.10.7, build 20.10.7-0ubuntu5~20.04.2

2.2 kubeadm kubectl kubectl

Since Google Source is not normally accessible in China, Ali Source is used instead.

# Configure Ali Source
sudo apt-get install -y ca-certificates curl software-properties-common apt-transport-https curl
curl -s | sudo apt-key add -
sudo tee /etc/apt/sources.list.d/kubernetes.list <<EOF
deb kubernetes-xenial main

# Install kubeadm kubectl kubectl after refreshing the software list
sudo apt-get update && sudo apt-get install -y kubelet kubeadm kubectl
# Lock Version
sudo apt-mark hold kubelet kubeadm kubectl

3. Building Clusters

3.1 Deploy the control-plane s node

Run etcd, API Server on control-plane node machine

sudo kubeadm init --config kubeadm-config.yaml


- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  - signing
  - authentication
kind: InitConfiguration
  bindPort: 6443
  criSocket: /var/run/dockershim.sock
  imagePullPolicy: IfNotPresent
  name: master
  taints: null
  timeoutForControlPlane: 4m0s
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
    dataDir: /var/lib/etcd
kind: ClusterConfiguration
kubernetesVersion: 1.23.4
  dnsDomain: cluster.local
scheduler: {}
kind: KubeletConfiguration
cgroupDriver: cgroupfs
  • Specify the warehouse imageRepository, where the domestic mirror warehouse imageRepository: registry is used. Cn-hangzhou. Aliyuncs. Com/google_ Containers
  • Specify the IP address advertiseAddress, advertiseAddress:

Default configurations can be printed via kubeadm config print init-defaults

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join --token abcdef.0123456789abcdef \
	--discovery-token-ca-cert-hash sha256:0549bc0df13330f30b3684dc6896c47c2e668e3c4138cbd5f0ecfb909d1c16de

When kubeadm init is finished, it prompts you to do three things:
a. Configuring environment variables
b. Install network plug-ins
c. Join nodes in clusters

a. Configuring environment variables

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

If you are a root user, you can execute the following commands:

export KUBECONFIG=/etc/kubernetes/admin.conf

b. Install network plug-ins

K8S has many network plugins, common ones are flannel, calico, etc. See more Installing Addons
Installing Calico requires only the following command to execute

kubectl apply -f

c. Join nodes in clusters

Not on control-plane s, on worker nodes

sudo kubeadm join --token abcdef.0123456789abcdef \
	--discovery-token-ca-cert-hash sha256:0549bc0df13330f30b3684dc6896c47c2e668e3c4138cbd5f0ecfb909d1c16de

If token expires, you can obtain it by following the command

kubeadm token create --print-join-command

More information on building control-plane nodes

3.2 Deploy worker nodes

Installing Docker, kubelet, kubeadm, kubectl is enough.
Then, execute the kubeadm join to join the cluster

3.3 Acceptance Cluster

Create a simple Nginx pod

kubectl apply -f

View the IP of the pod:-o wide

$ kubectl get pods -o wide
nginx   1/1     Running   0          5m57s   node02   <none>           <none>

Accessing the Nginx service

<!DOCTYPE html>
<title>Welcome to nginx!</title>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href=""></a>.<br/>
Commercial support is available at
<a href=""></a>.</p>

<p><em>Thank you for using nginx.</em></p>

The Nginx Welcome Page represents a successful cluster

4. Reference Articles

  1. Getting started
  2. Installing kubeadm
  3. Creating a cluster with kubeadm
  4. Kubernetes networking model
  5. Diagram K8S (01): Deploy version 1.23 K8S cluster based on Ubuntu 20.04
  6. Kubernetes mirror
  7. k8s - Error Correction Guide
  8. Network Configuration
  9. Introduction to Networking Modes

Topics: Docker Kubernetes Ubuntu Container