OpenYurt, Vivian Hu (WasmEdge)
Edge calculation It is a distributed application architecture that places computing resources (such as CPU and storage) close to the data source. In this way, it provides low latency, high performance and strong security for various applications.
However, similar to cloud computing in data center, edge computing also faces challenges such as workload distribution, management and scheduling across heterogeneous devices and networks. In order to meet these challenges in cloud computing, many developers will choose container based solutions. Therefore, developers are now introducing solutions such as Kubernetes (K8s) and containers into edge computing. Unfortunately, neither K8s nor Linux containers are designed for edge computing.
Potential problems with standard K8s on the edge include:
- The computing, storage and network resources of edge devices are limited. Running a complete K8s node on an edge device is often too resource intensive.
- K8s can only manage devices with public IP addresses. However, many edge devices do not expose their public IP.
- Edge devices are often offline, while the standard K8s requires their pod s to be always connected.
- Edge devices are usually connected through non TCP networks and require a special network interface to communicate with their K8s manager.
Therefore, developers have created a special Kubernetes distribution for edge computing. A leading edge optimized K8s release is OpenYurt . OpenYurt was originally created by Alibaba and is now a sandbox project of CNCF. OpenYurt is an open platform that includes the powerful editing capabilities of upstream Kubernetes. Therefore, users can use the same upstream Kubernetes API to deliver, manage, and monitor edge workloads.
Even if you use an image OpenYurt With such an edge optimized K8s distribution, the standard Linux container is still too bulky to run edge workloads on small devices. A typical Linux container image usually takes up space in gigabytes, and it takes a few seconds to start a full CPU. Considering that typical edge applications are usually thin, Linux containers occupy too much storage, CPU and battery resources on resource constrained devices. In addition, Linux container applications are not cross platform and unsafe on edge devices.
For edge containerized applications, compliance OCI standard and compatible k8s with Lightweight web assembly runtime, such as WasmEdge ，Compared to Linux container Is a better alternative. WasmEdge provides a capability based security sandbox, makes few assumptions about the underlying operating system services, and supports at the front end Multiple programming languages . In addition to Linux, macOS and Windows, WasmEdge can even run on real-time operating system (RTOS), like the Linux foundation seL4.
OpenYurt and WasmEdge The team showed how to use OpenYurt to manage WasmEdge applications in an edge cluster. OpenYurt and WasmEdge will finally bring cloud native container applications to edge computing! Let's start!
preparation
OpenYurt uses the centralized Kubernetes control panel to manage multiple edge nodes in the ECS. In addition, it extends the upstream Kubernetes to the edge. In this demonstration, we will set up two machines. One simulated cloud Node is called Master, and the other simulated edge Node is called Node. These two nodes constitute the simplest OpenYurt cluster in which OpenYurt components run.
First, close the swap partition
sudo swapoff -a free -m //verification
Next, configure the file node/etc/hosts of the two nodes
Then, load the kernel module br_netfilter and adjust the kernel parameters.
//load the module sudo modprobe br_netfilter //verify lsmod | grep br_netfilter // create k8s.conf cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sudo sysctl --system
Finally, set the RP filter value and set the file / etc / sysctl d/10-network-security. The values of the two parameters in conf are changed from 2 to 1, and / proc/sys/net/ipv4/ip_forward is set to 1.
sudo vi /etc/sysctl.d/10-network-security.confecho 1 > /proc/sys/net/ipv4/ip_forward sudo sysctl --system
Install container
Use the following command line to install containerd on the edge node.
export VERSION="1.5.7"
echo -e "Version: $VERSION"
echo -e "Installing libseccomp2 ..."
sudo apt install -y libseccomp2
echo -e "Installing wget"
sudo apt install -y wget
wget https://github.com/containerd/containerd/releases/download/v${VERSION}/cri-containerd-cni-${VERSION}-linux-amd64.tar.gz
wget https://github.com/containerd/containerd/releases/download/v${VERSION}/cri-containerd-cni-${VERSION}-linux-amd64.tar.gz.sha256sum sha256sum --check cri-containerd-cni-${VERSION}-linux-amd64.tar.gz.sha256sum
sudo tar --no-overwrite-dir -C / -xzf cri-containerd-cni-${VERSION}-linux-amd64.tar.gz
sudo systemctl daemon-reload
Since the crun project supports WasmEdge by default, we only need to configure containerd for runc. So we need to modify / etc / containerd / config The runc parameter in toml is curr, and add pod_annotation.
sudo mkdir -p /etc/containerd/ sudo bash -c "containerd config default > /etc/containerd/config.toml" wget https://raw.githubusercontent.com/second-state/wasmedge-containers-examples/main/containerd/containerd_config.diff sudo patch -d/ -p0 < containerd_config.diff
After that, restart containerd to make the configuration take effect.
systemctl start containerd
Install WasmEdge
Use the following installation script to install WasmEdge on the edge node.
curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash
Install and build crun
We need to build a crun binary file supporting WasmEdge on edge node. At present, the most direct way is to build crun from the source code. First, make sure you have installed crun dependencies in Ubuntu 20.04. If you are using other versions of Linux, please refer to Official documents.
sudo apt update
sudo apt install -y make git gcc build-essential pkgconf libtool \
libsystemd-dev libprotobuf-c-dev libcap-dev libseccomp-dev libyajl-dev \
go-md2man libtool autoconf python3 automake
Next, configure, build, and install crun binaries with WasmEdge support
sudo make install git clone https://github.com/containers/crun cd crun ./autogen.sh ./configure —with-wasmedge make sudo make install
Install OpenYurt
Now, we can install OpenYurt on the Edge node to control the containerd, crun and wasedge stacks under it.
The easiest way to build an OpenYurt cluster is to use the OpenYurt experience center. You just need to register an account for testing, and then you can get an OpenYurt cluster. Next, you can use the yurtctl join command line to join edge nodes. stay here See more details. At present, the OpenYurt component version provided by the OpenYurt experience center is 0.6.0 (the latest version), and the Kubernetes version is 1.20.11.
Alternatively, you can build OpenYurt clusters from scratch according to the following guidelines.
Set up a K8s cluster
Here, we use yurtctl convert to convert K8s cluster into OpenYurt cluster, so we need to build a K8s cluster first. It should be noted that if you use yurtctl init/join to set up OpenYurt cluster and join nodes, you can skip this step.
$ sudo apt-get update && sudo apt-get install -y ca-certificates curl software-properties-common apt-transport-https // add K8s source $ curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add - $ sudo tee /etc/apt/sources.list.d/kubernetes.list <<EOF $ deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main // install K8s components 1.18.9 $ sudo apt-get update && sudo apt-get install -y kubelet=1.18.9-00 kubeadm=1.18.9-00 kubectl=1.18.9-00 // Initialize the master node $ sudo kubeadm init --pod-network-cidr 172.16.0.0/16 \ --apiserver-advertise-address=192.168.3.167 \ --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers // join the work node $ kubeadm join 192.168.3.167:6443 --token 3zefbt.99e6denc1cxpk9fg \ --discovery-token-ca-cert-hash sha256:8077d4e7dd6eee64a999d56866ae4336073ed5ffc3f23281d757276b08b9b195
Install yurtctl tool
Use the following command line to install yurtctl. The yurtcli CLI tool can install / uninstall OpenYurt and convert a standard Kubernetes cluster to an OpenYurt cluster.
git clone https://github.com/openyurtio/openyurt.git cd openyurt make build WHAT=cmd/yurtctl
Install OpenYurt components
yurtctl convert --deploy-yurttunnel --cloud-nodes oy-master --provider kubeadm\ --yurt-controller-manager-image="openyurt/yurt-controller-manager:v0.5.0"\ --yurt-tunnel-agent-image="openyurt/yurt-tunnel-agent:v0.5.0"\ --yurt-tunnel-server-image="openyurt/yurt-tunnel-server:v0.5.0"\ --node-servant-image="openyurt/node-servant:latest"\ --yurthub-image="openyurt/yurthub:v0.5.0"
Note: it is necessary to change -- yurtctl service image = "openyurt / yurtctl service: v0.5.0" to -- node service image = "openyurt / node service: latest"
Run a simple WebAssembly app
Next, let's run the web assembly program through the Kubernetes cluster as a container in the pod. This section will start with Docker hub pulling the container image based on web assembly. If you want to know how to compile and package a web assembly program and publish it as a container image to the Docker hub, see WasmEdge Book.
Enter the following command from another terminal window of the master node, and then start using the cluster.
root@master:~# kubectl cluster-info Kubernetes master is running at https://192.168.3.167:6443 KubeDNS is running at https://192.168.3.167:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Because kubectl 1.18.9 has no annotations parameter, we need to adjust the command line here. If you are using OpenYurt Experience Center, the default versions are OpenYurt 0.6.0 and Kubernetes 1.20.11, please refer to the Kubernetes section To run the wasm application.
// kubectl 1.18.9
$ sudo kubectl run -it --rm --restart=Never wasi-demo --image=hydai/wasm-wasi-example:with-wasm-annotation --overrides='{"kind":"Pod","metadata":{"annotations":{"module.wasm.image/variant":"compat"}} , "apiVersion":"v1", "spec": {"hostNetwork": true}}' /wasi_example_main.wasm 50000000
// kubectl 1.20.11
$ sudo kubectl run -it --rm --restart=Never wasi-demo --image=hydai/wasm-wasi-example:with-wasm-annotation --annotations="module.wasm.image/variant=compat" --overrides='{"kind":"Pod", "apiVersion":"v1", "spec": {"hostNetwork": true}}' /wasi_example_main.wasm 50000000
The output of the container application is printed to the console. All Kubernetes versions are the same.
Random number: 1123434661 Random bytes: [25, 169, 202, 211, 22, 29, 128, 133, 168, 185, 114, 161, 48, 154, 56, 54, 99, 5, 229, 161, 225, 47, 85, 133, 90, 61, 156, 86, 3, 14, 10, 69, 185, 225, 226, 181, 141, 67, 44, 121, 157, 98, 247, 148, 201, 248, 236, 190, 217, 245, 131, 68, 124, 28, 193, 143, 215, 32, 184, 50, 71, 92, 148, 35, 180, 112, 125, 12, 152, 111, 32, 30, 86, 15, 107, 225, 39, 30, 178, 215, 182, 113, 216, 137, 98, 189, 72, 68, 107, 246, 108, 210, 148, 191, 28, 40, 233, 200, 222, 132, 247, 207, 239, 32, 79, 238, 18, 62, 67, 114, 186, 6, 212, 215, 31, 13, 53, 138, 97, 169, 28, 183, 235, 221, 218, 81, 84, 235] Printed from wasi: This is from a main function This is from a main function The env vars are as follows. The args are as follows. /wasi_example_main.wasm 50000000 File content is This is in a file pod "wasi-demo" deleted
Now you can view the status of the pod from the Kubernetes command line.
crictl ps -a
You can see the events from scheduling to running the web assembly workload in the log.
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID 0c176ed65599a 0423b8eb71e31 8 seconds ago Exited wasi-demo
That's it. next, You can try using OpenYurt to manage your WasmEdge app.
Future expectation
WasmEdge and OpenYurt Together, all the advantages of Kubernetes are introduced into edge computing. The above demo demonstrates a very simple usage scenario, which only touches on the corner of its great potential and prospect. Please follow these two projects on GitHub to get the latest information about our cooperation. At the same time, if you have applications that benefit from this technology path, we are also happy to listen!
- WasmEdge GitHub repo: https://github.com/WasmEdge/WasmEdge
- OpenYurt GitHub repo: https://github.com/openyurtio/openyurt