Taints exist on node and tolerances exist on pod.

Stain (Taint)

Composition of taint

Using the kubectl taint command, you can set a stain on a Node. After the Node is set with a stain, there is a mutually exclusive relationship between the Node and the Pod. You can make the Node refuse the scheduling execution of the Pod, and even expel the existing Pod of the Node.

The composition of each stain is as follows:

key=value:effect

Each stain has a key and value as the label of the stain, where value can be empty, and effect describes the function of the stain. Currently, taint effect supports the following three options:

• NoSchedule: indicates that k8s the Pod will not be scheduled to the Node with this stain
• PreferNoSchedule: indicates that k8s it will try to avoid scheduling the Pod to the Node with this stain
• NoExecute: indicates that k8s the Pod will not be scheduled to the Node with the stain, and the existing Pod on the Node will be expelled

Setting and removal of stains

View node stains

$ kubectl get nodes node-1 -o yaml
-
spec:
  podCIDR: 10.240.0.0/24
  podCIDRs:
  - 10.240.0.0/24
  taints:
  - effect: NoExecute
    key: department
    value: dev
  - effect: NoSchedule
    key: department
    value: dev

Examples of commands for setting and removing stains using kubectl are as follows:

# Set stain
kubectl taint nodes node1 key1=value1:NoSchedule

# Modify stain
kubectl taint nodes node-1 department=dev:NoSchedule

# Remove stains
kubectl taint nodes node1 key1:NoSchedule-

Tolerances

Node s with stains will generate mutually exclusive relationships between NoSchedule, PreferNoSchedule, NoExecute and Pod according to taint's effect, and Pod will not be scheduled to nodes to a certain extent. However, we can set tolerance on the Pod, which means that the Pod with tolerance can tolerate the existence of stains and can be scheduled to the nodes with stains.

Set tolerance points on the pod by setting the tolerances field in the spec of the Pod:

tolerations:
- key: "key1"
  operator: "Equal"
  value: "value1"
  effect: "NoSchedule"
  tolerationSeconds: 3600
- key: "key1"
  operator: "Equal"
  value: "value1"
  effect: "NoExecute"
- key: "key2"
  operator: "Exists"
  effect: "NoSchedule"

• The key, vaule and effect should be consistent with the taint set on the Node
• If the value of operator is Exists, the value value will be ignored
• Tolerance seconds is used to describe the time that can continue to run on the Pod when the Pod needs to be expelled

Here are two special cases of setting tolerance on Pod:

Example 1: when the key value is not specified, it means that all stains are tolerated

tolerations:
- operator: "Exists"

Example 2: when the effect value is not specified, it means that all stain effects are tolerated:

tolerations:
- key: "key"
  operator: "Exists"

Stain and tolerance examples

Cluster information

# Get cluster node
$ kubectl get nodes
-
NAME             STATUS   ROLES    AGE   VERSION
node-1   Ready    <none>   70d   v1.16.3
node-2   Ready    <none>   70d   v1.16.3
node-3   Ready    <none>   70d   v1.16.3

Stain the node-3 node (key=department, value=ops, effect=NoSchedule)

# Illegal pod scheduling is not allowed. After modifying the stain, illegal pod will not be expelled (key = Department, value=ops, effect=NoSchedule)
$ kubectl taint nodes node-3 department=ops:NoSchedule
-
node/node-3 tainted

Deploy an nginx instance with a daemon set

# Deploy as a DaemonSet
$ kubectl apply -f deploy-nginx.yaml
-
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-1
  namespace: default
  labels:
    web: nginx-1
spec:
  selector:
    matchLabels:
      web: nginx-1
  template:
    metadata:
      labels:
        web: nginx-1
    spec:
      containers:
      - name: nginx-1
        image: nginx:1.17
        ports:
        - containerPort: 80

# View instance
$ kubectl get pod -o wide
-
NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
nginx-1-2x6xs                   1/1     Running   0          3s      10.240.2.36   node-1   <none>           <none>
nginx-1-jvjzg                   1/1     Running   0          3s      10.240.1.41   node-3   <none>           <none>

conclusion: nginx Instance not in node-3 Deployed on the node, the stain configuration takes effect.

Add tolerance parameters and redeploy nginx instances

# First delete the previously deployed nginx image, then increase tolerance and redeploy
$ kubectl apply -f deploy.yaml
-
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-1
  labels:
    web: nginx-1
spec:
  selector:
    matchLabels:
      web: nginx-1
  template:
    metadata:
      labels:
        web: nginx-1
    spec:
      containers:
      - name: nginx-1
        image: nginx:1.17
        ports:
        - containerPort: 80 
      tolerations:
      - key: "department"
        operator: "Equal"
        value: "ops"
        effect: "NoSchedule"

# View instance
$ kubectl get pod -o wide
-
NAME                            READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
nginx-1-gz8rn                   1/1     Running   0          6s      10.240.1.42   node-1   <none>           <none>
nginx-1-j4c5v                   1/1     Running   0          6s      10.240.0.47   node-2   <none>           <none>
nginx-1-nvxbc                   1/1     Running   0          6s      10.240.2.37   node-3   <none>           <none>


conclusion: nginx Instance running on node-3 On the node, the new tolerance configuration takes effect.