k8s中taints和toleration的作用及其使用方式

1、概述

在pod进行调度的时候，如果使用了节点的亲和性，那么就会在调度的时候，调度到具有某些标签的节点上，这个是从pod的角度来说的，同时呢，也可以从节点node的角度出发来定义什么样的pod可以被调度到这个节点上。

在k8s中，可以通过taints和toleration的一起使用来达到这个目的。

2、在节点上配置taints（污点）

2.1、语法

在节点上设置taints

kubectl taint nodes node1 key1=value1:NoSchedule

移除节点上的taints

kubectl taint nodes node1 key1-

注意：在key的最后面加上“-”

2.2、示例

2.2.1、在节点上设置taints

kubectl taint nodes nccztsjb-node-23 role=master:NoSchedule

如果移除taints使用

kubectl taint nodes nccztsjb-node-23 role-

2.2.2、查看节点上的taints信息

[root@nccztsjb-node-23 ~]# kubectl describe nodes nccztsjb-node-23 | grep Taints
Taints:             role=master:NoSchedule
[root@nccztsjb-node-23 ~]# 

说明节点上已经被配置了这个taints。那么任何的pod默认是不会被调度到这个节点上的。

2.2.3、创建pod（多个副本）

kubectl create deployment nginx-taints --image=172.20.58.152/middleware/nginx:1.21.4 --replicas=5

查看pod的调度情况

[root@nccztsjb-node-23 ~]# kubectl get pod -l app=nginx-taints -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE               NOMINATED NODE   READINESS GATES
nginx-taints-6698889db5-2sngj   1/1     Running   0          7s    172.39.157.221   nccztsjb-node-24   <none>           <none>
nginx-taints-6698889db5-2t4nr   1/1     Running   0          7s    172.39.21.86     nccztsjb-node-25   <none>           <none>
nginx-taints-6698889db5-jjbwn   1/1     Running   0          7s    172.39.21.85     nccztsjb-node-25   <none>           <none>
nginx-taints-6698889db5-ljdlf   1/1     Running   0          7s    172.39.157.223   nccztsjb-node-24   <none>           <none>
nginx-taints-6698889db5-mtds6   1/1     Running   0          7s    172.39.157.222   nccztsjb-node-24   <none>           <none>
[root@nccztsjb-node-23 ~]# 

从查询的结果来看，pod没有被调度到nccztsjb-node-23的节点上。

那要让某些的pod被调度到这个节点上该怎么做呢？

继续往下看toleration的内容。

3、pod上设置toleration（容忍）

所谓的toleration即pod允许、容忍某些节点，可以进行调度。

那，我们看到，在2.2.3的步骤中，pod是不会在节点nccztsjb-node-23上进行发布的，那么要在这个节点上进行调度该如何操作呢？

就是要在pod上定义toleration。

3.1、语法

(1)operator: equal

tolerations:
- key: "key1"
  operator: "Equal"
  value: "value1"
  effect: "NoSchedule"

注意，如果operator如果是Equal，那么就要定义key和value，并且要和taints设置的值是一样的。

（2）operator: exists

tolerations:
- key: "key1"
  operator: "Exists"
  effect: "NoSchedule"

如果operator是exists的，那么就不需要设置value的值。

(3)operator为exists，但是不设置key

tolerations:
- operator: "Exists"
  effect: "NoSchedule"

3.2、示例

(1) operator为euqal的情况

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-taints
  namespace: default
spec:
  progressDeadlineSeconds: 600
  selector:
    matchLabels:
      app: nginx-taints
  replicas: 5
  template:
    metadata:
      labels:
        app: nginx-taints
    spec:
      containers:
      - image: 172.20.58.152/middleware/nginx:1.21.4
        imagePullPolicy: IfNotPresent
        name: nginx
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      tolerations:
      - key: "role"
        operator: "Equal"
        value: "master"
        effect: "NoSchedule"

创建deployment

kubectl replace -f nginx-taints.yaml

[root@nccztsjb-node-23 ~]# kubectl get pod -l app=nginx-taints -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE               NOMINATED NODE   READINESS GATES
nginx-taints-58b6ddd798-6dstj   1/1     Running   0          28s   172.39.21.84     nccztsjb-node-25   <none>           <none>
nginx-taints-58b6ddd798-dm9nn   1/1     Running   0          26s   172.39.157.225   nccztsjb-node-24   <none>           <none>
nginx-taints-58b6ddd798-ptm5m   1/1     Running   0          28s   172.39.157.224   nccztsjb-node-24   <none>           <none>
nginx-taints-58b6ddd798-rxgcg   1/1     Running   0          26s   172.39.21.92     nccztsjb-node-25   <none>           <none>
nginx-taints-58b6ddd798-ztnvb   1/1     Running   0          28s   172.39.209.74    nccztsjb-node-23   <none>           <none>

发现在重新应用了yaml之后，是可以在nccztsjb-node-23节点上进行pod的调度的。

(2)operator为exists的情况

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-taints
  namespace: default
spec:
  progressDeadlineSeconds: 600
  selector:
    matchLabels:
      app: nginx-taints
  replicas: 5
  template:
    metadata:
      labels:
        app: nginx-taints
    spec:
      containers:
      - image: 172.20.58.152/middleware/nginx:1.21.4
        imagePullPolicy: IfNotPresent
        name: nginx
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      tolerations:
      - key: "role"
        operator: "Exists"
        effect: "NoSchedule"

重新应用yaml

[root@nccztsjb-node-23 ~]# kubectl replace -f nginx-taints.yaml 
deployment.apps/nginx-taints replaced
[root@nccztsjb-node-23 ~]# kubectl get pod -l app=nginx-taints -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE               NOMINATED NODE   READINESS GATES
nginx-taints-6cb85bb844-6b8kr   1/1     Running   0          13s   172.39.157.226   nccztsjb-node-24   <none>           <none>
nginx-taints-6cb85bb844-k72xc   1/1     Running   0          13s   172.39.21.93     nccztsjb-node-25   <none>           <none>
nginx-taints-6cb85bb844-ql2zh   1/1     Running   0          11s   172.39.157.227   nccztsjb-node-24   <none>           <none>
nginx-taints-6cb85bb844-s48z8   1/1     Running   0          11s   172.39.21.90     nccztsjb-node-25   <none>           <none>
nginx-taints-6cb85bb844-wwwhb   1/1     Running   0          13s   172.39.209.75    nccztsjb-node-23   <none>           <none>

也一样是可以在节点nccztsjb-node-23上进行调度的。

OK，以上就是在node上设置tanints，默认没有任何的pod会被调度到该节点上，然后通过在pod上设置toleration允许pod被调度到这个节点上。注意：是允许、能够被调度到这个节点上，不是一定、必须调度到这个节点上（比如在有多个节点的情况下，其他的节点没有taints的设置是可以允许被调度上的）

(3)不设置key和value的场景

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-taints
  namespace: default
spec:
  progressDeadlineSeconds: 600
  selector:
    matchLabels:
      app: nginx-taints
  replicas: 5
  template:
    metadata:
      labels:
        app: nginx-taints
    spec:
      containers:
      - image: 172.20.58.152/middleware/nginx:1.21.4
        imagePullPolicy: IfNotPresent
        name: nginx
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      tolerations:
      - operator: "Exists"
        effect: "NoSchedule"

重新应用配置yaml

[root@nccztsjb-node-23 ~]# kubectl apply -f nginx-taints.yaml 
deployment.apps/nginx-taints created

[root@nccztsjb-node-23 ~]# kubectl get pod -l app=nginx-taints -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE               NOMINATED NODE   READINESS GATES
nginx-taints-548544bcdd-2ftwq   1/1     Running   0          10s   172.39.209.76    nccztsjb-node-23   <none>           <none>
nginx-taints-548544bcdd-6wvtp   1/1     Running   0          10s   172.39.157.229   nccztsjb-node-24   <none>           <none>
nginx-taints-548544bcdd-glbkv   1/1     Running   0          10s   172.39.21.89     nccztsjb-node-25   <none>           <none>
nginx-taints-548544bcdd-gn6wp   1/1     Running   0          10s   172.39.21.91     nccztsjb-node-25   <none>           <none>
nginx-taints-548544bcdd-kdkfx   1/1     Running   0          10s   172.39.157.228   nccztsjb-node-24   <none>           <none>
[root@nccztsjb-node-23 ~]# 

可以发现，pod还是依然可以被调度到节点nccztsjb-node-23上的。即如果没有key,value，将operator设置为exists是一样的效果。这个设置可以容忍任何的taints.

3.3、effect可以取哪些值

• NoSchedule
• PreferNoSchedule

• NoExecure

4、核心关注

• taints定义在node上，拒绝pod的运行
• toleration定义在pod上，让pod能够运行在标注了taint的node上。

taints的主要目标：让pod避开那些不合适的node. (不合适的node标注上taints)