k8s-Pod调度
Deployment全自动调度
NodeSelector定向调度
NodeAffinity亲和性
PodAffinity-Pod亲和性与互斥性
污点和容忍度
DaemonSet
Job
CronJob
Deployment升级策略
Deployment回滚
Deployment暂停和恢复
DeamonSet的更新策略
Pod的扩缩容
Deployment全自动调度
声明Deployment后，通过筛选标签对匹配的pod做副本控制。Deployment会创建一个RS和replicas个pod。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3 #副本数
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
删除pod后会自动创建新的。
删除Deployment后会关联删除RS和Pod,无法单独删除RS。
NodeSelector定向调度
从调度策略上来说，这三个pod由系统全自动完成调度，他们各自运行在哪个node节点，完全由master的scheduler经过一系列算法计算得出，用户无法干预调度过程和结果。1个被调度到了node1两个被调度到了node2。
在实际情况下，也可能需要将pod调度到指定节点上，可以通过给node打标签，和pod的nodeselector属性匹配，来达到定向调度的目的。
给node添加标签：kubectl label nodes nodename key=value
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
nodeSelector:
zone: north
pod全部调度到了node01上。注意如果pod使用了nodeselector但是没有匹配的node，则pod不会被创建。
删除node标签：kubectl label nodes nodename key-
创建deployment后发现pod一直在被创建中。
通过查看其中一个pod发现错误信息
给node添加上对应标签后，pod又自动创建了。
在pod创建后，删除node标签，pod正常运行。
但是如果pod被删除后，RS重新创建pod会失败。
如果同时给多个node打上匹配的标签，则也会调度到不同的pod上。
NodeAffinity亲和性
requiredDuringSchedulingIgnoredDuringExecution
表示pod必须部署到满足条件的节点上，如果没有满足条件的节点，就不停重试。其中IgnoreDuringExecution表示pod部署之后运行的时候，如果节点标签发生了变化，不再满足pod指定的条件，pod也会继续运行。
kubectl label node k8s-node02 disktype=ssd
kubectl get node --show-labels
在k8s02上打上disktype=ssd标签，然后pod选择策略指定label中包含 disktype=ssd,三个节点都调度到了k8s-node02
apiVersion: apps/v1
kind: Deployment
metadata:
name: d3
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In # In: label的值在某个列表中 NotIn：label的值不在某个列表中 Exists：某个label存在 DoesNotExist：某个label不存在 Gt：label的值大于某个值（字符串比较） Lt：label的值小于某个值（字符串比较）
values:
- ssd
如果operator修改为 NotIn则3个pod都会调度到k8s-node01上。
preferredDuringSchedulingIgnoredDuringExecution
表示优先部署到满足条件的节点上，如果没有满足条件的节点，就忽略这些条件，按照正常逻辑部署。
删除掉node02上的disktype标签后，没有满足affiinity的但是还是被调度到了。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d4
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: disktype
operator: NotIn # In: label的值在某个列表中 NotIn：label的值不在某个列表中 Exists：某个label存在 DoesNotExist：某个label不存在 Gt：label的值大于某个值（字符串比较） Lt：label的值小于某个值（字符串比较）
values:
- ssd
权重的值在1-100之间，看这个结果，越大权重越大。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d5
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: disktype
operator: In # In: label的值在某个列表中 NotIn：label的值不在某个列表中 Exists：某个label存在 DoesNotExist：某个label不存在 Gt：label的值大于某个值（字符串比较） Lt：label的值小于某个值（字符串比较）
values:
- ssd1
- weight: 100
preference:
matchExpressions:
- key: disktype
operator: In # In: label的值在某个列表中 NotIn：label的值不在某个列表中 Exists：某个label存在 DoesNotExist：某个label不存在 Gt：label的值大于某个值（字符串比较） Lt：label的值小于某个值（字符串比较）
values:
- ssd2
如果你同时指定了 nodeSelector 和 nodeAffinity，两者 必须都要满足， 才能将 Pod 调度到候选节点上。
如果你指定了多个与 nodeAffinity 类型关联的 nodeSelectorTerms， 只要其中一个 nodeSelectorTerms 满足的话，Pod 就可以被调度到节点上。
如果你指定了多个与同一 nodeSelectorTerms 关联的 matchExpressions， 则只有当所有 matchExpressions 都满足时 Pod 才可以被调度到节点上。
PodAffinity-Pod亲和性与互斥性
创建一个测试节点，一个pod被分到了node01。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d6
spec:
selector:
matchLabels:
security: s1 #Deployment会控制label相同的pod
replicas: 1
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
security: s1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
podAffinity
亲和性，会尽力与具有指定标签的pod调度到同一个node上。下边例子，app:v1与security：s1亲和
apiVersion: apps/v1
kind: Deployment
metadata:
name: d7
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: security
operator: In # In: label的值在某个列表中 NotIn：label的值不在某个列表中 Exists：某个label存在 DoesNotExist：某个label不存在 Gt：label的值大于某个值（字符串比较） Lt：label的值小于某个值（字符串比较）
values:
- s1
topologyKey: kubernetes.io/hostname
podAntiAffinity
互斥性，对具有指定标签的pod有互斥性，拒绝放到一个node上。下边例子，app:v2与security：s1互斥
apiVersion: apps/v1
kind: Deployment
metadata:
name: d8
spec:
selector:
matchLabels:
app: v2 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v2
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: security
operator: In # In: label的值在某个列表中 NotIn：label的值不在某个列表中 Exists：某个label存在 DoesNotExist：某个label不存在 Gt：label的值大于某个值（字符串比较） Lt：label的值小于某个值（字符串比较）
values:
- s1
topologyKey: kubernetes.io/hostname
污点和容忍度
给node01添加污点 taint0=taint00:NoSchedule 给node02添加污点 taint2-taint22:NoSchedule
kubectl taint nodes k8s-node01 taint0=taint00:NoSchedule
kubectl taint nodes k8s-node02 taint2=taint22:NoSchedule
[root@k8s-master01 ~]# kubectl describe node k8s-node01
Name: k8s-node01
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
disktype=ssd1
kubernetes.io/arch=amd64
kubernetes.io/hostname=k8s-node01
kubernetes.io/os=linux
Annotations: flannel.alpha.coreos.com/backend-data: {"VtepMAC":"16:30:be:e9:46:bb"}
flannel.alpha.coreos.com/backend-type: vxlan
flannel.alpha.coreos.com/kube-subnet-manager: true
flannel.alpha.coreos.com/public-ip: 192.168.180.130
kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Sat, 20 Aug 2022 22:07:32 +0800
Taints: taint0=taint00:NoSchedule
Unschedulable: false
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
NetworkUnavailable False Thu, 25 Aug 2022 15:38:42 +0800 Thu, 25 Aug 2022 15:38:42 +0800 FlannelIsUp Flannel is running on this node
MemoryPressure False Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:32 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:32 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:32 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:42 +0800 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.180.130
Hostname: k8s-node01
Capacity:
cpu: 1
ephemeral-storage: 17394Mi
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 995676Ki
pods: 110
Allocatable:
cpu: 1
ephemeral-storage: 16415037823
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 893276Ki
pods: 110
System Info:
Machine ID: f914368afc1643a4a6fda29f9b7d4da1
System UUID: ABE14D56-5112-0934-DFE1-70325811365B
Boot ID: c064f3ee-6080-40a7-8273-ee9b605a0adb
Kernel Version: 3.10.0-1160.el7.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://20.10.17
Kubelet Version: v1.15.1
Kube-Proxy Version: v1.15.1
PodCIDR: 10.244.1.0/24
Non-terminated Pods: (2 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system kube-flannel-ds-amd64-jsn6j 100m (10%) 100m (10%) 50Mi (5%) 50Mi (5%) 5d12h
kube-system kube-proxy-l89l8 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5d12h
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 100m (10%) 100m (10%)
memory 50Mi (5%) 50Mi (5%)
ephemeral-storage 0 (0%) 0 (0%)
Events: <none>
[root@k8s-master01 ~]# kubectl describe node k8s-node02
Name: k8s-node02
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
disktype=ssd2
kubernetes.io/arch=amd64
kubernetes.io/hostname=k8s-node02
kubernetes.io/os=linux
Annotations: flannel.alpha.coreos.com/backend-data: {"VtepMAC":"b2:58:6a:57:86:15"}
flannel.alpha.coreos.com/backend-type: vxlan
flannel.alpha.coreos.com/kube-subnet-manager: true
flannel.alpha.coreos.com/public-ip: 192.168.180.131
kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Wed, 24 Aug 2022 20:50:34 +0800
Taints: taint2=taint22:NoSchedule
Unschedulable: false
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
NetworkUnavailable False Thu, 25 Aug 2022 15:38:43 +0800 Thu, 25 Aug 2022 15:38:43 +0800 FlannelIsUp Flannel is running on this node
MemoryPressure False Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:34 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:34 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:34 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:44 +0800 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.180.131
Hostname: k8s-node02
Capacity:
cpu: 1
ephemeral-storage: 17394Mi
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 995676Ki
pods: 110
Allocatable:
cpu: 1
ephemeral-storage: 16415037823
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 893276Ki
pods: 110
System Info:
Machine ID: f914368afc1643a4a6fda29f9b7d4da1
System UUID: CBA34D56-4220-5A92-0FEB-563F66061138
Boot ID: 376e5260-63e9-46e2-bf34-2991cecc5526
Kernel Version: 3.10.0-1160.el7.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://20.10.17
Kubelet Version: v1.15.1
Kube-Proxy Version: v1.15.1
PodCIDR: 10.244.2.0/24
Non-terminated Pods: (2 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system kube-flannel-ds-amd64-8k8ww 100m (10%) 100m (10%) 50Mi (5%) 50Mi (5%) 38h
kube-system kube-proxy-t825f 0 (0%) 0 (0%) 0 (0%) 0 (0%) 38h
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 100m (10%) 100m (10%)
memory 50Mi (5%) 50Mi (5%)
ephemeral-storage 0 (0%) 0 (0%)
Events: <none>
启动一个普通的pod，发现这个pod一直是Pending状态，查看事件 nodes are available: 3 node(s) had taints that the pod didn't tolerate.两个node节点都有污染标记，都不能被pod容忍。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d6
spec:
selector:
matchLabels:
security: s1 #Deployment会控制label相同的pod
replicas: 1
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
security: s1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
[root@k8s-master01 home]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
d6-5cfddfb4fd-ffblh 0/1 Pending 0 2m <none> <none> <none> <none>
[root@k8s-master01 home]# kubectl describe pod d6-5cfddfb4fd-ffblh
Name: d6-5cfddfb4fd-ffblh
Namespace: default
Priority: 0
Node: <none>
Labels: pod-template-hash=5cfddfb4fd
security=s1
Annotations: <none>
Status: Pending
IP:
Controlled By: ReplicaSet/d6-5cfddfb4fd
Containers:
myapp01:
Image: 192.168.180.129:9999/myharbor/myapp:v1
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-6wl7b (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
default-token-6wl7b:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-6wl7b
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 76s (x2 over 2m21s) default-scheduler 0/3 nodes are available: 3 node(s) had taints that the pod didn't tolerate.
创建一个可以容忍taint0=taint00:NoSchedule的pod，node01上有这个污点，所以pod都调度到了node01
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint0
operator: Equal
value: taint00
effect: NoSchedule
operator:Equal
那么key和value都需要匹配node的taint，下面这个例子，value没有匹配则调度失败。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint0
operator: Equal
value: taint001
effect: NoSchedule
operator:Exists
则表示不匹配value。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint0
operator: Exists
effect: NoSchedule
删除node污点
kubectl taint nodes k8s-node01 taint0=taint00:NoSchedule-
kubectl taint nodes k8s-node02 taint2=taint22:NoSchedule-
PreferNoSchedule
更改污点类型为node污点类型，pod的容忍类型也修改为PreferNoSchedule虽然没有一个node可以匹配，但是还是可以成功调度。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint33
operator: Exists
effect: PreferNoSchedule
NoExecute
驱逐节点，看下面的情况，先创建三个pod，两个被分配到了node01。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment会控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通过这个模板创建pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在设置pod名称了，多个副本的情况下不能重名，会由自动生成
spec:
#restartPolicy: Always #deployment需要控制副本数量，所以重启策略必须是Always，默认也是Always，所以可以不写。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint33
operator: Exists
effect: NoExecute
tolerationSeconds: 60
现在两个node都没有设置任何的污点，然后将node01的污点类型修改为NoExecute，发现三个pod都被驱逐出了node01，调度到了node02下，因为node02没有任何的污点。tolerationSeconds是在node01上还能待的时间单位秒。
DaemonSet
DaemonSet确保全部（或者某些）节点上运行一个 Pod 的副本。 当有节点加入集群时， 也会为他们新增一个 Pod 。 当有节点从集群移除时，这些 Pod 也会被回收。删除 DaemonSet 将会删除它创建的所有 Pod。
DaemonSet支持NodeSelector,NodeAffinity来指定满足条件的Node范围进行调度，也支持Taints和Tolerations。
以下案例只有先启动了一个node，创建daemonSet后，在node01创建了一个pod。
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ds1
spec:
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
再次启动node02，无需做任何操作，node02上会自动调度一个pod。
Job
The Job "j2" is invalid: spec.template.spec.restartPolicy: Unsupported value: "Always": supported values: "OnFailure", "Never"
restartPolicy不能设置为Always
单次Job
通常只启动一个 Pod，除非该 Pod 失败。当 Pod 成功终止时，立即视 Job 为完成状态。
apiVersion: batch/v1
kind: Job
metadata:
name: j1
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
删除job会关联删除pod
多次Job
completions指定了job会创建几个pod，也就是会运行几次，是并行操作。
apiVersion: batch/v1
kind: Job
metadata:
name: j2
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
completions: 2
并行Job
parallelism最大并行数。completions最小完成数。
apiVersion: batch/v1
kind: Job
metadata:
name: j3
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
completions: 2 #成功运行的Pod个数，如果不设置，默认和parallelism
parallelism: 2 #并行运行的pod个数，parallelism默认值是1，completions默认值也是1
CronJob
CronJob 创建基于时隔重复调度的 Jobs CronJob 用于执行周期性的动作，例如备份、报告生成等。 这些任务中的每一个都应该配置为周期性重复的（例如：每天/每周/每月一次）； 你可以定义任务开始执行的时间间隔。
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: cj1
spec:
schedule: '*/1 * * * *'
jobTemplate:
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
Pod升级策略
RollingUpdate
这是默认的更新方式。设置spec.strategy.type=RollingUpdate，表示Deployment会以滚动更新的方式逐个更新pod，同时可以设置spec.strategy.rollingUpdate下的两个参数maxUnavailable和maxSurge来控制滚动更新过程。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1 #用于指定Deployment在更新过程中不可用状态的Pod数量的上限，可以是百分比
maxUnavailable: 1 #用于指定Deploymnet在更新Pod的过程中Pod总数量超过预期副本数量的最大值，可以是百分比
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
现在pod镜像需要被更新为192.168.180.129:9999/myharbor/myapp:v2通过命令更新。
kubectl set image deployment d1 myapp01=192.168.180.129:9999/myharbor/myapp:v2
[root@k8s-master01 home]# kubectl set image deployment d1 myapp01=192.168.180.129:9999/myharbor/myapp:v2
deployment.extensions/d1 image updated
[root@k8s-master01 home]# kubectl rollout status deployment d1
deployment "d1" successfully rolled out
[root@k8s-master01 home]# kubectl get pod
NAME READY STATUS RESTARTS AGE
d1-7c86978c57-cfvsj 1/1 Running 0 51s
d1-7c86978c57-dn748 1/1 Running 0 55s
d1-7c86978c57-mhtdp 1/1 Running 0 48s
[root@k8s-master01 home]# kubectl describe pod d1-7c86978c57-cfvsj
Name: d1-7c86978c57-cfvsj
Namespace: default
Priority: 0
Node: k8s-node02/192.168.180.131
Start Time: Sun, 28 Aug 2022 09:21:10 +0800
Labels: app=v1
pod-template-hash=7c86978c57
Annotations: <none>
Status: Running
IP: 10.244.2.168
Controlled By: ReplicaSet/d1-7c86978c57
Containers:
myapp01:
Container ID: docker://ffc089051facd31a242d199e21ca0d3d423cbcdadfbeaa49dd7b993330124f8d
Image: 192.168.180.129:9999/myharbor/myapp:v2
Image ID: docker-pullable://192.168.180.129:9999/myharbor/myapp@sha256:a97b2685e86ee13eaa0cb625e832fb195d39c0ccc8ef4bc7611aab6cac319e34
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
State: Running
Started: Sun, 28 Aug 2022 09:21:12 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-6wl7b (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-6wl7b:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-6wl7b
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 86s default-scheduler Successfully assigned default/d1-7c86978c57-cfvsj to k8s-node02
Normal Pulling 84s kubelet, k8s-node02 Pulling image "192.168.180.129:9999/myharbor/myapp:v2"
Normal Pulled 84s kubelet, k8s-node02 Successfully pulled image "192.168.180.129:9999/myharbor/myapp:v2"
Normal Created 84s kubelet, k8s-node02 Created container myapp01
Normal Started 84s kubelet, k8s-node02 Started container myapp01
kubectl describe deploy d1 查看deployment是如何升级的pod。
9d创建了3个pod
57创建了1个pod
9d缩减到2个pod
57扩容到2个pod
9d缩减到1个pod
57扩容到3个pod
9d缩减到0个pod
最后的结果是1个deployment2个replicaSet3个pod。
Recreate
设置spec.strategy.type=Recreate，表示Deployment在更新pod时，会先杀掉所有正在运行的pod，然后创建新的pod。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
strategy:
type: Recreate
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
9d直接将pod缩减到0个然后57创建了3个。
Deployment回滚
如果在Deployment升级过程中出现意外，比如写错新镜像的名称而导致升级失败，就需要回退到升级之前的旧版本，这时就需要使用到Deployment的回滚功能了。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
现在没有myapp:v3这个镜像，查看Deployment部署过程。
kubectl rollout status deployment d1
发现新的rs在创建pod时被卡在镜像拉去过程中。为了解决这个问题，我们需要回滚到之前稳定版本的Deployment。首先查看Deployment部署记录。
kubectl rollout history deployment d1
查看Deployment特定的版本信息。
kubectl rollout history deployment d1 --revision=1
[root@k8s-master01 ~]# kubectl rollout history deployment d1 --revision=1
deployment.extensions/d1 with revision #1
Pod Template:
Labels: app=v1
pod-template-hash=8b5b8699d
Containers:
myapp01:
Image: 192.168.180.129:9999/myharbor/myapp:v1
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
Environment: <none>
Mounts: <none>
Volumes: <none>
[root@k8s-master01 ~]# kubectl rollout history deployment d1 --revision=2
deployment.extensions/d1 with revision #2
Pod Template:
Labels: app=v1
pod-template-hash=6c659bd7dd
Containers:
myapp01:
Image: 192.168.180.129:9999/myharbor/myapp:v3
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
Environment: <none>
Mounts: <none>
Volumes: <none>
现在我们回滚到上一个版本，或者回滚到指定的版本。
回滚到上个版本：kubectl rollout undo deployment
回滚到指定版本：kubectl rollout undo deployment d1 --to-revision=1
Deployment暂停和恢复
对于一次复杂的Deployment配置修改，为了避免频繁触发Deployment的更新操作，可以先暂停Deployment的更新操作，然后进行配置修改，再恢复Depolyment。注意，在恢复暂停的Deployment前，无法回滚该Deployment。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
暂停命令
kubectl rollout pause deployment d1
查看Deployment更新记录，发现并没有触发新的Deployment部署操作。
恢复命令
kubectl rollout resume deploy d1
可以看到恢复后重新创建了一个新的rs，并且Depolyment也有了更新记录
kubectl describe deploy d1 查看更新信息
DeamonSet的更新策略
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ds1
spec:
selector:
matchLabels:
app: v1
updateStrategy:
type: OnDelete
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
OnDelete
在创建好新的DaemonSet配置后，新的pod并不会被自动创建，直到用户手动删除旧的pod才会触发新建操作。
手动删除其中一个pod，它才会触发更新操作
另外一个pod还是v1版本
RollingUpdate
旧版本的pod将自动被杀掉，然后自动部署新版本的pod，整过过程与普通的Deployment滚动升级一样是可控的。
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ds1
spec:
selector:
matchLabels:
app: v1
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
回滚与Deployment操作相同。
Pod的扩缩容
k8s对pod的扩缩容操作提供了手动和自动两种模式，对Deployment或RCC进行pod副本数量设置，即可一键完成。
手动扩缩容
通过Deployment将3个副本变成5个，如果副本数少于replicas则是缩容
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
kubectl scale deployment d1 --replicas 5
HPA自动扩缩容
关于HPA的概念参考官方文档
https://kubernetes.io/zh-cn/docs/tasks/run-application/horizontal-pod-autoscale/
apiVersion: apps/v1 kind: Deployment metadata: name: d1 spec: selector: matchLabels: app: v1 replicas: 1 template: metadata: labels: app: v1 spec: containers: - name: myapp01 image: 192.168.180.129:9999/myharbor/myapp:v1 imagePullPolicy: IfNotPresent ports: - name: tomcatport containerPort: 8080 command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"] resources: requests: cpu: 50m memory: 50Mi
Metrics-Server安装
参考这篇博客，安装时需要注意自己的k8s版本 https://blog.51cto.com/lingxudong/2545242
基于CPU扩容缩容
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: hpa1
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: d1
minReplicas: 1
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 20
基于内存扩容
apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: hpa2 spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: d1 minReplicas: 1 maxReplicas: 100 metrics: - type: Resource resource: name: memory target: type: AverageValue averageUtilization: 20
