kubenetes k8s label控制pod位置及创建SVC、外网访问
一、k8s label控制pod位置
1 默认情况下，Scheduler会将pod调度到所有可用的node。不过有些情况我们希望将pod部署到指定的node, 使用label可以实现这个功能。
|
kubectl get node --show-lables
|
2 查看label是否生效
|
[root@k8s-master01 ~]# kubectl get node --show-labels
NAME STATUS ROLES AGE VERSION LABELS
k8s-master01 Ready master 41d v1.15.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master01,kubernetes.io/os=linux,node-role.kubernetes.io/master=
k8s-node01 Ready <none> 41d v1.15.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node01,kubernetes.io/os=linux,onsite=kxc
k8s-node02 Ready <none> 41d v1.15.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node02,kubernetes.io/os=linux
|
3 编辑yml文件
|
[root@k8s-master01 ~]# vi nginx.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deployment0814
spec:
replicas: 3
template:
metadata:
labels:
app: web-server
spec:
containers:
- name: nginx
image: nginx:1.7.9
nodeSelector:
onsite: kxc
|
4 执行创建pod
|
[root@k8s-master01 ~]# kubectl apply -f nginx.yaml
deployment.extensions/nginx-deployment0814 created
|
5 查看是否pod是否都部署在node01上面
|
[root@k8s-master01 ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
http-displayment-6dbfc78c44-djjqg 1/1 Running 9 35d 10.255.1.64 k8s-node01 <none> <none>
http-displayment-6dbfc78c44-qzkdz 1/1 Running 9 35d 10.255.1.65 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-4vfnm 1/1 Running 8 30d 10.255.1.69 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-c4gf4 1/1 Running 8 30d 10.255.2.43 k8s-node02 <none> <none>
httpd-deployment-5dbd9b4457-ptzkl 1/1 Running 8 30d 10.255.1.68 k8s-node01 <none> <none>
httpd0814-5f4d9c656d-c7p5z 1/1 Running 0 20m 10.255.2.50 k8s-node02 <none> <none>
httpd0814-5f4d9c656d-vf9rw 1/1 Running 0 22m 10.255.1.71 k8s-node01 <none> <none>
myapp-pod 1/1 Running 68 36d 10.255.1.70 k8s-node01 <none> <none>
myhttpd-deploy-86c586bc96-89pmw 1/1 Running 4 6d7h 10.255.2.45 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-cvqtz 1/1 Running 4 6d7h 10.255.2.47 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-tttk4 1/1 Running 4 6d7h 10.255.1.66 k8s-node01 <none> <none>
nginx-deployment-557c97577b-kx4cx 1/1 Running 9 35d 10.255.2.46 k8s-node02 <none> <none>
nginx-deployment-557c97577b-q48lf 1/1 Running 9 35d 10.255.1.67 k8s-node01 <none> <none>
nginx-deployment-557c97577b-sl9l6 1/1 Running 9 35d 10.255.2.44 k8s-node02 <none> <none>
nginx-deployment0814-b98dfdc97-26lbf 1/1 Running 0 57s 10.255.1.74 k8s-node01 <none> <none>
nginx-deployment0814-b98dfdc97-44f4p 1/1 Running 0 57s 10.255.1.73 k8s-node01 <none> <none>
nginx-deployment0814-b98dfdc97-pm82h 1/1 Running 0 57s 10.255.1.72 k8s-node01 <none> <none>
|
6 删除label
|
[root@k8s-master01 ~]# kubectl label node k8s-node01 onsite-
node/k8s-node01 labeled
|
查看label是否消失
|
[root@k8s-master01 ~]# kubectl get node --show-labels
NAME STATUS ROLES AGE VERSION LABELS
k8s-master01 Ready master 41d v1.15.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master01,kubernetes.io/os=linux,node-role.kubernetes.io/master=
k8s-node01 Ready <none> 41d v1.15.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node01,kubernetes.io/os=linux
k8s-node02 Ready <none> 41d v1.15.1 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-node02,kubernetes.io/os=linux
|
不过此时pod并不会重新部署，依旧在node01上运行
|
[root@k8s-master01 ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
http-displayment-6dbfc78c44-djjqg 1/1 Running 9 35d 10.255.1.64 k8s-node01 <none> <none>
http-displayment-6dbfc78c44-qzkdz 1/1 Running 9 35d 10.255.1.65 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-4vfnm 1/1 Running 8 30d 10.255.1.69 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-c4gf4 1/1 Running 8 30d 10.255.2.43 k8s-node02 <none> <none>
httpd-deployment-5dbd9b4457-ptzkl 1/1 Running 8 30d 10.255.1.68 k8s-node01 <none> <none>
httpd0814-5f4d9c656d-c7p5z 1/1 Running 0 28m 10.255.2.50 k8s-node02 <none> <none>
httpd0814-5f4d9c656d-vf9rw 1/1 Running 0 30m 10.255.1.71 k8s-node01 <none> <none>
myapp-pod 1/1 Running 68 36d 10.255.1.70 k8s-node01 <none> <none>
myhttpd-deploy-86c586bc96-89pmw 1/1 Running 4 6d7h 10.255.2.45 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-cvqtz 1/1 Running 4 6d7h 10.255.2.47 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-tttk4 1/1 Running 4 6d7h 10.255.1.66 k8s-node01 <none> <none>
nginx-deployment-557c97577b-kx4cx 1/1 Running 9 35d 10.255.2.46 k8s-node02 <none> <none>
nginx-deployment-557c97577b-q48lf 1/1 Running 9 35d 10.255.1.67 k8s-node01 <none> <none>
nginx-deployment-557c97577b-sl9l6 1/1 Running 9 35d 10.255.2.44 k8s-node02 <none> <none>
nginx-deployment0814-b98dfdc97-26lbf 1/1 Running 0 9m15s 10.255.1.74 k8s-node01 <none> <none>
nginx-deployment0814-b98dfdc97-44f4p 1/1 Running 0 9m15s 10.255.1.73 k8s-node01 <none> <none>
nginx-deployment0814-b98dfdc97-pm82h 1/1 Running 0 9m15s 10.255.1.72 k8s-node01 <none> <none>
|
除非在nginx.yml中删除nodeSelector-设置，然后通过kubectl apply重新部署。
修改如下：
|
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deployment0814
spec:
replicas: 3
template:
metadata:
labels:
app: web-server
spec:
containers:
- name: nginx
image: nginx:1.7.9
# nodeSelector:
# onsite: kxc
|
重新执行
|
[root@k8s-master01 ~]# kubectl apply -f nginx.yaml
deployment.extensions/nginx-deployment0814 configured
|
结果看下，有调度到node02节点
|
[root@k8s-master01 ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
http-displayment-6dbfc78c44-djjqg 1/1 Running 9 35d 10.255.1.64 k8s-node01 <none> <none>
http-displayment-6dbfc78c44-qzkdz 1/1 Running 9 35d 10.255.1.65 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-4vfnm 1/1 Running 8 30d 10.255.1.69 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-c4gf4 1/1 Running 8 30d 10.255.2.43 k8s-node02 <none> <none>
httpd-deployment-5dbd9b4457-ptzkl 1/1 Running 8 30d 10.255.1.68 k8s-node01 <none> <none>
httpd0814-5f4d9c656d-c7p5z 1/1 Running 0 31m 10.255.2.50 k8s-node02 <none> <none>
httpd0814-5f4d9c656d-vf9rw 1/1 Running 0 33m 10.255.1.71 k8s-node01 <none> <none>
myapp-pod 1/1 Running 68 36d 10.255.1.70 k8s-node01 <none> <none>
myhttpd-deploy-86c586bc96-89pmw 1/1 Running 4 6d7h 10.255.2.45 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-cvqtz 1/1 Running 4 6d7h 10.255.2.47 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-tttk4 1/1 Running 4 6d7h 10.255.1.66 k8s-node01 <none> <none>
nginx-deployment-557c97577b-kx4cx 1/1 Running 9 35d 10.255.2.46 k8s-node02 <none> <none>
nginx-deployment-557c97577b-q48lf 1/1 Running 9 35d 10.255.1.67 k8s-node01 <none> <none>
nginx-deployment-557c97577b-sl9l6 1/1 Running 9 35d 10.255.2.44 k8s-node02 <none> <none>
nginx-deployment0814-557c97577b-4c7rq 1/1 Running 0 27s 10.255.1.75 k8s-node01 <none> <none>
nginx-deployment0814-557c97577b-876l4 1/1 Running 0 25s 10.255.1.76 k8s-node01 <none> <none>
nginx-deployment0814-557c97577b-s7ghh 1/1 Running 0 27s 10.255.2.51 k8s-node02 <none> <none>
|
二、创建SVC
1 将nginx.yaml修改以下内容
|
[root@k8s-master01 ~]# cat nginx.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deployment0814
spec:
replicas: 3
template:
metadata:
labels:
app: nginx0814-server
spec:
containers:
- name: nginx
image: nginx:1.7.9
# nodeSelector:
# onsite: kxc
|
2 创建SVC文件nginx-svc.yml
|
[root@k8s-master01 ~]# cat nginx-svc.yml
apiVersion: v1
kind: Service
metadata:
name: nginx-svc0814
namespace: default
spec:
type: ClusterIP
selector:
app: nginx0814-server
# release: stabel
ports:
- name: nginx-port0814
protocol: TCP
port: 30014
targetPort: 80
|
3 删除原来的nginx-deployment0814，重新部署
|
[root@k8s-master01 ~]# kubectl delete deployment nginx-deployment0814
deployment.extensions "nginx-deployment0814" deleted
[root@k8s-master01 ~]# kubectl apply -f nginx.yaml
deployment.extensions/nginx-deployment0814 created
|
查看pod相关信息
|
[root@k8s-master01 ~]# kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
http-displayment 2/2 2 2 36d
httpd-deployment 3/3 3 3 30d
httpd0814 2/2 2 2 12h
myhttpd-deploy 3/3 3 3 6d19h
nginx-deployment 3/3 3 3 35d
nginx-deployment0814 3/3 3 3 5s
[root@k8s-master01 ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
http-displayment-6dbfc78c44-djjqg 1/1 Running 10 35d 10.255.1.86 k8s-node01 <none> <none>
http-displayment-6dbfc78c44-qzkdz 1/1 Running 10 36d 10.255.1.85 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-4vfnm 1/1 Running 9 30d 10.255.1.81 k8s-node01 <none> <none>
httpd-deployment-5dbd9b4457-c4gf4 1/1 Running 9 30d 10.255.2.53 k8s-node02 <none> <none>
httpd-deployment-5dbd9b4457-ptzkl 1/1 Running 9 30d 10.255.1.82 k8s-node01 <none> <none>
httpd0814-5f4d9c656d-c7p5z 1/1 Running 1 12h 10.255.2.52 k8s-node02 <none> <none>
httpd0814-5f4d9c656d-vf9rw 1/1 Running 1 12h 10.255.1.77 k8s-node01 <none> <none>
myapp-pod 1/1 Running 73 36d 10.255.1.79 k8s-node01 <none> <none>
myhttpd-deploy-86c586bc96-89pmw 1/1 Running 5 6d19h 10.255.2.55 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-cvqtz 1/1 Running 5 6d19h 10.255.2.54 k8s-node02 <none> <none>
myhttpd-deploy-86c586bc96-tttk4 1/1 Running 5 6d19h 10.255.1.80 k8s-node01 <none> <none>
nginx-deployment-557c97577b-kx4cx 1/1 Running 10 35d 10.255.2.57 k8s-node02 <none> <none>
nginx-deployment-557c97577b-q48lf 1/1 Running 10 35d 10.255.1.78 k8s-node01 <none> <none>
nginx-deployment-557c97577b-sl9l6 1/1 Running 10 35d 10.255.2.56 k8s-node02 <none> <none>
nginx-deployment0814-59574fff8f-d7pmh 1/1 Running 0 18s 10.255.2.59 k8s-node02 <none> <none>
nginx-deployment0814-59574fff8f-f9d9t 1/1 Running 0 18s 10.255.1.87 k8s-node01 <none> <none>
nginx-deployment0814-59574fff8f-nzs98 1/1 Running 0 18s 10.255.2.60 k8s-node02 <none> <none>
|
4 创建svc
|
[root@k8s-master01 ~]# kubectl apply -f nginx-svc.yml
service/nginx-svc0814 created
[root@k8s-master01 ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 41d
mydb ClusterIP 10.254.129.95 <none> 80/TCP 36d
myhttpd-deploy ClusterIP 10.254.220.221 <none> 80/TCP 6d17h
myservice ClusterIP 10.254.217.74 <none> 80/TCP 36d
nginx-svc0814 ClusterIP 10.254.160.183 <none> 30014/TCP 9s
|
5 检测网络是否正常访问，在集群任意节点
|
[root@k8s-master01 ~]# curl http://10.254.160.183:30014
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
|
三、外网访问
service ClusterIP只能在集群访问，当然可以使用外部NAT或者LVS实现外网访问，k8s自带的NodePort可以轻易实现外部访问。
1 修改nginx-svc
|
[root@k8s-master01 ~]# vi nginx-svc.yml
apiVersion: v1
kind: Service
metadata:
name: nginx-svc0814
namespace: default
spec:
type: NodePort
selector:
app: nginx0814-server
# release: stabel
ports:
- name: nginx-port0814
protocol: TCP
port: 30014
targetPort: 80
nodePort: 31014
~
|
2 重新应用nginx-svc.yml
|
[root@k8s-master01 ~]# kubectl apply -f nginx-svc.yml
service/nginx-svc0814 configured
[root@k8s-master01 ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.254.0.1 <none> 443/TCP 41d
mydb ClusterIP 10.254.129.95 <none> 80/TCP 36d
myhttpd-deploy ClusterIP 10.254.220.221 <none> 80/TCP 6d17h
myservice ClusterIP 10.254.217.74 <none> 80/TCP 36d
nginx-svc0814 NodePort 10.254.160.183 <none> 30014:31014/TCP 18m
|
3 检验所有node节点的地址都可以访问
master01:
|
[root@k8s-master01 ~]# curl http://192.168.3.91:31014
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
|
node01：
|
[root@k8s-master01 ~]# curl http://192.168.3.94:31014
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
|
node02：
|
[root@k8s-master01 ~]# curl http://192.168.3.95:31014
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
|
iptables映射：
|
[root@k8s-master01 ~]# iptables-save | grep 31014
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/nginx-svc0814:nginx-port0814" -m tcp --dport 31014 -j KUBE-MARK-MASQ
-A KUBE-NODEPORTS -p tcp -m comment --comment "default/nginx-svc0814:nginx-port0814" -m tcp --dport 31014 -j KUBE-SVC-3MFC53WNCMNMNMFD
|
