k8s-学习笔记2-容器部署nginx负载均衡
deployment文件部署
# cat nginx_deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
strategy:
type: RollingUpdate
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
imagePullSecrets:
- name: default-secret
部署完成后，查看pod信息，可以看到两个pod的ip，和所在的node。
# kubectl get pods -l app=nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-645ccd8dc9-fmrtb 1/1 Running 0 72m 10.244.1.9 192.168.18.5-hd1 <none> <none> nginx-645ccd8dc9-w75cd 1/1 Running 0 72m 10.244.1.8 192.168.18.5-hd1 <none> <none>
因为是副本部署，所以当一个pod被删除，就会自动创建新的pod
可见，删除一个pod的下一刻，新的pod已经被创建，但还没准备好，也没有分配ip
# kubectl delete pod nginx-645ccd8dc9-fmrtb pod "nginx-645ccd8dc9-fmrtb" deleted # kubectl get pods -l app=nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-645ccd8dc9-pr8l7 0/1 ContainerCreating 0 5s <none> kibanaprod-192.168.18.5-hd1 <none> <none> nginx-645ccd8dc9-w75cd 1/1 Running 0 4h28m 10.244.1.8 kibanaprod-192.168.18.5-hd1 <none> <none>
过几秒钟就已经分配好，这个新的pod ip和原来的是不一样的。
# kubectl get pods -l app=nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-645ccd8dc9-pr8l7 1/1 Running 0 112s 10.244.1.13 kibanaprod-192.168.18.5-hd1 <none> <none> nginx-645ccd8dc9-w75cd 1/1 Running 0 4h30m 10.244.1.8 kibanaprod-192.168.18.5-hd1 <none> <none>
curl pod ip可以直接访问
从k8s master上traceroute发现是直接通过flannel网络达到位于另一个node上的pod。
# curl 10.244.1.9 working. Further configuration is required.</p> [root@masterpub-10.0.0.80-hd1 18:03 /data/k8s/nginx] # traceroute 10.244.1.9 traceroute to 10.244.1.9 (10.244.1.9), 30 hops max, 60 byte packets 1 10.244.1.0 (10.244.1.0) 1.321 ms * 1.225 ms 2 10.244.1.9 (10.244.1.9) 1.290 ms 1.248 ms 1.212 ms
service expose
--port svc端口
--target-port 容器端口
--type 类型 ClusterIP, NodePort, LoadBalancer, or ExternalName. Default is 'ClusterIP'
一个svc，将一个deployment对接起来
10.99.188.113这个地址是动态生成的，它只有在集群内部才能访问到。但master上无法访问，在node上使用curl就能访问。
# kubectl get deployment NAME READY UP-TO-DATE AVAILABLE AGE nginx 2/2 2 2 4h42m # kubectl expose deployment nginx --name nginx-svc --port=80 --target-port=80 service/nginx-svc exposed # kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 29d nginx-svc ClusterIP 10.99.188.113 <none> 80/TCP 92s
查看svc的详细信息
可以看到它的名称，名称空间，标签选择器，类型，ip，endpoint
# kubectl describe svc nginx-svc Name: nginx-svc Namespace: default Labels: <none> Annotations: <none> Selector: app=nginx Type: ClusterIP IP: 10.99.188.113 Port: <unset> 80/TCP TargetPort: 80/TCP Endpoints: 10.244.1.15:80,10.244.1.16:80 Session Affinity: None Events: <none>
动态修改svc的type，改成NodePort，这样外网就能访问到
# kubectl edit svc nginx-svc
# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 29d nginx-svc NodePort 10.99.188.113 <none> 80:31621/TCP 59m
这时外网访问k8s集群中任意一个node:port，都能连接到svc。
可以做一个外部的内网SLB，绑定一个外网，把所有node接入。
scale扩容
如果两个副本不够用，可以动态扩展
# kubectl scale --replicas=4 deployment nginx deployment.apps/nginx scaled # kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE nginx-645ccd8dc9-9xkwd 1/1 Running 0 16m nginx-645ccd8dc9-nfthl 1/1 Running 0 16m nginx-645ccd8dc9-q6nlt 1/1 Running 0 82s nginx-645ccd8dc9-x4gbk 1/1 Running 0 82s
也可以缩容，指定副本数，又只剩下2个了
# kubectl scale --replicas=2 deployment nginx deployment.apps/nginx scaled [root@masterpub-10.0.0.80-hd1 22:14 ~] NAME READY STATUS RESTARTS AGE nginx-645ccd8dc9-9xkwd 1/1 Running 0 18m nginx-645ccd8dc9-nfthl 1/1 Running 0 18m
更新/回滚deployment
查看目前的deployment，image是nginx
# kubectl describe deployment nginx
Containers:
nginx:
Image: nginx
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
查看目前deployment相关的pods，都在运行
# kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE nginx-645ccd8dc9-fj2kk 1/1 Running 0 3m25s nginx-645ccd8dc9-jfspl 1/1 Running 0 3m25s nginx-645ccd8dc9-l9686 1/1 Running 0 21m nginx-645ccd8dc9-nb8zl 1/1 Running 0 21m
更新image
# kubectl set image deployment/nginx nginx=nginx:1.9.1 deployment.apps/nginx image updated
# kubectl describe deployment nginx
Containers:
nginx:
Image: nginx:1.9.1
Port: <none>
Host Port: <none>
Environment: <none>
Mounts: <none>
Volumes: <none>
第一次查看时，pods正在滚动替换，创建新的pod，销毁旧的。
第二次查看时，已全部替换
# kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE nginx-544b8d8974-78k8d 0/1 ContainerCreating 0 5s nginx-544b8d8974-g28kz 1/1 Running 0 24s nginx-544b8d8974-tlx4h 0/1 ContainerCreating 0 24s nginx-645ccd8dc9-jfspl 1/1 Running 0 4m19s nginx-645ccd8dc9-l9686 0/1 Terminating 0 22m nginx-645ccd8dc9-nb8zl 1/1 Running 0 22m # kubectl get pods -l app=nginx NAME READY STATUS RESTARTS AGE nginx-544b8d8974-78k8d 1/1 Running 0 2m54s nginx-544b8d8974-g28kz 1/1 Running 0 3m13s nginx-544b8d8974-gnhqk 1/1 Running 0 2m33s nginx-544b8d8974-tlx4h 1/1 Running 0 3m13s
回滚刚才的更新
# kubectl rollout undo deployment/nginx deployment.apps/nginx rolled back # kubectl rollout status deployment/nginx Waiting for deployment "nginx" rollout to finish: 2 out of 4 new replicas have been updated... Waiting for deployment "nginx" rollout to finish: 2 out of 4 new replicas have been updated... Waiting for deployment "nginx" rollout to finish: 2 out of 4 new replicas have been updated... Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated... Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated... Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated... Waiting for deployment "nginx" rollout to finish: 3 out of 4 new replicas have been updated... Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "nginx" rollout to finish: 1 old replicas are pending termination... Waiting for deployment "nginx" rollout to finish: 3 of 4 updated replicas are available... deployment "nginx" successfully rolled out
coredns
service提供一个ip，但外界访问需要一个域名，这个解析的工作就是coredns来完成
# kubectl get pods -o wide -n kube-system|grep dns coredns-7ff77c879f-6wkt6 1/1 Running 0 29d 10.244.0.2 masterpub-10.0.0.80-hd1 <none> <none> coredns-7ff77c879f-m2crn 1/1 Running 0 29d 10.244.0.3 masterpub-10.0.0.80-hd1 <none> <none>
# kubectl get svc -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 29d
部署一个交互式的busybox测试，查看它的dns解析，可见就是coredns的svc地址
# kubectl run client-test --image=busybox -it --restart=Never If you don't see a command prompt, try pressing enter. / # cat /etc/resolv.conf nameserver 10.96.0.10 search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5
访问svc的域名，可见，已经成功解析
/ # wget nginx-svc Connecting to nginx-svc (10.99.188.113:80)
尝试删除deployment里的两个pod，然后再测试在pod客户端里访问域名
前面10s左右无法通信，后面就有了，说明deployment里的pod已经重新创建，svc不管后面怎么变化，都通信正常
/ # wget -O - -q http://nginx-svc wget: can't connect to remote host (10.99.188.113): Connection refused / # wget -O - -q http://nginx-svc wget: can't connect to remote host (10.99.188.113): Connection refused / # wget -O - -q http://nginx-svc <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title>
service文件部署
改成NodePort，就是映射到node的port上。这里没有写nodeport的值，就是随机，范围是30000多。
文件中使用标签选择器，就是将app:nginx的pod纳入service。
# cat nginx_deployment_svc
apiVersion: v1
kind: Service
metadata:
name: nginxservice
spec:
type: NodePort
selector:
app: nginx
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
- name: https
protocol: TCP
port: 443
targetPort: 443
查看service的信息，看到type，集群ip，端口映射。
查看这个service的endpoint信息，可见将两个pod已经纳入
# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginxservice NodePort 10.96.65.135 <none> 80:30205/TCP,443:32418/TCP 64m # kubectl get ep nginxservice NAME ENDPOINTS AGE nginxservice 10.244.1.8:443,10.244.1.9:443,10.244.1.8:80 + 1 more... 66m
访问顺序
这时外网就从node ip访问nginx service
先到node ip，再到cluster ip，再到pod ip
从仓库中拉取镜像部署
创建一个namespace
#cat namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: redis
kubectl apply -f namespace.yaml
kubectl get namespace
创建一个configmap
redis.conf
kubectl create configmap redis-conf --from-file=redis.conf -n redis
kubectl get configmap -n redis
