k8s service
创建 ClusterIP 类型的 Service
apiVersion: apps/v1 kind: Deployment metadata: name: my-deployment spec: selector: matchLabels: app: metrics department: sales replicas: 3 template: metadata: labels: app: metrics department: sales spec: containers: - name: hello image: "gcr.io/google-samples/hello-app:2.0"
将此清单复制到名为 my-deployment.yaml
的文件,然后创建该 Deployment:
kubectl apply -f my-deployment.yaml
root@ubuntu:~/k8s_server# mv nsexec.c my-deployment.yaml root@ubuntu:~/k8s_server# kubectl apply -f my-deployment.yaml deployment.apps/my-deployment created root@ubuntu:~/k8s_server# kubectl get pods NAME READY STATUS RESTARTS AGE kata-busybox 1/1 Running 0 207d kata-nginx 1/1 Running 0 205d my-deployment-68bdbbb5cc-2d2k6 0/1 ContainerCreating 0 7s my-deployment-68bdbbb5cc-7fkwp 0/1 ContainerCreating 0 7s my-deployment-68bdbbb5cc-lk5vr 0/1 ContainerCreating 0 7s root@ubuntu:~/k8s_server#
root@ubuntu:~/k8s_server# kubectl get pods NAME READY STATUS RESTARTS AGE kata-busybox 1/1 Running 0 207d kata-nginx 1/1 Running 0 205d my-deployment-68bdbbb5cc-2d2k6 0/1 ImagePullBackOff 0 16m my-deployment-68bdbbb5cc-7fkwp 0/1 ImagePullBackOff 0 16m my-deployment-68bdbbb5cc-lk5vr 0/1 ImagePullBackOff 0 16m root@ubuntu:~/k8s_server# kubectl delete pod my-deployment-68bdbbb5cc-2d2k6 my-deployment-68bdbbb5cc-7fkwp my-deployment-68bdbbb5cc-lk5vr pod "my-deployment-68bdbbb5cc-2d2k6" deleted pod "my-deployment-68bdbbb5cc-7fkwp" deleted pod "my-deployment-68bdbbb5cc-lk5vr" deleted
root@ubuntu:~/k8s_server# cat nginx.yaml apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 2 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 80
root@ubuntu:~/k8s_server# cat nginx-svc.yaml apiVersion: v1 kind: Service metadata: name: my-nginx labels: run: my-nginx spec: ports: - port: 80 protocol: TCP selector: run: my-nginx
述规约将创建一个 Service,对应具有标签 run: my-nginx
的 Pod,目标 TCP 端口 80, 并且在一个抽象的 Service 端口(targetPort
:容器接收流量的端口;port
:抽象的 Service 端口,可以使任何其它 Pod 访问该 Service 的端口)上暴露。 查看 Service API 对象 了解 Service 定义支持的字段列表。 查看你的 Service 资源:
root@ubuntu:~/k8s_server# kubectl create -f nginx-svc.yaml service/my-nginx created root@ubuntu:~/k8s_server# kubectl get svc my-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-nginx ClusterIP 10.110.79.116 <none> 80/TCP 8s root@ubuntu:~/k8s_server#
正如前面所提到的,一个 Service 由一组 backend Pod 组成。这些 Pod 通过 endpoints
暴露出来。 Service Selector 将持续评估,结果被 POST 到一个名称为 my-nginx
的 Endpoint 对象上。 当 Pod 终止后,它会自动从 Endpoint 中移除,新的能够匹配上 Service Selector 的 Pod 将自动地被添加到 Endpoint 中。 检查该 Endpoint,注意到 IP 地址与在第一步创建的 Pod 是相同的。
root@ubuntu:~/k8s_server# kubectl describe svc my-nginx Name: my-nginx Namespace: default Labels: run=my-nginx Annotations: <none> Selector: run=my-nginx Type: ClusterIP IP: 10.110.79.116 Port: <unset> 80/TCP TargetPort: 80/TCP Endpoints: 10.244.0.19:80,10.244.0.20:80 Session Affinity: None Events: <none>
root@ubuntu:~/k8s_server# kubectl get ep my-nginx NAME ENDPOINTS AGE my-nginx 10.244.0.19:80,10.244.0.20:80 2m35s root@ubuntu:~/k8s_server#
现在,能够从集群中任意节点上使用 curl 命令请求 Nginx Service <CLUSTER-IP>:<PORT>
。 注意 Service IP 完全是虚拟的,它从来没有走过网络,如果对它如何工作的原理感到好奇, 可以进一步阅读服务代理 的内容。
root@ubuntu:~/k8s_server# telnet 10.110.79.116 80 Trying 10.110.79.116... Connected to 10.110.79.116. Escape character is '^]'. ^CConnection closed by foreign host. root@ubuntu:~/k8s_server#
访问 Service
Kubernetes支持两种查找服务的主要模式: 环境变量和DNS。 前者开箱即用,而后者则需要[CoreDNS集群插件] CoreDNS 集群插件.
说明: 如果不需要服务环境变量(因为可能与预期的程序冲突,可能要处理的变量太多,或者仅使用DNS等),则可以通过在 pod spec 上将enableServiceLinks
标志设置为false
来禁用此模式。
环境变量
当 Pod 在 Node 上运行时,kubelet 会为每个活跃的 Service 添加一组环境变量。 这会有一个顺序的问题。想了解为何,检查正在运行的 Nginx Pod 的环境变量(Pod 名称将不会相同):
my-nginx-5dc4865748-jqx54 1/1 Running 0 18m my-nginx-5dc4865748-pcrbg 1/1 Running 0 18m root@ubuntu:~/k8s_server# kubectl exec my-nginx-5dc4865748-jqx54 -- printenv | grep SERVICE KUBERNETES_SERVICE_HOST=10.96.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 root@ubuntu:~/k8s_server# kubectl exec my-nginx-5dc4865748-pcrbg -- printenv | grep SERVICE KUBERNETES_SERVICE_HOST=10.96.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 root@ubuntu:~/k8s_server#
暴露 Service
对我们应用的某些部分,可能希望将 Service 暴露在一个外部 IP 地址上。 Kubernetes 支持两种实现方式:NodePort 和 LoadBalancer。 在上一段创建的 Service 使用了 NodePort
,因此 Nginx https 副本已经就绪, 如果使用一个公网 IP,能够处理 Internet 上的流量。
root@ubuntu:~/k8s_server# cat nginx-np.yaml apiVersion: v1 kind: Service metadata: name: my-nginx-np labels: run: my-nginx spec: type: NodePort selector: app: my-nginx ports: # 默认情况下,为了方便起见,`targetPort` 被设置为与 `port` 字段相同的值。 - port: 80 targetPort: 80 # 可选字段 # 默认情况下,为了方便起见,Kubernetes 控制平面会从某个范围内分配一个端口号(默认:30000-32767) nodePort: 30007
root@ubuntu:~/k8s_server# kubectl get svc my-nginx -o yaml apiVersion: v1 kind: Service metadata: creationTimestamp: "2021-05-12T07:10:09Z" labels: run: my-nginx managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:labels: .: {} f:run: {} f:spec: f:ports: .: {} k:{"port":80,"protocol":"TCP"}: .: {} f:port: {} f:protocol: {} f:targetPort: {} f:selector: .: {} f:run: {} f:sessionAffinity: {} f:type: {} manager: kubectl operation: Update time: "2021-05-12T07:10:09Z" name: my-nginx namespace: default resourceVersion: "39404283" selfLink: /api/v1/namespaces/default/services/my-nginx uid: 1bc222be-11ed-47b5-8363-83bf9b9e0f2a spec: clusterIP: 10.110.79.116 ports: - port: 80 protocol: TCP targetPort: 80 selector: run: my-nginx sessionAffinity: None type: ClusterIP status: loadBalancer: {}
root@ubuntu:~/k8s_server# kubectl get svc my-nginx-np -o yaml apiVersion: v1 kind: Service metadata: creationTimestamp: "2021-05-12T07:24:39Z" labels: run: my-nginx managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:labels: .: {} f:run: {} f:spec: f:externalTrafficPolicy: {} f:ports: .: {} k:{"port":80,"protocol":"TCP"}: .: {} f:nodePort: {} f:port: {} f:protocol: {} f:targetPort: {} f:selector: .: {} f:app: {} f:sessionAffinity: {} f:type: {} manager: kubectl operation: Update time: "2021-05-12T07:24:39Z" name: my-nginx-np namespace: default resourceVersion: "39406221" selfLink: /api/v1/namespaces/default/services/my-nginx-np uid: 56ee2fcd-c2dc-4455-b81f-515e9cd0bcce spec: clusterIP: 10.102.202.85 externalTrafficPolicy: Cluster ports: - nodePort: 30007 port: 80 protocol: TCP targetPort: 80 selector: app: my-nginx sessionAffinity: None type: NodePort status: loadBalancer: {}
root@ubuntu:~/k8s_server# netstat -pan | grep 30007 tcp 0 0 0.0.0.0:30007 0.0.0.0:* LISTEN 13725/kube-proxy root@ubuntu:~/k8s_server# kubectl get svc my-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-nginx ClusterIP 10.110.79.116 <none> 80/TCP 18m root@ubuntu:~/k8s_server# telnet 10.110.79.116 80 Trying 10.110.79.116... Connected to 10.110.79.116. Escape character is '^]'. ^CConnection closed by foreign host. root@ubuntu:~/k8s_server# kubectl get svc my-nginx-np NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-nginx-np NodePort 10.102.202.85 <none> 80:30007/TCP 4m39s root@ubuntu:~/k8s_server# telnet 10.102.202.85 30007 Trying 10.102.202.85... ^C
root@ubuntu:~/k8s_server# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 207d my-nginx ClusterIP 10.110.79.116 <none> 80/TCP 22m my-nginx-np NodePort 10.102.202.85 <none> 80:30007/TCP 7m55s
root@ubuntu:~/k8s_server# kubectl edit svc my-nginx-np
root@ubuntu:~/k8s_server# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 207d my-nginx ClusterIP 10.110.79.116 <none> 80/TCP 27m my-nginx-np NodePort 10.102.202.85 10.10.16.82 80:30007/TCP 13m root@ubuntu:~/k8s_server#
节点没有外网IP
root@ubuntu:~/k8s_server# kubectl get nodes -o yaml | grep ExternalIP -C 1
root@ubuntu:~/k8s_server# kubectl get nodes --output wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ubuntu Ready master 207d v1.18.1 10.10.16.82 <none> Ubuntu 18.04.3 LTS 5.0.0-23-generic containerd://1.3.7
root@ubuntu:~/k8s_server#
ClusterIP实现
root@ubuntu:~/k8s_server# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 207d my-nginx ClusterIP 10.110.79.116 <none> 8280/TCP 56m my-nginx-np NodePort 10.102.202.85 10.10.16.82 8180:32007/TCP 42m root@ubuntu:~/k8s_server# telnet 10.110.79.116 8280 Trying 10.110.79.116... Connected to 10.110.79.116. Escape character is '^]'. ^CConnection closed by foreign host.
其他机器上添加路由访问 custerip 8280
root@cloud:~/cmd# ip r add 10.110.79.116/32 via 10.10.16.82 root@cloud:~/cmd# telnet 10.110.79.116 8280 Trying 10.110.79.116... Connected to 10.110.79.116. Escape character is '^]'. ^CConnection closed by foreign host. root@cloud:~/cmd#
root@ubuntu:~/k8s_server# tcpdump -i enahisic2i0 tcp and host 10.10.16.47 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enahisic2i0, link-type EN10MB (Ethernet), capture size 262144 bytes 16:21:07.145276 IP 10.10.16.47.36654 > 10.110.79.116.8280: Flags [S], seq 1830641387, win 64240, options [mss 1460,sackOK,TS val 903079081 ecr 0,nop,wscale 7], length 0 16:21:07.145395 IP 10.110.79.116.8280 > 10.10.16.47.36654: Flags [S.], seq 941547984, ack 1830641388, win 64308, options [mss 1410,sackOK,TS val 3689254214 ecr 903079081,nop,wscale 7], length 0 16:21:07.145455 IP 10.10.16.47.36654 > 10.110.79.116.8280: Flags [.], ack 1, win 502, options [nop,nop,TS val 903079081 ecr 3689254214], length 0 ^C 3 packets captured 11 packets received by filter 0 packets dropped by kernel root@ubuntu:~/k8s_server#
root@ubuntu:~/k8s_server# tcpdump -i cni0 tcp and host 10.10.16.47 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on cni0, link-type EN10MB (Ethernet), capture size 262144 bytes 16:22:14.896119 IP 10.244.0.20.http > 10.10.16.47.36658: Flags [S.], seq 3152243450, ack 2069205458, win 64308, options [mss 1410,sackOK,TS val 3689321964 ecr 903146831,nop,wscale 7], length 0 16:22:30.142312 IP 10.244.0.20.http > 10.10.16.47.36658: Flags [.], ack 3, win 503, options [nop,nop,TS val 3689337211 ecr 903162078], length 0 16:22:31.157763 IP 10.244.0.20.http > 10.10.16.47.36658: Flags [.], ack 7, win 503, options [nop,nop,TS val 3689338226 ecr 903163093], length 0 16:22:31.157900 IP 10.244.0.20.http > 10.10.16.47.36658: Flags [P.], seq 1:312, ack 7, win 503, options [nop,nop,TS val 3689338226 ecr 903163093], length 311: HTTP: HTTP/1.1 400 Bad Request 16:22:31.157957 IP 10.244.0.20.http > 10.10.16.47.36658: Flags [F.], seq 312, ack 7, win 503, options [nop,nop,TS val 3689338226 ecr 903163093], length 0 16:22:31.158084 IP 10.244.0.20.http > 10.10.16.47.36658: Flags [.], ack 8, win 503, options [nop,nop,TS val 3689338226 ecr 903163094], length 0
root@ubuntu:~/k8s_server# conntrack -D
root@ubuntu:~/k8s_server# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 207d my-nginx ClusterIP 10.110.79.116 <none> 8280/TCP 78m my-nginx-np NodePort 10.102.202.85 10.10.16.82 8180:32007/TCP 64m
root@ubuntu:~/k8s_server# netstat -pan | grep -E '8280|8180|32007' tcp 0 0 10.10.16.82:8180 0.0.0.0:* LISTEN 13725/kube-proxy tcp 0 0 0.0.0.0:32007 0.0.0.0:* LISTEN 13725/kube-proxy
root@ubuntu:~/k8s_server# iptables -S -t nat | grep 8280 -A KUBE-SERVICES ! -s 10.244.0.0/16 -d 10.110.79.116/32 -p tcp -m comment --comment "default/my-nginx: cluster IP" -m tcp --dport 8280 -j KUBE-MARK-MASQ -A KUBE-SERVICES -d 10.110.79.116/32 -p tcp -m comment --comment "default/my-nginx: cluster IP" -m tcp --dport 8280 -j KUBE-SVC-BEPXDJBUHFCSYIC3
Chain KUBE-SVC-BEPXDJBUHFCSYIC3 (1 references) target prot opt source destination KUBE-SEP-WURJJT7GEXOC32WR all -- 0.0.0.0/0 0.0.0.0/0 /* default/my-nginx: */ statistic mode random probability 0.50000000000 KUBE-SEP-UFGUNP44NJC7LB3J all -- 0.0.0.0/0 0.0.0.0/0 /* default/my-nginx: */
Chain KUBE-SEP-UFGUNP44NJC7LB3J (1 references) target prot opt source destination KUBE-MARK-MASQ all -- 10.244.0.20 0.0.0.0/0 /* default/my-nginx: */ DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 /* default/my-nginx: */ tcp DNAT [unsupported revision] Chain KUBE-SEP-WURJJT7GEXOC32WR (1 references) target prot opt source destination KUBE-MARK-MASQ all -- 10.244.0.19 0.0.0.0/0 /* default/my-nginx: */ DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 /* default/my-nginx: */ tcp DNAT [unsupported revision]
NodePort
root@ubuntu:~/k8s_server# cat nginx-np.yaml apiVersion: v1 kind: Service metadata: name: my-nginx-np labels: run: my-nginx spec: ports: - nodePort: 31199 port: 8080 protocol: TCP targetPort: 80 selector: run: my-nginx type: NodePort status: loadBalancer: {}
本地访问port=8080成功
root@ubuntu:~/k8s_server# telnet 10.99.1.231 8080 Trying 10.99.1.231... Connected to 10.99.1.231. Escape character is '^]'. ^CConnection closed by foreign host.
远程访问31199
root@ubuntu:~/k8s_server# iptables -S -t nat | grep 31199 -A KUBE-NODEPORTS -p tcp -m comment --comment "default/my-nginx-np:" -m tcp --dport 31199 -j KUBE-MARK-MASQ -A KUBE-NODEPORTS -p tcp -m comment --comment "default/my-nginx-np:" -m tcp --dport 31199 -j KUBE-SVC-T5LLH4C2SBESAK4A root@ubuntu:~/k8s_server#
通过node ip=10.10.16.82 nodeport=31199访问
远程访问8080
root@cloud:~/cmd# ip r add 10.99.1.231/32 via 10.10.16.82 root@cloud:~/cmd# telnet 10.10.16.82 8080 Trying 10.10.16.82... telnet: Unable to connect to remote host: Connection refused root@cloud:~/cmd# telnet 10.99.1.231 8080 Trying 10.99.1.231... Connected to 10.99.1.231. Escape character is '^]'. ^CConnection closed by foreign host. root@cloud:~/cmd#
root@ubuntu:~/k8s_server# iptables -S -t nat | grep 8080 -A KUBE-SERVICES ! -s 10.244.0.0/16 -d 10.99.1.231/32 -p tcp -m comment --comment "default/my-nginx-np: cluster IP" -m tcp --dport 8080 -j KUBE-MARK-MASQ -A KUBE-SERVICES -d 10.99.1.231/32 -p tcp -m comment --comment "default/my-nginx-np: cluster IP" -m tcp --dport 8080 -j KUBE-SVC-T5LLH4C2SBESAK4A root@ubuntu:~/k8s_server#
总结
Kubernetes -- port, targetport, nodeport的区别
使用 Service 连接到应用
使用 Service 公开应用
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步