反向代理访问k8s中的应用
一、访问k8s中应用的方式
1. NodePort类型
type: NodePort ports: - port: 80 targetPort: 80 nodePort: 30008
实例：
[root@kub_master namespace]# cat nginx-svc.yaml apiVersion: v1 kind: Service metadata: name: myweb namespace: develop spec: type: NodePort ports: - port: 80 nodePort: 30008 targetPort: 80 selector: app: myweb
[root@kub_master namespace]# kubectl get all --namespace=develop NAME DESIRED CURRENT READY AGE rc/myweb 2 2 2 11m NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/myweb 192.168.72.198 <nodes> 80:30008/TCP 9m NAME READY STATUS RESTARTS AGE po/busybox 1/1 Running 9 9h po/myweb-1fd25 1/1 Running 0 11m po/myweb-z1524 1/1 Running 0 11m
测试访问
2. ClusterIP类型
type: ClusterIP ports: - port: 80 targetPort: 8080
实例
[root@kub_master namespace]# vim nginx-rc2.yaml [root@kub_master namespace]# cat nginx-rc2.yaml apiVersion: v1 kind: ReplicationController metadata: name: myweb2 namespace: develop spec: replicas: 2 selector: app: myweb2 template: metadata: labels: app: myweb2 spec: containers: - name: myweb2 image: 192.168.0.212:5000/nginx:1.15 ports: - containerPort: 80 [root@kub_master namespace]# kubectl create -f nginx-rc2.yaml replicationcontroller "myweb2" created [root@kub_master namespace]# vim nginx-svc2.yaml [root@kub_master namespace]# cat nginx-svc2.yaml apiVersion: v1 kind: Service metadata: name: myweb2 namespace: develop spec: type: ClusterIP ports: - port: 80 targetPort: 80 selector: app: myweb2 [root@kub_master namespace]# kubectl create -f nginx-svc2.yaml service "myweb2" created
[root@kub_master namespace]# kubectl get all --namespace=develop NAME DESIRED CURRENT READY AGE rc/myweb 2 2 2 34m rc/myweb2 2 2 2 19s NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/myweb 192.168.72.198 <nodes> 80:30008/TCP 32m svc/myweb2 192.168.109.59 <none> 80/TCP 10s NAME READY STATUS RESTARTS AGE po/busybox 1/1 Running 10 10h po/myweb-1fd25 1/1 Running 0 34m po/myweb-z1524 1/1 Running 0 34m po/myweb2-d85s5 1/1 Running 0 19s po/myweb2-qvxqg 1/1 Running 0 19s
访问演示
http://192.168.0.212:8080/api/v1/proxy/namespaces/develop/services/myweb2/
二、kubernetes proxy api接口
kubernetes API Server最主要的REST接口是资源对象的增、删、改、查，除此之外，它还提供了一类特殊的REST接口——kubernetes proxy API接口，这类接口的作用是代理REST请求，即kubernetes API Server 把收到的REST请求转发到某个node上的kubelet守护进程的REST端口上，由该kubelet进程负责响应。
1. kubernetes proxy API关于Node的接口的REST路径为/api/v1/proxy/nodes/{name}，其中{name}为节点的名称或者ip地址，包括以下几个具体的接口：
/api/v1/proxy/nodes/{name}/pods/ #列出指定节点内所有pod的信息
/api/v1/proxy/nodes/{name}/stats/ #列出指定节点内物理资源的统计信息
/api/v1/proxy/nodes/{name}/spec/ #列出指定节点内的概要信息
如用以下命令获取192.168.0.208节点上所有运行中的pod
[root@kub_master dashboard]# curl 192.168.0.212:8080/api/v1/proxy/nodes/192.168.0.208/pods | jq '.' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 6072 0 6072 0 0 2604k 0 --:--:-- --:--:-- --:--:-- 2964k { "kind": "PodList", "apiVersion": "v1", "metadata": {}, "items": [ { "metadata": { "name": "kube-dns-4072910292-4qb6c", "generateName": "kube-dns-4072910292-", "namespace": "kube-system", "selfLink": "/api/v1/namespaces/kube-system/pods/kube-dns-4072910292-4qb6c", "uid": "f4dac799-ffe8-11ea-8a8e-fa163e38ad0d", "resourceVersion": "651798", "creationTimestamp": "2020-09-26T11:10:59Z", "labels": { "k8s-app": "kube-dns", "pod-template-hash": "4072910292" }, "annotations": { "kubernetes.io/config.seen": "2020-09-26T19:25:28.297616251+08:00", "kubernetes.io/config.source": "api", "kubernetes.io/created-by": "{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicaSet\",\"namespace\":\"kube-system\",\"name\":\"kube-dns-4072910292\",\"uid\":\"f4d9c231-ffe8-11ea-8a8e-fa163e38ad0d\",\"apiVersion\":\"extensions\",\"resourceVersion\":\"651756\"}}\n", "scheduler.alpha.kubernetes.io/critical-pod": "", "scheduler.alpha.kubernetes.io/tolerations": "[{\"key\":\"CriticalAddonsOnly\", \"operator\":\"Exists\"}]" }, "ownerReferences": [ { "apiVersion": "extensions/v1beta1", "kind": "ReplicaSet", "name": "kube-dns-4072910292", "uid": "f4d9c231-ffe8-11ea-8a8e-fa163e38ad0d", "controller": true } ] }, "spec": { "containers": [ { "name": "kubedns", "image": "myhub.fdccloud.com/library/kubedns-amd64:1.9", "args": [ "--domain=cluster.local.", "--dns-port=10053", "--config-map=kube-dns", "--kube-master-url=http://192.168.0.212:8080", "--v=0" ], "ports": [ { "name": "dns-local", "containerPort": 10053, "protocol": "UDP" }, { "name": "dns-tcp-local", "containerPort": 10053, "protocol": "TCP" }, { "name": "metrics", "containerPort": 10055, "protocol": "TCP" } ], "env": [ { "name": "PROMETHEUS_PORT", "value": "10055" } ], "resources": { "limits": { "memory": "170Mi" }, "requests": { "cpu": "100m", "memory": "70Mi" } }, "livenessProbe": { "httpGet": { "path": "/healthz-kubedns", "port": 8080, "scheme": "HTTP" }, "initialDelaySeconds": 60, "timeoutSeconds": 5, "periodSeconds": 10, "successThreshold": 1, "failureThreshold": 5 }, "readinessProbe": { "httpGet": { "path": "/readiness", "port": 8081, "scheme": "HTTP" }, "initialDelaySeconds": 3, "timeoutSeconds": 5, "periodSeconds": 10, "successThreshold": 1, "failureThreshold": 3 }, "terminationMessagePath": "/dev/termination-log", "imagePullPolicy": "IfNotPresent" }, { "name": "dnsmasq", "image": "myhub.fdccloud.com/library/kube-dnsmasq-amd64:1.4", "args": [ "--cache-size=1000", "--no-resolv", "--server=127.0.0.1#10053" ], "ports": [ { "name": "dns", "containerPort": 53, "protocol": "UDP" }, { "name": "dns-tcp", "containerPort": 53, "protocol": "TCP" } ], "resources": { "requests": { "cpu": "150m", "memory": "10Mi" } }, "livenessProbe": { "httpGet": { "path": "/healthz-dnsmasq", "port": 8080, "scheme": "HTTP" }, "initialDelaySeconds": 60, "timeoutSeconds": 5, "periodSeconds": 10, "successThreshold": 1, "failureThreshold": 5 }, "terminationMessagePath": "/dev/termination-log", "imagePullPolicy": "IfNotPresent" }, { "name": "dnsmasq-metrics", "image": "myhub.fdccloud.com/library/dnsmasq-metrics-amd64:1.0", "args": [ "--v=2", "--logtostderr" ], "ports": [ { "name": "metrics", "containerPort": 10054, "protocol": "TCP" } ], "resources": { "requests": { "memory": "10Mi" } }, "livenessProbe": { "httpGet": { "path": "/metrics", "port": 10054, "scheme": "HTTP" }, "initialDelaySeconds": 60, "timeoutSeconds": 5, "periodSeconds": 10, "successThreshold": 1, "failureThreshold": 5 }, "terminationMessagePath": "/dev/termination-log", "imagePullPolicy": "IfNotPresent" }, { "name": "healthz", "image": "myhub.fdccloud.com/library/exechealthz-amd64:1.2", "args": [ "--cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null", "--url=/healthz-dnsmasq", "--cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null", "--url=/healthz-kubedns", "--port=8080", "--quiet" ], "ports": [ { "containerPort": 8080, "protocol": "TCP" } ], "resources": { "limits": { "memory": "50Mi" }, "requests": { "cpu": "10m", "memory": "50Mi" } }, "terminationMessagePath": "/dev/termination-log", "imagePullPolicy": "IfNotPresent" } ], "restartPolicy": "Always", "terminationGracePeriodSeconds": 30, "dnsPolicy": "Default", "nodeName": "192.168.0.208", "securityContext": {} }, "status": { "phase": "Running", "conditions": [ { "type": "Initialized", "status": "True", "lastProbeTime": null, "lastTransitionTime": "2020-09-26T11:10:59Z" }, { "type": "Ready", "status": "True", "lastProbeTime": null, "lastTransitionTime": "2020-09-26T11:25:35Z" }, { "type": "PodScheduled", "status": "True", "lastProbeTime": null, "lastTransitionTime": "2020-09-26T11:10:59Z" } ], "hostIP": "192.168.0.208", "podIP": "172.16.66.4", "startTime": "2020-09-26T11:10:59Z", "containerStatuses": [ { "name": "dnsmasq", "state": { "running": { "startedAt": "2020-09-26T11:11:03Z" } }, "lastState": {}, "ready": true, "restartCount": 0, "image": "myhub.fdccloud.com/library/kube-dnsmasq-amd64:1.4", "imageID": "docker-pullable://myhub.fdccloud.com/library/kube-dnsmasq-amd64@sha256:d68dc5377bbd81322dfdddc593bc45e3a2e042b93b5e2066bfb179747da74f1e", "containerID": "docker://f03dfe2baf71c9341f6e7a6e260b161740c74883d9031774c77ce7d672bf995b" }, { "name": "dnsmasq-metrics", "state": { "running": { "startedAt": "2020-09-26T11:11:04Z" } }, "lastState": {}, "ready": true, "restartCount": 0, "image": "myhub.fdccloud.com/library/dnsmasq-metrics-amd64:1.0", "imageID": "docker-pullable://myhub.fdccloud.com/library/dnsmasq-metrics-amd64@sha256:a1bfd78d01254c75cc14ecfad6568a9d76425506a5bc17c1a39c906c219c2f13", "containerID": "docker://b85c70b423b76c30ddc484195551ce7be498a8e3c98b41575d648fc098d8e576" }, { "name": "healthz", "state": { "running": { "startedAt": "2020-09-26T11:11:00Z" } }, "lastState": {}, "ready": true, "restartCount": 0, "image": "myhub.fdccloud.com/library/exechealthz-amd64:1.2", "imageID": "docker-pullable://myhub.fdccloud.com/library/exechealthz-amd64@sha256:b54c3595f9a8b38c8e8b84ce1721c969fdc2b1a1fb3c51ee18d4ecbf692a0cdb", "containerID": "docker://c25945854d8b9d33ed8929c8fc72be0b26f17073e4a2d33e3e99e4129b61a736" }, { "name": "kubedns", "state": { "running": { "startedAt": "2020-09-26T11:11:02Z" } }, "lastState": {}, "ready": true, "restartCount": 0, "image": "myhub.fdccloud.com/library/kubedns-amd64:1.9", "imageID": "docker-pullable://myhub.fdccloud.com/library/kubedns-amd64@sha256:1b21c69cd89b9bb47879ef94f03be2b0db194c7c04af4faa781cdd47474b88ec", "containerID": "docker://8ec91dce5a9c5c55fc2ee10dc2d302285d01bf88b3fcd006fbd22eb6d6a69051" } ] } } ] }
说明：这里获取的pod信息数据来自node而非etcd数据库。
2. kubernetes proxy API关于pod的相关接口，通过这些接口，我们可以访问pod里某个容器提供的服务
/api/v1/proxy/namespace/{namespace}/pods/{name} #访问pod
[root@kub_master dashboard]# kubectl get pods -o wide --namespace=develop NAME READY STATUS RESTARTS AGE IP NODE busybox 1/1 Running 12 12h 172.16.81.4 192.168.0.212 myweb-1fd25 1/1 Running 0 2h 172.16.46.4 192.168.0.184 myweb-z1524 1/1 Running 0 2h 172.16.81.5 192.168.0.212 myweb2-d85s5 1/1 Running 0 2h 172.16.46.5 192.168.0.184 myweb2-qvxqg 1/1 Running 0 2h 172.16.81.6 192.168.0.212 [root@kub_master dashboard]# curl http://192.168.0.212:8080/api/v1/proxy/namespaces/develop/pods/myweb2-d85s5/ <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
浏览器访问
因此，pod的proxy接口的作用和意义：在kubernetes集群外访问某个pod容器的服务（HTTP服务）时，可以用Proxy API实现，这种场景多用于管理的目的。如逐一排查Service的pod副本，检查哪个pod的服务存在异常问题。
3. kubernetes proxy API关于service的相关接口，通过这些接口，我们可以访问service
/api/v1/proxy/namespace/{namespace}/services/{name} #访问service
[root@kub_master dashboard]# curl http://192.168.0.212:8080/api/v1/proxy/namespaces/develop/services/myweb2/ <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
