Helm 安装部署Kubernetes的dashboard
Kubernetes Dashboard 是 k8s集群的一个 WEB UI管理工具，代码托管在 github 上，地址：https://github.com/kubernetes/dashboard
创建tls secret
通过https进行访问必需要使用证书和密钥，在Kubernetes中可以通过配置一个加密凭证（TLS secret）来提供。
这里只是拿来自己使用，创建一个自己签名的证书。
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout ./tls.key -out ./tls.crt -subj "/CN=192.168.236.130"
将会产生两个文件tls.key和tls.crt，你可以改成自己的文件名或放在特定的目录下（如果你是为公共服务器创建的，请保证这个不会被别人访问到）。后面的192.168.126.130是我的服务器IP地址，你可以改成自己的。
安装tls secret
下一步，将这两个文件的信息创建为一个Kubernetes的secret访问凭证，我将名称指定为 hongda-com-tls-secret，这在后面的Ingress配置时将会用到。如果你修改了这个名字，注意后面的配置yaml文件也需要同步修改。
kubectl -n kube-system create secret tls hongda-com-tls-secret --key ./tls.key --cert ./tls.crt
查看：
kubectl get secret -n kube-system |grep hongda
hongda-com-tls-secret kubernetes.io/tls 2 43s
安装
kubernetes-dashboard.yaml：
image:
repository: k8s.gcr.io/kubernetes-dashboard-amd64
tag: v1.10.1
ingress:
enabled: true
hosts:
- k8s.hongda.com
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
tls:
- secretName: hongda-com-tls-secret
hosts:
- k8s.hongda.com
nodeSelector:
node-role.kubernetes.io/edge: ''
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: PreferNoSchedule
rbac:
clusterAdminRole: true
相比默认配置，修改了以下配置项：
- ingress.enabled - 置为 true 开启 Ingress，用 Ingress 将 Kubernetes Dashboard 服务暴露出来，以便让我们浏览器能够访问
- ingress.annotations - 指定
ingress.class为 nginx，让我们安装 Nginx Ingress Controller 来反向代理 Kubernetes Dashboard 服务；由于 Kubernetes Dashboard 后端服务是以 https 方式监听的，而 Nginx Ingress Controller 默认会以 HTTP 协议将请求转发给后端服务，用secure-backends这个 annotation 来指示 Nginx Ingress Controller 以 HTTPS 协议将请求转发给后端服务 - ingress.hosts - 这里替换为证书配置的域名
- Ingress.tls - secretName 配置为 cert-manager 生成的免费证书所在的 Secret 资源名称，hosts 替换为证书配置的域名
- rbac.clusterAdminRole - 置为 true 让 dashboard 的权限够大，这样我们可以方便操作多个 namespace
命令安装：
helm install stable/kubernetes-dashboard \
-n kubernetes-dashboard \
--namespace kube-system \
-f kubernetes-dashboard.yaml
输出：
[root@master /]# helm install stable/kubernetes-dashboard -n kubernetes-dashboard --namespace kube-system -f kubernetes-dashboard.yaml
NAME: kubernetes-dashboard
LAST DEPLOYED: Tue Aug 6 16:11:37 2019
NAMESPACE: kube-system
STATUS: DEPLOYED
RESOURCES:
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
kubernetes-dashboard 0/1 1 0 <invalid>
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
kubernetes-dashboard-848b8dd798-gtddg 0/1 ContainerCreating 0 <invalid>
==> v1/Secret
NAME TYPE DATA AGE
kubernetes-dashboard Opaque 0 <invalid>
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-dashboard ClusterIP 10.108.244.10 <none> 443/TCP <invalid>
==> v1/ServiceAccount
NAME SECRETS AGE
kubernetes-dashboard 1 <invalid>
==> v1beta1/ClusterRoleBinding
NAME AGE
kubernetes-dashboard <invalid>
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
kubernetes-dashboard k8s.hongda.com 80, 443 <invalid>
NOTES:
*********************************************************************************
*** PLEASE BE PATIENT: kubernetes-dashboard may take a few minutes to install ***
*********************************************************************************
From outside the cluster, the server URL(s) are:
https://k8s.hongda.com
查看pods:
[root@master /]# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-5c98db65d4-gts57 1/1 Running 1 3d6h 10.244.2.2 slaver2 <none> <none>
coredns-5c98db65d4-qhwrw 1/1 Running 1 3d6h 10.244.1.2 slaver1 <none> <none>
etcd-master 1/1 Running 2 3d6h 18.16.202.163 master <none> <none>
kube-apiserver-master 1/1 Running 2 3d6h 18.16.202.163 master <none> <none>
kube-controller-manager-master 1/1 Running 6 3d6h 18.16.202.163 master <none> <none>
kube-flannel-ds-amd64-2lwl8 1/1 Running 0 3d1h 18.16.202.227 slaver1 <none> <none>
kube-flannel-ds-amd64-9bjck 1/1 Running 0 3d1h 18.16.202.95 slaver2 <none> <none>
kube-flannel-ds-amd64-gxxqg 1/1 Running 0 3d1h 18.16.202.163 master <none> <none>
kube-proxy-8cwj4 1/1 Running 0 107m 18.16.202.163 master <none> <none>
kube-proxy-j9zpz 1/1 Running 0 107m 18.16.202.227 slaver1 <none> <none>
kube-proxy-vfgjv 1/1 Running 0 107m 18.16.202.95 slaver2 <none> <none>
kube-scheduler-master 1/1 Running 6 3d6h 18.16.202.163 master <none> <none>
kubernetes-dashboard-64f97ccb4f-nbpkx 0/1 ImagePullBackOff 0 33m 10.244.0.4 master <none> <none>
tiller-deploy-6787c946f8-6b5tv 1/1 Running 0 44m 10.244.1.4 slaver1 <none> <none>
异常问题
查看线上版本：
[root@master /]# helm search kubernetes-dashboard
NAME CHART VERSION APP VERSION DESCRIPTION
stable/kubernetes-dashboard 0.6.0 1.8.3 General-purpose web UI for Kubernetes clusters
应该是版本不一致，阿里云里最新版本为1.8.3,而helm安装配置版本为1.10.1,所以导致没有拉取到镜像
添加新的仓库源
[root@master /]# helm repo add stable http://mirror.azure.cn/kubernetes/charts/
"stable" has been added to your repositories
[root@master /]# helm search kubernetes-dashboard
NAME CHART VERSION APP VERSION DESCRIPTION
stable/kubernetes-dashboard 1.8.0 1.10.1 General-purpose web UI for Kubernetes clusters
更换仓库以后，再次安装，还是一样的问题，查看
[root@master /]# kubectl get namespace
NAME STATUS AGE
default Active 3d8h
ingress-nginx Active 152m
kube-node-lease Active 3d8h
kube-public Active 3d8h
kube-system Active 3d8h
[root@master /]# kubectl describe pod kubernetes-dashboard-7ffdf885d6-t4htt -n kube-system
Name: kubernetes-dashboard-7ffdf885d6-t4htt
Namespace: kube-system
Priority: 0
Node: master/18.16.202.163
Start Time: Wed, 31 Jul 2019 16:46:40 +0800
Labels: app=kubernetes-dashboard
kubernetes.io/cluster-service=true
pod-template-hash=7ffdf885d6
release=kubernetes-dashboard
Annotations: <none>
Status: Pending
IP: 10.244.0.20
Controlled By: ReplicaSet/kubernetes-dashboard-7ffdf885d6
Containers:
kubernetes-dashboard:
Container ID:
Image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
Image ID:
Port: 8443/TCP
Host Port: 0/TCP
Args:
--auto-generate-certificates
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Limits:
cpu: 100m
memory: 50Mi
Requests:
cpu: 100m
memory: 50Mi
Liveness: http-get https://:8443/ delay=30s timeout=30s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/certs from kubernetes-dashboard-certs (rw)
/tmp from tmp-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kubernetes-dashboard-token-pph4g (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kubernetes-dashboard-certs:
Type: Secret (a volume populated by a Secret)
SecretName: kubernetes-dashboard
Optional: false
tmp-volume:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
kubernetes-dashboard-token-pph4g:
Type: Secret (a volume populated by a Secret)
SecretName: kubernetes-dashboard-token-pph4g
Optional: false
QoS Class: Guaranteed
Node-Selectors: node-role.kubernetes.io/edge=
Tolerations: node-role.kubernetes.io/master:NoSchedule
node-role.kubernetes.io/master:PreferNoSchedule
node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m47s default-scheduler Successfully assigned kube-system/kubernetes-dashboard-7ffdf885d6-t4htt to master
Normal Pulling 89s (x4 over 3m45s) kubelet, master Pulling image "k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3"
Warning Failed 74s (x4 over 3m30s) kubelet, master Failed to pull image "k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3": rpc error: code = Unknown desc = Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning Failed 74s (x4 over 3m30s) kubelet, master Error: ErrImagePull
Normal BackOff 61s (x6 over 3m30s) kubelet, master Back-off pulling image "k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3"
Warning Failed 46s (x7 over 3m30s) kubelet, master Error: ImagePullBackOff
明显是特么的拉取的k8s.gcr.io域名下面的，拉取不到。
好吧，我还是拉取不到。
解决问题
从Docker Hub中拉取一个相同版本的，替换
拉取
docker pull sacred02/kubernetes-dashboard-amd64:v1.10.1
替换
docker tag sacred02/kubernetes-dashboard-amd64:v1.10.1 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
删除
docker rmi sacred02/kubernetes-dashboard-amd64:v1.10.1
再次使用helm安装
helm install stable/kubernetes-dashboard -n kubernetes-dashboard --namespace kube-system -f kubernetes-dashboard.yaml
查看
[root@master /]# helm ls
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
kubernetes-dashboard 1 Wed Jul 31 17:11:35 2019 DEPLOYED kubernetes-dashboard-1.8.0 1.10.1 kube-system
nginx-ingress 1 Wed Jul 31 13:59:14 2019 DEPLOYED nginx-ingress-1.11.5 0.25.0 ingress-nginx
查看po,svc:
[root@master /]# kubectl get po,svc --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default pod/curl-6bf6db5c4f-vhsqc 1/1 Running 1 10d 10.244.2.3 slaver2 <none> <none>
ingress-nginx pod/nginx-ingress-controller-b89575c7f-2xtkk 1/1 Running 0 26m 18.16.202.163 master <none> <none>
ingress-nginx pod/nginx-ingress-default-backend-7b8b45bd49-g4mbz 1/1 Running 0 26m 10.244.0.23 master <none> <none>
kube-system pod/coredns-5c98db65d4-gts57 1/1 Running 7 11d 10.244.2.2 slaver2 <none> <none>
kube-system pod/coredns-5c98db65d4-qhwrw 1/1 Running 6 11d 10.244.1.2 slaver1 <none> <none>
kube-system pod/etcd-master 1/1 Running 4 11d 18.16.202.163 master <none> <none>
kube-system pod/kube-apiserver-master 1/1 Running 4 11d 18.16.202.163 master <none> <none>
kube-system pod/kube-controller-manager-master 1/1 Running 8 11d 18.16.202.163 master <none> <none>
kube-system pod/kube-flannel-ds-amd64-2lwl8 1/1 Running 0 11d 18.16.202.227 slaver1 <none> <none>
kube-system pod/kube-flannel-ds-amd64-9bjck 1/1 Running 0 11d 18.16.202.95 slaver2 <none> <none>
kube-system pod/kube-flannel-ds-amd64-gxxqg 1/1 Running 3 11d 18.16.202.163 master <none> <none>
kube-system pod/kube-proxy-8cwj4 1/1 Running 3 8d 18.16.202.163 master <none> <none>
kube-system pod/kube-proxy-j9zpz 1/1 Running 0 8d 18.16.202.227 slaver1 <none> <none>
kube-system pod/kube-proxy-vfgjv 1/1 Running 0 8d 18.16.202.95 slaver2 <none> <none>
kube-system pod/kube-scheduler-master 1/1 Running 8 11d 18.16.202.163 master <none> <none>
kube-system pod/kubernetes-dashboard-848b8dd798-gtddg 1/1 Running 0 40s 10.244.0.24 master <none> <none>
kube-system pod/tiller-deploy-6787c946f8-6b5tv 1/1 Running 0 8d 10.244.1.4 slaver1 <none> <none>
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 11d <none>
ingress-nginx service/nginx-ingress-controller LoadBalancer 10.111.25.193 <pending> 80:31577/TCP,443:31246/TCP 26m app=nginx-ingress,component=controller,release=nginx-ingress
ingress-nginx service/nginx-ingress-default-backend ClusterIP 10.106.126.222 <none> 80/TCP 26m app=nginx-ingress,component=default-backend,release=nginx-ingress
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 11d k8s-app=kube-dns
kube-system service/kubernetes-dashboard ClusterIP 10.108.244.10 <none> 443/TCP 40s app=kubernetes-dashboard,release=kubernetes-dashboard
kube-system service/tiller-deploy ClusterIP 10.98.116.74 <none> 44134/TCP 8d app=helm,name=tiller
token查看
[root@master /]# kubectl -n kube-system get secret | grep kubernetes-dashboard-token
kubernetes-dashboard-token-4v624 kubernetes.io/service-account-token 3 5m42s
[root@master /]# kubectl describe -n kube-system secret/kubernetes-dashboard-token-4v624
Name: kubernetes-dashboard-token-4v624
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: kubernetes-dashboard
kubernetes.io/service-account.uid: 6688cc3b-5f28-4e38-a37a-67c0927752ab
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi00djYyNCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjY2ODhjYzNiLTVmMjgtNGUzOC1hMzdhLTY3YzA5Mjc3NTJhYiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.Wq6xvzLSJNnt9Zg9u5J-85RB0-Slf6HMFfHzNwDGJDn3Yc2lfxL88YXi0ForX4Q9F0v96nt_GNKOm6DB8FGoKR3cALeWpeuoXSSY_ryY8tj6KFN1mrOlvVnRRgsk_lReOxLZexvR58OQ7N04pDrZ6Okr3PDB22i-31xPaVPBt6BhZU5ee6VZyXr7y3pj8VAJSki7tnr7ZRlG6WJizrMf25sZ9xdznwcGJ7yGz2gD3moYhNKQa5KPwcLOGTfg3GuLUNoQjdz5wUmvx4X2YMhfj6Fx7I3mZzr9whrfhO2PWuNtFheaKscSg2UyIPH5Zav9WTSzXxDedORh8BjX3cUJcQ
查看k8s.hongda.com
[root@master /]# ping k8s.hongda.com
PING k8s.hongda.com (13.209.58.121) 56(84) bytes of data.
From 18.16.202.169 (18.16.202.169): icmp_seq=2 Redirect Network(New nexthop: 18.16.202.1 (18.16.202.1))
From 18.16.202.169 (18.16.202.169): icmp_seq=3 Redirect Network(New nexthop: 18.16.202.1 (18.16.202.1))
^C
--- k8s.hongda.com ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2002ms
