十三，k8s集群web端管理工具dashboard部署
部署 dashboard
由于会被墙, 所以要加一步拉取镜像
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
#把镜像copy到其他节点
docker save k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 -o dashboard.tar
scp dashboard.tar 172.27.1.242:/root/
#在其他node节点导入镜像
docker load -i dashboard.tar
docker image ls|grep k8s.gcr.io/kubernetes-dashboard-amd64
正式开始安装dashboard
[root@master ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
[root@master ~]# kubectl apply -f kubernetes-dashboard.yaml
secret/kubernetes-dashboard-certs created
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard created
service/kubernetes-dashboard created
查看
[root@master ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-5c98db65d4-8mzfz 1/1 Running 0 44d
coredns-5c98db65d4-spjx8 1/1 Running 0 44d
etcd-master.kubernetes 1/1 Running 0 44d
kube-apiserver-master.kubernetes 1/1 Running 0 44d
kube-controller-manager-master.kubernetes 1/1 Running 0 44d
kube-flannel-ds-amd64-4szk7 1/1 Running 0 44d
kube-flannel-ds-amd64-b4ssp 1/1 Running 1 44d
kube-flannel-ds-amd64-nmklz 1/1 Running 0 44d
kube-flannel-ds-amd64-wjczq 1/1 Running 0 44d
kube-proxy-8fqsz 1/1 Running 0 44d
kube-proxy-bkrw4 1/1 Running 0 44d
kube-proxy-n75g8 1/1 Running 1 44d
kube-proxy-rmckk 1/1 Running 0 44d
kube-scheduler-master.kubernetes 1/1 Running 0 44d
kubernetes-dashboard-7d75c474bb-cqwh6 1/1 Running 0 18s # 已经启动成功
开放访问
dashboard默认采用的是ClusterIP, 用patch修改type为NodePort, 以便可以直接访问(也可以采用ingress)：
[root@master ~]# kubectl patch svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}' -n kube-system
service/kubernetes-dashboard patched
[root@master ~]# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 44d
kubernetes-dashboard NodePort 10.106.202.90 <none> 443:32211/TCP 3m16s
通过查看，dashboard的svc已经被映射到32211 这个端口，所以在浏览器直接输入任意节点的ip + 端口即可访问。
注意访问的地址是 https://IP:PORT，注意一定要是https,否则就会显示乱码
配置dashboard用户
Dashboard 登陆有两种认证方式：
- kubeconfig 文件认证
- token 令牌认证
下面对这两种认证都进行测试验证
1. token 令牌认证
创建一个 serviceAccount
在 kube-system 名称空间中创建
[root@master ~]# kubectl create serviceaccount dashboard-admin -n kube-system
[root@master ~]# kubectl describe sa/dashboard-admin -n kube-system
Name: dashboard-admin
Namespace: kube-system
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: dashboard-admin-token-wrdvz
Tokens: dashboard-admin-token-wrdvz
Events: <none>
dashboard-admin 绑定 clusterbinding 的 sa
[root@master ~]# kubectl create clusterrolebinding dashboard-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:\dashboard-admin
clusterrolebinding.rbac.authorization.k8s.io/dashboard-cluster-admin created
[root@master ~]# kubectl describe clusterrolebinding/dashboard-cluster-admin
Name: dashboard-cluster-admin
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: cluster-admin
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount dashboard-admin kube-system
这帮绑定后，serviceaccount 的 dashboard-admin 就有集群的管理员权限了。
获取 serviceaccount 的 dashboard-admin 的secret信息
在创建好serviceaccount 的 dashboard-admin 后，会在 kube-system名称空间下的secret 中，自动创建一个 以 dashboard-admin 开头的 secret，携带者 token 字样
[root@master ~]# kubectl get secret -n kube-system
NAME TYPE DATA AGE
attachdetach-controller-token-7nxx6 kubernetes.io/service-account-token 3 44d
bootstrap-signer-token-c44r9 kubernetes.io/service-account-token 3 44d
bootstrap-token-cgw8vp bootstrap.kubernetes.io/token 7 44d
certificate-controller-token-6682d kubernetes.io/service-account-token 3 44d
clusterrole-aggregation-controller-token-g7l5s kubernetes.io/service-account-token 3 44d
coredns-token-gsrxq kubernetes.io/service-account-token 3 44d
cronjob-controller-token-qch7l kubernetes.io/service-account-token 3 44d
daemon-set-controller-token-gmp2z kubernetes.io/service-account-token 3 44d
# 下面这个就是自动生成的
dashboard-admin-token-kzf2m kubernetes.io/service-account-token 3 6m50s
default-token-ll59x kubernetes.io/service-account-token 3 44d
... ...
... ...
从上已经查看到了，下面需要查看一下详细信息即可获取到token的值
[root@master ~]# kubectl describe secret/dashboard-admin-token-kzf2m -n kube-system
Name: dashboard-admin-token-wrdvz
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: dashboard-admin
kubernetes.io/service-account.uid: 4c593334-7f68-4190-85a2-f92fcab234d4
Type: kubernetes.io/service-account-token
Data
====
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4ta3pmMm0iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNzJjMzc1NjMtZGVkNS00NGJjLWJjOWYtYmVkMzY2NzY0Y2ZmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.YxTfrsJVnE0vrDRVXFEoomsN76v5l_mGA_-n0Zi0ADIJnqOvf2h34yHE2yGXphmz6qSWsDMRgPh7uk94wpSJexIZewvN8GpVDZn8xV7tBAsW_G6Ipe6ZAeXNCynfYMXPlR1Br-PW1oC7-7-aN1QPVwhEOcAQHv5EOqJ0fopPNfbh1WxsGyQCGS8Q3jBtuTI_Huwd4rKjAOaO6271uL74N2smiWD86DLXd-LVH0Qhom16Pbj-GTkJkxyiRlR-2Q8g9hsbGJNoOKVe9gv60qHDKAf8k51rwXdRoH4ZOJqveXMXNt8WQnwlexJqNY_unL4SydqdLF4_lYnPOla5atOf2g
ca.crt: 1025 bytes
上面查看到的token 后面的一大串就是，把这一串复制到页面中就可以登录。
注：实验时由于重启过一次service，导致了端口由32211变成了31718
2. kubeconfig 文件认证
创建测试一个权限小一点的，只能访问default名称空间资源的账户
创建一个 serviceAccount
[root@master pki]# kubectl create serviceaccount def-ns-admin -n default
serviceaccount/def-ns-admin created
[root@master pki]# kubectl describe sa/def-ns-admin -n default
Name: def-ns-admin
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: def-ns-admin-token-qfrqj
Tokens: def-ns-admin-token-qfrqj
Events: <none>
创建 rolebinding 绑定 def-ns-admin
[root@master pki]# kubectl create rolebinding def-ns-admin --clusterrole=admin --serviceaccount=default:def-ns-admin
rolebinding.rbac.authorization.k8s.io/def-ns-admin created
[root@master pki]# kubectl describe rolebinding/def-ns-admin -n default
Name: def-ns-admin
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: admin
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount def-ns-admin default
提示：这里和上面一样，到此步骤系统已经自动创建出一个secret，详细信息中包含token，可以直接使用此token登录dashboard，只能管理default名称空间下的资源。
创建集群和创建kubeconfig配置文件
[root@master pki]#
[root@master pki]# kubectl config set-cluster kubernetes --certificate-authority=./ca.crt --server="https://172.27.1.241:6443" --embed-certs=true --kubeconfig=/root/def-ns-admin.conf
Cluster "kubernetes" set.
[root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: kubernetes
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
拿到 def-ns-admin 用户的base64格式的token
先拿到对应的token后base64 解码
[root@master pki]# kubectl get secret def-ns-admin-token-rc4cz -o jsonpath={.data.token} | base64 -d
eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMK
这样就获取到解码后的值了，然后保存到变量中
[root@master pki]# DEF_NS_ADMIN_TOKEN=$(kubectl get secret def-ns-admin-token-rc4cz -o jsonpath={.data.token} | base64 -d)
这样执行后就保存下来了
设置 set-credentials 保存至 def-ns-admin.conf
[root@master pki]# kubectl config set-credentials def-ns-admin --token=${DEF_NS_ADMIN_TOKEN} --kubeconfig=/root/def-ns-admin.conf
[root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: kubernetes
contexts: []
current-context: ""
kind: Config
preferences: {}
users:
- name: def-ns-admin # 认证的serviceaccount
user:
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMKP7xSqGKeF0QHlw
配置成功
配置 def-ns-admin.conf 上下文
[root@master pki]# kubectl config set-context def-ns-admin@kubernetes --cluster=kubernetes --user=def-ns-admin --kubeconfig=/root/def-ns-admin.conf
Context "def-ns-admin@kubernetes" created.
[root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf
apiVersion: v1
clusters:
- cluster: # 这里是上面创建的集群
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443 # 这里是集群地址
name: kubernetes # 这里集群名称
contexts:
- context: # 上面创建的上下文
cluster: kubernetes # 集群
user: def-ns-admin # serviceaccount
name: def-ns-admin@kubernetes
current-context: "" # 这里还没有配置当前上下文
kind: Config
preferences: {}
users:
- name: def-ns-admin
user:
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMKP7xSqGKeF0QHlw
配置当前上下文
[root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: def-ns-admin
name: def-ns-admin@kubernetes
current-context: def-ns-admin@kubernetes # 当前上下文用户
kind: Config
preferences: {}
users:
- name: def-ns-admin
user:
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMKP7xSqGKeF0QHlw
这样，就把当前 上下文的用户配置成功
测试使用kubeconfig文件登陆
下载刚刚生成好的 def-ns-admin.conf
然后测试登陆:
基于 tls 验证
创建 dashboard 专用的 tls 证书
进入到 kubernetes 集群的 master 节点中的 /etc/kubernetes/pki/ 目录
这个目录是专门存放 kubernetes 集群证书的目录
生成dashboard 用户的私钥
[root@master ~]# cd /etc/kubernetes/pki/
[root@master pki]# (umask 077; openssl genrsa -out dashboard.key 2048)
Generating RSA private key, 2048 bit long modulus
........................+++
.........................................................+++
e is 65537 (0x10001)
[root@master pki]# ll dashboard.key
-rw------- 1 root root 1679 Aug 22 17:51 dashboard.key
生成证书签署请求
[root@master pki]# openssl req -new -key dashboard.key -out dashboard.csr -subj "/O=tracy/CN=dashboard"
[root@master pki]# ll dashboard.csr
-rw-r--r-- 1 root root 911 Aug 22 17:54 dashboard.csr
给dashboard证书签证
这里需要注意，一定要kubernetes集群中的 ca.crt 和 ca.key 来签署，这样才能在此集群中使用。
[root@master pki]# openssl x509 -req -in dashboard.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out dashboard.crt -days 365
Signature ok
subject=/O=tracy/CN=dashboard
Getting CA Private Key
这样就签署完成了
创建 dashboard 对应的 secret
[root@master pki]# kubectl create secret generic dashboard-cert -n kube-system --from-file=dashboard.crt --from-file=dashboard.key
secret/dashboard-cert created
[root@master pki]# kubectl get secret -n kube-system dashboard-cert
NAME TYPE DATA AGE
dashboard-cert Opaque 2 16s
[root@master pki]# kubectl describe secret -n kube-system dashboard-cert
Name: dashboard-cert
Namespace: kube-system
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
dashboard.crt: 1001 bytes
dashboard.key: 1679 bytes
Dashboard部署总结：
1、部署：
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-
flannel.yml
2、将Service改为NodePort
kubectl patch svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}' -n kube-system
3、认证：
认证时的账号必须为ServiceAccount：被dashboard pod拿来由kubernetes进行认证；
token:
（1）创建ServiceAccount，根据其管理目标，使用rolebinding或clusterrolebinding绑定至合理role或
clusterrole;
（2）获取到此ServiceAccount的secret，查看secret的详细信息，其中就有token；
kubeconfig: 把ServiceAccount的token封装为kubeconfig文件
（1）创建ServiceAccount，根据其管理目标，使用rolebinding或clusterrolebinding绑定至合理role或
clusterrole;
（2）kubectl get secret | awk '/^ServiceAccount/{print $1}'
KUBE_TOKEN=$(kubectl get secret SERVCIEACCOUNT_SERRET_NAME -o jsonpath={.data.token} |
base64 -d)
（3）生成kubeconfig文件
kubectl config set-cluster --kubeconfig=/PATH/TO/SOMEFILE
kubectl config set-credentials NAME --token=$KUBE_TOKEN --kubeconfig=/PATH/TO/SOMEFILE
kubectl config set-context
kubectl config use-context
kubernetes集群的管理方式：
1、命令式：create, run, expose, delete, edit, ...
2、命令式配置文件: create -f /PATH/TO/RESOURCE_CONFIGURATION_FILE, delete -f, replace -f
3、声明式配置文件: apply -f, patch,
