Kubernetes探索学习001--Centos7.6使用kubeadm快速部署Kubernetes集群

Centos7.6使用kubeadm快速部署kubernetes集群

为什么要使用kubeadm来部署kubernetes？因为kubeadm是kubernetes原生的部署工具，简单快捷方便，便于新手快速搭建学习，通过kubeadm配合kubernetes相关组件的docker镜像部署出来的集群环境和二进制文件搭建起来的集群环境基本上没什么区别。但是需要注意这种方式不建议用于生产环境！主要用于研究学习kubernetes! 关于kubeadm: Easily bootstrap a secure Kubernetes cluster

1.1.服务器规划

主机名	内网ip地址	角色	系统版本
kubernetes01	10.5.0.206	Master	CentOS Linux release 7.6.1810 (Core)
kubernetes02	10.5.0.207	Worker	CentOS Linux release 7.6.1810 (Core)
kubernetes03	10.5.0.208	Worker	CentOS Linux release 7.6.1810 (Core)
kubernetes04	10.5.0.209	Worker	CentOS Linux release 7.6.1810 (Core)
kubernetes05	10.5.0.210	Worker	CentOS Linux release 7.6.1810 (Core)
kubernetes06	10.5.0.213	Worker	CentOS Linux release 7.6.1810 (Core)
kubernetes07	10.5.0.214	Worker	CentOS Linux release 7.6.1810 (Core)
kubernetes08	10.5.0.218	Worker	CentOS Linux release 7.6.1810 (Core)
kubernetes09	10.5.0.219	Worker	CentOS Linux release 7.6.1810 (Core)

1.2.Master节点

Master 节点主要包含了三个Kubernetes项目中最最最重要的组件：apiserver，scheduler，controller-manager！
apiserver：提供了管理集群的API接口
scheduler：负责分配调度Pod到集群内的node节点
controller-manager：由一系列的控制器组成，通过apiserver监控整个集群的状态

1.2.1.确认系统版本，修改主机名

1.查看系统版本
[root@iZ2ze7ftggknd1fplnxygqZ ~]# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core)
2.修改主机名
hostnamectl set-hostname kubernetes01
3.别忘了修改/etc/hosts文件
[root@kubernetes01 ~]# cat /etc/hosts
127.0.0.1       localhost       localhost.localdomain   localhost4      localhost4.localdomain4
::1     localhost       localhost.localdomain   localhost6      localhost6.localdomain6
# kubernetes-cluster
10.5.0.206 kubernetes01
...

1.2.2.关闭防火墙

systemctl stop firewalld && systemctl disable firewalld

1.2.3.检查selinux是否关闭

[root@kubernetes01 ~]# setenforce 0
setenforce: SELinux is disabled

1.2.4.提前处理路由问题

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1    
vm.swappiness=0
EOF
之后
sysctl --system

1.2.5.安装docker-ce, 注意docker-ce的版本和kubernetes版本的兼容性！

使用yum安装docekr-ce，版本v18.06.1
[root@kubernetes01 ~]# yum -y install yum-utils device-mapper-persistent-data lvm2
[root@kubernetes01 ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@kubernetes01 ~]# yum -y install docker-ce-18.06.1.ce
[root@kubernetes01 ~]# /bin/systemctl start docker.service 
[root@kubernetes01 ~]# docker --version 
Docker version 18.06.1-ce, build e68fc7a

1.2.6.安装kubelet kubeadm kubectl

1.配置某云的yum源
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
EOF
2.安装key文件
wget https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
rpm -import rpm-package-key.gpg
3.yum安装
yum install -y kubelet-1.12.1
yum install -y kubectl-1.12.1
yum install -y kubeadm-1.12.1

1.2.7.版本检查

[root@kubernetes01 ~]# kubelet --version
Kubernetes v1.12.1
[root@kubernetes01 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[root@kubernetes01 ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:43:08Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

v1.12.1kubeadm需要的kubernetes组件docker镜像版本:
k8s.gcr.io/kube-apiserver:v1.12.1
k8s.gcr.io/kube-controller-manager:v1.12.1
k8s.gcr.io/kube-scheduler:v1.12.1
k8s.gcr.io/kube-proxy:v1.12.1
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.2.24
k8s.gcr.io/coredns:1.2.2

1.2.8.下载kubernetes相关组件的docker镜像

由于国内网络环境的“特殊性”，这里另辟蹊径。
[root@kubernetes01 ~]# cat pull_k8s_images.sh 
#!/bin/bash
images=(kube-proxy:v1.12.1 kube-scheduler:v1.12.1 kube-controller-manager:v1.12.1
kube-apiserver:v1.12.1
etcd:3.2.24 coredns:1.2.2 pause:3.1 )
for imageName in ${images[@]} ; do
docker pull anjia0532/google-containers.${imageName}
docker tag anjia0532/google-containers.$imageName k8s.gcr.io/$imageName
docker rmi anjia0532/google-containers.$imageName
done

1.2.9.查看镜像信息

各位还记得开头提起过的scheduler,controller-manager,apiserver这三个基本组件的作用吗？😂别忘记～～
[root@kubernetes01 ~]# docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy                v1.12.1             61afff57f010        5 months ago        96.6MB
k8s.gcr.io/kube-apiserver            v1.12.1             dcb029b5e3ad        5 months ago        194MB
k8s.gcr.io/kube-scheduler            v1.12.1             d773ad20fd80        5 months ago        58.3MB
k8s.gcr.io/kube-controller-manager   v1.12.1             aa2dd57c7329        5 months ago        164MB
k8s.gcr.io/etcd                      3.2.24              3cab8e1b9802        6 months ago        220MB
k8s.gcr.io/coredns                   1.2.2               367cdc8433a4        7 months ago        39.2MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        15 months ago       742kB

1.2.10.使用kubeadm部署kubernetes集群master节点

[root@kubernetes01 ~]# kubeadm init --kubernetes-version=v1.12.1 
preflight检测没有问题后经过一段时间，看到这样的提示算是完成了对Kubernetes Master节点的部署。
Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.5.0.206:6443 --token bh3pih.cuir6xpjl7zn7pf2 --discovery-token-ca-cert-hash sha256:ae00fc1ad4a680c01be4deaae6f6e4cf554867664bc5c16e0b3f98d4f2adcf2c

在开始使用之前，需要以常规用户身份运行以下命令: 上面那段英文中有说明！
因为Kubernetes集群默认是需要加密访问的！
so执行👇
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

1.2.11.健康检查

1.查看主要组件的健康状态
[root@kubernetes01 ~]# kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok                   
controller-manager   Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   
2.查看master节点状态
[root@kubernetes01 ~]# kubectl get nodes
NAME           STATUS     ROLES    AGE     VERSION
kubernetes01   NotReady   master   4m15s   v1.12.1

1.2.12.部署网络插件weave

[root@kubernetes01 ~]# kubectl apply -f https://git.io/weave-kube-1.6
serviceaccount/weave-net created
serviceaccount/weave-net created
clusterrole.rbac.authorization.k8s.io/weave-net created
clusterrolebinding.rbac.authorization.k8s.io/weave-net created
role.rbac.authorization.k8s.io/weave-net created
rolebinding.rbac.authorization.k8s.io/weave-net created
daemonset.extensions/weave-net created
等一会儿，查看Master节点状态，STATUS已经变了，这是因为部署的网络组件生效了
[root@kubernetes01 ~]# kubectl get nodes
NAME                STATUS   ROLES    AGE   VERSION
kubernetes-master   Ready    master   21m   v1.12.1

1.2.13查看Master节点上网络weave相关Pod的状态

[root@kubernetes01 ~]# kubectl get pods -n kube-system -l name=weave-net -o wide
NAME              READY   STATUS    RESTARTS   AGE     IP           NODE                NOMINATED NODE
weave-net-vhs56   2/2     Running   0          6m59s   10.5.0.206   kubernetes-master   <none>

1.2.14部署可视化插件

1.获取可视化插件docker镜像，修改tag
docker pull anjia0532/google-containers.kubernetes-dashboard-amd64:v1.10.0
docker tag  anjia0532/google-containers.kubernetes-dashboard-amd64:v1.10.0   k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
docker rmi  anjia0532/google-containers.kubernetes-dashboard-amd64:v1.10.0 
2.获取并修改可视化插件YAML文件的最后部分，便于后期通过token登陆可视化页面，这里需要特别注意的是暴露了30001端口，这如果在生产环境是极不安全的！
[root@kubernetes01 ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
[root@kubernetes01 ~]# tail -n 20 kubernetes-dashboard.yaml
        effect: NoSchedule

---
# ------------------- Dashboard Service ------------------- #

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  type: NodePort
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 30001
  selector:
    k8s-app: kubernetes-dashboard
3.部署可视化插件
[root@kubernetes01 ~]# kubectl apply -f kubernetes-dashboard.yaml
secret/kubernetes-dashboard-certs created
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard created
service/kubernetes-dashboard configured
4.查看可视化插件对应的Pod状态
[root@kubernetes01 ~]# kubectl get pods -n kube-system |  grep dash
kubernetes-dashboard-65c76f6c97-f29nm   1/1     Running   0          3m8s
5.获取token值
[root@kubernetes01 ~]# kubectl -n kube-system describe $(kubectl -n kube-system get secret -n kube-system -o name | grep namespace) | grep token
Name:         namespace-controller-token-mt4sh
Type:  kubernetes.io/service-account-token
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJuYW1lc3BhY2UtY29udHJvbGxlci10b2tlbi1tdDRzaCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJuYW1lc3BhY2UtY29udHJvbGxlciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImY5YzE3YWQzLTUxYzItMTFlOS05NWZiLTAwMTYzZTBlNDRiYyIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpuYW1lc3BhY2UtY29udHJvbGxlciJ9.W2flckBO8CrzGyJzw2aJH5obQSjy4PNSll7uHOiIXPk4dnOTEzI-BfM4C9QrNDjbNTu8gIdLHntLj1181Sf_sRMidB_vhUPg6CFA1zy3XmYH21eVqjSxEBNXMSfrJHBgXnBzaHieaXqF55_etABB0j4xLM7V-bRsQ9AB0G3cv1IYU_gYG3BozksvAObmDEY4GgCI7f0-nu2YRqOMPJPhXWzKOGUvBBPyj171Xo06QvF6p9zpTMSoLa3aV-gU4XA2nMf2_aDdgFrGVI4p95ziewyu0o-W-DiEnXW1hRtwgg-PRe3QPU9ps3TALlr3U8rwh3xVmlqnRuNGVDqzmclVdQ
访问https://10.5.0.206:30001通过token登陆控制面板,注意是https协议！

1.2.15部署容器存储插件

这里需要知道Rook项目是基于Ceph的Kubernetes存储插件，一个可用于生产级别的做持久化存储的插件，值得好好把玩。

cd /usr/local/src
yum -y install git
git clone https://github.com/rook/rook.git
cd /usr/local/src/rook/cluster/examples/kubernetes/ceph
kubectl apply -f operator.yaml
kubectl apply -f cluster.yaml 

1.3.Worker节点

和安装Master节点相似，首先把准备工作做好，主机名修改，关闭防火墙，提前处理路由问题，配置yum源等等，由于节点数9个，所以这里简单的使用了下ansible playbook配合shell脚本进行安装，节省时间。
1.docker-ce的安装脚本
cat install_dockerce.sh 
#!/bin/bash
yum -y install yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum -y install docker-ce-18.06.1.ce
2.kubernetes相关组件的安装脚本
cat install_kubectl.sh
#!/bin/bash
# install kubelet and kubeadm and kubectls
yum install -y kubelet-1.12.1
yum install -y kubectl-1.12.1
yum install -y kubeadm-1.12.1
# install kube-proxy and pause
images=(kube-proxy:v1.12.1 pause:3.1 )
for imageName in ${images[@]} ; do
docker pull anjia0532/google-containers.$imageName
docker tag anjia0532/google-containers.$imageName k8s.gcr.io/$imageName
docker rmi anjia0532/google-containers.$imageName
done
# join cluster
kubeadm join 10.5.0.206:6443 --token bh3pih.cuir6xpjl7zn7pf2 --discovery-token-ca-cert-hash sha256:ae00fc1ad4a680c01be4deaae6f6e4cf554867664bc5c16e0b3f98d4f2adcf2c

1.4其它

遇到的一些小问题:
kubeadmv1.12.1无法正确安装的问题，节点报错[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]:的问题，从k8s.gcr.io拉取镜像失败的问题，这些问题都很好解决,卡住了别怕！一点一点儿克服困难。

1.5总结

文章中使用kubeadm部署了1台Kubernetes Master节点，部署了9台Kubernetes Worker节点，部署了可视化插件，部署了容器存储插件，部署了容器的网络插件。总的来说kubeadm是玩起来是相当方便😄，但是缺点也显而易见，比如没有做到Master的高可用，安全性不足等等等😭...so并不具备生产环境使用的标准。这里个人推荐生产环境研究使用kubeasz和kubespray部署！最后的最后，学习kubernetes需要的就是探索精神！☀️
PS:服务器使用的是国内某☁️的机器
欢迎大家留言讨论哦～～～