kubernetes的安装

• 获取源码

最新安装包下载地址，GitHub下载地址

本次实验的1.10.0的二进制包下载，百度网盘

• 机器环境

Kubernetes Roles	IP地址	Hostname
Master	192.168.142.161	kubernetes-node1.example.com
Node	192.168.142.162	kubernetes-node2.example.com
Node	192.168.142.163	kubernetes-node3.example.com

Master端配置

• 配置kube-apiserver服务

将kube-apiserver的可执行文件复制到/usr/bin目录下
然后编辑systemd的服务文件
vim /usr/lib/systemd/system/kube-apiserver.service

[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=etcd.service
Wants=etcd.service


[Service]
EnvironmentFile=/etc/kubernetes/apiserver
ExecStart=/usr/bin/kube-apiserver $KUBE_API_ARGS
Restart=on-failure
LimitNOFILE=65536


[Install]
WantedBy=multi-user.target

基于CA签名的双向数字证书认证方式进行认证
生成过程如下：

(1)为kube-apiserver生成一个数字证书，并用CA证书进行签名。
(2)为kube-apiserver进程配置证书相关的启动参数，包括CA证书（用于验证客户端证书的签名真伪、自己经过CA签名后的证书及私钥）。
(3)为每个访问Kubernetes API Server的客户端进程生成自己的数字证书，也都用CA证书进行签名，在相关程序的启动参数中增加CA证书、自己的证书等相关参数。

设置kube-apiserver的CA证书相关的文件和启动参数

OpenSSL工具在Master服务器上创建CA证书和私钥相关的文件

openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=example.com" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048

生成文件如下：

ca.crt  ca.key  server.key

创建Master_ssl.cnf文件，生成x509 v3版本证书.在该文件中主要需要设置Master服务器的hostname，IP地址，以及Kubernetes Master Service的虚拟服务器名称和该虚拟服务器的clusterIP地址。

DNS.5为Master服务器的HostName，IP.1为Kubernetes Master Service的Cluster IP，IP.2为Master服务器的IP。

[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = kubernets-node1.example.com
IP.1 = 169.169.0.1
IP.2 = 192.168.142.161

基于Master_ssl.cnf生成server.csr和server.crt。
在生成server.csr时，-subj参数中/CN指定的名字需要为Master所在的主机名。

openssl req -new -key server.key -subj "/CN=kubernets-node1.example.com" -config /etc/kubernetes/master_ssl.cnf -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile /etc/kubernetes/master_ssl.cnf -out server.crt

现在有6个文件：

ca.crt ca.key ca.srl server.crt server.csr server.key

cp ca.crt ca.key ca.srl server.crt server.csr server.key /var/run/kubernetes/

指定配置文件/etc/kubernetes/apiserver的内容，具体内容如下：

vim /etc/kubernetes/apiserver

KUBE_API_ARGS="--etcd-servers=http://192.168.142.161:2379,http://192.168.142.162:2379,http://192.168.142.163:2379 --bind-address=0.0.0.0 --secure-port=443 --insecure-port=0 --client-ca-file=/var/run/kubernetes/ca.crt --tls-private-key-file=/var/run/kubernetes/server.key --tls-cert-file=/var/run/kubernetes/server.crt --service-cluster-ip-range=169.169.0.0/16 --service-node-port-range=1-65535 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ResourceQuota --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

• 配置kube-controller-manager服务

kube-controller-manager依赖于kube-apiserver服务。

配置启动文件

cat /usr/lib/systemd/system/kube-controller-manager.service 
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service


[Service]
EnvironmentFile=/etc/kubernetes/controller-manager
ExecStart=/usr/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
#Type=notify
LimitNOFILE=65536


[Install]
WantedBy=multi-user.target

设置kube-controller-manager的客户端证书、私钥

openssl genrsa -out cs_client.key 2048
openssl req -new -key cs_client.key -subj "/CN=kubernets-node1.example.com" -out cs_client.csr
openssl x509 -req -in cs_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000

其中在生成cs_client.crt时，-CA参数和-CAkey参数使用的是apiserver的ca.crt和ca.key文件，然后将这些文件复制到一个目录中（/var/run/kubernetes）

接下来创建/etc/kubernetes/kubeconfig文件（kube-controller-manager与kube-scheduler共用）
内容如下

cat /etc/kubernetes/kubeconfig
apiVersion: v1
kind: Config
users:
- name: controllermanager
  user:
    client-certificate: /var/run/kubernetes/cs_client.crt
    client-key: /var/run/kubernetes/cs_client.key
clusters:
- name: local
  cluster:
    certificate-authority: /var/run/kubernetes/ca.crt
contexts:
- context:
    cluster: local
    user: controllermanager
  name: my-context
current-context: my-contex

然后设置kube-controller-manager的启动参数

cat /etc/kubernetes/controller-manager
KUBE_CONTROLLER_MANAGER_ARGS="--master=https://192.168.142.161 --service-account-private-key-file=/var/run/kubernetes/server.key --root-ca-file=/var/run/kubernetes/ca.crt --kubeconfig=/etc/kubernetes/kubeconfig --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

• 配置kube-scheduler服务

kube-scheduler服务也依赖于kube-apiserver服务

cat /usr/lib/systemd/system/kube-scheduler.service

[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes
After=kube-apiserver.service
Wants=kube-apiserver.service


[Service]
EnvironmentFile=/etc/kubernetes/scheduler
ExecStart=/usr/bin/kube-scheduler $KUBE_SCHEDULER_ARGS
Restart=on-failure
#Type=notify
LimitNOFILE=65536


[Install]
WantedBy=multi-user.target

复用kube-controller-manager创建的客户端证书

配置开机启动参数

cat /etc/kubernetes/scheduler
KUBE_SCHEDULER_ARGS="--master=https://192.168.142.161 --kubeconfig=/etc/kubernetes/kubeconfig --logtostderr=false --log-dir=/var/lib/kubernetes --v=2"

至此Master端安装完成。启动所有服务：

systemctl start kube-apiserver systemctl start kube-controller-manager systemctl start kube-kube-scheduler

配置Node上的kubelet和kube-proxy

kubelet服务依赖docker，这里我们需要安装docker。安装过程如下：

如果你之前安装过 docker，请先删掉
yum remove docker docker-common docker-selinux docker-engine
安装依赖
yum install -y yum-utils device-mapper-persistent-data lvm2
下载repo文件
wget -O /etc/yum.repos.d/docker-ce.repo https://download.docker.com/linux/centos/docker-ce.repo
替换为国内的源镜像站
sed -i 's+download-stage.docker.com+mirrors.tuna.tsinghua.edu.cn/docker-ce+' /etc/yum.repos.d/docker-ce.repo
安装docker
yum install docker-ce
启动
systemctl enable docker
systemctl start docker

1：首先复制kube-apiserver的ca.crt和ca.key文件到node上，在生成kubelet_client.crt时-CA参数和-CAkey参数使用的是apiserver的ca.crt和ca.key文件。在生成kube_let.csr时-subj参数中的“/CN”设置为Node的IP地址。

openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=192.168.142.162" -out kubelet_client.csr
openssl x509 -req -in kubelet_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

2：然后创建/etc/kubernetes/kubeconfig文件。（kubelet和kube-proxy进程共用）,配置客户端证书等相关参数：

cat /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: kubelet
  user:
    client-certificate: /var/run/kubernetes/kubelet_client.crt
    client-key: /var/run/kubernetes/kubelet_client.key
clusters:
- name: local
  cluster:
    server: https://192.168.142.161
    certificate-authority: /var/run/kubernetes/ca.crt
contexts:
- context:
    cluster: local
    user: kubelet
  name: my-context
current-context: my-context

3：设置kubelet服务的启动参数

cat /etc/kubernetes/kubelet

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=192.168.142.162 --pod-infra-container-image=registry-vpc.cn-beijing.aliyuncs.com/k8s_len/pause-amd64:3.0 --fail-swap-on=false --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

4：设置kube-proxy的启动参数

cat /etc/kubernetes/kube-proxy

KUBE_PROXY_ARGS="--master=https://192.168.142.161 --kubeconfig=/etc/kubernetes/kubeconfig --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

5：定义开机服务的启动项

cat /usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/kubernetes/kubernetes
After=docker.service
Wants=docker.service


[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=/etc/kubernetes/kubeconfig.yaml
EnvironmentFile=/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet $KUBELET_ARGS
Restart=on-failure
#Type=notify
#LimitNOFILE=65536


[Install]
WantedBy=multi-user.target

cat /usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-proxy Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.service
Wants=network.service


[Service]
EnvironmentFile=/etc/kubernetes/kube-proxy
ExecStart=/usr/bin/kube-proxy $KUBE_PROXY_ARGS
Restart=on-failure
#Type=notify
LimitNOFILE=65536


[Install]
WantedBy=multi-user.target