kubernetes单Master集群部署-多Master集群部署(8)

一、多Master集群部署

1、拷贝文件

 从Master01上拷贝相关文件：

 

服务文件：

修改配置文件kube-apiserver：10.11.97.191为master02的IP

KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://10.11.97.191:2379,https://10.11.97.192:2379,https://10.11.97.71:2379 \
--bind-address=10.11.97.191 \
--secure-port=6443 \
--advertise-address=10.11.97.191 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--kubelet-https=true \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/opt/kubernetes/ssl/server.pem  \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"

　　

2、启动服务

systemctl start kube-apiserver

systemctl start kube-controller-manager

systemctl start kube-scheduler

服务启动成功。

 

3、apiserver高可用配置

通过Nginx进行4层负载均衡：

stream {
        log_format  main  "$remote_addr $upstream_addr $time_local $status";
        access_log /var/log/nginx/k8s-access.log main;
        upstream k8s-apiserver {
           server 10.11.97.191:6443;
           server 10.11.97.192:6443;
      }

        server {
                listen 0.0.0.0:6443;
                proxy_pass k8s-apiserver;
        }
}

　　这样，就使用Nginx的https://10.11.97.191:6443替代了apiserver的地址了。

修改Node节点连接apiserver的地址：

bootstrap.kubeconfig

kubelet.kubeconfig

kube-proxy.kubeconfig

修改地方：apiserverIP统一修改为

server: https://10.11.97.191:6443

　　

重启Node节点的服务。

4、LB高可用

使用keepalived对LB进行高可用配置。

keepalived的Master节点：

IP地址：

 

 cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     xuequn@kingsoft.com
   }
   notification_email_from zabbix@kingsoft.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx_web {

script "/etc/keepalived/check_nginx.sh"

}



vrrp_instance VI_NGINX_WEB {
    state BACKUP
    interface eth0
    virtual_router_id 60
    priority 190
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass ZabbixServer
    }
   track_script {
   
    chk_nginx_web

   }
    virtual_ipaddress {
        10.11.97.187
    }

}

　　

keepalived的Slave节点：

IP地址：

cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     xuequn@kingsoft.com
   }
   notification_email_from zabbix@kingsoft.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx_web {

script "/etc/keepalived/check_nginx.sh"

}



vrrp_instance VI_NGINX_WEB {
    state MASTER
    interface eth0
    virtual_router_id 60
    priority 200
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass ZabbixServer
    }
   track_script {
   
    #chk_nginx_web

   }
    virtual_ipaddress {
        10.11.97.187
    }

}

　　

健康检查脚本：

cat  /etc/keepalived/check_nginx.sh

#!/bin/bash
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")

if [ "$count" -eq 0 ];then
    systemctl stop keepalived
fi

　　

启动keepalived集群后，vip绑定在master上：

至此，k8s集群多Master高可用，LB高可用、Etcd高可用都已完成，其中任何一个节点挂掉都不会影响正常业务。