随笔 - 366  文章 - 0  评论 - 101  阅读 - 30万

kubeadm搭建k8s 3master 高可用 keepalived + haproxy

?
1
2
3
4
5
192.168.1.222 k8s-master11 m1
192.168.1.223 k8s-master12 m2
192.168.1.224 k8s-master13 m3
192.168.1.225 k8s-node01 n1
192.168.1.226 k8s-node02 n2

keepalived和haproxy装在n1和n2上，VIP为   192.168.1.123 

haproxy

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
yum -y install haproxy keepalived
[root@k8s-node01 docker]# cat /etc/haproxy/haproxy.cfg
global
    log 127.0.0.1 local3 info
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000
#---------------------------------------------------------------------
frontend  k8s-api
   bind *:6443
   mode tcp
   default_backend             apiserver
#---------------------------------------------------------------------
backend apiserver
    balance     roundrobin
    mode tcp
    server  k8s-master11 192.168.1.222:6443 check weight 1 maxconn 2000 check inter 2000 rise 2 fall 3
    server  k8s-master12 192.168.1.223:6443 check weight 1 maxconn 2000 check inter 2000 rise 2 fall 3
    server  k8s-master13 192.168.1.224:6443 check weight 1 maxconn 2000 check inter 2000 rise 2 fall 3

　keepalived

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
[root@k8s-node01 docker]# cat /etc/keepalived/keepalived.conf
vrrp_script check_haproxy {
    script "/etc/keepalived/check_haproxy.sh"
    interval 3
}
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
    192.168.1.123
    }
     track_script {
        check_haproxy
     }
}
 
 
 
 
[root@k8s-node01 keepalived]# cat check_haproxy.sh
#!/bin/bash
systemctl status haproxy > /dev/null
if [[ $? != 0 ]];then
        echo "haproxy is down,close the keepalived"
        systemctl stop keepalived
fi

　　准备工作 参考  https://www.cnblogs.com/hanxing/p/16028100.html    （到 kubeadm init --config=kubeadm-config.yaml   之前）

?
1
kubeadm-config
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
[root@k8s-master11 ~]# cat kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.1.222    #这里别写vip，就写m1的ip
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master11
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  certSANs:
  - "192.168.1.123"         #VIP地址
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
#imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers   #阿里云的镜像站点
controlPlaneEndpoint: "192.168.1.123:6443"    #VIP的地址和端口
kind: ClusterConfiguration
kubernetesVersion: v1.15.1
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
  podSubnet: "10.244.0.0/16"
  scheduler: {}
---
  apiVersion: kubeproxy.config.k8s.io/v1alpha1
  kind: KubeProxyConfiguration
  featureGates:
  SupportIPVSProxyMode: true
  mode: ipvs

　

 m1操作

 kubeadm init --config /root/kubeadm-config.yaml --upload-certs 

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
[root@k8s-master11 ~]# kubeadm init --config=kubeadm-config.yaml --experimental-upload-certs     
Flag --experimental-upload-certs has been deprecated, use --upload-certs instead
W0327 13:21:37.385237   58645 strict.go:54] error unmarshaling configuration schema.GroupVersionKind{Group:"kubeadm.k8s.io", Version:"v1beta2", Kind:"ClusterConfiguration"}: error unmarshaling JSON: while decoding JSON: json: unknown field "scheduler"
W0327 13:21:37.387762   58645 strict.go:54] error unmarshaling configuration schema.GroupVersionKind{Group:"kubeproxy.config.k8s.io", Version:"v1alpha1", Kind:"KubeProxyConfiguration"}: error unmarshaling JSON: while decoding JSON: json: unknown field "SupportIPVSProxyMode"
[init] Using Kubernetes version: v1.15.1
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.13. Latest validated version: 18.09
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master11 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.1.222 192.168.1.123 192.168.1.123]
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master11 localhost] and IPs [192.168.1.222 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master11 localhost] and IPs [192.168.1.222 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 45.547778 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
a5608f6eb8831204eeb138e9d2ae553a1c59c76c2c5547e3d41303c5f86974e0
[mark-control-plane] Marking the node k8s-master11 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node k8s-master11 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: abcdef.0123456789abcdef
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
 
Your Kubernetes control-plane has initialized successfully!
 
To start using your cluster, you need to run the following as a regular user:
 
  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
 
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
 
You can now join any number of the control-plane node running the following command on each as root:
 
  kubeadm join 192.168.1.123:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:dd05586525feb49ccd6e19b894b6a30d1f7d0ee5ef8cc9345ec63665f8011fd5 \
    --control-plane --certificate-key a5608f6eb8831204eeb138e9d2ae553a1c59c76c2c5547e3d41303c5f86974e0
 
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
 
Then you can join any number of worker nodes by running the following on each as root:
 
kubeadm join 192.168.1.123:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:dd05586525feb49ccd6e19b894b6a30d1f7d0ee5ef8cc9345ec63665f8011fd5

　　

m2和m3操作上面日志的第一个join，n1和n2操作第二个join，再分别执行

 

?
1
mkdir -p $HOME/.kube ;ln -s  /etc/kubernetes/admin.conf  $HOME/.kube/config

　　

m1操作flannel

 

?
1
2
3
4
5
6
7
8
9
10
11
[root@k8s-master11 ~]# kubectl apply -f fla.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created

　fla.yml 内容如下  这里用的是 kube-flannel-ds-amd64

 

　　

搭建完毕，检查集群

 

?
1
2
3
4
5
6
7
[root@k8s-master11 ~]# kubectl get node
NAME           STATUS   ROLES    AGE    VERSION
k8s-master11   Ready    master   106m   v1.15.1
k8s-master12   Ready    master   100m   v1.15.1
k8s-master13   Ready    <none>   95m    v1.15.1
k8s-node01     Ready    <none>   95m    v1.15.1
k8s-node02     Ready    <none>   95m    v1.15.1

　　

　

posted on   寒星12345678999  阅读(651)  评论(0)  编辑  收藏  举报
相关博文：
· Kubernetes k8s 安装 配置
· k8s 添加master （删除m1 m2 添加 m4 m5）
· kubeadm部署k8s集群
· kubeadm 搭建 K8s
· 部署双主高可用K8S集群
阅读排行：
· DeepSeek 开源周回顾「GitHub 热点速览」
· 物流快递公司核心技术能力-地址解析分单基础技术分享
· .NET 10首个预览版发布：重大改进与新特性概览！
· AI与.NET技术实操系列（二）：开始使用ML.NET
· .NET10 - 预览版1新功能体验（一）
历史上的今天：
2015-03-27 必看
2015-03-27 面试题收集
< 2025年3月 >
日 一 二 三 四 五 六
23 24 25 26 27 28 1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31 1 2 3 4 5

Powered by: 博客园 Copyright © 2025 寒星12345678999
Powered by .NET 9.0 on Kubernetes
点击右上角即可分享
微信分享提示
AI FOR CODE 大赛