kubeasz 安装 Kubernetes 1.23.1 集群

kubeasz 安装 Kubernetes 1.23.1 集群

`kubeasz` 安装 `Kubernetes 1.23.1` 集群

kubeasz 为 GitHub 上开源项目，内部使用 ansible 自动安装 k8s 集群

服务	版本
CentOS	7.8
Docker	20.10.9
Kubernetes	1.23.1
Ansible	2.9.27
Kubeasz	3.2.0

文档使用的 kubeasz 文档存放在 GitHub

集群规划

节点名称	IP地址	节点角色
m1	192.168.0.200	master, ansible, kubeasz
m2	192.168.0.201	master
m3	192.168.0.202	master
n1	192.168.0.205	node
n2	192.168.0.206	node
n3	192.168.0.207	node
n4	192.168.0.208	node

获取 `kubeasz` 代码

mastr 节点存放 kubeasz 代码

[root@localhost ~]# yum install -y lrzsz vim tree unzip

从 GitHub 上下载 3.2.0 版本代码，并上传服务器

[root@localhost ~]# cd /data/
[root@localhost data]# mv /tmp/kubeasz-3.2.0.zip ./
[root@localhost data]# unzip kubeasz-3.2.0.zip

准备工作

修改主机名

所有节点修改主机名

$ hostnamectl set-hostname [m1|m2|m3|n1|n2|n3|n4]
$ bash

设置免密登陆

master 节点设置其他节点免密登陆

# 创建密钥
[root@m1 data]# ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):    
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:i/OYoJVEAsXiQN8BLze89lP0Jp0TwMtoMQhOkYQifaM root@m1
The key's randomart image is:
+---[RSA 2048]----+
|o=+*=.. ..       |
|=o*.*..o ..      |
|=..*.B  =...     |
| .Eoo oo.oo o    |
|    .o. So *     |
|   .......o .    |
|    + oo.        |
|   o . =.        |
|  .   o .        |
+----[SHA256]-----+

# cp 密钥到对应主机
[root@m1 data]# ssh-copy-id [m1|m2|m3]
root@m1's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'm1'"
and check to make sure that only the key(s) you wanted were added.

[root@m1 data]# ssh-copy-id [n1|n2|n3|n4]

# 测试免密登陆
[root@m1 data]# ssh n2
Last login: Tue Nov 22 15:00:42 2022 from 192.168.100.133
[root@n2 ~]# hostname
n2

修改 `/etc/hosts` 文件

master 节点设置后，分发给其他节点

[root@m1 data]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.0.200 m1
192.168.0.201 m2
192.168.0.202 m3

192.168.0.205 n1
192.168.0.206 n2
192.168.0.207 n3
192.168.0.208 n4

分发 /etc/hosts 文件

[root@m1 data]# for i in  {1,2,5,6,7,8}; do scp /etc/hosts 192.168.0.20${i}:/etc/hosts ; done
hosts                                100%  254   387.2KB/s   00:00    
hosts                                100%  254   335.7KB/s   00:00    
hosts                                100%  254   325.7KB/s   00:00    
hosts                                100%  254   369.4KB/s   00:00    
hosts                                100%  254   325.7KB/s   00:00    
hosts                                100%  254   369.4KB/s   00:00 

# 检查 /etc/hosts 信息
[root@m1 data]# ssh n1
cLast login: Tue Nov 22 15:02:18 2022 from 192.168.100.133

[root@n1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.0.200 m1
192.168.0.201 m2
192.168.0.202 m3

192.168.0.205 n1
192.168.0.206 n2
192.168.0.207 n3
192.168.0.208 n4

安装 ansible 服务

master 节点安装 ansible

# 安装 epel 源
[root@m1 data]# yum -y install epel-release

# 安装 ansible 服务
[root@m1 data]# yum -y install ansible

# 查看服务
[root@m1 data]# ansible --version
ansible 2.9.27
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Oct 14 2020, 14:45:30) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

安装 `Kubernetes` 集群

下载项目源码、二进制及离线镜像

[root@m1 data]# cd kubeasz-3.2.0
[root@m1 kubeasz-3.2.0]# ./ezdown -D

上述脚本运行成功后，所有文件（ kubeasz 代码、二进制、离线镜像）均已整理好放入目录 /etc/kubeasz

创建集群配置实例

[root@m1 kubeasz-3.2.0]# ./ezctl new k8s
2023-04-20 15:09:47 DEBUG generate custom cluster files in /etc/kubeasz/clusters/k8s
2023-04-20 15:09:47 DEBUG set version of common plugins
2023-04-20 15:09:47 DEBUG cluster k8s: files successfully created.
2023-04-20 15:09:47 INFO next steps 1: to config '/etc/kubeasz/clusters/k8s/hosts'
2023-04-20 15:09:47 INFO next steps 2: to config '/etc/kubeasz/clusters/k8s/config.yml'

然后根据提示配置 /etc/kubeasz/clusters/k8s/hosts 和 /etc/kubeasz/clusters/k8s/config.yml
根据前面节点规划修改 hosts 文件和其他集群层面的主要配置选项
其他集群组件等配置项可以在 config.yml 文件中修改。

集群信息确定

GitHub文档地址后续文档的配置信息，可直接查看 GitHub 仓库

/etc/kubeasz/clusters/k8s/hosts 文件信息

# 'etcd' cluster should have odd member(s) (1,3,5,...)
[etcd]
192.168.0.200
192.168.0.201
192.168.0.202

# master node(s)
[kube_master]
192.168.0.200
192.168.0.201
192.168.0.202


# work node(s)
[kube_node]
192.168.0.205
192.168.0.206
192.168.0.207
192.168.0.208

# [optional] harbor server, a private docker registry
# 'NEW_INSTALL': 'true' to install a harbor server; 'false' to integrate with existed one
[harbor]
#192.168.1.8 NEW_INSTALL=false

# [optional] loadbalance for accessing k8s from outside
[ex_lb]
#192.168.1.6 LB_ROLE=backup EX_APISERVER_VIP=192.168.1.250 EX_APISERVER_PORT=8443
#192.168.1.7 LB_ROLE=master EX_APISERVER_VIP=192.168.1.250 EX_APISERVER_PORT=8443

# [optional] ntp server for the cluster
#########################################################
# 未启用 chrony 时钟同步，使用公有云服务器，本身自带了chrony 同步
# 并且默认的 chronyd 版本和云上版本不一致，导致时钟查看存在问题
#########################################################
[chrony]


[all:vars]
# --------- Main Variables ---------------
# Secure port for apiservers
SECURE_PORT="6443"

# Cluster container-runtime supported: docker, containerd
CONTAINER_RUNTIME="docker"

# Network plugins supported: calico, flannel, kube-router, cilium, kube-ovn
#########################################################################################
# 此处网络模型选择 flannel 配合后续的 config.yaml 文件
# 使用的是 vlan DIRECT_ROUTING 模式，等同于 hostgw 模式
# 也可以使用网络插件：calico 等，我们选择 flannel 是因为阿里云的 192.168.0.0/24 网段
# vpc 过老，不支持 ipip 模式，无法部署 calico ipip 模式和 flannel vlan 非 DIRECT_ROUTING 模式
# 例如阿里云 10.0.0.0/24 网段，就是使用的 calico ipip 模式
#########################################################################################    
CLUSTER_NETWORK="flannel"

# Service proxy mode of kube-proxy: 'iptables' or 'ipvs'
PROXY_MODE="ipvs"

# K8S Service CIDR, not overlap with node(host) networking
SERVICE_CIDR="192.168.100.0/22"

# Cluster CIDR (Pod CIDR), not overlap with node(host) networking
CLUSTER_CIDR="192.168.192.0/20"

# NodePort Range
NODE_PORT_RANGE="20000-49999"

# Cluster DNS Domain
CLUSTER_DNS_DOMAIN="cluster.local"

# -------- Additional Variables (don't change the default value right now) ---
# Binaries Directory
bin_dir="/opt/kube/bin"

# Deploy Directory (kubeasz workspace)
base_dir="/etc/kubeasz"

# Directory for a specific cluster
cluster_dir="{{ base_dir }}/clusters/k8s"

# CA and other components cert/key Directory
ca_dir="/etc/kubernetes/ssl"

/etc/kubeasz/clusters/k8s/config.yml 文件信息

############################
# prepare
############################
# 可选离线安装系统软件包 (offline|online)
INSTALL_SOURCE: "online"

# 可选进行系统安全加固 github.com/dev-sec/ansible-collection-hardening
OS_HARDEN: false

# 设置时间源服务器【重要：集群内机器时间必须同步】
ntp_servers:
  - "ntp1.aliyun.com"
  - "time1.cloud.tencent.com"
  - "0.cn.pool.ntp.org"

# 设置允许内部时间同步的网络段，比如"10.0.0.0/8"，默认全部允许
local_network: "192.168.0.0/24"


############################
# role:deploy
############################
# default: ca will expire in 100 years
# default: certs issued by the ca will expire in 50 years
CA_EXPIRY: "876000h"
CERT_EXPIRY: "438000h"

# kubeconfig 配置参数
CLUSTER_NAME: "prod"
CONTEXT_NAME: "context-{{ CLUSTER_NAME }}"

# k8s version
K8S_VER: "1.23.1"

############################
# role:etcd
############################
# 设置不同的wal目录，可以避免磁盘io竞争，提高性能
ETCD_DATA_DIR: "/var/lib/etcd"
ETCD_WAL_DIR: ""


############################
# role:runtime [containerd,docker]
############################
# ------------------------------------------- containerd
# [.]启用容器仓库镜像
ENABLE_MIRROR_REGISTRY: false

# [containerd]基础容器镜像
SANDBOX_IMAGE: "easzlab/pause:3.6"

# [containerd]容器持久化存储目录
CONTAINERD_STORAGE_DIR: "/var/lib/containerd"

# ------------------------------------------- docker
# [docker]容器存储目录
DOCKER_STORAGE_DIR: "/data/docker"

# [docker]开启Restful API
ENABLE_REMOTE_API: false

# [docker]信任的HTTP仓库
INSECURE_REG: '["127.0.0.1/8", "harbor.evescn.com"]'


############################
# role:kube-master
############################
# k8s 集群 master 节点证书配置，可以添加多个ip和域名（比如增加公网ip和域名）
MASTER_CERT_HOSTS:
  - "k8s.evescn.com"
  #- "www.test.com"

# node 节点上 pod 网段掩码长度（决定每个节点最多能分配的pod ip地址）
# 如果flannel 使用 --kube-subnet-mgr 参数，那么它将读取该设置为每个节点分配pod网段
# https://github.com/coreos/flannel/issues/847
################################################################
# 修改了 NODE_CIDR_LEN，使用 fannel 模式，子网设置过大担心 ip 不够用
# CLUSTER_CIDR="192.168.192.0/20" 
# NODE_CIDR_LEN: 25
# 我们的 NODE 节点能部署 2^5 = 32台，每台配置: 16c/32g，部署业务足够了
# 如果使用默认 24，那 NODE 节点能部署 2^4 = 16 台，担心业务不够用
################################################################
NODE_CIDR_LEN: 25


############################
# role:kube-node
############################
# Kubelet 根目录
KUBELET_ROOT_DIR: "/var/lib/kubelet"

# node节点最大pod 数
################################################################
# NODE_CIDR_LEN: 25
# 网段内的可用 ip = 128 -2 > 110，node 节点的每个 pod 都能分配 ip
################################################################
MAX_PODS: 110

# 配置为kube组件（kubelet,kube-proxy,dockerd等）预留的资源量
# 数值设置详见templates/kubelet-config.yaml.j2
################################################################
# 启用了资源预留，role 文件中需要修改，否则服务部署报错，详细看 其他修改项目
################################################################
KUBE_RESERVED_ENABLED: "yes"

# k8s 官方不建议草率开启 system-reserved, 除非你基于长期监控，了解系统的资源占用状况；
# 并且随着系统运行时间，需要适当增加资源预留，数值设置详见templates/kubelet-config.yaml.j2
# 系统预留设置基于 4c/8g 虚机，最小化安装系统服务，如果使用高性能物理机可以适当增加预留
# 另外，集群安装时候apiserver等资源占用会短时较大，建议至少预留1g内存
SYS_RESERVED_ENABLED: "no"

# haproxy balance mode
BALANCE_ALG: "roundrobin"


############################
# role:network [flannel,calico,cilium,kube-ovn,kube-router]
############################
# ------------------------------------------- flannel
# [flannel]设置flannel 后端"host-gw","vxlan"等
###########################################################################################
# 此处使用 vlan DIRECT_ROUTING 模式，是因为阿里云的 192.168.0.0/24 网段 vpc 过老，不支持 ipip 模式
###########################################################################################  
FLANNEL_BACKEND: "vxlan"
# DIRECT_ROUTING: false
DIRECT_ROUTING: true

# [flannel] flanneld_image: "quay.io/coreos/flannel:v0.10.0-amd64"
flannelVer: "v0.15.1"
flanneld_image: "easzlab/flannel:{{ flannelVer }}"

# [flannel]离线镜像tar包
flannel_offline: "flannel_{{ flannelVer }}.tar"

# ------------------------------------------- calico
# [calico]设置 CALICO_IPV4POOL_IPIP=“off”,可以提高网络性能，条件限制详见 docs/setup/calico.md
CALICO_IPV4POOL_IPIP: "off"

# [calico]设置 calico-node使用的host IP，bgp邻居通过该地址建立，可手工指定也可以自动发现
IP_AUTODETECTION_METHOD: "can-reach={{ groups['kube_master'][0] }}"

# [calico]设置calico 网络 backend: brid, vxlan, none
CALICO_NETWORKING_BACKEND: "brid"

# [calico]更新支持calico 版本: [v3.3.x] [v3.4.x] [v3.8.x] [v3.15.x]
calico_ver: "v3.19.3"

# [calico]calico 主版本
calico_ver_main: "{{ calico_ver.split('.')[0] }}.{{ calico_ver.split('.')[1] }}"

# [calico]离线镜像tar包
calico_offline: "calico_{{ calico_ver }}.tar"

# ------------------------------------------- cilium
# [cilium]CILIUM_ETCD_OPERATOR 创建的 etcd 集群节点数 1,3,5,7...
ETCD_CLUSTER_SIZE: 1

# [cilium]镜像版本
cilium_ver: "v1.4.1"

# [cilium]离线镜像tar包
cilium_offline: "cilium_{{ cilium_ver }}.tar"

# ------------------------------------------- kube-ovn
# [kube-ovn]选择 OVN DB and OVN Control Plane 节点，默认为第一个master节点
OVN_DB_NODE: "{{ groups['kube_master'][0] }}"

# [kube-ovn]离线镜像tar包
kube_ovn_ver: "v1.5.3"
kube_ovn_offline: "kube_ovn_{{ kube_ovn_ver }}.tar"

# ------------------------------------------- kube-router
# [kube-router]公有云上存在限制，一般需要始终开启 ipinip；自有环境可以设置为 "subnet"
OVERLAY_TYPE: "full"

# [kube-router]NetworkPolicy 支持开关
FIREWALL_ENABLE: "true"

# [kube-router]kube-router 镜像版本
kube_router_ver: "v0.3.1"
busybox_ver: "1.28.4"

# [kube-router]kube-router 离线镜像tar包
kuberouter_offline: "kube-router_{{ kube_router_ver }}.tar"
busybox_offline: "busybox_{{ busybox_ver }}.tar"


############################
# role:cluster-addon
############################
# coredns 自动安装
dns_install: "yes"
corednsVer: "1.8.6"
ENABLE_LOCAL_DNS_CACHE: true
dnsNodeCacheVer: "1.21.1"
# 设置 local dns cache 地址
LOCAL_DNS_CACHE: "169.254.20.10"

# metric server 自动安装
metricsserver_install: "yes"
metricsVer: "v0.5.2"

# dashboard 自动安装
dashboard_install: "no"
dashboardVer: "v2.4.0"
dashboardMetricsScraperVer: "v1.0.7"

# ingress 自动安装
############################################################################
# 修改了 traefik 版本，主要是 10.3.0 这个版本好像 traefik 不支持跨名称空间访问 svc
# 升级为了 10.9.0 版本，部署的 yaml 文件也进行了修改，详细的看后面 其他修改项目
############################################################################
ingress_install: "yes"
ingress_backend: "traefik"
# traefik_chart_ver: "10.3.0"
traefik_chart_ver: "10.9.0"

# prometheus 自动安装
prom_install: "no"
prom_namespace: "monitor"
prom_chart_ver: "12.10.6"

# nfs-provisioner 自动安装
##################################################################################
# 启用了 nfs 部署，nfs_server 可以替换为云上对于的 nas 服务地址，使用云 nas 而不是自建 nas
##################################################################################
nfs_provisioner_install: "yes"
nfs_provisioner_namespace: "kube-system"
nfs_provisioner_ver: "v4.0.2"
nfs_storage_class: "nfs-client-storageclass"
nfs_server: "10.0.0.100"
nfs_path: "/"

############################
# role:harbor
############################
# harbor version，完整版本号
HARBOR_VER: "v2.1.3"
HARBOR_DOMAIN: "harbor.yourdomain.com"
HARBOR_TLS_PORT: 8443

# if set 'false', you need to put certs named harbor.pem and harbor-key.pem in directory 'down'
HARBOR_SELF_SIGNED_CERT: false

# install extra component
HARBOR_WITH_NOTARY: false
HARBOR_WITH_TRIVY: false
HARBOR_WITH_CLAIR: false
HARBOR_WITH_CHARTMUSEUM: true

其他修改项目
- 启用资源预留的修改

KUBE_RESERVED_ENABLED: "yes"

    # roles/kube-node/templates/kubelet-config.yaml.j2
    # 修改前配置详情
    enforceNodeAllocatable:
    - pods
    {% if KUBE_RESERVED_ENABLED == "yes" %}
    - kube-reserved
    {% endif %}
    {% if SYS_RESERVED_ENABLED == "yes" %}
    - system-reserved
    {% endif %}

    # 修改后配置详情，删除了 KUBE_RESERVED_ENABLED == yes 的选项
    enforceNodeAllocatable:
    - pods
    {% if SYS_RESERVED_ENABLED == "yes" %}
    - system-reserved
    {% endif %}

其他修改项目
- traefik 升级为 10.9.0 版本

# 下载 10.9.0 版本 traefik 
helm repo add traefik https://traefik.github.io/charts
helm pull traefik/traefik --version 10.9.0
mv traefik-10.9.0.tgz /etc/kubeasz/roles/cluster-addon/files

# 修改 traefik 文件，helm 部署的 values.yaml 文件信息，启用跨名称空间访问 svc
# /etc/kubeasz/roles/cluster-addon/templates/traefik/values.yaml.j2

roles/cluster-addon/templates/traefik/values.yaml.j2

一键安装

# 自动执行 01-07 playbook 脚本
[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s all

或者分布安装

[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s 01
[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s 02
[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s 03
[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s 04
[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s 05
[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s 06
[root@m1 kubeasz-3.2.0]# ./ezctl setup k8s 07

分布安装 01 02 03 ...... 分布对应 playbooks 目录下 playbook 脚本

[root@m1 kubeasz-3.2.0]# ls playbooks/ -lh
total 88K
-rw-r--r--. 1 root root  443 Mar 28  2021 01.prepare.yml
-rw-r--r--. 1 root root   58 Mar 28  2021 02.etcd.yml
-rw-r--r--. 1 root root  209 Mar 28  2021 03.runtime.yml
-rw-r--r--. 1 root root  470 Mar 28  2021 04.kube-master.yml
-rw-r--r--. 1 root root  140 Mar 28  2021 05.kube-node.yml
-rw-r--r--. 1 root root  408 Mar 28  2021 06.network.yml
-rw-r--r--. 1 root root   77 Mar 28  2021 07.cluster-addon.yml
-rw-r--r--. 1 root root   34 Mar 28  2021 10.ex-lb.yml
-rw-r--r--. 1 root root 3.9K Mar 28  2021 11.harbor.yml
-rw-r--r--. 1 root root 1.6K Mar 28  2021 21.addetcd.yml
-rw-r--r--. 1 root root 1.5K Mar 28  2021 22.addnode.yml
-rw-r--r--. 1 root root 1.1K Mar 28  2021 23.addmaster.yml
-rw-r--r--. 1 root root 3.0K Mar 28  2021 31.deletcd.yml
-rw-r--r--. 1 root root 1.3K Mar 28  2021 32.delnode.yml
-rw-r--r--. 1 root root 1.4K Mar 28  2021 33.delmaster.yml
-rw-r--r--. 1 root root 1.8K Mar 28  2021 90.setup.yml
-rw-r--r--. 1 root root 1.2K Mar 28  2021 91.start.yml
-rw-r--r--. 1 root root 1.1K Mar 28  2021 92.stop.yml
-rw-r--r--. 1 root root 1.1K Mar 28  2021 93.upgrade.yml
-rw-r--r--. 1 root root 1.8K Mar 28  2021 94.backup.yml
-rw-r--r--. 1 root root  999 Mar 28  2021 95.restore.yml
-rw-r--r--. 1 root root  337 Mar 28  2021 99.clean.yml

ezctl 脚本部分代码

......
function usage-setup(){
  echo -e "\033[33mUsage:\033[0m ezctl setup <cluster> <step>"
  cat <<EOF
available steps:
    01  prepare            to prepare CA/certs & kubeconfig & other system settings
    02  etcd               to setup the etcd cluster
    03  container-runtime  to setup the container runtime(docker or containerd)
    04  kube-master        to setup the master nodes
    05  kube-node          to setup the worker nodes
    06  network            to setup the network plugin
    07  cluster-addon      to setup other useful plugins
    90  all                to run 01~07 all at once
    10  ex-lb              to install external loadbalance for accessing k8s from outside
    11  harbor             to install a new harbor server or to integrate with an existed one

examples: ./ezctl setup test-k8s 01  (or ./ezctl setup test-k8s prepare)
          ./ezctl setup test-k8s 02  (or ./ezctl setup test-k8s etcd)
          ./ezctl setup test-k8s all
EOF
}

......
function setup() {
    [[ -d "clusters/$1" ]] || { logger error "invalid config, run 'ezctl new $1' first"; return 1; }
    [[ -f "bin/kube-apiserver" ]] || { logger error "no binaries founded, run 'ezdown -D' fist"; return 1; }

    PLAY_BOOK="dummy.yml"
    case "$2" in
      (01|prepare)
          PLAY_BOOK="01.prepare.yml"
          ;;
      (02|etcd)
          PLAY_BOOK="02.etcd.yml"
          ;;
      (03|container-runtime)
          PLAY_BOOK="03.runtime.yml"
          ;;
      (04|kube-master)
          PLAY_BOOK="04.kube-master.yml"
          ;;
      (05|kube-node)
          PLAY_BOOK="05.kube-node.yml"
          ;;
      (06|network)
          PLAY_BOOK="06.network.yml"
          ;;
      (07|cluster-addon)
          PLAY_BOOK="07.cluster-addon.yml"
          ;;
      (90|all)
          PLAY_BOOK="90.setup.yml"
          ;;
      (10|ex-lb)
          PLAY_BOOK="10.ex-lb.yml"
          ;;
      (11|harbor)
          PLAY_BOOK="11.harbor.yml"
          ;;
      (*)
          usage-setup
          exit 1
          ;;
    esac

    logger info "cluster:$1 setup step:$2 begins in 5s, press any key to abort:\n"
    ! (read -r -t5 -n1) || { logger warn "setup abort"; return 1; }

    ansible-playbook -i "clusters/$1/hosts" -e "@clusters/$1/config.yml" "playbooks/$PLAY_BOOK" || return 1
}

检查集群

# 验证集群版本 
[root@m1 kubeasz-3.2.0]# kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2021-12-16T11:41:01Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2021-12-16T11:34:54Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}

# 验证节点就绪 (Ready) 状态
[root@m1 kubeasz-3.2.0]# kubectl get node 
NAME            STATUS                     ROLES    AGE   VERSION
192.168.0.200   Ready,SchedulingDisabled   master   10d   v1.23.1
192.168.0.201   Ready,SchedulingDisabled   master   10d   v1.23.1
192.168.0.202   Ready,SchedulingDisabled   master   10d   v1.23.1
192.168.0.205   Ready                      node     10d   v1.23.1
192.168.0.206   Ready                      node     10d   v1.23.1
192.168.0.207   Ready                      node     8d    v1.23.1
192.168.0.208   Ready                      node     8d    v1.23.1

# 验证集群pod状态，默认已安装网络插件、coredns、metrics-server等
[root@m1 kubeasz-3.2.0]# kubectl get pod -n kube-system 
NAME                                      READY   STATUS    RESTARTS   AGE
coredns-596755dbff-r57m5                  1/1     Running   0          9d
kube-flannel-ds-75x6v                     1/1     Running   0          10d
kube-flannel-ds-cm2f4                     1/1     Running   0          8d
kube-flannel-ds-dpmhz                     1/1     Running   0          8d
kube-flannel-ds-drk8b                     1/1     Running   0          10d
kube-flannel-ds-gzz72                     1/1     Running   0          10d
kube-flannel-ds-hjcqj                     1/1     Running   0          10d
kube-flannel-ds-n8x6r                     1/1     Running   0          10d
metrics-server-5d648558d9-tj4db           1/1     Running   0          10d
nfs-client-provisioner-59486ccf56-bfzzp   1/1     Running   0          10d
node-local-dns-6hmpj                      1/1     Running   0          9d
node-local-dns-8x6lc                      1/1     Running   0          9d
node-local-dns-kz7h7                      1/1     Running   0          8d
node-local-dns-s6xmg                      1/1     Running   0          9d
node-local-dns-vhfbr                      1/1     Running   0          9d
node-local-dns-w8xng                      1/1     Running   0          8d
node-local-dns-z2rxg                      1/1     Running   0          9d

# 验证集群服务状态
[root@m1 kubeasz-3.2.0]# kubectl get svc -A 
NAME                               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                        AGE
kubernetes                         ClusterIP   192.168.100.1    <none>        443/TCP                        10d
kube-dns                           ClusterIP   192.168.100.2    <none>        53/UDP,53/TCP,9153/TCP         10d
kube-dns-upstream                  ClusterIP   192.168.100.94   <none>        53/UDP,53/TCP                  10d
metrics-server                     ClusterIP   192.168.103.89   <none>        443/TCP                        10d
node-local-dns                     ClusterIP   None             <none>        9253/TCP                       10d

集群补充

新增 Node 节点

[root@m1 kubeasz-3.2.0]# ./ezctl -h
Usage: ezctl COMMAND [args]
-------------------------------------------------------------------------------------
Cluster setups:
    list                             to list all of the managed clusters
    checkout    <cluster>            to switch default kubeconfig of the cluster
    new         <cluster>            to start a new k8s deploy with name 'cluster'
    setup       <cluster>  <step>    to setup a cluster, also supporting a step-by-step way
    start       <cluster>            to start all of the k8s services stopped by 'ezctl stop'
    stop        <cluster>            to stop all of the k8s services temporarily
    upgrade     <cluster>            to upgrade the k8s cluster
    destroy     <cluster>            to destroy the k8s cluster
    backup      <cluster>            to backup the cluster state (etcd snapshot)
    restore     <cluster>            to restore the cluster state from backups
    start-aio                        to quickly setup an all-in-one cluster with 'default' settings

Cluster ops:
    add-etcd    <cluster>  <ip>      to add a etcd-node to the etcd cluster
    add-master  <cluster>  <ip>      to add a master node to the k8s cluster
    add-node    <cluster>  <ip>      to add a work node to the k8s cluster
    del-etcd    <cluster>  <ip>      to delete a etcd-node from the etcd cluster
    del-master  <cluster>  <ip>      to delete a master node from the k8s cluster
    del-node    <cluster>  <ip>      to delete a work node from the k8s cluster

Extra operation:
    kcfg-adm    <cluster>  <args>    to manage client kubeconfig of the k8s cluster

Use "ezctl help <command>" for more information about a given command.

# 新增 2台 node 节点
[root@m1 kubeasz-3.2.0]# ./ezctl add-node k8s 192.168.0.209
[root@m1 kubeasz-3.2.0]# ./ezctl add-node k8s 192.168.0.210

posted @ 2023-04-20 16:11 evescn 阅读(1090) 评论(0) 编辑收藏举报

刷新页面返回顶部

evescn

kubeasz 安装 Kubernetes 1.23.1 集群

kubeasz 安装 Kubernetes 1.23.1 集群

集群规划

获取 kubeasz 代码

准备工作

修改主机名

设置免密登陆

修改 /etc/hosts 文件

安装 ansible 服务

安装 Kubernetes 集群

下载项目源码、二进制及离线镜像

创建集群配置实例

集群信息确定

一键安装

检查集群

集群补充

公告

`kubeasz` 安装 `Kubernetes 1.23.1` 集群

获取 `kubeasz` 代码

修改 `/etc/hosts` 文件

安装 `Kubernetes` 集群