KubeSphere 3.2.0 离线安装步骤

一、简介

KubeSphere 是在 Kubernetes 之上构建的以应用为中心的多租户容器平台，提供全栈的 IT 自动化运维的能力，简化企业的 DevOps 工作流。KubeSphere 提供了 运维友好的向导式操作界面，帮助企业快速构建一个强大和功能丰富的容器云平台，包括 Kubernetes 资源管理、DevOps (CI/CD)、应用生命周期管理、微服务治理 (Service Mesh)、多租户管理、监控日志、告警通知、存储与网络管理、GPU support 等功能，未来还将提供 多集群管理、Network Policy、镜像仓库管理 等功能。KubeSphere 愿景是打造一个基于 Kubernetes 的云原生分布式操作系统，它的架构可以很方便地与云原生生态系统进行即插即用（plug-and-play）的集成。

二、环境信息

两台机器，一台用于下载安装包，一台用于部署

role	IP	hostname	desc
packer	192.168.1.100	packer	可联网下载软件包
master、worker、registry	192.168.1.101	k8s	主节点、工作节点、镜像

三、部署步骤

在联网的机器上下载所需文件

使用国内yum镜像源

#使用国内yum镜像源
sudo mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup

sudo curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo

sudo sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo

sudo yum clean all

sudo yum makecache

online 下载yum安装的一些工具,以便捷安装docker,单独保存到online目录下,离线集群可不安装此目录下的包
```
sudo yum install -y yum-utils device-mapper-persistent-data lvm2 --downloadonly --downloaddir=/opt/software/package/online
```

下载一些安装k8s所需要的基础包,保存在k8s目录下

sudo yum install -y chrony openssl openssl-devel socat epel-release conntrack-tools --downloadonly --downloaddir=/opt/software/kubesphere/package/k8s

安装yum工具

sudo rpm -Uvh --force --nodeps /opt/software/package/online/*.rpm

安装时间同步工具,有些https链接时间不一直,会无法下载

修改 chrony配置文件,设置国内时间服务器
sudo vi /etc/chrony.conf
#注释删除原来的,增加国内
server ntp1.aliyun.com iburst
server ntp2.aliyun.com iburst
server time1.cloud.tencent.com iburst
server time2.cloud.tencent.com iburst

重启chronyd以生效配置,并设置开机启动

sudo systemctl restart chronyd && systemctl enable chronyd

设置时区

sudo timedatectl set-ntp true && timedatectl set-timezone Asia/Shanghai

检查是否可用
```
sudo chronyc activity -v
```

添加docker源

sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

sudo sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo

更新yum 元信息
```
sudo yum makecache fast
```

保存docker安装包

sudo yum -y install docker-ce --downloadonly --downloaddir=/opt/software/kubesphere/package/k8s

安装docker

sudo rpm -Uvh --force --nodeps /opt/software/kubesphere/package/k8s/*.rpm

设置开机启动docker

sudo systemctl restart docker && systemctl enable docker

docker镜像加速配置

此配置可到阿里云容器镜像服务-镜像工具-镜像加速器中获取

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://xxxx.mirror.aliyuncs.com"]
}
EOF

重启docker及守护进程

sudo systemctl daemon-reload && systemctl restart docker

拉取并保存registry2镜像仓库文件

sudo docker pull registry:2.7.1
mkdir -p /opt/software/kubesphere && cd /opt/software/kubesphere
sudo docker save -o /opt/software/kubesphere/docker.registry-2.7.1.tar registry:2.7.1

下载k8s安装包及拉取docker镜像

curl -L -O https://github.com/kubesphere/ks-installer/releases/download/v3.2.0/offline-installation-tool.sh

下载镜像清单文件

curl -L -O https://github.com/kubesphere/ks-installer/releases/download/v3.2.0/images-list.txt

# 默认的images-list.txt缺少一些镜像,需要在##kubesphere-images下追加
kubesphere/pause:3.4.1
kubesphere/kube-apiserver:v1.21.5
kubesphere/kube-proxy:v1.21.5
kubesphere/kube-controller-manager:v1.21.5
kubesphere/kube-scheduler:v1.21.5
kubesphere/k8s-dns-node-cache:1.15.12
kubesphere/kubectl:v1.21.0
coredns/coredns:1.8.0
calico/cni:v3.20.0
calico/kube-controllers:v3.20.0
calico/node:v3.20.0
calico/pod2daemon-flexvol:v3.20.0
openebs/provisioner-localpv:2.10.1
openebs/linux-utils:2.10.0

下载kubekey

export KKZONE=cn;curl -sfL https://get-kk.kubesphere.io | VERSION=v1.2.0 sh -

给脚本赋予执行权限

chmod +x /opt/software/kubesphere/offline-installation-tool.sh

下载指定版本的k8s二进制文件

export KKZONE=cn;export KUBERNETES_VERSION="v1.21.5";./offline-installation-tool.sh -b

修改镜像清单文件
```
mv images-list.txt images-list-add.txt
```
下载kubesphere-images 根据images-list.txt配置，调用docker save 保存docker镜像文件
```
./offline-installation-tool.sh -s -l images-list-add.txt -d ./kubesphere-images
```

下载crictl

cd /opt/software/kubesphere/kubekey/v1.21.5/amd64
curl -L -O https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.22.0/crictl-v1.22.0-linux-amd64.tar.gz

制作registry文件

mkdir -p /opt/software/registry/docker/certs

生成默认ca

openssl genrsa -out /opt/software/kubesphere/docker/certs/ca.key 2048
openssl req -x509 -new -nodes -key /opt/software/kubesphere/docker/certs/ca.key -subj "/CN=ca.kubekey.local" -days 36500 -out /opt/software/kubesphere/docker/certs/ca.crt

生成证书

openssl req -new -sha256 \
    -key /opt/software/kubesphere/docker/certs/ca.key \
    -subj "/C=CN/ST=Beijing/L=Beijing/O=UnitedStack/OU=Devops/CN=dockerhub.kubekey.local" \
    -reqexts SAN \
    -config <(cat /etc/pki/tls/openssl.cnf \
        <(printf "[SAN]\nsubjectAltName=DNS:dockerhub.kubekey.local")) \
    -out /opt/software/kubesphere/docker/certs/domain.csr \
    -keyout /opt/software/kubesphere/docker/certs/domain.key

签名证书

openssl x509 -req -days 365000 \
    -in /opt/software/kubesphere/docker/certs/domain.csr \
    -CA /opt/software/kubesphere/docker/certs/ca.crt \
    -CAkey /opt/software/kubesphere/docker/certs/ca.key -CAcreateserial \
    -extfile <(printf "subjectAltName=DNS:dockerhub.kubekey.local") \
    -out /opt/software/kubesphere/docker/certs/domain.crt

打包离线部署包

cd /opt/software/
zip -q -r  kubesphere.zip  kubesphere/

离线机器部署操作

上传打包好的离线部署包

解压文件

mkdir -p /opt/software
unzip kubesphere.zip

tar -zxvf /opt/software/kubesphere/kubekey-v1.2.0-linux-amd64.tar.gz -C /opt/software/kubesphere/

备份文件

cp -r /opt/software/kubesphere/docker /opt/module/kubesphere
cp /opt/software/kubesphere/images-list-add.txt /opt/module/kubesphere
cp /opt/software/kubesphere/offline-installation-tool.sh /opt/module/kubesphere

私库的证书

mkdir -p /etc/docker/certs.d/dockerhub.kubekey.local

cp /opt/module/kubesphere/docker/certs/ca.crt /etc/docker/certs.d/dockerhub.kubekey.local/ca.crt

设置hosts

echo 192.168.1.101 dockerhub.kubekey.local >> /etc/hosts
echo 192.168.1.101 k8s >> /etc/hosts

关闭防火墙

systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld

关闭selinux

# 临时关闭
setenforce 0
#永久关闭
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/sysconfig/selinux
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config

关闭swap

swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab

允许iptables检查桥接流量

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sudo sysctl --system

安装k8s所需要的基础包

mkdir -p /opt/software/kubesphere
sudo rpm -Uvh --force --nodeps /opt/software/kubesphere/package/k8s/*.rpm

安装时间同步工具

#修改 chrony配置文件,设置国内时间服务器
sudo vi /etc/chrony.conf
#注释删除原来的,增加内网主节点的
server k8s100 iburst

重启chronyd以生效配置,并设置开机启动

sudo systemctl restart chronyd && systemctl enable chronyd

设置时区

sudo timedatectl set-ntp true && timedatectl set-timezone Asia/Shanghai

检查是否可用
```
sudo chronyc activity -v
```

上传下载好的docker安装包，并进行安装

wget https://download.docker.com/linux/static/stable/x86_64/docker-20.10.2.tgz

tar xvf docker-20.10.2.tgz

cd docker-20.10.2

sudo cp docker/* /usr/bin/

sudo dockerd &

docker info

将docker注册成系统服务

sudo vi /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
 
[Service]
Type=notify
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
 
[Install]
WantedBy=multi-user.target
<<<end file
systemctl start/stop docker
systemctl enable/disable docker

离线安装docker-compose

curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

chmod +x /usr/bin/docker-compose

docker-compose --version

开启docker服务

sudo systemctl start docker
sudo systemctl stop docker

设置开机启动
```
systemctl enable docker
```

导入镜像

docker load -i /opt/software/kubesphere/docker.registry-2.7.1.tar

启动registry2镜像私库

docker run -d \
  --restart=always \
  --name registry \
  -v /opt/module/kubesphere/docker/certs:/certs \
  -v /opt/module/kubesphere/docker/registry:/var/lib/registry \
  -e REGISTRY_HTTP_ADDR=0.0.0.0:443 \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/ca.key \
  -p 443:443 \
  registry:2.7.1

推送镜像至私有仓库

cd /opt/software/kubesphere

chmod +x offline-installation-tool.sh

./offline-installation-tool.sh -l images-list-add.txt -d ./kubesphere-images -r dockerhub.kubekey.local

到浏览器查看导入镜像的列表

https://dockerhub.kubekey.local/v2/_catalog

部署KubeSphere

修改config-sample.yaml文件

#修改hosts,roleGroups

spec:
  hosts:
  - {name: k8s, address: 192.168.1.100, internalAddress: 192.168.1.101, user: root, password: toor}
  roleGroups:
    etcd:
    - k8s
    master: 
    - k8s
    worker:
    - k8s
    
#修改registry
    privateRegistry: dockerhub.kubekey.local #指向镜像私库
    
#开启应用商店--可选
  openpitrix:
    store:
      enabled: true

拷贝文件

mkdir -p /opt/module/kubesphere
cp /opt/software/kubesphere/kk /opt/module/kubesphere
cp -r /opt/software/kubesphere/kubekey /opt/module/kubesphere
cp /opt/software/kubesphere/config-sample.yaml /opt/module/kubesphere

执行安装

cd /opt/module/kubesphere/

chmod +x kk

unset KKZONE

./kk create cluster -f config-sample.yaml

posted on 2022-08-23 14:23 白糖℃ 阅读(675) 评论(0) 编辑收藏举报

KubeSphere 3.2.0 离线安装步骤

一、简介

二、 环境信息

三、部署步骤

二、环境信息