kubesphere离线包制作和离线安装

离线包制作

1. 执行以下命令下载 KubeKey 并解压

   export KKZONE=cn
   curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -

2. 新建文件manifest.yaml并加入以下内容

   ---
   apiVersion: kubekey.kubesphere.io/v1alpha2
   kind: Manifest
   metadata:
     name: sample
   spec:
     arches:
     - amd64
     operatingSystems:
     - arch: amd64
       type: linux
       id: centos
       version: "7"
       repository:
         iso:
           localPath:
           url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/centos7-rpms-amd64.iso
     - arch: amd64
       type: linux
       id: ubuntu
       version: "20.04"
       repository:
         iso:
           localPath:
           url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/ubuntu-20.04-debs-amd64.iso
     kubernetesDistributions:
     - type: kubernetes
       version: v1.23.15
     components:
       helm:
         version: v3.9.0
       cni:
         version: v1.2.0
       etcd:
         version: v3.4.13
       calicoctl:
         version: v3.23.2
      ## For now, if your cluster container runtime is containerd, KubeKey will add a docker 20.10.8 container runtime in the below list.
      ## The reason is KubeKey creates a cluster with containerd by installing a docker first and making kubelet connect the socket file of containerd which docker contained.
       containerRuntimes:
       - type: docker
         version: 20.10.8
       - type: containerd
         version: 1.6.4
       crictl:
         version: v1.24.0
       docker-registry:
         version: "2"
       harbor:
         version: v2.5.3
       docker-compose:
         version: v2.2.2
     images:
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.23.15
     - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.8.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.23.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.12.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:3.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:3.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nfs-subdir-external-provisioner:v4.0.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.15.12
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-installer:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-console:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-controller-manager:v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.22.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.21.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.20.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kubefed:v0.8.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tower:v0.2.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z
     - registry.cn-beijing.aliyuncs.com/kubesphereio/mc:RELEASE.2019-08-07T23-14-43Z
     - registry.cn-beijing.aliyuncs.com/kubesphereio/snapshot-controller:v4.0.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx-ingress-controller:v1.1.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/defaultbackend-amd64:1.4
     - registry.cn-beijing.aliyuncs.com/kubesphereio/metrics-server:v0.4.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.0.25-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/alpine:3.14
     - registry.cn-beijing.aliyuncs.com/kubesphereio/openldap:1.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/netshoot:v1.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/cloudcore:v1.13.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/iptables-manager:v1.13.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/edgeservice:v0.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/gatekeeper:v3.5.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/openpitrix-jobs:v3.3.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-apiserver:ks-v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-controller:ks-v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-tools:ks-v3.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-jenkins:v3.4.0-2.319.3-1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/inbound-agent:4.10-2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18-podman
     - registry.cn-beijing.aliyuncs.com/kubesphereio/s2ioperator:v3.2.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/s2irun:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/s2i-binary:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-runtime:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-8-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-6-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-4-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-36-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-35-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-34-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/python-27-centos7:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd:v2.3.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd-applicationset:v0.4.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/dex:v2.30.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:6.2.6-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/configmap-reload:v0.7.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus:v2.39.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-config-reloader:v0.55.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-operator:v0.55.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-state-metrics:v2.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/node-exporter:v1.3.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/alertmanager:v0.23.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/thanos:v0.31.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/grafana:8.3.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager-operator:v2.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager:v2.3.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-tenant-sidecar:v3.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-curator:v5.7.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-oss:6.8.22
     - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch:2.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-dashboards:2.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-curator:v0.0.5
     - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentbit-operator:v0.14.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/docker:19.03
     - registry.cn-beijing.aliyuncs.com/kubesphereio/fluent-bit:v1.9.4
     - registry.cn-beijing.aliyuncs.com/kubesphereio/log-sidecar-injector:v1.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/filebeat:6.7.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-operator:v0.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-exporter:v0.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-ruler:v0.6.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-operator:v0.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-webhook:v0.2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/pilot:1.14.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/proxyv2:1.14.6
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-operator:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-agent:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-collector:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-query:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-es-index-cleaner:1.29
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali-operator:v1.50.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali:v1.50
     - registry.cn-beijing.aliyuncs.com/kubesphereio/busybox:1.31.1
     - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx:1.14-alpine
     - registry.cn-beijing.aliyuncs.com/kubesphereio/wget:1.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/hello:plain-text
     - registry.cn-beijing.aliyuncs.com/kubesphereio/wordpress:4.8-apache
     - registry.cn-beijing.aliyuncs.com/kubesphereio/hpa-example:latest
     - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentd:v1.4.2-2.0
     - registry.cn-beijing.aliyuncs.com/kubesphereio/perl:latest
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-productpage-v1:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v1:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v2:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-details-v1:1.16.2
     - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-ratings-v1:1.16.3
     - registry.cn-beijing.aliyuncs.com/kubesphereio/scope:1.13.0

3. 导出制品 artifact

   export KKZONE=cn
   ./kk artifact export -m manifest.yaml -o kubesphere.tar.gz

离线安装

1. 将下载的 KubeKey 和制品 artifact 通过 U 盘等介质拷贝至离线环境安装节点
2. 执行以下命令创建离线集群配置文件

./kk create config --with-kubesphere v3.4.1 --with-kubernetes v1.23.15 -f config-sample.yaml

3. 修改刚刚生成的配置文件

按照实际环境配置修改节点信息。
必须指定 registry 仓库部署节点（用于 KubeKey 部署自建 Harbor 仓库）。
registry 里必须指定 type 类型为 harbor，否则默认安装 docker registry。

4. 执行以下命令安装镜像仓库

./kk init registry -f config-sample.yaml -a kubesphere.tar.gz

5. 创建 Harbor 项目

由于 Harbor 项目存在访问控制（RBAC）的限制，即只有指定角色的用户才能执行某些操作。如果您未创建项目，则镜像不能被推送到 Harbor。Harbor 中有两种类型的项目：
公共项目（Public）：任何用户都可以从这个项目中拉取镜像。
私有项目（Private）：只有作为项目成员的用户可以拉取镜像。
Harbor 管理员账号：admin，密码：Harbor12345。Harbor 安装文件在 /opt/harbor

• 登录 Harbor 仓库创建项目。将项目设置为公开以便所有用户都能够拉取镜像。

6. 再次修改集群配置文件config-sample.yaml

  ...
  registry:
    type: harbor
    auths:
      "dockerhub.kubekey.local":
        username: admin
        password: Harbor12345
    privateRegistry: "dockerhub.kubekey.local"
    namespaceOverride: "kubesphereio"
    registryMirrors: []
    insecureRegistries: []
  addons: []

备注
新增 auths 配置增加 dockerhub.kubekey.local 和账号密码。
privateRegistry 增加 dockerhub.kubekey.local。
namespaceOverride 增加 kubesphereio

7. 执行以下命令安装 KubeSphere 集群

./kk create cluster -f config-sample.yaml -a kubesphere.tar.gz

8. 执行以下命令查看集群状态

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

安装完成后就能看到登录信息