二、K8s集群安装:Kubesphere容器平台篇
二、KubeSphere
默认的 dashboard 没啥用,我们用 kubesphere 可以打通全部的 devops 链路。
Kubesphere 集成了很多套件,集群要求较高
Kuboard 也很不错,集群要求不高
1、简介
KubeSphere 是一款面向云原生设计的开源项目,在目前主流容器调度平台 Kubernetes 之
上构建的分布式多租户容器管理平台,提供简单易用的操作界面以及向导式操作方式,在降
低用户使用容器调度平台学习成本的同时,极大降低开发、测试、运维的日常工作的复杂度。
2、安装
1、前提条件
2、安装前提环境
1、安装 helm(master 节点执行)
Helm 是 Kubernetes 的包管理器。包管理器类似于我们在 Ubuntu 中使用的 apt、Centos
中使用的 yum 或者 Python 中的 pip 一样,能快速查找、下载和安装软件包。Helm 由客
户端组件 helm 和服务端组件 Tiller 组成, 能够将一组 K8S 资源打包统一管理, 是查找、共
享和使用为 Kubernetes 构建的软件的最佳方式。
1)、安装方法一(以下命令执行可能失败,因为被墙了,可以使用后面的安装方法)
curl -L https://git.io/get_helm.sh | bash
墙原因,上传我们给定的 get_helm.sh,chmod 700 然后./get_helm.sh
可能有文件格式兼容性问题,用 vi 打开该 sh 文件,输入:
:set ff
回车,显示 fileformat=dos,重新设置下文件格式:
:set ff=unix
保存退出:
:wq
安装方法二:
1.根据系统下载所需版本
下载链接:https://get.helm.sh/helm-v2.16.2-linux-amd64.tar.gz ,建议使用迅雷
2.用xftp将其拖入k8s目录,解压压缩包(以linux-amd64为例)
tar -zxvf helm-v2.16.2-linux-amd64.tar.gz
3.将文件移动到PATH目录中(以linux-amd64为例)
sudo mv linux-amd64/helm /usr/bin/helm
最后,执行命令,出现以下信息即部署成功
[root@k8s-node1 k8s]# helm version Client: &version.Version{SemVer:"v2.16.2", GitCommit:"bbdfe5e7803a12bbdf97e94cd847859890cf4050", GitTreeState:"clean"}
2)、验证版本
helm version
3)、创建权限(master 执行)
创建 helm-rbac.yaml,写入如下内容
apiVersion: v1 kind: ServiceAccount metadata: name: tiller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: tiller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system
应用配置
kubectl apply -f helm-rbac.yaml
2、安装 Tiller(master 执行)
1、初始化,以下两个命令均可,只是镜像地址不同。
helm init --service-account=tiller --tiller-image=jessestuart/tiller:v2.16.2 --history-max 300
或者
helm init --service-account=tiller --tiller-image=registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.16.2 --history-max 300
--tiller-image 指定镜像,否则会被墙。大家使用这个镜像比较好
jessestuart/tiller:v2.16.2
等待节点上部署的 tiller 完成即可
可能遇到的问题1:
Error: https://kubernetes-charts.storage.googleapis.com is not a valid chart repository or cannot be reached: failed to fetch https://kubernetes-charts.storage.googleapis.com/index.yaml : 403 Forbidden
解决办法:
helm init --upgrade -i registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.16.2 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
可能遇到的问题2:(解决k8s helm安装tiller出现ImagePullBackOff,ErrImagePull错误)
使用 helm version 命令查看版本,client与server的版本如果不一致,可能导致后面的kubesphere安装失败。
查看pod状态,如果出现ImagePullBackOff,ErrImagePull错误
kubectl get pods -n kube-system
解决方法:
删除tiller deployment,顺带会一起删除tiller pod
kubectl delete -n kube-system deployment tiller-deploy
或者
helm reset -f
然后重新执行上面的安装即可。
2、测试
helm install stable/nginx-ingress --name nginx-ingress
helm ls
helm delete nginx-ingress
3、使用语法
#创建一个 chart 范例
helm create helm-chart
#检查 chart 语法
helm lint ./helm-chart
#使用默认 chart 部署到 k8s
helm install --name example1 ./helm-chart --set service.type=NodePort
#kubectl get pod 查看是否部署成功
3、安装 OpenEBS(master 执行)
kubectl describe node k8s-node1 | grep Taint #确定 master 节点是否有 taint kubectl taint nodes k8s-node1 node-role.kubernetes.io/master:NoSchedule- #取消 taint kubectl apply -f https://openebs.github.io/charts/openebs-operator-1.5.0.yaml #若该命令执行失败,可能被墙,参考下面的方法
如果被墙,可以创建以下文件:openebs-operator-1.5.0.yaml,
文件内容如下:

# This manifest deploys the OpenEBS control plane components, with associated CRs & RBAC rules # NOTE: On GKE, deploy the openebs-operator.yaml in admin context # Create the OpenEBS namespace apiVersion: v1 kind: Namespace metadata: name: openebs --- # Create Maya Service Account apiVersion: v1 kind: ServiceAccount metadata: name: openebs-maya-operator namespace: openebs --- # Define Role that allows operations on K8s pods/deployments kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: openebs-maya-operator rules: - apiGroups: ["*"] resources: ["nodes", "nodes/proxy"] verbs: ["*"] - apiGroups: ["*"] resources: ["namespaces", "services", "pods", "pods/exec", "deployments", "deployments/finalizers", "replicationcontrollers", "replicasets", "events", "endpoints", "configmaps", "secrets", "jobs", "cronjobs"] verbs: ["*"] - apiGroups: ["*"] resources: ["statefulsets", "daemonsets"] verbs: ["*"] - apiGroups: ["*"] resources: ["resourcequotas", "limitranges"] verbs: ["list", "watch"] - apiGroups: ["*"] resources: ["ingresses", "horizontalpodautoscalers", "verticalpodautoscalers", "poddisruptionbudgets", "certificatesigningrequests"] verbs: ["list", "watch"] - apiGroups: ["*"] resources: ["storageclasses", "persistentvolumeclaims", "persistentvolumes"] verbs: ["*"] - apiGroups: ["volumesnapshot.external-storage.k8s.io"] resources: ["volumesnapshots", "volumesnapshotdatas"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] - apiGroups: ["apiextensions.k8s.io"] resources: ["customresourcedefinitions"] verbs: [ "get", "list", "create", "update", "delete", "patch"] - apiGroups: ["*"] resources: [ "disks", "blockdevices", "blockdeviceclaims"] verbs: ["*" ] - apiGroups: ["*"] resources: [ "cstorpoolclusters", "storagepoolclaims", "storagepoolclaims/finalizers", "cstorpoolclusters/finalizers", "storagepools"] verbs: ["*" ] - apiGroups: ["*"] resources: [ "castemplates", "runtasks"] verbs: ["*" ] - apiGroups: ["*"] resources: [ "cstorpools", "cstorpools/finalizers", "cstorvolumereplicas", "cstorvolumes", "cstorvolumeclaims"] verbs: ["*" ] - apiGroups: ["*"] resources: [ "cstorpoolinstances", "cstorpoolinstances/finalizers"] verbs: ["*" ] - apiGroups: ["*"] resources: [ "cstorbackups", "cstorrestores", "cstorcompletedbackups"] verbs: ["*" ] - apiGroups: ["coordination.k8s.io"] resources: ["leases"] verbs: ["get", "watch", "list", "delete", "update", "create"] - apiGroups: ["admissionregistration.k8s.io"] resources: ["validatingwebhookconfigurations", "mutatingwebhookconfigurations"] verbs: ["get", "create", "list", "delete", "update", "patch"] - nonResourceURLs: ["/metrics"] verbs: ["get"] - apiGroups: ["*"] resources: [ "upgradetasks"] verbs: ["*" ] --- # Bind the Service Account with the Role Privileges. # TODO: Check if default account also needs to be there kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: openebs-maya-operator subjects: - kind: ServiceAccount name: openebs-maya-operator namespace: openebs roleRef: kind: ClusterRole name: openebs-maya-operator apiGroup: rbac.authorization.k8s.io --- apiVersion: apps/v1 kind: Deployment metadata: name: maya-apiserver namespace: openebs labels: name: maya-apiserver openebs.io/component-name: maya-apiserver openebs.io/version: 1.5.0 spec: selector: matchLabels: name: maya-apiserver openebs.io/component-name: maya-apiserver replicas: 1 strategy: type: Recreate rollingUpdate: null template: metadata: labels: name: maya-apiserver openebs.io/component-name: maya-apiserver openebs.io/version: 1.5.0 spec: serviceAccountName: openebs-maya-operator containers: - name: maya-apiserver imagePullPolicy: IfNotPresent image: quay.io/openebs/m-apiserver:1.5.0 ports: - containerPort: 5656 env: # OPENEBS_IO_KUBE_CONFIG enables maya api service to connect to K8s # based on this config. This is ignored if empty. # This is supported for maya api server version 0.5.2 onwards #- name: OPENEBS_IO_KUBE_CONFIG # value: "/home/ubuntu/.kube/config" # OPENEBS_IO_K8S_MASTER enables maya api service to connect to K8s # based on this address. This is ignored if empty. # This is supported for maya api server version 0.5.2 onwards #- name: OPENEBS_IO_K8S_MASTER # value: "http://172.28.128.3:8080" # OPENEBS_NAMESPACE provides the namespace of this deployment as an # environment variable - name: OPENEBS_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace # OPENEBS_SERVICE_ACCOUNT provides the service account of this pod as # environment variable - name: OPENEBS_SERVICE_ACCOUNT valueFrom: fieldRef: fieldPath: spec.serviceAccountName # OPENEBS_MAYA_POD_NAME provides the name of this pod as # environment variable - name: OPENEBS_MAYA_POD_NAME valueFrom: fieldRef: fieldPath: metadata.name # If OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG is false then OpenEBS default # storageclass and storagepool will not be created. - name: OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG value: "true" # OPENEBS_IO_INSTALL_DEFAULT_CSTOR_SPARSE_POOL decides whether default cstor sparse pool should be # configured as a part of openebs installation. # If "true" a default cstor sparse pool will be configured, if "false" it will not be configured. # This value takes effect only if OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG # is set to true - name: OPENEBS_IO_INSTALL_DEFAULT_CSTOR_SPARSE_POOL value: "false" # OPENEBS_IO_CSTOR_TARGET_DIR can be used to specify the hostpath # to be used for saving the shared content between the side cars # of cstor volume pod. # The default path used is /var/openebs/sparse #- name: OPENEBS_IO_CSTOR_TARGET_DIR # value: "/var/openebs/sparse" # OPENEBS_IO_CSTOR_POOL_SPARSE_DIR can be used to specify the hostpath # to be used for saving the shared content between the side cars # of cstor pool pod. This ENV is also used to indicate the location # of the sparse devices. # The default path used is /var/openebs/sparse #- name: OPENEBS_IO_CSTOR_POOL_SPARSE_DIR # value: "/var/openebs/sparse" # OPENEBS_IO_JIVA_POOL_DIR can be used to specify the hostpath # to be used for default Jiva StoragePool loaded by OpenEBS # The default path used is /var/openebs # This value takes effect only if OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG # is set to true #- name: OPENEBS_IO_JIVA_POOL_DIR # value: "/var/openebs" # OPENEBS_IO_LOCALPV_HOSTPATH_DIR can be used to specify the hostpath # to be used for default openebs-hostpath storageclass loaded by OpenEBS # The default path used is /var/openebs/local # This value takes effect only if OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG # is set to true #- name: OPENEBS_IO_LOCALPV_HOSTPATH_DIR # value: "/var/openebs/local" - name: OPENEBS_IO_JIVA_CONTROLLER_IMAGE value: "quay.io/openebs/jiva:1.5.0" - name: OPENEBS_IO_JIVA_REPLICA_IMAGE value: "quay.io/openebs/jiva:1.5.0" - name: OPENEBS_IO_JIVA_REPLICA_COUNT value: "3" - name: OPENEBS_IO_CSTOR_TARGET_IMAGE value: "quay.io/openebs/cstor-istgt:1.5.0" - name: OPENEBS_IO_CSTOR_POOL_IMAGE value: "quay.io/openebs/cstor-pool:1.5.0" - name: OPENEBS_IO_CSTOR_POOL_MGMT_IMAGE value: "quay.io/openebs/cstor-pool-mgmt:1.5.0" - name: OPENEBS_IO_CSTOR_VOLUME_MGMT_IMAGE value: "quay.io/openebs/cstor-volume-mgmt:1.5.0" - name: OPENEBS_IO_VOLUME_MONITOR_IMAGE value: "quay.io/openebs/m-exporter:1.5.0" - name: OPENEBS_IO_CSTOR_POOL_EXPORTER_IMAGE value: "quay.io/openebs/m-exporter:1.5.0" - name: OPENEBS_IO_HELPER_IMAGE value: "quay.io/openebs/linux-utils:1.5.0" # OPENEBS_IO_ENABLE_ANALYTICS if set to true sends anonymous usage # events to Google Analytics - name: OPENEBS_IO_ENABLE_ANALYTICS value: "true" - name: OPENEBS_IO_INSTALLER_TYPE value: "openebs-operator" # OPENEBS_IO_ANALYTICS_PING_INTERVAL can be used to specify the duration (in hours) # for periodic ping events sent to Google Analytics. # Default is 24h. # Minimum is 1h. You can convert this to weekly by setting 168h #- name: OPENEBS_IO_ANALYTICS_PING_INTERVAL # value: "24h" livenessProbe: exec: command: - /usr/local/bin/mayactl - version initialDelaySeconds: 30 periodSeconds: 60 readinessProbe: exec: command: - /usr/local/bin/mayactl - version initialDelaySeconds: 30 periodSeconds: 60 --- apiVersion: v1 kind: Service metadata: name: maya-apiserver-service namespace: openebs labels: openebs.io/component-name: maya-apiserver-svc spec: ports: - name: api port: 5656 protocol: TCP targetPort: 5656 selector: name: maya-apiserver sessionAffinity: None --- apiVersion: apps/v1 kind: Deployment metadata: name: openebs-provisioner namespace: openebs labels: name: openebs-provisioner openebs.io/component-name: openebs-provisioner openebs.io/version: 1.5.0 spec: selector: matchLabels: name: openebs-provisioner openebs.io/component-name: openebs-provisioner replicas: 1 strategy: type: Recreate rollingUpdate: null template: metadata: labels: name: openebs-provisioner openebs.io/component-name: openebs-provisioner openebs.io/version: 1.5.0 spec: serviceAccountName: openebs-maya-operator containers: - name: openebs-provisioner imagePullPolicy: IfNotPresent image: quay.io/openebs/openebs-k8s-provisioner:1.5.0 env: # OPENEBS_IO_K8S_MASTER enables openebs provisioner to connect to K8s # based on this address. This is ignored if empty. # This is supported for openebs provisioner version 0.5.2 onwards #- name: OPENEBS_IO_K8S_MASTER # value: "http://10.128.0.12:8080" # OPENEBS_IO_KUBE_CONFIG enables openebs provisioner to connect to K8s # based on this config. This is ignored if empty. # This is supported for openebs provisioner version 0.5.2 onwards #- name: OPENEBS_IO_KUBE_CONFIG # value: "/home/ubuntu/.kube/config" - name: NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName - name: OPENEBS_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace # OPENEBS_MAYA_SERVICE_NAME provides the maya-apiserver K8s service name, # that provisioner should forward the volume create/delete requests. # If not present, "maya-apiserver-service" will be used for lookup. # This is supported for openebs provisioner version 0.5.3-RC1 onwards #- name: OPENEBS_MAYA_SERVICE_NAME # value: "maya-apiserver-apiservice" livenessProbe: exec: command: - pgrep - ".*openebs" initialDelaySeconds: 30 periodSeconds: 60 --- apiVersion: apps/v1 kind: Deployment metadata: name: openebs-snapshot-operator namespace: openebs labels: name: openebs-snapshot-operator openebs.io/component-name: openebs-snapshot-operator openebs.io/version: 1.5.0 spec: selector: matchLabels: name: openebs-snapshot-operator openebs.io/component-name: openebs-snapshot-operator replicas: 1 strategy: type: Recreate template: metadata: labels: name: openebs-snapshot-operator openebs.io/component-name: openebs-snapshot-operator openebs.io/version: 1.5.0 spec: serviceAccountName: openebs-maya-operator containers: - name: snapshot-controller image: quay.io/openebs/snapshot-controller:1.5.0 imagePullPolicy: IfNotPresent env: - name: OPENEBS_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace livenessProbe: exec: command: - pgrep - ".*controller" initialDelaySeconds: 30 periodSeconds: 60 # OPENEBS_MAYA_SERVICE_NAME provides the maya-apiserver K8s service name, # that snapshot controller should forward the snapshot create/delete requests. # If not present, "maya-apiserver-service" will be used for lookup. # This is supported for openebs provisioner version 0.5.3-RC1 onwards #- name: OPENEBS_MAYA_SERVICE_NAME # value: "maya-apiserver-apiservice" - name: snapshot-provisioner image: quay.io/openebs/snapshot-provisioner:1.5.0 imagePullPolicy: IfNotPresent env: - name: OPENEBS_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace # OPENEBS_MAYA_SERVICE_NAME provides the maya-apiserver K8s service name, # that snapshot provisioner should forward the clone create/delete requests. # If not present, "maya-apiserver-service" will be used for lookup. # This is supported for openebs provisioner version 0.5.3-RC1 onwards #- name: OPENEBS_MAYA_SERVICE_NAME # value: "maya-apiserver-apiservice" livenessProbe: exec: command: - pgrep - ".*provisioner" initialDelaySeconds: 30 periodSeconds: 60 --- # This is the node-disk-manager related config. # It can be used to customize the disks probes and filters apiVersion: v1 kind: ConfigMap metadata: name: openebs-ndm-config namespace: openebs labels: openebs.io/component-name: ndm-config data: # udev-probe is default or primary probe which should be enabled to run ndm # filterconfigs contails configs of filters - in their form fo include # and exclude comma separated strings node-disk-manager.config: | probeconfigs: - key: udev-probe name: udev probe state: true - key: seachest-probe name: seachest probe state: false - key: smart-probe name: smart probe state: true filterconfigs: - key: os-disk-exclude-filter name: os disk exclude filter state: true exclude: "/,/etc/hosts,/boot" - key: vendor-filter name: vendor filter state: true include: "" exclude: "CLOUDBYT,OpenEBS" - key: path-filter name: path filter state: true include: "" exclude: "loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/dm-,/dev/md" --- apiVersion: apps/v1 kind: DaemonSet metadata: name: openebs-ndm namespace: openebs labels: name: openebs-ndm openebs.io/component-name: ndm openebs.io/version: 1.5.0 spec: selector: matchLabels: name: openebs-ndm openebs.io/component-name: ndm updateStrategy: type: RollingUpdate template: metadata: labels: name: openebs-ndm openebs.io/component-name: ndm openebs.io/version: 1.5.0 spec: # By default the node-disk-manager will be run on all kubernetes nodes # If you would like to limit this to only some nodes, say the nodes # that have storage attached, you could label those node and use # nodeSelector. # # e.g. label the storage nodes with - "openebs.io/nodegroup"="storage-node" # kubectl label node <node-name> "openebs.io/nodegroup"="storage-node" #nodeSelector: # "openebs.io/nodegroup": "storage-node" serviceAccountName: openebs-maya-operator hostNetwork: true containers: - name: node-disk-manager image: quay.io/openebs/node-disk-manager-amd64:v0.4.5 imagePullPolicy: Always securityContext: privileged: true volumeMounts: - name: config mountPath: /host/node-disk-manager.config subPath: node-disk-manager.config readOnly: true - name: udev mountPath: /run/udev - name: procmount mountPath: /host/proc readOnly: true - name: sparsepath mountPath: /var/openebs/sparse env: # namespace in which NDM is installed will be passed to NDM Daemonset # as environment variable - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace # pass hostname as env variable using downward API to the NDM container - name: NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName # specify the directory where the sparse files need to be created. # if not specified, then sparse files will not be created. - name: SPARSE_FILE_DIR value: "/var/openebs/sparse" # Size(bytes) of the sparse file to be created. - name: SPARSE_FILE_SIZE value: "10737418240" # Specify the number of sparse files to be created - name: SPARSE_FILE_COUNT value: "0" livenessProbe: exec: command: - pgrep - ".*ndm" initialDelaySeconds: 30 periodSeconds: 60 volumes: - name: config configMap: name: openebs-ndm-config - name: udev hostPath: path: /run/udev type: Directory # mount /proc (to access mount file of process 1 of host) inside container # to read mount-point of disks and partitions - name: procmount hostPath: path: /proc type: Directory - name: sparsepath hostPath: path: /var/openebs/sparse --- apiVersion: apps/v1 kind: Deployment metadata: name: openebs-ndm-operator namespace: openebs labels: name: openebs-ndm-operator openebs.io/component-name: ndm-operator openebs.io/version: 1.5.0 spec: selector: matchLabels: name: openebs-ndm-operator openebs.io/component-name: ndm-operator replicas: 1 strategy: type: Recreate template: metadata: labels: name: openebs-ndm-operator openebs.io/component-name: ndm-operator openebs.io/version: 1.5.0 spec: serviceAccountName: openebs-maya-operator containers: - name: node-disk-operator image: quay.io/openebs/node-disk-operator-amd64:v0.4.5 imagePullPolicy: Always readinessProbe: exec: command: - stat - /tmp/operator-sdk-ready initialDelaySeconds: 4 periodSeconds: 10 failureThreshold: 1 env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name # the service account of the ndm-operator pod - name: SERVICE_ACCOUNT valueFrom: fieldRef: fieldPath: spec.serviceAccountName - name: OPERATOR_NAME value: "node-disk-operator" - name: CLEANUP_JOB_IMAGE value: "quay.io/openebs/linux-utils:1.5.0" --- apiVersion: apps/v1 kind: Deployment metadata: name: openebs-admission-server namespace: openebs labels: app: admission-webhook openebs.io/component-name: admission-webhook openebs.io/version: 1.5.0 spec: replicas: 1 strategy: type: Recreate rollingUpdate: null selector: matchLabels: app: admission-webhook template: metadata: labels: app: admission-webhook openebs.io/component-name: admission-webhook openebs.io/version: 1.5.0 spec: serviceAccountName: openebs-maya-operator containers: - name: admission-webhook image: quay.io/openebs/admission-server:1.5.0 imagePullPolicy: IfNotPresent args: - -alsologtostderr - -v=2 - 2>&1 env: - name: OPENEBS_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: ADMISSION_WEBHOOK_NAME value: "openebs-admission-server" --- apiVersion: apps/v1 kind: Deployment metadata: name: openebs-localpv-provisioner namespace: openebs labels: name: openebs-localpv-provisioner openebs.io/component-name: openebs-localpv-provisioner openebs.io/version: 1.5.0 spec: selector: matchLabels: name: openebs-localpv-provisioner openebs.io/component-name: openebs-localpv-provisioner replicas: 1 strategy: type: Recreate template: metadata: labels: name: openebs-localpv-provisioner openebs.io/component-name: openebs-localpv-provisioner openebs.io/version: 1.5.0 spec: serviceAccountName: openebs-maya-operator containers: - name: openebs-provisioner-hostpath imagePullPolicy: Always image: quay.io/openebs/provisioner-localpv:1.5.0 env: # OPENEBS_IO_K8S_MASTER enables openebs provisioner to connect to K8s # based on this address. This is ignored if empty. # This is supported for openebs provisioner version 0.5.2 onwards #- name: OPENEBS_IO_K8S_MASTER # value: "http://10.128.0.12:8080" # OPENEBS_IO_KUBE_CONFIG enables openebs provisioner to connect to K8s # based on this config. This is ignored if empty. # This is supported for openebs provisioner version 0.5.2 onwards #- name: OPENEBS_IO_KUBE_CONFIG # value: "/home/ubuntu/.kube/config" - name: NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName - name: OPENEBS_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace # OPENEBS_SERVICE_ACCOUNT provides the service account of this pod as # environment variable - name: OPENEBS_SERVICE_ACCOUNT valueFrom: fieldRef: fieldPath: spec.serviceAccountName - name: OPENEBS_IO_ENABLE_ANALYTICS value: "true" - name: OPENEBS_IO_INSTALLER_TYPE value: "openebs-operator" - name: OPENEBS_IO_HELPER_IMAGE value: "quay.io/openebs/linux-utils:1.5.0" livenessProbe: exec: command: - pgrep - ".*localpv" initialDelaySeconds: 30 periodSeconds: 60 ---
然后执行命令:
kubectl apply -f openebs-operator-1.5.0.yaml
#安装 openebs
#查看效果
kubectl get sc -n openebs
#将 openebs-hostpath 设置为 默认的 StorageClass:
kubectl patch storageclass openebs-hostpath -p \ '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
#至此,OpenEBS 的 LocalPV 已作为默认的存储类型创建成功。由于在文档开头手动去掉
了 master 节点的 Taint,我们可以在安装完 OpenEBS 后将 master 节点 Taint 加上,避
免业务相关的工作负载调度到 master 节点抢占 master 资源
kubectl taint nodes k8s-node1 node-role.kubernetes.io=master:NoSchedule
3、最小化安装 kubesphere
若您的集群可用的资源符合 CPU > 1 Core,可用内存 > 2 G,可以参考以下命令开启
KubeSphere 最小化安装:
kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/master/kubesphere-minimal.yaml
以上命令可能执行失败,因为被墙了或者失效了,可以创建以下文件 kubesphere-minimal.yaml,内容如下:

--- apiVersion: v1 kind: Namespace metadata: name: kubesphere-system --- apiVersion: v1 data: ks-config.yaml: | --- persistence: storageClass: "" etcd: monitoring: False endpointIps: 192.168.0.7,192.168.0.8,192.168.0.9 port: 2379 tlsEnable: True common: mysqlVolumeSize: 20Gi minioVolumeSize: 20Gi etcdVolumeSize: 20Gi openldapVolumeSize: 2Gi redisVolumSize: 2Gi metrics_server: enabled: False console: enableMultiLogin: False # enable/disable multi login port: 30880 monitoring: prometheusReplicas: 1 prometheusMemoryRequest: 400Mi prometheusVolumeSize: 20Gi grafana: enabled: False logging: enabled: False elasticsearchMasterReplicas: 1 elasticsearchDataReplicas: 1 logsidecarReplicas: 2 elasticsearchMasterVolumeSize: 4Gi elasticsearchDataVolumeSize: 20Gi logMaxAge: 7 elkPrefix: logstash containersLogMountedPath: "" kibana: enabled: False openpitrix: enabled: False devops: enabled: False jenkinsMemoryLim: 2Gi jenkinsMemoryReq: 1500Mi jenkinsVolumeSize: 8Gi jenkinsJavaOpts_Xms: 512m jenkinsJavaOpts_Xmx: 512m jenkinsJavaOpts_MaxRAM: 2g sonarqube: enabled: False postgresqlVolumeSize: 8Gi servicemesh: enabled: False notification: enabled: False alerting: enabled: False kind: ConfigMap metadata: name: ks-installer namespace: kubesphere-system --- apiVersion: v1 kind: ServiceAccount metadata: name: ks-installer namespace: kubesphere-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: creationTimestamp: null name: ks-installer rules: - apiGroups: - "" resources: - '*' verbs: - '*' - apiGroups: - apps resources: - '*' verbs: - '*' - apiGroups: - extensions resources: - '*' verbs: - '*' - apiGroups: - batch resources: - '*' verbs: - '*' - apiGroups: - rbac.authorization.k8s.io resources: - '*' verbs: - '*' - apiGroups: - apiregistration.k8s.io resources: - '*' verbs: - '*' - apiGroups: - apiextensions.k8s.io resources: - '*' verbs: - '*' - apiGroups: - tenant.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - certificates.k8s.io resources: - '*' verbs: - '*' - apiGroups: - devops.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - monitoring.coreos.com resources: - '*' verbs: - '*' - apiGroups: - logging.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - jaegertracing.io resources: - '*' verbs: - '*' - apiGroups: - storage.k8s.io resources: - '*' verbs: - '*' - apiGroups: - admissionregistration.k8s.io resources: - '*' verbs: - '*' --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: ks-installer subjects: - kind: ServiceAccount name: ks-installer namespace: kubesphere-system roleRef: kind: ClusterRole name: ks-installer apiGroup: rbac.authorization.k8s.io --- apiVersion: apps/v1 kind: Deployment metadata: name: ks-installer namespace: kubesphere-system labels: app: ks-install spec: replicas: 1 selector: matchLabels: app: ks-install template: metadata: labels: app: ks-install spec: serviceAccountName: ks-installer containers: - name: installer image: kubesphere/ks-installer:v2.1.1 imagePullPolicy: "Always"
执行命令,仅针对 kubesphere2.1 版本
kubectl apply -f kubesphere-minimal.yaml
或者,仅针对 kubesphere3.1 版本
先创建以下文件 kubesphere-installer.yaml、cluster-configuration.yaml,内容如下:
kubesphere-installer.yaml:

--- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: clusterconfigurations.installer.kubesphere.io spec: group: installer.kubesphere.io versions: - name: v1alpha1 served: true storage: true schema: openAPIV3Schema: type: object properties: spec: type: object x-kubernetes-preserve-unknown-fields: true status: type: object x-kubernetes-preserve-unknown-fields: true scope: Namespaced names: plural: clusterconfigurations singular: clusterconfiguration kind: ClusterConfiguration shortNames: - cc --- apiVersion: v1 kind: Namespace metadata: name: kubesphere-system --- apiVersion: v1 kind: ServiceAccount metadata: name: ks-installer namespace: kubesphere-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: ks-installer rules: - apiGroups: - "" resources: - '*' verbs: - '*' - apiGroups: - apps resources: - '*' verbs: - '*' - apiGroups: - extensions resources: - '*' verbs: - '*' - apiGroups: - batch resources: - '*' verbs: - '*' - apiGroups: - rbac.authorization.k8s.io resources: - '*' verbs: - '*' - apiGroups: - apiregistration.k8s.io resources: - '*' verbs: - '*' - apiGroups: - apiextensions.k8s.io resources: - '*' verbs: - '*' - apiGroups: - tenant.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - certificates.k8s.io resources: - '*' verbs: - '*' - apiGroups: - devops.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - monitoring.coreos.com resources: - '*' verbs: - '*' - apiGroups: - logging.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - jaegertracing.io resources: - '*' verbs: - '*' - apiGroups: - storage.k8s.io resources: - '*' verbs: - '*' - apiGroups: - admissionregistration.k8s.io resources: - '*' verbs: - '*' - apiGroups: - policy resources: - '*' verbs: - '*' - apiGroups: - autoscaling resources: - '*' verbs: - '*' - apiGroups: - networking.istio.io resources: - '*' verbs: - '*' - apiGroups: - config.istio.io resources: - '*' verbs: - '*' - apiGroups: - iam.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - notification.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - auditing.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - events.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - core.kubefed.io resources: - '*' verbs: - '*' - apiGroups: - installer.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - storage.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - security.istio.io resources: - '*' verbs: - '*' - apiGroups: - monitoring.kiali.io resources: - '*' verbs: - '*' - apiGroups: - kiali.io resources: - '*' verbs: - '*' - apiGroups: - networking.k8s.io resources: - '*' verbs: - '*' - apiGroups: - kubeedge.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - types.kubefed.io resources: - '*' verbs: - '*' - apiGroups: - monitoring.kubesphere.io resources: - '*' verbs: - '*' - apiGroups: - application.kubesphere.io resources: - '*' verbs: - '*' --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: ks-installer subjects: - kind: ServiceAccount name: ks-installer namespace: kubesphere-system roleRef: kind: ClusterRole name: ks-installer apiGroup: rbac.authorization.k8s.io --- apiVersion: apps/v1 kind: Deployment metadata: name: ks-installer namespace: kubesphere-system labels: app: ks-install spec: replicas: 1 selector: matchLabels: app: ks-install template: metadata: labels: app: ks-install spec: serviceAccountName: ks-installer containers: - name: installer image: kubespheredev/ks-installer:master imagePullPolicy: "Always" resources: limits: cpu: "1" memory: 1Gi requests: cpu: 20m memory: 100Mi volumeMounts: - mountPath: /etc/localtime name: host-time readOnly: true volumes: - hostPath: path: /etc/localtime type: "" name: host-time
cluster-configuration.yaml:

--- apiVersion: installer.kubesphere.io/v1alpha1 kind: ClusterConfiguration metadata: name: ks-installer namespace: kubesphere-system labels: version: v3.1.1 spec: persistence: storageClass: "" # If there is no default StorageClass in your cluster, you need to specify an existing StorageClass here. authentication: jwtSecret: "" # Keep the jwtSecret consistent with the Host Cluster. Retrieve the jwtSecret by executing "kubectl -n kubesphere-system get cm kubesphere-config -o yaml | grep -v "apiVersion" | grep jwtSecret" on the Host Cluster. local_registry: "" # Add your private registry address if it is needed. # dev_tag: "" # Add your kubesphere image tag you want to install, by default it's same as ks-install release version. etcd: monitoring: false # Enable or disable etcd monitoring dashboard installation. You have to create a Secret for etcd before you enable it. endpointIps: localhost # etcd cluster EndpointIps. It can be a bunch of IPs here. port: 2379 # etcd port. tlsEnable: true common: core: console: enableMultiLogin: true # Enable or disable simultaneous logins. It allows different users to log in with the same account at the same time. port: 30880 type: NodePort # apiserver: # Enlarge the apiserver and controller manager's resource requests and limits for the large cluster # resources: {} # controllerManager: # resources: {} redis: enabled: false volumeSize: 2Gi # Redis PVC size. openldap: enabled: false volumeSize: 2Gi # openldap PVC size. minio: volumeSize: 20Gi # Minio PVC size. monitoring: # type: external # Whether to specify the external prometheus stack, and need to modify the endpoint at the next line. endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090 # Prometheus endpoint to get metrics data. GPUMonitoring: # Enable or disable the GPU-related metrics. If you enable this switch but have no GPU resources, Kubesphere will set it to zero. enabled: false gpu: # Install GPUKinds. The default GPU kind is nvidia.com/gpu. Other GPU kinds can be added here according to your needs. kinds: - resourceName: "nvidia.com/gpu" resourceType: "GPU" default: true es: # Storage backend for logging, events and auditing. # master: # volumeSize: 4Gi # The volume size of Elasticsearch master nodes. # replicas: 1 # The total number of master nodes. Even numbers are not allowed. # resources: {} # data: # volumeSize: 20Gi # The volume size of Elasticsearch data nodes. # replicas: 1 # The total number of data nodes. # resources: {} logMaxAge: 7 # Log retention time in built-in Elasticsearch. It is 7 days by default. elkPrefix: logstash # The string making up index names. The index name will be formatted as ks-<elk_prefix>-log. basicAuth: enabled: false username: "" password: "" externalElasticsearchUrl: "" externalElasticsearchPort: "" alerting: # (CPU: 0.1 Core, Memory: 100 MiB) It enables users to customize alerting policies to send messages to receivers in time with different time intervals and alerting levels to choose from. enabled: false # Enable or disable the KubeSphere Alerting System. # thanosruler: # replicas: 1 # resources: {} auditing: # Provide a security-relevant chronological set of records,recording the sequence of activities happening on the platform, initiated by different tenants. enabled: false # Enable or disable the KubeSphere Auditing Log System. # operator: # resources: {} # webhook: # resources: {} devops: # (CPU: 0.47 Core, Memory: 8.6 G) Provide an out-of-the-box CI/CD system based on Jenkins, and automated workflow tools including Source-to-Image & Binary-to-Image. enabled: false # Enable or disable the KubeSphere DevOps System. jenkinsMemoryLim: 2Gi # Jenkins memory limit. jenkinsMemoryReq: 1500Mi # Jenkins memory request. jenkinsVolumeSize: 8Gi # Jenkins volume size. jenkinsJavaOpts_Xms: 512m # The following three fields are JVM parameters. jenkinsJavaOpts_Xmx: 512m jenkinsJavaOpts_MaxRAM: 2g events: # Provide a graphical web console for Kubernetes Events exporting, filtering and alerting in multi-tenant Kubernetes clusters. enabled: false # Enable or disable the KubeSphere Events System. # operator: # resources: {} # exporter: # resources: {} # ruler: # enabled: true # replicas: 2 # resources: {} logging: # (CPU: 57 m, Memory: 2.76 G) Flexible logging functions are provided for log query, collection and management in a unified console. Additional log collectors can be added, such as Elasticsearch, Kafka and Fluentd. enabled: false # Enable or disable the KubeSphere Logging System. containerruntime: docker logsidecar: enabled: true replicas: 2 # resources: {} metrics_server: # (CPU: 56 m, Memory: 44.35 MiB) It enables HPA (Horizontal Pod Autoscaler). enabled: false # Enable or disable metrics-server. monitoring: storageClass: "" # If there is an independent StorageClass you need for Prometheus, you can specify it here. The default StorageClass is used by default. # kube_rbac_proxy: # resources: {} # kube_state_metrics: # resources: {} # prometheus: # replicas: 1 # Prometheus replicas are responsible for monitoring different segments of data source and providing high availability. # volumeSize: 20Gi # Prometheus PVC size. # resources: {} # operator: # resources: {} # adapter: # resources: {} # node_exporter: # resources: {} # alertmanager: # replicas: 1 # AlertManager Replicas. # resources: {} # notification_manager: # resources: {} # operator: # resources: {} # proxy: # resources: {} gpu: # GPU monitoring-related plugins installation. nvidia_dcgm_exporter: enabled: false # resources: {} multicluster: clusterRole: none # host | member | none # You can install a solo cluster, or specify it as the Host or Member Cluster. network: networkpolicy: # Network policies allow network isolation within the same cluster, which means firewalls can be set up between certain instances (Pods). # Make sure that the CNI network plugin used by the cluster supports NetworkPolicy. There are a number of CNI network plugins that support NetworkPolicy, including Calico, Cilium, Kube-router, Romana and Weave Net. enabled: false # Enable or disable network policies. ippool: # Use Pod IP Pools to manage the Pod network address space. Pods to be created can be assigned IP addresses from a Pod IP Pool. type: none # Specify "calico" for this field if Calico is used as your CNI plugin. "none" means that Pod IP Pools are disabled. topology: # Use Service Topology to view Service-to-Service communication based on Weave Scope. type: none # Specify "weave-scope" for this field to enable Service Topology. "none" means that Service Topology is disabled. openpitrix: # An App Store that is accessible to all platform tenants. You can use it to manage apps across their entire lifecycle. store: enabled: false # Enable or disable the KubeSphere App Store. servicemesh: # (0.3 Core, 300 MiB) Provide fine-grained traffic management, observability and tracing, and visualized traffic topology. enabled: false # Base component (pilot). Enable or disable KubeSphere Service Mesh (Istio-based). kubeedge: # Add edge nodes to your cluster and deploy workloads on edge nodes. enabled: false # Enable or disable KubeEdge. cloudCore: nodeSelector: {"node-role.kubernetes.io/worker": ""} tolerations: [] cloudhubPort: "10000" cloudhubQuicPort: "10001" cloudhubHttpsPort: "10002" cloudstreamPort: "10003" tunnelPort: "10004" cloudHub: advertiseAddress: # At least a public IP address or an IP address which can be accessed by edge nodes must be provided. - "" # Note that once KubeEdge is enabled, CloudCore will malfunction if the address is not provided. nodeLimit: "100" service: cloudhubNodePort: "30000" cloudhubQuicNodePort: "30001" cloudhubHttpsNodePort: "30002" cloudstreamNodePort: "30003" tunnelNodePort: "30004" edgeWatcher: nodeSelector: {"node-role.kubernetes.io/worker": ""} tolerations: [] edgeWatcherAgent: nodeSelector: {"node-role.kubernetes.io/worker": ""} tolerations: [] gatekeeper: # Provide admission policy and rule management, A validating (mutating TBA) webhook that enforces CRD-based policies executed by Open Policy Agent. enabled: false # Enable or disable Gatekeeper. # controller_manager: # resources: {} # audit: # resources: {}
再执行:
kubectl apply -f kubesphere-installer.yaml kubectl apply -f cluster-configuration.yaml # 可以编辑devops为true
查看安装日志,请耐心等待安装成功。
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
KubeSphere 卸载(无需操作)
kubectl delete -f kubesphere-minimal.yaml
4、完整化安装(看电脑配置是否满足)
若集群可用 CPU > 8 Core 且可用内存 > 16 G,可以使用以下命令完整安装 KubeSphere。
注意,应确保集群中有一个节点的可用内存大于 8 G。
1、一条命令
kubectl apply -f https://raw.githubusercontent.com/kubesphere/ks-installer/v2.1.1/kubesphere-complete-setup.yaml
以上命令可能执行失败,因为被墙了或者失效了,可以创建以下文件 kubesphere-complete-setup.yaml,内容如下:

执行命令
kubectl apply -f kubesphere-complete-setup.yaml
参照 https://github.com/kubesphere/ks-installer/tree/master 修改部分配置
2、查看进度
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
如果看到以下信息,就代表安装成功了!!!

3、解决问题重启 installer
kubectl delete pod ks-installer-75b8d89dff-f7r2g -n kubesphere-system
4、metrics-server 部署
分类:
k8s
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· 微软正式发布.NET 10 Preview 1:开启下一代开发框架新篇章
· C# 集成 DeepSeek 模型实现 AI 私有化(本地部署与 API 调用教程)
· DeepSeek R1 简明指南:架构、训练、本地部署及硬件要求
· 没有源码,如何修改代码逻辑?
· NetPad:一个.NET开源、跨平台的C#编辑器