Kubernetes14-Helm：Kubernetes应用包管理

1、Helm概述

1.1、Helm介绍

• 随着容器技术逐渐被企业接受，在Kubernetes上已经能便捷地部署简单的应用了。但对于复杂的应用中间件，在Kubernetes上进行容器化部署并非易事，通常需要先研究Docker镜像的运行需求、环境变量等内容，并能为这些容器定制存储、网络等设置，最后设计和编写Deployment、ConfigMap、Service及Ingress等相关YAML配置文件，再提交给Kubernetes部署。这些复杂的过程将逐步被Helm应用包管理工具实现。

• Helm是一个由CNCF孵化和管理的项目，用于对需要在Kubernetes上部署的复杂应用进行定义、安装和更新。Helm以Chart的方式对应用软件进行描述，可以方便地创建、版本化、共享和发布复杂的应用软件。

• Helm是Kubernetes的软件包管理工具。包管理器类似Centos中使用的yum或者Python中的pip一样，能快速查找、下载和安装软件包。

1.2、Helm相关组件及概念

• Chart：一个Helm包，其中包含运行一个应用所需要的工具和资源定义，还可能包含Kubernetes集群中的服务定义。可以理解为包含了一组定义 Kubernetes 资源相关的YAML文件。
• Release：在Kubernetes集群上运行的一个Chart实例。在同一个集群上，一个Chart可以被安装多次。例如有一个MySQL Chart，如果想在服务器上运行两个MySQL数据库，就可以基于这个Chart安装两次。每次安装都会生成新的Release，会有独立的Release名称。
• Repository：Helm chart的仓库，Repository本质上是一个Web服务器，该服务器保存了一系列的Chart软件包以供用户下载，并且提供了一个该Repository的Chart包的清单文件以供查询。Helm可以同时管理多个不同的Repository。

• 简单来说，Helm整个系统的主要任务就是，在仓库中查找需要的Chart，然后将Chart以Release的形式安装到Kubernetes集群中。

1.3、Helm原理

• helm架构图
  • Helm v2是C/S架构，主要分为客户端helm和服务端Tiller。
  • Helm v3移除了Tiller，只有helm。

2、安装Helm v3

• HelmClient是一个客户端，拥有对Repository、Chart、Release等对象的管理能力。

• 使用二进制包安装helm
  • 下载HelmClient：https://get.helm.sh/helm-v3.8.2-linux-amd64.tar.gz

]# tar zvfx helm-v3.8.2-linux-amd64.tar.gz

//helm是一个命令行工具
]# ls -l ./linux-amd64/
-rwxr-xr-x 1 3434 3434 44630016 4月  14 01:47 helm
-rw-r--r-- 1 3434 3434    11373 4月  14 01:56 LICENSE
-rw-r--r-- 1 3434 3434     3367 4月  14 01:56 README.md

//将helm放到bin目录下
]# cp linux-amd64/helm /usr/local/bin/

• 查看安装版本

]# helm version
version.BuildInfo{Version:"v3.8.2", GitCommit:"6e3701edea09e5d55a8ca2aae03a68917630e91b", GitTreeState:"clean", GoVersion:"go1.17.5"}

3、Helm的基本使用

]# helm --help
Common actions for Helm:（Helm的常见动作）
  helm search：搜索图charts
  helm pull：下载charts到本地目录
  helm install：将charts部署到Kubernetes
  helm list：查看已经部署的charts的实例

Environment variables:（环境变量）
  $HELM_CACHE_HOME：设置存储缓存文件的位置。
  $HELM_CONFIG_HOME：设置配置文件。
  $HELM_DATA_HOME：设置数据的存储位置
  $HELM_DEBUG：是否在调试模式下运行
  $HELM_DRIVER：设置后端的存储驱动。取值为：configmap、secret、memory、sql。
  $HELM_DRIVER_SQL_CONNECTION_STRING：设置SQL存储驱动程序应该使用的连接字符串。
  $HELM_MAX_HISTORY：设置helm release history的最大次数。
  $HELM_NAMESPACE：设置命名空间。
  $HELM_NO_PLUGINS：禁用插件。设置HELM_NO_PLUGINS=1禁用插件。
  $HELM_PLUGINS：将路径设置为plugins目录
  $HELM_REGISTRY_CONFIG：设置注册表（registry）配置文件的路径。
  $HELM_REPOSITORY_CACHE：设置仓库（repository）缓存目录的路径。
  $HELM_REPOSITORY_CONFIG：设置仓库（repositories）文件的路径。
  $KUBECONFIG：设置Kubernetes认证配置文件(default "~/.kube/config")
  $HELM_KUBEAPISERVER：设置Kubernetes API服务器端点进行身份验证。
  $HELM_KUBECAFILE：设置Kubernetes证书颁发机构文件（CA）。
  $HELM_KUBEASGROUPS：使用逗号分隔的列表设置要用于模拟的组。
  $HELM_KUBEASUSER：设置用户名以模拟操作。
  $HELM_KUBECONTEXT：设置kubeconfig上下文（context）的名称。
  $HELM_KUBETOKEN：设置用于认证的不记名KubeToken。

Helm根据以下配置顺序存储缓存、配置和数据:
  如果设置了HELM_*_HOME环境变量，则将使用它
  否则，在支持XDG基本目录规范的系统上，将使用XDG变量
  当没有设置其他位置时，将根据操作系统使用默认位置

默认情况下，默认目录依赖于操作系统。默认值如下所示:
    Operating System     Cache Path                    Configuration Path                 Data Path               |
|------------------|---------------------------|--------------------------------|-------------------------|
    Linux                $HOME/.cache/helm             $HOME/.config/helm                 $HOME/.local/share/helm |
    macOS                $HOME/Library/Caches/helm     $HOME/Library/Preferences/helm     $HOME/Library/helm      |
    Windows              %TEMP%\helm                   %APPDATA%\helm                     %APPDATA%\helm          |

Usage:
  helm [command]

Available Commands:
  completion  为指定的shell生成自动完成脚本
  create      创建一个具有给定名称的新chart
  dependency  管理chart的依赖关系
  env         管理客户端环境信息
  get         查看指定release（实例）的扩展信息
  help        关于任何命令的帮助
  history     获取releases历史
  install     安装一个chart
  lint        检查chart中可能出现的问题
  list        releases列表
  package     将chart目录打包到chart存档中
  plugin      安装，列出，或卸载Helm插件
  pull        从仓库下载chart，并(可选)将其解压到本地目录中
  push        将chart推送到仓库
  registry    登录到注册中心或从注册中心注销
  repo        添加、列出、删除、更新和索引本地chart仓库
  rollback    将release回滚到以前的修订release
  search      在chart仓库中搜索chart
  show        显示chart的信息
  status      显示命名release的状态
  template    本地呈现模板
  test        为发行版运行测试
  uninstall   卸载一个release
  upgrade     升级一个release
  verify      验证给定路径上的chart是否已签名且有效
  version     打印客户端版本信息

Flags:
    --debug                       启用详细输出
  -h, --help                        帮助
      --kube-apiserver string       Kubernetes API server的地址和端口
      --kube-as-group stringArray   指定用户组。可以重复使用该参数以指定多个组。
      --kube-as-user string         指定用户
      --kube-ca-file string         Kubernetes API服务器连接的证书颁发机构文件（CA）
      --kube-context string         要使用的kubeconfig上下文的名称
      --kube-token string           用于身份验证的承载令牌
      --kubeconfig string           kubeconfig文件的路径
  -n, --namespace string            指定命名空间
      --registry-config string      注册表（registry）配置文件的路径(default "/root/.config/helm/registry/config.json")
      --repository-cache string     包含缓存仓库（repository）索引的文件的路径(default "/root/.cache/helm/repository")
      --repository-config string    包含仓库名称和url的文件的路径(default "/root/.config/helm/repositories.yaml")

Use "helm [command] --help" for more information about a command.

3.1、chart仓库（repository）

• 添加、删除、列出和索引本地chart repositories。

• helm官方：https://hub.helm.sh/
• 开源社镜像：
  • http://mirror.kaiyuanshe.cn/kubernetes/charts/
  • http://mirror.azure.cn/kubernetes/charts/

]# helm repo --help
Usage:
  helm repo [command]

Available Commands:
  add         将一个chart仓库添加到本地
  index       给定一个包含打包chart的目录，生成一个索引文件
  list        列出本地chart仓库
  remove      删除一个或多个本地chart仓库
  update      更新本地chart仓库

示例：

//添加chart仓库到本地（文件加载到了${HOME}/.cache/helm/repository/）
]# helm repo add kaiyuanshe http://mirror.kaiyuanshe.cn/kubernetes/charts/
]# helm repo add azure http://mirror.azure.cn/kubernetes/charts/
]# helm repo add bitnami https://charts.bitnami.com/bitnami

//列出本地chart仓库
]# helm repo list
NAME      	URL                                           
azure     	http://mirror.azure.cn/kubernetes/charts/     
kaiyuanshe	http://mirror.kaiyuanshe.cn/kubernetes/charts/
bitnami   	https://charts.bitnami.com/bitnami

//删除本地chart仓库
]# helm repo remove kaiyuanshe

//更新本地chart仓库
]# helm repo update

可能可以使用的helm chart网址

helm repo add stable https://charts.helm.sh/stable
helm repo add aliyunstable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add azureincubator https://mirror.azure.cn/kubernetes/charts-incubator/
helm repo add gitlab https://charts.gitlab.io
helm repo add kiwigrid https://kiwigrid.github.io
helm repo add azure https://mirror.azure.cn/kubernetes/charts/
helm repo add aliyun https://apphub.aliyuncs.com/
helm repo add saleor https://k.github.io/saleor-helm
helm repo add elastic https://helm.elastic.co
helm repo add harbor https://helm.goharbor.io
helm repo add reactiveops-stable https://charts.reactiveops.com/stable
helm repo add buttahtoast https://buttahtoast.github.io/helm-charts/
helm repo add bedag https://bedag.github.io/helm-charts/
helm repo add grafana https://grafana.github.io/helm-charts
helm repo add prometheus https://prometheus-community.github.io/helm-charts
helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo add minio https://helm.min.io
helm repo add incubator https://charts.helm.sh/incubator
helm repo add codimd https://helm.codimd.dev/
helm repo add ibm https://raw.githubusercontent.com/IBM/charts/master/repo/stable
helm repo add ibm-entitled https://raw.githubusercontent.com/IBM/charts/master/repo/entitled
helm repo add ibm-community https://raw.githubusercontent.com/IBM/charts/master/repo/community
helm repo add eks https://aws.github.io/eks-charts
helm repo add argo https://argoproj.github.io/argo-helm
helm repo add microsoft https://microsoft.github.io/charts/repo
helm repo add privatebin https://privatebin.github.io/helm-chart
helm repo add jetstack https://charts.jetstack.io
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo add fabric8 https://fabric8.io/helm
helm repo add taito-charts https://taitounited.github.io/taito-charts
helm repo add yugabytedb https://charts.yugabyte.com
helm repo add strimzi https://strimzi.io/charts/
helm repo add infracloudio https://infracloudio.github.io/charts
helm repo add stackstorm https://helm.stackstorm.com/
helm repo add gresearch https://g-research.github.io/charts/
helm repo add kubeless-functions-charts https://kubeless-functions-charts.storage.googleapis.com
helm repo add openebs-charts https://openebs.github.io/charts/
helm repo add jenkins https://charts.jenkins.io
helm repo add openfaas https://openfaas.github.io/faas-netes
helm repo add agones https://agones.dev/chart/stable
helm repo add kong https://charts.konghq.com
helm repo add appscode https://charts.appscode.com/stable/
helm repo add banzaicloud-stable https://kubernetes-charts.banzaicloud.com/branch/master
helm repo add presslabs https://presslabs.github.io/charts
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo add nvidiaoperator https://nvidia.github.io/gpu-operator
helm repo add akomljen-charts https://raw.githubusercontent.com/komljen/helm-charts/master/charts/
helm repo add fluent https://fluent.github.io/helm-charts
helm repo add fluxcd https://charts.fluxcd.io
 
helm repo add kubesphere https://charts.kubesphere.io/main
helm repo add influx https://influx-charts.storage.googleapis.com
 
helm repo add mesosphere-staging https://mesosphere.github.io/charts/staging
helm repo add hazelcast https://hazelcast-charts.s3.amazonaws.com/
helm repo add tscharts https://technosophos.github.com/tscharts
helm repo add ibm-helm https://raw.githubusercontent.com/IBM/charts/master/repo/ibm-helm
helm repo add buildkite https://buildkite.github.io/charts/
helm repo add fairwinds-stable https://charts.fairwinds.com/stable
helm repo add influxdata https://helm.influxdata.com/
helm repo add monocular https://helm.github.io/monocular
helm repo add oteemocharts https://oteemo.github.io/charts
helm repo add honeycomb https://honeycombio.github.io/helm-charts
helm repo add tricksterproxy https://helm.tricksterproxy.io/
helm repo add mogaal https://mogaal.github.io/helm-charts/
helm repo add soluto https://charts.soluto.io
helm repo add logicmonitor https://logicmonitor.github.io/k8s-helm-charts
helm repo add everpeace https://everpeace.github.io/helm-charts
helm repo add kubevious https://helm.kubevious.io
helm repo add nginx-stable https://helm.nginx.com/stable
helm repo add nginx-edge https://helm.nginx.com/edge
helm repo add redhat-charts https://redhat-developer.github.io/redhat-helm-charts
helm repo add center https://repo.chartcenter.io
helm repo add kongz https://charts.kong-z.com
helm repo add streamnative https://charts.streamnative.io
helm repo add loki https://grafana.github.io/loki/charts
helm repo add lensesio https://lensesio.github.io/kafka-helm-charts/
helm repo add cribl https://criblio.github.io/helm-charts/
helm repo add wunderio https://storage.googleapis.com/charts.wdr.io
helm repo add mvisonneau https://charts.visonneau.fr/
helm repo add lowess-helm-charts https://lowess.github.io/helm-charts/
helm repo add tkestackstable https://raw.githubusercontent.com/tkestack/charts/master/repo/stable
helm repo add gruntwork https://helmcharts.gruntwork.io
helm repo add 3loc https://3loc.github.io/charts
helm repo add aerokube https://charts.aerokube.com/
helm repo add preferred-ai https://charts.preferred.ai
helm repo add spotinst https://spotinst.github.io/spotinst-kubernetes-helm-charts
helm repo add dex https://charts.dexidp.io
helm repo add verdaccio https://charts.verdaccio.org
helm repo add devstack https://apps.devstack.com
helm repo add cyberark https://cyberark.github.io/helm-charts
helm repo add containersolutions https://containersolutions.github.io/helm-charts/
helm repo add pcktdmp https://pcktdmp.github.io/charts
helm repo add trieb.work https://trieb-work.github.io/helm-charts/
helm repo add curity https://curityio.github.io/idsvr-helm/
helm repo add preferred-ai https://charts.preferred.ai
helm repo add dlc https://dcos-labs.github.io/charts/
helm repo add supergiant https://supergiant.github.io/charts
 
helm repo add juliohm1978 https://raw.githubusercontent.com/juliohm1978/chart-repo-example/master/index
helm repo add whyeasy-helm-charts https://whyeasy.github.io/helm-charts
helm repo add utkuozdemir https://utkuozdemir.org/helm-charts
helm repo add vsphere-tmm https://vsphere-tmm.github.io/helm-charts
helm repo add kilip https://charts.itstoni.com/
helm repo add dumrauf https://dumrauf-helm-charts-repo.s3.amazonaws.com
helm repo add shakahl https://shakahl.github.io/helm-charts/
helm repo add aquasecurity https://aquasecurity.github.io/helm-charts/
helm repo add gocd https://gocd.github.io/helm-chart
helm repo add emagtechlabs  https://emagtechlabs.github.io/helm-charts/
helm repo add lsstsqre https://lsst-sqre.github.io/charts/
helm repo add estahn https://estahn.github.io/charts/
helm repo add fairfaxmedia https://charts.ffx.io/
helm repo add spot https://charts.spot.io
helm repo add googlestable https://kubernetes-charts.storage.googleapis.com/
# helm repo add emc-mongoose https://emc-mongoose.github.io/mongoose-helm-charts/
helm repo add equinor-charts https://equinor.github.io/helm-charts/charts/
helm repo add cloudflow-helm-charts https://lightbend.github.io/cloudflow-helm-charts/
 
helm repo add graphql-gateway https://raw.githubusercontent.com/hansehe/graphql-gateway/master/helm/charts
helm repo add drpsychick https://drpsychick.github.io/charts
helm repo add csi-hostpath https://speedwing.github.io/csi-hostpath-helm
helm repo add kesterriley-repo https://kesterriley.github.io/my-helm-charts/
helm repo add estafette https://helm.estafette.io
helm repo add helm-consensys https://artifacts.consensys.net/public/helm/helm/charts/
helm repo add balle-petersen https://charts.balle-petersen.org
helm repo add rossumai https://rossumai.github.io/helm-charts
helm repo add curiedfcharts https://curie-data-factory.github.io/helm-charts
helm repo add maxirus https://maxirus.github.io/helm-charts/
helm repo add rookout https://helm-charts.rookout.com
helm repo add naps https://naps.github.io/helm-charts/
helm repo add jacobhelm https://jacobbaek.github.io/helmcharts
helm repo add nervos https://k.github.io/nervos-helm
helm repo add kubenav https://kubenav.github.io/helm-repository
helm repo add graillus https://graillus.github.io/helm-charts
helm repo add fikaworks https://fikaworks.github.io/helm-charts
helm repo add adamrushuk https://adamrushuk.github.io/charts
helm repo add ealenn https://ealenn.github.io/charts
helm repo add locust https://raw.githubusercontent.com/hansehe/locust/master/helm/charts
helm repo add ot-helm https://ot-container-kit.github.io/helm-charts
helm repo add nlnwa https://nlnwa.github.io/charts
helm repo add jmriebold https://jmriebold.github.io/charts
helm repo add ChronixDB https://raw.githubusercontent.com/ChronixDB/chronix.ops/master/helm/repo/
helm repo add avanier https://avanier.github.io/ax-helm-charts/
helm repo add jx3 https://storage.googleapis.com/jenkinsxio/charts
helm repo add sushi https://helm.sushivalidator.com
helm repo add haproxytech https://haproxytech.github.io/helm-charts
# helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
helm repo add rancher-latest https://releases.rancher.com/server-charts/latest
helm repo add keel https://charts.keel.sh
helm repo add rancher-stable     https://releases.rancher.com/server-charts/stable
helm repo add helm-cabin https://nick-triller.github.io/helm-cabin/
helm repo add snyk-charts https://snyk.github.io/kubernetes-monitor/
helm repo add portainer https://portainer.github.io/k8s/
helm repo add spark-operator https://googlecloudplatform.github.io/spark-on-k8s-operator
helm repo add podinfo https://stefanprodan.github.io/podinfo
 
helm repo add uswitch https://uswitch.github.io/kiam-helm-charts/charts/
helm repo add longhorn https://charts.longhorn.io
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add kubevela https://charts.kubevela.net/core
helm repo add pravega https://charts.pravega.io
helm repo add mojo2600 https://mojo2600.github.io/pihole-kubernetes/
helm repo add openwhisk https://openwhisk.apache.org/charts
helm repo add kubeinvaders https://lucky-sideburn.github.io/helm-charts/
helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
helm repo add scylla-operator https://storage.googleapis.com/scylla-operator-charts/stable
helm repo add artifact-hub https://artifacthub.github.io/hub/chart/
helm repo add mattermost https://helm.mattermost.com
helm repo add crossplane-master https://charts.crossplane.io/master/
helm repo add k8s-at-home https://k8s-at-home.com/charts/
helm repo add runix https://helm.runix.net
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo add astronomer https://helm.astronomer.io
helm repo add kedacore https://kedacore.github.io/charts
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
 
helm repo add katafygio https://bpineau.github.io/katafygio
helm repo add yunikorn  https://apache.github.io/incubator-yunikorn-release
helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
 
helm repo add kubernetic https://harbur.github.io/kubernetic-charts
helm repo add kuma https://kumahq.github.io/charts
helm repo add stratos https://cloudfoundry.github.io/stratos
helm repo add sonatype https://sonatype.github.io/helm3-charts/
helm repo add traefik https://helm.traefik.io/traefik
helm repo add mittwald https://helm.mittwald.de
helm repo add containerum https://charts.containerum.io
helm repo add bokysan https://bokysan.github.io/docker-postfix/
helm repo add zaqar https://lsantos.me/zaqar/helm
helm repo add mailu https://mailu.github.io/helm-charts/
helm repo add k8s-ovpn https://raw.githubusercontent.com/suda/k8s-ovpn-chart/master/
 
helm repo add atlassian-data-center https://atlassian.github.io/data-center-helm-charts
helm repo add ygqygq2 https://ygqygq2.github.io/charts/
helm repo add bytectl https://bytectl.github.io/helm-charts
helm repo add edu https://iac-source.github.io/helm-charts
helm repo add jenkins-x https://jenkins-x-charts.github.io/v2/
helm repo add stakater https://stakater.github.io/stakater-charts
helm repo add opencord https://charts.opencord.org/
helm repo add twuni https://helm.twun.io
helm repo add truecharts https://truecharts.org/
helm repo add deliveryhero https://charts.deliveryhero.io/
helm repo add vineyard https://vineyard.oss-ap-southeast-1.aliyuncs.com/charts/
 
helm repo add gmelillo https://helm.melillo.me
helm repo add cloudnativeapp https://cloudnativeapp.github.io/charts/curated/
 
helm repo add k8tz https://k8tz.github.io/k8tz/
helm repo add dirigible https://eclipse.github.io/dirigible
helm repo add budibase https://budibase.github.io/budibase/
 
helm repo add billimek https://billimek.com/billimek-charts/
helm repo add node-red https://schwarzit.github.io/node-red-chart/
helm repo add cocainefarm https://kube.cat/chartrepo/cocainefarm
helm repo add genuitec https://genuitec.github.io/CodeTogether/helm/
helm repo add taskmedia https://helm.task.media
helm repo add okgolove https://okgolove.github.io/helm-charts/
helm repo add devtron-labs https://raw.githubusercontent.com/devtron-labs/charts/gh-pages
helm repo add cert-manager-alidns-webhook https://devmachine-fr.github.io/cert-manager-alidns-webhook
helm repo add truecharts2 https://charts.truecharts.org/
 
helm repo add vsphere-cpi https://kubernetes.github.io/cloud-provider-vsphere
 
helm repo update

3.2、搜索Chart

]# helm search --help
Usage:
  helm search [command]

Available Commands:
  hub         在公网上的chart仓库中搜索chart，默认chart仓库是https://hub.helm.sh
  repo        在本地chart仓库中搜索chart，不需要连接公网。

示例：

//搜索mongodb
]# helm search repo mongodb
]# helm search hub mongodb

3.3、查看Chart的信息

]# helm show -h
Usage:
  helm show [command]

Aliases:
  show, inspect

Available Commands:
  all         显示chart的所有信息
  chart       显示chart的定义（definition）
  crds        显示chart的CRDs
  readme      显示chart的README
  values      显示chart的values

3.4、下载、上传chart

//从仓库下载chart
helm pull [chart URL | repo/chartname] [...] [flags]
//将chart推送到仓库
helm push [chart] [remote] [flags]

示例：

//下载到mongodb的chart到本地当前目录
]# helm pull bitnami/mongodb

3.5、运行并管理release

1、运行release

//使用仓库直接运行release
]# helm install mongodb-name1 bitnami/mongodb
//使用压缩包运行release
]# helm install mongodb-name2 mongodb-13.1.7.tgz
//使用解压后的目录运行release
]# helm install mongodb-name3 mongodb/

2、查看已经运行的release

]# helm list
NAME        	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART         	APP VERSION
mongodb-name1	default  	1       	2022-10-09 17:54:13.504601252 +0800 CST	deployed	mongodb-13.1.7	6.0.2 

3、获取release的状态

]# helm status mongodb-name1

4、删除releases

]# helm uninstall mongodb-name1

5、获取releases部署历史

]# helm history mongodb-name2
REVISION	UPDATED                 	STATUS  	CHART         	APP VERSION	DESCRIPTION     
1       	Sun Oct  9 17:54:13 2022	deployed	mongodb-13.1.7	6.0.2      	Install complete

6、升级和回滚

//升级一个release
helm upgrade [RELEASE] [CHART] [flags]
//回滚release
helm rollback <RELEASE> [REVISION] [flags]

4、部署redis集群

• 部署前要先实现实现StorageClass（动态创建PV）。
  • 共享存储查看：https://www.cnblogs.com/maiblogs/p/16392831.html，这里使用“6.2、动态存储卷”。

1、创建名称空间

]# kubectl create namespace redis-namespace

2、部署redis集群

//获取chart仓库
]# helm repo add bitnami https://charts.bitnami.com/bitnami

//获取redis的chart
]# helm search repo redis
NAME                                	CHART VERSION	APP VERSION	DESCRIPTION                                       
bitnami/redis                       	17.3.2       	7.0.5      	Redis(R) is an open source, advanced key-value ...
bitnami/redis-cluster               	8.2.4        	7.0.5      	Redis(R) is an open source, scalable, distribut...
...

//部署bitnami/redis-cluster到k8s（storageClass的名字是nfs-client）
helm install my-release bitnami/redis-cluster -n redis-namespace --set "persistence.storageClass=nfs-client"

3、查看

//查看server
]# kubectl get svc -n redis-namespace
NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)              AGE
my-release-redis-cluster            ClusterIP   10.20.161.245   <none>        6379/TCP             11m
my-release-redis-cluster-headless   ClusterIP   None            <none>        6379/TCP,16379/TCP   11m

//查看pods
]# kubectl get pods -n redis-namespace
NAME                         READY   STATUS    RESTARTS   AGE
my-release-redis-cluster-0   1/1     Running   1          12m
my-release-redis-cluster-1   1/1     Running   1          12m
my-release-redis-cluster-2   1/1     Running   1          12m
my-release-redis-cluster-3   1/1     Running   1          12m
my-release-redis-cluster-4   1/1     Running   1          12m
my-release-redis-cluster-5   1/1     Running   1          12m

//查看pvc
]# kubectl get pvc -n redis-namespace
NAME                                    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
redis-data-my-release-redis-cluster-0   Bound    pvc-7b1b341c-19a3-4d2c-853f-ee82b5dc4622   8Gi        RWO            nfs-client     12m
redis-data-my-release-redis-cluster-1   Bound    pvc-c8cd942a-027a-4c7a-aa13-c9dec2a0e081   8Gi        RWO            nfs-client     12m
redis-data-my-release-redis-cluster-2   Bound    pvc-9fceae30-2a73-4bcd-8da5-88d40c93a060   8Gi        RWO            nfs-client     12m
redis-data-my-release-redis-cluster-3   Bound    pvc-d7ee0fbb-8b87-4a2e-93d5-6fef1f78273e   8Gi        RWO            nfs-client     12m
redis-data-my-release-redis-cluster-4   Bound    pvc-7f4d2476-1261-4429-b1c4-77c8a701d97a   8Gi        RWO            nfs-client     12m
redis-data-my-release-redis-cluster-5   Bound    pvc-e816270f-0150-4b64-9016-2dca13abedce   8Gi        RWO            nfs-client     12m

//查看pv
]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                   STORAGECLASS   REASON   AGE
pvc-7b1b341c-19a3-4d2c-853f-ee82b5dc4622   8Gi        RWO            Delete           Bound    redis-namespace/redis-data-my-release-redis-cluster-0   nfs-client              12m
pvc-7f4d2476-1261-4429-b1c4-77c8a701d97a   8Gi        RWO            Delete           Bound    redis-namespace/redis-data-my-release-redis-cluster-4   nfs-client              12m
pvc-9fceae30-2a73-4bcd-8da5-88d40c93a060   8Gi        RWO            Delete           Bound    redis-namespace/redis-data-my-release-redis-cluster-2   nfs-client              12m
pvc-c8cd942a-027a-4c7a-aa13-c9dec2a0e081   8Gi        RWO            Delete           Bound    redis-namespace/redis-data-my-release-redis-cluster-1   nfs-client              12m
pvc-d7ee0fbb-8b87-4a2e-93d5-6fef1f78273e   8Gi        RWO            Delete           Bound    redis-namespace/redis-data-my-release-redis-cluster-3   nfs-client              12m
pvc-e816270f-0150-4b64-9016-2dca13abedce   8Gi        RWO            Delete           Bound    redis-namespace/redis-data-my-release-redis-cluster-5   nfs-client              12m

4、登录redis

//启动redis客户端
]# kubectl run --namespace redis-namespace my-release-redis-cluster-client --rm --tty -i --restart='Never' \
    --env REDIS_PASSWORD=$REDIS_PASSWORD \
    --image docker.io/bitnami/redis-cluster:7.0.5-debian-11-r0 -- bash

//登录redis服务器
/$ redis-cli -c -h my-release-redis-cluster -a euhiu3dXmQ

//查看集群节点
6379> cluster nodes
a18a5aa0a16f2fd9d389fe83a39e1bcf28611d3d 10.10.2.4:6379@16379 master - 0 1665371671970 1 connected 0-5460
de4021c3c1611382270a90cad0bb43025d24e7b2 10.10.2.6:6379@16379 slave a18a5aa0a16f2fd9d389fe83a39e1bcf28611d3d 0 1665371670000 1 connected
b54b8cb69585febff3d46420e6b99a1a2c244232 10.10.1.5:6379@16379 master - 0 1665371672000 2 connected 5461-10922
8de10b6d34d3afcebc39120bcd28a07306140eef 10.10.1.6:6379@16379 slave b54b8cb69585febff3d46420e6b99a1a2c244232 0 1665371671000 2 connected
8b72dfdaac832d01c3669e9086c0272909eb256b 10.10.2.5:6379@16379 myself,master - 0 1665371669000 3 connected 10923-16383
bc55361f4715ed0ecaeab2b455d722e0f871b732 10.10.1.7:6379@16379 slave 8b72dfdaac832d01c3669e9086c0272909eb256b 0 1665371672980 3 connected

5、查看service和statefulset的配置文件

• 查看无头service

]# kubectl edit service my-release-redis-cluster -n redis-namespace
apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: my-release
    meta.helm.sh/release-namespace: redis-namespace
  creationTimestamp: "2022-10-10T02:43:08Z"
  labels:
    app.kubernetes.io/instance: my-release
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: redis-cluster
    helm.sh/chart: redis-cluster-8.2.4
  name: my-release-redis-cluster
  namespace: redis-namespace
  resourceVersion: "2055"
  uid: c0f5afe5-a1ba-4d2c-89ec-912f6911e3e9
spec:
  clusterIP: 10.20.161.245
  clusterIPs:
  - 10.20.161.245
  ports:
  - name: tcp-redis
    port: 6379
    protocol: TCP
    targetPort: tcp-redis
  selector:
    app.kubernetes.io/instance: my-release
    app.kubernetes.io/name: redis-cluster
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

• 查看service

]# kubectl edit service my-release-redis-cluster-headless -n redis-namespace
apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: my-release
    meta.helm.sh/release-namespace: redis-namespace
  creationTimestamp: "2022-10-10T02:43:08Z"
  labels:
    app.kubernetes.io/instance: my-release
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: redis-cluster
    helm.sh/chart: redis-cluster-8.2.4
  name: my-release-redis-cluster-headless
  namespace: redis-namespace
  resourceVersion: "2053"
  uid: 64e1dd30-ebc1-4f04-9488-14cb13cd3a2f
spec:
  clusterIP: None
  clusterIPs:
  - None
  ports:
  - name: tcp-redis
    port: 6379
    protocol: TCP
    targetPort: tcp-redis
  - name: tcp-redis-bus
    port: 16379
    protocol: TCP
    targetPort: tcp-redis-bus
  publishNotReadyAddresses: true
  selector:
    app.kubernetes.io/instance: my-release
    app.kubernetes.io/name: redis-cluster
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

• 查看StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  annotations:
    meta.helm.sh/release-name: my-release
    meta.helm.sh/release-namespace: redis-namespace
  creationTimestamp: "2022-10-10T02:43:09Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: my-release
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: redis-cluster
    helm.sh/chart: redis-cluster-8.2.4
  name: my-release-redis-cluster
  namespace: redis-namespace
  resourceVersion: "2582"
  uid: 38181325-2620-4290-b471-5646374ecb07
spec:
  podManagementPolicy: Parallel
  replicas: 6
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: my-release
      app.kubernetes.io/name: redis-cluster
  serviceName: my-release-redis-cluster-headless
  template:
    metadata:
      annotations:
        checksum/config: edb77e240c0c6f850a0491f3ee75b41734497b84c85758df6333acad7c2c322e
        checksum/scripts: 34a440ce2ff43b6313a12562e0482364dd1fe3d2fe1ad9bb5c92170d6e55b19f
        checksum/secret: 0613daff23bf184e662d188826eaf2691eb74e4af7a35604998c942da6d6b879
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: my-release
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: redis-cluster
        helm.sh/chart: redis-cluster-8.2.4
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchLabels:
                  app.kubernetes.io/instance: my-release
                  app.kubernetes.io/name: redis-cluster
              namespaces:
              - redis-namespace
              topologyKey: kubernetes.io/hostname
            weight: 1
      containers:
      - args:
        - |
          # Backwards compatibility change
          if ! [[ -f /opt/bitnami/redis/etc/redis.conf ]]; then
              echo COPYING FILE
              cp  /opt/bitnami/redis/etc/redis-default.conf /opt/bitnami/redis/etc/redis.conf
          fi
          pod_index=($(echo "$POD_NAME" | tr "-" "\n"))
          pod_index="${pod_index[-1]}"
          if [[ "$pod_index" == "0" ]]; then
            export REDIS_CLUSTER_CREATOR="yes"
            export REDIS_CLUSTER_REPLICAS="1"
          fi
          /opt/bitnami/scripts/redis-cluster/entrypoint.sh /opt/bitnami/scripts/redis-cluster/run.sh
        command:
        - /bin/bash
        - -c
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: REDIS_NODES
          value: 'my-release-redis-cluster-0.my-release-redis-cluster-headless my-release-redis-cluster-1.my-release-redis-cluster-headless
            my-release-redis-cluster-2.my-release-redis-cluster-headless my-release-redis-cluster-3.my-release-redis-cluster-headless
            my-release-redis-cluster-4.my-release-redis-cluster-headless my-release-redis-cluster-5.my-release-redis-cluster-headless '
        - name: REDISCLI_AUTH
          valueFrom:
            secretKeyRef:
              key: redis-password
              name: my-release-redis-cluster
        - name: REDIS_PASSWORD
          valueFrom:
            secretKeyRef:
              key: redis-password
              name: my-release-redis-cluster
        - name: REDIS_AOF_ENABLED
          value: "yes"
        - name: REDIS_TLS_ENABLED
          value: "no"
        - name: REDIS_PORT
          value: "6379"
        image: docker.io/bitnami/redis-cluster:7.0.5-debian-11-r0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          exec:
            command:
            - sh
            - -c
            - /scripts/ping_liveness_local.sh 5
          failureThreshold: 5
          initialDelaySeconds: 5
          periodSeconds: 5
          successThreshold: 1
          timeoutSeconds: 6
        name: my-release-redis-cluster
        ports:
        - containerPort: 6379
          name: tcp-redis
          protocol: TCP
        - containerPort: 16379
          name: tcp-redis-bus
          protocol: TCP
        readinessProbe:
          exec:
            command:
            - sh
            - -c
            - /scripts/ping_readiness_local.sh 1
          failureThreshold: 5
          initialDelaySeconds: 5
          periodSeconds: 5
          successThreshold: 1
          timeoutSeconds: 2
        resources: {}
        securityContext:
          runAsNonRoot: true
          runAsUser: 1001
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /scripts
          name: scripts
        - mountPath: /bitnami/redis/data
          name: redis-data
        - mountPath: /opt/bitnami/redis/etc/redis-default.conf
          name: default-config
          subPath: redis-default.conf
        - mountPath: /opt/bitnami/redis/etc/
          name: redis-tmp-conf
      dnsPolicy: ClusterFirst
      enableServiceLinks: false
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1001
        runAsUser: 1001
      serviceAccount: default
      serviceAccountName: default
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 493
          name: my-release-redis-cluster-scripts
        name: scripts
      - configMap:
          defaultMode: 420
          name: my-release-redis-cluster-default
        name: default-config
      - emptyDir: {}
        name: redis-tmp-conf
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: my-release
        app.kubernetes.io/name: redis-cluster
      name: redis-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 8Gi
      storageClassName: nfs-client
      volumeMode: Filesystem
    status:
      phase: Pending
status:
  collisionCount: 0
  currentReplicas: 6
  currentRevision: my-release-redis-cluster-577c47b8dd
  observedGeneration: 1
  readyReplicas: 6
  replicas: 6
  updateRevision: my-release-redis-cluster-577c47b8dd
  updatedReplicas: 6

1

#                                                                                                                                       #