Kubernetes14-Helm:Kubernetes应用包管理
1、Helm概述
1.1、Helm介绍
- 随着容器技术逐渐被企业接受,在Kubernetes上已经能便捷地部署简单的应用了。但对于复杂的应用中间件,在Kubernetes上进行容器化部署并非易事,通常需要先研究Docker镜像的运行需求、环境变量等内容,并能为这些容器定制存储、网络等设置,最后设计和编写Deployment、ConfigMap、Service及Ingress等相关YAML配置文件,再提交给Kubernetes部署。这些复杂的过程将逐步被Helm应用包管理工具实现。
- Helm是一个由CNCF孵化和管理的项目,用于对需要在Kubernetes上部署的复杂应用进行定义、安装和更新。Helm以Chart的方式对应用软件进行描述,可以方便地创建、版本化、共享和发布复杂的应用软件。
- Helm是Kubernetes的软件包管理工具。包管理器类似Centos中使用的yum或者Python中的pip一样,能快速查找、下载和安装软件包。
1.2、Helm相关组件及概念
- Chart:一个Helm包,其中包含运行一个应用所需要的工具和资源定义,还可能包含Kubernetes集群中的服务定义。可以理解为包含了一组定义 Kubernetes 资源相关的YAML文件。
- Release:在Kubernetes集群上运行的一个Chart实例。在同一个集群上,一个Chart可以被安装多次。例如有一个MySQL Chart,如果想在服务器上运行两个MySQL数据库,就可以基于这个Chart安装两次。每次安装都会生成新的Release,会有独立的Release名称。
- Repository:Helm chart的仓库,Repository本质上是一个Web服务器,该服务器保存了一系列的Chart软件包以供用户下载,并且提供了一个该Repository的Chart包的清单文件以供查询。Helm可以同时管理多个不同的Repository。
- 简单来说,Helm整个系统的主要任务就是,在仓库中查找需要的Chart,然后将Chart以Release的形式安装到Kubernetes集群中。
1.3、Helm原理
- helm架构图
- Helm v2是C/S架构,主要分为客户端helm和服务端Tiller。
- Helm v3移除了Tiller,只有helm。
2、安装Helm v3
- HelmClient是一个客户端,拥有对Repository、Chart、Release等对象的管理能力。
- 使用二进制包安装helm
- 下载HelmClient:https://get.helm.sh/helm-v3.8.2-linux-amd64.tar.gz
]# tar zvfx helm-v3.8.2-linux-amd64.tar.gz //helm是一个命令行工具 ]# ls -l ./linux-amd64/ -rwxr-xr-x 1 3434 3434 44630016 4月 14 01:47 helm -rw-r--r-- 1 3434 3434 11373 4月 14 01:56 LICENSE -rw-r--r-- 1 3434 3434 3367 4月 14 01:56 README.md //将helm放到bin目录下 ]# cp linux-amd64/helm /usr/local/bin/
- 查看安装版本
]# helm version version.BuildInfo{Version:"v3.8.2", GitCommit:"6e3701edea09e5d55a8ca2aae03a68917630e91b", GitTreeState:"clean", GoVersion:"go1.17.5"}
3、Helm的基本使用
]# helm --help Common actions for Helm:(Helm的常见动作) helm search:搜索图charts helm pull:下载charts到本地目录 helm install:将charts部署到Kubernetes helm list:查看已经部署的charts的实例 Environment variables:(环境变量) $HELM_CACHE_HOME:设置存储缓存文件的位置。 $HELM_CONFIG_HOME:设置配置文件。 $HELM_DATA_HOME:设置数据的存储位置 $HELM_DEBUG:是否在调试模式下运行 $HELM_DRIVER:设置后端的存储驱动。取值为:configmap、secret、memory、sql。 $HELM_DRIVER_SQL_CONNECTION_STRING:设置SQL存储驱动程序应该使用的连接字符串。 $HELM_MAX_HISTORY:设置helm release history的最大次数。 $HELM_NAMESPACE:设置命名空间。 $HELM_NO_PLUGINS:禁用插件。设置HELM_NO_PLUGINS=1禁用插件。 $HELM_PLUGINS:将路径设置为plugins目录 $HELM_REGISTRY_CONFIG:设置注册表(registry)配置文件的路径。 $HELM_REPOSITORY_CACHE:设置仓库(repository)缓存目录的路径。 $HELM_REPOSITORY_CONFIG:设置仓库(repositories)文件的路径。 $KUBECONFIG:设置Kubernetes认证配置文件(default "~/.kube/config") $HELM_KUBEAPISERVER:设置Kubernetes API服务器端点进行身份验证。 $HELM_KUBECAFILE:设置Kubernetes证书颁发机构文件(CA)。 $HELM_KUBEASGROUPS:使用逗号分隔的列表设置要用于模拟的组。 $HELM_KUBEASUSER:设置用户名以模拟操作。 $HELM_KUBECONTEXT:设置kubeconfig上下文(context)的名称。 $HELM_KUBETOKEN:设置用于认证的不记名KubeToken。 Helm根据以下配置顺序存储缓存、配置和数据: 如果设置了HELM_*_HOME环境变量,则将使用它 否则,在支持XDG基本目录规范的系统上,将使用XDG变量 当没有设置其他位置时,将根据操作系统使用默认位置 默认情况下,默认目录依赖于操作系统。默认值如下所示: Operating System Cache Path Configuration Path Data Path | |------------------|---------------------------|--------------------------------|-------------------------| Linux $HOME/.cache/helm $HOME/.config/helm $HOME/.local/share/helm | macOS $HOME/Library/Caches/helm $HOME/Library/Preferences/helm $HOME/Library/helm | Windows %TEMP%\helm %APPDATA%\helm %APPDATA%\helm | Usage: helm [command] Available Commands: completion 为指定的shell生成自动完成脚本 create 创建一个具有给定名称的新chart dependency 管理chart的依赖关系 env 管理客户端环境信息 get 查看指定release(实例)的扩展信息 help 关于任何命令的帮助 history 获取releases历史 install 安装一个chart lint 检查chart中可能出现的问题 list releases列表 package 将chart目录打包到chart存档中 plugin 安装,列出,或卸载Helm插件 pull 从仓库下载chart,并(可选)将其解压到本地目录中 push 将chart推送到仓库 registry 登录到注册中心或从注册中心注销 repo 添加、列出、删除、更新和索引本地chart仓库 rollback 将release回滚到以前的修订release search 在chart仓库中搜索chart show 显示chart的信息 status 显示命名release的状态 template 本地呈现模板 test 为发行版运行测试 uninstall 卸载一个release upgrade 升级一个release verify 验证给定路径上的chart是否已签名且有效 version 打印客户端版本信息 Flags: --debug 启用详细输出 -h, --help 帮助 --kube-apiserver string Kubernetes API server的地址和端口 --kube-as-group stringArray 指定用户组。可以重复使用该参数以指定多个组。 --kube-as-user string 指定用户 --kube-ca-file string Kubernetes API服务器连接的证书颁发机构文件(CA) --kube-context string 要使用的kubeconfig上下文的名称 --kube-token string 用于身份验证的承载令牌 --kubeconfig string kubeconfig文件的路径 -n, --namespace string 指定命名空间 --registry-config string 注册表(registry)配置文件的路径(default "/root/.config/helm/registry/config.json") --repository-cache string 包含缓存仓库(repository)索引的文件的路径(default "/root/.cache/helm/repository") --repository-config string 包含仓库名称和url的文件的路径(default "/root/.config/helm/repositories.yaml") Use "helm [command] --help" for more information about a command.
3.1、chart仓库(repository)
- 添加、删除、列出和索引本地chart repositories。
- helm官方:https://hub.helm.sh/
- 开源社镜像:
- http://mirror.kaiyuanshe.cn/kubernetes/charts/
- http://mirror.azure.cn/kubernetes/charts/
]# helm repo --help Usage: helm repo [command] Available Commands: add 将一个chart仓库添加到本地 index 给定一个包含打包chart的目录,生成一个索引文件 list 列出本地chart仓库 remove 删除一个或多个本地chart仓库 update 更新本地chart仓库
示例:
//添加chart仓库到本地(文件加载到了${HOME}/.cache/helm/repository/) ]# helm repo add kaiyuanshe http://mirror.kaiyuanshe.cn/kubernetes/charts/ ]# helm repo add azure http://mirror.azure.cn/kubernetes/charts/ ]# helm repo add bitnami https://charts.bitnami.com/bitnami //列出本地chart仓库 ]# helm repo list NAME URL azure http://mirror.azure.cn/kubernetes/charts/ kaiyuanshe http://mirror.kaiyuanshe.cn/kubernetes/charts/ bitnami https://charts.bitnami.com/bitnami //删除本地chart仓库 ]# helm repo remove kaiyuanshe //更新本地chart仓库 ]# helm repo update
可能可以使用的helm chart网址
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
helm repo add stable https://charts.helm.sh/stable helm repo add aliyunstable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts helm repo add bitnami https://charts.bitnami.com/bitnami helm repo add azureincubator https://mirror.azure.cn/kubernetes/charts-incubator/ helm repo add gitlab https://charts.gitlab.io helm repo add kiwigrid https://kiwigrid.github.io helm repo add azure https://mirror.azure.cn/kubernetes/charts/ helm repo add aliyun https://apphub.aliyuncs.com/ helm repo add saleor https://k.github.io/saleor-helm helm repo add elastic https://helm.elastic.co helm repo add harbor https://helm.goharbor.io helm repo add reactiveops-stable https://charts.reactiveops.com/stable helm repo add buttahtoast https://buttahtoast.github.io/helm-charts/ helm repo add bedag https://bedag.github.io/helm-charts/ helm repo add grafana https://grafana.github.io/helm-charts helm repo add prometheus https://prometheus-community.github.io/helm-charts helm repo add hashicorp https://helm.releases.hashicorp.com helm repo add minio https://helm.min.io helm repo add incubator https://charts.helm.sh/incubator helm repo add codimd https://helm.codimd.dev/ helm repo add ibm https://raw.githubusercontent.com/IBM/charts/master/repo/stable helm repo add ibm-entitled https://raw.githubusercontent.com/IBM/charts/master/repo/entitled helm repo add ibm-community https://raw.githubusercontent.com/IBM/charts/master/repo/community helm repo add eks https://aws.github.io/eks-charts helm repo add argo https://argoproj.github.io/argo-helm helm repo add microsoft https://microsoft.github.io/charts/repo helm repo add privatebin https://privatebin.github.io/helm-chart helm repo add jetstack https://charts.jetstack.io helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo add fabric8 https://fabric8.io/helm helm repo add taito-charts https://taitounited.github.io/taito-charts helm repo add yugabytedb https://charts.yugabyte.com helm repo add strimzi https://strimzi.io/charts/ helm repo add infracloudio https://infracloudio.github.io/charts helm repo add stackstorm https://helm.stackstorm.com/ helm repo add gresearch https://g-research.github.io/charts/ helm repo add kubeless-functions-charts https://kubeless-functions-charts.storage.googleapis.com helm repo add openebs-charts https://openebs.github.io/charts/ helm repo add jenkins https://charts.jenkins.io helm repo add openfaas https://openfaas.github.io/faas-netes helm repo add agones https://agones.dev/chart/stable helm repo add kong https://charts.konghq.com helm repo add appscode https://charts.appscode.com/stable/ helm repo add banzaicloud-stable https://kubernetes-charts.banzaicloud.com/branch/master helm repo add presslabs https://presslabs.github.io/charts helm repo add nvidia https://helm.ngc.nvidia.com/nvidia helm repo add nvidiaoperator https://nvidia.github.io/gpu-operator helm repo add akomljen-charts https://raw.githubusercontent.com/komljen/helm-charts/master/charts/ helm repo add fluent https://fluent.github.io/helm-charts helm repo add fluxcd https://charts.fluxcd.io helm repo add kubesphere https://charts.kubesphere.io/main helm repo add influx https://influx-charts.storage.googleapis.com helm repo add mesosphere-staging https://mesosphere.github.io/charts/staging helm repo add hazelcast https://hazelcast-charts.s3.amazonaws.com/ helm repo add tscharts https://technosophos.github.com/tscharts helm repo add ibm-helm https://raw.githubusercontent.com/IBM/charts/master/repo/ibm-helm helm repo add buildkite https://buildkite.github.io/charts/ helm repo add fairwinds-stable https://charts.fairwinds.com/stable helm repo add influxdata https://helm.influxdata.com/ helm repo add monocular https://helm.github.io/monocular helm repo add oteemocharts https://oteemo.github.io/charts helm repo add honeycomb https://honeycombio.github.io/helm-charts helm repo add tricksterproxy https://helm.tricksterproxy.io/ helm repo add mogaal https://mogaal.github.io/helm-charts/ helm repo add soluto https://charts.soluto.io helm repo add logicmonitor https://logicmonitor.github.io/k8s-helm-charts helm repo add everpeace https://everpeace.github.io/helm-charts helm repo add kubevious https://helm.kubevious.io helm repo add nginx-stable https://helm.nginx.com/stable helm repo add nginx-edge https://helm.nginx.com/edge helm repo add redhat-charts https://redhat-developer.github.io/redhat-helm-charts helm repo add center https://repo.chartcenter.io helm repo add kongz https://charts.kong-z.com helm repo add streamnative https://charts.streamnative.io helm repo add loki https://grafana.github.io/loki/charts helm repo add lensesio https://lensesio.github.io/kafka-helm-charts/ helm repo add cribl https://criblio.github.io/helm-charts/ helm repo add wunderio https://storage.googleapis.com/charts.wdr.io helm repo add mvisonneau https://charts.visonneau.fr/ helm repo add lowess-helm-charts https://lowess.github.io/helm-charts/ helm repo add tkestackstable https://raw.githubusercontent.com/tkestack/charts/master/repo/stable helm repo add gruntwork https://helmcharts.gruntwork.io helm repo add 3loc https://3loc.github.io/charts helm repo add aerokube https://charts.aerokube.com/ helm repo add preferred-ai https://charts.preferred.ai helm repo add spotinst https://spotinst.github.io/spotinst-kubernetes-helm-charts helm repo add dex https://charts.dexidp.io helm repo add verdaccio https://charts.verdaccio.org helm repo add devstack https://apps.devstack.com helm repo add cyberark https://cyberark.github.io/helm-charts helm repo add containersolutions https://containersolutions.github.io/helm-charts/ helm repo add pcktdmp https://pcktdmp.github.io/charts helm repo add trieb.work https://trieb-work.github.io/helm-charts/ helm repo add curity https://curityio.github.io/idsvr-helm/ helm repo add preferred-ai https://charts.preferred.ai helm repo add dlc https://dcos-labs.github.io/charts/ helm repo add supergiant https://supergiant.github.io/charts helm repo add juliohm1978 https://raw.githubusercontent.com/juliohm1978/chart-repo-example/master/index helm repo add whyeasy-helm-charts https://whyeasy.github.io/helm-charts helm repo add utkuozdemir https://utkuozdemir.org/helm-charts helm repo add vsphere-tmm https://vsphere-tmm.github.io/helm-charts helm repo add kilip https://charts.itstoni.com/ helm repo add dumrauf https://dumrauf-helm-charts-repo.s3.amazonaws.com helm repo add shakahl https://shakahl.github.io/helm-charts/ helm repo add aquasecurity https://aquasecurity.github.io/helm-charts/ helm repo add gocd https://gocd.github.io/helm-chart helm repo add emagtechlabs https://emagtechlabs.github.io/helm-charts/ helm repo add lsstsqre https://lsst-sqre.github.io/charts/ helm repo add estahn https://estahn.github.io/charts/ helm repo add fairfaxmedia https://charts.ffx.io/ helm repo add spot https://charts.spot.io helm repo add googlestable https://kubernetes-charts.storage.googleapis.com/ # helm repo add emc-mongoose https://emc-mongoose.github.io/mongoose-helm-charts/ helm repo add equinor-charts https://equinor.github.io/helm-charts/charts/ helm repo add cloudflow-helm-charts https://lightbend.github.io/cloudflow-helm-charts/ helm repo add graphql-gateway https://raw.githubusercontent.com/hansehe/graphql-gateway/master/helm/charts helm repo add drpsychick https://drpsychick.github.io/charts helm repo add csi-hostpath https://speedwing.github.io/csi-hostpath-helm helm repo add kesterriley-repo https://kesterriley.github.io/my-helm-charts/ helm repo add estafette https://helm.estafette.io helm repo add helm-consensys https://artifacts.consensys.net/public/helm/helm/charts/ helm repo add balle-petersen https://charts.balle-petersen.org helm repo add rossumai https://rossumai.github.io/helm-charts helm repo add curiedfcharts https://curie-data-factory.github.io/helm-charts helm repo add maxirus https://maxirus.github.io/helm-charts/ helm repo add rookout https://helm-charts.rookout.com helm repo add naps https://naps.github.io/helm-charts/ helm repo add jacobhelm https://jacobbaek.github.io/helmcharts helm repo add nervos https://k.github.io/nervos-helm helm repo add kubenav https://kubenav.github.io/helm-repository helm repo add graillus https://graillus.github.io/helm-charts helm repo add fikaworks https://fikaworks.github.io/helm-charts helm repo add adamrushuk https://adamrushuk.github.io/charts helm repo add ealenn https://ealenn.github.io/charts helm repo add locust https://raw.githubusercontent.com/hansehe/locust/master/helm/charts helm repo add ot-helm https://ot-container-kit.github.io/helm-charts helm repo add nlnwa https://nlnwa.github.io/charts helm repo add jmriebold https://jmriebold.github.io/charts helm repo add ChronixDB https://raw.githubusercontent.com/ChronixDB/chronix.ops/master/helm/repo/ helm repo add avanier https://avanier.github.io/ax-helm-charts/ helm repo add jx3 https://storage.googleapis.com/jenkinsxio/charts helm repo add sushi https://helm.sushivalidator.com helm repo add haproxytech https://haproxytech.github.io/helm-charts # helm repo add rancher-stable https://releases.rancher.com/server-charts/stable helm repo add rancher-latest https://releases.rancher.com/server-charts/latest helm repo add keel https://charts.keel.sh helm repo add rancher-stable https://releases.rancher.com/server-charts/stable helm repo add helm-cabin https://nick-triller.github.io/helm-cabin/ helm repo add snyk-charts https://snyk.github.io/kubernetes-monitor/ helm repo add portainer https://portainer.github.io/k8s/ helm repo add spark-operator https://googlecloudplatform.github.io/spark-on-k8s-operator helm repo add podinfo https://stefanprodan.github.io/podinfo helm repo add uswitch https://uswitch.github.io/kiam-helm-charts/charts/ helm repo add longhorn https://charts.longhorn.io helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo add kubevela https://charts.kubevela.net/core helm repo add pravega https://charts.pravega.io helm repo add mojo2600 https://mojo2600.github.io/pihole-kubernetes/ helm repo add openwhisk https://openwhisk.apache.org/charts helm repo add kubeinvaders https://lucky-sideburn.github.io/helm-charts/ helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller helm repo add scylla-operator https://storage.googleapis.com/scylla-operator-charts/stable helm repo add artifact-hub https://artifacthub.github.io/hub/chart/ helm repo add mattermost https://helm.mattermost.com helm repo add crossplane-master https://charts.crossplane.io/master/ helm repo add k8s-at-home https://k8s-at-home.com/charts/ helm repo add runix https://helm.runix.net helm repo add falcosecurity https://falcosecurity.github.io/charts helm repo add astronomer https://helm.astronomer.io helm repo add kedacore https://kedacore.github.io/charts helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets helm repo add katafygio https://bpineau.github.io/katafygio helm repo add yunikorn https://apache.github.io/incubator-yunikorn-release helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts helm repo add kubernetic https://harbur.github.io/kubernetic-charts helm repo add kuma https://kumahq.github.io/charts helm repo add stratos https://cloudfoundry.github.io/stratos helm repo add sonatype https://sonatype.github.io/helm3-charts/ helm repo add traefik https://helm.traefik.io/traefik helm repo add mittwald https://helm.mittwald.de helm repo add containerum https://charts.containerum.io helm repo add bokysan https://bokysan.github.io/docker-postfix/ helm repo add zaqar https://lsantos.me/zaqar/helm helm repo add mailu https://mailu.github.io/helm-charts/ helm repo add k8s-ovpn https://raw.githubusercontent.com/suda/k8s-ovpn-chart/master/ helm repo add atlassian-data-center https://atlassian.github.io/data-center-helm-charts helm repo add ygqygq2 https://ygqygq2.github.io/charts/ helm repo add bytectl https://bytectl.github.io/helm-charts helm repo add edu https://iac-source.github.io/helm-charts helm repo add jenkins-x https://jenkins-x-charts.github.io/v2/ helm repo add stakater https://stakater.github.io/stakater-charts helm repo add opencord https://charts.opencord.org/ helm repo add twuni https://helm.twun.io helm repo add truecharts https://truecharts.org/ helm repo add deliveryhero https://charts.deliveryhero.io/ helm repo add vineyard https://vineyard.oss-ap-southeast-1.aliyuncs.com/charts/ helm repo add gmelillo https://helm.melillo.me helm repo add cloudnativeapp https://cloudnativeapp.github.io/charts/curated/ helm repo add k8tz https://k8tz.github.io/k8tz/ helm repo add dirigible https://eclipse.github.io/dirigible helm repo add budibase https://budibase.github.io/budibase/ helm repo add billimek https://billimek.com/billimek-charts/ helm repo add node-red https://schwarzit.github.io/node-red-chart/ helm repo add cocainefarm https://kube.cat/chartrepo/cocainefarm helm repo add genuitec https://genuitec.github.io/CodeTogether/helm/ helm repo add taskmedia https://helm.task.media helm repo add okgolove https://okgolove.github.io/helm-charts/ helm repo add devtron-labs https://raw.githubusercontent.com/devtron-labs/charts/gh-pages helm repo add cert-manager-alidns-webhook https://devmachine-fr.github.io/cert-manager-alidns-webhook helm repo add truecharts2 https://charts.truecharts.org/ helm repo add vsphere-cpi https://kubernetes.github.io/cloud-provider-vsphere helm repo update
3.2、搜索Chart
]# helm search --help Usage: helm search [command] Available Commands: hub 在公网上的chart仓库中搜索chart,默认chart仓库是https://hub.helm.sh repo 在本地chart仓库中搜索chart,不需要连接公网。
示例:
//搜索mongodb ]# helm search repo mongodb ]# helm search hub mongodb
3.3、查看Chart的信息
]# helm show -h Usage: helm show [command] Aliases: show, inspect Available Commands: all 显示chart的所有信息 chart 显示chart的定义(definition) crds 显示chart的CRDs readme 显示chart的README values 显示chart的values
3.4、下载、上传chart
//从仓库下载chart helm pull [chart URL | repo/chartname] [...] [flags] //将chart推送到仓库 helm push [chart] [remote] [flags]
示例:
//下载到mongodb的chart到本地当前目录 ]# helm pull bitnami/mongodb
3.5、运行并管理release
1、运行release
//使用仓库直接运行release ]# helm install mongodb-name1 bitnami/mongodb //使用压缩包运行release ]# helm install mongodb-name2 mongodb-13.1.7.tgz //使用解压后的目录运行release ]# helm install mongodb-name3 mongodb/
2、查看已经运行的release
]# helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION mongodb-name1 default 1 2022-10-09 17:54:13.504601252 +0800 CST deployed mongodb-13.1.7 6.0.2
3、获取release的状态
]# helm status mongodb-name1
4、删除releases
]# helm uninstall mongodb-name1
5、获取releases部署历史
]# helm history mongodb-name2 REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 1 Sun Oct 9 17:54:13 2022 deployed mongodb-13.1.7 6.0.2 Install complete
6、升级和回滚
//升级一个release helm upgrade [RELEASE] [CHART] [flags] //回滚release helm rollback <RELEASE> [REVISION] [flags]
4、部署redis集群
- 部署前要先实现实现StorageClass(动态创建PV)。
- 共享存储查看:https://www.cnblogs.com/maiblogs/p/16392831.html,这里使用“6.2、动态存储卷”。
1、创建名称空间
]# kubectl create namespace redis-namespace
2、部署redis集群
//获取chart仓库 ]# helm repo add bitnami https://charts.bitnami.com/bitnami //获取redis的chart ]# helm search repo redis NAME CHART VERSION APP VERSION DESCRIPTION bitnami/redis 17.3.2 7.0.5 Redis(R) is an open source, advanced key-value ... bitnami/redis-cluster 8.2.4 7.0.5 Redis(R) is an open source, scalable, distribut... ... //部署bitnami/redis-cluster到k8s(storageClass的名字是nfs-client) helm install my-release bitnami/redis-cluster -n redis-namespace --set "persistence.storageClass=nfs-client"
3、查看
//查看server ]# kubectl get svc -n redis-namespace NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-release-redis-cluster ClusterIP 10.20.161.245 <none> 6379/TCP 11m my-release-redis-cluster-headless ClusterIP None <none> 6379/TCP,16379/TCP 11m //查看pods ]# kubectl get pods -n redis-namespace NAME READY STATUS RESTARTS AGE my-release-redis-cluster-0 1/1 Running 1 12m my-release-redis-cluster-1 1/1 Running 1 12m my-release-redis-cluster-2 1/1 Running 1 12m my-release-redis-cluster-3 1/1 Running 1 12m my-release-redis-cluster-4 1/1 Running 1 12m my-release-redis-cluster-5 1/1 Running 1 12m //查看pvc ]# kubectl get pvc -n redis-namespace NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE redis-data-my-release-redis-cluster-0 Bound pvc-7b1b341c-19a3-4d2c-853f-ee82b5dc4622 8Gi RWO nfs-client 12m redis-data-my-release-redis-cluster-1 Bound pvc-c8cd942a-027a-4c7a-aa13-c9dec2a0e081 8Gi RWO nfs-client 12m redis-data-my-release-redis-cluster-2 Bound pvc-9fceae30-2a73-4bcd-8da5-88d40c93a060 8Gi RWO nfs-client 12m redis-data-my-release-redis-cluster-3 Bound pvc-d7ee0fbb-8b87-4a2e-93d5-6fef1f78273e 8Gi RWO nfs-client 12m redis-data-my-release-redis-cluster-4 Bound pvc-7f4d2476-1261-4429-b1c4-77c8a701d97a 8Gi RWO nfs-client 12m redis-data-my-release-redis-cluster-5 Bound pvc-e816270f-0150-4b64-9016-2dca13abedce 8Gi RWO nfs-client 12m //查看pv ]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-7b1b341c-19a3-4d2c-853f-ee82b5dc4622 8Gi RWO Delete Bound redis-namespace/redis-data-my-release-redis-cluster-0 nfs-client 12m pvc-7f4d2476-1261-4429-b1c4-77c8a701d97a 8Gi RWO Delete Bound redis-namespace/redis-data-my-release-redis-cluster-4 nfs-client 12m pvc-9fceae30-2a73-4bcd-8da5-88d40c93a060 8Gi RWO Delete Bound redis-namespace/redis-data-my-release-redis-cluster-2 nfs-client 12m pvc-c8cd942a-027a-4c7a-aa13-c9dec2a0e081 8Gi RWO Delete Bound redis-namespace/redis-data-my-release-redis-cluster-1 nfs-client 12m pvc-d7ee0fbb-8b87-4a2e-93d5-6fef1f78273e 8Gi RWO Delete Bound redis-namespace/redis-data-my-release-redis-cluster-3 nfs-client 12m pvc-e816270f-0150-4b64-9016-2dca13abedce 8Gi RWO Delete Bound redis-namespace/redis-data-my-release-redis-cluster-5 nfs-client 12m
4、登录redis
//启动redis客户端 ]# kubectl run --namespace redis-namespace my-release-redis-cluster-client --rm --tty -i --restart='Never' \ --env REDIS_PASSWORD=$REDIS_PASSWORD \ --image docker.io/bitnami/redis-cluster:7.0.5-debian-11-r0 -- bash //登录redis服务器 /$ redis-cli -c -h my-release-redis-cluster -a euhiu3dXmQ //查看集群节点 6379> cluster nodes a18a5aa0a16f2fd9d389fe83a39e1bcf28611d3d 10.10.2.4:6379@16379 master - 0 1665371671970 1 connected 0-5460 de4021c3c1611382270a90cad0bb43025d24e7b2 10.10.2.6:6379@16379 slave a18a5aa0a16f2fd9d389fe83a39e1bcf28611d3d 0 1665371670000 1 connected b54b8cb69585febff3d46420e6b99a1a2c244232 10.10.1.5:6379@16379 master - 0 1665371672000 2 connected 5461-10922 8de10b6d34d3afcebc39120bcd28a07306140eef 10.10.1.6:6379@16379 slave b54b8cb69585febff3d46420e6b99a1a2c244232 0 1665371671000 2 connected 8b72dfdaac832d01c3669e9086c0272909eb256b 10.10.2.5:6379@16379 myself,master - 0 1665371669000 3 connected 10923-16383 bc55361f4715ed0ecaeab2b455d722e0f871b732 10.10.1.7:6379@16379 slave 8b72dfdaac832d01c3669e9086c0272909eb256b 0 1665371672980 3 connected
5、查看service和statefulset的配置文件
- 查看无头service
]# kubectl edit service my-release-redis-cluster -n redis-namespace apiVersion: v1 kind: Service metadata: annotations: meta.helm.sh/release-name: my-release meta.helm.sh/release-namespace: redis-namespace creationTimestamp: "2022-10-10T02:43:08Z" labels: app.kubernetes.io/instance: my-release app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: redis-cluster helm.sh/chart: redis-cluster-8.2.4 name: my-release-redis-cluster namespace: redis-namespace resourceVersion: "2055" uid: c0f5afe5-a1ba-4d2c-89ec-912f6911e3e9 spec: clusterIP: 10.20.161.245 clusterIPs: - 10.20.161.245 ports: - name: tcp-redis port: 6379 protocol: TCP targetPort: tcp-redis selector: app.kubernetes.io/instance: my-release app.kubernetes.io/name: redis-cluster sessionAffinity: None type: ClusterIP status: loadBalancer: {}
- 查看service
]# kubectl edit service my-release-redis-cluster-headless -n redis-namespace apiVersion: v1 kind: Service metadata: annotations: meta.helm.sh/release-name: my-release meta.helm.sh/release-namespace: redis-namespace creationTimestamp: "2022-10-10T02:43:08Z" labels: app.kubernetes.io/instance: my-release app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: redis-cluster helm.sh/chart: redis-cluster-8.2.4 name: my-release-redis-cluster-headless namespace: redis-namespace resourceVersion: "2053" uid: 64e1dd30-ebc1-4f04-9488-14cb13cd3a2f spec: clusterIP: None clusterIPs: - None ports: - name: tcp-redis port: 6379 protocol: TCP targetPort: tcp-redis - name: tcp-redis-bus port: 16379 protocol: TCP targetPort: tcp-redis-bus publishNotReadyAddresses: true selector: app.kubernetes.io/instance: my-release app.kubernetes.io/name: redis-cluster sessionAffinity: None type: ClusterIP status: loadBalancer: {}
- 查看StatefulSet
apiVersion: apps/v1 kind: StatefulSet metadata: annotations: meta.helm.sh/release-name: my-release meta.helm.sh/release-namespace: redis-namespace creationTimestamp: "2022-10-10T02:43:09Z" generation: 1 labels: app.kubernetes.io/instance: my-release app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: redis-cluster helm.sh/chart: redis-cluster-8.2.4 name: my-release-redis-cluster namespace: redis-namespace resourceVersion: "2582" uid: 38181325-2620-4290-b471-5646374ecb07 spec: podManagementPolicy: Parallel replicas: 6 revisionHistoryLimit: 10 selector: matchLabels: app.kubernetes.io/instance: my-release app.kubernetes.io/name: redis-cluster serviceName: my-release-redis-cluster-headless template: metadata: annotations: checksum/config: edb77e240c0c6f850a0491f3ee75b41734497b84c85758df6333acad7c2c322e checksum/scripts: 34a440ce2ff43b6313a12562e0482364dd1fe3d2fe1ad9bb5c92170d6e55b19f checksum/secret: 0613daff23bf184e662d188826eaf2691eb74e4af7a35604998c942da6d6b879 creationTimestamp: null labels: app.kubernetes.io/instance: my-release app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: redis-cluster helm.sh/chart: redis-cluster-8.2.4 spec: affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - podAffinityTerm: labelSelector: matchLabels: app.kubernetes.io/instance: my-release app.kubernetes.io/name: redis-cluster namespaces: - redis-namespace topologyKey: kubernetes.io/hostname weight: 1 containers: - args: - | # Backwards compatibility change if ! [[ -f /opt/bitnami/redis/etc/redis.conf ]]; then echo COPYING FILE cp /opt/bitnami/redis/etc/redis-default.conf /opt/bitnami/redis/etc/redis.conf fi pod_index=($(echo "$POD_NAME" | tr "-" "\n")) pod_index="${pod_index[-1]}" if [[ "$pod_index" == "0" ]]; then export REDIS_CLUSTER_CREATOR="yes" export REDIS_CLUSTER_REPLICAS="1" fi /opt/bitnami/scripts/redis-cluster/entrypoint.sh /opt/bitnami/scripts/redis-cluster/run.sh command: - /bin/bash - -c env: - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: REDIS_NODES value: 'my-release-redis-cluster-0.my-release-redis-cluster-headless my-release-redis-cluster-1.my-release-redis-cluster-headless my-release-redis-cluster-2.my-release-redis-cluster-headless my-release-redis-cluster-3.my-release-redis-cluster-headless my-release-redis-cluster-4.my-release-redis-cluster-headless my-release-redis-cluster-5.my-release-redis-cluster-headless ' - name: REDISCLI_AUTH valueFrom: secretKeyRef: key: redis-password name: my-release-redis-cluster - name: REDIS_PASSWORD valueFrom: secretKeyRef: key: redis-password name: my-release-redis-cluster - name: REDIS_AOF_ENABLED value: "yes" - name: REDIS_TLS_ENABLED value: "no" - name: REDIS_PORT value: "6379" image: docker.io/bitnami/redis-cluster:7.0.5-debian-11-r0 imagePullPolicy: IfNotPresent livenessProbe: exec: command: - sh - -c - /scripts/ping_liveness_local.sh 5 failureThreshold: 5 initialDelaySeconds: 5 periodSeconds: 5 successThreshold: 1 timeoutSeconds: 6 name: my-release-redis-cluster ports: - containerPort: 6379 name: tcp-redis protocol: TCP - containerPort: 16379 name: tcp-redis-bus protocol: TCP readinessProbe: exec: command: - sh - -c - /scripts/ping_readiness_local.sh 1 failureThreshold: 5 initialDelaySeconds: 5 periodSeconds: 5 successThreshold: 1 timeoutSeconds: 2 resources: {} securityContext: runAsNonRoot: true runAsUser: 1001 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /scripts name: scripts - mountPath: /bitnami/redis/data name: redis-data - mountPath: /opt/bitnami/redis/etc/redis-default.conf name: default-config subPath: redis-default.conf - mountPath: /opt/bitnami/redis/etc/ name: redis-tmp-conf dnsPolicy: ClusterFirst enableServiceLinks: false restartPolicy: Always schedulerName: default-scheduler securityContext: fsGroup: 1001 runAsUser: 1001 serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 volumes: - configMap: defaultMode: 493 name: my-release-redis-cluster-scripts name: scripts - configMap: defaultMode: 420 name: my-release-redis-cluster-default name: default-config - emptyDir: {} name: redis-tmp-conf updateStrategy: rollingUpdate: partition: 0 type: RollingUpdate volumeClaimTemplates: - apiVersion: v1 kind: PersistentVolumeClaim metadata: creationTimestamp: null labels: app.kubernetes.io/instance: my-release app.kubernetes.io/name: redis-cluster name: redis-data spec: accessModes: - ReadWriteOnce resources: requests: storage: 8Gi storageClassName: nfs-client volumeMode: Filesystem status: phase: Pending status: collisionCount: 0 currentReplicas: 6 currentRevision: my-release-redis-cluster-577c47b8dd observedGeneration: 1 readyReplicas: 6 replicas: 6 updateRevision: my-release-redis-cluster-577c47b8dd updatedReplicas: 6
1
# #