Helm程序包管理器
业务的容器化及微服务化过程基本上都是通过将单体大应用分解为多个小的服务并进行容器化编排运行来实现，这种构建逻辑分解了单体应用的复杂性，让每个微服务都能够独立进行部署和扩展，实现了敏捷开发和运维。但是，微服务化拆解巨大的单体应用为巨量的微服务程序，几乎必然导致了应用管理复杂度的增加，例如，在kubernetes系统上，每个应用基本上都有着不止一个资源，而每个应用在不同的环境中存在使用不同的配置参数的可能性等复杂问题。因此，出现了helm。简单的说，Helm的目标是在两分钟内就可以在Kubernetes上安装应用程序。
Helm可以解决的问题：1）运维人员写好资源文件模板；2）交给开发人员填写参数即可
helm官网：https://v3.helm.sh/zh/docs/
https://helm.sh/
helm 官方的chart站点：https://hub.kubeapps.com/
一、Helm基础
Helm是kubernetes的包管理工具，相当于linux环境下的yum/apg-get命令等，可用于实现帮助用户查找、分享及使用Kubernetes应用程序。它的核心打包功能组件称为chart，可以帮助用户创建、安装及升级复杂应用。
Helm将kubernetes的资源（如Deployments、Services或ConfigMap等）打包到一个Charts中，制作并测试完成的各个Charts将保存都Charts仓库进行存储和分发。另外，Helm实现了可配置的发布，它支持应用配置的版本管理，简化了Kubernetes部署应用的版本控制、打包、发布、删除和更新等操作。简单的说，Helm其实就是一个基于kubernetes的程序包（资源包）管理器，它将一个应用的相关资源组织成为Charts，并通过Charts管理程序包，其使用优势可简单总结为以下几个方面：
1）管理复杂应用：Charts能够描述哪怕是最复杂的程序结构，其提供了可重复使用的应用安装定义
2）易于升级：使用就地升级和自定义钩子来解决更新的难题
3）简单分享：Charts易于通过公共或私有服务完成版本化、分享及主机构建
4）回滚：可使用“helm rollback”命令轻松实现快速回滚
1. Helm的核心术语
Helm将kubernetes应用的相关配置组织为Charts，并通过它完成应用的常规管理操作。通常来说，使用Charts管理应用的流程包括从0开始创建Charts、将Charts及其相关的文件打包为归档格式、将Charts存储于仓库中并与之交互、在kubernetes集群中安装或卸载Charts以及管理经Helm安装的应用的版本发行周期。因此，对Helm来说，它具有以下几个关键的概念：
1）Charts：即一个Helm程序包，它包含了运行一个kubernetes应用所需要的镜像、依赖关系和资源定义等，必要时还会包含service的定义；它类似于API 的dpkg文件或yum的rpm文件。
2）Repository：Charts仓库，用于集中存储和分发Charts，类似于Perl的CPAN，或者Python的Pypi
3）Config：应用程序实例化安装运行时使用的配置信息
4）Release：应用程序实例化配置后运行于kubernetes集群中的一个Charts实例；在同一个集群上，一个Charts可以使用不同的Config重复安装多次，每次安装都会创建一个新的Release。
事实上，Charts更像是存储于kubernetes集群之外的程序，它的每次安装是指在集群中使用专用配置运行一个示例，执行过程有点类似于在操作系统上基于程序启动一个进程。
2. Helm架构
1）v2版本的Helm
v2版本的Helm主要有Helm 客户端，Tiller服务器和Charts仓库组成。
Helm客户端是命令行客户端工具，采用go语言编写，基于gRPC协议与Tiller Server交互。它主要完成如下任务：
（1）本地Charts开发；
（2）管理Charts仓库；
（3）与Tiller Server交互：发送Charts以安装、查询Release的相关信息以升级或卸载已有的Release。
Tiller Server是托管运行于kubernetes集群之中的容器化服务应用，它接收来自Helm客户端的请求，并在必要时与kubernetes API Server进行交互。它主要完成以下任务：
（1）监听来自于Helm客户端的请求；
（2）合并Charts和配置以构建一个Release；
（3）向Kubernetes集群安装Charts并对相应的Release进行跟踪；
（4）升级和卸载Charts。
通常，用户于Helm客户端本地遵循其格式编写Charts文件，而后即可部署于kubernetes集群之上运行为一个特定的Release。仅在有分发需求时，才应该将同一应用的Charts文件打包成归档压缩格式提交到特定的Charts仓库。仓库既可以运行为公共托管平台，也可以是用户自建的服务器，仅供特定的组织或个人使用。
2）v3版本的Helm
2019年11月13日，Helm团队发布Helm v3的第一个稳定版本，v2版本到v3版本架构变化如下图所示：
v3版本主要变化如下：(架构变化）
（1）Helm服务端Tiller被删除
（2）Release名称可以在不同命名空间重用
（3）支持将Chart推送至Docker镜像仓库中
（4）使用JSONSchema验证chartvalues
K8s版本支持的各个helm版本，请参考https://helm.sh/zh/docs/topics/version_skew/。
3. 安装Helm Client
Helm的安装方式有两种：预编译的二进制程序和源码编译安装。这里介绍预编译的二进制程序的安装方式。Helm的每个发行版本都提供了主流操作系统的专用版本，主要包括Linux、Mac OS和windows，用户安装前按需下载合用的平台上的相关发行版本即可。Helm项目托管在GitHub上，项目地址为：https://github.com/helm/helm/releases。
安装前首先下载合适版本的压缩包并将其展开。本示例中使用的是3.8.1的版本。
[root@k8s-master1 ~]# mkdir helm [root@k8s-master1 ~]# cd helm/ [root@k8s-master1 helm]# tar -zxvf helm-v3.8.1-linux-amd64.tar.gz linux-amd64/ linux-amd64/helm linux-amd64/LICENSE linux-amd64/README.md
而后，将其二进制程序文件复制或移动到系统PATH环境变量指向的目录中即可。如/usr/local/bin/ 目录（管理员用户才有写入此文件至此目录的权限）：
[root@k8s-master1 helm]# mv linux-amd64/helm /usr/bin/
Helm的各种管理功能均可通过其子命令完成，获取其使用帮助，直接使用“help”子命令即可：
[root@k8s-master1 helm]# helm help The Kubernetes package manager Common actions for Helm: - helm search: search for charts - helm pull: download a chart to your local directory to view - helm install: upload the chart to Kubernetes - helm list: list releases of charts ....
查看helm的版本，使用如下命令：
[root@k8s-master1 helm]# helm version
version.BuildInfo{Version:"v3.8.1", GitCommit:"5cb9af4b1b271d11d7a97a71df3ac337dd94ad37", GitTreeState:"clean", GoVersion:"go1.17.5"}
[root@k8s-master1 helm]#
4. 配置国内存放chart仓库的地址
Charts是Helm的程序包，它们存储于Charts仓库中。Kubernetes官方的Charts仓库保存了一系列精心制作的和维护的Charts，仓库的默认名称为“stable”。安装Charts到kubernetes集群时，Helm首先会到Kubernetes官方的Charts仓库中获取到相关的Charts，而后将其安装并创建为Release。
官方仓库（https://hub.kubeapps.com/charts/incubator）官方chart仓库，国内可能无法访问
微软仓库（http://mirror.azure.cn/kubernetes/charts/）这个仓库推荐，基本上官网有的chart这里都有，国内可能无法访问
阿里云仓库（https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts）
"helm repo" 相关命令可用于管理使用的Charts仓库。“helm repo add”命令能够添加Charts仓库的元数据信息，“helm repo update"命令能够更新使用的默认仓库的元数据信息，“helm repo list”显示配置了哪些charts仓库等，其命令及执行结果如下所示：
1）添加阿里云的chart仓库
[root@k8s-master1 helm]# helm repo add aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts "aliyun" has been added to your repositories
2）添加bitnami的chart仓库
[root@k8s-master1 helm]# helm repo add bitnami https://charts.bitnami.com/bitnami "bitnami" has been added to your repositories
3）更新chart仓库
[root@k8s-master1 helm]# helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "aliyun" chart repository ...Successfully got an update from the "bitnami" chart repository Update Complete. ⎈Happy Helming!⎈
4）查看配置的chart仓库有哪些
[root@k8s-master1 helm]# helm repo list NAME URL aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts bitnami https://charts.bitnami.com/bitnami [root@k8s-master1 helm]#
5）删除chart仓库地址
[root@k8s-master1 helm]# helm repo remove aliyun "aliyun" has been removed from your repositories You have new mail in /var/spool/mail/root [root@k8s-master1 helm]# helm repo list NAME URL bitnami https://charts.bitnami.com/bitnami
6）重新添加阿里云的chart仓库
[root@k8s-master1 helm]# helm repo add aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts "aliyun" has been added to your repositories [root@k8s-master1 helm]# helm repo list NAME URL bitnami https://charts.bitnami.com/bitnami aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts [root@k8s-master1 helm]#
7）更新chart仓库
[root@k8s-master1 helm]# helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "aliyun" chart repository ...Successfully got an update from the "bitnami" chart repository Update Complete. ⎈Happy Helming!⎈
8）从指定chart仓库地址搜索chart
[root@k8s-master1 helm]# helm search repo aliyun NAME CHART VERSION APP VERSION DESCRIPTION aliyun/acs-engine-autoscaler 2.1.3 2.1.1 Scales worker nodes within agent pools aliyun/aerospike 0.1.7 v3.14.1.2 A Helm chart for Aerospike in Kubernetes aliyun/anchore-engine 0.1.3 0.1.6 Anchore container analysis and policy evaluatio... aliyun/artifactory 7.0.3 5.8.4 Universal Repository Manager supporting all maj... aliyun/artifactory-ha 0.1.0 5.8.4 Universal Repository Manager supporting all maj... aliyun/aws-cluster-autoscaler 0.3.2 Scales worker nodes within autoscaling groups. aliyun/bitcoind 0.1.0 0.15.1 Bitcoin is an innovative payment network and a ... aliyun/buildkite 0.2.1 3 Agent for Buildkite aliyun/centrifugo 2.0.0 1.7.3 Centrifugo is a real-time messaging server. aliyun/cert-manager 0.2.2 0.2.3 A Helm chart for cert-manager aliyun/chaoskube 0.6.2 0.6.1 Chaoskube periodically kills random pods in you... aliyun/chronograf 0.4.2 Open-source web application written in Go and R... aliyun/cluster-autoscaler 0.4.2 1.1.0 Scales worker nodes within autoscaling groups. aliyun/cockroachdb 0.6.5 1.1.5 CockroachDB is a scalable, survivable, strongly... aliyun/concourse 1.0.2 3.9.0 Concourse is a simple and scalable CI system. aliyun/consul 1.3.1 1.0.0 Highly available and distributed service discov... aliyun/coredns 0.8.0 1.0.1 CoreDNS is a DNS server that chains plugins and... aliyun/coscale 0.2.0 3.9.1 CoScale Agent aliyun/dask-distributed 2.0.0 Distributed computation in Python aliyun/datadog 0.10.9 DataDog Agent aliyun/docker-registry 1.0.3 2.6.2 A Helm chart for Docker Registry aliyun/dokuwiki 0.2.2 DokuWiki is a standards-compliant, simple to us... aliyun/drupal 0.11.8 8.4.5 One of the most versatile open source content m... aliyun/elastalert 0.1.1 0.1.21 ElastAlert is a simple framework for alerting o... aliyun/elasticsearch-exporter 0.1.2 1.0.2 Elasticsearch stats exporter for Prometheus aliyun/etcd-operator 0.7.0 0.7.0 CoreOS etcd-operator Helm chart for Kubernetes aliyun/external-dns 0.4.9 0.4.8 Configure external DNS servers (AWS Route53, Go... aliyun/factorio 0.3.0 Factorio dedicated server. aliyun/fluent-bit 0.2.11 0.12.11 Fast and Lightweight Log/Data Forwarder for Lin... aliyun/g2 0.3.0 0.5.0 G2 by AppsCode - Gearman in Golang aliyun/gcloud-endpoints 0.1.0 Develop, deploy, protect and monitor your APIs ... aliyun/gcloud-sqlproxy 0.2.3 Google Cloud SQL Proxy aliyun/gcp-night-king 1.0.0 1 A Helm chart for GCP Night King aliyun/ghost 2.1.13 1.21.3 A simple, powerful publishing platform that all... aliyun/gitlab-ce 0.2.1 GitLab Community Edition aliyun/gitlab-ee 0.2.1 GitLab Enterprise Edition aliyun/grafana 0.7.0 The leading tool for querying and visualizing t... aliyun/hadoop 1.0.4 2.7.3 The Apache Hadoop software library is a framewo... aliyun/heapster 0.2.7 Heapster enables Container Cluster Monitoring a... aliyun/influxdb 0.8.2 Scalable datastore for metrics, events, and rea... aliyun/ipfs 0.2.0 v0.4.9 A Helm chart for the Interplanetary File System aliyun/jasperreports 0.2.5 6.4.2 The JasperReports server can be used as a stand... aliyun/jenkins 0.13.5 2.73 Open source continuous integration server. It s... aliyun/joomla 0.5.7 3.8.5 PHP content management system (CMS) for publish... aliyun/kanister-operator 0.2.0 0.2.0 Kanister-operator Helm chart for Kubernetes aliyun/kapacitor 0.5.0 InfluxDB's native data processing engine. It ca... aliyun/keel 0.2.1 0.4.2 Open source, tool for automating Kubernetes dep... aliyun/kibana 0.2.2 6.0.0 Kibana is an open source data visualization plu... aliyun/kong 0.1.2 0.12.1 Kong is open-source API Gateway and Microservic... aliyun/kube-lego 0.4.0 DEPRECATED Automatically requests certificates ... aliyun/kube-ops-view 0.4.1 Kubernetes Operational View - read-only system ... aliyun/kube-state-metrics 0.5.3 1.1.0 Install kube-state-metrics to generate and expo... aliyun/kube2iam 0.8.0 Provide IAM credentials to pods based on annota... aliyun/kubed 0.3.0 0.4.0 Kubed by AppsCode - Kubernetes daemon aliyun/kubernetes-dashboard 0.6.0 1.8.3 General-purpose web UI for Kubernetes clusters aliyun/lamp 0.1.4 Modular and transparent LAMP stack chart suppor... aliyun/linkerd 0.4.0 Service mesh for cloud native apps aliyun/locust 0.1.2 A modern load testing framework aliyun/luigi 2.7.2 Luigi is a Python module that helps you build c... aliyun/magento 0.6.3 2.2.3 A feature-rich flexible e-commerce solution. It... aliyun/mailhog 2.2.0 1.0.0 An e-mail testing tool for developers aliyun/mariadb 2.1.6 10.1.31 Fast, reliable, scalable, and easy to use open-... aliyun/mcrouter 0.1.0 0.36.0 Mcrouter is a memcached protocol router for sca... aliyun/mediawiki 0.6.3 1.30.0 Extremely powerful, scalable software and a fea... aliyun/memcached 2.0.1 Free & open source, high-performance, distribut... aliyun/metabase 0.3.2 v0.27.2 The easy, open source way for everyone in your ... aliyun/minecraft 0.2.0 Minecraft server aliyun/minio 0.5.5 Distributed object storage server built for clo... aliyun/mongodb 0.4.27 3.7.1 NoSQL document-oriented database that stores JS... aliyun/mongodb-replicaset 2.3.1 3.6 NoSQL document-oriented database that stores JS... aliyun/moodle 0.4.5 3.4.1 Moodle is a learning platform designed to provi... aliyun/msoms 0.1.2 1.0.0-30 A chart for deploying omsagent as a daemonset K... aliyun/mssql-linux 0.1.7 SQL Server 2017 Linux Helm Chart aliyun/mysql 0.3.5 Fast, reliable, scalable, and easy to use open-... aliyun/namerd 0.2.0 Service that manages routing for multiple linke... aliyun/neo4j 0.5.0 3.2.3 Neo4j is the world's leading graph database aliyun/newrelic-infrastructure 0.0.4 0.0.12 A Helm chart to deploy the New Relic Infrastruc... aliyun/nginx-ingress 0.9.5 0.10.2 An nginx Ingress controller that uses ConfigMap... aliyun/nginx-lego 0.3.1 Chart for nginx-ingress-controller and kube-lego aliyun/odoo 0.7.3 11.0.20180115 A suite of web based open source business apps. aliyun/opencart 0.6.2 3.0.2 A free and open source e-commerce platform for ... aliyun/openvpn 2.0.2 A Helm chart to install an openvpn server insid... aliyun/orangehrm 0.5.2 4.0.0 OrangeHRM is a free HR management system that o... aliyun/osclass 0.5.2 3.7.4 Osclass is a php script that allows you to quic... aliyun/owncloud 0.5.7 10.0.7 A file sharing server that puts the control and... aliyun/pachyderm 0.1.5 1.6.7 Pachyderm is a large-scale container-based work... aliyun/parse 0.3.6 2.7.2 Parse is a platform that enables users to add a... aliyun/percona 0.3.0 free, fully compatible, enhanced, open source d... aliyun/percona-xtradb-cluster 0.0.2 5.7.19 free, fully compatible, enhanced, open source d... aliyun/phabricator 0.5.15 2018.8.0 Collection of open source web applications that... aliyun/phpbb 0.6.4 3.2.2 Community forum that supports the notion of use... aliyun/postgresql 0.9.1 Object-relational database management system (O... aliyun/prestashop 0.5.3 1.7.2 A popular open source ecommerce solution. Profe... aliyun/prometheus 5.4.0 Prometheus is a monitoring system and time seri... aliyun/prometheus-to-sd 0.1.0 0.2.2 Scrape metrics stored in prometheus format and ... aliyun/quassel 0.2.2 0.12.4 Quassel IRC is a modern, cross-platform, distri... aliyun/rabbitmq 0.6.21 3.7.3 Open source message broker software that implem... aliyun/rabbitmq-ha 1.0.0 3.7.3 Highly available RabbitMQ cluster, the open sou... aliyun/redis 1.1.15 4.0.8 Open source, advanced key-value store. It is of... aliyun/redis-ha 2.0.1 Highly available Redis cluster with multiple se... aliyun/redmine 2.0.4 3.4.4 A flexible project management web application. aliyun/rethinkdb 0.1.1 The open-source database for the realtime web aliyun/risk-advisor 2.0.0 Risk Advisor add-on module for Kubernetes aliyun/rocketchat 0.1.2 Prepare to take off with the ultimate chat plat... aliyun/sapho 0.2.1 A micro application development and integration... aliyun/searchlight 0.3.0 5.0.0 Searchlight by AppsCode - Alerts for Kubernetes aliyun/selenium 0.2.6 3.9.1 Chart for selenium grid aliyun/sematext-docker-agent 0.1.2 Sematext Docker Agent aliyun/sensu 0.2.0 Sensu monitoring framework backed by the Redis ... aliyun/sentry 0.1.9 8.17 Sentry is a cross-platform crash reporting and ... aliyun/sonarqube 0.3.6 6.5 Sonarqube is an open sourced code quality scann... aliyun/sonatype-nexus 0.1.6 3.5.1 Sonatype Nexus is an open source repository man... aliyun/spark 0.1.10 Fast and general-purpose cluster computing system. aliyun/spartakus 1.1.3 Collect information about Kubernetes clusters t... aliyun/spinnaker 0.4.0 1.6.0 Open source, multi-cloud continuous delivery pl... aliyun/spotify-docker-gc 0.1.2 A simple Docker container and image garbage col... aliyun/stash 0.4.0 0.6.2 Stash by AppsCode - Backup your Kubernetes Volumes aliyun/sugarcrm 0.2.4 6.5.26 SugarCRM enables businesses to create extraordi... aliyun/suitecrm 0.3.7 7.9.12 SuiteCRM is a completely open source enterprise... aliyun/sumokube 0.1.2 Sumologic Log Collector aliyun/sumologic-fluentd 0.2.1 Sumologic Log Collector aliyun/swift 0.5.0 0.7.2 swift by AppsCode - Ajax friendly Helm Tiller P... aliyun/sysdig 0.4.0 Sysdig Monitor and Secure agent aliyun/telegraf 0.3.2 Telegraf is an agent written in Go for collecti... aliyun/testlink 0.4.18 1.9.16 Web-based test management system that facilitat... aliyun/traefik 1.24.1 1.5.3 A Traefik based Kubernetes ingress controller w... aliyun/uchiwa 0.2.3 Dashboard for the Sensu monitoring framework aliyun/verdaccio 0.2.0 2.7.3 A lightweight private npm proxy registry (sinop... aliyun/voyager 3.1.0 6.0.0-rc.0 Voyager by AppsCode - Secure Ingress Controller... aliyun/weave-cloud 0.1.2 Weave Cloud is a add-on to Kubernetes which pro... aliyun/weave-scope 0.9.2 1.6.5 A Helm chart for the Weave Scope cluster visual... aliyun/wordpress 0.8.8 4.9.4 Web publishing platform for building blogs and ... aliyun/zeppelin 1.0.0 0.7.2 Web-based notebook that enables data-driven, in... aliyun/zetcd 0.1.6 0.0.3 CoreOS zetcd Helm chart for Kubernetes
二、 helm基本用法
1. 搜索Chart
#查看阿里云chart仓库中的memcached [root@k8s-master1 helm]# helm search repo aliyun |grep memcached aliyun/mcrouter 0.1.0 0.36.0 Mcrouter is a memcached protocol router for sca... aliyun/memcached 2.0.1 Free & open source, high-performance, distribut... You have new mail in /var/spool/mail/root
2. 查看chart信息
[root@k8s-master1 helm]# helm show chart aliyun/memcached apiVersion: v1 description: Free & open source, high-performance, distributed memory object caching system. home: http://memcached.org/ icon: https://upload.wikimedia.org/wikipedia/en/thumb/2/27/Memcached.svg/1024px-Memcached.svg.png keywords: - memcached - cache maintainers: - email: gtaylor@gc-taylor.com name: Greg Taylor name: memcached sources: - https://github.com/docker-library/memcached version: 2.0.1 You have new mail in /var/spool/mail/root [root@k8s-master1 helm]#
3. 下载chart包到本地
[root@k8s-master1 helm]# helm pull aliyun/memcached [root@k8s-master1 helm]# tar zxvf memcached-2.0.1.tgz memcached/Chart.yaml tar: memcached/Chart.yaml: implausibly old time stamp 1970-01-01 08:00:00 memcached/values.yaml tar: memcached/values.yaml: implausibly old time stamp 1970-01-01 08:00:00 memcached/templates/NOTES.txt tar: memcached/templates/NOTES.txt: implausibly old time stamp 1970-01-01 08:00:00 memcached/templates/_helpers.tpl tar: memcached/templates/_helpers.tpl: implausibly old time stamp 1970-01-01 08:00:00 memcached/templates/pdb.yaml tar: memcached/templates/pdb.yaml: implausibly old time stamp 1970-01-01 08:00:00 memcached/templates/statefulset.yaml tar: memcached/templates/statefulset.yaml: implausibly old time stamp 1970-01-01 08:00:00 memcached/templates/svc.yaml tar: memcached/templates/svc.yaml: implausibly old time stamp 1970-01-01 08:00:00 memcached/.helmignore tar: memcached/.helmignore: implausibly old time stamp 1970-01-01 08:00:00 memcached/README.md tar: memcached/README.md: implausibly old time stamp 1970-01-01 08:00:00 [root@k8s-master1 helm]# cd memcached memcached/ memcached-2.0.1.tgz [root@k8s-master1 helm]# cd memcached [root@k8s-master1 memcached]# ll total 12 -rwxr-xr-x 1 root root 415 Jan 1 1970 Chart.yaml -rwxr-xr-x 1 root root 2804 Jan 1 1970 README.md drwxr-xr-x 2 root root 99 Dec 10 14:42 templates -rwxr-xr-x 1 root root 882 Jan 1 1970 values.yaml
其中，Chart.yaml: chart的基本信息，包括版本名字之类
templates: 存放k8s的部署资源模板，通过渲染变量得到部署文件
values.yaml：存放全局变量，templates下的文件可以调用
[root@k8s-master1 memcached]# cd templates/ You have new mail in /var/spool/mail/root [root@k8s-master1 templates]# ll total 20 -rwxr-xr-x 1 root root 536 Jan 1 1970 _helpers.tpl -rwxr-xr-x 1 root root 570 Jan 1 1970 NOTES.txt -rwxr-xr-x 1 root root 373 Jan 1 1970 pdb.yaml -rwxr-xr-x 1 root root 2305 Jan 1 1970 statefulset.yaml -rwxr-xr-x 1 root root 420 Jan 1 1970 svc.yaml
其中，_helpers.tpl 存放能够复用的模板
NOTES.txt 为用户提供一个关于chart部署后使用说明的文件
4. 部署Charts
helm部署memcached服务
1）修改statefulset.yaml文件
(1)apiVersion后面的value值变成apps/v1
(2)spec下添加selector字段:
selector:
matchLabels:
app: {{ template "memcached.fullname" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
(3)删除affinity亲和性配置
2）安装memcached
[root@k8s-master1 memcached]# helm install memcached ./
NAME: memcached
LAST DEPLOYED: Sat Dec 10 14:59:40 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Memcached can be accessed via port 11211 on the following DNS name from within your cluster:
memcached-memcached.default.svc.cluster.local
If you'd like to test your instance, forward the port locally:
export POD_NAME=$(kubectl get pods --namespace default -l "app=memcached-memcached" -o jsonpath="{.items[0].metadata.name}")
kubectl port-forward $POD_NAME 11211
In another tab, attempt to set a key:
$ echo -e 'set mykey 0 60 5\r\nhello\r' | nc localhost 11211
You should see:
STORED
3）验证memcache是否部署成功
[root@k8s-master1 memcached]# kubectl get pods -o wide |grep memcached memcached-memcached-0 1/1 Running 0 10m 10.244.36.70 k8s-node1 <none> <none> memcached-memcached-1 1/1 Running 0 10m 10.244.36.69 k8s-node1 <none> <none> memcached-memcached-2 1/1 Running 0 10m 10.244.36.71 k8s-node1 <none> <none> You have new mail in /var/spool/mail/root [root@k8s-master1 memcached]# kubectl get svc -o wide |grep memcached memcached-memcached ClusterIP None <none> 11211/TCP 13m app=memcached-memcached You have new mail in /var/spool/mail/root [root@k8s-master1 memcached]# kubectl get sts -o wide |grep memcached memcached-memcached 3/3 14m memcached-memcached memcached:1.4.36-alpine
4）测试memecached服务是否正常
根据安装完成后的NOTES提示，执行以下命令，进行测试访问：
#查看是否有nc命令，若无安装此命令
[root@k8s-master1 memcached]# which nc /usr/bin/which: no nc in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin) You have new mail in /var/spool/mail/root [root@k8s-master1 memcached]# yum install nc -y
#测试memcached
[root@k8s-master1 memcached]# export POD_NAME=$(kubectl get pods --namespace default -l "app=memcached-memcached" -o jsonpath="{.items[0].metadata.name}")
You have new mail in /var/spool/mail/root
[root@k8s-master1 memcached]# kubectl port-forward $POD_NAME 11211
Forwarding from 127.0.0.1:11211 -> 11211
Forwarding from [::1]:11211 -> 11211
Handling connection for 11211
#另外一个终端，执行如下命令：
[root@k8s-master1 helm]# echo -e 'set mykey 0 60 5\r\nhello\r' | nc localhost 11211
STORED
You have new mail in /var/spool/mail/root
[root@k8s-master1 helm]#
5. release 相关操作
1）查看release发布状态
[root@k8s-master1 memcached]# helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION memcached default 1 2022-12-10 14:59:40.259697603 +0800 CST deployed memcached-2.0.1 [root@k8s-master1 memcached]#
2）删除release
[root@k8s-master1 memcached]# helm delete memcached release "memcached" uninstalled
#查看资源memcached资源是否删除
[root@k8s-master1 memcached]# kubectl get pods |grep memcache You have new mail in /var/spool/mail/root
从上述结果可以看到：删除release会把release下对应的资源也删除。
三、Helm Charts自定义模板
Charts是Helm使用的kubernetes程序包打包格式，一个charts就是一个描述一组kubernetes资源的文件的集合。从物理的角度来描述，Charts是一个遵循特定规范的目录结构，它能够打包成为一个可用于部署的版本化归档文件。
1. Charts文件组织结构
一个Charts就是按特定格式组织的目录结构，目录名即为Charts名，目录名称本身不包含版本信息。安装好helm之后我们可以开始自定义chart，那么需要先创建出一个模板如下：
[root@k8s-master1 helm]# helm create myapp Creating myapp You have new mail in /var/spool/mail/root [root@k8s-master1 helm]# cd myapp/ [root@k8s-master1 myapp]# ll total 8 drwxr-xr-x 2 root root 6 Dec 10 20:31 charts -rw-r--r-- 1 root root 1141 Dec 10 20:31 Chart.yaml drwxr-xr-x 3 root root 162 Dec 10 20:31 templates -rw-r--r-- 1 root root 1872 Dec 10 20:31 values.yaml [root@k8s-master1 myapp]# tree ./ ./ ├── charts ├── Chart.yaml ├── templates │ ├── deployment.yaml │ ├── _helpers.tpl │ ├── hpa.yaml │ ├── ingress.yaml │ ├── NOTES.txt │ ├── serviceaccount.yaml │ ├── service.yaml │ └── tests │ └── test-connection.yaml └── values.yaml 3 directories, 10 files
目录中除了charts/和templates/ 是目录之外，其他的都是文件。它们的基本功用如下：
charts/: #目录，用于存放所依赖的子chart
Chart.yaml: #当前Charts的描述信息，包括名字、描述信息、版本等。yaml格式的文件
values.yaml：当前Charts用到的默认配置值，这些值会在安装时应用到 GO 模板生成部署文件
templates/：#模板目录，存放当前Charts用到的模板文件，可应用于Charts生成有效的kubernetes的资源清单文件
templates/deployment.yaml #deployment资源的go模板文件
templates/NOTES.txt：#chart 的 “帮助文本”。这会在用户运行 helm install 时显示给用户
templates/_helpers.tpl：#放置模板助手的地方，可以在整个 chart 中重复使用
templates/hpa.yaml #水平pod自动扩缩容go模板文件
templates/ingress.yaml #七层代理go模板文件
templates/service.yaml #service的go模板文件
2. Chart.yaml文件组织格式
Chart.yaml文件用于提供Charts相关的各种元数据，如名称、版本、关键词、维护者信息、使用的模板引擎等，它是一个Charts必备的核心文件，主要包括以下字段：
[root@k8s-master1 myapp]# cat Chart.yaml apiVersion: v2 #用于描述对应chart使用的api版本，默认是v2版本 name: myapp #用于描述对应chart的名称，必选字段 description: A Helm chart for Kubernetes #用于描述对应chart的说明简介，可选字段 # A chart can be either an 'application' or a 'library' chart. # # Application charts are a collection of templates that can be packaged into versioned archives # to be deployed. # # Library charts provide useful utilities or functions for the chart developer. They're included as # a dependency of application charts to inject those utilities and functions into the rendering # pipeline. Library charts do not define any templates and therefore cannot be deployed. type: application #用户描述对应chart是应用程序还是库文件，应用程序类型的chart，它可以运行为一个release，但库类型的chart不能运行为release，它只能作为依赖被application类型的chart所使用 # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) version: 0.1.0 #用于描述对应chart版本 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to # follow Semantic Versioning. They should reflect the version the application is using. # It is recommended to use it with quotes. appVersion: "1.16.0" #用于描述对应chart内部程序的版本信息
3.go模板文件渲染
[root@k8s-master1 myapp]# cat templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "myapp.fullname" . }} #表示取myapp的全名
labels:
{{- include "myapp.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "myapp.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "myapp.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "myapp.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
该部署清单模板文件，主要用go模板语言来写的，其中{{ include "myapp.fullname" . }}就表示取myapp的全名；{{ .Values.image.repository }}这段代码表示读取当前目录下的values文件中的image.repository字段的值；{{ .Values.image.tag | default .Chart.AppVersion }}表示对于values文件中image.tag的值或者读取default.chart文件中的AppVersion字段的值；简单讲go模板就是应用对应go模板语法来定义关属性的的值；一般都是从values.yaml文件中加载对应字段的值作为模板文件相关属性的值。
nindent 4：表示首行缩进4个字母
TRUNC（NUMBER）表示截断数字
4. values.yaml文件编写
[root@k8s-master1 myapp]# cat values.yaml
# Default values for myapp.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
replicaCount: 1
image:
repository: nginx
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
type: ClusterIP
port: 80
ingress:
enabled: false
className: ""
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: chart-example.local
paths:
- path: /
pathType: ImplementationSpecific
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
比如要引用values.yaml文件中的image字段下的tag字段的值，可以在模板文件中写成{{ .Values.image.tag }}；如果在命令行使用--set选项来应用可以写成 image.tag；修改对应的值可以直接编辑对应values.yaml文件中对应字段的值，也可以直接使用--set 指定对应字段的对应值即可；默认情况在命令行使用--set选项给出的值，都会直接被替换，没有给定的值，默认还是使用values.yaml文件中给定的默认值。
5. 部署release
[root@k8s-master1 myapp]# helm install myapp .
NAME: myapp
LAST DEPLOYED: Sat Dec 10 20:58:55 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=myapp" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
[root@k8s-master1 myapp]# kubectl get pods -o wide |grep myapp
myapp-777f4ccb8c-qxw6v 1/1 Running 0 37s 10.244.36.73 k8s-node1 <none> <none>
6. 测试部署的release
通过上述NOTES中的命令提示运行相关的命令获取访问端点后即可访问相应的服务：
[root@k8s-master1 myapp]# export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=myapp,app.kubernetes.io/instance=myapp" -o jsonpath="{.items[0].metadata.name}")
You have new mail in /var/spool/mail/root
[root@k8s-master1 myapp]# export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
[root@k8s-master1 myapp]# echo "Visit http://127.0.0.1:8080 to use your application"
Visit http://127.0.0.1:8080 to use your application
[root@k8s-master1 myapp]# kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80
#另外一个终端执行以下命令
[root@k8s-master1 ~]# curl http://127.0.0.1:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
四、Helm常用命令
1. 检查values语法格式
通常使用“helm lint”命令确认Charts是否遵循最佳实践且模板格式良好：
[root@k8s-master1 myapp]# helm lint /root/helm/myapp ==> Linting /root/helm/myapp [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed
根据执行的结果可看到语法正确。多数情况下，“helm lint”命令报告的错误信息，根据其错误提示中的行号信息即能定位出错误所在。确保一切问题都得以解决之后，即可通过“helm install”命令调试运行以查看由Charts定义的容器化应用是否能够正确部署。
[root@k8s-master1 myapp]# helm install myapp --dry-run . --set service.type=NodePort
NAME: myapp
LAST DEPLOYED: Sat Dec 10 21:29:07 2022
NAMESPACE: default
STATUS: pending-install
REVISION: 1
HOOKS:
---
# Source: myapp/templates/tests/test-connection.yaml
apiVersion: v1
kind: Pod
metadata:
name: "myapp-test-connection"
labels:
helm.sh/chart: myapp-0.1.0
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.16.0"
app.kubernetes.io/managed-by: Helm
annotations:
"helm.sh/hook": test
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['myapp:80']
restartPolicy: Never
MANIFEST:
---
# Source: myapp/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: myapp
labels:
helm.sh/chart: myapp-0.1.0
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.16.0"
app.kubernetes.io/managed-by: Helm
---
# Source: myapp/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp
labels:
helm.sh/chart: myapp-0.1.0
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.16.0"
app.kubernetes.io/managed-by: Helm
spec:
type: NodePort
ports:
- port: 80
targetPort: http
protocol: TCP
name: http
selector:
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp
---
# Source: myapp/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
labels:
helm.sh/chart: myapp-0.1.0
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp
app.kubernetes.io/version: "1.16.0"
app.kubernetes.io/managed-by: Helm
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp
template:
metadata:
labels:
app.kubernetes.io/name: myapp
app.kubernetes.io/instance: myapp
spec:
serviceAccountName: myapp
securityContext:
{}
containers:
- name: myapp
securityContext:
{}
image: "nginx:1.16.0"
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
resources:
{}
NOTES:
1. Get the application URL by running these commands:
export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services myapp)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
确认上述命令输出信息无误后，移除命令中的“--dry-run”选项后再次运行命令即可完成应用部署。
2. upgrade升级release
[root@k8s-master1 myapp]# kubectl get svc |grep myapp
myapp ClusterIP 10.111.220.29 <none> 80/TCP 12m
[root@k8s-master1 myapp]# helm upgrade --set service.type="NodePort" myapp .
Release "myapp" has been upgraded. Happy Helming!
NAME: myapp
LAST DEPLOYED: Sat Dec 10 21:12:11 2022
NAMESPACE: default
STATUS: deployed
REVISION: 2
NOTES:
1. Get the application URL by running these commands:
export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services myapp)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
[root@k8s-master1 myapp]# kubectl get svc |grep myapp
myapp NodePort 10.111.220.29 <none> 80:31620/TCP 13m
通过上面可以看到Service的type类型已经变成了NodePort，通过浏览器访问结果如下：
3. 回滚release
1）查看历史版本：
[root@k8s-master1 myapp]# helm history myapp REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 1 Sat Dec 10 20:58:55 2022 superseded myapp-0.1.0 1.16.0 Install complete 2 Sat Dec 10 21:12:11 2022 deployed myapp-0.1.0 1.16.0 Upgrade complete
2）回滚myapp到版本1
[root@k8s-master1 myapp]# helm rollback myapp 1 Rollback was a success! Happy Helming! [root@k8s-master1 myapp]# kubectl get svc -o wide |grep myapp myapp ClusterIP 10.111.220.29 <none> 80/TCP 18m app.kubernetes.io/instance=myapp,app.kubernetes.io/name=myapp
可以看到service已经完成回滚了
4. 打包Chart
[root@k8s-master1 myapp]# helm package /root/helm/myapp Successfully packaged chart and saved it to: /root/helm/myapp/myapp-0.1.0.tgz You have new mail in /var/spool/mail/root [root@k8s-master1 myapp]# ls -lrt myapp-0.1.0.tgz -rw-r--r-- 1 root root 3754 Dec 10 21:19 myapp-0.1.0.tgz
5. helm相关命令说明
