第十一章 Helm-kubernetes的包管理器（中）
11.5 chart详解
chart由一系列文件组成，这些文件描述了K8s部署应用时需要的资源，比如Servcie、Deployment、PersistentVolmeClaim、Secret、ConfigMap等。
chart可以很复杂，部署整个应用，比如包含HTTP servers、Database、消息中间件、Cache等。
chart将这些文件放置在预定义的目录结构中，通常被打包成tar包，而且标注上版本信息，便于Helm部署。
11.5.1 chart目录结构
一旦安装了某个chart，就可以在 ~/.helm/cache/archive中找到chart的tar包。
kubeusr@GalaxyKubernetesMaster:~$ ls ~/.helm/cache/archive mysql-0.10.1.tgz redis-3.7.6.tgz
解压mysql-0.10.1.tgz
tar zxvf mysql-0.10.1.tgz -C /home
（1）Chart.yaml
描述chart的概要信息。
appVersion: 5.7.14 description: Fast, reliable, scalable, and easy to use open-source relational database system. engine: gotpl home: https://www.mysql.com/ icon: https://www.mysql.com/common/logos/logo-mysql-170x115.png keywords: - mysql - database - sql maintainers: - email: o.with@sportradar.com name: olemarkus - email: viglesias@google.com name: viglesiasce name: mysql # 必填的 sources: - https://github.com/kubernetes/charts - https://github.com/docker-library/mysql version: 0.10.1 # 必填的
(2) README.md
# MySQL [MySQL](https://MySQL.org) is one of the most popular database servers in the world. Notable users include Wikipedia, Facebook and Google. ## Introduction This chart bootstraps a single node MySQL deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. ## Prerequisites - Kubernetes 1.6+ with Beta APIs enabled - PV provisioner support in the underlying infrastructure ## Installing the Chart To install the chart with the release name `my-release`: ```bash $ helm install --name my-release stable/mysql ``` The command deploys MySQL on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. By default a random password will be generated for the root user. If you'd like to set your own password change the mysqlRootPassword in the values.yaml. You can retrieve your root password by running the following command. Make sure to replace [YOUR_RELEASE_NAME]: printf $(printf '\%o' `kubectl get secret [YOUR_RELEASE_NAME]-mysql -o jsonpath="{.data.mysql-root-password[*]}"`) > **Tip**: List all releases using `helm list` ## Uninstalling the Chart To uninstall/delete the `my-release` deployment: ```bash $ helm delete my-release ``` The command removes all the Kubernetes components associated with the chart and deletes the release. ## Configuration The following table lists the configurable parameters of the MySQL chart and their default values. | Parameter | Description | Default | | -------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------------------------------------------- | | `image` | `mysql` image repository. | `mysql` | | `imageTag` | `mysql` image tag. | `5.7.14` | | `imagePullPolicy` | Image pull policy | `IfNotPresent` | | `existingSecret` | Use Existing secret for Password details | `nil` | | `extraVolumes` | Additional volumes as a string to be passed to the `tpl` function | | | `extraVolumeMounts` | Additional volumeMounts as a string to be passed to the `tpl` function | | | `extraInitContainers` | Additional init containers as a string to be passed to the `tpl` function | | | `mysqlRootPassword` | Password for the `root` user. Ignored if existing secret is provided | Random 10 characters | | `mysqlUser` | Username of new user to create. | `nil` | | `mysqlPassword` | Password for the new user. Ignored if existing secret is provided | Random 10 characters | | `mysqlDatabase` | Name for new database to create. | `nil` | | `livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | 30 | | `livenessProbe.periodSeconds` | How often to perform the probe | 10 | | `livenessProbe.timeoutSeconds` | When the probe times out | 5 | | `livenessProbe.successThreshold` | Minimum consecutive successes for the probe to be considered successful after having failed. | 1 | | `livenessProbe.failureThreshold` | Minimum consecutive failures for the probe to be considered failed after having succeeded. | 3 | | `readinessProbe.initialDelaySeconds` | Delay before readiness probe is initiated | 5 | | `readinessProbe.periodSeconds` | How often to perform the probe | 10 | | `readinessProbe.timeoutSeconds` | When the probe times out | 1 | | `readinessProbe.successThreshold` | Minimum consecutive successes for the probe to be considered successful after having failed. | 1 | | `readinessProbe.failureThreshold` | Minimum consecutive failures for the probe to be considered failed after having succeeded. | 3 | | `persistence.enabled` | Create a volume to store data | true | | `persistence.size` | Size of persistent volume claim | 8Gi RW | | `persistence.storageClass` | Type of persistent volume claim | nil (uses alpha storage class annotation) | | `persistence.accessMode` | ReadWriteOnce or ReadOnly | ReadWriteOnce | | `persistence.existingClaim` | Name of existing persistent volume | `nil` | | `persistence.subPath` | Subdirectory of the volume to mount | `nil` | | `persistence.annotations` | Persistent Volume annotations | {} | | `nodeSelector` | Node labels for pod assignment | {} | | `metrics.enabled` | Start a side-car prometheus exporter | `false` | | `metrics.image` | Exporter image | `prom/mysqld-exporter` | | `metrics.imageTag` | Exporter image | `v0.10.0` | | `metrics.imagePullPolicy` | Exporter image pull policy | `IfNotPresent` | | `metrics.resources` | Exporter resource requests/limit | `nil` | | `metrics.livenessProbe.initialDelaySeconds` | Delay before metrics liveness probe is initiated | 15 | | `metrics.livenessProbe.timeoutSeconds` | When the probe times out | 5 | | `metrics.readinessProbe.initialDelaySeconds` | Delay before metrics readiness probe is initiated | 5 | | `metrics.readinessProbe.timeoutSeconds` | When the probe times out | 1 | | `resources` | CPU/Memory resource requests/limits | Memory: `256Mi`, CPU: `100m` | | `configurationFiles` | List of mysql configuration files | `nil` | | `ssl.enabled` | Setup and use SSL for MySQL connections | `false` | | `ssl.secret` | Name of the secret containing the SSL certificates | mysql-ssl-certs | | `ssl.certificates[0].name` | Name of the secret containing the SSL certificates | `nil` | | `ssl.certificates[0].ca` | CA certificate | `nil` | | `ssl.certificates[0].cert` | Server certificate (public key) | `nil` | | `ssl.certificates[0].key` | Server key (private key) | `nil` | | `imagePullSecrets` | Name of Secret resource containing private registry credentials | `nil` | | `initializationFiles` | List of SQL files which are run after the container started | `nil` | | `timezone` | Container and mysqld timezone (TZ env) | `nil` (UTC depending on image) | | `podAnnotations` | Map of annotations to add to the pods | `{}` | Some of the parameters above map to the env variables defined in the [MySQL DockerHub image](https://hub.docker.com/_/mysql/). Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example, ```bash $ helm install --name my-release \ --set mysqlRootPassword=secretpassword,mysqlUser=my-user,mysqlPassword=my-password,mysqlDatabase=my-database \ stable/mysql ``` The above command sets the MySQL `root` account password to `secretpassword`. Additionally it creates a standard database user named `my-user`, with the password `my-password`, who has access to a database named `my-database`. Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```bash $ helm install --name my-release -f values.yaml stable/mysql ``` > **Tip**: You can use the default [values.yaml](values.yaml) ## Persistence The [MySQL](https://hub.docker.com/_/mysql/) image stores the MySQL data and configurations at the `/var/lib/mysql` path of the container. By default a PersistentVolumeClaim is created and mounted into that directory. In order to disable this functionality you can change the values.yaml to disable persistence and use an emptyDir instead. > *"An emptyDir volume is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node. When a Pod is removed from a node for any reason, the data in the emptyDir is deleted forever."* ## Custom MySQL configuration files The [MySQL](https://hub.docker.com/_/mysql/) image accepts custom configuration files at the path `/etc/mysql/conf.d`. If you want to use a customized MySQL configuration, you can create your alternative configuration files by passing the file contents on the `configurationFiles` attribute. Note that according to the MySQL documentation only files ending with `.cnf` are loaded. ```yaml configurationFiles: mysql.cnf: |- [mysqld] skip-host-cache skip-name-resolve sql-mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION mysql_custom.cnf: |- [mysqld] ``` ## MySQL initialization files The [MySQL](https://hub.docker.com/_/mysql/) image accepts *.sh, *.sql and *.sql.gz files at the path `/docker-entrypoint-initdb.d`. These files are being run exactly once for container initialization and ignored on following container restarts. If you want to use initialization scripts, you can create initialization files by passing the file contents on the `initializationFiles` attribute. ```yaml initializationFiles: first-db.sql: |- CREATE DATABASE IF NOT EXISTS first DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; second-db.sql: |- CREATE DATABASE IF NOT EXISTS second DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; ``` ## SSL This chart supports configuring MySQL to use [encrypted connections](https://dev.mysql.com/doc/refman/5.7/en/encrypted-connections.html) with TLS/SSL certificates provided by the user. This is accomplished by storing the required Certificate Authority file, the server public key certificate, and the server private key as a Kubernetes secret. The SSL options for this chart support the following use cases: * Manage certificate secrets with helm * Manage certificate secrets outside of helm ## Manage certificate secrets with helm Include your certificate data in the `ssl.certificates` section. For example: ``` ssl: enabled: false secret: mysql-ssl-certs certificates: - name: mysql-ssl-certs ca: |- -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- cert: |- -----BEGIN CERTIFICATE----- ... -----END CERTIFICATE----- key: |- -----BEGIN RSA PRIVATE KEY----- ... -----END RSA PRIVATE KEY----- ``` > **Note**: Make sure your certificate data has the correct formatting in the values file. ## Manage certificate secrets outside of helm 1. Ensure the certificate secret exist before installation of this chart. 2. Set the name of the certificate secret in `ssl.secret`. 3. Make sure there are no entries underneath `ssl.certificates`. To manually create the certificate secret from local files you can execute: ``` kubectl create secret generic mysql-ssl-certs \ --from-file=ca.pem=./ssl/certificate-authority.pem \ --from-file=server-cert.pem=./ssl/server-public-key.pem \ --from-file=server-key.pem=./ssl/server-private-key.pem ``` > **Note**: `ca.pem`, `server-cert.pem`, and `server-key.pem` **must** be used as the key names in this generic secret. If you are using a certificate your configurationFiles must include the three ssl lines under [mysqld] ``` [mysqld] ssl-ca=/ssl/ca.pem ssl-cert=/ssl/server-cert.pem ssl-key=/ssl/server-key.pem ```
(3) LICENSE
描述chart的许可信息，此文件为可选。
(4) requirements.yaml
指定chart的依赖关系，安装过程中，依赖的chart也会被安装。
(5) values.yaml
chart支持在安装时根据参数进行定制化配置，而values.xml则提供了这些配置参数的默认值。
## mysql image version ## ref: https://hub.docker.com/r/library/mysql/tags/ ## image: "mysql" imageTag: "5.7.14" ## Specify password for root user ## ## Default: random 10 character string # mysqlRootPassword: testing ## Create a database user ## # mysqlUser: ## Default: random 10 character string # mysqlPassword: ## Allow unauthenticated access, uncomment to enable ## # mysqlAllowEmptyPassword: true ## Create a database ## # mysqlDatabase: ## Specify an imagePullPolicy (Required) ## It's recommended to change this to 'Always' if the image tag is 'latest' ## ref: http://kubernetes.io/docs/user-guide/images/#updating-images ## imagePullPolicy: IfNotPresent extraVolumes: | # - name: extras # emptyDir: {} extraVolumeMounts: | # - name: extras # mountPath: /usr/share/extras # readOnly: true extraInitContainers: | # - name: do-something # image: busybox # command: ['do', 'something'] # Optionally specify an array of imagePullSecrets. # Secrets must be manually created in the namespace. # ref: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod # imagePullSecrets: # - name: myRegistryKeySecretName ## Node selector ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector nodeSelector: {} livenessProbe: initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 successThreshold: 1 failureThreshold: 3 readinessProbe: initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds: 1 successThreshold: 1 failureThreshold: 3 ## Persist data to a persistent volume persistence: enabled: true ## database data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## # storageClass: "-" accessMode: ReadWriteOnce size: 8Gi annotations: {} ## Configure resource requests and limits ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ ## resources: requests: memory: 256Mi cpu: 100m # Custom mysql configuration files used to override default mysql settings configurationFiles: {} # mysql.cnf: |- # [mysqld] # skip-name-resolve # ssl-ca=/ssl/ca.pem # ssl-cert=/ssl/server-cert.pem # ssl-key=/ssl/server-key.pem # Custom mysql init SQL files used to initialize the database initializationFiles: {} # first-db.sql: |- # CREATE DATABASE IF NOT EXISTS first DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; # second-db.sql: |- # CREATE DATABASE IF NOT EXISTS second DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; metrics: enabled: false image: prom/mysqld-exporter imageTag: v0.10.0 imagePullPolicy: IfNotPresent resources: {} annotations: {} # prometheus.io/scrape: "true" # prometheus.io/port: "9104" livenessProbe: initialDelaySeconds: 15 timeoutSeconds: 5 readinessProbe: initialDelaySeconds: 5 timeoutSeconds: 1 ## Configure the service ## ref: http://kubernetes.io/docs/user-guide/services/ service: ## Specify a service type ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types type: ClusterIP port: 3306 # nodePort: 32000 ssl: enabled: false secret: mysql-ssl-certs certificates: # - name: mysql-ssl-certs # ca: |- # -----BEGIN CERTIFICATE----- # ... # -----END CERTIFICATE----- # cert: |- # -----BEGIN CERTIFICATE----- # ... # -----END CERTIFICATE----- # key: |- # -----BEGIN RSA PRIVATE KEY----- # ... # -----END RSA PRIVATE KEY----- ## Populates the 'TZ' system timezone environment variable ## ref: https://dev.mysql.com/doc/refman/5.7/en/time-zone-support.html ## ## Default: nil (mysql will use image's default timezone, normally UTC) ## Example: 'Australia/Sydney' # timezone: # To be added to the database server pod(s) podAnnotations: {}
(6) templates:
k8s各种资源的配置模板都在这。Helm会将values.yaml中的参数值注入模板中，生成标准的YAML配置文件。
模板是chart的最重要的部分，也是Helm最强大的地方。模板增加了应用部署的灵活性，能够适用不同的环境。
（7）templates/NOTES.txt
chart的简易适用文档。
11.5.2 chart模板
大部分属性变成了 {{XXX}}。 这些实际上是模板语法。 Helm采用GO语言的模板编写chart。Go模板非常强大，支持变量、对象、函数、流控制等功能。
{{- if not .Values.existingSecret }}
apiVersion: v1
kind: Secret
metadata:
name: {{ template "mysql.fullname" . }} # 定义secret的name。关键字template的作用是引用一个模板mysql.fullname
labels:
app: {{ template "mysql.fullname" . }}
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
type: Opaque
data:
{{ if .Values.mysqlRootPassword }}
mysql-root-password: {{ .Values.mysqlRootPassword | b64enc | quote }}
{{ else }}
mysql-root-password: {{ randAlphaNum 10 | b64enc | quote }}
{{ end }}
{{ if .Values.mysqlPassword }}
mysql-password: {{ .Values.mysqlPassword | b64enc | quote }}
{{ else }}
mysql-password: {{ randAlphaNum 10 | b64enc | quote }}
{{ end }}
{{- if .Values.ssl.enabled }}
{{ if .Values.ssl.certificates }}
{{- range .Values.ssl.certificates }}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ .name }}
labels:
app: {{ template "mysql.fullname" $ }}
chart: "{{ $.Chart.Name }}-{{ $.Chart.Version }}"
release: "{{ $.Release.Name }}"
heritage: "{{ $.Release.Service }}"
type: Opaque
data:
ca.pem: {{ .ca | b64enc }}
server-cert.pem: {{ .cert | b64enc }}
server-key.pem: {{ .key | b64enc }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
11.5.3 再次实践MySQL chartt
安装之前需要清楚chart的使用方法。这些信息保存在values.yaml和README.MD,可以使用如下命令查看：
阅读注释可以知道MySQL chart支持哪些参数，安装前需要哪些准备。
kubeusr@GalaxyKubernetesMaster:~$ helm inspect values stable/mysql #输出的是Values.yaml的内容 ## mysql image version ## ref: https://hub.docker.com/r/library/mysql/tags/ ## image: "mysql" imageTag: "5.7.14" ## Specify password for root user ## ## Default: random 10 character string # mysqlRootPassword: testing ## Create a database user ## # mysqlUser: ## Default: random 10 character string # mysqlPassword: ## Allow unauthenticated access, uncomment to enable ## # mysqlAllowEmptyPassword: true ## Create a database ## # mysqlDatabase: ## Specify an imagePullPolicy (Required) ## It's recommended to change this to 'Always' if the image tag is 'latest' ## ref: http://kubernetes.io/docs/user-guide/images/#updating-images ## imagePullPolicy: IfNotPresent extraVolumes: | # - name: extras # emptyDir: {} extraVolumeMounts: | # - name: extras # mountPath: /usr/share/extras # readOnly: true extraInitContainers: | # - name: do-something # image: busybox # command: ['do', 'something'] # Optionally specify an array of imagePullSecrets. # Secrets must be manually created in the namespace. # ref: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod # imagePullSecrets: # - name: myRegistryKeySecretName ## Node selector ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector nodeSelector: {} livenessProbe: initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 successThreshold: 1 failureThreshold: 3 readinessProbe: initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds: 1 successThreshold: 1 failureThreshold: 3 ## Persist data to a persistent volume persistence: enabled: true ## database data Persistent Volume Storage Class ## If defined, storageClassName: <storageClass> ## If set to "-", storageClassName: "", which disables dynamic provisioning ## If undefined (the default) or set to null, no storageClassName spec is ## set, choosing the default provisioner. (gp2 on AWS, standard on ## GKE, AWS & OpenStack) ## # storageClass: "-" accessMode: ReadWriteOnce size: 8Gi annotations: {} ## Configure resource requests and limits ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ ## resources: requests: memory: 256Mi cpu: 100m # Custom mysql configuration files used to override default mysql settings configurationFiles: {} # mysql.cnf: |- # [mysqld] # skip-name-resolve # ssl-ca=/ssl/ca.pem # ssl-cert=/ssl/server-cert.pem # ssl-key=/ssl/server-key.pem # Custom mysql init SQL files used to initialize the database initializationFiles: {} # first-db.sql: |- # CREATE DATABASE IF NOT EXISTS first DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; # second-db.sql: |- # CREATE DATABASE IF NOT EXISTS second DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; metrics: enabled: false image: prom/mysqld-exporter imageTag: v0.10.0 imagePullPolicy: IfNotPresent resources: {} annotations: {} # prometheus.io/scrape: "true" # prometheus.io/port: "9104" livenessProbe: initialDelaySeconds: 15 timeoutSeconds: 5 readinessProbe: initialDelaySeconds: 5 timeoutSeconds: 1 ## Configure the service ## ref: http://kubernetes.io/docs/user-guide/services/ service: ## Specify a service type ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types type: ClusterIP port: 3306 # nodePort: 32000 ssl: enabled: false secret: mysql-ssl-certs certificates: # - name: mysql-ssl-certs # ca: |- # -----BEGIN CERTIFICATE----- # ... # -----END CERTIFICATE----- # cert: |- # -----BEGIN CERTIFICATE----- # ... # -----END CERTIFICATE----- # key: |- # -----BEGIN RSA PRIVATE KEY----- # ... # -----END RSA PRIVATE KEY----- ## Populates the 'TZ' system timezone environment variable ## ref: https://dev.mysql.com/doc/refman/5.7/en/time-zone-support.html ## ## Default: nil (mysql will use image's default timezone, normally UTC) ## Example: 'Australia/Sydney' # timezone: # To be added to the database server pod(s) podAnnotations: {}
chart定义了一个PVC， 申请存储空间。 因为实验环境不支持动态供给，所以要先申请：
kubectl apply -f mysql-pv.ymal # 这个文件是自定义的。
接下来就可以安装chart了：
2 定制化安装chart
可以接受values.yml的默认值，也可定制化，比如设置mysqlRootPassword。Helm支持两种方法传递参数：
(1) 指定自己的values文件, 通常做法是 helm inspect values mysql > myvalues.yaml生成values文件，然后设置mysqlRootPassword,最后执行
helm install --values=myvalues.yaml mysql
(2) 使用--set 直接传入参数值
helm install stable/mysql --set mysqlRootPassword=abc123-n myNAME: my
通过helm list 和 helm status XXX 可以查看chart的最新状态。
11.5.4 升级和回滚release
helm upgrade # 可以对已经发布的release进行升级 通过--values 或 --set应用新的配置。
helm history my # 可以查看所有版本
helm rollback my 1 # 可以回滚到任何版本
