Kubernetes进阶实战读书笔记：helm Charts

一、 helm Charts文件组织结构

Charts是HELM使用Kubernetes程序包打包格式、一个Chart就是一个描述一组Kubernetes资源的文件的集合
事实上、一个单独的Charts既能用于部署简单应用，例如一个memcached pod 也能部署复杂的应用如http服务器db服务器cache服务器

例如一个wordpress Charts的目录结构应该如下所示

[root@master stable]# tree wordpress/
wordpress/
├── Chart.yaml
├── OWNERS
├── README.md
├── requirements.lock
├── requirements.yaml
├── templates
│   ├── deployment.yaml
│   ├── externaldb-secrets.yaml
│   ├── _helpers.tpl
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── pvc.yaml
│   ├── secrets.yaml
│   ├── servicemonitor.yaml
│   ├── svc.yaml
│   ├── tests
│   │   └── test-mariadb-connection.yaml
│   └── tls-secrets.yaml
├── values-production.yaml
├── values.schema.json
└── values.yaml

1、Chart.yaml：当前Charts的描述信息、yaml格式的文件
2、LICENSE：当前Charts的许可信息，纯文本文件；此为可选文件
3、README.md：易读格式的README文件；可选
4、当前Charts依赖关系描述文件；可选
5、values.yaml：当前Charts用到的默认配置值。
6、ci：目录、存放当前Charts依赖到的所有Charts文件
7、templates：目录、存放当前Charts用到的模板文件，可应用于Charts生成有效的Kubernetes清单文件

二、Charts文件组织格式

1、例如一个redis Charts的Chart.yaml应该如下所示

[root@master redis]# cat Chart.yaml 
apiVersion: v1
name: redis
version: 10.5.7
appVersion: 5.0.7
# The redis chart is deprecated and no longer maintained. For details deprecation, see the PROCESSES.md file.
deprecated: true   #当前Chart是否已废弃、可选字段，布尔值
description: DEPRECATED Open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.
keywords:
- redis
- keyvalue
- database
home: http://redis.io/
icon: https://bitnami.com/assets/stacks/redis/img/redis-stack-220x234.png
sources:
- https://github.com/bitnami/bitnami-docker-redis
maintainers: []
engine: gotpl

maintainers项目维护者信息

[root@master jenkins]# cat Chart.yaml
apiVersion: v1
name: jenkins
home: https://jenkins.io/
version: 2.3.0
appVersion: lts
description: Open source continuous integration server. It supports multiple SCM tools
including CVS, Subversion and Git. It can execute Apache Ant and Apache Maven-based
projects as well as arbitrary scripts.
sources:
- https://github.com/jenkinsci/jenkins
- https://github.com/jenkinsci/docker-jnlp-slave
- https://github.com/maorfr/kube-tasks
- https://github.com/jenkinsci/configuration-as-code-plugin
maintainers: #项目维护者信息、主要嵌套name、email和URL几个属性组成；可选字段
- name: lachie83
email: lachlan.evenson@microsoft.com
- name: viglesiasce
email: viglesias@google.com
- name: maorfr
email: maor.friedman@redhat.com
- name: torstenwalter
email: mail@torstenwalter.de
- name: mogaal
email: garridomota@gmail.com
- name: wmcdona89
email: wmcdona89@gmail.com
icon: https://wiki.jenkins-ci.org/download/attachments/2916393/logo.png

三、模板和值

模板遵循Go模板语言格式，并支持50种以上的来自Spring库的模板函数附件，以及为数补多少的其他专用函数。所有的模板文件都存储与templates目中
在当前被引用时，此目录中的所有模板文件都会传递给模板引擎进行处理

模板引擎中用的值(value)有如下两种提供方式

1、通过的文件提供，通过用于提供默认值
2、在运行"helm install" 命令时传递包含所需要的的自定义值YAML文件；此处传递的值会覆盖默认值

下面的示例是wordpress中的模板文件deployment的部分内容

[root@master stable]# cat wordpress/templates/deployment.yaml 
apiVersion: {{ template "wordpress.deployment.apiVersion" . }}
kind: Deployment
metadata:
name: {{ template "wordpress.fullname" . }}
labels: {{- include "wordpress.labels" . | nindent 4 }}
spec:
selector:
matchLabels: {{- include "wordpress.matchLabels" . | nindent 6 }}
{{- if .Values.updateStrategy }}
strategy: {{ toYaml .Values.updateStrategy | nindent 4 }}
{{- end }}
replicas: {{ .Values.replicaCount }}
template:
metadata:
labels: {{- include "wordpress.labels" . | nindent 8 }}
{{- if or .Values.podAnnotations .Values.metrics.enabled }}
annotations:
{{- if .Values.podAnnotations }}
{{- include "wordpress.tplValue" (dict "value" .Values.podAnnotations "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.metrics.podAnnotations }}
{{- include "wordpress.tplValue" (dict "value" .Values.metrics.podAnnotations "context" $) | nindent 8 }}
{{- end }}
{{- end }}
spec:
{{- include "wordpress.imagePullSecrets" . | indent 6 }}
{{- if .Values.schedulerName }}
schedulerName: {{ .Values.schedulerName | quote }}
{{- end }}
hostAliases:
- ip: "127.0.0.1"
hostnames:
- "status.localhost"

而在values.yaml一类的文件中定义时，既可以将它定义为全局作用于、也可以定义为仅供Charts目录下的某个Charts所使用

一般来说上级Charts可以访问下级的Charts中的值、而下级Charts不能访问其上级Charts的值

service:
type: LoadBalancer
## HTTP Port
##
port: 80
## HTTPS Port
##
httpsPort: 443
## HTTPS Target Port
## defaults to https unless overridden to the specified port.
## if you want the target port to be "http" or "80" you can specify that here.
##
httpsTargetPort: https
## Metrics Port
##
metricsPort: 9117
## Node Ports to expose
## nodePorts:
## http: <to set explicitly, choose port between 30000-32767>
## https: <to set explicitly, choose port between 30000-32767>
## metrics: <to set explicitly, choose port between 30000-32767>
nodePorts:
http: ""
https: ""
metrics: ""

Go 模板语法请参考godoc站点中内容、地址为：https://godoc.org/text/template

四、自定义Charts

1、生成一个空Charts

[root@master ~]# helm create luoahongchart
Creating luoahongchart

[root@master ~]# tree luoahongchart/
luoahongchart/
├── charts
├── Chart.yaml
├── templates
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests
│   └── test-connection.yaml
└── values.yaml

3 directories, 9 files

由命令生成的各文件还有着各自应该具有的通用组织结构框架、例如Chart.yaml文件的默认内容如下

[root@master luoahongchart]# cat Chart.yaml 
apiVersion: v1
appVersion: "1.0"
description: A Helm chart for Kubernetes
name: luoahongchart
version: 0.1.0

事实上，它甚至直接在values.yaml 将要使用的镜像文件定义中为nginx生成了一个可直接安装容器化nginx应用的Charts，
期中的部分内容如下所示：

[root@master luoahongchart]# grep -vE "#|^$" values.yaml 
replicaCount: 1
image:
repository: nginx
tag: stable
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
name:
podSecurityContext: {}
securityContext: {}
service:
type: ClusterIP
port: 80

因此，用户仅需要在各文件现有框架的基础上按需进行修改即可定义出所需的Chart来

2.修改Charts以部署自定义服务

这里以此前使用的容器应用""为例来说明如何定义一个Charts

[root@master luoahongchart]# grep -vE "#|^$" values.yaml 
replicaCount: 1
image:
repository: kubernetes/myapp #更改nginx为kubernetes/myapp
tag: v1 #更改stable为v1
pullPolicy: IfNotPresent 
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
name:
podSecurityContext: {}
securityContext: {}
service:
type: ClusterIP
port: 80

而后通过"helm lint" 命令确认修改后的是否遵循最佳实践且模板格式良好

[root@master ~]# ll
total 4
-rw-------. 1 root root 1404 Apr 21 09:52 anaconda-ks.cfg
drwxr-xr-x 2 root root 108 Jul 9 17:21 k8s
drwxr-xr-x 4 root root 93 Jul 13 16:19 luoahongchart
[root@master ~]# helm lint luoahongchart
==> Linting luoahongchart
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, no failures

多数情况下，"helm lint"命令报告的错误信息、根据其错误提示中的行号信息即能定位出错误所在、确保一切问题都得以解决之后、即可通过"helm install"命令调试运行以查看由Charts定义的容器化应用是否能够正确部署

[root@master ~]# helm install --name myapp --dry-run --debug ./luoahongchart --set service.type=NodePort
[debug] Created tunnel using local port: '38624'

[debug] SERVER: "127.0.0.1:38624"

[debug] Original chart version: ""
[debug] CHART PATH: /root/luoahongchart

NAME: myapp
REVISION: 1
RELEASED: Mon Jul 13 16:34:30 2020
CHART: luoahongchart-0.1.0
USER-SUPPLIED VALUES:
service:
type: NodePort

确认上述命令输出信息无误后、移除命令中的"--dry-run" 选项后再次运行命令即可部署完成应用的部署

[root@master ~]# helm install --name myapp ./luoahongchart --set service.type=NodePort
NAME: myapp
LAST DEPLOYED: Mon Jul 13 16:36:37 2020
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
myapp-luoahongchart 0/1 0 0 0s

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
myapp-luoahongchart-6777bd6b65-fslx2 0/1 ContainerCreating 0 0s

==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
myapp-luoahongchart NodePort 10.99.123.248 <none> 80:30435/TCP 0s

==> v1/ServiceAccount
NAME SECRETS AGE
myapp-luoahongchart 1 0s


NOTES:
1. Get the application URL by running these commands:
export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services myapp-luoahongchart)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT

而后、通过删除NOTES中的命令提示运行相关的命令获取访问端点后即可通过浏览器访问相应的服务

[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
myapp-luoahongchart-6777bd6b65-fslx2 1/1 Running 0 6m5s 10.244.2.19 nodes2 <none> <none>

[root@master ~]#export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services myapp-luoahongchart)
[root@master ~]#export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
[root@master ~]#echo http://$NODE_IP:$NODE_PORT
http://192.168.118.18:30435

而后通过浏览器访问测试所部属的myapp应用

3.Charts仓库

至此、一个自定义的基于本地设定完成、不过、它仅能用于本地访问、当然用户可以通过"helm package"命令将其打包为tar格式后分享给团队或者社区：

[root@master ~]# helm package ./luoahongchart
Successfully packaged chart and saved it to: /root/luoahongchart-0.1.0.tgz

[root@master ~]# helm serve
Regenerating index. This may take a moment.
Now serving you on 127.0.0.1:8879

此命令会占据当前终端，于是、另起一个终端即可测试访问仓库服务中Charts：

[root@master ~]# helm search local
NAME CHART VERSION	APP VERSION	DESCRIPTION 
local/luoahongchart 0.1.0 1.0 A Helm chart for Kubernetes 
incubator/puppet-forge	0.1.8 1.10.0 Distribute locally developed Puppet modules and proxy to ...

向外分享创建好的，只能自己基于web服务器程序来满足、如果要外部访问建议使用https的服务器提供仓库服务

添加

[root@master luoahongchart]# helm repo add incubator https://kubernetes-charts.storage.googleapis.com
"incubator" has been added to your repositories

helm repo add incubator https://kubernetes-charts.storage.googleapis.com

查看

[root@master luoahongchart]# helm repo list
NAME URL 
local http://127.0.0.1:8879/charts 
stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
incubator	https://kubernetes-charts.storage.googleapis.com

更新

[root@master ~]# helm repo update
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "stable" chart repository
...Unable to get an update from the "incubator" chart repository (https://kubernetes-charts-incubator.storage.googleapis.com):
Get https://kubernetes-charts-incubator.storage.googleapis.com/index.yaml: dial tcp 34.64.4.80:443: connect: connection timed out
Update Complete.

而删除制定的仓库配置"helm repo remove <REPO_NAME>"

4、配置依赖关系

构建存在依赖关系的charts时、还需要为其定义依赖项、例如、前面创建的myapp依赖于数据库管库系统MYSQL时、在luoahongchart
的目录中创建如下文件：

[root@master ~]# cat ./luoahongchart/requirements.yaml 
dependencies:
- name: mysql
version: 0.6.0
repository: https://kubernetes-charts.storage.googleapis.com

而后、需要运行"helm dependency update" 命令为Charts更新依赖关系

运行下面的命令来引入定义的MySQL依赖项时、会自动下载MySQL相关的charts程序包至./luoahongchart/charts/子目录中

[root@master ~]# helm dependency update ./luoahongchart
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused
...Successfully got an update from the "stable" chart repository
...Successfully got an update from the "incubator" chart repository
Update Complete.
Saving 1 charts
Downloading mysql from repo https://kubernetes-charts.storage.googleapis.com
Deleting outdated charts

更新过程中helm会自动生成一个锁定文件requirements.lock、以便后续再次获取依赖关系时使用已知的工作版本

[root@master ~]# ll ./luoahongchart
total 16
drwxr-xr-x 2 root root 29 Jul 13 17:31 charts
-rw-r--r-- 1 root root 109 Jul 13 16:05 Chart.yaml
-rw-r--r-- 1 root root 237 Jul 13 17:31 requirements.lock
-rw-r--r-- 1 root root 108 Jul 13 17:24 requirements.yaml
drwxr-xr-x 3 root root 146 Jul 13 16:05 templates
-rw-r--r-- 1 root root 1519 Jul 13 16:19 values.yaml

程序包至./luoahongchart/charts/子目录中

[root@master ~]# ll ./luoahongchart/charts/
total 8
-rw-r--r-- 1 root root 7589 Jul 13 17:31 mysql-0.6.0.tgz

此时、再次部署myapp Charts、就会同事部署依赖到mysql Charts、另外、用户也可以手动将锁依赖到的程序包直接放置于luoahongchart/charts/目录中
来定义依赖关系、此时不要在使用requirements.yaml文件