helm 部署 Nacos 集群
目录
资源清单
本文安装
Nacos依赖K8S集群和helm，本文不提供K8S集群和helm安装方式
使用此文档部署，需要自行解决 storageClass 问题 （ NFS, ceph, openebs等 ）
使用此文档部署，需要自行解决 外部数据库问题 （ mysql 主从 ）
使用此文档部署，需要自行解决 ingress nginx 问题，部署服务会定义ingress nginx （ 也可以不启用 ingress，使用 NodePort 进行访问 ）
| 软件 | 版本 |
|---|---|
| chart | 2.0.1 |
| nacos | 2.0.4 |
| kubernetes | v1.19.3 |
| helm | v3.8.1 |
| mysql 主从 | 5.7.26 |
| ingress nginx | 3.15.2 |
helm 安装 Nacos 集群
1. 添加 nacos helm 仓库
$ helm repo add ygqygq2 https://ygqygq2.github.io/charts/
2. 查询 nacos 资源
$ helm search repo nacos
NAME CHART VERSION APP VERSION DESCRIPTION
ygqygq2/nacos 2.0.1 2.0.4 Chart for nacos, an easy-to-use dynamic service...
3. 拉取 nacos chart 到本地
$ mkdir /root/nacos && cd /root/nacos
# 拉取 chart 到本地 /root/nacos 目录
$ helm pull ygqygq2/nacos --version 2.0.1
$ tar -xvf nacos-2.0.1.tgz
$ cp nacos/values.yaml ./values-test.yaml
# 查看当前目录层级
$ tree -L 2
.
├── nacos
│ ├── Chart.yaml
│ ├── LICENSE
│ ├── README.md
│ ├── templates
│ └── values.yaml
├── nacos-2.0.1.tgz
└── value-test.yaml
4. 对本地 values-test.yaml 修改
- 查看集群
storageclasses
$ kubectl get storageclasses.storage.k8s.io
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
openebs-jiva-default jiva.csi.openebs.io Delete Immediate true 33d
- 修改配置
如需定义其他配置，可以查看
nacos/values.yaml文件配置，然后复制到values-test.yaml中进行定义
$ cat values-test.yaml
## 全局配置定义 storageClass: ""，会使用集群配置的 openebs 提供的 storageClass，
## 使用此文档部署，需要自行解决 storageClass 问题 (ceph, nfs, 公有云提供的 nfs)
# 副本个数
replicaCount: 3
service:
type: ClusterIP # 一般不用修改
ingressPort: 8848
ports:
http:
port: 8848 # Service port number for client-a port.
protocol: TCP # Service port protocol for client-a port.
persistentVolume: # 是否存储持久化
enabled: true # 启用持久化存储
storageClass: "openebs-jiva-default" # 设置 storageClass
accessMode: ReadWriteOnce
annotations: {}
# helm.sh/resource-policy: keep
size: 10Gi # 大小为 10G
ingress: # 是否使用nginx暴露域名或端口
enabled: true # 启用 ingress
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
ingressClassName: ""
path: /nacos
pathType: ImplementationSpecific
hosts:
- nacos.evescn.com # 设置域名
tls: [] # 未启动 https认证
mysql:
enabled: false
# 启用外部存储，而不是自建 mysql 主从
# 可以自建 nacos 用户，这里此处直接使用了 root 账户进行 mysql 集群登陆
# 需要去数据库新建 nacos 数据库
external:
mysqlMasterHost: "mysql-cluster-mysql.test-middleware"
mysqlDatabase: "nacos"
mysqlMasterPort: "3306"
mysqlMasterUser: "root"
mysqlMasterPassword: "root123"
mysqlSlaveHost: "mysql-cluster-mysql-slave.test-middleware"
mysqlSlavePort: "3306"
5. 修改模板文件
root/nacos/nacos/secret copy.yaml模板文件需要修改，否则安装会报错
- 直接安装报错信息
$ helm install nacos-cluster nacos -f value-test.yaml | tee 2.log
Error: INSTALLATION FAILED: YAML parse error on nacos/templates/secret.yaml: error converting YAML to JSON: yaml: line 12: could not find expected ':'
# /root/nacos/nacos/secret.yml 模板文件渲染报错
- 原模板文件
$ cat /root/nacos/nacos/secret.yaml.copy
apiVersion: v1
kind: Secret
metadata:
name: {{ template "nacos.fullname" . }}
labels:
app: {{ template "nacos.name" . }}
chart: {{ template "nacos.chart" . }}
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
{{- if .Values.labels }}
{{ toYaml .Values.labels | indent 4 }}
{{- end }}
data:
{{- if not .Values.mysql.enabled }}
{{ toYaml .Values.mysql.external | indent 2 | b64enc }}
{{- else }}
mysqlMasterHost: {{ (include "nacos.mysql.primaryHost" .) | b64enc | quote }}
mysqlDatabase: {{ .Values.mysql.auth.database | b64enc | quote }}
mysqlMasterPort: {{ "3306" | b64enc }}
mysqlMasterUser: {{ .Values.mysql.auth.username | b64enc | quote }}
mysqlMasterPassword: {{ .Values.mysql.auth.password | b64enc | quote }}
mysqlSlaveHost: {{ (include "nacos.mysql.secondaryHost" .) | b64enc | quote }}
mysqlSlavePort: {{ "3306" | b64enc }}
{{- end }}
{{- if .Values.secret.enabled }}
{{- range $key, $value := .Values.secret.data }}
{{ $key }}: {{ $value | b64enc | quote }}
{{- end }}
{{- end }}
- 修改后的模板文件
$ cat /root/nacos/nacos/templates/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: {{ template "nacos.fullname" . }}
labels:
app: {{ template "nacos.name" . }}
chart: {{ template "nacos.chart" . }}
release: "{{ .Release.Name }}"
heritage: "{{ .Release.Service }}"
{{- if .Values.labels }}
{{ toYaml .Values.labels | indent 4 }}
{{- end }}
data: # 修改了 data中 if else 配置， 对 if else 配置进行了调整
{{- if .Values.mysql.enabled }}
mysqlMasterHost: {{ (include "nacos.mysql.primaryHost" .) | b64enc | quote }}
mysqlDatabase: {{ .Values.mysql.auth.database | b64enc | quote }}
mysqlMasterPort: {{ "3306" | b64enc }}
mysqlMasterUser: {{ .Values.mysql.auth.username | b64enc | quote }}
mysqlMasterPassword: {{ .Values.mysql.auth.password | b64enc | quote }}
mysqlSlaveHost: {{ (include "nacos.mysql.secondaryHost" .) | b64enc | quote }}
mysqlSlavePort: {{ "3306" | b64enc }}
{{- else }}
mysqlMasterHost: {{ .Values.mysql.external.mysqlMasterHost | b64enc | quote }}
mysqlDatabase: {{ .Values.mysql.external.mysqlDatabase | b64enc | quote }}
mysqlMasterPort: {{ .Values.mysql.external.mysqlMasterPort | b64enc }}
mysqlMasterUser: {{ .Values.mysql.external.mysqlMasterUser | b64enc | quote }}
mysqlMasterPassword: {{ .Values.mysql.external.mysqlMasterPassword | b64enc | quote}}
mysqlSlaveHost: {{ .Values.mysql.external.mysqlSlaveHost | b64enc | quote }}
mysqlSlavePort: {{ .Values.mysql.external.mysqlSlavePort | b64enc }}
{{- end }}
{{- if .Values.secret.enabled }}
{{- range $key, $value := .Values.secret.data }}
{{ $key }}: {{ $value | b64enc | quote }}
{{- end }}
{{- end }}
6. 创建数据库
# 启动一个临时容器连接 mysql 数据库
$ kubectl run mysql-cluster-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:5.7.26 --namespace test-middleware --command -- bash
## 登陆 MySQL Master节点
$ mysql -h mysql-cluster-mysql.test-middleware -uroot -p
Enter password: # root123
mysql> create database nacos default character set utf8 collate utf8_general_ci;
7. 安装依赖
nacos helm依赖其他第三方chart:mysql和common，需要提前安装，否则直接安装报错
$ cd /root/nacos
# 直接安装报错提示
$ helm install nacos-cluster nacos/ -f value-test.yaml
Error: INSTALLATION FAILED: An error occurred while checking for chart dependencies. You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: common, mysql
# 安装第三方 chart
$ cd /root/nacos/nacos/
$ helm dependency build
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "ygqygq2" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 2 charts
Downloading common from repo https://charts.bitnami.com/bitnami
Downloading mysql from repo https://charts.bitnami.com/bitnami
Deleting outdated charts
# 查看当前目录层级
$ tree -L 2
.
├── nacos
│ ├── Chart.lock
│ ├── charts
│ │ ├── common-1.13.1.tgz
│ │ └── mysql-8.9.6.tgz
│ ├── Chart.yaml
│ ├── LICENSE
│ ├── README.md
│ ├── secret\ copy.yaml
│ ├── templates
│ └── values.yaml
├── nacos-2.0.1.tgz
└── value-test.yaml
8. 安装 nacos 集群
# 创建 test-middleware 名称空间
$ kubectl create ns test-middleware
# 安装 nacos 集群
$ helm -n test-middleware install nacos-cluster nacos -f value-test.yaml
## helm -n NAMESAPCE install SERVER_NAME FILE_NAME -f CONFIG_FILE
-n 指定 kubernetes 集群名称空间
-f 指定使用的配置文件，文件中定义的配置可以覆盖 mysql/values.yaml 文件中配置
NAME: nacos-cluster
LAST DEPLOYED: Mon May 9 21:26:22 2022
NAMESPACE: test-middleware
STATUS: deployed
REVISION: 1
NOTES:
The nacos has been installed.
Nacos can be accessed:
* The application URL:
http://nacos.evescn.com/nacos
* Within your cluster, at the following DNS name at port 8848:
nacos-cluster.test-middleware.svc
* From outside the cluster, run these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace test-middleware -l "app=nacos,release=nacos-cluster" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:8848/nacos to use nacos"
kubectl port-forward --namespace test-middleware $POD_NAME 8848:8848
# The default user is: nacos
# The default password is: nacos
9. 验证 nacos 集群
- 查看 pod
$ kubectl get pods --namespace test-middleware -l "app=nacos,release=nacos-cluster"
NAME READY STATUS RESTARTS AGE
nacos-cluster-0 1/1 Running 0 24m
nacos-cluster-1 1/1 Running 0 22m
nacos-cluster-2 1/1 Running 0 18m
- 查看 pvc
$ kubectl -n test-middleware get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-storage-nacos-cluster-0 Bound pvc-0438916d-7ff6-4642-acb2-703e72ddb6b5 10Gi RWO openebs-jiva-default 7m53s
data-storage-nacos-cluster-1 Bound pvc-652cb7b9-1a9c-428c-be56-45dfaa3d0c88 10Gi RWO openebs-jiva-default 6m7s
data-storage-nacos-cluster-2 Bound pvc-08a67a3e-8855-4a3b-b918-4de97c96cb33 10Gi RWO openebs-jiva-default 2m11s
- 查看 MySQL 数据库
# 启动一个临时容器连接 mysql 数据库
$ kubectl run mysql-cluster-mysql-client --rm --tty -i --restart='Never' --image docker.io/bitnami/mysql:5.7.26 --namespace test-middleware --command -- bash
# 登陆 MySQL Master节点
$ mysql -h mysql-cluster-mysql.test-middleware -uroot -p
Enter password: # root123
mysql> use nacos;
mysql> show tables;
+----------------------+
| Tables_in_nacos |
+----------------------+
| config_info |
| config_info_aggr |
| config_info_beta |
| config_info_tag |
| config_tags_relation |
| group_capacity |
| his_config_info |
| permissions |
| roles |
| tenant_capacity |
| tenant_info |
| users |
+----------------------+
12 rows in set (0.00 sec)
- 登陆
web页面查看
http://nacos.evescn.com/nacos/
参考文档
https://artifacthub.io/packages/helm/ygqygq2/nacos?modal=values
# 修改 helm 模板参考文档
https://helm.sh/zh/docs/chart_template_guide/control_structures/
https://www.cnblogs.com/ssgeek/p/15511387.html
