k8s部署nacos集群

一、部署依赖及环境

1.部署依赖

MySQL
storageClass

2.部署环境

阿里云ACK
MySQL 5.7
k8s 1.24
nacos 2.2.3
namespace 为default自行修改

二、创建及初始化数据库

wget https://raw.githubusercontent.com/alibaba/nacos/develop/distribution/conf/mysql-schema.sql
mysql -uroot -p
CREATE DATABASE IF NOT EXISTS nacos DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_general_ci;
use nacos;
source < /root/mysql-schema.sql;
CREATE USER 'nocos'@'%' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON nacos.* TO 'nocos'@'%';
FLUSH PRIVILEGES;
exit;

三、创建naocs连接MySQL配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
  namespace: default  
data:
  mysql.host: "xxxxx.mysql.rds.aliyuncs.com"
  mysql.db.name: "nacos"
  mysql.port: "3306"
  mysql.user: "nacos"
  mysql.password: "xxxxxxxxxxx"

四、创建svc

apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  namespace: default    
  labels:
    app: nacos
spec:
  publishNotReadyAddresses: true 
  ports:
    - port: 8848
      name: server
      targetPort: 8848
    - port: 9848
      name: client-rpc
      targetPort: 9848
    - port: 9849
      name: raft-rpc
      targetPort: 9849
    ## 兼容1.4.x版本的选举端口
    - port: 7848
      name: old-raft-rpc
      targetPort: 7848
  clusterIP: None
  selector:
    app: nacos

五、创建stateful

1.注意事项

该文件参考官方文档修改
1.pvc配置
这里 volume.beta.kubernetes.io/storage-class: "alicloud-disk-nas"中将alicloud-disk-nas替换成你自己的storageClass，如果没有storage-Class则需要修改文件添加对应的PVC/PVP配置
2.镜像配置
镜像我指定了具体版本，所以下载策略也同时修改了
3.认证配置，参考官方文档自行配置

- name: NACOS_AUTH_ENABLE
value: "true"
- name: NACOS_AUTH_IDENTITY_KEY
value: "nacosAuthKey"
- name: NACOS_AUTH_IDENTITY_VALUE
value: "nacosSecurtyValue"
- name: NACOS_AUTH_TOKEN
value: "SecretKey012345678901234567890123456789012345678901234567890123456789"
- name: NACOS_AUTH_TOKEN_EXPIRE_SECONDS
value: 18000

2.配置如下

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: default 
spec:
  podManagementPolicy: Parallel
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - nacos
              topologyKey: "kubernetes.io/hostname"
      initContainers:
        - name: peer-finder-plugin-install
          image: nacos/nacos-peer-finder-plugin:1.1
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - mountPath: /home/nacos/plugins/peer-finder
              name: data
              subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: IfNotPresent
          image: nacos/nacos-server:v2.2.3
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - containerPort: 8848
              name: client-port
            - containerPort: 9848
              name: client-rpc
            - containerPort: 9849
              name: raft-rpc
            - containerPort: 7848
              name: old-raft-rpc
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: SERVICE_NAME
              value: "nacos-headless"
            - name: DOMAIN_NAME
              value: "cluster.local"
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.password
            - name: SPRING_DATASOURCE_PLATFORM
              value: "mysql"
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: NACOS_APPLICATION_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_AUTH_ENABLE
              value: "true"
            - name: NACOS_AUTH_IDENTITY_KEY
              value: "nacosAuthKey"
            - name: NACOS_AUTH_IDENTITY_VALUE
              value: "nacosSecurtyValue"
            - name: NACOS_AUTH_TOKEN
              value: "SecretKey012345678901234567890123456789012345678901234567890123456789"
            - name: NACOS_AUTH_TOKEN_EXPIRE_SECONDS
              value: "18000"
          volumeMounts:
            - name: data
              mountPath: /home/nacos/plugins/peer-finder
              subPath: peer-finder
            - name: data
              mountPath: /home/nacos/data
              subPath: data
            - name: data
              mountPath: /home/nacos/logs
              subPath: logs
  volumeClaimTemplates:
    - metadata:
        name: data
        annotations:
          volume.beta.kubernetes.io/storage-class: "alicloud-disk-nas"
      spec:
        accessModes: [ "ReadWriteMany" ]
        resources:
          requests:
            storage: 20Gi
  selector:
    matchLabels:
      app: nacos

检查cluster是否一致

for i in {0..2}; do echo nacos-$i; kubectl exec nacos-$i -c nacos -- cat conf/cluster.conf; done

六、创建ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/service-weight: ""
  labels:
    ingress-controller: nginx
  name: nacos.inktech.ltd
  namespace: default
spec:
  rules:
  - host: nacos.xxx.com
    http:
      paths:
      - backend:
          service:
            name: nacos-headless
            port:
              number: 8848
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - nacos.xxx.com
    secretName: xxx.com-tls

参考文档：https://github.com/nacos-group/nacos-k8s/blob/master/README-CN.md