k8s_mysql主从

https://cdn.modb.pro/db/428365

https://blog.51cto.com/u_15048360/3121394

https://blog.csdn.net/qq_48480384/article/details/127423322

错误信息：https://blog.csdn.net/weixin_49471976/article/details/126497093，

centos7上的docker容器没有hostname命令，不能用`hostname`获取，解决方法
1.yum install hostname
2.使用$HOSTNAME

一.部署storageclass相关

1.创建serviceaccount

k8s集群上个可以只有一个serviceaccount权限文件，然后provisioner工具绑定，起的名称不一样即可

nfs-client-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  namespace: storage-class
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: storage-class
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  namespace: storage-class
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  namespace: storage-class
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: storage-class
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

2.下载nfs-client-provisioner工具

provisioner-mysql.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-client-provisioner-mysql  #自定义名称
  namespace: storage-class
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner 
    spec:
      #绑定service account名称，使得自动化配置工具能访问pv、pvc等资源
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner-mysql #自定义名称
          # 下载nfs-client-provisioner工具
          image: quay.io/external_storage/nfs-client-provisioner:latest
          env:
            - name: PROVISIONER_NAME #给nfs-client-provisioner起个名字
              value: nfs-storage-mysql
            - name: NFS_SERVER
              value: 192.168.181.141 #nfs server地址
            - name: NFS_PATH
              value: /data/mysqldata #nfs共享路径
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes #具体挂载路径
      volumes:       #定义一个nfs类型的volume，将nfs挂载到容器中
        - name: nfs-client-root
          nfs:
            server: 192.168.181.141
            path: /data/mysqldata

3.storageclass

storageclass.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storageclass-mysql
provisioner: nfs-storage-mysql  #匹配找到env中PROVISIONER_NAME的值，通过其创建pv
reclaimPolicy: Retain        #回收策略

二.mysql相关

1.configmap

configmap.yaml，初始化mysql配置文件

apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql
  labels:
    app: mysql
data:
  master.cnf: |
    # Apply this config only on the master.
    [mysqld]
    log-bin
  slave.cnf: |
    # Apply this config only on slaves.
    [mysqld]
    super-read-only

2.service

mysql-service.yaml，创建无头和有头service，无头用来部署satefulset控制器，有头用来外部访问

apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  ports:
  - name: mysql
    port: 3306
  clusterIP: None
  selector:
    app: mysql
---
# Client service for connecting to any MySQL instance for reads.
# For writes, you must instead connect to the master: mysql-0.mysql.
apiVersion: v1
kind: Service
metadata:
  name: mysql-read
  labels:
    app: mysql
spec:
  ports:
  - name: mysql
    port: 3306
  selector:
    app: mysql

3.satefulset

mysql-satefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  serviceName: mysql
  replicas: 2 
  template:
    metadata:
      labels:
        app: mysql
    spec:
      initContainers:
      - name: init-mysql
        image: mysql:5.7
        imagePullPolicy: IfNotPresent
        command:
        - /bin/sh
        - "-c"
        - |
          set -ex
          [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
          ordinal=${BASH_REMATCH[1]}
          if [[ $ordinal -eq 0 ]]; then
            cp /mnt/config-map/master.cnf /mnt/conf.d/my.cnf
          else
            cp /mnt/config-map/slave.cnf /mnt/conf.d/my.cnf
          fi 
          cat <<EOF >> /mnt/conf.d/my.cnf
          server-id=$((100 + $ordinal))
          port=3306
          datadir=/var/lib/mysql
          socket=/var/lib/mysql/mysql.sock 
          [mysqld_safe]
          log-error=/var/log/mariadb/mariadb.log
          pid-file=/var/run/mariadb/mariadb.pid
          EOF
        volumeMounts:
        - name: conf
          mountPath: /mnt/conf.d
        - name: config-map
          mountPath: /mnt/config-map
      containers:
      - name: mysql
        image: mysql:5.7
        lifecycle:
          postStart:
            exec:
              command: 
                - /bin/sh
                - "-c"
                - |
                  set -ex
                  sleep 5
                  user='repl'
                  [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1
                  ordinal=${BASH_REMATCH[1]}
                  if [[ $ordinal -eq 0 ]]; then
                    result=`mysql -e "select user from mysql.user"`
                    if [[ $result =~ $user ]]; then
                      logfile=`mysql -e "show master status"|awk 'NR==2{print $1}'`
                      position=`mysql -e "show master status"|awk 'NR==2{print $2}'`
                      echo $logfile > /position/a1.txt
                      echo $position > /position/a2.txt     
                    else 
                      mysql -h127.0.0.1 -e "create user 'repl'@'%' identified by 'slavepass';grant replication slave on *.* to 'repl'@'%';flush privileges;"
                      logfile=`mysql -e "show master status"|awk 'NR==2{print $1}'`
                      position=`mysql -e "show master status"|awk 'NR==2{print $2}'`
                      echo $logfile > /position/a1.txt
                      echo $position > /postion/a2.txt
                    fi
                  else
                   #如果已经开启了slave，重启集群后不用再开始，因为状态都保存在/var/lib/mysql下
                   array=($(mysql  -e "show slave status\G"|grep "Running" |awk '{print $2}'))
                   if [ "${array[0]}" == "Yes" ] && [ "${array[1]}" == "Yes" ];then
                     echo 1
                   else
                     logfile=`cat /position/a1.txt`
                     position=`cat /position/a2.txt`
                     mysql -h127.0.0.1 -e "CHANGE MASTER TO MASTER_HOST='mysql-0.mysql',MASTER_USER='repl',MASTER_PASSWORD='slavepass',MASTER_LOG_FILE='$logfile',MASTER_LOG_POS=$position;"
                     mysql -h127.0.0.1 -e "start slave;"
                   fi
                  fi
        env:
        - name: MYSQL_ALLOW_EMPTY_PASSWORD 
          value: "1"
        ports:
        - name: mysql
          containerPort: 3306
        volumeMounts:
        # name为data的用的是storageclass，自动生成pv、pvc。会数据隔离
        - name: data
          mountPath: /var/lib/mysql
          subPath: mysql
        - name: data
          mountPath: /var/log
          subPath: log
        - name: data
          mountPath: /var/run
          subPath: run
        # initContainers中生成，containers中挂载上
        - name: conf
          mountPath: /etc/my.cnf
          subPath: my.cnf
        # name为position的因为是手动写volumes，因此所有pod用的是一个nfs，此处主mysql生成的postion和logfile会存储在/positon，从mysql从挂载的/position上读取
        - name: position
          mountPath: /position
        resources:
          requests:
            cpu: 500m
            memory: 1G
        livenessProbe:
          exec:
            command: ["mysqladmin", "ping"]
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
        readinessProbe:
          exec:
            # Check we can execute queries over TCP (skip-networking is off).
            command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]
          initialDelaySeconds: 5
          periodSeconds: 2
          timeoutSeconds: 1
      volumes:
      - name: conf
        emptyDir: {}
      - name: config-map
        configMap:
          name: mysql
      - name: position
        nfs:
          path: /data/mysqldata/position
          server: 192.168.181.141
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      storageClassName: "nfs-storageclass-mysql"
      accessModes: ["ReadWriteMany"]
      resources:
        requests:
          storage: 0.5Gi

注意点：

1.command和postStart不一定谁先执行，command在文件不写，是有默认命令，尽量不要改动，因此在poststart加上sleep 5,等容器加载完再执行
2.initContainers要生成或操作的文件只会在挂载目录显示，在pod中没有，只能去宿主机上找，但是containers下的配置可以挂载上并操作
3.手动volumes写nfs时多个pod使用的同一个存储，多pod之间可同步数据，基于storageclass生成不同的存储空间，因此多个pod之间无法同步数据

所有的配置安装完成，kubectl exec -it mysql-0 -- sh进入主库，创建库或表，再kubectl exec -itt mysql-1 -- sh进入从库，可看到相同的。

三.测试

1.写数据

我们可以尝试向这个 MySQL 集群发起请求，执行一些 SQL 操作来验证它是否正常

kubectl run mysql-client --image=mysql:5.7 -i --rm --restart=Never -- mysql -h mysql-0.mysql << EOF
CREATE DATABASE test;
CREATE TABLE test.messages (message VARCHAR(250));
INSERT INTO test.messages VALUES ('hello');
EOF

2.读数据

如上所示，我们通过启动一个容器，使用 MySQL client 执行了创建数据库和表、以及插入数据的操作。需要注意的是，我们连接的 MySQL 的地址必须是 mysql-0.mysql（即：Master 节点的 DNS A 记录, 因为POD 之间通过DNS A 记录互相通信）只有 Master 节点才能处理写操作。

而通过连接 mysql-read 这个 Service，我们就可以用 SQL 进行读操作，如下所示

kubectl run mysql-client --image=mysql:5.7 -i -t --rm --restart=Never --\
  mysql -h mysql-read -e "SELECT * FROM test.messages"

注意：k8s的yaml文件下载的镜像都是存放在node上的，上面的mysql:5.7也是存放在node上的，kubectl run mysql-client -- image=xx是命令行启动一个pod，代替了yaml文件