|NO.Z.00265|——————————|^^ 部署 ^^|——|KuberNetes&中间件容器化及helm.V12|——|中间件.v01|RabbitMQ|部署|

一、RabbitMQ概述

### --- RabbitMQ概述

~~~     相比redis：RabbitMQ是支持服务发现的，可以利用k8s的服务发现机制；然后自动的配置集群，
~~~     说明即使没有后端存储，也是可以运行在k8s集群中的。
~~~     而且RabbitMQ扩容和缩容是非常简单的一件事，因为它用的是k8s的服务发现机制，查找符合条件的集群节点。自动的加入到集群当中。
~~~     # 建议使用StatefulSet来安装RabbitMQ

### --- 官方地址：

~~~     https://github.com/dotbalo/k8s/tree/master/k8s-rabbitmq-cluster

---

一、RabbitMQ安装

### --- 拉取RabbitMQ的程序包

[root@k8s-master01 RabbitMQ]# git clone https://github.com/dotbalo/k8s.git
Cloning into 'k8s'...
remote: Enumerating objects: 683, done.
remote: Counting objects: 100% (51/51), done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 683 (delta 18), reused 21 (delta 4), pack-reused 632
Receiving objects: 100% (683/683), 5.02 MiB | 975.00 KiB/s, done.
Resolving deltas: 100% (294/294), done.

二、创建rabbitmq.namespace

### --- 创建namespace

[root@k8s-master01 rabbit-cluster]# kubectl create ns public-service
namespace/public-service created

三、创建rabbitmq-configmap.yaml配置文件

### --- 创建rabbitmq-configmap-yaml文件
~~~     模板地址：https://github.com/dotbalo/k8s/blob/master/k8s-rabbitmq-cluster/rabbitmq-configmap.yaml

[root@k8s-master01 rabbit-cluster]# vim rabbitmq-configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: rmq-cluster-config
  namespace: public-service
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
data:
    enabled_plugins: |
      [rabbitmq_management,rabbitmq_peer_discovery_k8s].
    rabbitmq.conf: |
      loopback_users.guest = false
      ## Clustering
      cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
      cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
      cluster_formation.k8s.address_type = hostname
      #################################################
      # public-service is rabbitmq-cluster's namespace#
      #################################################
      cluster_formation.k8s.hostname_suffix = .rmq-cluster.public-service.svc.cluster.local
      cluster_formation.node_cleanup.interval = 10
      cluster_formation.node_cleanup.only_log_warning = true
      cluster_partition_handling = autoheal
      ## queue master locator
      queue_master_locator=min-masters

### --- 注：

  name: rmq-cluster-config                              // 创建了一个configmap，它的名称是rmq-cluster-config
  namespace: public-service                             // namespace
data:                                                   // 它会创建2个文件
    enabled_plugins: |                                  // 第一个，它需要开启的插件
      [rabbitmq_management,rabbitmq_peer_discovery_k8s].// 使用这个插件自动发现这个集群有多少个redis，然后根据自动发现加入集群或者缩容集群
    rabbitmq.conf: |
      loopback_users.guest = false                      // 第二个，集群的配置文件；根据这些配置来实现自动发现

四、创建rabbitmq-configmap

### --- 创建configmap

[root@k8s-master01 rabbit-cluster]# kubectl create -f rabbitmq-configmap.yaml 
configmap/rmq-cluster-config created

五、创建rabbitmq-secret.yaml配置文件

### --- 编写secret配置文件
~~~     注：保存的是一些它的配置信息；账号密码url等信息

[root@k8s-master01 rabbit-cluster]# vim rabbitmq-secret.yaml 
kind: Secret
apiVersion: v1
metadata:
  name: rmq-cluster-secret
  namespace: public-service
stringData:
  cookie: ERLANG_COOKIE
  password: RABBITMQ_PASS
  url: amqp://RABBITMQ_USER:RABBITMQ_PASS@rmq-cluster-balancer
  username: RABBITMQ_USER
type: Opaque

### --- 注：service，反代到另一个rabbitmq节点

  name: rmq-cluster-secret
  namespace: public-service
  cookie: ERLANG_COOKIE
  password: RABBITMQ_PASS
  url: amqp://RABBITMQ_USER:RABBITMQ_PASS@rmq-cluster-balancer  
  username: RABBITMQ_USER

六、创建rabbitmq-secret

### --- 创建secret

[root@k8s-master01 rabbit-cluster]# kubectl create -f rabbitmq-secret.yaml 
secret/rmq-cluster-secret created

七、创建rabbitmq-service.yaml

### --- 编写service的yaml文件；创建的是2个service，可以写在一个配置文件中

[root@k8s-master01 rabbit-cluster]# vim rabbitmq-svc.yaml
kind: Service
apiVersion: v1
metadata:
  labels:
    app: rmq-cluster
  name: rmq-cluster
  namespace: public-service
spec:
  clusterIP: None
  ports:
  - name: amqp
    port: 5672
    targetPort: 5672
  selector:
    app: rmq-cluster
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: rmq-cluster
    type: LoadBalancer
  name: rmq-cluster-balancer
  namespace: public-service
spec:
  ports:
  - name: http
    port: 15672
    protocol: TCP
    targetPort: 15672
  - name: amqp
    port: 5672
    protocol: TCP
    targetPort: 5672
  selector:
    app: rmq-cluster
  type: NodePort        # 可以更改成ClusterIP ingress配置一个域名访问

八、创建rabbitmq-service

### --- 创建service

[root@k8s-master01 rabbit-cluster]# kubectl create -f rabbitmq-svc.yaml 
service/rmq-cluster created
service/rmq-cluster-balancer created

### --- 查看创建的service
~~~     注：创建了2个service
~~~     第一个：无头service，是用于rabbitmq通讯用的
~~~     第二个：rmq-cluster-balancer是客户端连接rabbitmq用的

[root@k8s-master01 rabbit-cluster]# kubectl get svc -n public-service
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                          AGE
rmq-cluster            ClusterIP   None            <none>        5672/TCP                         32m
rmq-cluster-balancer   NodePort    10.101.215.84   <none>        15672:31549/TCP,5672:31578/TCP   32m

九、创建rabbitmq-rbac.yaml

### --- 编写rbac的yaml文件

[root@k8s-master01 rabbit-cluster]# vim rabbitmq-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: rmq-cluster
  namespace: public-service
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: rmq-cluster
  namespace: public-service
rules:
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: rmq-cluster
  namespace: public-service
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: rmq-cluster
subjects:
- kind: ServiceAccount
  name: rmq-cluster
  namespace: public-service

十、创建rabbitmq-rbac

### --- 创建rbac

[root@k8s-master01 rabbit-cluster]# kubectl create -f rabbitmq-rbac.yaml 
serviceaccount/rmq-cluster created          // rbac的名称是：rmq-cluster，后面实验需要用到
role.rbac.authorization.k8s.io/rmq-cluster created
rolebinding.rbac.authorization.k8s.io/rmq-cluster created

十一、创建rmq-cluster-StatefulSet.yaml

### --- 创建rmq-cluster

[root@k8s-master01 rabbit-cluster]# vim rabbitmq-cluster-sts.yaml
kind: StatefulSet
apiVersion: apps/v1
metadata:
  labels:
    app: rmq-cluster
  name: rmq-cluster
  namespace: public-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: rmq-cluster
  serviceName: rmq-cluster
  template:
    metadata:
      labels:
        app: rmq-cluster
    spec:
      containers:
      - args:
        - -c
        - cp -v /etc/rabbitmq/rabbitmq.conf ${RABBITMQ_CONFIG_FILE}; exec docker-entrypoint.sh
          rabbitmq-server
        command:
        - sh
        env:
        - name: RABBITMQ_DEFAULT_USER
          valueFrom:
            secretKeyRef:
              key: username
              name: rmq-cluster-secret
        - name: RABBITMQ_DEFAULT_PASS
          valueFrom:
            secretKeyRef:
              key: password
              name: rmq-cluster-secret
        - name: RABBITMQ_ERLANG_COOKIE
          valueFrom:
            secretKeyRef:
              key: cookie
              name: rmq-cluster-secret
        - name: K8S_SERVICE_NAME
          value: rmq-cluster
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: RABBITMQ_USE_LONGNAME
          value: "true"
        - name: RABBITMQ_NODENAME
          value: rabbit@$(POD_NAME).rmq-cluster.$(POD_NAMESPACE).svc.cluster.local
        - name: RABBITMQ_CONFIG_FILE
          value: /var/lib/rabbitmq/rabbitmq.conf
        image: rabbitmq:3.8.3-management
        imagePullPolicy: IfNotPresent
        livenessProbe:
          exec:
            command:
            - rabbitmqctl
            - status
          initialDelaySeconds: 30
          timeoutSeconds: 10
        name: rabbitmq
        ports:
        - containerPort: 15672
          name: http
          protocol: TCP
        - containerPort: 5672
          name: amqp
          protocol: TCP
        readinessProbe:
          exec:
            command:
            - rabbitmqctl
            - status
          initialDelaySeconds: 10
          timeoutSeconds: 10
        volumeMounts:
        - mountPath: /etc/rabbitmq
          name: config-volume
          readOnly: false
        - mountPath: /var/lib/rabbitmq
          name: rabbitmq-storage
          readOnly: false
      serviceAccountName: rmq-cluster
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          items:
          - key: rabbitmq.conf
            path: rabbitmq.conf
          - key: enabled_plugins
            path: enabled_plugins
          name: rmq-cluster-config
        name: config-volume
      - name: rabbitmq-storage
        emptyDir: {}

### --- 注：

kind: StatefulSet
apiVersion: apps/v1                             // apps/v1betal已经不用了 ，更改为apps/v1
  replicas: 3                                   // 副本数为3
        - cp -v /etc/rabbitmq/rabbitmq.conf ${RABBITMQ_CONFIG_FILE}; exec docker-entrypoint.sh    // 配置是通过configmap挂载进来的，拷贝到/var/lib/rabbitmq/rabbitmq.conf这个路径下
        - name: RABBITMQ_DEFAULT_USER           // 配置了账号密码
              key: cookie                       // cookie配置
              name: rmq-cluster-secret
        - name: K8S_SERVICE_NAME                // service name
          value: /var/lib/rabbitmq/rabbitmq.con // 前面定义的变量；cp -v定义的变量
              name: rmq-cluster-secret          // rmq-cluster-secret和secret名称保持一致
      serviceAccountName: rmq-cluster           // 就是刚才创建的service，它是有集群的查看权限的。可以查看到当前有多少个RabbitMQ实例，可以创建成集群
        name: config-volume
       -name：rabbitmq-storage
        emptyDir: {}                            // 若需要持久化数据的情况下，可以更改为后端存储即可

十二、创建rmq-cluster-StatefulSet

### --- 查看rmq-cluster-secret.yaml    

[root@k8s-master01 ~]# kubectl get secret -n public-service rmq-cluster-secret -oyaml
apiVersion: v1
data:
  cookie: RVJMQU5HX0NPT0tJRQ==
  password: UkFCQklUTVFfUEFTUw==        // 密码
  url: YW1xcDovL1JBQkJJVE1RX1VTRVI6UkFCQklUTVFfUEFTU0BybXEtY2x1c3Rlci1iYWxhbmNlcg==
  username: UkFCQklUTVFfVVNFUg==        // 账号
    manager: kubectl-create             // 名称

### --- 创建rmq-cluster-StatefulSet

[root@k8s-master01 rabbit-cluster]# kubectl create -f rabbitmq-cluster-sts.yaml 
statefulset.apps/rmq-cluster created

十三、查看创建的rmq-cluster资源

### --- 查看创建的rmq-cluster

[root@k8s-master01 rabbit-cluster]# kubectl get po -n public-service -owide
NAME            READY   STATUS    RESTARTS   AGE    IP             NODE           NOMINATED NODE   READINESS GATES
rmq-cluster-0   1/1     Running   0          102s   172.25.92.70   k8s-master01   <none>           <none>
rmq-cluster-1   1/1     Running   0          71s    172.25.92.75   k8s-master02   <none>           <none>
rmq-cluster-2   0/1     Running   0          27s    172.25.92.74   k8s-master03   <none>           <none>

### --- 查看rmq-cluster的日志信息

[root@k8s-master01 rabbit-cluster]# kubectl logs -f rmq-cluster-0 -n public-service
 * rabbitmq_management
 * rabbitmq_web_dispatch
 * rabbitmq_peer_discovery_k8s
 * rabbitmq_management_agent
 * rabbitmq_peer_discovery_common
 completed with 5 plugins.

### --- 查看创建的容器，执行操作

[root@k8s-master01 rabbit-cluster]# kubectl exec -ti rmq-cluster-0 -n public-service -- bash
root@rmq-cluster-0:/# env | grep DEFAULT    // 查看配置的变量
RABBITMQ_DEFAULT_PASS=RABBITMQ_PASS         // 密码
RABBITMQ_DEFAULT_USER=RABBITMQ_USER         // 账号

### --- 查看它的service

[root@k8s-master01 rabbit-cluster]# kubectl get svc -n public-service
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                          AGE
rmq-cluster            ClusterIP   None            <none>        5672/TCP                         47h
rmq-cluster-balancer   NodePort    10.101.215.84   <none>        15672:31549/TCP,5672:31578/TCP   47h

### --- 查看service对应的ep

[root@k8s-master01 rabbit-cluster]# kubectl get ep -n public-service
NAME                   ENDPOINTS                                                AGE
rmq-cluster            172.25.92.70:5672                                        2d1h
rmq-cluster-balancer   172.25.92.70:15672,172.25.92.75:15672172.25.92.70:5672   2d1h

---

 

 

 

 

 

 

 

 

 

---

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart

                                                                                                                                                   ——W.S.Landor

---