k8s 部署 (二) EMQ X 集群
本文部署的 EMQ X Broker 版本为 4.3.1
一、RBAC 鉴权
集群需要使用到 Kubernetes 的 API Server，但是普通 Pod 是没有权限访问的，需要授权：
---
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: default
name: emqx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: emqx
namespace: default
rules:
- apiGroups:
- ''
resources:
- endpoints
verbs:
- get
- watch
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: emqx
namespace: default
roleRef:
kind: Role
name: emqx
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: emqx
namespace: default
如果没有授权，会有如下报错：
Ekka(AutoCluster): Discovery error: {403,"{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"endpoints "emqx-headless" is forbidden: User "system:serviceaccount:default:default" cannot get resource "endpoints" in API group "" in the namespace "default"","reason":"Forbidden","details":{"name":"emqx-headless","kind":"endpoints"},"code":403}"}
二、新增 ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
name: emqx-cm
data:
NAME: "emqx"
CLUSTER__DISCOVERY: "k8s"
CLUSTER__K8S__ADDRESS_TYPE: "ip"
CLUSTER__K8S__APISERVER: "https://IP:PORT"
CLUSTER__K8S__NAMESPACE: "default"
CLUSTER__K8S__SERVICE_NAME: "emqx-headless"
CLUSTER__K8S__APP_NAME: "emqx"
默认情况下 EMQ X 使用带有 EMQX 的前缀的环境变量来覆盖配置文件中的配置项环境变量名称到配置文件键值名称映射规则如下:将 EMQX 前缀移除；大写字符替换成小写；双下划线 __ 替换成点 . 详见：使用环境变量修改配置
- cluster.kubernetes.apiserver 为 kubernetes apiserver 的地址，可以通过 kubectl cluster-info 命令获取
- cluster.kubernetes.service_name 为 Service 的 name
- cluster.kubernetes.app_name 为 EMQ X Broker 的 node.name 中 @ 符号之前的部分，需要同时将集群中 EMQ X Broker 设置为统一的 node.name 的前缀
三、新增 Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: emqx
name: emqx
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: emqx
template:
metadata:
labels:
app: emqx
spec:
serviceAccountName: emqx
containers:
- envFrom:
- prefix: EMQX_
configMapRef:
name: emqx-cm
image: emqx/emqx:4.3.1
imagePullPolicy: IfNotPresent
livenessProbe:
exec:
command:
- emqx_ctl
- status
failureThreshold: 3
initialDelaySeconds: 60
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: emqx
ports:
- name: mqtt
protocol: TCP
containerPort: 1883
- name: mqttssl
protocol: TCP
containerPort: 8883
- name: mgmt
protocol: TCP
containerPort: 8081
- name: websocket
protocol: TCP
containerPort: 8083
- name: wss
protocol: TCP
containerPort: 8084
- name: dashboard
protocol: TCP
containerPort: 18083
resources:
limits:
cpu: '2'
memory: 2Gi
requests:
cpu: '1'
memory: 2Gi
volumeMounts:
- mountPath: /etc/localtime
name: volume-localtime
volumes:
- hostPath:
path: /etc/localtime
type: ''
name: volume-localtime
- 1883 MQTT 协议端口
- 8883 MQTT/SSL 端口
- 8083 MQTT/WebSocket 端口
- 8084 MQTT/WebSocket/SSL 端口
- 8081 管理 API 端口
- 18083 Dashboard 端口
四、新增 Service
apiVersion: v1
kind: Service
metadata:
name: emqx-headless
labels:
app: emqx-headless
spec:
type: ClusterIP
clusterIP: None
ports:
- name: mqtt
port: 1883
protocol: TCP
targetPort: 1883
- name: mqttssl
port: 8883
protocol: TCP
targetPort: 8883
- name: mgmt
port: 8081
protocol: TCP
targetPort: 8081
- name: websocket
port: 8083
protocol: TCP
targetPort: 8083
- name: wss
port: 8084
protocol: TCP
targetPort: 8084
- name: dashboard
port: 18083
protocol: TCP
targetPort: 18083
selector:
app: emqx
五、放行 TCP 端口
见：阿里云 k8s 部署 Spring Cloud Alibaba 微服务实践 (五) Kubernetes TCP Ingress
参考：
