kubernetes部署elasticsearch-6.6.2

事先环境准备：
1.k8s环境
2.集群存储，本文使用的是ceph

 

以下用到的配置文件：
es_configmap.yaml
es_ing.yaml
es_statefulset.yaml
es_svc.yaml

 

1. 配置文件 es_svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: hotes-cluster-9300
spec:
  clusterIP: None
  selector:
    app: es-cluster
  ports:
    - port: 9300
      name: inner

---
apiVersion: v1
kind: Service
metadata:
  name: hotes-cluster-9200
spec:
  selector:
    app: es-cluster
  ports:
    - name: http
      port: 9200
      targetPort: 9200
      nodePort: 8831
  type: NodePort

 

把hotes-cluster-9300的clusterIP设置为:None,被称作headless service（可以参考https://www.jianshu.com/p/a6d8b28c88a2）
hotes-cluster-9200设置了9200，用于外部访问es

2. 配置文件 es_ing.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hotes-cluster
spec:
  rules:
    - host: www.hotes.com
      http:
        paths:
          - backend:
              serviceName: hotes-cluster-9200
              servicePort: 9200
            path: /

 

设置一个域名，用于外部访问es

3. 配置文件 es_configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: hotes-config
data:
  elasticsearch.yml: |
    node.name: ${HOSTNAME}  
    cluster.name: hotes-backup 
    network.host: "0.0.0.0"
    bootstrap.memory_lock: false
    discovery.zen.ping.unicast.hosts: esnode-0.hotes-cluster-9300.default.svc.cluster.local.,esnode-1.hotes-cluster-9300.default.svc.cluster.local.,esnode-2.hotes-cluster-9300.default.svc.cluster.local.
    discovery.zen.minimum_master_nodes: 1

node.name 节点名

cluster.name 集群名字
discovery.zen.ping.unicast.hosts 集群各节点的名字，像这样使用headless service方式，解析出来的IP进行通信，不需要走kube-proxy，默认走的是9300端口，如果输入的是主机名被解析成了多个地址，节点将会尝试连接所有地址，同时只选择其中一个可用的
pod的dns规则：{stateful-set-name}-{0…N}.{service-name}.{namespace}.svc.cluster.local.（注意后面还有一个.）
这里配置的service-name是headless service name

discovery.zen.minimum_master_nodes 为了防止过多的数据丢失，配置这个参数很重要。

如果没有这个配置，当集群遇到网络问题时，有可能会分离成两个独立的集群（脑裂现象），这样会导致集群节点丢失，从而丢失数据，因为即使网络好了以后，已经分成的两个集群并不会再合并，为避免脑裂现象，需要设置master候选节点的数量：（master_eligible_nodes/2）+1

 

 

4. 配置文件 es_statefulset.yaml

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: esnode
  labels:
    app: es-cluster
spec:
  serviceName: hotes-cluster-9300
  replicas: 3
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: es-cluster
    spec:
      #securityContext:
      #  fsGroup: 1000
      initContainers:
        - name: fix-permissions
          image: video-harbor.ks-live.com/public/busybox:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
          volumeMounts:
            - name: es-data
              mountPath: /usr/share/elasticsearch/data
        - name: init-ulimit
          image: xxx.com/public/busybox:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          command: ["sh", "-c", "ulimit -n 655350"]          
        - name: init-sysctl
          image: xxx.com/public/busybox:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
      nodeSelector: 
        zone: "xxx-indexer-yuanzhan_json"
      containers:
        - name: elasticsearch
          resources:
            requests:
              memory: 300Mi
              cpu: 0.01
            limits:
              memory: 60.5Gi
              cpu: 15
          securityContext:
            privileged: true
            runAsUser: 0
            capabilities:
              add:
                - IPC_LOCK
                - SYS_RESOURCE
          image: xxx.com/st/elasticsearch-oss:6.6.2
          imagePullPolicy: IfNotPresent
          env:
            - name: ES_JAVA_OPTS
              value: "-Xms9800m -Xmx9800m"
            - name: HOSTNAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
          #command: ["/bin/sleep","9000"]
          readinessProbe:
            httpGet:
              scheme: HTTP
              path: /_cluster/health?local=true
              port: 9200
            initialDelaySeconds: 5
          ports:
            - containerPort: 9200
              name: es-http
            - containerPort: 9300
              name: es-transport
          volumeMounts:
            - name: es-data
              mountPath: /usr/share/elasticsearch/data
            - name: elasticsearch-config
              mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
              subPath: elasticsearch.yml
      volumes:
        - name: elasticsearch-config
          configMap:
            name: hotes-config
            items:
              - key: elasticsearch.yml
                path: elasticsearch.yml
  volumeClaimTemplates:
    - metadata:
        name: es-data
      spec:
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 1Gi
        storageClassName: rbd

使用elasticsearch-oss:6.6.2的镜像，镜像内elasticsearch用户的id是1000，将configmap挂载到 /usr/share/elasticsearch/config/elasticsearch.yml，并赋给elasticsearch权限，使用ceph搭建的分布式存储，动态创建1G的空间用于存储数据

 

5. 使用域名 http://www.hotes.com/_cluster/state/nodes?pretty 查看集群状态

或者通过nodePort的方式 http://host-ipaddress:8831/_cluster/state/nodes?pretty 查看集群状态

 

参考：https://github.com/13428282016/elasticsearch-CN/wiki/es-setup--elasticsearch