helm 安装 EFK ES Xpack认证

资源清单

本文安装 EFK 依赖 K8S集群 和 helm ，本文不提供 K8S集群 和 helm 安装方式

使用此文档部署，需要自行解决 storageClass 问题（ NFS, ceph, openebs等）

软件	版本
chart	7.16.3
elasticsearch	7.16.3
filebeat	7.16.3
kibana	7.16.3
kubernetes	v1.19.3
helm	v3.8.1

`helm` 安装 `efk 集群`

1. 添加 `elastic` 的仓库

$ helm repo add elastic https://helm.elastic.co

2. 查询 `efk` 资源

$ helm repo update
$ helm search repo elasticsearch
NAME                                	CHART VERSION	APP VERSION	DESCRIPTION                                       
elastic/elasticsearch               	7.17.3       	7.17.3     	Official Elastic helm chart for Elasticsearch                    

$ helm search repo filebeat
NAME            	CHART VERSION	APP VERSION	DESCRIPTION                             
elastic/filebeat	7.17.3       	7.17.3     	Official Elastic helm chart for Filebeat

$ helm search repo kibana
NAME                     	CHART VERSION	APP VERSION	DESCRIPTION                                       
elastic/kibana           	7.17.3       	7.17.3     	Official Elastic helm chart for Kibana

`helm` 安装 `elasticsearch 集群` 启用 Xpack 认证

1. 创建集群证书

$ mkdir -p /root/elk/es/certs && cd /root/elk/es/certs

# 运行容器生成证书
docker run --name elastic-charts-certs -i -w /app elasticsearch:7.16.3 /bin/sh -c  \
  "elasticsearch-certutil ca --out /app/elastic-stack-ca.p12 --pass '' && \
    elasticsearch-certutil cert --ca /app/elastic-stack-ca.p12 --pass '' --ca-pass '' --out /app/elastic-certificates.p12"

# 从容器中将生成的证书拷贝出来
docker cp elastic-charts-certs:/app/elastic-certificates.p12 ./ 

# 删除容器
docker rm -f elastic-charts-certs

# 将 pcks12 中的信息分离出来，写入文件
openssl pkcs12 -nodes -passin pass:'' -in elastic-certificates.p12 -out elastic-certificate.pem

2. 添加证书到集群

$ cd /root/elk/es/certs

# 创建 test-middleware 名称空间
$ kubectl create ns test-middleware

# 添加证书
$ kubectl -n test-middleware create secret generic elastic-certificates --from-file=elastic-certificates.p12

$ kubectl -n test-middleware create secret generic elastic-certificate-pem --from-file=elastic-certificate.pem

# 设置集群用户名密码，用户名不建议修改 ( es 和 kibana 都会使用此账户密码信息 )
$ kubectl -n test-middleware create secret generic elastic-credentials \
  --from-literal=username=elastic --from-literal=password=admin@123
 
 # 查看创建的 secret
$ kubectl -n test-middleware get secret
NAME                                          TYPE                                  DATA   AGE
elastic-certificate-pem                       Opaque                                1      11m
elastic-certificates                          Opaque                                1      11m
elastic-credentials                           Opaque                                2      11m

3. 拉取 `elasticsearch chart` 到本地

$ cd /root/elk/es/

# 拉取 chart 到本地 /root/elk/es 目录
$ helm pull elastic/elasticsearch --version 7.16.3

$ tar -zxvf elasticsearch-7.16.3.tgz
$ cp elasticsearch/values.yaml ./values-test.yaml

# 查看当前目录层级
$ tree -L 2
.
├── certs
│   ├── elastic-certificate.pem
│   └── elastic-certificates.p12
├── elasticsearch
│   ├── Chart.yaml
│   ├── examples
│   ├── Makefile
│   ├── README.md
│   ├── templates
│   └── values.yaml
├── elasticsearch-7.16.3.tgz
└── values-test.yaml

4. 对本地 `values-test.yaml` 修改

查看集群 storageclasses

$ kubectl get storageclasses.storage.k8s.io 
NAME                   PROVISIONER           RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
openebs-jiva-default   jiva.csi.openebs.io   Delete          Immediate              true                   33d

修改配置

$ cat values-test.yaml 

## 配置文件中定义 storageClassName: ""，会使用集群配置的 openebs 提供的 storageClass，
## 使用此文档部署，需要自行解决 storageClass 问题 (ceph, nfs, 公有云提供的 nfs)

---
clusterName: "elasticsearch"
nodeGroup: "master"


roles:
  master: "true"
  ingest: "true"
  data: "true"
  remote_cluster_client: "true"
  ml: "true"


# 定义集群数量
replicas: 3
minimumMasterNodes: 2


# 启用 ES 的 xpack 认证，认证文件为上述定义 elastic-certificate secret
esConfig:
  elasticsearch.yml: |
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.transport.ssl.verification_mode: certificate
    xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 


# 定义 ES 集群认证的账户密码信息，引用上述定义的 elastic-credentials secret
extraEnvs:
  - name: ELASTIC_USERNAME
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
  - name: ELASTIC_PASSWORD
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password


# 挂载上述定义 elastic-certificate secret 到 POD，xpack 定义使用
secretMounts:
  - name: elastic-certificates
    secretName: elastic-certificates
    path: /usr/share/elasticsearch/config/certs
    defaultMode: 0755


image: "elasticsearch"
imageTag: "7.16.3"
imagePullPolicy: "IfNotPresent"


esJavaOpts: "" # example: "-Xmx1g -Xms1g"


volumeClaimTemplate:
  # 定义 storageClass 使用的类型
  storageClassName: "openebs-jiva-default"
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 30Gi


# 启用持久化存储
persistence:
  enabled: true
  labels:
    # Add default labels for the volumeClaimTemplate of the StatefulSet
    enabled: false
  annotations: {}


service:
  enabled: true
  labels: {}
  labelsHeadless: {}
  type: ClusterIP
  nodePort: ""
  annotations: {}
  httpPortName: http
  transportPortName: transport
  loadBalancerIP: ""
  loadBalancerSourceRanges: []
  externalTrafficPolicy: ""


nodeSelector: {}


ingress:
  enabled: false
  annotations: {}
  # kubernetes.io/ingress.class: nginx
  # kubernetes.io/tls-acme: "true"
  className: "nginx"
  pathtype: ImplementationSpecific
  hosts:
    - host: chart-example.local
      paths:
        - path: /
  tls: []

5. 安装 `elasticsearch 集群`

# 安装 elasticsearch 集群
$ helm -n test-middleware install elasticsearch-cluster elasticsearch -f values-test.yaml

## helm -n NAMESAPCE install SERVER_NAME FILE_NAME -f CONFIG_FILE
-n 指定 kubernetes 集群名称空间
-f 指定使用的配置文件，文件中定义的配置可以覆盖 elasticsearch/values.yaml 文件中配置


NAME: elasticsearch-cluster
LAST DEPLOYED: Thu Jun  2 22:33:31 2022
NAMESPACE: test-middleware
STATUS: deployed
REVISION: 1
NOTES:
1. Watch all cluster members come up.
  $ kubectl get pods --namespace=test-middleware -l app=elasticsearch-master -w
2. Test cluster health using Helm test.
  $ helm --namespace=test-middleware test elasticsearch-cluster

6. 查看部署的 `elasticsearch` 集群

$ helm -n test-middleware list
NAME                 	NAMESPACE      	REVISION	UPDATED                                	STATUS  	CHART               	APP VERSION
elasticsearch-cluster	test-middleware	1       	2022-06-02 22:57:55.112229514 -0400 EDT	deployed	elasticsearch-7.16.3	7.16.3

$ kubectl get pods --namespace=test-middleware -l app=elasticsearch-master
NAME                     READY   STATUS    RESTARTS   AGE
elasticsearch-master-0   1/1     Running   0          8m57s
elasticsearch-master-1   1/1     Running   0          8m35s
elasticsearch-master-2   1/1     Running   0          8m35s

查看服务使用的 storageclass

# 查看 pvc
$ kubectl -n test-middleware get pvc
NAME                                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
elasticsearch-master-elasticsearch-master-0   Bound    pvc-778c6765-59cf-4b70-9029-29d216fb4648   30Gi       RWO            openebs-jiva-default   11m
elasticsearch-master-elasticsearch-master-1   Bound    pvc-40e32e2a-049a-4b79-bfb8-be75dd00f517   30Gi       RWO            openebs-jiva-default   11m
elasticsearch-master-elasticsearch-master-2   Bound    pvc-a71bd761-66bb-482a-abfd-93fd6164088a   30Gi       RWO            openebs-jiva-default   11m

# 查看 pv
$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                                 STORAGECLASS           REASON   AGE
pvc-40e32e2a-049a-4b79-bfb8-be75dd00f517   30Gi       RWO            Delete           Bound    test-middleware/elasticsearch-master-elasticsearch-master-1           openebs-jiva-default            11m
pvc-778c6765-59cf-4b70-9029-29d216fb4648   30Gi       RWO            Delete           Bound    test-middleware/elasticsearch-master-elasticsearch-master-0           openebs-jiva-default            11m
pvc-a71bd761-66bb-482a-abfd-93fd6164088a   30Gi       RWO            Delete           Bound    test-middleware/elasticsearch-master-elasticsearch-master-2           openebs-jiva-default            11m

7. 连接 `elasticsearch 集群` 验证服务

$ kubectl -n test-middleware exec -it elasticsearch-master-0 -- bash

elasticsearch@elasticsearch-master-0:~$ curl -u elastic:admin@123 localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

elasticsearch@elasticsearch-master-0:~$ curl -u elastic:admin@123 localhost:9200/_cat/nodes            
10.244.3.131 17 65  7 0.13 0.28 0.48 cdfhilmrstw - elasticsearch-master-0
10.244.2.50  53 69 10 0.04 0.18 0.35 cdfhilmrstw - elasticsearch-master-1
10.244.1.71  59 71  9 0.00 0.20 0.49 cdfhilmrstw * elasticsearch-master-2

`helm` 安装 `filebeat`

1. 拉取 `filebeat chart` 到本地

$ mkdir -p /root/elk/filebeat/ && cd /root/elk/filebeat/

# 拉取 chart 到本地 /root/elk/filebeat 目录
$ helm pull elastic/filebeat --version 7.16.3

$ tar -zxvf filebeat-7.16.3.tgz
$ cp filebeat/values.yaml values-test.yaml

# 查看当前目录层级
$ tree -L 2
.
├── filebeat
│   ├── Chart.yaml
│   ├── examples
│   ├── Makefile
│   ├── README.md
│   ├── templates
│   └── values.yaml
├── filebeat-7.16.3.tgz
└── values-test.yaml

2. 对本地 `values-test.yaml` 修改

修改配置

$ cat values-test.yaml 

---
daemonset:
  enabled: false


deployment:
  # Include the daemonset
  enabled: true
  filebeatConfig:
    filebeat.yml: |
      filebeat.inputs:
      - type: container
        paths:
          - /var/log/containers/*.log
        processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"

      output.elasticsearch:
        host: '${NODE_NAME}'
        hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'

  nodeSelector: {}

  resources:
    requests:
      cpu: "100m"
      memory: "100Mi"
    limits:
      cpu: "1000m"
      memory: "200Mi"


image: "docker.elastic.co/beats/filebeat"
imageTag: "7.16.3"
imagePullPolicy: "IfNotPresent"
imagePullSecrets: []

replicas: 1

3. 安装 `filebeat`

# 安装 filebeat
$ helm -n test-middleware install filebeat filebeat -f values-test.yaml

## helm -n NAMESAPCE install SERVER_NAME FILE_NAME -f CONFIG_FILE
-n 指定 kubernetes 集群名称空间
-f 指定使用的配置文件，文件中定义的配置可以覆盖 filebeat/values.yaml 文件中配置


NAME: filebeat
LAST DEPLOYED: Fri Jun  3 01:50:52 2022
NAMESPACE: test-middleware
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Watch all containers come up.
  $ kubectl get pods --namespace=test-middleware -l app=filebeat-filebeat -w

4. 查看部署的 `filebeat`

$ helm -n test-middleware list
NAME                 	NAMESPACE      	REVISION	UPDATED                                	STATUS  	CHART               	APP VERSION
filebeat             	test-middleware	1       	2022-06-03 07:20:19.380955035 -0400 EDT	deployed	filebeat-7.16.3     	7.16.3 

$ kubectl get pods --namespace=test-middleware -l app=filebeat-filebeat
NAME                                 READY   STATUS             RESTARTS   AGE
filebeat-filebeat-5b67d8479b-zlprk   1/1     Running            0          4m58s

`helm` 安装 `kibana`

1. 拉取 `kibana chart` 到本地

$ mkdir -p /root/elk/kibana/ && cd /root/elk/kibana/

# 拉取 chart 到本地 /root/elk/kibana 目录
$ helm pull elastic/kibana --version 7.16.3

$ tar -zxvf kibana-7.16.3.tgz
$ cp kibana/values.yaml values-test.yaml

# 查看当前目录层级
$ tree -L 2
.
├── kibana
│   ├── Chart.yaml
│   ├── examples
│   ├── Makefile
│   ├── README.md
│   ├── templates
│   └── values.yaml
├── kibana-7.16.3.tgz
└── values-test.yaml

2. 对本地 `values-test.yaml` 修改

修改配置

$ cat values-test.yaml 

---
elasticsearchHosts: "http://elasticsearch-master:9200"


replicas: 1


extraEnvs:
  - name: ELASTICSEARCH_USERNAME
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: username
  - name: ELASTICSEARCH_PASSWORD
    valueFrom:
      secretKeyRef:
        name: elastic-credentials
        key: password
  - name: "NODE_OPTIONS"
    value: "--max-old-space-size=1800"


image: "kibana"
imageTag: "7.16.3"
imagePullPolicy: "IfNotPresent"
nodeSelector: {}


kibanaConfig:
  kibana.yml: |
    i18n.locale: "zh-CN"


service:
  type: ClusterIP
  loadBalancerIP: ""
  port: 5601
  nodePort: ""
  labels: {}
  annotations:
    {}
    # cloud.google.com/load-balancer-type: "Internal"
    # service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
    # service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    # service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
    # service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
  loadBalancerSourceRanges:
    []
    # 0.0.0.0/0
  httpPortName: http


ingress:
  enabled: true
  className: "nginx"
  pathtype: ImplementationSpecific
  annotations: {}
  # kubernetes.io/ingress.class: nginx
  # kubernetes.io/tls-acme: "true"
  hosts:
    - host: kibana.evescn.com
      paths:
        - path: /

3. 安装 `kibana`

# 安装 kibana
$ helm -n test-middleware install kibana kibana -f values-test.yaml

## helm -n NAMESAPCE install SERVER_NAME FILE_NAME -f CONFIG_FILE
-n 指定 kubernetes 集群名称空间
-f 指定使用的配置文件，文件中定义的配置可以覆盖 kibana/values.yaml 文件中配置


NAME: kibana
LAST DEPLOYED: Fri Jun  3 02:21:20 2022
NAMESPACE: test-middleware
STATUS: deployed
REVISION: 1
TEST SUITE: None

4. 查看部署的 `kibana`

$ helm -n test-middleware list
NAME                 	NAMESPACE      	REVISION	UPDATED                                	STATUS  	CHART               	APP VERSION
kibana               	test-middleware	1       	2022-06-03 02:21:20.846637173 -0400 EDT	deployed	kibana-7.16.3       	7.16.3 

$ kubectl get pod --namespace=test-middleware -l app=kibana
NAME                               READY   STATUS              RESTARTS   AGE
kibana-kibana-d4fcd8979-nldsw      1/1     Running             0          2m15s

参考文档

https://bbs.huaweicloud.com/blogs/303085
https://www.cnblogs.com/aresxin/p/helm-es6.html

https://artifacthub.io/packages/helm/elastic/elasticsearch/7.16.3
https://artifacthub.io/packages/helm/elastic/filebeat/7.16.3
https://artifacthub.io/packages/helm/elastic/kibana/7.16.3

posted @ 2022-06-03 21:06 evescn 阅读(821) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

evescn

helm 安装 EFK ES Xpack认证

资源清单

helm 安装 efk 集群

1. 添加 elastic 的仓库

2. 查询 efk 资源

helm 安装 elasticsearch 集群 启用 Xpack 认证

1. 创建集群证书

2. 添加证书到集群

3. 拉取 elasticsearch chart 到本地

4. 对本地 values-test.yaml 修改

5. 安装 elasticsearch 集群

6. 查看部署的 elasticsearch 集群

7. 连接 elasticsearch 集群 验证服务

helm 安装 filebeat

1. 拉取 filebeat chart 到本地

2. 对本地 values-test.yaml 修改

3. 安装 filebeat

4. 查看部署的 filebeat

helm 安装 kibana

1. 拉取 kibana chart 到本地

2. 对本地 values-test.yaml 修改

3. 安装 kibana

4. 查看部署的 kibana

参考文档

公告

`helm` 安装 `efk 集群`

1. 添加 `elastic` 的仓库

2. 查询 `efk` 资源

`helm` 安装 `elasticsearch 集群` 启用 Xpack 认证

3. 拉取 `elasticsearch chart` 到本地

4. 对本地 `values-test.yaml` 修改

5. 安装 `elasticsearch 集群`

6. 查看部署的 `elasticsearch` 集群

7. 连接 `elasticsearch 集群` 验证服务

`helm` 安装 `filebeat`

1. 拉取 `filebeat chart` 到本地

2. 对本地 `values-test.yaml` 修改

3. 安装 `filebeat`

4. 查看部署的 `filebeat`

`helm` 安装 `kibana`

1. 拉取 `kibana chart` 到本地

2. 对本地 `values-test.yaml` 修改

3. 安装 `kibana`

4. 查看部署的 `kibana`