rocky Elasticsearch 8.7.1集群 x-spack 安全验证 及 集群内部TLS加密传输 （ca）

目录

• 简介
• 环境准备
• 安装
  • 配置 hostname 解析
  • 安装
  • systemd 脚本
• ca证书
• 配置
• 给所有 ES 配置相同的用户密码
• 启动查看

 

简介

常规部署 Elasticsearch 集群时，不管是集群之间的数据传输，或者是 Client 访问Elasticsearch 集群时 均不需要相关验证，可通过对外提供的http接口，直接访问到ES的内部数据

这情况下，相对来说安全度没有保障，那么本次部署一套 基于 x-spack 安全验证的安全认证

其实不光是 对外提供服务的 9200 端口需要验证，集群内服务端口 9300 之间数据通信，也需要安全机制，本次使用自签ca证书，用于集群内部加密通信

说明： x-spack 组件是收费的，但好的是基础安全验证是其中的免费的，不用担心商用问题；

环境准备

 

系统版本	主机名	IP	ES 版本	ES 用户端口	ES 集群端口
Rocky Linux release 9.2 (Blue Onyx)	es01	192.168.8.114	8.7.1	9200	9300
Rocky Linux release 9.2 (Blue Onyx)	es02	192.168.8.115	8.7.1	9200	9300
Rocky Linux release 9.2 (Blue Onyx)	es03	192.168.8.116	8.7.1	9200	9300

安装

现在下载的 elasticsearch 安装包中，自带 jdk ，无需像以前老版本一样，还要需要安装jdk环境，方便很多。

官方下载地址：https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.7.0-linux-x86_64.tar.gz

需要优化文件描述符

useradd es

cat >> /etc/security/limits.conf <<EOF
*   hard    nofile  65536
*   soft    nofile  65536
*   hard    nproc   5000
*   soft    nproc   5000
es soft memlock unlimited
es hard memlock unlimited

EOF

echo 'vm.max_map_count=262144' >>  /etc/sysctl.conf
sysctl -p

配置 hostname 解析

所有节点配置好 hostname 解析

cat >> /etc/hosts <<EOF

192.168.8.114 es01
192.168.8.115 es02
192.168.8.116 es03
EOF

安装

useradd -s /sbin/nologin -M es
cd /opt/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.7.0-linux-x86_64.tar.gz
tar xf elasticsearch-8.7.1-linux-x86_64.tar.gz
ls -al /usr/local/elasticsearch-8.7.1 

生成ca证书

证书签发在es02上操作即可

通过查看官网集群证书的创建方式分为两种：

1. 通过 elasticsearch-certutil 命令逐一创建证书
2. 使用 elasticsearch-certutil 的 Silent Mode 创建

这里使用简约的 Silent Mode 创建；

进入到 ES 的目录：

cd /usr/local/elasticsearch-8.7.1
elasticsearch-8.7.1

创建证书所需的 instances.yml 文件，具体格式请查看官网：官网集群证书

cat >config/certs/instances.yml<<EOF
instances:
  - name: "es01" 
    ip: 
      - "192.168.8.114"
  - name: "es02"
    ip:
      - "192.168.8.115"
  - name: "es03"
    ip:
      - "192.168.8.116"
EOF

注解: name 为实例名

然后执行

    bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip

   unzip config/certs/ca.zip -d config/certs;

bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;

 unzip config/certs/certs.zip -d config/certs;

然后把对应的 目录 拷贝到对应的服务器，并做如下操作：

mv node02 config/certs

[es@es02 elasticsearch]# rsync -avz es01 es@192.168.8.114:/usr/local/elasticsearch/config/certs
[es@es02 elasticsearch]# rsync -avz es01 es@192.168.8.116:/usr/local/elasticsearch/config/certs

配置

es02 配置:

cat > /usr/local/elasticsearch/config/elasticsearch.yml <EOFnode.name: es02
cluster.name: YnGames
network.host: 192.168.8.115
http.host: 0.0.0.0
transport.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["192.168.8.114", "192.168.8.115", "192.168.8.116"]
discovery.seed_hosts: ["192.168.8.114", "192.168.8.116"]
bootstrap.memory_lock: true
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/es02/es02.key
xpack.security.http.ssl.certificate: certs/es02/es02.crt
xpack.security.http.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/es02/es02.key
xpack.security.transport.ssl.certificate: certs/es02/es02.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.verification_mode: certificate
xpack.license.self_generated.type: basic
http.cors.enabled: true
http.cors.allow-origin: "*"
EOF

es03 配置：

cat > /usr/local/elasticsearch/config/elasticsearch.yml <EOFnode.name: es03
network.host: 192.168.8.116
cluster.name: YnGames
http.host: 192.168.8.116
transport.host: 192.168.8.116
http.port: 9200
cluster.initial_master_nodes: ["192.168.8.114", "192.168.8.115", "192.168.8.116"]
discovery.seed_hosts: ["192.168.8.114", "192.168.8.115"]
bootstrap.memory_lock: true
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/es03/es03.key
xpack.security.http.ssl.certificate: certs/es03/es03.crt
xpack.security.http.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/es03/es03.key
xpack.security.transport.ssl.certificate: certs/es03/es03.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.verification_mode: certificate
xpack.license.self_generated.type: basic
http.cors.enabled: true
http.cors.allow-origin: "*"
EOF

es01 配置：

cat > /usr/local/elasticsearch/config/elasticsearch.yml <EOFnode.name: es01
network.host: 192.168.8.114
cluster.name: YnGames
http.host: 0.0.0.0
transport.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["192.168.8.114", "192.168.8.115", "192.168.8.116"]
discovery.seed_hosts: ["192.168.8.115", "192.168.8.116"]
bootstrap.memory_lock: true
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/es01/es01.key
xpack.security.http.ssl.certificate: certs/es01/es01.crt
xpack.security.http.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/es01/es01.key
xpack.security.transport.ssl.certificate: certs/es01/es01.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca/ca.crt
xpack.security.transport.ssl.verification_mode: certificate
xpack.license.self_generated.type: basic
http.cors.enabled: true
http.cors.allow-origin: "*"
EOF

给所有 ES 配置相同的用户密码，

给所有 ES 配置相同的用户密码,注意：所有节点都要执行

使用命令: ./bin/elasticsearch-users useradd username -p password -r superuser

-r 表示角色，superuser 是超级用户

./bin/elasticsearch-users useradd test -p password123 -r superuser

./elasticsearch-reset-password -u user   重置密码

 

添加了用户，并需要给这个用户添加角色不然会报错

角色授权

bin/elasticsearch-users roles -a superuser test

bin/elasticsearch-users roles -a kibana_system test  //这里一定要授权kibana_system ，不然kibana无法登录，即使有超级权限也不行

启动查看

启动所有节点的 elasticsearch ；

./bin/elasticsearch -d

[root@localhost elasticsearch-8.7.1]# curl -uadmin:viu@1234 https://192.168.8.115:9200/_cat/nodes?v -k  
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.8.114 28 67 0 0.00 0.01 0.00 cdfhilmrstw - es01
192.168.8.115 22 90 3 0.02 0.02 0.00 cdfhilmrstw * es02
192.168.8.116 24 67 1 0.11 0.04 0.01 cdfhilmrstw - es03

kibana配置

server.port: 5601
server.host: "192.168.8.115"
server.maxPayload: 1048576
server.name: "kibana"
server.ssl.enabled: false
elasticsearch.hosts: ["https://192.168.8.115:9200"，"https://192.168.8.114:9200"，"https://192.168.8.116:9200"]
elasticsearch.username: "admin"
elasticsearch.password: "viu@1234"
elasticsearch.pingTimeout: 1500
elasticsearch.requestTimeout: 30000
i18n.locale: "zh-CN"
elasticsearch.ssl.certificateAuthorities: config/certs/ca/ca.crt   //拷贝es下ca证书即可
elasticsearch.ssl.verificationMode: certificate

 

---------------或者跳过证书

 

server.port: 5601
server.host: "192.168.8.115"
server.maxPayload: 1048576
server.name: "kibana"
server.ssl.enabled: false
elasticsearch.hosts: ["https://192.168.8.115:9200"]
elasticsearch.username: "admin"
elasticsearch.password: "viu@1234"
elasticsearch.pingTimeout: 1500
elasticsearch.requestTimeout: 30000
i18n.locale: "zh-CN"
#elasticsearch.ssl.certificateAuthorities: config/certs/ca/ca.crt
#elasticsearch.ssl.verificationMode: certificate
elasticsearch.ssl.verificationMode: none

 

 

 

---------

filebeats

output.elasticsearch:
hosts: ["https://xxxxg:443"]
username: "elastic"
password: ""
index: "cloudgame-%{+yyyy.MM}"
ssl:
verification_mode: none