Linux安装ElastSearch

Linux安装ES

准备好Linux系统，软件安装前需要对当前系统做一些优化配置

系统配置修改

一、内存优化

在/etc/sysctl.conf添加如下内容：

1. fs.file-max=655360 系统最大打开文件描述符数
2. vm.max_map_count=655360 限制一个进程拥有虚拟内存区域的大小
3. sysctl -p生效

[root@localhost /] vi /etc/sysctl.conf 
[root@localhost /] cat /etc/sysctl.conf 
fs.file-max=655360
vm.max_map_count=655360
[root@localhost /] sysctl -p
fs.file-max = 655360
vm.max_map_count = 655360

二、修改最大文件打开数量

修改 /etc/security/limits.conf 文件
(nofile)最大开打开文件描述符
(nproc)最大用户进程数
(memlock)最大锁定内存地址空间

[root@localhost /] vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 65536
* hard nproc 65536
* soft memlock unlimited
* hard memlock unlimited

三、进程数限制

修改 /etc/security/limits.d/90-nproc.conf

将1024修改为65536

重新登陆 ulimit -a 查看是否生效

系统差异有的可能是 20-nproc.conf

[root@localhost /] vi /etc/security/limits.d/90-nproc.conf
*          soft    nproc     65536
root       soft    nproc     unlimited
[root@localhost ~] ulimit -u
65536

完成以上配置需要重启服务器 reboot

ElasticSearch安装

⚠️ 自行下载相应版本安装包安 https://www.elastic.co/cn/downloads/past-releases#elasticsearch ，装ES之前确保已经安装了jdk环境。启动ES服务时，不能使用root账号启动，切换创建的用户

一、上传解压重命名

将ES压缩包上传到/home/下

[root@localhost home] cd /home/
[root@localhost home] pwd
/home
[root@localhost home] ll
总用量 338228
-rw-r--r--. 1 root root 346342976 3月  15 14:47 elasticsearch-7.15.0-linux-aarch64.tar.gz

解压压缩包

[root@localhost home] tar -zxf elasticsearch-7.15.0-linux-aarch64.tar.gz 
[root@localhost home]# ll
总用量 338228
drwxr-xr-x. 9 root root       155 9月  16 11:07 elasticsearch-7.15.0
-rw-r--r--. 1 root root 346342976 3月  15 14:47 elasticsearch-7.15.0-linux-aarch64.tar.gz

重命名文件夹

[root@localhost home] mv elasticsearch-7.15.0 elasticsearch

创建快照路径

[root@localhost home] mkdir -p /home/elasticsearch/snapshot/

二、创建用户并授权

> 创建`es_user` 组 创建 `es_user`用户 设置用户密码

```shell
[root@localhost home] groupadd es_user
[root@localhost home] useradd es_user -g es_user
[root@localhost home] passwd es_user
更改用户 es_user 的密码 。
新的 密码：
无效的密码： 密码少于 8 个字符
重新输入新的 密码：
passwd：所有的身份验证令牌已经成功更新。
```

> 将文件`elasticsearch `的拥有者设为 `es_user`

```shell
[es_user@localhost home] chown -R es_user:es_user elasticsearch
[es_user@localhost home] ll
总用量 338228
drwxr-xr-x. 9 es_user es_user       155 9月  16 11:07 elasticsearch
-rw-r--r--. 1 root    root    346342976 3月  15 14:47 elasticsearch-7.15.0-linux-aarch64.tar.gz
drwx------. 2 es_user es_user        62 3月  15 15:18 es_user
```

三、修改配置文件

切换当前用户

[es_user@localhost home] su es_user

修改配置文件 vi /home/elasticsearch/config/elasticsearch.yml

# 集群名称，同一个集群其他节点名称要和主节点相同
cluster.name: my-application
# 节点名称唯一，每一个节点都需不同
node.name: node-1
# 快照备份路径
path.repo: /home/elasticsearch/snapshot/
# 数据存放路径，默认 es 根目录下 可选
#path.data: /path/to/data
# 日志存放路径，默认 es 根目录下 可选
#path.logs: /path/to/logs
# true主节点 子节点 false
node.master: true

# 绑定 IP 当前主机IP 或 0.0.0.0
network.host: 0.0.0.0

# 端口
http.port: 9200

# 集群发现，集群需要配置
#discovery.seed_hosts: ["127.0.0.1"]

# 各个节点列表,集群需要配置
cluster.initial_master_nodes: ["node-1"]

# 开启系统监控日志收集
xpack.monitoring.collection.enabled: true
# 数据保留时间默认 7天
xpack.monitoring.history.duration: 7d
xpack.ml.enabled: false

四、启动ES服务

ES根目录下的bin目录启动 es

[es_user@localhost home] cd elasticsearch/bin/

启动ES，进入ES ./bin 目录下执行; -d 后台运行

[es_user@localhost bin] ./elasticsearch -d

验证是否启动成功，输出以下信息证明启动成功

[root@localhost ~] curl http://127.0.0.1:9200
{
  "name" : "node-1",
  "cluster_name" : "my6666",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.15.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "79d65f6e357953a5b3cbcc5e2c7c21073d89aa29",
    "build_date" : "2021-09-16T03:05:29.143308416Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

ES开启SSL加密传输

在开启SSL认证之前，请确认您的ES服务器可以成功启动，以及相关环境配置都没有问题，使用es_user用户进行操作

生成证书

进入ES安装路径下,pwd 查看当前路径，请勿使用root账号操作，切换至普通用户或es用户

[root@localhost elasticsearch] pwd
/home/elasticsearch

生成ca授权证书

[es_user@localhost elasticsearch]$ ./bin/elasticsearch-certutil ca
Please enter the desired output file [elastic-stack-ca.p12]: 回车即可
Enter password for elastic-stack-ca.p12 : 回车即可

查看当前目录会生成一个 elastic-stack-ca.p12 证书文件

[es_user@localhost elasticsearch]$ ls
bin  config  elastic-stack-ca.p12  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.asciidoc

基于证书生成秘钥证书

[es_user@localhost elasticsearch]$ ./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 
Enter password for CA (elastic-stack-ca.p12) : 回车即可
Please enter the desired output file [elastic-certificates.p12]: 回车即可
Enter password for elastic-certificates.p12 :回车即可

查看当前目录会生成一个 elastic-certificates.p12 证书

[es_user@localhost elasticsearch]$ ls
bin  config  elastic-certificates.p12  elastic-stack-ca.p12  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.asciidoc

根据证书文件导出一份CA公钥文件，用于后续各应用配置文件(filebeat,logstash)中引用CA公钥时使用：

[es_user@localhost elasticsearch]$ openssl pkcs12 -clcerts -nokeys -in elastic-stack-ca.p12 -out ca.pem

在当前目录的 config 目录下创建一个 certs 目录用于存放证书文件

[es_user@localhost elasticsearch]$ mkdir -p config/certs

拷贝当前证书文件elastic-certificates.p12 到 config/certs并查看是否拷贝成功

[es_user@localhost elasticsearch]$ cp elastic-certificates.p12 config/certs/
[es_user@localhost elasticsearch]$ ls config/certs/
elastic-certificates.p12

添加SSL证书

添加证书时需要先停止ES服务

通过以下命令查看ES是否启动，如果启动使 kill -9 进程pid 结束进程，如下所示当前ES并未启动

[es_user@localhost elasticsearch]$ ps -ef|grep elasticsearch
es_user    9616 116449  0 14:44 pts/2    00:00:00 grep --color=auto elasticsearch

编辑config/elasticsearch.yml 配置文件

[es_user@localhost elasticsearch]$ vi config/elasticsearch.yml

在配置文件底部增加以下内容

# 开启安全验证
xpack.security.enabled: true
# 设置密码时改配置为false，设置成功将此配置设置为true，并且重启服务
xpack.security.http.ssl.enabled: false
xpack.security.http.ssl.keystore.path: /home/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: /home/elasticsearch/config/certs/elastic-certificates.p12

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /home/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /home/elasticsearch/config/certs/elastic-certificates.p12

启动ES服务 ./bin/elasticsearch 前台启动，窗口关闭服务停止，./bin/elasticsearch -d 后台启动

[es_user@localhost elasticsearch]$ ./bin/elasticsearch

生成账号密码

执行以下命令系统自动生成不同角色账号，在执行命令时需要等待ES完全启动成功，elastic 账号类似root账号有系统最高权限。将该账号信息配置到Java application-xxx.yaml配置中，生成成功后妥善保管所有账号密码

[es_user@localhost elasticsearch]$ ./bin/elasticsearch-setup-passwords auto 
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.x86_64/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y


Changed password for user apm_system
PASSWORD apm_system = Gy2A1L9QPNArAEFdgLSq

Changed password for user kibana_system
PASSWORD kibana_system = bvkPOKij4H0peAtGICjY

Changed password for user kibana
PASSWORD kibana = bvkPOKij4H0peAtGICjY

Changed password for user logstash_system
PASSWORD logstash_system = Cw8pWQpqQWF0pvHfmZqo

Changed password for user beats_system
PASSWORD beats_system = qIqZTl8jNDuys39zUxOF

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = BMg3JiXs4PauCnTNGdYW

Changed password for user elastic
PASSWORD elastic = j80MPels5jfrf9E7PM89

重启ES服务

重启之前，先停掉ES服务，修改配置文件，开启SSL认证

到此ES SSL加密结束

[es_user@localhost elasticsearch]$ vi config/elasticsearch.yml
xpack.security.http.ssl.enabled: true
[es_user@localhost elasticsearch]$ ./bin/elasticsearch

完整配置文件示例

# 集群名称
cluster.name: big_data
# 节点名称
node.name: node-1
# 主节点
node.master: true
# 绑定IP地址
network.host: 192.168.0.114
# 端口
http.port: 9200
# 集群发现
discovery.seed_hosts: ["192.168.0.114"]
# 集群主节点
cluster.initial_master_nodes: ["node-1"]
# 快照备份路径
path.repo: /home/elasticsearch/snapshot/
# 开启系统监控日志收集
xpack.monitoring.collection.enabled: true
# 数据保留时间默认 7天
xpack.monitoring.history.duration: 7d
# 关闭ES机器学习
xpack.ml.enabled: false
# 开启系统安全
xpack.security.enabled: true

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: /home/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.truststore.path: /home/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.http.ssl.client_authentication: "optional"

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /home/elasticsearch/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /home/elasticsearch/config/certs/elastic-certificates.p12