es分布式集群搭建

调整系统参数

vi /etc/sysctl.conf 
#添加下面配置：
vm.max_map_count= 262144
#生效
sysctl -p

es配置文件修改

cat jvm.options

-Xms16g
-Xmx16g

配置证书

cd /usr/share/elasticsearch/bin
./elasticsearch-certutil ca  # 默认回车，输入密码
./elasticsearch-certutil cert --ca elastic-stack-ca.p12  # 输入密码，默认回车，输入密码

将生成的elastic-certificates.p12、elastic-stack-ca.p12传送至其他节点的conf目录下面

scp *..p12 root@172.16.76.135:/data/elasticsearch-7.16.3/config

所有节点导入证书

cd  /data/elasticsearch-7.16.3/bin
sudo ./elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
sudo ./elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

配置elasticsearch.yml文件

 # ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data/elasticsearch-7.16.3/data
#
# Path to log files:
#
path.logs: /data/elasticsearch-7.16.3/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 0.0.0.0
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["172.16.76.128", "172.16.76.133","172.16.76.134","172.16.76.135"]

#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["node-1", "node-2","node-3"]

#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Security ----------------------------------
#
#                                 *** WARNING ***
#
# Elasticsearch security features are not enabled by default.
# These features are free, but require configuration changes to enable them.
# This means that users don’t have to provide credentials and can get full access
# to the cluster. Network connections are also not encrypted.
#
# To protect your data, we strongly encourage you to enable the Elasticsearch security features.
# Refer to the following documentation for instructions.
#
# https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html
transport.tcp.port: 9300
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /data/elasticsearch-7.16.3/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /data/elasticsearch-7.16.3/config/elastic-certificates.p12
xpack.monitoring.collection.cluster.stats.timeout: '30s'
ingest.geoip.downloader.enabled: false

修改权限

切换普通用户启动es

在其中一台节点生成密码

cd app/es7/bin/
./elasticsearch-setup-passwords interactive

-------------------------------------------------------------------------------

扩展集群

1. 修改所有集群节点配置文件，添加新增节点

discovery.seed_hosts: ["172.16.76.128", "172.16.76.133","172.16.76.134","172.16.76.135"]    # 集群节点

cluster.initial_master_nodes: ["node-1", "node-2","node-3","node-4"]   # 可以竞选master节点

2. 拷贝证书文件到新节点的config目录，并导入证书

scp *..p12 root@172.16.76.135:/data/elasticsearch-7.16.3/config

cd  /data/elasticsearch-7.16.3/bin
sudo ./elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
sudo ./elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

3. 修改权限

4. 重启es

--------------------------------------------------------------------------------

集群维护下线与加入

es下线节点
curl  -u elastic:密码    -H "Content-Type:application/json"   -XPUT http://172.16.76.128:9200/_cluster/settings -d '{"transient" :{"cluster.routing.allocation.exclude._ip" : "172.16.76.128"}}'

取消节点禁用策略
curl  -u elastic:密码    -H "Content-Type:application/json"   -XPUT http://172.16.76.128:9200/_cluster/settings -d '{"transient" :{"cluster.routing.allocation.exclude._ip" : null}}'

查看集群健康状态

curl -k -u elastic:密码 -XGET http://localhost:9200/_cluster/health?pretty -i

查找未分配的分片

  ​​curl -s "http://localhost:9200/_cat/shards" | grep UNASSIGNED​​

将没有成功分配的shard，进行手动迁移到目标节点

curl -XPOST 'localhost:9200/_cluster/reroute' -d '{
"commands" : [ {
"allocate" : {
"index" : "rs_wx_test",#unsigned 的索引
"shard" : 1,#出现unsigned的切片编号
"node" : "AfUyuXmGTESHXpwi4OExxx",#目标节点
"allow_primary" : true
}
}]
}'

因为故障（异常）致使的分片分配不了，尝试进行retry

POST /_cluster/reroute?retry_failed=true