elastersearch7.6.1搭建及基本操作详解
vim /etc/security/limits.conf
* soft nofile 655350 * hard nofile 655350 * soft nofile 655350 * hard nofile 655350 es soft memlock unlimited es hard memlock unlimited
vim /etc/sysctl.conf
vm.max_map_count=262144 fs.file-max=655360 vm.swappiness=1
保存后执行 sysctl -p
编辑配置文件
vim elastersearch.yaml
cluster.name: elastersearch node.name: cs-es1 path.data: /home/es/data path.logs: /home/es/logs # 是否锁住内存，避免交换内存带来性能损失，默认为false bootstrap.memory_lock: true # 本机ip network.host: 192.168.1.2 http.port: 9200 transport.tcp.port: 9300 # 节点角色，默认data master client 都有,如果分开，则改为 false node.data: true node.master: true # 所有节点ip discovery.seed_hosts: ["192.168.1.2","192.168.1.3","192.168.1.4"] # 初始化 master 节点 cluster.initial_master_nodes: ["192.168.1.2","192.168.1.3","192.168.1.4"] discovery.zen.ping_timeout: 60s discovery.zen.join_timeout: 60s # 忽略不符合master的节点 discovery.zen.master_election.ignore_non_master_pings: true # 选举最多同意数 discovery.zen.mininum_master_nodes: 1 gateway.recover_after_nodes: 1 gateway.recover_after_time: 5m gateway.expected_nodes: 1 # xpack,如果不需要开启，注释掉 xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
此配置文件为模板，如需多台服务器组集群，则要将本模板的node.name、network.host字段更改为实际的名称和地址，discovery.seed_hosts和cluster.initial_master_nodes也要做相应的更改
若开启 xpath ,需要创建证书
执行命令生成证书
bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""
需要将此证书放在每台节点的 config/ 目录下
然后重启es，然后设置密码
需要为4个用户分别设置密码：elastic、kibana、logstash_system、beats_system
1.可以自动设置密码
bin/elasticsearch-setup-passwords auto
2.也可以手动设置密码
bin/elasticsearch-setup-passwords interactive
访问时加上密码访问即可
curl -u elastic:baidu@123 '192.168.1.2/_cat/nodes'
如果kibina等需要链接ES，需要在连接的配置文件中指定密码
加入systemd管理
vim /usr/lib/systemd/system/elasticsearch.service
[Unit] Description=es user elasticsearch.service After=network.target [Service] Type=simple User=es Group=es ExecStart=/home/es/elasticsearch7/bin/elasticsearch ExecStop=/bin/kill -9 $MAINPID Restart=no LimitMEMLOCK=infinity LimitNOFILE=1048576 LimitNPROC=1048576 PrivateTmp=true [Install] WantedBy=multi-user.target
其中参数不太理解可以参考此博文：http://www.jinbuguo.com/systemd/systemd.service.html
重新加载systemd
systemctl daemon-reload
systemctl start elasticsearch.service
基本操作
1、查看集群健康状态
curl '10.231.13.177:9200/_cluster/health?pretty' { "cluster_name" : "elasticsearch", "status" : "green", //集群的状态红绿灯，绿：健康，黄：亚健康，红：病态 "timed_out" : false, "number_of_nodes" : 4, //节点数 "number_of_data_nodes" : 1, //数据节点数 "active_primary_shards" : 0, //分片数，0个Index库 "active_shards" : 0, //正常的分片数 "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 } ## 查看指定索引库的健康状态 http://localhost:9200/_cluster/health/index_name?pretty
2、查询集群所有节点
curl '10.231.13.177:9200/_cat/nodes?v' ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name 10.231.66.181 18 96 14 3.12 4.11 4.44 di - ngd-es5-elasticsearch-data-0 10.231.66.172 26 96 14 3.12 4.11 4.44 - - ngd-es5-elasticsearch-client-7795566789-6qc5x 10.231.66.183 40 96 14 3.12 4.11 4.44 m - ngd-es5-elasticsearch-master-0 10.231.64.85 35 99 2 0.37 0.63 0.76 m * ngd-es5-elasticsearch-master-1 heap.percent：堆内存占的内存百分比 ram.percent：物理内存占用百分比 cpu：表示使用的cpu核心 load_1m load_5m load_15m：1分钟 5分钟 15分钟 占用系统cpu百分比 node.role：表示节点能充当的角色主(m)、数据(di) 、client(-)节点，节点未区分开为dilm master：表示当前是否为主节点，*表示当前为主 ## 查看MASTER节点 curl '10.233.0.93:9200/_cat/master?v' id host ip node 2oaDofmzQ6KQK9BlorvO0Q 10.233.110.41 10.233.110.41 ngd-es5-elasticsearch-master-1
3、列出所有索引
curl '10.231.13.177:9200/_cat/indices?v' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open vfantest acyI9dArRAqTOHNAwGO_oQ 5 1 0 0 650b 650b
当前只有一个索引
4、查看各个节点的总分片数及分布情况，及磁盘使用率
curl '10.233.8.243:9200/_cat/allocation?v' shards disk.indices disk.used disk.avail disk.total disk.percent host ip node 28 84.8kb 284.4gb 57.8gb 342.2gb 83 10.233.64.133 10.233.64.133 ngd-es7-master-1 29 68.2kb 140.3gb 201.9gb 342.2gb 40 10.233.66.72 10.233.66.72 ngd-es7-master-2 29 106.4kb 91.4gb 250.8gb 342.2gb 26 10.233.65.219 10.233.65.219 ngd-es7-master-0
5、查看集群分片的详细分布，分片的大小等
curl '10.233.8.243:9200/_cat/shards?v' index shard prirep state docs store ip node dialog-statistics-scene-202106 1 p STARTED 0 283b 10.233.64.133 ngd-es7-master-1 dialog-statistics-scene-202106 1 r STARTED 0 283b 10.233.65.219 ngd-es7-master-0 dialog-statistics-scene-202106 2 p STARTED 0 283b 10.233.66.72 ngd-es7-master-2 dialog-statistics-scene-202106 2 r STARTED 0 283b 10.233.65.219 ngd-es7-master-0 dialog-statistics-scene-202106 0 r STARTED 0 283b 10.233.64.133 ngd-es7-master-1 dialog-statistics-scene-202106 0 p STARTED 0 283b 10.233.66.72 ngd-es7-master-2 search-suggest-online-data 1 r STARTED 0 283b 10.233.64.133 ngd-es7-master-1 search-suggest-online-data 1 p STARTED 0 283b 10.233.65.219 ngd-es7-master-0 search-suggest-online-data 2 p STARTED 0 283b 10.233.66.72 ngd-es7-master-2 search-suggest-online-data 2 r STARTED 0 283b 10.233.65.219 ngd-es7-master-0 search-suggest-online-data 0 r STARTED 0 283b 10.233.64.133 ngd-es7-master-1 search-suggest-online-data 0 p STARTED 0 283b 10.233.66.72 ngd-es7-master-2
6、查看 segments 分布信息
curl '10.233.8.243:9200/_cat/segments?v' index shard prirep ip segment generation docs.count docs.deleted size size.memory committed searchable version compound data-record-index-202106 0 p 10.233.64.133 _0 0 9 0 19.4kb 6477 true true 8.4.0 true data-record-index-202106 0 r 10.233.66.72 _0 0 9 0 19.4kb 6477 true true 8.4.0 true data-record-index-202106 1 p 10.233.64.133 _0 0 7 0 19.4kb 6461 true true 8.4.0 true data-record-index-202106 1 p 10.233.64.133 _1 1 9 0 38.1kb 6621 true true 8.4.0 true data-record-index-202106 1 r 10.233.65.219 _0 0 7 0 19.4kb 6461 true true 8.4.0 true data-record-index-202106 1 r 10.233.65.219 _1 1 9 0 38.1kb 6621 true true 8.4.0 true data-record-index-202106 2 p 10.233.66.72 _0 0 4 0 16.2kb 6437 true true 8.4.0 true data-record-index-202106 2 p 10.233.66.72 _1 1 5 0 24.4kb 6541 true true 8.4.0 true data-record-index-202106 2 r 10.233.65.219 _0 0 4 0 16.2kb 6437 true true 8.4.0 true data-record-index-202106 2 r 10.233.65.219 _1 1 5 0 24.4kb 6541 true true 8.4.0 true
7、创建一个索引：
## 可以在创建中指定settings 和 mappings，加-d '{}' 参数指定即可 curl -XPUT '10.231.13.177:9200/20210122?pretty' -H 'content-type: application/json' -d '{ "acknowledged" : true, "shards_acknowledged" : true }' settings格式： "settings": { "number_of_shards": 5, "number_of_replicas": 1 } mappings格式： "mappings": { "sms_up": { "dynamic_templates": [ { "strings_as_keywords": { "match_mapping_type": "string", "mapping": { "type": "keyword" } } } ], ## 再次查看索引 curl '10.231.13.177:9200/_cat/indices?v' health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open vfantest acyI9dArRAqTOHNAwGO_oQ 5 1 0 0 650b 650b yellow open 20210122 urFLBfkQRGmIQVcXqITG5w 5 1 0 0 260b 260b
看到名称为"20210122"的索引已经创建，它有5个私有的分片以及一个副本
8、插入和获取
现在来插入一些数据到集群索引，插入时必须要指定数据类型，例如 external 类型 ID为 1 的数据
curl -XPUT 'localhost:9200/20210122/external/1?pretty' -d ' > { > "name": "Tony", > "sex": "boy" > }' 回显： { "_index" : "20210122", "_type" : "external", "_id" : "1", "_version" : 1, "result" : "created", "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "created" : true }
返回为""created" : true" 代表创建成功
获取 20210122 索引下类型为 external 、ID =1 的数据，pretty参数表示返回结果格式美观。
curl 'localhost:9200/20210122/external/1?pretty' { "_index" : "20210122", "_type" : "external", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "name" : "Tony", "sex" : "boy" } }
9、删除索引
curl -XDELETE 'localhost:9200/20210122' {"acknowledged":true}
10、查看ES有哪些插件
curl '10.233.0.93:9200/_cat/plugins?v' name component version ngd-es5-elasticsearch-master-2 analysis-ik 5.4.2 ngd-es5-elasticsearch-master-2 discovery-kubernetes 5.4.2 ngd-es5-elasticsearch-master-2 elasticsearch-analysis-ansj 5.4.2.0 ngd-es5-elasticsearch-master-2 ingest-geoip 5.4.2 ngd-es5-elasticsearch-master-2 ingest-user-agent 5.4.2 ngd-es5-elasticsearch-master-2 x-pack 5.4.2 ... ngd-es5-elasticsearch-client-7795566789-6w2ns x-pack 5.4.2
11、查看当前ES节点信息
curl '10.233.0.93:9200' { "name" : "ngd-es5-elasticsearch-client-7795566789-6w2ns", "cluster_name" : "elasticsearch", "cluster_uuid" : "KJAcgz-xQ1awOxG4T-WOlA", "version" : { "number" : "5.4.2", "build_hash" : "929b078", "build_date" : "2017-06-15T02:29:28.122Z", "build_snapshot" : false, "lucene_version" : "6.5.1" }, "tagline" : "You Know, for Search" }
You Know，for Search 表示节点正常
