elasticsearch（基础）
一、创建postman项目
1.创建空间
2.新建测试项目
| GET http://10.0.0.101:9200/_cat/nodes |
二、ES的常见术语
| 索引(index)： |
| 用户写入ES集群的逻辑单元。 |
| 分片(shard): |
| 一个索引最少一个分片。 |
| 将索引的数据分布式的存储在ES集群。 |
| 副本(replica): |
| 一个分片可以有0个或多个副本。 |
| 为同一个分片数据提供数据冗余。 |
| 文档(docment): |
| 实际存储数据的媒介。 |
| 这些文档存储在分片中。 |
| |
| 主分片和副本分片的区别: |
| 主分片可以用于读写操作(rw)。 |
| 副本分片仅能用于读取操作(ro)。 |
| |
| 集群的颜色: |
| - green |
| 表示所有的主分片和副本分片均正常工作。 |
| |
| - yellow |
| 表示有部分副本分片不正常工作。 |
| |
| - red |
| 表示有部分主分片不正常工作。 |
注意事项：
分片数量不能修改，副本可以修改
分片数量为啥不能修改
因为文档存取计算公式 hash(文档id)%分片数量
【文档写入哪个分片编号计算公式： hash(文档ID) % primary_shards_numbers】
假设现在文档id是10，分片数量是3，那么取余是1，文档存到分片1，读取的时候也是分片1
现在文档id是10，分片数量变成5，那么取余是0，读取的时候变成分片0，就会读取不到
副本数量不能大于等于主机数量，否则会有个副本起不来。集群变黄
三、索引管理:
1.查看索引
1.1 查看所有的索引
| curl -X GET 10.0.0.101:9200/_cat/indices |
2.创建索引
2.1 创建默认索引，默认是一个分片和一个副本
| curl -X PUT 10.0.0.101:9200/linux-es |
2.2 创建指定的分片
| curl -XPUT 10.0.0.101:9200/linux-es-001 |
| { |
| "settings":{ |
| "number_of_shards": 3 |
| } |
| } |
2.3 创建指定的分片和副本
| curl -XPUT 10.0.0.101:9200/linux-es-003 |
| { |
| "settings":{ |
| "number_of_shards": 5, |
| "number_of_replicas":2 |
| } |
| } |
| |
3.修改索引
3.1 修改副本
| curl -XPUT 10.0.0.101:9200/linux-es-003/_settings |
| { |
| "number_of_replicas": 1 |
| } |
| 副本修改为1个 |
3.2 修改分片，不能修改
| curl -XPUT 10.0.0.101:9200/linux-es-003/_settings |
| { |
| "number_of_shards": 10 |
| } |
| |
4.删除索引
4.1 删除单个索引
| curl -XDELETE 10.0.0.101:9200/linux-es-003 |
4.2基于通配符删除多个索引
| curl -XDELETE 10.0.0.101:9200/linux-es-* |
5.索引别名
5.1添加索引别名
| POST http://10.0.0.101:9200/_aliases |
| { |
| "actions": [ |
| { |
| "add": { |
| "index": "linux-es", |
| "alias": "Linux2023" |
| } |
| }, |
| { |
| "add": { |
| "index": "linux-es-001", |
| "alias": "Linux2023" |
| } |
| } |
| ] |
| } |
5.2 查看索引别名
| GET http://10.0.0.101:9200/_aliases |
5.3 删除索引别名
| POST http://10.0.0.101:9200/_aliases |
| { |
| "actions": [ |
| { |
| "remove": { |
| "index": "linux-es", |
| "alias": "Linux2023" |
| } |
| } |
| ] |
| } |
| |
5.4修改索引别名
| POST http://10.0.0.101:9200/_aliases |
| { |
| "actions": [ |
| { |
| "remove": { |
| "index": "linux-es-001", |
| "alias": "Linux2023" |
| } |
| }, |
| { |
| "add": { |
| "index": "linux-es-001", |
| "alias": "Linux运维" |
| } |
| } |
| ] |
| } |
6.索引管理
6.1 关闭索引
| POST 10.0.0.101:9200/linux-es/_close |
| |
| 注：索引关闭意味着该索引无法进行读写操作，但数据不会被删除 |
6.2 打开索引
| POST 10.0.0.101:9200/linux-es/_open |
补充：
| Restful风格程序: |
| RESTFUL是一种网络应用程序的设计风格和开发方式，基于HTTP，可以使用XML格式定义或JSON格式定义。 |
| REST（英文：Representational State Transfer，简称REST）描述了一个架构样式的网络系统，比如 web 应用程序。 |
| REST首次出现在2000年Roy Fielding的博士论文中，Roy Fielding是HTTP规范的主要编写者之一。 |
| |
| |
| JSON语法: |
| 基础数据类型: |
| 字符串: |
| "linux" |
| "2022" |
| "" |
| 数字: |
| 0 |
| 1 |
| 2 |
| ... |
| 布尔值: |
| true |
| false |
| 空值: |
| null |
| |
| 高级数据类型: |
| 数组: |
| ["linux","北京",2022,null,true,{"age":"40","hobby":"linux"}] |
| |
| 对象: |
| {"name":"test", "age":40, "address":"北京", "hobby":["Linux","思想课"],"other":null} |
四、文档管理
1.创建文档
1.1不指定文档ID
| POST 10.0.0.101:9200/linux-es-student/_doc |
| { |
| "name": "stu01", |
| "hobby": ["吃鸡","rap"] |
| } |
1.2指定文档ID
| POST 10.0.0.101:9200/linux-es-student/_doc/1001 |
| { |
| "name": "stu2", |
| "hobby": ["看电影","熬夜"] |
| } |
2.文档修改
2.1 全量更新
| |
| POST 10.0.0.101:9200/linux-es-student/_doc/jS-5TIcBfCCSVc1HyFYP |
| { |
| "name": "stu3" |
| } |
2.2局部更新
| POST 10.0.0.101:9200/linux-es-student/_doc/1001/_update |
| { |
| "doc":{ |
| "age":20, |
| "hobby":["烫头"] |
| } |
| } |
3.文档查看
| GET 10.0.0.101:9200/linux-es-student/_search |
4.删除文档
| DELTE 10.0.0.101:9200/linux-es-student/_doc/1001 |
5.文档的批量操作
5.1批量创建
| POST 10.0.0.101:9200/_bulk |
| { "create": { "_index": "linux-es-elk"} } |
| { "name": "stu1","hobby":["Linux"] } |
| { "create": { "_index": "linux-es-elk","_id": 1002} } |
| { "name": "stu2","hobby":["JAVA"] } |
| { "create": { "_index": "linux-es-elk","_id": 1001} } |
| { "name": "stu3","hobby":["JS"] } |
提示这个表示body要有一个空行
5.2 批量修改
| POST 10.0.0.101:9200/_bulk |
| { "update" : {"_id" : "1001", "_index" : "linux-es-elk"} } |
| { "doc" : {"name" : "student01"} } |
| { "update" : {"_id" : "1002", "_index" : "linux-es-elk"} } |
| { "doc" : {"name" : "student02"} } |
5.3查询文档
| POST 10.0.0.101:9200/_mget |
| { |
| "docs": [ |
| { |
| "_index": "linux-es-elk", |
| "_id": "1001" |
| }, |
| { |
| "_index": "linux-es-elk", |
| "_id": "1002" |
| } |
| ] |
| } |
5.4 批量删除
| POST 10.0.0.101:9200/_bulk |
| { "delete" : { "_index" : "linux-es-elk", "_id" : "1001" } } |
| { "delete" : { "_index" : "linux-es-elk", "_id" : "1002" } } |
6.自定义数据类型-映射(mappings)
6.1 IP案例:
6.1.1 创建索引时指定映射关系
| PUT http://10.0.0.101:9200/linux-es-ip |
| { |
| "mappings": { |
| "properties": { |
| "ipaddress": { |
| "type": "ip" |
| } |
| } |
| } |
| } |
6.1.2查看索引的映射关系
| GET http://10.0.0.101:9200/linux-es-ip |
6.1.3创建测试数据
| POST 10.0.0.101:9200/_bulk |
| { "create": { "_index": "linux-es-ip"} } |
| { "name": "stu01","ipaddress": "172.16.15.15" } |
| { "create": { "_index": "linux-es-ip"} } |
| { "name": "stu02","ipaddress": "10.0.0.56" } |
6.1.4 查看IP的网段
| GET/POST 10.0.0.101:9200/linux-es-ip/_search |
| { |
| "query": { |
| "match" : { |
| "ipaddress": "10.0.0.0/24" |
| } |
| } |
| } |
6.2date案例:
6.2.1创建索引时指定映射关系
| PUT http://10.0.0.101:9200/linux-es-date |
| { |
| "mappings": { |
| "properties": { |
| "birthday": { |
| "type": "date", |
| "format": "yyyy-MM-dd" |
| } |
| } |
| } |
| } |
6.2.2 查看索引的映射关系
| GET http://10.0.0.101:9200/linux-es-date |
6.2.3创建测试数据
| POST 10.0.0.101:9200/_bulk |
| { "create": { "_index": "linux-es-date"} } |
| { "name": "stu01","birthday": "1991-10-01" } |
| { "create": { "_index": "linux-es-date"} } |
| { "name": "stu02","birthday": "2003-05-01" } |
| { "create": { "_index": "linux-es-date"} } |
| { "name": "stu03","birthday": "1996-05-01" } |
6.2.4查看年龄，从大到小
| GET/POST 10.0.0.101:9200/linux-es-date/_search |
| { |
| "sort": { "birthday": "asc"} |
| } |
6.3综合案例
6.3.1创建索引
| PUT http://10.0.0.101:9200/linux-es-elk-2023 |
6.3.2 查看索引信息
| GET http://10.0.0.101:9200/linux-es-elk-2023 |
6.3.3 为已创建的索引修改数据类型
| PUT http://10.0.0.101:9200/linux-es-elk-2023/_mapping |
| { |
| "properties": { |
| "name": { |
| "type": "text", |
| "index": true |
| }, |
| "gender": { |
| "type": "keyword", |
| "index": true |
| }, |
| "province": { |
| "type": "keyword", |
| "index": true |
| }, |
| "city": { |
| "type": "keyword", |
| "index": false |
| }, |
| "email": { |
| "type": "keyword" |
| }, |
| "ip_addr": { |
| "type": "ip" |
| }, |
| "birthday": { |
| "type": "date", |
| "format": "yyyy-MM-dd" |
| } |
| } |
| } |
6.3.4 添加测试数据
| POST http://10.0.0.101:9200/_bulk |
| { "create": { "_index": "linux-es-elk-2023"}} |
| { "name": "stu1","gender":"男性的","telephone":"1111111111","province":"广西","city":"北海市","email":"stu1@qq.com","ip_addr":"192.168.25.201","birthday":"1999-04-05"} |
| { "create": { "_index": "linux-es-elk-2023"}} |
| { "name": "stu2","gender":"女性的","telephone":"222222222","province":"河南","city":"濮阳市","email":"stu2@qq.com","ip_addr":"192.168.15.31","birthday":"2003-09-05","hobby":["Linux"]} |
6.3.5查看数据-基于gender-匹配keyword类型
| GET http://10.0.0.101:9200/linux-es-elk-2023/_search |
| { |
| "query":{ |
| "match":{ |
| "gender": "女" |
| } |
| } |
| } |
6.3.6查看数据-基于name字段搜索-匹配text类型
| GET http://10.0.0.101:9200/linux-es-elk-2023/_search |
| { |
| "query":{ |
| "match":{ |
| "name": "stu" |
| } |
| } |
| } |
6.3.7 查看数据-基于email字段搜索-匹配keyword类型
| GET http://10.0.0.101:9200/linux-es-elk-2023/_search |
| { |
| "query":{ |
| "match":{ |
| "email": "stu1@qq.com" |
| } |
| } |
| } |
6.3.8查看数据-基于ip_addr字段搜索-匹配ip类型
| GET http://10.0.0.101:9200/linux-es-elk-2023/_search |
| { |
| "query": { |
| "match" : { |
| "ip_addr": "192.168.15.0/24" |
| } |
| } |
| } |
6.3.9查看数据-基于city字段搜索，无法完成，该字段无法被检索
| GET http://10.0.0.101:9200/linux-es-elk-2023/_search |
| { |
| "query":{ |
| "match":{ |
| "city": "濮阳市" |
| } |
| } |
| } |
五、分词器
1.内置的标准分词器-分析英文
| GET http://10.0.0.101:9200/_analyze |
| { |
| "analyzer": "standard", |
| "text": "My name is wh, and I'm 18 years old !" |
| } |
| |
| { |
| "tokens": [ |
| { |
| "token": "my", |
| "start_offset": 0, |
| "end_offset": 2, |
| "type": "<ALPHANUM>", |
| "position": 0 |
| }, |
| { |
| "token": "name", |
| "start_offset": 3, |
| "end_offset": 7, |
| "type": "<ALPHANUM>", |
| "position": 1 |
| }, |
| { |
| "token": "is", |
| "start_offset": 8, |
| "end_offset": 10, |
| "type": "<ALPHANUM>", |
| "position": 2 |
| }, |
| { |
| "token": "wh", |
| "start_offset": 11, |
| "end_offset": 13, |
| "type": "<ALPHANUM>", |
| "position": 3 |
| }, |
| { |
| "token": "and", |
| "start_offset": 16, |
| "end_offset": 19, |
| "type": "<ALPHANUM>", |
| "position": 4 |
| }, |
| { |
| "token": "i'm", |
| "start_offset": 20, |
| "end_offset": 23, |
| "type": "<ALPHANUM>", |
| "position": 5 |
| }, |
| { |
| "token": "18", |
| "start_offset": 24, |
| "end_offset": 26, |
| "type": "<NUM>", |
| "position": 6 |
| }, |
| { |
| "token": "years", |
| "start_offset": 27, |
| "end_offset": 32, |
| "type": "<ALPHANUM>", |
| "position": 7 |
| }, |
| { |
| "token": "old", |
| "start_offset": 33, |
| "end_offset": 36, |
| "type": "<ALPHANUM>", |
| "position": 8 |
| } |
| ] |
| } |
温馨提示:
标准分词器模式使用空格和符号进行切割分词的。
2.内置的标准分词器-分析中文并不友好
| GET http://10.0.0.101:9200/_analyze |
| { |
| "analyzer": "standard", |
| "text": "我爱北京天安门!" |
| } |
| |
| { |
| "tokens": [ |
| { |
| "token": "我", |
| "start_offset": 0, |
| "end_offset": 1, |
| "type": "<IDEOGRAPHIC>", |
| "position": 0 |
| }, |
| { |
| "token": "爱", |
| "start_offset": 1, |
| "end_offset": 2, |
| "type": "<IDEOGRAPHIC>", |
| "position": 1 |
| }, |
| { |
| "token": "北", |
| "start_offset": 2, |
| "end_offset": 3, |
| "type": "<IDEOGRAPHIC>", |
| "position": 2 |
| }, |
| { |
| "token": "京", |
| "start_offset": 3, |
| "end_offset": 4, |
| "type": "<IDEOGRAPHIC>", |
| "position": 3 |
| }, |
| { |
| "token": "天", |
| "start_offset": 4, |
| "end_offset": 5, |
| "type": "<IDEOGRAPHIC>", |
| "position": 4 |
| }, |
| { |
| "token": "安", |
| "start_offset": 5, |
| "end_offset": 6, |
| "type": "<IDEOGRAPHIC>", |
| "position": 5 |
| }, |
| { |
| "token": "门", |
| "start_offset": 6, |
| "end_offset": 7, |
| "type": "<IDEOGRAPHIC>", |
| "position": 6 |
| } |
| ] |
| } |
3.安装IK分词器
3.1 创建IK分词器目录
| mkdir /es/softwares/es7/elasticsearch-7.17.5/plugins/ik |
3.2 解压软件包
| [root@elk101.com ~] |
| -rw-r--r-- 1 root root 4504823 Apr 4 17:21 elasticsearch-analysis-ik-7.17.5.zip |
| [root@elk101.com ~] |
3.3 重启服务
4.测试IK中文分词器
4.1 测试IK中文分词器-细粒度拆分
| GET http://10.0.0.101:9200/_analyze |
| { |
| "analyzer": "ik_max_word", |
| "text": "我爱北京天安门!" |
| } |
| |
| { |
| "tokens": [ |
| { |
| "token": "我", |
| "start_offset": 0, |
| "end_offset": 1, |
| "type": "CN_CHAR", |
| "position": 0 |
| }, |
| { |
| "token": "爱", |
| "start_offset": 1, |
| "end_offset": 2, |
| "type": "CN_CHAR", |
| "position": 1 |
| }, |
| { |
| "token": "北京", |
| "start_offset": 2, |
| "end_offset": 4, |
| "type": "CN_WORD", |
| "position": 2 |
| }, |
| { |
| "token": "天安门", |
| "start_offset": 4, |
| "end_offset": 7, |
| "type": "CN_WORD", |
| "position": 3 |
| }, |
| { |
| "token": "天安", |
| "start_offset": 4, |
| "end_offset": 6, |
| "type": "CN_WORD", |
| "position": 4 |
| }, |
| { |
| "token": "门", |
| "start_offset": 6, |
| "end_offset": 7, |
| "type": "CN_CHAR", |
| "position": 5 |
| } |
| ] |
| } |
4.2 测试IK中文分词器-粗粒度拆分
| GET http://10.0.0.101:9200/_analyze |
| { |
| "analyzer": "ik_smart", |
| "text": "我爱北京天安门!" |
| } |
| |
| { |
| "tokens": [ |
| { |
| "token": "我", |
| "start_offset": 0, |
| "end_offset": 1, |
| "type": "CN_CHAR", |
| "position": 0 |
| }, |
| { |
| "token": "爱", |
| "start_offset": 1, |
| "end_offset": 2, |
| "type": "CN_CHAR", |
| "position": 1 |
| }, |
| { |
| "token": "北京", |
| "start_offset": 2, |
| "end_offset": 4, |
| "type": "CN_WORD", |
| "position": 2 |
| }, |
| { |
| "token": "天安门", |
| "start_offset": 4, |
| "end_offset": 7, |
| "type": "CN_WORD", |
| "position": 3 |
| } |
| ] |
| } |
5.自定义IK分词器的字典
| |
| [root@elk101.com ~] |
| [root@elk101.com /es/softwares/es7/elasticsearch-7.17.5/plugins/ik/config] |
| total 8260 |
| -rw-r--r-- 1 root root 5225922 Jan 18 2022 extra_main.dic |
| -rw-r--r-- 1 root root 63188 Jan 18 2022 extra_single_word.dic |
| -rw-r--r-- 1 root root 63188 Jan 18 2022 extra_single_word_full.dic |
| -rw-r--r-- 1 root root 10855 Jan 18 2022 extra_single_word_low_freq.dic |
| -rw-r--r-- 1 root root 156 Jan 18 2022 extra_stopword.dic |
| -rw-r--r-- 1 root root 625 Jan 18 2022 IKAnalyzer.cfg.xml |
| -rw-r--r-- 1 root root 3058510 Jan 18 2022 main.dic |
| -rw-r--r-- 1 root root 123 Jan 18 2022 preposition.dic |
| -rw-r--r-- 1 root root 1824 Jan 18 2022 quantifier.dic |
| -rw-r--r-- 1 root root 164 Jan 18 2022 stopword.dic |
| -rw-r--r-- 1 root root 192 Jan 18 2022 suffix.dic |
| -rw-r--r-- 1 root root 752 Jan 18 2022 surname.dic |
| |
| |
| cat > linux-es.dic <<'EOF' |
| 德玛西亚 |
| 艾欧尼亚 |
| 亚索 |
| 上号 |
| 带你飞 |
| 贼6 |
| EOF |
| |
| |
| [root@elk101.com /es/softwares/es7/elasticsearch-7.17.5/plugins/ik/config] |
| <?xml version="1.0" encoding="UTF-8"?> |
| <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd"> |
| <properties> |
| <comment>IK Analyzer 扩展配置</comment> |
| <!--用户可以在这里配置自己的扩展字典 --> |
| <entry key="ext_dict">linux-es.dic</entry> |
| <!--用户可以在这里配置自己的扩展停止词字典--> |
| <entry key="ext_stopwords"></entry> |
| <!--用户可以在这里配置远程扩展字典 --> |
| <!-- <entry key="remote_ext_dict">words_location</entry> --> |
| <!--用户可以在这里配置远程扩展停止词字典--> |
| <!-- <entry key="remote_ext_stopwords">words_location</entry> --> |
| </properties> |
| |
| |
| [root@elk101.com /es/softwares/es7/elasticsearch-7.17.5/plugins/ik/config] |
| |
| |
| GET http://10.0.0.101:9200/_analyze |
| { |
| "analyzer": "ik_smart", |
| "text": "嗨，哥们! 上号，我德玛西亚和艾欧尼亚都有号! 我亚索贼6，肯定能带你飞!!!" |
| } |
六、安装kibana集成ES集群
1.下载软件包
| [root@elk103.com ~] |
| -rw-r--r-- 1 root root 268453142 Apr 4 18:03 kibana-7.17.5-x86_64.rpm |
2.安装kibana
3.修改kibana配置文件
| vim /etc/kibana/kibana.yml |
| ... |
| |
| server.host: "0.0.0.0" |
| |
| elasticsearch.hosts: ["http://10.0.0.101:9200","http://10.0.0.102:9200","http://10.0.0.103:9200"] |
| |
| i18n.locale: "zh-CN" |
4.启动kibana
5.访问kibana界面
七、索引模板
什么是索引模板:
指的是创建索引的一种方式。用户可以根据需求自定义对应的索引模板。
1.查看索引模板
1.1 查看所有的索引模板
| GET http://10.0.0.103:9200/_template |
1.2 查看单个索引模板
| GET http://10.0.0.103:9200/_template/.monitoring-es |
2.创建/修改索引模板
| POST http://10.0.0.103:9200/_template/linux-es |
| { |
| "aliases": { |
| "DBA": {}, |
| "SRE": {}, |
| "K8S": {} |
| }, |
| "index_patterns": [ |
| "linux-es*" |
| ], |
| "settings": { |
| "index": { |
| "number_of_shards": 3, |
| "number_of_replicas": 0 |
| } |
| }, |
| "mappings": { |
| "properties":{ |
| "ip_addr": { |
| "type": "ip" |
| }, |
| "access_time": { |
| "type": "date" |
| }, |
| "address": { |
| "type" :"text" |
| }, |
| "name": { |
| "type": "keyword" |
| } |
| } |
| } |
| } |
3.删除索引模板:
| DELETE http://10.0.0.103:9200/_template/linux-es |
八、DSL
什么是DSL：
Elasticsearch 提供了基于JSON的完整 Query DSL（Domain Specific Language，领域特定语言）来定义查询
1.准备数据:
1.1创建索引添加映射关系
| PUT http://10.0.0.101:9200/linux-es-shopping |
| { |
| "mappings": { |
| "properties": { |
| "item": { |
| "type": "text" |
| }, |
| "title": { |
| "type": "text" |
| }, |
| "price": { |
| "type": "double" |
| }, |
| "type": { |
| "type": "keyword" |
| }, |
| "group": { |
| "type": "long" |
| }, |
| "auther": { |
| "type": "text" |
| }, |
| "birthday": { |
| "type": "date", |
| "format": "yyyy-MM-dd" |
| }, |
| "province": { |
| "type": "keyword" |
| }, |
| "city": { |
| "type": "keyword" |
| }, |
| "remote_ip": { |
| "type": "ip" |
| } |
| } |
| } |
| } |
1.2 导入数据
| POST 10.0.0.101:9200/_bulk |
| 参考"商品.json"内容即可 |
2.全文检索-match查询
| GET http://10.0.0.101:9200/linux-es-shopping/_search |
| { |
| "query":{ |
| "match":{ |
| "auther":"王辉" |
| } |
| } |
| } |
| |
| |
3.精确匹配-match_phrase查询
| http://10.0.0.101:9200/linux-es-shopping/_search |
| { |
| "query":{ |
| "match_phrase":{ |
| "auther":"王辉" |
| } |
| } |
| } |
| |
| |
4.全量查询-match_all查询
| http://10.0.0.101:9200/linux-es-shopping/_search |
| { |
| "query": { |
| "match_all": {} |
| } |
| } |
| |
| |
5.分页查询
5.1每页显示3条数据，查询第四页
| http://10.0.0.101:9200/linux-es-shopping/_search |
| { |
| "query":{ |
| "match_phrase":{ |
| "auther":"王辉" |
| } |
| }, |
| "size": 3, |
| "from":9 |
| } |
5.2查询第5组数据，每页显示7条数据，查询第9页
| http://10.0.0.101:9200/linux-es-shopping/_search |
| { |
| "query":{ |
| "match":{ |
| "group":5 |
| } |
| }, |
| "size":7, |
| "from": 56 |
| } |
相关参数说明:
size:
指定每页显示多少条数据，默认值为10.
from:
指定跳过数据偏移量的大小，默认值为0，即默认看第一页。
查询指定页码的from值 = "(页码 - 1) * 每页数据大小(size)"
温馨提示:
生产环境中，不建议深度分页，百度的页码数量控制在76页左右。
6.使用"_source"查看的指定字段
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query":{ |
| "match_phrase":{ |
| "auther":"王辉" |
| } |
| }, |
| "_source":["title","auther","price"] |
| } |
| |
| |
| { |
| "_index": "linux-es-shopping", |
| "_type": "_doc", |
| "_id": "1xhaUYcBEK9jVpqWKwch", |
| "_score": 5.9686093, |
| "_source": { |
| "price": 19.9, |
| "auther": "王辉", |
| "title": "果麦经典：小王子（畅销400万册纪念版，李继宏口碑译作，作者基金会官方认证简体中文版）" |
| } |
| }, |
7.查询存在某个字段的文档-exists
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "exists" : { |
| "field": "hobby" |
| } |
| } |
| } |
8.语法高亮
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "match_phrase": { |
| "title": "孙子兵法" |
| } |
| }, |
| "highlight": { |
| "pre_tags": [ |
| "<span style='color:red;'>" |
| ], |
| "post_tags": [ |
| "</span>" |
| ], |
| "fields": { |
| "title": {} |
| } |
| } |
| } |
| |
| |
| { |
| "_index": "linux-es-shopping", |
| "_type": "_doc", |
| "_id": "ABhaUYcBEK9jVpqWKwgh", |
| "_score": 24.361904, |
| "_source": { |
| "item": "https://item.jd.com/12693530.html", |
| "title": "孙子兵法", |
| "price": 9.00, |
| "type": "bookProducts", |
| "group": 5, |
| "auther": "孟凡夫", |
| "birthday": "1999-09-01", |
| "province": "辽宁", |
| "city": "沈阳", |
| "remote_ip": "101.28.133.39" |
| }, |
| "highlight": { |
| "title": [ |
| "<span style='color:red;'>孙</span><span style='color:red;'>子</span><span style='color:red;'>兵</span><span style='color:red;'>法</span>" |
| ] |
| } |
| }, |
9.排序查询
9.1 升序查询作者是于萌最便宜商品及价格
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query":{ |
| "match_phrase":{ |
| "auther":"于萌" |
| } |
| }, |
| "sort":{ |
| "price":{ |
| "order": "asc" |
| } |
| }, |
| "size":1 |
| } |
9.2 降序查询作者是于萌最贵的商品及价格
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query":{ |
| "match_phrase":{ |
| "auther":"于萌" |
| } |
| }, |
| "sort":{ |
| "price":{ |
| "order": "desc" |
| } |
| }, |
| "size":1 |
| } |
| |
| 相关字段说明: |
| sort: |
| 基于指定的字段进行排序。此处为指定的是"price" |
| order: |
| 指定排序的规则，分为"asc"(升序)和"desc"(降序)。 |
10.多条件查询-bool
10.1查看作者是于萌且商品价格为24.90
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "must": [ |
| { |
| "match_phrase": { |
| "auther": "于萌" |
| } |
| }, |
| { |
| "match": { |
| "price": 24.90 |
| } |
| } |
| ] |
| } |
| } |
| } |
10.2 查看作者是于萌或者是高超的商品并降序排序
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "should": [ |
| { |
| "match_phrase": { |
| "auther": "于萌" |
| } |
| }, |
| { |
| "match_phrase": { |
| "auther": "高超" |
| } |
| } |
| ] |
| } |
| }, |
| "sort":{ |
| "price":{ |
| "order": "desc" |
| } |
| } |
| } |
| |
| |
| |
10.3 查看作者是于萌或者是高超且商品价格为168或者198
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "should": [ |
| { |
| "match_phrase": { |
| "auther": "于萌" |
| } |
| }, |
| { |
| "match_phrase": { |
| "auther": "高超" |
| } |
| }, |
| { |
| "match": { |
| "price": 168.00 |
| } |
| }, |
| { |
| "match": { |
| "price": 198.00 |
| } |
| } |
| ], |
| "minimum_should_match": "2" |
| } |
| } |
| } |
| |
| |
10.4查看作者不是于萌或者是高超且商品价格为168或者198的商品
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "must_not": [ |
| { |
| "match_phrase": { |
| "auther": "于萌" |
| } |
| }, |
| { |
| "match_phrase": { |
| "auther": "高超" |
| } |
| } |
| ], |
| "should": [ |
| { |
| "match": { |
| "price": 168.00 |
| } |
| }, |
| { |
| "match": { |
| "price": 198.00 |
| } |
| } |
| ], |
| "minimum_should_match": 1 |
| } |
| } |
| } |
10.5综合案例
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "must": [ |
| { |
| "match_phrase": { |
| "title": "零食" |
| } |
| } |
| ], |
| "must_not": [ |
| { |
| "match_phrase": { |
| "auther": "于萌" |
| } |
| }, |
| { |
| "match_phrase": { |
| "auther": "高超" |
| } |
| } |
| ], |
| "should": [ |
| { |
| "match": { |
| "price": 168.00 |
| } |
| }, |
| { |
| "match": { |
| "price": 9.9 |
| } |
| }, |
| { |
| "match": { |
| "price": 19.9 |
| } |
| } |
| ], |
| "minimum_should_match": 1 |
| } |
| }, |
| "highlight": { |
| "pre_tags": [ |
| "<span style='color:red;'>" |
| ], |
| "post_tags": [ |
| "</span>" |
| ], |
| "fields": { |
| "title": {} |
| } |
| }, |
| "_source": [ |
| "title", |
| "price", |
| "auther" |
| ], |
| "sort": { |
| "price": { |
| "order": "desc" |
| } |
| } |
| } |
11.过滤查询
11.1查询3组成员产品价格3599到10500的商品的最便宜的3个
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "must": [ |
| { |
| "match": { |
| "group": 3 |
| } |
| } |
| ], |
| "filter": { |
| "range": { |
| "price": { |
| "gte": 3599, |
| "lte": 10500 |
| } |
| } |
| } |
| } |
| }, |
| "sort": { |
| "price": { |
| "order": "asc" |
| } |
| }, |
| "size": 3 |
| } |
11.2查询3,4,6这3个组的最贵的3个产品且不包含酒的商品
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query":{ |
| "bool":{ |
| "must_not":[ |
| { |
| "match":{ |
| "title": "酒" |
| } |
| } |
| ], |
| "should": [ |
| { |
| "match":{ |
| "group":3 |
| } |
| }, |
| { |
| "match":{ |
| "group":4 |
| } |
| }, |
| { |
| "match":{ |
| "group":6 |
| } |
| } |
| ] |
| } |
| }, |
| "sort":{ |
| "price":{ |
| "order": "desc" |
| } |
| }, |
| "size":3 |
| } |
12.精确匹配多个值-terms
12.1查询商品价格为9.9和19.9的商品
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "terms": { |
| "price": [ |
| 9.9, |
| 19.9 |
| ] |
| } |
| } |
| } |
| |
13.多词搜索
13.1 多词搜索包含"小面包"关键字的所有商品
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "must": [ |
| { |
| "match": { |
| "title": { |
| "query": "小面包", |
| "operator": "and" |
| } |
| } |
| } |
| ] |
| } |
| }, |
| "highlight": { |
| "pre_tags": [ |
| "<h1>" |
| ], |
| "post_tags": [ |
| "</h1>" |
| ], |
| "fields": { |
| "title": {} |
| } |
| } |
| } |
| |
and表示字段需要包含查询的所有字符，不需要挨着，顺序也不要对
or 表示字段需要包含查询的所有字符的某一个
14.权重案例
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "bool": { |
| "must": [ |
| { |
| "match": { |
| "title": { |
| "query": "小面包", |
| "operator": "and" |
| } |
| } |
| } |
| ], |
| "should": [ |
| { |
| "match": { |
| "title": { |
| "query": "下午茶", |
| "boost": 5 |
| } |
| } |
| }, |
| { |
| "match": { |
| "title": { |
| "query": "良品铺子", |
| "boost": 2 |
| } |
| } |
| } |
| ] |
| } |
| }, |
| "highlight": { |
| "pre_tags": [ |
| "<h1>" |
| ], |
| "post_tags": [ |
| "</h1>" |
| ], |
| "fields": { |
| "title": {} |
| } |
| } |
| } |
15.聚合查询
15.1统计每个组收集的商品数量
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| |
| "aggs": { |
| "linux-es_group": { |
| "terms":{ |
| "field": "group" |
| } |
| } |
| }, |
| "size": 0 |
| } |
15.2 统计2组最贵的商品
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "match": { |
| "group": 2 |
| } |
| }, |
| "aggs": { |
| "linux-es_max_shopping": { |
| "max": { |
| "field": "price" |
| } |
| } |
| }, |
| "sort":{ |
| "price":{ |
| "order":"desc" |
| } |
| }, |
| "size": 1 |
| } |
15.3统计3组最便宜的商品
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "match": { |
| "group": 3 |
| } |
| }, |
| "aggs": { |
| "linux-es_min_shopping": { |
| "min": { |
| "field": "price" |
| } |
| } |
| }, |
| "sort":{ |
| "price":{ |
| "order": "asc" |
| } |
| }, |
| "size": 1 |
| } |
15.4 统计4组商品的平均价格
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "match": { |
| "group": 4 |
| } |
| }, |
| "aggs": { |
| "linux-es_avg_shopping": { |
| "avg": { |
| "field": "price" |
| } |
| } |
| }, |
| "size": 0 |
| } |
15.5统计买下5组所有商品要多少钱
| http://10.0.0.101:9200/linux-es-shopping/_search |
| |
| { |
| "query": { |
| "match": { |
| "group":5 |
| } |
| }, |
| "aggs": { |
| "linux-es_sum_shopping": { |
| "sum": { |
| "field": "price" |
| } |
| } |
| }, |
| "size": 0 |
| } |
16.图形化界面查看
http://10.0.0.103:5601/
九、集群迁移实战
1.搭建两套多实例集群
| |
| [root@elk101.com ~] |
| 10.0.0.103 60 84 3 0.13 0.13 0.18 cdfhilmrstw - elk103.com |
| 10.0.0.101 43 46 1 0.07 0.04 0.07 cdfhilmrstw - elk101.com |
| 10.0.0.102 43 32 2 0.06 0.06 0.08 cdfhilmrstw * elk102.com |
| |
| [root@elk101.com ~] |
| 10.0.0.101 56 46 0 0.03 0.03 0.06 mdi * elk101.com |
| 10.0.0.102 43 46 43 1.07 0.29 0.16 mdi - elk102.com |
| 10.0.0.103 46 84 2 0.11 0.12 0.17 mdi - elk103.com |
2.同集群迁移实战
| POST http://10.0.0.103:9200/_reindex |
| { |
| "source": { |
| "index": "linux-es-shopping" |
| }, |
| "dest": { |
| "index": "linux-es-shopping-new" |
| } |
| } |
3.不同集群迁移
3.1 所有主机修改配置文件
| vim /es/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml |
| ... |
| |
| reindex.remote.whitelist: "10.0.0.*:19200" |
3.2 所有节点重启ES7服务
3.3 es6节点创建测试数据
| 10.0.0.101:19200/linux-es-student/_doc |
| { |
| "name": "stu05", |
| "age": 33, |
| "hobby": ["看书","运动","rap"] |
| } |
3.4 迁移数据
| POST http://10.0.0.103:9200/_reindex |
| { |
| "source": { |
| "index": "linux-es-student", |
| "remote": { |
| "host": "http://10.0.0.101:19200" |
| }, |
| "query": { |
| "bool": { |
| "filter": { |
| "range": { |
| "age": { |
| "gt": 25 |
| } |
| } |
| } |
| } |
| } |
| }, |
| "dest": { |
| "index": "linux-es-student-2023" |
| } |
| } |
| |
| |
十、集群API
1.ES集群健康状态API
| |
| yum -y install epel-release |
| yum -y install jq |
| |
| |
| [root@elk101.com ~] |
| { |
| "cluster_name": "linux-es", |
| "status": "green", |
| "timed_out": false, |
| "number_of_nodes": 3, |
| "number_of_data_nodes": 3, |
| "active_primary_shards": 12, |
| "active_shards": 24, |
| "relocating_shards": 0, |
| "initializing_shards": 0, |
| "unassigned_shards": 0, |
| "delayed_unassigned_shards": 0, |
| "number_of_pending_tasks": 0, |
| "number_of_in_flight_fetch": 0, |
| "task_max_waiting_in_queue_millis": 0, |
| "active_shards_percent_as_number": 100 |
| } |
| [root@elk101.com ~] |
| "green" |
| 相关参数说明: |
| cluster_name |
| 集群的名称。 |
| |
| status |
| 集群的健康状态，基于其主分片和副本分片的状态。 |
| ES集群有以下三种状态: |
| green |
| 所有分片都已分配。 |
| yellow |
| 所有主分片都已分配，但一个或多个副本分片未分配。 |
| 如果集群中的某个节点发生故障，则在修复该节点之前，某些数据可能不可用。 |
| red |
| 一个或多个主分片未分配，因此某些数据不可用。这可能会在集群启动期间短暂发生，因为分配了主分片。 |
| |
| timed_out |
| 是否在参数false指定的时间段内返回响应（默认情况下30秒）。 |
| |
| number_of_nodes |
| 集群内的节点数。 |
| |
| number_of_data_nodes |
| 作为专用数据节点的节点数。 |
| |
| active_primary_shards |
| 可用主分片的数量。 |
| |
| active_shards |
| 可用主分片和副本分片的总数。 |
| |
| relocating_shards |
| 正在重定位的分片数。 |
| |
| initializing_shards |
| 正在初始化的分片数。 |
| |
| unassigned_shards |
| 未分配的分片数。 |
| |
| delayed_unassigned_shards |
| 分配因超时设置而延迟的分片数。 |
| |
| number_of_pending_tasks |
| 尚未执行的集群级别更改的数量。 |
| |
| number_of_in_flight_fetch |
| 未完成的提取次数。 |
| |
| task_max_waiting_in_queue_millis |
| 自最早启动的任务等待执行以来的时间（以毫秒为单位）。 |
| |
| active_shards_percent_as_number |
| 集群中活动分片的比率，以百分比表示。 |
2.ES集群的设置及优先级（settings）
| 如果您使用多种方法配置相同的设置，Elasticsearch 会按以下优先顺序应用这些设置： |
| (1)Transient setting(临时配置，集群重启后失效) |
| (2)Persistent setting(持久化配置，集群重启后依旧生效) |
| (3)elasticsearch.yml setting(配置文件) |
| (4)Default setting value(默认设置值) |
2.1查询集群的所有配置信息
| GET http://10.0.0.103:9200/_cluster/settings?include_defaults=true&flat_settings=true |
2.2修改集群的配置信息
| PUT http://10.0.0.103:9200/_cluster/settings |
| { |
| "transient": { |
| "cluster.routing.allocation.enable": "none" |
| } |
| } |
| 相关参数说明: |
| "cluster.routing.allocation.enable": |
| "all": |
| 允许所有分片类型进行分配。 |
| "primaries" |
| 仅允许分配主分片。 |
| "new_primaries" |
| 仅允许新创建索引分配主分片。 |
| "none": |
| 不允许分配任何类型的分配。 |
| |
| 参考链接: |
| https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-get-settings.html |
| https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-update-settings.html |
3.集群状态API
| 集群状态是一种内部数据结构，它跟踪每个节点所需的各种信息，包括： |
| (1)集群中其他节点的身份和属性 |
| (2)集群范围的设置 |
| (3)索引元数据，包括每个索引的映射和设置 |
| (4)集群中每个分片副本的位置和状态 |
| 推荐阅读: |
| https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-state.html |
3.1 查看集群的状态信息
| GET http://10.0.0.103:9200/_cluster/state |
3.2 只查看节点信息
| GET http://10.0.0.103:9200/_cluster/state/nodes |
3.3查看nodes,version,routing_table这些信息，并且查看以"linux-es*"开头的所有索引
| http://10.0.0.103:9200/_cluster/state/nodes,version,routing_table/linux-es* |
4.集群统计API
| Cluster Stats API 允许从集群范围的角度检索统计信息。返回基本索引指标（分片数量、存储大小、内存使用情况）和有关构成集群的当前节点的信息（数量、角色、操作系统、jvm 版本、内存使用情况、cpu 和已安装的插件）。 |
| 推荐阅读: |
| https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-stats.html |
4.1查看统计信息
| GET http://10.0.0.103:9200/_cluster/stats |
5.查看集群的分片分配情况（allocation）
| 集群分配解释API的目的是为集群中的分片分配提供解释。 |
| |
| 对于未分配的分片，解释 API 提供了有关未分配分片的原因的解释。 |
| |
| 对于分配的分片，解释 API 解释了为什么分片保留在其当前节点上并且没有移动或重新平衡到另一个节点。 |
| |
| 当您尝试诊断分片未分配的原因或分片继续保留在其当前节点上的原因时，此 API 可能非常有用，而您可能会对此有所期待。 |
| 推荐阅读: |
| https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-allocation-explain.html |
5.1分析teacher索引的0号分片未分配的原因
| |
| |
| PUT http://10.0.0.103:9200/_cluster/settings |
| { |
| "transient": { |
| "cluster.routing.allocation.enable": "none" |
| } |
| } |
| |
| PUT 10.0.0.101:9200/linux-es-001 |
| { |
| "acknowledged": true, |
| "shards_acknowledged": false, |
| "index": "linux-es-001" |
| } |
| |
| |
| GET http://10.0.0.101:9200/_cluster/allocation/explain |
| { |
| "index": "linux-es-001", |
| "shard": 0, |
| "primary": true |
| } |
| |
| |
| |
| { |
| "index": "linux-es-001", |
| "shard": 0, |
| "primary": true, |
| "current_state": "unassigned", |
| "unassigned_info": { |
| "reason": "INDEX_CREATED", |
| "at": "2023-04-05T13:09:16.809Z", |
| "last_allocation_status": "no" |
| }, |
| "can_allocate": "no", |
| "allocate_explanation": "cannot allocate because allocation is not permitted to any of the nodes", |
| |
| |
| |
| PUT http://10.0.0.103:9200/_cluster/settings |
| { |
| "transient": { |
| "cluster.routing.allocation.enable": "all" |
| } |
| } |
6.集群分片重路由API（reroute）
| reroute 命令允许手动更改集群中各个分片的分配。 |
| |
| 例如，可以将分片从一个节点显式移动到另一个节点，可以取消分配，并且可以将未分配的分片显式分配给特定节点。 |
| 推荐阅读: |
| https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-reroute.html |
6.1将"linux-es-001"索引的0号分片从elk102节点移动到elk101节点。
| POST http://10.0.0.101:9200/_cluster/reroute |
| { |
| "commands": [ |
| { |
| "move": { |
| "index": "linux-es-001", |
| "shard": 0, |
| "from_node": "elk102.com", |
| "to_node": "elk101.com" |
| } |
| } |
| ] |
| } |
6.2取消副本分片的分配，其副本会重新初始化分配
| POST http://10.0.0.101:9200/_cluster/reroute |
| { |
| "commands": [ |
| { |
| "cancel": { |
| "index": "linux-es-001", |
| "shard": 0, |
| "node": "elk103.com" |
| } |
| } |
| ] |
| } |
十一、ES集群原理
1.文档的写入原理
2.单个文档的读取操作
3.文档的全量查询
4.ES底层存储原理剖析
5.倒排索引技术
| 原表数据: |
| 文章ID 文章context |
| 1001 My name is wh, I love linux, .... 10w+ |
| 1002 My name is test, I love java, .... 20w+ |
| 1003 I love kafka, .... 30w+ |
| |
| |
| |
| 倒排索引技术：（倒排表） |
| My: |
| - 1001 20 count : 8分 |
| - 1002 300 count : 100分 |
| ... |
| name |
| - 1001 |
| - 1002 |
| .. |
| is |
| Jason |
| Yin |
| I |
| love |
| linux |
| - 1001 100c |
| - 1010 20c |
| ... |
| hash(1001) % primary-shard ----> shard-number[...] |
6.面试题: 分片底层时如何工作的？
答: 分片底层对应的是一个Lucene库，而Lucene底层使用倒排索引技术实现。
7.es索引概念
| 正排索引(正向索引): |
| 我们MySQL为例，用id字段存储博客文章的编号，用context存储文件的内容。 |
| CREATE TABLE blog (id INT PRIMARY KEY AUTO_INCREMENT, context TEXT); |
| INSERT INTO blog VALUES (1,'I am wh, I love Linux ...') |
| 此时，如果我们查询文章内容包含"Jason Yin"的词汇的时候，就比较麻烦了，因为要进行全表扫描。 |
| SELECT * FROM blog WHERE context LIKE 'wh'; |
| |
| |
| 倒排索引(反向索引): |
| ES使用一种称为"倒排索引"的结构，它适用于快速的全文检索。 |
| 倒排索引中有以下三个专业术语: |
| 词条: |
| 指的是最小的存储和查询单元，换句话说，指的是您想要查询的关键字(词)。 |
| 对应英文而言通常指的是一个单词，而对于中文而言，对应的是一个词组。 |
| 词典(字典): |
| 它是词条的集合，底层通常基于"Btree+"和"HASHMap"实现。 |
| 倒排表: |
| 记录了词条出现在什么位置，出现的频率是什么。 |
| 倒排表中的每一条记录我们称为倒排项。 |
| |
| 倒排索引的搜索过程: |
| (1)首先根据用户需要查询的词条进行分词后，将分词后的各个词条字典进行匹配，验证词条在词典中是否存在; |
| (2)如果上一步搜索结果发现词条不在字典中，则结束本次搜索，如果在词典中，就需要去查看倒排表中的记录(倒排项); |
| (3)根据倒排表中记录的倒排项来定位数据在哪个文档中存在，而后根据这些文档的"_id"来获取指定的数据; |
| |
| |
| 综上所述，假设有10亿篇文章，对于mysql不创建索引的情况下，会进行全表扫描搜索"Jason Yin"。而对于ES而言，其只需要将倒排表中返回的id进行扫描即可，而无须进行全量查询。 |
8.es角色说明
| [root@elk103.com ~] |
| 10.0.0.102 40 45 7 0.21 0.44 0.29 cdfhilmrstw * elk102.com |
| 10.0.0.101 43 46 3 0.19 0.23 0.21 cdfhilmrstw - elk101.com |
| 10.0.0.103 48 60 24 1.00 1.03 0.44 cdfhilmrstw - elk103.com |
| |
| 角色说明: |
| c : |
| Cold data |
| d : |
| data node |
| f : |
| frozen node |
| h : |
| hot node |
| i : |
| ingest node |
| l : |
| machine learning node |
| m : |
| master eligible node |
| r : |
| remote cluster client node |
| s : |
| content node |
| t : |
| transform node |
| v : |
| voting-only node |
| w : |
| warm node |
| - : |
| coordinating node only |
| |
| 常用的角色说明: |
| data node: |
| 指的是存储数据的节点。 |
| node.data: true |
| |
| master node: |
| 控制ES集群，并维护集群的状态(cluster state，包括节点信息，索引信息等，ES集群每个节点都有一份)。 |
| node.master: true |
| |
| coordinating: |
| 协调节点可以处理请求的节点，ES集群所有的节点均为协调节点，该角色无法取消。 |
9.master角色和data角色分离实战
| 修改elk103位master节点，修改elk101和elk102为数据节点 |
9.1 停止所有节点的ES服务
9.2 所有节点清空数据
| rm -rf /es/{data,logs}/es7/* /tmp/* |
9.3 修改配置文件
| [root@elk103.com ~] |
| 76 cluster.initial_master_nodes: ["elk103.com"] |
| 101 node.data: false |
| 102 node.master: true |
| |
| |
| [root@elk101.com ~] |
| 76 cluster.initial_master_nodes: ["elk103.com"] |
| 100 node.data: true |
| 101 node.master: false |
| [root@elk102.com ~] |
| 76 cluster.initial_master_nodes: ["elk103.com"] |
| 100 node.data: true |
| 101 node.master: false |
9.4 测试
| [root@elk103.com ~] |
| 10.0.0.102 39 43 23 0.54 0.39 0.28 cdfhilrstw - elk102.com |
| 10.0.0.101 45 43 22 0.44 0.38 0.27 cdfhilrstw - elk101.com |
| 10.0.0.103 50 57 24 0.74 0.56 0.41 ilmr * elk103.com |
| |
| |
9.5 创建索引测试
| (6)创建索引测试，观察elk103是否能够存储分片。 |
| PUT 10.0.0.101:9200/linux-es-001 |
| { |
| "settings":{ |
| "number_of_shards": 10, |
| "number_of_replicas":0 |
| } |
| } |
【推荐】国内首个AI IDE，深度理解中文开发场景，立即下载体验Trae
【推荐】编程新体验，更懂你的AI，立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包，你的智能百科全书，全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell：AI 加持，快人一步