ES的基本语法

1 基本使用

　　使用kibaba来执行语句

　　es安装和kibaba安装

 

1）添加PUT

　　已存在会直接覆盖

PUT /myindex/_doc/1
{
  "id":1,
  "name":"手机",
  "price":3888.8,
  "desc":"美国有苹果，中国有菠萝"
}

 

2）修改POST

POST/myindex/_doc/1
{
  "id":1,
  "name":"手机",
  "price":3888.8,
  "desc":"美国有苹果，中国有菠萝"
}

 

3）查询GET

GET /jsnh/user/1/

 

4）删除DELETE

DELETE /jsnh/user/1/

 

2 查询

2.1 测试数据准备

PUT /product/_doc/1
{
    "name" : "xiaomi phone",
    "desc" :  "shouji zhong de zhandouji",
    "price" :  3999,
    "tags": [ "xingjiabi", "fashao", "buka" ]
}
PUT /product/_doc/2
{
    "name" : "xiaomi nfc phone",
    "desc" :  "zhichi quangongneng nfc,shouji zhong de jianjiji",
    "price" :  4999,
    "tags": [ "xingjiabi", "fashao", "gongjiaoka" ]
}


PUT /product/_doc/3
{
    "name" : "nfc phone",
    "desc" :  "shouji zhong de hongzhaji",
    "price" :  2999,
    "tags": [ "xingjiabi", "fashao", "menjinka" ]
}

PUT /product/_doc/4
{
    "name" : "xiaomi erji",
    "desc" :  "erji zhong de huangmenji",
    "price" :  999,
    "tags": [ "low", "bufangshui", "yinzhicha" ]
}

PUT /product/_doc/5
{
    "name" : "hongmi erji",
    "desc" :  "erji zhong de kendeji",
    "price" :  399,
    "tags": [ "lowbee", "xuhangduan", "zhiliangx" ]
}

 

2.2 简单查询

1）查询product/_doc下所有

GET /product/_doc/_search

 

2）简单条件查询

　　返回name中包含xiaomi或者phone的关键字的

　　es采取的是关键字查询索引，xiaomi phone会被分词分为 xiaomi和phone，再去查找索引

GET /product/_doc/_search?q=name:xiaomi phone

 

3）简单条件+分页排序

　　页码为0，每页为2，安装price排正序

GET /product/_doc/_search?q=name:xiaomi phone&from=0&size=2&sort=price:asc

 

2.3 复杂查询

2.3.1 查询所有match_all

　　无条件查询所有

    GET /product/_doc/_search
{
  "query":{
    "match_all": {}
  }
}

 

2.3.2 单字段条件查询match

　　match会对查询条件进行分词处理，得到一些关键字，然后每个关键字去相等匹配，只要有一个匹配到，就可以返回，再根据相关度排序

　　返回name中包含xiaomi或者phone关键字的

        GET /product/_doc/_search
{
  "query":{
    "match": {
      "name":"xiaomi phone"
    }
  }
}

 

2.3.3 多字段条件匹配multi_match

　　multi_match和match的区别在于，可以对多个字段进行匹配查询

　　查询name字段或desc字段中包含phone关键字的的，multi_match多个字段单条件去匹配

    GET /product/_doc/_search
{
  "query":{
    "multi_match": {
      "query":"phone",
      "fields":["name","desc"]
    }
  }
}

 

2.3.4 排序sort

    GET /product/_doc/_search
{
  "query":{
    "multi_match": {
      "query":"phone",
      "fields":["name","desc"]
    }
  },
  "sort":{
    "price":"asc"
  }
}

 更复杂的排序查看第6块内容

 

2.3.5 指定返回字段

　　可以指定只返回部分字段，不用全部字段都返回

　　只返回字段name和price

    GET /product/_doc/_search
{
  "query":{
    "multi_match": {
      "query":"phone",
      "fields":["name","desc"]
    }
  },
  "sort":{
    "price":"asc"
  },
  "_source":["name","price"]
}

 

2.3.6 分页

GET /product/_doc/_search
{
  "query":{
    "multi_match": {
      "query":"phone",
      "fields":["name","desc"]
    }
  },
  "sort":{
    "price":"asc"
  },
  "_source":["name","price"],
  "from":0,
  "size":2
}

 

2.3.7 不分词查询term

　　查询条件不会被分词，而是整个作为一个关键字去查询

　　获取name包含关键字xiaomi erji的，由于关键字中没有xiaomi erji，只有xiaomi和erji这两个关键字，所以查不到

GET /product/_doc/_search
{
  "query":{
    "term": {
      "name":"xiaomi erji"
    }
  },
  "sort":{
    "price":"asc"
  },
  "_source":["name","price"],
  "from":0,
  "size":2
}

 

2.3.8 不分词查询terms

　　和term的区别在于，对一个字段可以查询多个条件，满足一个就可以

　　获取name包含关键字包含xiaomi erji或者phone关键字的

    GET /product/_doc/_search
{
  "query":{
    "terms": {
      "name":["hongmi erji","phone"]
    }
  },
  "sort":{
    "price":"asc"
  },
  "_source":["name","price"],
  "from":0,
  "size":2
}

 

2.3.9 验证分词会怎么分_analyze

　　查询字符串"xiaomi nfc zhineng phone huhu"会被分成哪几个关键词

 GET /_analyze
{
  "analyzer": "standard",
  "text":"xiaomi nfc zhineng phone huhu"
}

 

2.3.10 短语搜索match_phrase

　　match_phrase还是分词后去搜的

　　目标文档需要包含分词后的所有关键词

　　目标文档还要保持这些词的顺序和文档中的一致

　　添加测试数据

PUT /myindex/_doc/1
{
  "id":1,
  "name":"手机",
  "price":3888.8,
  "desc":"USA has apple China has pineapple"
}
PUT /myindex/_doc/2
{
  "id":1,
  "name":"手机",
  "price":3888.8,
  "desc":"USA has apple China has pear"
}

PUT /myindex/_doc/3
{
  "id":1,
  "name":"手机",
  "price":3888.8,
  "desc":"USA has apple China has pear and pineapple"
}

　　查询

GET /myindex/_search
{
  "query":{
    "match_phrase":{
      "desc":"China has pineapple"
    }
  }
}

　　查询结果，只有一条。说明match_phrase是分词后去查询的，且查询的结果必须包含查询条件的这几个关键词，且顺序连续一致

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 0.6026137,
    "hits" : [
      {
        "_index" : "myindex",
        "_type" : "mytype",
        "_id" : "1",
        "_score" : 0.6026137,
        "_source" : {
          "id" : 1,
          "name" : "手机",
          "price" : 3888.8,
          "desc" : "USA has apple China has pineapple"
        }
      }
    ]
  }
}

 

2.4 组合条件查询bool

　　bool标识组合条件

 

2.4.1 must

　　must标识必须满足的条件

 

1）查询name中包含xiaomi关键字的并且desc中包含shouji关键字的

{
  "query":{
    "bool":{
      "must":
      [
        {
          "match":{
            "name":"xiaomi"
          }
        },
        {
          "match":{
            "desc":"shouji"
          }
        }
      ]
    }
  }
}

 

2.4.2 filter

　　filter也是必须满足的条件，和must的区别是不计相关度分数。

 

1）查询name包含xiaomi关键字的，desc包含shouji关键字的，price大于4000的(这个条件不参与相关度分数的计算)

GET /product/_doc/_search
{
  "query":{
    "bool":{
      "must":
      [
        {
          "match":{
            "name":"xiaomi"
          }
        },
        {
          "match":{
            "desc":"shouji"
          }
        }
      ],
      "filter":[
          {
            "range":{
              "price":{
                "gt":4000
              }
            }
          }
        ]
    }
  }
}

 

2.4.3 should

　　相当于or匹配

　　minimum_should_match表示shoule中的条件有几个必须满足，可以为0。

　　如果bool查询包含至少一个should子句，而没有must或 filter子句，则默认值为1。否则，默认值为0

 

1）查询出name包含xiaomi关键字或者desc包含shouji关键字的　

    GET /product/_doc/_search
{
  "query":{
    "bool":{
      "should":
      [
        {
          "match":{
            "name":"xiaomi"
          }
        },
        {
          "match":{
            "desc":"shouji"
          }
        }
      ],
      "minimum_should_match":1
    }
  }
}

 

2）查询出name包含xiaomi关键字并且desc包含shouji关键字的

    GET /product/_doc/_search
{
  "query":{
    "bool":{
      "should":
      [
        {
          "match":{
            "name":"xiaomi"
          }
        },
        {
          "match":{
            "desc":"shouji"
          }
        }
      ],
      "minimum_should_match":2
    }
  }
}

 

3）如下，因为存在must，所以minimum_should_match值为0，相当于should里的条件不存在，也就是查询price大于1999的

GET /product/_doc/_search
{
  "query":{
    "bool":{
      "should":
      [
        {
          "match":{
            "name":"xiaomi"
          }
        },
        {
          "match":{
            "desc":"shouji"
          }
        }
      ],
      "must":[
          {
            "range":{
              "price":{
                "gt":1999
              }
            }
          }
        ]
    }
  }
}

 

4）查询价格大于1999，并且(name包含xiaomi关键字或者desc包含shouji关键字)

GET /product/_doc/_search
{
  "query":{
    "bool":{
      "should":
      [
        {
          "match":{
            "name":"xiaomi"
          }
        },
        {
          "match":{
            "desc":"shouji"
          }
        }
      ],
      "minimum_should_match":1,
      "must":[
          {
            "range":{
              "price":{
                "gt":1999
              }
            }
          }
        ]
    }
  }
}

 

5）查询价格大于1999，并且(name包含xiaomi关键字，并且desc包含shouji关键字)

GET /product/_doc/_search
{
  "query":{
    "bool":{
      "should":
      [
        {
          "match":{
            "name":"xiaomi"
          }
        },
        {
          "match":{
            "desc":"shouji"
          }
        }
      ],
      "minimum_should_match":2,
      "must":[
          {
            "range":{
              "price":{
                "gt":1999
              }
            }
          }
        ]
    }
  }
}

 

2.4.4 must not

　　必须不满足，且不计算相关度分数

 

1）查询价格不能大于4000的，也就是价格小于等于4000的

GET /product/_doc/_search
{
  "query":{
    "bool":{
      "must_not":[
          {
            "range":{
              "price":{
                "gt":4000
              }
            }
          }
        ]
    }
  }
}

 

3 聚合查询

3.1 简介

　　ES聚合查询主要又三种模式,分别是分桶聚合(Bucket aggregations)、指标聚合(Metrics aggregations)、管道聚合(Pipeline aggregations),三种模式处理的业务场景不同,下面开始简要分析下.

 

3.2 分桶聚合

　　分桶聚合类似与关系型数据库的Group By查询,按照指定的条件,进行分组统计.下面用一张网络图(来自马士兵教育)来解释

　　图中首先按照手机的品牌进行分桶统计数量,接着在小米手机的分桶基础上,再按照小米手机的档次进行二次分桶(分桶的嵌套查询)统计数量.

　　分桶聚合大致就是为了完成以上需求的

 

3.3 指标聚合

　　指标聚合主要是计算指标的Avg(平均值)、Max(最大值)、Min(最小值)、Sum(求和)、Cardinality(去重)、ValueCount(记数)、Stats(统计聚合)、Top Hits(聚合)等.下面用一张网络图(来自马士兵教育)来解释

 

 　　可以通过指标聚合计算某个班级、某个学科的最高分、最低分等等

 

3.4 管道聚合

　　管道聚合主要用于对聚和结果的二次聚合,举个例子,这里需要计算某个商城中的各个品牌手机价格平均值中最小的手机品牌.

　　这里第一步需要计算各个手机品牌价格的平均值,接着计算平均值中的最小值,这里就需要用到管道聚合

 

3.5 示例数据准备

3.4.1 创建mappings

PUT food
{
  "settings": {
    "number_of_shards": 3,  
    "number_of_replicas": 1 
  }, 
  "mappings": {
    "date_detection": false, 
    "properties": {
      "CreateTime":{
        "type":"date",
        "format": "yyyy-MM-dd HH:mm:ss" 
      },
      "Desc":{
        "type": "text",
        "fields": {
          "keyword":{
            "type":"keyword", 
            "ignore_above":256
          }
        }

      },
      "Level":{
        "type": "text",
        "fields": {
          "keyword":{
            "type":"keyword",
            "ignore_above":256
          }
        }
      },
      "Name":{
        "type": "text",
        "fields": {
          "keyword":{
            "type":"keyword",
            "ignore_above":256
          }
        }
      },
      "Price":{
        "type": "float"
      },
      "Tags":{
        "type": "text",
        "fields": {
          "keyword":{
            "type":"keyword",
            "ignore_above":256
          }
        }
      },
      "Type":{
        "type": "text",
        "fields": {
          "keyword":{
            "type":"keyword",
            "ignore_above":256
          }
        }
      }
    }
  }
}

 

3.4.2 添加数据

PUT food/_doc/1
{
  "CreateTime":"2022-06-06 11:11:11",
  "Desc":"青菜 yyds 营养价值很高,很好吃",
  "Level":"普通蔬菜",
  "Name":"青菜",
  "Price":11.11,
  "Tags":["性价比","营养","绿色蔬菜"],
  "Type":"蔬菜"
}

PUT food/_doc/2
{
  "CreateTime":"2022-06-06 13:11:11",
  "Desc":"大白菜 好吃 便宜 水分多",
  "Level":"普通蔬菜",
  "Name":"大白菜",
  "Price":12.11,
  "Tags":["便宜","好吃","白色蔬菜"],
  "Type":"蔬菜"
}

PUT food/_doc/3
{
  "CreateTime":"2022-06-07 13:11:11",
  "Desc":"芦笋来自国外进口的蔬菜,西餐标配",
  "Level":"中等蔬菜",
  "Name":"芦笋",
  "Price":66.11,
  "Tags":["有点贵","国外","绿色蔬菜","营养价值高"],
  "Type":"蔬菜"
}

PUT food/_doc/4
{
  "CreateTime":"2022-07-07 13:11:11",
  "Desc":"苹果 yyds 好吃 便宜 水分多 营养",
  "Level":"普通水果",
  "Name":"苹果",
  "Price":11.11,
  "Tags":["性价比","易种植","水果","营养"],
  "Type":"水果"
}

PUT food/_doc/5
{
  "CreateTime":"2022-07-09 13:11:11",
  "Desc":"榴莲 非常好吃 很贵 吃一个相当于吃一只老母鸡",
  "Level":"高级水果",
  "Name":"榴莲",
  "Price":100.11,
  "Tags":["贵","水果","营养"],
  "Type":"水果"
}

PUT food/_doc/6
{
  "CreateTime":"2022-07-08 13:11:11",
  "Desc":"猫砂王榴莲 榴莲中的战斗机",
  "Level":"高级水果",
  "Name":"猫砂王榴莲",
  "Price":300.11,
  "Tags":["超级贵","进口","水果","非常好吃"],
  "Type":"水果"
}

 

3.5 分桶查询示例

　　查询各个标签的产品数据,如超级贵的食物有多少个,并按照标签属性进行升序排列

GET food/_search
{
  "size": 0, 
  "aggs": {
    "tags_aggs": {
      "terms": {
        "field": "Tags.keyword", 
        "size": 20,   
        "order": {
          "_count": "asc" 
        }
      }
    }
  }
}

　　查询结果

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 6,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "tags_aggs" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "便宜",
          "doc_count" : 1
        },
        {
          "key" : "国外",
          "doc_count" : 1
        },
        {
          "key" : "好吃",
          "doc_count" : 1
        },
        {
          "key" : "易种植",
          "doc_count" : 1
        },
        {
          "key" : "有点贵",
          "doc_count" : 1
        },
        {
          "key" : "白色蔬菜",
          "doc_count" : 1
        },
        {
          "key" : "营养价值高",
          "doc_count" : 1
        },
        {
          "key" : "贵",
          "doc_count" : 1
        },
        {
          "key" : "超级贵",
          "doc_count" : 1
        },
        {
          "key" : "进口",
          "doc_count" : 1
        },
        {
          "key" : "非常好吃",
          "doc_count" : 1
        },
        {
          "key" : "性价比",
          "doc_count" : 2
        },
        {
          "key" : "绿色蔬菜",
          "doc_count" : 2
        },
        {
          "key" : "水果",
          "doc_count" : 3
        },
        {
          "key" : "营养",
          "doc_count" : 3
        }
      ]
    }
  }
}

 

3.5 指标查询示例

　　现在按照价格统计以下,所有食物价格的最贵的、所有食物价格的最便宜的、所有食物价格的平均值、所有食物价格的总和

GET food/_search
{
  "size": 0, 
  "aggs": {
    "max_price":{
      "max": {
        "field": "Price"
      }
    },
    "min_price":{
      "min": {
        "field": "Price"
      }
    },
    "avg_price":{
      "avg": {
        "field": "Price"
      }
    },
    "sum_price":{
      "sum":{
        "field": "Price"
      }
    }
  }
}

　　查询结果如下

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 6,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "max_price" : {
      "value" : 300.1099853515625
    },
    "min_price" : {
      "value" : 11.109999656677246
    },
    "avg_price" : {
      "value" : 83.44333092371623
    },
    "sum_price" : {
      "value" : 500.65998554229736
    }
  }
}

　　过stats进行快速的查询

GET food/_search
{
  "size": 0, 
  "aggs": {
    "price_stats": {
      "stats": {
        "field": "Price"
      }
    }
  }
}

　　查询结果

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 6,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "price_stats" : {
      "count" : 6,
      "min" : 11.109999656677246,
      "max" : 300.1099853515625,
      "avg" : 83.44333092371623,
      "sum" : 500.65998554229736
    }
  }
}

　　

3.6 管道聚合示例

　　查询各个物分类中价格平均值最低的食物分类

GET food/_search
{
  "size": 0, 
  "aggs": {
    "type_bucket": {
      
      "terms": {
        "field": "Type.keyword"
      },
      
      "aggs": {
        "price_bucket": {
          "avg": {
            "field": "Price"
          }
        }
      }
    },
    "myresult":{
      "min_bucket": {
        "buckets_path": "type_bucket>price_bucket"
      }
    }
  }
}

　　查询结果

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 3,
    "successful" : 3,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 6,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "type_bucket" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "水果",
          "doc_count" : 3,
          "price_bucket" : {
            "value" : 137.1099952061971
          }
        },
        {
          "key" : "蔬菜",
          "doc_count" : 3,
          "price_bucket" : {
            "value" : 29.77666664123535
          }
        }
      ]
    },
    "myresult" : {
      "value" : 29.77666664123535,
      "keys" : [
        "蔬菜"
      ]
    }
  }
}

 

　　更复杂的排序多聚合示例查看：https://www.cnblogs.com/jthr/p/17141273.html 的第6块内容