ES high-level-client和DSL学习

一、ElasticSearch 简介

中文官方文档链接

https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.6/java-rest-high-getting-started.html

 

1、简介
ElasticSearch 是一个基于 Lucene 的搜索服务器。它提供了一个分布式多员工能力的全文搜索引擎，基于 RESTful web 接口。Elasticsearch 是用 Java 语言开发的，并作为 Apache 许可条款下的开放源码发布，是一种流行的企业级搜索引擎。ElasticSearch 用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。
2、特性

    分布式的文档存储引擎
    分布式的搜索引擎和分析引擎
    分布式，支持PB级数据

3、使用场景

    搜索领域：如百度、谷歌，全文检索等。
    门户网站：访问统计、文章点赞、留言评论等。
    广告推广：记录员工行为数据、消费趋势、员工群体进行定制推广等。
    信息采集：记录应用的埋点数据、访问日志数据等，方便大数据进行分析。

 

二、ElasticSearch 基础概念

 

1、ElaticSearch 和 DB 的关系

在 Elasticsearch 中，文档归属于一种类型 type，而这些类型存在于索引 index 中，我们可以列一些简单的不同点，来类比传统关系型数据库：

    Relational DB -> Databases -> Tables -> Rows -> Columns
    Elasticsearch -> Indices -> Types -> Documents -> Fields

Elasticsearch 集群可以包含多个索引 indices，每一个索引可以包含多个类型 types，每一个类型包含多个文档 documents，然后每个文档包含多个字段 Fields。而在 DB 中可以有多个数据库 Databases，每个库中可以有多张表 Tables，没个表中又包含多行Rows，每行包含多列Columns。
2、索引
索引基本概念（indices）：索引是含义相同属性的文档集合，是 ElasticSearch 的一个逻辑存储，可以理解为关系型数据库中的数据库，ElasticSearch 可以把索引数据存放到一台服务器上，也可以 sharding 后存到多台服务器上，每个索引有一个或多个分片，每个分片可以有多个副本。
索引类型（index_type）：索引可以定义一个或多个类型，文档必须属于一个类型。在 ElasticSearch 中，一个索引对象可以存储多个不同用途的对象，通过索引类型可以区分单个索引中的不同对象，可以理解为关系型数据库中的表。每个索引类型可以有不同的结构，但是不同的索引类型不能为相同的属性设置不同的类型。
3、文档
文档（document）：文档是可以被索引的基本数据单位。存储在 ElasticSearch 中的主要实体叫文档 document，可以理解为关系型数据库中表的一行记录。每个文档由多个字段构成，ElasticSearch 是一个非结构化的数据库，每个文档可以有不同的字段，并且有一个唯一的标识符。
4、映射
映射（mapping）:ElasticSearch 的 Mapping 非常类似于静态语言中的数据类型：声明一个变量为 int 类型的变量，以后这个变量都只能存储 int 类型的数据。同样的，一个 number 类型的 mapping 字段只能存储 number 类型的数据。同语言的数据类型相比，Mapping 还有一些其他的含义，Mapping 不仅告诉 ElasticSearch 一个 Field 中是什么类型的值， 它还告诉 ElasticSearch 如何索引数据以及数据是否能被搜索到。ElaticSearch 默认是动态创建索引和索引类型的 Mapping 的。这就相当于无需定义 Solr 中的 Schema，无需指定各个字段的索引规则就可以索引文件，很方便。但有时方便就代表着不灵活。比如，ElasticSearch 默认一个字段是要做分词的，但我们有时要搜索匹配整个字段却不行。如有统计工作要记录每个城市出现的次数。对于 name 字段，若记录 new york 文本，ElasticSearch 可能会把它拆分成 new 和 york 这两个词，分别计算这个两个单词的次数，而不是我们期望的 new york。

三、SpringBoot 项目引入 ElasticSearch 依赖

 

下面介绍下 SpringBoot 如何通过 elasticsearch-rest-high-level-client 工具操作 ElasticSearch，这里需要说一下，为什么没有使用 Spring 家族封装的 spring-data-elasticsearch。
主要原因是灵活性和更新速度，Spring 将 ElasticSearch 过度封装，让开发者很难跟 ES 的 DSL 查询语句进行关联。再者就是更新速度，ES 的更新速度是非常快，但是 spring-data-elasticsearch 更新速度比较缓慢。
由于上面两点，所以选择了官方推出的 Java 客户端 elasticsearch-rest-high-level-client，它的代码写法跟 DSL 语句很相似，懂 ES 查询的使用其上手很快。

1、Maven 引入相关依赖

1. <modelVersion>4.0.0</modelVersion>
2. <parent>
3. <groupId>org.springframework.boot</groupId>
4. <artifactId>spring-boot-starter-parent</artifactId>
5. <version>2.2.5.RELEASE</version>
6. <relativePath/> <!-- lookup parent from repository -->
7. </parent>
8.  
9. <groupId>org.example</groupId>
10. <artifactId>elasticsearch</artifactId>
11. <version>1.0-SNAPSHOT</version>
12.  
13. <properties>
14. <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
15. <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
16. <java.version>1.8</java.version>
17. <!-- 打包跳过测试模块 -->
18. <skipTests>true</skipTests>
19. </properties>
20.  
21. <dependencies>
22. <dependency>
23. <groupId>org.springframework.boot</groupId>
24. <artifactId>spring-boot-starter-web</artifactId>
25.  
26. </dependency>
27. <!--es客户端，不使用springboot封装的客户端-->
28. <dependency>
29. <groupId>org.elasticsearch.client</groupId>
30. <artifactId>elasticsearch-rest-high-level-client</artifactId>
31. <version>7.5.2</version>
32. </dependency>
33.  
34. <!--json解析工具-->
35. <dependency>
36. <groupId>com.alibaba</groupId>
37. <artifactId>fastjson</artifactId>
38. <version>1.2.68</version>
39. </dependency>
40. <!-- commons-lang3 -->
41. <dependency>
42. <groupId>org.apache.commons</groupId>
43. <artifactId>commons-lang3</artifactId>
44. <version>3.10</version>
45. </dependency>
46. <!--lombok-->
47. <dependency>
48. <groupId>org.projectlombok</groupId>
49. <artifactId>lombok</artifactId>
50. <version>1.18.12</version>
51. <scope>provided</scope>
52. </dependency>
53. <dependency>
54. <groupId>org.springframework.boot</groupId>
55. <artifactId>spring-boot-starter-test</artifactId>
56. <scope>test</scope>
57. <exclusions>
58. <exclusion>
59. <groupId>org.junit.vintage</groupId>
60. <artifactId>junit-vintage-engine</artifactId>
61. </exclusion>
62. </exclusions>
63. </dependency>
64. </dependencies>
65.  
66. <build>
67. <plugins>
68. <plugin>
69. <groupId>org.springframework.boot</groupId>
70. <artifactId>spring-boot-maven-plugin</artifactId>
71. </plugin>
72. </plugins>
73. </build>

2、ElasticSearch 连接配置

(1)、application.yml 配置文件

1. #spring
2. spring:
3. application:
4. name: springboot-test
5. #elasticsearch
6. elasticsearch:
7. schema: http
8. address: 127.0.0.1:9200
9. connectTimeout: 5000
10. socketTimeout: 5000
11. connectionRequestTimeout: 5000
12. maxConnectNum: 100
13. maxConnectPerRoute: 100

(2)、java 连接配置类
这里需要写一个 Java 配置类：

1. import org.apache.http.HttpHost;
2. import org.elasticsearch.client.RestClient;
3. import org.elasticsearch.client.RestClientBuilder;
4. import org.elasticsearch.client.RestHighLevelClient;
5. import org.springframework.beans.factory.annotation.Value;
6. import org.springframework.context.annotation.Bean;
7. import org.springframework.context.annotation.Configuration;
8.  
9. import java.util.ArrayList;
10. import java.util.List;
11.  
12.  
13. /**
14. * @Deacription ElasticSearch 配置
15. * @Author wenyt
16. * @Date 2020/3/18 9:12
17. * @Version 1.0
18. **/
19. @Configuration
20. public class ElasticSearchConfiguration {
21. /** 协议 */
22. @Value("${elasticsearch.schema:http}")
23. private String schema;
24.  
25. /** 集群地址，如果有多个用“,”隔开 */
26. @Value("${elasticsearch.address}")
27. private String address;
28.  
29. /** 连接超时时间 */
30. @Value("${elasticsearch.connectTimeout}")
31. private int connectTimeout;
32.  
33. /** Socket 连接超时时间 */
34. @Value("${elasticsearch.socketTimeout}")
35. private int socketTimeout;
36.  
37. /** 获取连接的超时时间 */
38. @Value("${elasticsearch.connectionRequestTimeout}")
39. private int connectionRequestTimeout;
40.  
41. /** 最大连接数 */
42. @Value("${elasticsearch.maxConnectNum}")
43. private int maxConnectNum;
44.  
45. /** 最大路由连接数 */
46. @Value("${elasticsearch.maxConnectPerRoute}")
47. private int maxConnectPerRoute;
48.  
49. @Bean(name = "restHighLevelClient")
50. public RestHighLevelClient restHighLevelClient() {
51. // 拆分地址
52. List<HttpHost> hostLists = new ArrayList<>();
53. String[] hostList = address.split(",");
54. for (String addr : hostList) {
55. String host = addr.split(":")[0];
56. String port = addr.split(":")[1];
57. hostLists.add(new HttpHost(host, Integer.parseInt(port), schema));
58. }
59. // 转换成 HttpHost 数组
60. HttpHost[] httpHost = hostLists.toArray(new HttpHost[]{});
61. // 构建连接对象
62. RestClientBuilder builder = RestClient.builder(httpHost);
63. // 异步连接延时配置
64. builder.setRequestConfigCallback(requestConfigBuilder -> {
65. requestConfigBuilder.setConnectTimeout(connectTimeout);
66. requestConfigBuilder.setSocketTimeout(socketTimeout);
67. requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeout);
68. return requestConfigBuilder;
69. });
70. // 异步连接数配置
71. builder.setHttpClientConfigCallback(httpClientBuilder -> {
72. httpClientBuilder.setMaxConnTotal(maxConnectNum);
73. httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
74. return httpClientBuilder;
75. });
76. return new RestHighLevelClient(builder);
77. }
78. }

 

四、索引操作示例

通过 Kibana 的 Restful 工具操作与对应的 Java 代码操作的两个示例。

1、Restful 操作示例

创建索引
创建名为 mydlq-user 的索引与对应 Mapping。

PUT /mydlq-user
{
  "mappings": {
    "doc": {
      "dynamic": true,
      "properties": {
        "name": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "address": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "remark": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "age": {
          "type": "integer"
        },
        "salary": {
          "type": "float"
        },
        "birthDate": {
          "type": "date",
          "format": "yyyy-MM-dd"
        },
        "createTime": {
          "type": "date"
        }
      }
    }
  }
}

2、Java 代码示例

1. import lombok.extern.slf4j.Slf4j;
2. import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
3. import org.elasticsearch.action.admin.indices.create.CreateIndexResponse;
4. import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
5. import org.elasticsearch.action.support.master.AcknowledgedResponse;
6. import org.elasticsearch.client.RequestOptions;
7. import org.elasticsearch.client.RestHighLevelClient;
8. import org.elasticsearch.common.settings.Settings;
9. import org.elasticsearch.common.xcontent.XContentBuilder;
10. import org.elasticsearch.common.xcontent.XContentFactory;
11. import org.springframework.beans.factory.annotation.Autowired;
12. import org.springframework.stereotype.Service;
13. import java.io.IOException;
14.  
15. @Slf4j
16. @Service
17. public class IndexService2 {
18.  
19. @Autowired
20. private RestHighLevelClient restHighLevelClient;
21.  
22. /**
23. * 创建索引
24. */
25. public void createIndex() {
26. try {
27. // 创建 Mapping
28. XContentBuilder mapping = XContentFactory.jsonBuilder()
29. .startObject()
30. .field("dynamic", true)
31. .startObject("properties")
32. .startObject("name")
33. .field("type","text")
34. .startObject("fields")
35. .startObject("keyword")
36. .field("type","keyword")
37. .endObject()
38. .endObject()
39. .endObject()
40. .startObject("address")
41. .field("type","text")
42. .startObject("fields")
43. .startObject("keyword")
44. .field("type","keyword")
45. .endObject()
46. .endObject()
47. .endObject()
48. .startObject("remark")
49. .field("type","text")
50. .startObject("fields")
51. .startObject("keyword")
52. .field("type","keyword")
53. .endObject()
54. .endObject()
55. .endObject()
56. .startObject("age")
57. .field("type","integer")
58. .endObject()
59. .startObject("salary")
60. .field("type","float")
61. .endObject()
62. .startObject("birthDate")
63. .field("type","date")
64. .field("format", "yyyy-MM-dd")
65. .endObject()
66. .startObject("createTime")
67. .field("type","date")
68. .endObject()
69. .endObject()
70. .endObject();
71. // 创建索引配置信息，配置
72. Settings settings = Settings.builder()
73. .put("index.number_of_shards", 1)
74. .put("index.number_of_replicas", 0)
75. .build();
76. // 新建创建索引请求对象，然后设置索引类型（ES 7.0 将不存在索引类型）和 mapping 与 index 配置
77. CreateIndexRequest request = new CreateIndexRequest("mydlq-user", settings);
78. request.mapping("doc", mapping);
79. // RestHighLevelClient 执行创建索引
80. CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);
81. // 判断是否创建成功
82. boolean isCreated = createIndexResponse.isAcknowledged();
83. log.info("是否创建成功：{}", isCreated);
84. } catch (IOException e) {
85. log.error("", e);
86. }
87. }
88.  
89. /**
90. * 删除索引
91. */
92. public void deleteIndex() {
93. try {
94. // 新建删除索引请求对象
95. DeleteIndexRequest request = new DeleteIndexRequest("mydlq-user");
96. // 执行删除索引
97. AcknowledgedResponse acknowledgedResponse = restHighLevelClient.indices().delete(request, RequestOptions.DEFAULT);
98. // 判断是否删除成功
99. boolean siDeleted = acknowledgedResponse.isAcknowledged();
100. log.info("是否删除成功：{}", siDeleted);
101. } catch (IOException e) {
102. log.error("", e);
103. }
104. }
105.  
106. }

五、文档操作示例

1、Restful 操作示例

在索引 mydlq-user 中增加一条文档信息。

POST /mydlq-user/
{
    "address": "北京市",
    "age": 29,
    "birthDate": "1990-01-10",
    "createTime": 1579530727699,
    "name": "张三",
    "remark": "来自北京市的张先生",
    "salary": 100
}

 

获取文档信息
获取 mydlq-user 的索引 id=1 的文档信息。

GET /mydlq-user/doc/1

更新文档信息
更新之前创建的 id=1 的文档信息。

PUT /mydlq-user/1
{
    "address": "北京市海淀区",
    "age": 29,
    "birthDate": "1990-01-10",
    "createTime": 1579530727699,
    "name": "张三",
    "remark": "来自北京市的张先生",
    "salary": 100
} 

删除文档信息
删除之前创建的 id=1 的文档信息。

DELETE /mydlq-user/1

2、Java 代码示例

1. import club.mydlq.elasticsearch.model.entity.UserInfo;
2. import com.alibaba.fastjson.JSON;
3. import lombok.extern.slf4j.Slf4j;
4. import org.elasticsearch.action.delete.DeleteRequest;
5. import org.elasticsearch.action.delete.DeleteResponse;
6. import org.elasticsearch.action.get.GetRequest;
7. import org.elasticsearch.action.get.GetResponse;
8. import org.elasticsearch.action.index.IndexRequest;
9. import org.elasticsearch.action.index.IndexResponse;
10. import org.elasticsearch.action.update.UpdateRequest;
11. import org.elasticsearch.action.update.UpdateResponse;
12. import org.elasticsearch.client.RequestOptions;
13. import org.elasticsearch.client.RestHighLevelClient;
14. import org.elasticsearch.common.xcontent.XContentType;
15. import org.springframework.beans.factory.annotation.Autowired;
16. import org.springframework.stereotype.Service;
17. import java.io.IOException;
18. import java.util.Date;
19.  
20. @Slf4j
21. @Service
22. public class IndexService {
23.  
24. @Autowired
25. private RestHighLevelClient restHighLevelClient;
26.  
27. /**
28. * 增加文档信息
29. */
30. public void addDocument() {
31. try {
32. // 创建索引请求对象
33. IndexRequest indexRequest = new IndexRequest("mydlq-user", "doc", "1");
34. // 创建员工信息
35. UserInfo userInfo = new UserInfo();
36. userInfo.setName("张三");
37. userInfo.setAge(29);
38. userInfo.setSalary(100.00f);
39. userInfo.setAddress("北京市");
40. userInfo.setRemark("来自北京市的张先生");
41. userInfo.setCreateTime(new Date());
42. userInfo.setBirthDate("1990-01-10");
43. // 将对象转换为 byte 数组
44. byte[] json = JSON.toJSONBytes(userInfo);
45. // 设置文档内容
46. indexRequest.source(json, XContentType.JSON);
47. // 执行增加文档
48. IndexResponse response = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
49. log.info("创建状态：{}", response.status());
50. } catch (Exception e) {
51. log.error("", e);
52. }
53. }
54.  
55. /**
56. * 获取文档信息
57. */
58. public void getDocument() {
59. try {
60. // 获取请求对象
61. GetRequest getRequest = new GetRequest("mydlq-user", "doc", "1");
62. // 获取文档信息
63. GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);
64. // 将 JSON 转换成对象
65. if (getResponse.isExists()) {
66. UserInfo userInfo = JSON.parseObject(getResponse.getSourceAsBytes(), UserInfo.class);
67. log.info("员工信息：{}", userInfo);
68. }
69. } catch (IOException e) {
70. log.error("", e);
71. }
72. }
73.  
74. /**
75. * 更新文档信息
76. */
77. public void updateDocument() {
78. try {
79. // 创建索引请求对象
80. UpdateRequest updateRequest = new UpdateRequest("mydlq-user", "doc", "1");
81. // 设置员工更新信息
82. UserInfo userInfo = new UserInfo();
83. userInfo.setSalary(200.00f);
84. userInfo.setAddress("北京市海淀区");
85. // 将对象转换为 byte 数组
86. byte[] json = JSON.toJSONBytes(userInfo);
87. // 设置更新文档内容
88. updateRequest.doc(json, XContentType.JSON);
89. // 执行更新文档
90. UpdateResponse response = restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
91. log.info("创建状态：{}", response.status());
92. } catch (Exception e) {
93. log.error("", e);
94. }
95. }
96.  
97. /**
98. * 删除文档信息
99. */
100. public void deleteDocument() {
101. try {
102. // 创建删除请求对象
103. DeleteRequest deleteRequest = new DeleteRequest("mydlq-user", "doc", "1");
104. // 执行删除文档
105. DeleteResponse response = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
106. log.info("删除状态：{}", response.status());
107. } catch (IOException e) {
108. log.error("", e);
109. }
110. }
111.  
112. }

 

六、查询操作示例

1、精确查询(term)

(1) Restful 操作示例
精确查询精确查询
查询地址为 北京市通州区 的人员信息：查询条件不会进行分词，但是查询内容可能会分词，导致查询不到。之前在创建索引时设置 Mapping 中 address 字段存在 keyword 字段是专门用于不分词查询的子字段。

GET mydlq-user/_search
{
  "query": {
    "term": {
      "address.keyword": {
        "value": "北京市通州区"
      }
    }
  }
}

精确查询-多内容查询精确查询
查询地址为 北京市丰台区、北京市昌平区 或 北京市大兴区 的人员信息：

GET mydlq-user/_search
{
  "query": {
    "terms": {
      "address.keyword": [
        "北京市丰台区",
        "北京市昌平区",
        "北京市大兴区"
      ]
    }
  }
}

(2) Java 代码示例

1. import club.mydlq.elasticsearch.model.entity.UserInfo;
2. import com.alibaba.fastjson.JSON;
3. import lombok.extern.slf4j.Slf4j;
4. import org.elasticsearch.action.search.SearchRequest;
5. import org.elasticsearch.action.search.SearchResponse;
6. import org.elasticsearch.client.RequestOptions;
7. import org.elasticsearch.client.RestHighLevelClient;
8. import org.elasticsearch.index.query.QueryBuilders;
9. import org.elasticsearch.rest.RestStatus;
10. import org.elasticsearch.search.SearchHit;
11. import org.elasticsearch.search.SearchHits;
12. import org.elasticsearch.search.builder.SearchSourceBuilder;
13. import org.springframework.beans.factory.annotation.Autowired;
14. import org.springframework.stereotype.Service;
15. import java.io.IOException;
16.  
17. @Slf4j
18. @Service
19. public class TermQueryService {
20.  
21. @Autowired
22. private RestHighLevelClient restHighLevelClient;
23.  
24. /**
25. * 精确查询（查询条件不会进行分词，但是查询内容可能会分词，导致查询不到）
26. */
27. public void termQuery() {
28. try {
29. // 构建查询条件（注意：termQuery 支持多种格式查询，如 boolean、int、double、string 等，这里使用的是 string 的查询）
30. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
31. searchSourceBuilder.query(QueryBuilders.termQuery("address.keyword", "北京市通州区"));
32. // 创建查询请求对象，将查询对象配置到其中
33. SearchRequest searchRequest = new SearchRequest("mydlq-user");
34. searchRequest.source(searchSourceBuilder);
35. // 执行查询，然后处理响应结果
36. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
37. // 根据状态和数据条数验证是否返回了数据
38. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
39. SearchHits hits = searchResponse.getHits();
40. for (SearchHit hit : hits) {
41. // 将 JSON 转换成对象
42. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
43. // 输出查询信息
44. log.info(userInfo.toString());
45. }
46. }
47. } catch (IOException e) {
48. log.error("", e);
49. }
50. }
51.  
52. /**
53. * 多个内容在一个字段中进行查询
54. */
55. public void termsQuery() {
56. try {
57. // 构建查询条件（注意：termsQuery 支持多种格式查询，如 boolean、int、double、string 等，这里使用的是 string 的查询）
58. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
59. searchSourceBuilder.query(QueryBuilders.termsQuery("address.keyword", "北京市丰台区", "北京市昌平区", "北京市大兴区"));
60. // 创建查询请求对象，将查询对象配置到其中
61. SearchRequest searchRequest = new SearchRequest("mydlq-user");
62. searchRequest.source(searchSourceBuilder);
63. // 执行查询，然后处理响应结果
64. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
65. // 根据状态和数据条数验证是否返回了数据
66. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
67. SearchHits hits = searchResponse.getHits();
68. for (SearchHit hit : hits) {
69. // 将 JSON 转换成对象
70. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
71. // 输出查询信息
72. log.info(userInfo.toString());
73. }
74. }
75. } catch (IOException e) {
76. log.error("", e);
77. }
78. }
79.  
80. }

2、匹配查询(match)

(1) Restful 操作示例
匹配查询全部数据与分页
匹配查询符合条件的所有数据，并且设置以 salary 字段升序排序，并设置分页：

GET mydlq-user/_search
{
  "query": {
    "match_all": {}
  },
  "from": 0,
  "size": 10,
  "sort": [
    {
      "salary": {
        "order": "asc"
      }
    }
  ]
}

匹配查询数据
匹配查询地址为 通州区 的数据：

GET mydlq-user/_search
{
  "query": {
    "match": {
      "address": "通州区"
    }
  }
}

词语匹配查询
词语匹配进行查询，匹配 address 中为 北京市通州区 的员工信息：

GET mydlq-user/_search
{
  "query": {
    "match_phrase": {
      "address": "北京市通州区"
    }
  }
}

内容多字段查询
查询在字段 address、remark 中存在 北京 内容的员工信息：

GET mydlq-user/_search
{
  "query": {
    "multi_match": {
      "query": "北京",
      "fields": ["address","remark"]
    }
  }
}

(2) Java 代码示例

1. import club.mydlq.elasticsearch.model.entity.UserInfo;
2. import com.alibaba.fastjson.JSON;
3. import lombok.extern.slf4j.Slf4j;
4. import org.elasticsearch.action.search.SearchRequest;
5. import org.elasticsearch.action.search.SearchResponse;
6. import org.elasticsearch.client.RequestOptions;
7. import org.elasticsearch.client.RestHighLevelClient;
8. import org.elasticsearch.index.query.MatchAllQueryBuilder;
9. import org.elasticsearch.index.query.QueryBuilders;
10. import org.elasticsearch.rest.RestStatus;
11. import org.elasticsearch.search.SearchHit;
12. import org.elasticsearch.search.SearchHits;
13. import org.elasticsearch.search.builder.SearchSourceBuilder;
14. import org.elasticsearch.search.sort.SortOrder;
15. import org.springframework.beans.factory.annotation.Autowired;
16. import org.springframework.stereotype.Service;
17. import java.io.IOException;
18.  
19. @Slf4j
20. @Service
21. public class MatchQueryService {
22.  
23. @Autowired
24. private RestHighLevelClient restHighLevelClient;
25.  
26. /**
27. * 匹配查询符合条件的所有数据，并设置分页
28. */
29. public Object matchAllQuery() {
30. try {
31. // 构建查询条件
32. MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
33. // 创建查询源构造器
34. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
35. searchSourceBuilder.query(matchAllQueryBuilder);
36. // 设置分页
37. searchSourceBuilder.from(0);
38. searchSourceBuilder.size(3);
39. // 设置排序
40. searchSourceBuilder.sort("salary", SortOrder.ASC);
41. // 创建查询请求对象，将查询对象配置到其中
42. SearchRequest searchRequest = new SearchRequest("mydlq-user");
43. searchRequest.source(searchSourceBuilder);
44. // 执行查询，然后处理响应结果
45. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
46. // 根据状态和数据条数验证是否返回了数据
47. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
48. SearchHits hits = searchResponse.getHits();
49. for (SearchHit hit : hits) {
50. // 将 JSON 转换成对象
51. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
52. // 输出查询信息
53. log.info(userInfo.toString());
54. }
55. }
56. } catch (IOException e) {
57. log.error("", e);
58. }
59. }
60.  
61. /**
62. * 匹配查询数据
63. */
64. public Object matchQuery() {
65. try {
66. // 构建查询条件
67. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
68. searchSourceBuilder.query(QueryBuilders.matchQuery("address", "*通州区"));
69. // 创建查询请求对象，将查询对象配置到其中
70. SearchRequest searchRequest = new SearchRequest("mydlq-user");
71. searchRequest.source(searchSourceBuilder);
72. // 执行查询，然后处理响应结果
73. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
74. // 根据状态和数据条数验证是否返回了数据
75. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
76. SearchHits hits = searchResponse.getHits();
77. for (SearchHit hit : hits) {
78. // 将 JSON 转换成对象
79. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
80. // 输出查询信息
81. log.info(userInfo.toString());
82. }
83. }
84. } catch (IOException e) {
85. log.error("", e);
86. }
87. }
88.  
89. /**
90. * 词语匹配查询
91. */
92. public Object matchPhraseQuery() {
93. try {
94. // 构建查询条件
95. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
96. searchSourceBuilder.query(QueryBuilders.matchPhraseQuery("address", "北京市通州区"));
97. // 创建查询请求对象，将查询对象配置到其中
98. SearchRequest searchRequest = new SearchRequest("mydlq-user");
99. searchRequest.source(searchSourceBuilder);
100. // 执行查询，然后处理响应结果
101. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
102. // 根据状态和数据条数验证是否返回了数据
103. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
104. SearchHits hits = searchResponse.getHits();
105. for (SearchHit hit : hits) {
106. // 将 JSON 转换成对象
107. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
108. // 输出查询信息
109. log.info(userInfo.toString());
110. }
111. }
112. } catch (IOException e) {
113. log.error("", e);
114. }
115. }
116.  
117. /**
118. * 内容在多字段中进行查询
119. */
120. public Object matchMultiQuery() {
121. try {
122. // 构建查询条件
123. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
124. searchSourceBuilder.query(QueryBuilders.multiMatchQuery("北京市", "address", "remark"));
125. // 创建查询请求对象，将查询对象配置到其中
126. SearchRequest searchRequest = new SearchRequest("mydlq-user");
127. searchRequest.source(searchSourceBuilder);
128. // 执行查询，然后处理响应结果
129. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
130. // 根据状态和数据条数验证是否返回了数据
131. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
132. SearchHits hits = searchResponse.getHits();
133. for (SearchHit hit : hits) {
134. // 将 JSON 转换成对象
135. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
136. // 输出查询信息
137. log.info(userInfo.toString());
138. }
139. }
140. } catch (IOException e) {
141. log.error("", e);
142. }
143. }
144.  
145. }

 

3、模糊查询(fuzzy)

(1) Restful 操作示例
模糊查询所有以 三 结尾的姓名

GET mydlq-user/_search
{
  "query": {
    "fuzzy": {
      "name": "三"
    }
  }
}

(2) Java 代码示例

1. import club.mydlq.elasticsearch.model.entity.UserInfo;
2. import com.alibaba.fastjson.JSON;
3. import lombok.extern.slf4j.Slf4j;
4. import org.elasticsearch.action.search.SearchRequest;
5. import org.elasticsearch.action.search.SearchResponse;
6. import org.elasticsearch.client.RequestOptions;
7. import org.elasticsearch.client.RestHighLevelClient;
8. import org.elasticsearch.common.unit.Fuzziness;
9. import org.elasticsearch.index.query.QueryBuilders;
10. import org.elasticsearch.rest.RestStatus;
11. import org.elasticsearch.search.SearchHit;
12. import org.elasticsearch.search.SearchHits;
13. import org.elasticsearch.search.builder.SearchSourceBuilder;
14. import org.springframework.beans.factory.annotation.Autowired;
15. import org.springframework.stereotype.Service;
16. import java.io.IOException;
17.  
18. @Slf4j
19. @Service
20. public class FuzzyQueryService {
21.  
22. @Autowired
23. private RestHighLevelClient restHighLevelClient;
24.  
25. /**
26. * 模糊查询所有以 “三” 结尾的姓名
27. */
28. public Object fuzzyQuery() {
29. try {
30. // 构建查询条件
31. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
32. searchSourceBuilder.query(QueryBuilders.fuzzyQuery("name", "三").fuzziness(Fuzziness.AUTO));
33. // 创建查询请求对象，将查询对象配置到其中
34. SearchRequest searchRequest = new SearchRequest("mydlq-user");
35. searchRequest.source(searchSourceBuilder);
36. // 执行查询，然后处理响应结果
37. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
38. // 根据状态和数据条数验证是否返回了数据
39. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
40. SearchHits hits = searchResponse.getHits();
41. for (SearchHit hit : hits) {
42. // 将 JSON 转换成对象
43. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
44. // 输出查询信息
45. log.info(userInfo.toString());
46. }
47. }
48. } catch (IOException e) {
49. log.error("", e);
50. }
51. }
52.  
53. }

 

4、范围查询(range)

(1) Restful 操作示例
查询岁数 ≥ 30 岁的员工数据

GET /mydlq-user/_search
{
  "query": {
    "range": {
      "age": {
        "gte": 30
      }
    }
  }
}

查询生日距离现在 30 年间的员工数据：

GET mydlq-user/_search
{
  "query": {
    "range": {
      "birthDate": {
        "gte": "now-30y"
      }
    }
  }
}

(2) Java 代码示例

1. import club.mydlq.elasticsearch.model.entity.UserInfo;
2. import com.alibaba.fastjson.JSON;
3. import lombok.extern.slf4j.Slf4j;
4. import org.elasticsearch.action.search.SearchRequest;
5. import org.elasticsearch.action.search.SearchResponse;
6. import org.elasticsearch.client.RequestOptions;
7. import org.elasticsearch.client.RestHighLevelClient;
8. import org.elasticsearch.index.query.QueryBuilders;
9. import org.elasticsearch.rest.RestStatus;
10. import org.elasticsearch.search.SearchHit;
11. import org.elasticsearch.search.SearchHits;
12. import org.elasticsearch.search.builder.SearchSourceBuilder;
13. import org.springframework.beans.factory.annotation.Autowired;
14. import org.springframework.stereotype.Service;
15. import java.io.IOException;
16.  
17. @Slf4j
18. @Service
19. public class RangeQueryService {
20.  
21. @Autowired
22. private RestHighLevelClient restHighLevelClient;
23.  
24. /**
25. * 查询岁数 ≥ 30 岁的员工数据
26. */
27. public void rangeQuery() {
28. try {
29. // 构建查询条件
30. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
31. searchSourceBuilder.query(QueryBuilders.rangeQuery("age").gte(30));
32. // 创建查询请求对象，将查询对象配置到其中
33. SearchRequest searchRequest = new SearchRequest("mydlq-user");
34. searchRequest.source(searchSourceBuilder);
35. // 执行查询，然后处理响应结果
36. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
37. // 根据状态和数据条数验证是否返回了数据
38. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
39. SearchHits hits = searchResponse.getHits();
40. for (SearchHit hit : hits) {
41. // 将 JSON 转换成对象
42. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
43. // 输出查询信息
44. log.info(userInfo.toString());
45. }
46. }
47. } catch (IOException e) {
48. log.error("", e);
49. }
50. }
51.  
52. /**
53. * 查询距离现在 30 年间的员工数据
54. * [年(y)、月(M)、星期(w)、天(d)、小时(h)、分钟(m)、秒(s)]
55. * 例如：
56. * now-1h 查询一小时内范围
57. * now-1d 查询一天内时间范围
58. * now-1y 查询最近一年内的时间范围
59. */
60. public void dateRangeQuery() {
61. try {
62. // 构建查询条件
63. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
64. // includeLower（是否包含下边界）、includeUpper（是否包含上边界）
65. searchSourceBuilder.query(QueryBuilders.rangeQuery("birthDate")
66. .gte("now-30y").includeLower(true).includeUpper(true));
67. // 创建查询请求对象，将查询对象配置到其中
68. SearchRequest searchRequest = new SearchRequest("mydlq-user");
69. searchRequest.source(searchSourceBuilder);
70. // 执行查询，然后处理响应结果
71. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
72. // 根据状态和数据条数验证是否返回了数据
73. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
74. SearchHits hits = searchResponse.getHits();
75. for (SearchHit hit : hits) {
76. // 将 JSON 转换成对象
77. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
78. // 输出查询信息
79. log.info(userInfo.toString());
80. }
81. }
82. } catch (IOException e) {
83. log.error("", e);
84. }
85. }
86.  
87. }

5、通配符查询(wildcard)

(1) Restful 操作示例
查询所有以 “三” 结尾的姓名：

GET mydlq-user/_search
{
  "query": {
    "wildcard": {
      "name.keyword": {
        "value": "*三"
      }
    }
  }
}

(2) Java 代码示例

1. import club.mydlq.elasticsearch.model.entity.UserInfo;
2. import com.alibaba.fastjson.JSON;
3. import lombok.extern.slf4j.Slf4j;
4. import org.elasticsearch.action.search.SearchRequest;
5. import org.elasticsearch.action.search.SearchResponse;
6. import org.elasticsearch.client.RequestOptions;
7. import org.elasticsearch.client.RestHighLevelClient;
8. import org.elasticsearch.index.query.QueryBuilders;
9. import org.elasticsearch.rest.RestStatus;
10. import org.elasticsearch.search.SearchHit;
11. import org.elasticsearch.search.SearchHits;
12. import org.elasticsearch.search.builder.SearchSourceBuilder;
13. import org.springframework.beans.factory.annotation.Autowired;
14. import org.springframework.stereotype.Service;
15. import java.io.IOException;
16.  
17. @Slf4j
18. @Service
19. public class WildcardQueryService {
20.  
21. @Autowired
22. private RestHighLevelClient restHighLevelClient;
23.  
24. /**
25. * 查询所有以 “三” 结尾的姓名
26. *
27. * *：表示多个字符（0个或多个字符）
28. * ?：表示单个字符
29. */
30. public Object wildcardQuery() {
31. try {
32. // 构建查询条件
33. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
34. searchSourceBuilder.query(QueryBuilders.wildcardQuery("name.keyword", "*三"));
35. // 创建查询请求对象，将查询对象配置到其中
36. SearchRequest searchRequest = new SearchRequest("mydlq-user");
37. searchRequest.source(searchSourceBuilder);
38. // 执行查询，然后处理响应结果
39. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
40. // 根据状态和数据条数验证是否返回了数据
41. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
42. SearchHits hits = searchResponse.getHits();
43. for (SearchHit hit : hits) {
44. // 将 JSON 转换成对象
45. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
46. // 输出查询信息
47. log.info(userInfo.toString());
48. }
49. }
50. } catch (IOException e) {
51. log.error("", e);
52. }
53. }
54.  
55. }

6、布尔查询(bool)

(1) Restful 操作示例
查询出生在 1990-1995 年期间，且地址在 北京市昌平区、北京市大兴区、北京市房山区 的员工信息：

GET /mydlq-user/_search
{
  "query": {
    "bool": {
      "filter": {
        "range": {
          "birthDate": {
            "format": "yyyy",
            "gte": 1990,
            "lte": 1995
          }
        }
      },
      "must": [
        {
          "terms": {
            "address.keyword": [
              "北京市昌平区",
              "北京市大兴区",
              "北京市房山区"
            ]
          }
        }
      ]
    }
  }
}

(2) Java 代码示例

1. import club.mydlq.elasticsearch.model.entity.UserInfo;
2. import com.alibaba.fastjson.JSON;
3. import lombok.extern.slf4j.Slf4j;
4. import org.elasticsearch.action.search.SearchRequest;
5. import org.elasticsearch.action.search.SearchResponse;
6. import org.elasticsearch.client.RequestOptions;
7. import org.elasticsearch.client.RestHighLevelClient;
8. import org.elasticsearch.index.query.BoolQueryBuilder;
9. import org.elasticsearch.index.query.QueryBuilders;
10. import org.elasticsearch.rest.RestStatus;
11. import org.elasticsearch.search.SearchHit;
12. import org.elasticsearch.search.SearchHits;
13. import org.elasticsearch.search.builder.SearchSourceBuilder;
14. import org.springframework.beans.factory.annotation.Autowired;
15. import org.springframework.stereotype.Service;
16. import java.io.IOException;
17.  
18. @Slf4j
19. @Service
20. public class BoolQueryService {
21.  
22. @Autowired
23. private RestHighLevelClient restHighLevelClient;
24.  
25. public Object boolQuery() {
26. try {
27. // 创建 Bool 查询构建器
28. BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
29. // 构建查询条件
30. boolQueryBuilder.must(QueryBuilders.termsQuery("address.keyword", "北京市昌平区", "北京市大兴区", "北京市房山区"))
31. .filter().add(QueryBuilders.rangeQuery("birthDate").format("yyyy").gte("1990").lte("1995"));
32. // 构建查询源构建器
33. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
34. searchSourceBuilder.query(boolQueryBuilder);
35. // 创建查询请求对象，将查询对象配置到其中
36. SearchRequest searchRequest = new SearchRequest("mydlq-user");
37. searchRequest.source(searchSourceBuilder);
38. // 执行查询，然后处理响应结果
39. SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
40. // 根据状态和数据条数验证是否返回了数据
41. if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {
42. SearchHits hits = searchResponse.getHits();
43. for (SearchHit hit : hits) {
44. // 将 JSON 转换成对象
45. UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);
46. // 输出查询信息
47. log.info(userInfo.toString());
48. }
49. }
50. }catch (IOException e){
51. log.error("",e);
52. }
53. }
54.  
55. }

七、聚合查询操作示例

1、Metric 聚合分析

(1) Restful 操作示例
统计员工总数、工资最高值、工资最低值、工资平均工资、工资总和：

GET /mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_stats": {
      "stats": {
        "field": "salary"
      }
    }
  }
}

统计员工工资最低值：

GET /mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_min": {
      "min": {
        "field": "salary"
      }
    }
  }
}

统计员工工资最高值：

GET /mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_max": {
      "max": {
        "field": "salary"
      }
    }
  }
}

统计员工工资平均值：

GET /mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_avg": {
      "avg": {
        "field": "salary"
      }
    }
  }
}

统计员工工资总值：

GET /mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_sum": {
      "sum": {
        "field": "salary"
      }
    }
  }
}

统计员工总数：

GET /mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "employee_count": {
      "value_count": {
        "field": "salary"
      }
    }
  }
}

统计员工工资百分位：

GET /mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_percentiles": {
      "percentiles": {
        "field": "salary"
      }
    }
  }
}

(2) Java 代码示例

1. import lombok.extern.slf4j.Slf4j;
2. import org.elasticsearch.action.search.SearchRequest;
3. import org.elasticsearch.action.search.SearchResponse;
4. import org.elasticsearch.client.RequestOptions;
5. import org.elasticsearch.client.RestHighLevelClient;
6. import org.elasticsearch.rest.RestStatus;
7. import org.elasticsearch.search.aggregations.AggregationBuilder;
8. import org.elasticsearch.search.aggregations.AggregationBuilders;
9. import org.elasticsearch.search.aggregations.Aggregations;
10. import org.elasticsearch.search.aggregations.metrics.avg.ParsedAvg;
11. import org.elasticsearch.search.aggregations.metrics.max.ParsedMax;
12. import org.elasticsearch.search.aggregations.metrics.min.ParsedMin;
13. import org.elasticsearch.search.aggregations.metrics.percentiles.ParsedPercentiles;
14. import org.elasticsearch.search.aggregations.metrics.percentiles.Percentile;
15. import org.elasticsearch.search.aggregations.metrics.stats.ParsedStats;
16. import org.elasticsearch.search.aggregations.metrics.sum.ParsedSum;
17. import org.elasticsearch.search.aggregations.metrics.sum.SumAggregationBuilder;
18. import org.elasticsearch.search.aggregations.metrics.valuecount.ParsedValueCount;
19. import org.elasticsearch.search.builder.SearchSourceBuilder;
20. import org.springframework.beans.factory.annotation.Autowired;
21. import org.springframework.stereotype.Service;
22. import java.io.IOException;
23.  
24. @Slf4j
25. @Service
26. public class AggrMetricService {
27.  
28. @Autowired
29. private RestHighLevelClient restHighLevelClient;
30.  
31. /**
32. * stats 统计员工总数、员工工资最高值、员工工资最低值、员工平均工资、员工工资总和
33. */
34. public Object aggregationStats() {
35. String responseResult = "";
36. try {
37. // 设置聚合条件
38. AggregationBuilder aggr = AggregationBuilders.stats("salary_stats").field("salary");
39. // 查询源构建器
40. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
41. searchSourceBuilder.aggregation(aggr);
42. // 设置查询结果不返回，只返回聚合结果
43. searchSourceBuilder.size(0);
44. // 创建查询请求对象，将查询条件配置到其中
45. SearchRequest request = new SearchRequest("mydlq-user");
46. request.source(searchSourceBuilder);
47. // 执行请求
48. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
49. // 获取响应中的聚合信息
50. Aggregations aggregations = response.getAggregations();
51. // 输出内容
52. if (RestStatus.OK.equals(response.status()) || aggregations != null) {
53. // 转换为 Stats 对象
54. ParsedStats aggregation = aggregations.get("salary_stats");
55. log.info("-------------------------------------------");
56. log.info("聚合信息:");
57. log.info("count：{}", aggregation.getCount());
58. log.info("avg：{}", aggregation.getAvg());
59. log.info("max：{}", aggregation.getMax());
60. log.info("min：{}", aggregation.getMin());
61. log.info("sum：{}", aggregation.getSum());
62. log.info("-------------------------------------------");
63. }
64. // 根据具体业务逻辑返回不同结果，这里为了方便直接将返回响应对象Json串
65. responseResult = response.toString();
66. } catch (IOException e) {
67. log.error("", e);
68. }
69. return responseResult;
70. }
71.  
72. /**
73. * min 统计员工工资最低值
74. */
75. public Object aggregationMin() {
76. String responseResult = "";
77. try {
78. // 设置聚合条件
79. AggregationBuilder aggr = AggregationBuilders.min("salary_min").field("salary");
80. // 查询源构建器
81. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
82. searchSourceBuilder.aggregation(aggr);
83. searchSourceBuilder.size(0);
84. // 创建查询请求对象，将查询条件配置到其中
85. SearchRequest request = new SearchRequest("mydlq-user");
86. request.source(searchSourceBuilder);
87. // 执行请求
88. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
89. // 获取响应中的聚合信息
90. Aggregations aggregations = response.getAggregations();
91. // 输出内容
92. if (RestStatus.OK.equals(response.status()) || aggregations != null) {
93. // 转换为 Min 对象
94. ParsedMin aggregation = aggregations.get("salary_min");
95. log.info("-------------------------------------------");
96. log.info("聚合信息:");
97. log.info("min：{}", aggregation.getValue());
98. log.info("-------------------------------------------");
99. }
100. // 根据具体业务逻辑返回不同结果，这里为了方便直接将返回响应对象Json串
101. responseResult = response.toString();
102. } catch (IOException e) {
103. log.error("", e);
104. }
105. return responseResult;
106. }
107.  
108. /**
109. * max 统计员工工资最高值
110. */
111. public Object aggregationMax() {
112. String responseResult = "";
113. try {
114. // 设置聚合条件
115. AggregationBuilder aggr = AggregationBuilders.max("salary_max").field("salary");
116. // 查询源构建器
117. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
118. searchSourceBuilder.aggregation(aggr);
119. searchSourceBuilder.size(0);
120. // 创建查询请求对象，将查询条件配置到其中
121. SearchRequest request = new SearchRequest("mydlq-user");
122. request.source(searchSourceBuilder);
123. // 执行请求
124. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
125. // 获取响应中的聚合信息
126. Aggregations aggregations = response.getAggregations();
127. // 输出内容
128. if (RestStatus.OK.equals(response.status()) || aggregations != null) {
129. // 转换为 Max 对象
130. ParsedMax aggregation = aggregations.get("salary_max");
131. log.info("-------------------------------------------");
132. log.info("聚合信息:");
133. log.info("max：{}", aggregation.getValue());
134. log.info("-------------------------------------------");
135. }
136. // 根据具体业务逻辑返回不同结果，这里为了方便直接将返回响应对象Json串
137. responseResult = response.toString();
138. } catch (IOException e) {
139. log.error("", e);
140. }
141. return responseResult;
142. }
143.  
144. /**
145. * avg 统计员工工资平均值
146. */
147. public Object aggregationAvg() {
148. String responseResult = "";
149. try {
150. // 设置聚合条件
151. AggregationBuilder aggr = AggregationBuilders.avg("salary_avg").field("salary");
152. // 查询源构建器
153. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
154. searchSourceBuilder.aggregation(aggr);
155. searchSourceBuilder.size(0);
156. // 创建查询请求对象，将查询条件配置到其中
157. SearchRequest request = new SearchRequest("mydlq-user");
158. request.source(searchSourceBuilder);
159. // 执行请求
160. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
161. // 获取响应中的聚合信息
162. Aggregations aggregations = response.getAggregations();
163. // 输出内容
164. if (RestStatus.OK.equals(response.status()) || aggregations != null) {
165. // 转换为 Avg 对象
166. ParsedAvg aggregation = aggregations.get("salary_avg");
167. log.info("-------------------------------------------");
168. log.info("聚合信息:");
169. log.info("avg：{}", aggregation.getValue());
170. log.info("-------------------------------------------");
171. }
172. // 根据具体业务逻辑返回不同结果，这里为了方便直接将返回响应对象Json串
173. responseResult = response.toString();
174. } catch (IOException e) {
175. log.error("", e);
176. }
177. return responseResult;
178. }
179.  
180. /**
181. * sum 统计员工工资总值
182. */
183. public Object aggregationSum() {
184. String responseResult = "";
185. try {
186. // 设置聚合条件
187. SumAggregationBuilder aggr = AggregationBuilders.sum("salary_sum").field("salary");
188. // 查询源构建器
189. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
190. searchSourceBuilder.aggregation(aggr);
191. searchSourceBuilder.size(0);
192. // 创建查询请求对象，将查询条件配置到其中
193. SearchRequest request = new SearchRequest("mydlq-user");
194. request.source(searchSourceBuilder);
195. // 执行请求
196. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
197. // 获取响应中的聚合信息
198. Aggregations aggregations = response.getAggregations();
199. // 输出内容
200. if (RestStatus.OK.equals(response.status()) || aggregations != null) {
201. // 转换为 Sum 对象
202. ParsedSum aggregation = aggregations.get("salary_sum");
203. log.info("-------------------------------------------");
204. log.info("聚合信息:");
205. log.info("sum：{}", String.valueOf((aggregation.getValue())));
206. log.info("-------------------------------------------");
207. }
208. // 根据具体业务逻辑返回不同结果，这里为了方便直接将返回响应对象Json串
209. responseResult = response.toString();
210. } catch (IOException e) {
211. log.error("", e);
212. }
213. return responseResult;
214. }
215.  
216. /**
217. * count 统计员工总数
218. */
219. public Object aggregationCount() {
220. String responseResult = "";
221. try {
222. // 设置聚合条件
223. AggregationBuilder aggr = AggregationBuilders.count("employee_count").field("salary");
224. // 查询源构建器
225. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
226. searchSourceBuilder.aggregation(aggr);
227. searchSourceBuilder.size(0);
228. // 创建查询请求对象，将查询条件配置到其中
229. SearchRequest request = new SearchRequest("mydlq-user");
230. request.source(searchSourceBuilder);
231. // 执行请求
232. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
233. // 获取响应中的聚合信息
234. Aggregations aggregations = response.getAggregations();
235. // 输出内容
236. if (RestStatus.OK.equals(response.status()) || aggregations != null) {
237. // 转换为 ValueCount 对象
238. ParsedValueCount aggregation = aggregations.get("employee_count");
239. log.info("-------------------------------------------");
240. log.info("聚合信息:");
241. log.info("count：{}", aggregation.getValue());
242. log.info("-------------------------------------------");
243. }
244. // 根据具体业务逻辑返回不同结果，这里为了方便直接将返回响应对象Json串
245. responseResult = response.toString();
246. } catch (IOException e) {
247. log.error("", e);
248. }
249. return responseResult;
250. }
251.  
252. /**
253. * percentiles 统计员工工资百分位
254. */
255. public Object aggregationPercentiles() {
256. String responseResult = "";
257. try {
258. // 设置聚合条件
259. AggregationBuilder aggr = AggregationBuilders.percentiles("salary_percentiles").field("salary");
260. // 查询源构建器
261. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
262. searchSourceBuilder.aggregation(aggr);
263. searchSourceBuilder.size(0);
264. // 创建查询请求对象，将查询条件配置到其中
265. SearchRequest request = new SearchRequest("mydlq-user");
266. request.source(searchSourceBuilder);
267. // 执行请求
268. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
269. // 获取响应中的聚合信息
270. Aggregations aggregations = response.getAggregations();
271. // 输出内容
272. if (RestStatus.OK.equals(response.status()) || aggregations != null) {
273. // 转换为 Percentiles 对象
274. ParsedPercentiles aggregation = aggregations.get("salary_percentiles");
275. log.info("-------------------------------------------");
276. log.info("聚合信息:");
277. for (Percentile percentile : aggregation) {
278. log.info("百分位：{}：{}", percentile.getPercent(), percentile.getValue());
279. }
280. log.info("-------------------------------------------");
281. }
282. // 根据具体业务逻辑返回不同结果，这里为了方便直接将返回响应对象Json串
283. responseResult = response.toString();
284. } catch (IOException e) {
285. log.error("", e);
286. }
287. return responseResult;
288. }
289.  
290. }

 

2、Bucket 聚合分析

(1) Restful 操作示例
按岁数进行聚合分桶，统计各个岁数员工的人数

GET mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "age_bucket": {
      "terms": {
        "field": "age",
        "size": "10"
      }
    }
  }
}

按工资范围进行聚合分桶，统计工资在 3000-5000、5000-9000 和 9000 以上的员工信息：

GET mydlq-user/_search
{
  "aggs": {
    "salary_range_bucket": {
      "range": {
        "field": "salary",
        "ranges": [
          {
            "key": "低级员工",
            "to": 3000
          },{
            "key": "中级员工",
            "from": 5000,
            "to": 9000
          },{
            "key": "高级员工",
            "from": 9000
          }
        ]
      }
    }
  }
}

按照时间范围进行分桶，统计 1985-1990 年和 1990-1995 年出生的员工信息：

GET mydlq-user/_search
{
  "size": 10,
  "aggs": {
    "date_range_bucket": {
      "date_range": {
        "field": "birthDate",
        "format": "yyyy",
        "ranges": [
          {
            "key": "出生日期1985-1990的员工",
            "from": "1985",
            "to": "1990"
          },{
            "key": "出生日期1990-1995的员工",
            "from": "1990",
            "to": "1995"
          }
        ]
      }
    }
  }
}

按工资多少进行聚合分桶，设置统计的最小值为 0，最大值为 12000，区段间隔为 3000：

GET mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_histogram": {
      "histogram": {
        "field": "salary",
        "extended_bounds": {
          "min": 0,
          "max": 12000
        },
        "interval": 3000
      }
    }
  }
}

按出生日期进行分桶：

GET mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "birthday_histogram": {
      "date_histogram": {
        "format": "yyyy",
        "field": "birthDate",
        "interval": "year"
      }
    }
  }
}

(2) Java 代码示例

1. import lombok.extern.slf4j.Slf4j;
2. import org.elasticsearch.action.search.SearchRequest;
3. import org.elasticsearch.action.search.SearchResponse;
4. import org.elasticsearch.client.RequestOptions;
5. import org.elasticsearch.client.RestHighLevelClient;
6. import org.elasticsearch.rest.RestStatus;
7. import org.elasticsearch.search.aggregations.AggregationBuilder;
8. import org.elasticsearch.search.aggregations.AggregationBuilders;
9. import org.elasticsearch.search.aggregations.Aggregations;
10. import org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramInterval;
11. import org.elasticsearch.search.aggregations.bucket.histogram.Histogram;
12. import org.elasticsearch.search.aggregations.bucket.range.Range;
13. import org.elasticsearch.search.aggregations.bucket.terms.Terms;
14. import org.elasticsearch.search.builder.SearchSourceBuilder;
15. import org.springframework.beans.factory.annotation.Autowired;
16. import org.springframework.stereotype.Service;
17. import java.io.IOException;
18. import java.util.List;
19.  
20. @Slf4j
21. @Service
22. public class AggrBucketService {
23.  
24. @Autowired
25. private RestHighLevelClient restHighLevelClient;
26.  
27. /**
28. * 按岁数进行聚合分桶
29. */
30. public Object aggrBucketTerms() {
31. try {
32. AggregationBuilder aggr = AggregationBuilders.terms("age_bucket").field("age");
33. // 查询源构建器
34. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
35. searchSourceBuilder.size(10);
36. searchSourceBuilder.aggregation(aggr);
37. // 创建查询请求对象，将查询条件配置到其中
38. SearchRequest request = new SearchRequest("mydlq-user");
39. request.source(searchSourceBuilder);
40. // 执行请求
41. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
42. // 获取响应中的聚合信息
43. Aggregations aggregations = response.getAggregations();
44. // 输出内容
45. if (RestStatus.OK.equals(response.status())) {
46. // 分桶
47. Terms byCompanyAggregation = aggregations.get("age_bucket");
48. List<? extends Terms.Bucket> buckets = byCompanyAggregation.getBuckets();
49. // 输出各个桶的内容
50. log.info("-------------------------------------------");
51. log.info("聚合信息:");
52. for (Terms.Bucket bucket : buckets) {
53. log.info("桶名：{} | 总数：{}", bucket.getKeyAsString(), bucket.getDocCount());
54. }
55. log.info("-------------------------------------------");
56. }
57. } catch (IOException e) {
58. log.error("", e);
59. }
60. }
61.  
62. /**
63. * 按工资范围进行聚合分桶
64. */
65. public Object aggrBucketRange() {
66. try {
67. AggregationBuilder aggr = AggregationBuilders.range("salary_range_bucket")
68. .field("salary")
69. .addUnboundedTo("低级员工", 3000)
70. .addRange("中级员工", 5000, 9000)
71. .addUnboundedFrom("高级员工", 9000);
72. // 查询源构建器
73. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
74. searchSourceBuilder.size(0);
75. searchSourceBuilder.aggregation(aggr);
76. // 创建查询请求对象，将查询条件配置到其中
77. SearchRequest request = new SearchRequest("mydlq-user");
78. request.source(searchSourceBuilder);
79. // 执行请求
80. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
81. // 获取响应中的聚合信息
82. Aggregations aggregations = response.getAggregations();
83. // 输出内容
84. if (RestStatus.OK.equals(response.status())) {
85. // 分桶
86. Range byCompanyAggregation = aggregations.get("salary_range_bucket");
87. List<? extends Range.Bucket> buckets = byCompanyAggregation.getBuckets();
88. // 输出各个桶的内容
89. log.info("-------------------------------------------");
90. log.info("聚合信息:");
91. for (Range.Bucket bucket : buckets) {
92. log.info("桶名：{} | 总数：{}", bucket.getKeyAsString(), bucket.getDocCount());
93. }
94. log.info("-------------------------------------------");
95. }
96. } catch (IOException e) {
97. log.error("", e);
98. }
99. }
100.  
101. /**
102. * 按照时间范围进行分桶
103. */
104. public Object aggrBucketDateRange() {
105. try {
106. AggregationBuilder aggr = AggregationBuilders.dateRange("date_range_bucket")
107. .field("birthDate")
108. .format("yyyy")
109. .addRange("1985-1990", "1985", "1990")
110. .addRange("1990-1995", "1990", "1995");
111. // 查询源构建器
112. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
113. searchSourceBuilder.size(0);
114. searchSourceBuilder.aggregation(aggr);
115. // 创建查询请求对象，将查询条件配置到其中
116. SearchRequest request = new SearchRequest("mydlq-user");
117. request.source(searchSourceBuilder);
118. // 执行请求
119. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
120. // 获取响应中的聚合信息
121. Aggregations aggregations = response.getAggregations();
122. // 输出内容
123. if (RestStatus.OK.equals(response.status())) {
124. // 分桶
125. Range byCompanyAggregation = aggregations.get("date_range_bucket");
126. List<? extends Range.Bucket> buckets = byCompanyAggregation.getBuckets();
127. // 输出各个桶的内容
128. log.info("-------------------------------------------");
129. log.info("聚合信息:");
130. for (Range.Bucket bucket : buckets) {
131. log.info("桶名：{} | 总数：{}", bucket.getKeyAsString(), bucket.getDocCount());
132. }
133. log.info("-------------------------------------------");
134. }
135. } catch (IOException e) {
136. log.error("", e);
137. }
138. }
139.  
140. /**
141. * 按工资多少进行聚合分桶
142. */
143. public Object aggrBucketHistogram() {
144. try {
145. AggregationBuilder aggr = AggregationBuilders.histogram("salary_histogram")
146. .field("salary")
147. .extendedBounds(0, 12000)
148. .interval(3000);
149. // 查询源构建器
150. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
151. searchSourceBuilder.size(0);
152. searchSourceBuilder.aggregation(aggr);
153. // 创建查询请求对象，将查询条件配置到其中
154. SearchRequest request = new SearchRequest("mydlq-user");
155. request.source(searchSourceBuilder);
156. // 执行请求
157. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
158. // 获取响应中的聚合信息
159. Aggregations aggregations = response.getAggregations();
160. // 输出内容
161. if (RestStatus.OK.equals(response.status())) {
162. // 分桶
163. Histogram byCompanyAggregation = aggregations.get("salary_histogram");
164. List<? extends Histogram.Bucket> buckets = byCompanyAggregation.getBuckets();
165. // 输出各个桶的内容
166. log.info("-------------------------------------------");
167. log.info("聚合信息:");
168. for (Histogram.Bucket bucket : buckets) {
169. log.info("桶名：{} | 总数：{}", bucket.getKeyAsString(), bucket.getDocCount());
170. }
171. log.info("-------------------------------------------");
172. }
173. } catch (IOException e) {
174. log.error("", e);
175. }
176. }
177.  
178. /**
179. * 按出生日期进行分桶
180. */
181. public Object aggrBucketDateHistogram() {
182. try {
183. AggregationBuilder aggr = AggregationBuilders.dateHistogram("birthday_histogram")
184. .field("birthDate")
185. .interval(1)
186. .dateHistogramInterval(DateHistogramInterval.YEAR)
187. .format("yyyy");
188. // 查询源构建器
189. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
190. searchSourceBuilder.size(0);
191. searchSourceBuilder.aggregation(aggr);
192. // 创建查询请求对象，将查询条件配置到其中
193. SearchRequest request = new SearchRequest("mydlq-user");
194. request.source(searchSourceBuilder);
195. // 执行请求
196. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
197. // 获取响应中的聚合信息
198. Aggregations aggregations = response.getAggregations();
199. // 输出内容
200. if (RestStatus.OK.equals(response.status())) {
201. // 分桶
202. Histogram byCompanyAggregation = aggregations.get("birthday_histogram");
203.  
204. List<? extends Histogram.Bucket> buckets = byCompanyAggregation.getBuckets();
205. // 输出各个桶的内容
206. log.info("-------------------------------------------");
207. log.info("聚合信息:");
208. for (Histogram.Bucket bucket : buckets) {
209. log.info("桶名：{} | 总数：{}", bucket.getKeyAsString(), bucket.getDocCount());
210. }
211. log.info("-------------------------------------------");
212. }
213. } catch (IOException e) {
214. log.error("", e);
215. }
216. }
217.  
218. }

3、Metric 与 Bucket 聚合分析

(1) Restful 操作示例
按照员工岁数分桶、然后统计每个岁数员工工资最高值:

GET mydlq-user/_search
{
  "size": 0,
  "aggs": {
    "salary_bucket": {
      "terms": {
        "field": "age",
        "size": "10"
      },
      "aggs": {
        "salary_max_user": {
          "top_hits": {
            "size": 1,
            "sort": [
              {
                "salary": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}

(2) Java 代码示例

1. import lombok.extern.slf4j.Slf4j;
2. import org.elasticsearch.action.search.SearchRequest;
3. import org.elasticsearch.action.search.SearchResponse;
4. import org.elasticsearch.client.RequestOptions;
5. import org.elasticsearch.client.RestHighLevelClient;
6. import org.elasticsearch.rest.RestStatus;
7. import org.elasticsearch.search.SearchHit;
8. import org.elasticsearch.search.aggregations.AggregationBuilder;
9. import org.elasticsearch.search.aggregations.AggregationBuilders;
10. import org.elasticsearch.search.aggregations.Aggregations;
11. import org.elasticsearch.search.aggregations.bucket.terms.Terms;
12. import org.elasticsearch.search.aggregations.metrics.tophits.ParsedTopHits;
13. import org.elasticsearch.search.builder.SearchSourceBuilder;
14. import org.elasticsearch.search.sort.SortOrder;
15. import org.springframework.beans.factory.annotation.Autowired;
16. import org.springframework.stereotype.Service;
17. import java.io.IOException;
18. import java.util.List;
19.  
20. @Slf4j
21. @Service
22. public class AggrBucketMetricService {
23.  
24. @Autowired
25. private RestHighLevelClient restHighLevelClient;
26.  
27. /**
28. * topHits 按岁数分桶、然后统计每个员工工资最高值
29. */
30. public Object aggregationTopHits() {
31. try {
32. AggregationBuilder testTop = AggregationBuilders.topHits("salary_max_user")
33. .size(1)
34. .sort("salary", SortOrder.DESC);
35. AggregationBuilder salaryBucket = AggregationBuilders.terms("salary_bucket")
36. .field("age")
37. .size(10);
38. salaryBucket.subAggregation(testTop);
39. // 查询源构建器
40. SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
41. searchSourceBuilder.size(0);
42. searchSourceBuilder.aggregation(salaryBucket);
43. // 创建查询请求对象，将查询条件配置到其中
44. SearchRequest request = new SearchRequest("mydlq-user");
45. request.source(searchSourceBuilder);
46. // 执行请求
47. SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
48. // 获取响应中的聚合信息
49. Aggregations aggregations = response.getAggregations();
50. // 输出内容
51. if (RestStatus.OK.equals(response.status())) {
52. // 分桶
53. Terms byCompanyAggregation = aggregations.get("salary_bucket");
54. List<? extends Terms.Bucket> buckets = byCompanyAggregation.getBuckets();
55. // 输出各个桶的内容
56. log.info("-------------------------------------------");
57. log.info("聚合信息:");
58. for (Terms.Bucket bucket : buckets) {
59. log.info("桶名：{}", bucket.getKeyAsString());
60. ParsedTopHits topHits = bucket.getAggregations().get("salary_max_user");
61. for (SearchHit hit:topHits.getHits()){
62. log.info(hit.getSourceAsString());
63. }
64. }
65. log.info("-------------------------------------------");
66. }
67. } catch (IOException e) {
68. log.error("", e);
69. }
70. }
71.  
72. }