Elasticsearch Java高级客户端
1. 概述
Java REST Client 有两种风格：
- Java Low Level REST Client ：用于Elasticsearch的官方低级客户端。它允许通过http与Elasticsearch集群通信。将请求编排和响应反编排留给用户自己处理。它兼容所有的Elasticsearch版本。（PS：学过WebService的话，对编排与反编排这个概念应该不陌生。可以理解为对请求参数的封装，以及对响应结果的解析）
- Java High Level REST Client ：用于Elasticsearch的官方高级客户端。它是基于低级客户端的，它提供很多API，并负责请求的编排与响应的反编排。（PS：就好比是，一个是传自己拼接好的字符串，并且自己解析返回的结果；而另一个是传对象，返回的结果也已经封装好了，直接是对象，更加规范了参数的名称以及格式，更加面对对象一点）
（PS：所谓低级与高级，我觉得一个很形象的比喻是，面向过程编程与面向对象编程）
在 Elasticsearch 7.0 中不建议使用TransportClient，并且在8.0中会完全删除TransportClient。因此，官方更建议我们用Java High Level REST Client，它执行HTTP请求，而不是序列号的Java请求。既然如此，这里就直接用高级了。
2. Java High Level REST Client （高级REST客户端）
2.1. Maven仓库
<dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>6.5.4</version> </dependency>
2.2. 依赖
- org.elasticsearch.client:elasticsearch-rest-client
- org.elasticsearch:elasticsearch
2.3. 初始化
RestHighLevelClient client = new RestHighLevelClient( RestClient.builder( new HttpHost("localhost", 9200, "http"), new HttpHost("localhost", 9201, "http")));
高级客户端内部会创建低级客户端用于基于提供的builder执行请求。低级客户端维护一个连接池，并启动一些线程，因此当你用完以后应该关闭高级客户端，并且在内部它将会关闭低级客户端，以释放这些资源。关闭客户端可以使用close()方法：
client.close();
2.4. 文档API
2.4.1. 添加文档
IndexRequest
IndexRequest request = new IndexRequest("posts", "doc", "1"); String jsonString = "{\"user\":\"kimchy\",\"postDate\":\"2013-01-30\",\"message\":\"trying out Elasticsearch\"}"; request.source(jsonString, XContentType.JSON);
提供文档source的方式还有很多，比如：
通过Map的方式提供文档source
通过XContentBuilder方式提供source
通过Object的方式（键值对）提供source
可选参数
同步执行
异步执行
你也可以异步执行 IndexRequest，为此你需要指定一个监听器来处理这个异步响应结果：
一个典型的监听器看起来是这样的：
IndexResponse
如果有版本冲突，将会抛出ElasticsearchException
同样的异常也有可能发生在当opType设置为create的时候，且相同索引、相同类型、相同ID的文档已经存在时。例如：
2.4.2. 查看文档
Get Request
可选参数
同步执行
异步执行
Get Response
当索引不存在，或者指定的文档的版本不存在时，响应状态吗是404，并且抛出ElasticsearchException
2.4.3. 文档是否存在
2.4.4. 删除文档
Delete Request
可选参数
同添加
2.5. 搜索API
Search Request
基本格式是这样的：
大多数查询参数被添加到 SearchSourceBuilder
可选参数
SearchSourceBuilder
控制检索行为的大部分选项都可以在SearchSourceBuilder中设置。下面是一个常见选项的例子：
在这个例子中，我们首先创建了一个SearchSourceBuilder对象，并且带着默认选项。然后设置了一个term查询，接着设置检索的位置和数量，最后设置超时时间
在设置完这些选项以后，我们只需要把SearchSourceBuilder加入到SearchRequest中即可
构建Query
用QueryBuilder来创建Serarch Query。QueryBuilder支持Elasticsearch DSL中每一种Query
例如：
还可以通过QueryBuilders工具类来创建QueryBuilder对象，例如：
无论是用哪种方式创建，最后一定要把QueryBuilder添加到SearchSourceBuilder中
排序
SearchSourceBuilder 可以添加一个或多个 SortBuilder
SortBuilder有四种实现：FieldSortBuilder、GeoDistanceSortBuilder、ScoreSortBuilder、ScriptSortBuilder
聚集函数
同步执行
异步执行
从查询响应中取出文档
3. 示例
3.1. 准备数据
3.1.1. 安装IK分词器插件
./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.5.4/elasticsearch-analysis-ik-6.5.4.zip
3.1.2. 创建索引
curl -X PUT "localhost:9200/book" -H 'Content-Type: application/json' -d' { "mappings":{ "_doc":{ "properties":{ "id":{ "type":"integer" }, "name":{ "type":"text", "analyzer":"ik_max_word", "search_analyzer":"ik_max_word" }, "author":{ "type":"text", "analyzer":"ik_max_word", "search_analyzer":"ik_max_word" }, "category":{ "type":"integer" }, "price":{ "type":"double" }, "status":{ "type":"short" }, "sellReason":{ "type":"text", "analyzer":"ik_max_word", "search_analyzer":"ik_max_word" }, "sellTime":{ "type":"date", "format":"yyyy-MM-dd" } } } } } '
3.1.3. 数据预览
3.2. 示例代码
3.2.1. 完整的pom.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.1.1.RELEASE</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.cjs.example</groupId> <artifactId>elasticsearch-demo</artifactId> <version>0.0.1-SNAPSHOT</version> <name>elasticsearch-demo</name> <description></description> <properties> <java.version>1.8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-thymeleaf</artifactId> </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>6.5.4</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>3.8</version> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.54</version> </dependency> <dependency> <groupId>ch.qos.logback</groupId> <artifactId>logback-core</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build> </project>
3.2.2. 配置
package com.cjs.example.elasticsearch.config; import org.apache.http.HttpHost; import org.elasticsearch.client.RestClient; import org.elasticsearch.client.RestHighLevelClient; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; /** * @author ChengJianSheng * @date 2019-01-07 */ @Configuration public class ElasticsearchClientConfig { @Bean public RestHighLevelClient restHighLevelClient() { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder( new HttpHost("localhost", 9200, "http"))); return client; } }
3.2.3. domain
package com.cjs.example.elasticsearch.domain.model; import lombok.Data; import java.io.Serializable; /** * 图书 * @author ChengJianSheng * @date 2019-01-07 */ @Data public class BookModel implements Serializable { private Integer id; // 图书ID private String name; // 图书名称 private String author; // 作者 private Integer category; // 图书分类 private Double price; // 图书价格 private String sellReason; // 上架理由 private String sellTime; // 上架时间 private Integer status; // 状态（1：可售，0：不可售） }
3.2.4. Controller
package com.cjs.example.elasticsearch.controller; import com.alibaba.fastjson.JSON; import com.cjs.example.elasticsearch.domain.common.BaseResult; import com.cjs.example.elasticsearch.domain.common.Page; import com.cjs.example.elasticsearch.domain.model.BookModel; import com.cjs.example.elasticsearch.domain.vo.BookRequestVO; import com.cjs.example.elasticsearch.service.BookService; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.*; /** * 文档操作 * @author ChengJianSheng * @date 2019-01-07 */ @Slf4j @RestController @RequestMapping("/book") public class BookController { @Autowired private BookService bookService; /** * 列表分页查询 */ @GetMapping("/list") public BaseResult list(BookRequestVO bookRequestVO) { Page<BookModel> page = bookService.list(bookRequestVO); if (null == page) { return BaseResult.error(); } return BaseResult.ok(page); } /** * 查看文档 */ @GetMapping("/detail") public BaseResult detail(Integer id) { if (null == id) { return BaseResult.error("ID不能为空"); } BookModel book = bookService.detail(id); return BaseResult.ok(book); } /** * 添加文档 */ @PostMapping("/add") public BaseResult add(@RequestBody BookModel bookModel) { bookService.save(bookModel); log.info("插入文档成功！请求参数: {}", JSON.toJSONString(bookModel)); return BaseResult.ok(); } /** * 修改文档 */ @PostMapping("/update") public BaseResult update(@RequestBody BookModel bookModel) { Integer id = bookModel.getId(); if (null == id) { return BaseResult.error("ID不能为空"); } BookModel book = bookService.detail(id); if (null == book) { return BaseResult.error("记录不存在"); } bookService.update(bookModel); log.info("更新文档成功！请求参数: {}", JSON.toJSONString(bookModel)); return BaseResult.ok(); } /** * 删除文档 */ @GetMapping("/delete") public BaseResult delete(Integer id) { if (null == id) { return BaseResult.error("ID不能为空"); } bookService.delete(id); return BaseResult.ok(); } }
3.2.5. Service
package com.cjs.example.elasticsearch.service.impl; import com.alibaba.fastjson.JSON; import com.cjs.example.elasticsearch.domain.common.Page; import com.cjs.example.elasticsearch.domain.model.BookModel; import com.cjs.example.elasticsearch.domain.vo.BookRequestVO; import com.cjs.example.elasticsearch.service.BookService; import lombok.extern.slf4j.Slf4j; import org.apache.commons.lang3.StringUtils; import org.elasticsearch.action.ActionListener; import org.elasticsearch.action.DocWriteResponse; import org.elasticsearch.action.delete.DeleteRequest; import org.elasticsearch.action.delete.DeleteResponse; import org.elasticsearch.action.get.GetRequest; import org.elasticsearch.action.get.GetResponse; import org.elasticsearch.action.index.IndexRequest; import org.elasticsearch.action.index.IndexResponse; import org.elasticsearch.action.search.SearchRequest; import org.elasticsearch.action.search.SearchResponse; import org.elasticsearch.action.support.replication.ReplicationResponse; import org.elasticsearch.action.update.UpdateRequest; import org.elasticsearch.action.update.UpdateResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.common.unit.TimeValue; import org.elasticsearch.index.query.BoolQueryBuilder; import org.elasticsearch.index.query.QueryBuilders; import org.elasticsearch.rest.RestStatus; import org.elasticsearch.search.SearchHit; import org.elasticsearch.search.SearchHits; import org.elasticsearch.search.builder.SearchSourceBuilder; import org.elasticsearch.search.sort.FieldSortBuilder; import org.elasticsearch.search.sort.SortOrder; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import org.springframework.util.CollectionUtils; import java.io.IOException; import java.util.*; import java.util.stream.Collectors; /** * @author ChengJianSheng * @date 2019-01-07 */ @Slf4j @Service public class BookServiceImpl implements BookService { private static final String INDEX_NAME = "book"; private static final String INDEX_TYPE = "_doc"; @Autowired private RestHighLevelClient client; @Override public Page<BookModel> list(BookRequestVO bookRequestVO) { int pageNo = bookRequestVO.getPageNo(); int pageSize = bookRequestVO.getPageSize(); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.from(pageNo - 1); sourceBuilder.size(pageSize); sourceBuilder.sort(new FieldSortBuilder("id").order(SortOrder.ASC)); // sourceBuilder.query(QueryBuilders.matchAllQuery()); BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); if (StringUtils.isNotBlank(bookRequestVO.getName())) { boolQueryBuilder.must(QueryBuilders.matchQuery("name", bookRequestVO.getName())); } if (StringUtils.isNotBlank(bookRequestVO.getAuthor())) { boolQueryBuilder.must(QueryBuilders.matchQuery("author", bookRequestVO.getAuthor())); } if (null != bookRequestVO.getStatus()) { boolQueryBuilder.must(QueryBuilders.termQuery("status", bookRequestVO.getStatus())); } if (StringUtils.isNotBlank(bookRequestVO.getSellTime())) { boolQueryBuilder.must(QueryBuilders.termQuery("sellTime", bookRequestVO.getSellTime())); } if (StringUtils.isNotBlank(bookRequestVO.getCategories())) { String[] categoryArr = bookRequestVO.getCategories().split(","); List<Integer> categoryList = Arrays.asList(categoryArr).stream().map(e->Integer.valueOf(e)).collect(Collectors.toList()); BoolQueryBuilder categoryBoolQueryBuilder = QueryBuilders.boolQuery(); for (Integer category : categoryList) { categoryBoolQueryBuilder.should(QueryBuilders.termQuery("category", category)); } boolQueryBuilder.must(categoryBoolQueryBuilder); } sourceBuilder.query(boolQueryBuilder); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices(INDEX_NAME); searchRequest.source(sourceBuilder); try { SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT); RestStatus restStatus = searchResponse.status(); if (restStatus != RestStatus.OK) { return null; } List<BookModel> list = new ArrayList<>(); SearchHits searchHits = searchResponse.getHits(); for (SearchHit hit : searchHits.getHits()) { String source = hit.getSourceAsString(); BookModel book = JSON.parseObject(source, BookModel.class); list.add(book); } long totalHits = searchHits.getTotalHits(); Page<BookModel> page = new Page<>(pageNo, pageSize, totalHits, list); TimeValue took = searchResponse.getTook(); log.info("查询成功！请求参数: {}, 用时{}毫秒", searchRequest.source().toString(), took.millis()); return page; } catch (IOException e) { log.error("查询失败！原因: {}", e.getMessage(), e); } return null; } @Override public void save(BookModel bookModel) { Map<String, Object> jsonMap = new HashMap<>(); jsonMap.put("id", bookModel.getId()); jsonMap.put("name", bookModel.getName()); jsonMap.put("author", bookModel.getAuthor()); jsonMap.put("category", bookModel.getCategory()); jsonMap.put("price", bookModel.getPrice()); jsonMap.put("sellTime", bookModel.getSellTime()); jsonMap.put("sellReason", bookModel.getSellReason()); jsonMap.put("status", bookModel.getStatus()); IndexRequest indexRequest = new IndexRequest(INDEX_NAME, INDEX_TYPE, String.valueOf(bookModel.getId())); indexRequest.source(jsonMap); client.indexAsync(indexRequest, RequestOptions.DEFAULT, new ActionListener<IndexResponse>() { @Override public void onResponse(IndexResponse indexResponse) { String index = indexResponse.getIndex(); String type = indexResponse.getType(); String id = indexResponse.getId(); long version = indexResponse.getVersion(); log.info("Index: {}, Type: {}, Id: {}, Version: {}", index, type, id, version); if (indexResponse.getResult() == DocWriteResponse.Result.CREATED) { log.info("写入文档"); } else if (indexResponse.getResult() == DocWriteResponse.Result.UPDATED) { log.info("修改文档"); } ReplicationResponse.ShardInfo shardInfo = indexResponse.getShardInfo(); if (shardInfo.getTotal() != shardInfo.getSuccessful()) { log.warn("部分分片写入成功"); } if (shardInfo.getFailed() > 0) { for (ReplicationResponse.ShardInfo.Failure failure : shardInfo.getFailures()) { String reason = failure.reason(); log.warn("失败原因: {}", reason); } } } @Override public void onFailure(Exception e) { log.error(e.getMessage(), e); } }); } @Override public void update(BookModel bookModel) { Map<String, Object> jsonMap = new HashMap<>(); jsonMap.put("sellReason", bookModel.getSellReason()); UpdateRequest request = new UpdateRequest(INDEX_NAME, INDEX_TYPE, String.valueOf(bookModel.getId())); request.doc(jsonMap); try { UpdateResponse updateResponse = client.update(request, RequestOptions.DEFAULT); } catch (IOException e) { log.error("更新失败！原因: {}", e.getMessage(), e); } } @Override public void delete(int id) { DeleteRequest request = new DeleteRequest(INDEX_NAME, INDEX_TYPE, String.valueOf(id)); try { DeleteResponse deleteResponse = client.delete(request, RequestOptions.DEFAULT); if (deleteResponse.status() == RestStatus.OK) { log.info("删除成功！id: {}", id); } } catch (IOException e) { log.error("删除失败！原因: {}", e.getMessage(), e); } } @Override public BookModel detail(int id) { GetRequest getRequest = new GetRequest(INDEX_NAME, INDEX_TYPE, String.valueOf(id)); try { GetResponse getResponse = client.get(getRequest, RequestOptions.DEFAULT); if (getResponse.isExists()) { String source = getResponse.getSourceAsString(); BookModel book = JSON.parseObject(source, BookModel.class); return book; } } catch (IOException e) { log.error("查看失败！原因: {}", e.getMessage(), e); } return null; } }
3.2.6. 页面
<!DOCTYPE html> <html lang="zh"> <head> <meta charset="UTF-8"> <title>图书列表</title> <link rel="stylesheet" href="/bootstrap-4/css/bootstrap.min.css"> <link rel="stylesheet" href="/bootstrap-table/bootstrap-table.css"> <script src="jquery-3.3.1.min.js"></script> <script src="/bootstrap-4/js/bootstrap.min.js"></script> <script src="/bootstrap-table/bootstrap-table.js"></script> <script src="/bootstrap-table/locale/bootstrap-table-zh-CN.js"></script> <script> $(function(){ $('#table').bootstrapTable({ url: '/book/list', method: 'get', sidePagination: 'server', responseHandler: function(res) { // 加载服务器数据之前的处理程序，可以用来格式化数据。参数：res为从服务器请求到的数据。 var result = {}; result.total = res.data.totalCount; result.rows = res.data.pageList; return result; }, pagination: true, pageSize: 3, // 初始PageSize queryParams: function(params) { var req = { pageSize: params.limit, pageNo: params.offset + 1 }; return req; }, striped: true, search: true, columns: [{ field: 'id', title: 'ID' }, { field: 'name', title: '名称' }, { field: 'author', title: '作者' }, { field: 'price', title: '单价' }, { field: 'sellTime', title: '上架时间' }, { field: 'status', title: '状态', formatter: function(value) { if (value == 1) { return '<span style="color: green">可售</span>'; } else { return '<span style="color: red">不可售</span>'; } } }, { field: 'category', title: '分类', formatter: function(value) { if (value == 10010) { return '中国当代小说'; } else if (value == 10011) { return '武侠小说'; } else if (value == 10012) { return '爱情小说'; } else if (value == 10013) { return '中国当代随笔'; } } }, { field: 'sellReason', title: '上架理由' }, { title: '操作', formatter: function() { return '<a href="#">修改</a> <a href="#">删除</a>'; } } ] }); }); </script> </head> <body> <div class="table-responsive" style="padding: 10px 30px"> <table id="table" class="table text-nowrap"></table> </div> </body> </html>
3.3. 演示
重点演示几个查询
返回结果：
{
"code": 200,
"success": true,
"msg": "SUCCESS",
"data": {
"pageNumber": 1,
"pageSize": 10,
"totalCount": 2,
"pageList": [
{
"id": 2,
"name": "倚天屠龙记（全四册）",
"author": "金庸",
"category": 10011,
"price": 70.4,
"sellReason": "武林至尊，宝刀屠龙，号令天下，莫敢不从。",
"sellTime": "2018-11-11",
"status": 1
},
{
"id": 3,
"name": "神雕侠侣",
"author": "金庸",
"category": 10011,
"price": 70,
"sellReason": "风陵渡口初相遇，一见杨过误终身",
"sellTime": "2018-11-11",
"status": 1
}
]
}
}
上面的查询对应的Elasticsearch DSL是这样的：
{
"from":0,
"size":10,
"query":{
"bool":{
"must":[
{
"match":{
"author":{
"query":"金庸",
"operator":"OR",
"prefix_length":0,
"max_expansions":50,
"fuzzy_transpositions":true,
"lenient":false,
"zero_terms_query":"NONE",
"auto_generate_synonyms_phrase_query":true,
"boost":1
}
}
},
{
"term":{
"status":{
"value":1,
"boost":1
}
}
},
{
"bool":{
"should":[
{
"term":{
"category":{
"value":10010,
"boost":1
}
}
},
{
"term":{
"category":{
"value":10011,
"boost":1
}
}
},
{
"term":{
"category":{
"value":10012,
"boost":1
}
}
}
],
"adjust_pure_negative":true,
"boost":1
}
}
],
"adjust_pure_negative":true,
"boost":1
}
},
"sort":[
{
"id":{
"order":"asc"
}
}
]
}
3.4. 工程结构
4. 参考
https://github.com/medcl/elasticsearch-analysis-ik
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-search.html
https://bootstrap-table.wenzhixin.net.cn/documentation/
5. 其它相关
