Elasticsearch 入门实战(5)--Java API Client 使用一(Index,Document,Ingest,SQL APIs)
本文主要介绍 Elasticsearch Java API Client 的使用,相关的环境及软件信息如下:CentOS 7.6.1810、Java 1.8.0_341(客户端用)、Elasticsearch 8.13.4、elasticsearch-java 8.13.4。
1、Java API Client 的特点
- Strongly typed requests and responses for all Elasticsearch APIs.
- Blocking and asynchronous versions of all APIs.
- Use of fluent builders and functional patterns to allow writing concise yet readable code when creating complex nested structures.
- Seamless integration of application classes by using an object mapper such as Jackson or any JSON-B implementation.
- Delegates protocol handling to an http client such as the Java Low Level REST Client that takes care of all transport-level concerns: HTTP connection pooling, retries, node discovery, and so on.
2、引入依赖
<dependency> <groupId>co.elastic.clients</groupId> <artifactId>elasticsearch-java</artifactId> <version>8.13.4</version> </dependency>
3、使用
Elasticsearch Java API Client 通过 API 的方式来组装请求数据,避免直接编写 JSON 字符串;请求数据的详细说明可参考:Elasticsearch 入门实战(3)--REST API 使用。
3.1、连接及关闭
Java API Client 底层依赖 Java Low Level REST Client,需先创建 Low Level REST Client。
private ElasticsearchTransport transport; private ElasticsearchClient client; @Before public void before() { RestClient restClient = RestClient.builder( new HttpHost("10.49.196.10", 9200), new HttpHost("10.49.196.11", 9200), new HttpHost("10.49.196.12", 9200)).build(); ObjectMapper objectMapper = new ObjectMapper(); transport = new RestClientTransport(restClient, new JacksonJsonpMapper(objectMapper)); client = new ElasticsearchClient(transport); } @After public void after() throws IOException { client.shutdown(); }
3.2、Index APIs
3.2.1、Create index API(创建索引)
/** * 创建索引 */ @Test public void createIndex() throws IOException { CreateIndexResponse response = client.indices().create(builder -> builder .settings(indexSettingsBuilder -> indexSettingsBuilder.numberOfReplicas("1").numberOfShards("2")) .mappings(typeMappingBuilder -> typeMappingBuilder .properties("age", propertyBuilder -> propertyBuilder.integer(integerNumberPropertyBuilder -> integerNumberPropertyBuilder)) .properties("name", propertyBuilder -> propertyBuilder.keyword(keywordPropertyBuilder -> keywordPropertyBuilder)) .properties("poems", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart"))) .properties("about", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart"))) .properties("success", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart"))) ) .index(INDEX_NAME)); log.info("response={}", response); }
3.2.2、Delete index API(删除索引)
/** * 删除索引 */ @Test public void deleteIndex() throws IOException { DeleteIndexResponse response = client.indices().delete(builder -> builder.index(INDEX_NAME)); log.info("response={}", response); }
3.2.3、Get index API(查看索引信息)
/** * 查询索引 */ @Test public void getIndex() throws IOException { //查询所有索引,也可以使用 * GetIndexResponse response = client.indices().get(builder -> builder.index("_all")); log.info("response={}", response); response = client.indices().get(builder -> builder.index(INDEX_NAME));//单个索引 log.info("response={}", response); }
3.2.4、Get mapping API(查询 mapping 信息)
/** * 查询 mapping 信息 */ @Test public void getMapping() throws IOException { //查询所有字段信息 GetMappingResponse response = client.indices().getMapping(builder -> builder.index(INDEX_NAME)); log.info("response={}", response); //查询某些字段信息 GetFieldMappingResponse response2 = client.indices().getFieldMapping(builder -> builder.index(INDEX_NAME).fields("age")); log.info("response2={}", response2); }
3.2.5、Update mapping API(修改 mapping 信息)
/** * 修改 mapping 信息 * 可以新增字段,已有字段只能修改字段的 search_analyzer 属性 */ @Test public void modifyMapping() throws IOException { PutMappingResponse response = client.indices().putMapping(typeMappingBuilder -> typeMappingBuilder .index(INDEX_NAME) .properties("success", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word"))) .properties("desc", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word"))) ); log.info("response={}", response); }
3.2.6、Get index settings API(查询 setting 信息)
/** * 查询 setting 信息 */ @Test public void getSetting() throws IOException { //查询所有参数 GetIndicesSettingsResponse response = client.indices().getSettings(builder -> builder.index(INDEX_NAME)); log.info("response={}", response); //查询单个参数 response = client.indices().getSettings(builder -> builder.index(INDEX_NAME).name("index.number_of_replicas")); log.info("response={}", response); //返回中包含默认参数信息 response = client.indices().getSettings(builder -> builder.index(INDEX_NAME).includeDefaults(true)); log.info("response={}", response); }
3.2.7、Update index settings API(修改 setting 信息)
/** * 修改 setting 信息 */ @Test public void modifySetting() throws IOException { PutIndicesSettingsResponse response = client.indices().putSettings(builder -> builder .index(INDEX_NAME) .settings(indexSettingsBuilder -> indexSettingsBuilder.maxResultWindow(10000))); log.info("response={}", response); }
3.2.8、Analyze API(查看分词器分词结果)
/** * 查看分词器分词结果 */ @Test public void analyze() throws IOException { AnalyzeResponse response = client.indices().analyze(builder -> builder .analyzer("ik_max_word") .text("唐代伟大的现实主义文学作家")); logger.info("response={}", response); }
3.3、Document APIs
3.3.1、Index API(创建/覆盖文档)
/** * 创建/覆盖文档 */ @Test public void createDoc() throws IOException { Map<String, Object> doc = new HashMap<>(); doc.put("age", 30); doc.put("name", "李白"); doc.put("poems", "静夜思"); doc.put("about", "字太白"); doc.put("success", "创造了古代浪漫主义文学高峰、歌行体和七绝达到后人难及的高度"); //没有对应 id 的文档就创建,有就覆盖更新所有字段(相当于先删除再新增)并更新版本号。 IndexResponse response = client.index(builder -> builder.index(INDEX_NAME).id("1").document(doc)); log.info(response.toString()); Poet poet = new Poet(); poet.setAge(40); poet.setName("杜甫"); poet.setPoems("登高"); poet.setAbout("字子美"); poet.setSuccess("唐代伟大的现实主义文学作家,唐诗思想艺术的集大成者"); //不设置 id 将新增,id 将自动生成。 response = client.index(builder -> builder.index(INDEX_NAME).document(poet)); log.info(response.toString()); Poet poet2 = new Poet(31, "杜甫", "登高", "字子美", "唐代伟大的现实主义文学作家,唐诗思想艺术的集大成者"); //新增文档,需要设置 id。 CreateResponse response2 = client.create(builder -> builder.index(INDEX_NAME).id("2").document(poet2)); log.info("response2={}", response2); }
3.3.2、Delete API(删除文档)
/** * 删除文档 */ @Test public void deleteDoc() throws IOException { DeleteResponse response = client.delete(builder -> builder.index(INDEX_NAME).id("2")); log.info(response.toString()); }
3.3.3、Get API(根据 id 查询文档)
/** * 根据 id 查询文档 */ @Test public void id() throws IOException { GetResponse<Map> response = client.get(builder -> builder.index(INDEX_NAME).id("1"), Map.class); log.info("response={}", response); }
3.3.4、Update API(修改文档)
/** * 修改文档,只修改设置的字段 */ @Test public void updateDoc() throws IOException { Map<String, Object> doc = new HashMap<>(); doc.put("age", 33); doc.put("name", "李白2"); UpdateResponse response = client.update(builder -> builder.index(INDEX_NAME).id("10").doc(doc), Map.class); log.info("response={}", response); //文档不存在时插入 Poet poet = new Poet(); poet.setAge(40); poet.setName("杜甫2"); response = client.update(builder -> builder.index(INDEX_NAME).id("11").doc(poet).docAsUpsert(true), Poet.class); log.info("response={}", response); }
3.3.5、Bulk API(批量操作)
/** * 批量操作 */ @Test public void bulk() throws IOException { List<BulkOperation> list = new ArrayList<>(); Map<String, Object> doc = new HashMap<>(); doc.put("age", 30); doc.put("name", "李白"); doc.put("poems", "静夜思"); doc.put("about", "字太白"); doc.put("success", "创造了古代浪漫主义文学高峰、歌行体和七绝达到后人难及的高度"); //新增操作 list.add(new BulkOperation.Builder().create(builder -> builder.index(INDEX_NAME).id("12345").document(doc)).build()); //更新操作 list.add(new BulkOperation.Builder().update(builder -> builder .index(INDEX_NAME).id("1000") .action(updateActionBuilder -> updateActionBuilder .doc(doc))).build() ); //新增或覆盖操作 list.add(new BulkOperation.Builder().index(builder -> builder.index(INDEX_NAME).id("10").document(doc)).build()); //删除操作 list.add(new BulkOperation.Builder().delete(builder -> builder.index(INDEX_NAME).id("1")).build()); BulkResponse response = client.bulk(builder -> builder.index(INDEX_NAME).operations(list)); log.info("response={}", response); }
3.3.6、Delete by query API(查询删除)
/** * 查询删除,根据查询结果删除数据 */ @Test public void deleteByQuery() throws IOException { DeleteByQueryResponse response = client.deleteByQuery(deleteByQueryRequestBuilder -> deleteByQueryRequestBuilder .index(INDEX_NAME) .query(queryBuilder -> queryBuilder .term(termQueryBuilder -> termQueryBuilder .field("name") .value("李白") ) ) ); log.info("response={}", response); }
3.3.7、Update By Query API(查询更新)
/** * 查询更新,根据查询结果更新数据;如果不指定查询条件将更新所有数据 */ @Test public void updateByQuery() throws IOException { UpdateByQueryResponse response = client.updateByQuery(updateByQueryRequestBuilder -> updateByQueryRequestBuilder .index(INDEX_NAME) .query(queryBuilder -> queryBuilder .term(termQueryBuilder -> termQueryBuilder .field("name") .value("杜甫") ) ) .script(scriptBuilder -> scriptBuilder .inline(inlineScriptBuilder -> inlineScriptBuilder .source("ctx._source.age=88") ) ) ); log.info("response={}", response); }
3.4、Ingest APIs
3.4.1、Create or update pipeline API(创建或修改管道)
/** * 创建或修改管道 */ @Test public void createOrUpdatePipeline() throws IOException { PutPipelineResponse response = client.ingest().putPipeline(builder -> builder .id("my-pipeline") .description("My optional pipeline description") .processors(processorBuilder -> processorBuilder .set(setProcessorBuilder -> setProcessorBuilder .field("test-field") .value(JsonData.of("test-value"))))); log.info("response={}", response); }
3.4.2、Delete pipeline API(删除管道)
/** * 删除管道 */ @Test public void deletePipeline() throws IOException { DeletePipelineResponse response = client.ingest().deletePipeline(builder -> builder.id("my-pipeline")); log.info("response={}", response); }
3.4.3、Simulate pipeline API(模拟管道处理)
/** * 模拟管道处理 */ @Test public void simulatePipeline() throws IOException { Map<String, Object> source = new HashMap<>(); source.put("test-field", "aa"); SimulateResponse response = client.ingest().simulate(builder -> builder .id("my-pipeline") .docs(documentBuilder -> documentBuilder .index("my-index") .id("123") .source(JsonData.of(source)))); log.info("response={}", response); }
3.5、SQL APIs
3.5.1、SQL 查询
/** * SQL查询 */ @Test public void sql() throws IOException { QueryResponse response = client.sql().query(builder -> builder .format("json").query("SELECT * FROM \"" + INDEX_NAME + "\" where name='杜甫' limit 1")); log.info(response.toString()); }
3.6、完整代码
package com.abc.demo.es; import co.elastic.clients.elasticsearch.ElasticsearchClient; import co.elastic.clients.elasticsearch.core.*; import co.elastic.clients.elasticsearch.core.bulk.BulkOperation; import co.elastic.clients.elasticsearch.indices.*; import co.elastic.clients.elasticsearch.ingest.*; import co.elastic.clients.elasticsearch.sql.QueryResponse; import co.elastic.clients.json.JsonData; import co.elastic.clients.json.jackson.JacksonJsonpMapper; import co.elastic.clients.transport.ElasticsearchTransport; import co.elastic.clients.transport.rest_client.RestClientTransport; import com.fasterxml.jackson.databind.ObjectMapper; import lombok.AllArgsConstructor; import lombok.Data; import lombok.NoArgsConstructor; import lombok.extern.slf4j.Slf4j; import org.apache.http.HttpHost; import org.elasticsearch.client.RestClient; import org.junit.After; import org.junit.Before; import org.junit.Test; import java.io.IOException; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; @Slf4j public class ElasticsearchJavaCase { private static final String INDEX_NAME = "poet-index"; private ElasticsearchTransport transport; private ElasticsearchClient client; @Before public void before() { RestClient restClient = RestClient.builder( new HttpHost("10.49.196.10", 9200), new HttpHost("10.49.196.11", 9200), new HttpHost("10.49.196.12", 9200)).build(); ObjectMapper objectMapper = new ObjectMapper(); transport = new RestClientTransport(restClient, new JacksonJsonpMapper(objectMapper)); client = new ElasticsearchClient(transport); } @After public void after() throws IOException { transport.close(); } /** * 创建索引 */ @Test public void createIndex() throws IOException { CreateIndexResponse response = client.indices().create(builder -> builder .settings(indexSettingsBuilder -> indexSettingsBuilder.numberOfReplicas("1").numberOfShards("2")) .mappings(typeMappingBuilder -> typeMappingBuilder .properties("age", propertyBuilder -> propertyBuilder.integer(integerNumberPropertyBuilder -> integerNumberPropertyBuilder)) .properties("name", propertyBuilder -> propertyBuilder.keyword(keywordPropertyBuilder -> keywordPropertyBuilder)) .properties("poems", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart"))) .properties("about", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart"))) .properties("success", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart"))) ) .index(INDEX_NAME)); log.info("response={}", response); } /** * 删除索引 */ @Test public void deleteIndex() throws IOException { DeleteIndexResponse response = client.indices().delete(builder -> builder.index(INDEX_NAME)); log.info("response={}", response); } /** * 查询索引 */ @Test public void getIndex() throws IOException { //查询所有索引,也可以使用 * GetIndexResponse response = client.indices().get(builder -> builder.index("*")); log.info("response={}", response); response = client.indices().get(builder -> builder.index(INDEX_NAME));//单个索引 log.info("response={}", response); } /** * 查询 mapping 信息 */ @Test public void getMapping() throws IOException { //查询所有字段信息 GetMappingResponse response = client.indices().getMapping(builder -> builder.index(INDEX_NAME)); log.info("response={}", response); //查询某些字段信息 GetFieldMappingResponse response2 = client.indices().getFieldMapping(builder -> builder.index(INDEX_NAME).fields("age")); log.info("response2={}", response2); } /** * 修改 mapping 信息 * 可以新增字段,已有字段只能修改字段的 search_analyzer 属性 */ @Test public void modifyMapping() throws IOException { PutMappingResponse response = client.indices().putMapping(typeMappingBuilder -> typeMappingBuilder .index(INDEX_NAME) .properties("name", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word"))) .properties("desc", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word"))) ); log.info("response={}", response); } /** * 查询 setting 信息 */ @Test public void getSetting() throws IOException { //查询所有参数 GetIndicesSettingsResponse response = client.indices().getSettings(builder -> builder.index(INDEX_NAME)); log.info("response={}", response); //查询单个参数 response = client.indices().getSettings(builder -> builder.index(INDEX_NAME).name("index.number_of_replicas")); log.info("response={}", response); //返回中包含默认参数信息 response = client.indices().getSettings(builder -> builder.index(INDEX_NAME).includeDefaults(true)); log.info("response={}", response); } /** * 修改 setting 信息 */ @Test public void modifySetting() throws IOException { PutIndicesSettingsResponse response = client.indices().putSettings(builder -> builder .index(INDEX_NAME) .settings(indexSettingsBuilder -> indexSettingsBuilder.maxResultWindow(10000))); log.info("response={}", response); } /** * 查看分词器分词结果 */ @Test public void analyze() throws IOException { AnalyzeResponse response = client.indices().analyze(builder -> builder .analyzer("ik_max_word") .text("唐代伟大的现实主义文学作家")); log.info("response={}", response); } /** * 创建/覆盖文档 */ @Test public void createDoc() throws IOException { Map<String, Object> doc = new HashMap<>(); doc.put("age", 30); doc.put("name", "李白"); doc.put("poems", "静夜思"); doc.put("about", "字太白"); doc.put("success", "创造了古代浪漫主义文学高峰、歌行体和七绝达到后人难及的高度"); //没有对应 id 的文档就创建,有就覆盖更新所有字段(相当于先删除再新增)并更新版本号。 IndexResponse response = client.index(builder -> builder.index(INDEX_NAME).id("1").document(doc)); log.info(response.toString()); Poet poet = new Poet(); poet.setAge(40); poet.setName("杜甫"); poet.setPoems("登高"); poet.setAbout("字子美"); poet.setSuccess("唐代伟大的现实主义文学作家,唐诗思想艺术的集大成者"); //不设置 id 将新增,id 将自动生成。 response = client.index(builder -> builder.index(INDEX_NAME).document(poet)); log.info(response.toString()); Poet poet2 = new Poet(31, "杜甫", "登高", "字子美", "唐代伟大的现实主义文学作家,唐诗思想艺术的集大成者"); //新增文档,需要设置 id。 CreateResponse response2 = client.create(builder -> builder.index(INDEX_NAME).id("2").document(poet2)); log.info("response2={}", response2); } /** * 删除文档 */ @Test public void deleteDoc() throws IOException { DeleteResponse response = client.delete(builder -> builder.index(INDEX_NAME).id("2")); log.info(response.toString()); } /** * 根据 id 查询文档 */ @Test public void id() throws IOException { GetResponse<Map> response = client.get(builder -> builder.index(INDEX_NAME).id("1"), Map.class); log.info("response={}", response); } /** * 修改文档,只修改设置的字段 */ @Test public void updateDoc() throws IOException { Map<String, Object> doc = new HashMap<>(); doc.put("age", 33); doc.put("name", "李白2"); UpdateResponse response = client.update(builder -> builder.index(INDEX_NAME).id("10").doc(doc), Map.class); log.info("response={}", response); //文档不存在时插入 Poet poet = new Poet(); poet.setAge(40); poet.setName("杜甫2"); response = client.update(builder -> builder.index(INDEX_NAME).id("11").doc(poet).docAsUpsert(true), Poet.class); log.info("response={}", response); } /** * 批量操作 */ @Test public void bulk() throws IOException { List<BulkOperation> list = new ArrayList<>(); Map<String, Object> doc = new HashMap<>(); doc.put("age", 30); doc.put("name", "李白"); doc.put("poems", "静夜思"); doc.put("about", "字太白"); doc.put("success", "创造了古代浪漫主义文学高峰、歌行体和七绝达到后人难及的高度"); //新增操作 list.add(new BulkOperation.Builder().create(builder -> builder.index(INDEX_NAME).id("12345").document(doc)).build()); //更新操作 list.add(new BulkOperation.Builder().update(builder -> builder .index(INDEX_NAME).id("1000") .action(updateActionBuilder -> updateActionBuilder .doc(doc))).build() ); //新增或覆盖操作 list.add(new BulkOperation.Builder().index(builder -> builder.index(INDEX_NAME).id("10").document(doc)).build()); //删除操作 list.add(new BulkOperation.Builder().delete(builder -> builder.index(INDEX_NAME).id("1")).build()); BulkResponse response = client.bulk(builder -> builder.index(INDEX_NAME).operations(list)); log.info("response={}", response); } /** * 查询删除,根据查询结果删除数据 */ @Test public void deleteByQuery() throws IOException { DeleteByQueryResponse response = client.deleteByQuery(deleteByQueryRequestBuilder -> deleteByQueryRequestBuilder .index(INDEX_NAME) .query(queryBuilder -> queryBuilder .term(termQueryBuilder -> termQueryBuilder .field("name") .value("李白") ) ) ); log.info("response={}", response); } /** * 查询更新,根据查询结果更新数据;如果不指定查询条件将更新所有数据 */ @Test public void updateByQuery() throws IOException { UpdateByQueryResponse response = client.updateByQuery(updateByQueryRequestBuilder -> updateByQueryRequestBuilder .index(INDEX_NAME) .query(queryBuilder -> queryBuilder .term(termQueryBuilder -> termQueryBuilder .field("name") .value("杜甫") ) ) .script(scriptBuilder -> scriptBuilder .inline(inlineScriptBuilder -> inlineScriptBuilder .source("ctx._source.age=88") ) ) ); log.info("response={}", response); } /** * 创建或修改管道 */ @Test public void createOrUpdatePipeline() throws IOException { PutPipelineResponse response = client.ingest().putPipeline(builder -> builder .id("my-pipeline") .description("My optional pipeline description") .processors(processorBuilder -> processorBuilder .set(setProcessorBuilder -> setProcessorBuilder .field("test-field") .value(JsonData.of("test-value"))))); log.info("response={}", response); } /** * 删除管道 */ @Test public void deletePipeline() throws IOException { DeletePipelineResponse response = client.ingest().deletePipeline(builder -> builder.id("my-pipeline")); log.info("response={}", response); } /** * 模拟管道处理 */ @Test public void simulatePipeline() throws IOException { Map<String, Object> source = new HashMap<>(); source.put("test-field", "aa"); SimulateResponse response = client.ingest().simulate(builder -> builder .id("my-pipeline") .docs(documentBuilder -> documentBuilder .index("my-index") .id("123") .source(JsonData.of(source)))); log.info("response={}", response); } /** * SQL查询 */ @Test public void sql() throws IOException { QueryResponse response = client.sql().query(builder -> builder .format("json").query("SELECT * FROM \"" + INDEX_NAME + "\" where name='杜甫' limit 1")); log.info(response.toString()); } @Data @AllArgsConstructor @NoArgsConstructor static class Poet { private Integer age; private String name; private String poems; private String about; /**成就*/ private String success; } }
详细的 Elasticsearch Java API Client 使用说明,请参考官网文档:https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/index.html。