ElasticSearch知识汇总

核心概念

索引（index）:ElasticSearch存储数据的地方，可以理解成关系型数据库中的数据库概念。
映射（mapping）:mapping定义了每个字段的类型、字段所使用的分词器等。相当于关系型数据库中的表结构。
文档（document）:Elasticsearch中的最小数据单元，常以json格式显示。一个document相当于关系型数据库中的一行数据。
倒排索引:一个倒排索引由文档中所有不重复词的列表构成，对于其中每个词，对应一个包含它的文档id列表。
类型（type）:一种type就像一类表。如用户表、角色表等。ES 7.x以后，将逐步移除type这个概念，现在的操作已经不再使用，默认_doc

操作

操作索引

添加索引

PUT 索引名称

查询索引

查询单个索引信息

GET 索引名

查询多个索引信息

GET 索引名1,索引名2

查询所有索引信息

GET _all

删除索引

DELETE 索引名

关闭索引

POST 索引名称/_close

打开索引

POST 索引名称/_open

操作映射

数据类型

字符串

text：会分词、不支持聚合
keyword：不分词、支持聚数值

布尔
二进制
范围类型
日期

复杂数据类型：json

数组[]
对象{}

添加映射

创建索引并添加映射

查询映射

添加字段

操作文档

添加文档，指定id

添加文档，不指定id

查询指定id的文档

查询所有文档

删除指定id文档

查询文档

词条查询term

词条查询不会分析查询条件，只有当词条和查询字符串完全匹配时才匹配搜索
Kibana操作

全文查询match

全文查询会分析查询条件，先将查询条件进行分词，然后查询，求并集
Kibana操作

创建索引，使用IK分词器

整合springboot

引入依赖

<properties>
    <java.version>1.8</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.junit.vintage</groupId>
                <artifactId>junit-vintage-engine</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!--引入es的坐标-->
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>7.4.0</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-client</artifactId>
        <version>7.4.0</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>7.4.0</version>
    </dependency>

</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
        </plugin>
    </plugins>
</build>

引导类

@SpringBootApplication
public class ElasticsearchDemoApplication {
    public static void main(String[] args) {
        SpringApplication.run(ElasticsearchDemoApplication.class, args);
    }
}

配置文件application.yml

elasticsearch:
  host: 192.168.149.135
  port: 9200

配置类

@Configuration
@ConfigurationProperties(prefix = "elasticsearch")
public class ElasticSearchConfig {
    private String host;
    private int port;

    public String getHost() {
        return host;
    }

    public void setHost(String host) {
        this.host = host;
    }

    public int getPort() {
        return port;
    }

    public void setPort(int port) {
        this.port = port;
    }

    @Bean
    public RestHighLevelClient client(){
        return new RestHighLevelClient(RestClient.builder(
                new HttpHost(
                        host,
                        port,
                        "http"
                )
        ));
    }
}

测试类

@SpringBootTest
class ElasticsearchDemoApplicationTests {

    @Autowired
    private RestHighLevelClient client;

    @Test
    void contextLoads() {
       /* //1.创建ES客户端对象
        RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(
           new HttpHost(
                   "192.168.149.135",
                   9200,
                   "http"
           )
        ));*/

        System.out.println(client);

    }
}

api操作文档

添加

以map格式

  Map map = new HashMap();
        map.put("name","小胖");
        IndexRequest request = new IndexRequest("goods_index2").id("5").source(map);
        client.index(request, RequestOptions.DEFAULT);

以对象格式

       Student student=new Student();
        student.setName("大胖");
        String string = JSON.toJSONString(student);
        IndexRequest goods_index2 = new IndexRequest("goods_index2").id("6").source(string, XContentType.JSON);
        client.index(goods_index2,RequestOptions.DEFAULT);

修改

和增加文档一样，ID存在为修改，不存在为增加

查询

 GetRequest request = new GetRequest("goods_index2").id("5");
        GetResponse documentFields = client.get(request, RequestOptions.DEFAULT);
        System.out.println(documentFields);

删除

  DeleteRequest deleteRequest = new DeleteRequest("goods_index2").id("5");
        DeleteResponse delete = client.delete(deleteRequest, RequestOptions.DEFAULT);
        System.out.println(delete);

批量操作

Bulk 批量操作是将文档的增删改查一些列操作，通过一次请求全都做完。减少网络传输次数。
语法

action：执行的操作
metadata：操作的信息，如索引名称，ID等
data：操作的数据，如增加数据，对象的具体信息

基本操作

批量导入

BulkRequest bulkRequest = new BulkRequest();
        for (int i = 0; i < 20; i++) {
            IndexRequest indexRequest = new IndexRequest("ssw");
            Map map = new HashMap();
            map.put("age",i);
            indexRequest.id(i+"");
            indexRequest.source(map);
            bulkRequest.add(indexRequest);
        }
        client.bulk(bulkRequest,RequestOptions.DEFAULT);

查询

查询所有的文档

kibanna

javaapi

/**
 * 查询所有
 *  1. matchAll
 *  2. 将查询结果封装为Goods对象，装载到List中
 *  3. 分页。默认显示10条
 */
@Test
public void testMatchAll() throws IOException {
    //2. 构建查询请求对象，指定查询的索引名称
    SearchRequest searchRequest = new SearchRequest("goods");
    //4. 创建查询条件构建器SearchSourceBuilder
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();

    //6. 查询条件
    QueryBuilder query = QueryBuilders.matchAllQuery();//查询所有文档
    //5. 指定查询条件
    sourceBuilder.query(query);

    //3. 添加查询条件构建器 SearchSourceBuilder
    searchRequest.source(sourceBuilder);

    // 8 . 添加分页信息
    sourceBuilder.from(0);
    sourceBuilder.size(100);

    //1. 查询,获取查询结果
    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    //7. 获取命中对象 SearchHits
    SearchHits searchHits = searchResponse.getHits();
    //7.1 获取总记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);


    List<Goods> goodsList = new ArrayList<>();
    //7.2 获取Hits数据  数组
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        //获取json字符串格式的数据
        String sourceAsString = hit.getSourceAsString();
        //转为java对象
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);
        goodsList.add(goods);

    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

term查询：

不会对查询条件进行分词

kibana

javaapi

/**
 * termQuery:词条查询
 */
@Test
public void testTermQuery() throws IOException {

    SearchRequest searchRequest = new SearchRequest("goods");
    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
    QueryBuilder query = QueryBuilders.termQuery("title","华为");//term词条查询
    sourceBulider.query(query);

    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();

        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);

        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

match查询

会对查询条件进行分词。
然后将分词后的查询条件和词条进行等值匹配
默认取并集（OR）

javaapi

/**
 * matchQuery:词条分词查询
 */
@Test
public void testMatchQuery() throws IOException {

    SearchRequest searchRequest = new SearchRequest("goods");
    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
    MatchQueryBuilder query = QueryBuilders.matchQuery("title", "华为手机");
    query.operator(Operator.AND);//求并集
    sourceBulider.query(query);

    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();

        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);

        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

模糊查询

wildcard查询

会对查询条件进行分词。还可以使用通配符 ?（任意单个字符）和 * （0个或多个字符）

/**
 * 模糊查询:WildcardQuery
 */
@Test
public void testWildcardQuery() throws IOException {


    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
    WildcardQueryBuilder query = QueryBuilders.wildcardQuery("title", "华*");
    sourceBulider.query(query);
    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();
        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);
        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

regexp查询

正则查询

/**
 * 模糊查询:regexpQuery
 */
@Test
public void testRegexpQuery() throws IOException {


    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
    RegexpQueryBuilder query = QueryBuilders.regexpQuery("title", "\\w+(.)*");
    sourceBulider.query(query);
    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();
        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);
        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

prefix查询

前缀查询

/**
 * 模糊查询:perfixQuery
 */
@Test
public void testPrefixQuery() throws IOException {

    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
    PrefixQueryBuilder query = QueryBuilders.prefixQuery("brandName", "三");
    sourceBulider.query(query);
    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();
        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);
        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

范围查询

查找指定字段在指定范围内包含值，也可以排序

/**
 * 1. 范围查询：rangeQuery
 * 2. 排序
 */
@Test
public void testRangeQuery() throws IOException {
    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();
    //范围查询
    RangeQueryBuilder query = QueryBuilders.rangeQuery("price");
    //指定下限
    query.gte(2000);
    //指定上限
    query.lte(3000);

    sourceBulider.query(query);
    //排序
    sourceBulider.sort("price", SortOrder.DESC);
    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();
        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);
        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

queryString查询

会对查询条件进行分词。
然后将分词后的查询条件和词条进行等值匹配
默认取并集（OR）
可以指定多个查询字段

/**
 * queryString
 */
@Test
public void testQueryStringQuery() throws IOException {


    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();

    //queryString
    QueryStringQueryBuilder query = QueryBuilders.queryStringQuery("华为手机")
            .field("title").field("categoryName").field("brandName")
            .defaultOperator(Operator.AND);

    sourceBulider.query(query);

    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();
        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);
        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

布尔查询

连接方式

must（and）：条件必须成立
must_not（not）：条件必须不成立
should（or）：条件可以成立
filter：条件必须成立，性能比must高。不会计算得分

语法

javaapi

/**
 * 布尔查询：boolQuery
 * 1. 查询品牌名称为:华为
 * 2. 查询标题包含：手机
 * 3. 查询价格在：2000-3000
 */
@Test
public void testBoolQuery() throws IOException {
    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();

    //1.构建boolQuery
    BoolQueryBuilder query = QueryBuilders.boolQuery();

    //2.构建各个查询条件
    //2.1 查询品牌名称为:华为
    QueryBuilder termQuery = QueryBuilders.termQuery("brandName","华为");
    query.must(termQuery);

    //2.2. 查询标题包含：手机
    QueryBuilder matchQuery = QueryBuilders.matchQuery("title","手机");
    query.filter(matchQuery);

    //2.3 查询价格在：2000-3000
    QueryBuilder rangeQuery = QueryBuilders.rangeQuery("price");
    ((RangeQueryBuilder) rangeQuery).gte(2000);
    ((RangeQueryBuilder) rangeQuery).lte(3000);
    query.filter(rangeQuery);

    //3.使用boolQuery连接

    sourceBulider.query(query);

    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();

        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);

        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }
}

聚合查询

分类

指标聚合

相当于MySQL的聚合函数。max、min、avg、sum等

桶聚合

相当于MySQL的 group by 操作。不要对text类型的数据进行分组，会失败。

javaapi

/**
 * 聚合查询：桶聚合，分组查询
 * 1. 查询title包含手机的数据
 * 2. 查询品牌列表
 */
@Test
public void testAggQuery() throws IOException {


    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();

    // 1. 查询title包含手机的数据
    MatchQueryBuilder query = QueryBuilders.matchQuery("title", "手机");

    sourceBulider.query(query);

    // 2. 查询品牌列表
    /*
    参数：
        1. 自定义的名称，将来用于获取数据
        2. 分组的字段
     */
    AggregationBuilder agg = AggregationBuilders.terms("goods_brands").field("brandName").size(100);
    sourceBulider.aggregation(agg);

    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();

        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);

        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }

    // 获取聚合结果
    Aggregations aggregations = searchResponse.getAggregations();

    Map<String, Aggregation> aggregationMap = aggregations.asMap();

    //System.out.println(aggregationMap);
    Terms goods_brands = (Terms) aggregationMap.get("goods_brands");

    List<? extends Terms.Bucket> buckets = goods_brands.getBuckets();

    List brands = new ArrayList();
    for (Terms.Bucket bucket : buckets) {
        Object key = bucket.getKey();
        brands.add(key);
    }

    for (Object brand : brands) {
        System.out.println(brand);
    }

}

高亮查询

三要素：高亮字段、前缀、后缀

kibanna

javaapi

/**
 *
 * 高亮查询：
 *  1. 设置高亮
 *      * 高亮字段
 *      * 前缀
 *      * 后缀
 *  2. 将高亮了的字段数据，替换原有数据
 */
@Test
public void testHighLightQuery() throws IOException {

    SearchRequest searchRequest = new SearchRequest("goods");

    SearchSourceBuilder sourceBulider = new SearchSourceBuilder();

    // 1. 查询title包含手机的数据
    MatchQueryBuilder query = QueryBuilders.matchQuery("title", "手机");

    sourceBulider.query(query);

    //设置高亮
    HighlightBuilder highlighter = new HighlightBuilder();
    //设置三要素
    highlighter.field("title");
    highlighter.preTags("<font color='red'>");
    highlighter.postTags("</font>");

    sourceBulider.highlighter(highlighter);

    // 2. 查询品牌列表
    /*
    参数：
        1. 自定义的名称，将来用于获取数据
        2. 分组的字段
     */
    AggregationBuilder agg = AggregationBuilders.terms("goods_brands").field("brandName").size(100);
    sourceBulider.aggregation(agg);

    searchRequest.source(sourceBulider);

    SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

    SearchHits searchHits = searchResponse.getHits();
    //获取记录数
    long value = searchHits.getTotalHits().value;
    System.out.println("总记录数："+value);

    List<Goods> goodsList = new ArrayList<>();
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
        String sourceAsString = hit.getSourceAsString();

        //转为java
        Goods goods = JSON.parseObject(sourceAsString, Goods.class);

        // 获取高亮结果，替换goods中的title
        Map<String, HighlightField> highlightFields = hit.getHighlightFields();
        HighlightField HighlightField = highlightFields.get("title");
        Text[] fragments = HighlightField.fragments();
        //替换
        goods.setTitle(fragments[0].toString());


        goodsList.add(goods);
    }

    for (Goods goods : goodsList) {
        System.out.println(goods);
    }

    // 获取聚合结果
    Aggregations aggregations = searchResponse.getAggregations();

    Map<String, Aggregation> aggregationMap = aggregations.asMap();

    //System.out.println(aggregationMap);
    Terms goods_brands = (Terms) aggregationMap.get("goods_brands");

    List<? extends Terms.Bucket> buckets = goods_brands.getBuckets();

    List brands = new ArrayList();
    for (Terms.Bucket bucket : buckets) {
        Object key = bucket.getKey();
        brands.add(key);
    }

    for (Object brand : brands) {
        System.out.println(brand);
    }

}

posted on 2020-03-21 19:05 赟麟阅读(554) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

ElasticSearch知识汇总

导航

公告