【尚硅谷】ElasticSearch入门到精通2021最新教程
一、ElasticSearch概述
ElasticSearch是一个分布式,RESTFUL风格的搜索和数据分析引擎
二、Elasticsearch 安装
下载地址:
https://www.elastic.co/cn/downloads/past-releases#elasticsearch
1、选择windows7.8.0版本解压到本地
2、解压后,进入 bin 文件目录,点击 elasticsearch.bat 文件启动 ES 服务
ES目录结构详解
注意:9300 端口为 Elasticsearch 集群间组件的通信端口,9200 端口为浏览器访问的 http 协议 RESTful 端口。
3、打开浏览器(推荐使用谷歌浏览器),输入地址:http://localhost:9200,测试结果
三、Elasticsearch 基本操作
1、索引操作
1)创建索引
PUT http://127.0.0.1:9200/shopping
重复创建索引会报错
【注意】创建索引只能使用PUT请求
2)查看所有索引
GET http://127.0.0.1:9200/_cat/indices?v
3)查看单个索引
GET http://127.0.0.1:9200/shopping
4)删除索引
2、文档操作
1)创建文档
POST请求和PUT请求都可以创建文档
创建文档时,POST可以携带ID也可以不携带ID,但是PUT请求必须要携带ID
创建文档时,POST请求携带ID,如果ES中有相同的文档,则为修改,但是version一直增加
创建文档时,PUT请求携带ID,如果ES中有相同的文档,则为修改,但是version不会增加
2)查看文档(根据ID查看文档)
3)修改文档(全量覆盖)
4)修改文档字段 (局部字段更新)
注意requestbody格式
5)删除文档
6)根据条件删除文档
3、映射操作
1)创建映射
PUT http://127.0.0.1:9200/student/_mapping { "properties": { "name":{ "type": "text", "index": true }, "sex":{ "type": "text", "index": false }, "age":{ "type": "long", "index": false } } }
映射字段说明:
字段名:任意填写,下面指定许多属性,例如:title、subtitle、images、price
type:类型,Elasticsearch 中支持的数据类型非常丰富,说几个关键的:
String 类型,又分两种:
text:可分词
keyword:不可分词,数据会作为完整字段进行匹配
Numerical:数值类型,分两类
基本数据类型:long、integer、short、byte、double、float、half_float
浮点数的高精度类型:scaled_float
Date:日期类型
Array:数组类型
Object:对象
index:是否索引,默认为 true,也就是说你不进行任何配置,所有字段都会被索引。
true:字段会被索引,则可以用来进行搜索
false:字段不会被索引,不能用来搜索
store:是否将数据进行独立存储,
默认为 false 原始的文本会存储在_source 里面,默认情况下其他提取出来的字段都不是独立存 储 的,是从_source 里面提取出来的。当然你也可以独立的存储某个字段,只要设置 "store": true 即可,获取独立存储的字段要比从_source 中解析快得多,但是也会占用 更多的空间,所以要根据实际业务需求来设置。
analyzer:分词器,这里的 ik_max_word 即使用 ik 分词器
2)查看映射
GET http://127.0.0.1:9200/student/_mapping
4、高级查询
1)查询所有文档 match_all
http://127.0.0.1:9200/shopping/_search { "query":{ "match_all":{} } }
2)匹配查询 match
http://127.0.0.1:9200/shopping/_search { "query":{ "match":{ "title":"手机" } } }
3)多字段匹配查询
http://127.0.0.1:9200/shopping/_search { "query": { "multi_match": { "query": "手机", "fields": ["title","category"] } } }
4)关键字精确查询 term
term 查询,精确的关键词匹配查询,不对查询条件进行分词。
http://127.0.0.1:9200/shopping/_search { "query": { "term": { "title":{ "value":"小米手机" } } } }
5)多关键字精确查询 term
terms 查询和 term 查询一样,但它允许你指定多值进行匹配。 如果这个字段包含了指定值中的任何一个值,那么这个文档满足条件
http://127.0.0.1:9200/shopping/_search { "query": { "term": { "title":["小米","华为"] } } }
6)指定查询字段 _source
http://127.0.0.1:9200/shopping/_search { "_source": ["name","nickname"], "query": { "terms": { "title": ["小米"] } } }
也可以通过
includes:来指定想要显示的字段
excludes:来指定不想要显示的字段
http://127.0.0.1:9200/shopping/_search { "_source": { "includes": ["name","nickname"] }, "query": { "terms": { "nickname": ["zhangsan"] } } }
7)组合查询 bool
8)范围查询 range
http://127.0.0.1:9200/shopping/_search { "query": { "range": { "age": { "gte": 30, "lte": 35 } } } }
9)模糊查询 fuzzy
返回包含与搜索字词相似的字词的文档。
为了找到相似的术语,fuzzy 查询会在指定的编辑距离内创建一组搜索词的所有可能的变体 或扩展。然后查询返回每个扩展的完全匹配
http://127.0.0.1:9200/shopping/_search { "query": { "fuzzy": { "title": { "value": "zhangsan" } } } }
10)字段排序 sort
http://127.0.0.1:9200/shopping/_search { "query":{ "match":{ "name":"zhangsan" } }, "sort":[ { "age":{ "order":"desc" } } ] }
11)高亮查询 highlight
在进行关键字搜索时,搜索出的内容中的关键字会显示不同的颜色,称之为高亮
Elasticsearch 可以对查询内容中的关键字部分,进行标签和样式(高亮)的设置。 在使用 match 查询的同时,加上一个 highlight 属性:
pre_tags:前置标签
post_tags:后置标签
fields:需要高亮的字段
12)分页查询
from:当前页的起始索引,默认从 0 开始。 from = (pageNum - 1) * size
size:每页显示多少条
http://127.0.0.1:9200/shopping/_search { "query": { "match_all": {} }, "from": 0, "size": 2 }
13)聚合查询
①、求最大值
http://127.0.0.1:9200/shopping/_search { "aggs":{ "max_age":{ "max":{"field":"age"} } } }
②、求最小值
http://127.0.0.1:9200/shopping/_search { "aggs":{ "min_age":{ "min":{"field":"age"} } } }
③、对字段求和
http://127.0.0.1:9200/shopping/_search { "aggs":{ "sum_age":{ "sum":{"field":"age"} } } }
④、求平均值
http://127.0.0.1:9200/shopping/_search { "aggs":{ "avg_age":{ "avg":{"field":"age"} } } }
⑤、State 聚合
stats 聚合,对某个字段一次性返回 count,max,min,avg 和 sum 五个指标
http://127.0.0.1:9200/shopping/_search { "aggs":{ "stats_age":{ "stats":{"field":"price"} } } }
⑥、桶聚合查询
桶聚和相当于 sql 中的 group by 语句
terms聚合,分组统计
http://127.0.0.1:9200/shopping/_search { "aggs":{ "price_agg":{ "terms":{"field":"price"} } } }
四、Java操作 ElasticSearch
1、引入pom文件
<dependencies> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>7.8.0</version> </dependency> <!-- elasticsearch 的客户端 --> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.8.0</version> </dependency> <!-- elasticsearch 依赖 2.x 的 log4j --> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>2.8.2</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.8.2</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.9.9</version> </dependency> <!-- junit 单元测试 --> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> </dependency> </dependencies>
2、创建es客户端
public class EsTestClient { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.2", 9200, "http"))); System.out.println(client); //关闭Es客户端 client.close(); } }
3、索引操作
1)创建索引
public class EsTestClient { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、创建索引 CreateIndexRequest createIndexRequest = new CreateIndexRequest("user"); CreateIndexResponse createIndexResponse = client.indices().create(createIndexRequest, RequestOptions.DEFAULT); boolean acknowledged = createIndexResponse.isAcknowledged(); System.out.println("创建索引"+ (acknowledged ? "成功":"失败")); //关闭Es客户端 client.close(); } }
在postman中查询
2)查看索引
//2、查看索引 GetIndexRequest getIndexResponse = new GetIndexRequest("user"); GetIndexResponse getIndexResponse = client.indices().get(getIndexRequest, RequestOptions.DEFAULT);
3)删除索引
//3、删除索引 DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest("user"); AcknowledgedResponse deleteResponse = client.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);
4、文档操作
1)插入文档
public class EsTestDocInsert { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、插入文档 IndexRequest indexRequest = new IndexRequest(); indexRequest.index("user").id("00001"); User user = new User(); user.setName("张三"); user.setAge(15); user.setSex("男"); ObjectMapper mapper = new ObjectMapper(); String userStr = mapper.writeValueAsString(user); indexRequest.source(userStr, XContentType.JSON); IndexResponse reponse = client.index(indexRequest, RequestOptions.DEFAULT); } }
2)修改文档
public class EsTestDocUpdate { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、修改文档 UpdateRequest updateRequest = new UpdateRequest(); updateRequest.index("user").id("00001"); updateRequest.doc(XContentType.JSON,"sex","女的"); UpdateResponse updateResponse = client.update(updateRequest, RequestOptions.DEFAULT); System.out.println(updateResponse.getResult()); } }
3)查询文档(根据ID进行查询)
public class EsTestDocGet { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //根据ID查询文档 GetRequest getRequest = new GetRequest(); getRequest.index("user").id("00001"); GetResponse res = client.get(getRequest, RequestOptions.DEFAULT); System.out.println(res.getSourceAsString()); } }
4)删除文档
public class EsTestDocDelete { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //根据ID查询文档 DeleteRequest deleteRequest = new DeleteRequest(); deleteRequest.index("user").id("00001"); DeleteResponse res = client.delete(deleteRequest, RequestOptions.DEFAULT); System.out.println(res.getResult()); } }
5)批量新增
public class EsTestDocInsertBatch { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); //1、批量插入文档 BulkRequest bulkRequest = new BulkRequest(); bulkRequest.add(new IndexRequest().index("user").source(XContentType.JSON, "name","张三")); bulkRequest.add(new IndexRequest().index("user").source(XContentType.JSON, "name","李四")); bulkRequest.add(new IndexRequest().index("user").source(XContentType.JSON, "name","王五")); BulkResponse response = client.bulk(bulkRequest, RequestOptions.DEFAULT); System.out.println(response.getItems()); client.close(); } }
5、高级插叙
1)match_all 查询
public class EsTestDocQuery { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("user"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()); searchRequest.source(sourceBuilder); SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); SearchHit[] hits1 = hits.getHits(); for (SearchHit documentFields : hits1) { System.out.println(documentFields); } client.close(); } }
2)分页查询
public class EsTestDocPageQuery { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("user"); // 构建查询的请求体 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); // 分页查询 // 当前页其实索引(第一条数据的顺序号),from sourceBuilder.from(0); // 每页显示多少条 size sourceBuilder.size(2); searchRequest.source(sourceBuilder); SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT); // 查询匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //输出每条查询的结果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
3)数据排序
public class EsTestDocSortQuery { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest searchRequest = new SearchRequest(); searchRequest.indices("user"); // 构建查询的请求体 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); // 排序 sourceBuilder.sort("age", SortOrder.ASC); searchRequest.source(sourceBuilder); SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT); // 查询匹配 SearchHits hits = response.getHits(); SearchHit[] hits1 = hits.getHits(); for (SearchHit documentFields : hits1) { System.out.println(documentFields); } client.close(); } }
4)过滤字段
public class EsTestDocFilterQuery { public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); // 创建搜索请求对象 SearchRequest request = new SearchRequest(); request.indices("student"); // 构建查询的请求体 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); //查询字段过滤 String[] excludes = {}; String[] includes = {"name", "age"}; sourceBuilder.fetchSource(includes, excludes); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查询匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //输出每条查询的结果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
5)bool查询
public class EsTestDocFilterQuery { public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); // 创建搜索请求对象 SearchRequest request = new SearchRequest(); request.indices("student"); // 构建查询的请求体 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); // 必须包含 boolQueryBuilder.must(QueryBuilders.matchQuery("age", "30")); // 一定不含 boolQueryBuilder.mustNot(QueryBuilders.matchQuery("name", "zhangsan")); // 可能包含 boolQueryBuilder.should(QueryBuilders.matchQuery("sex", "男")); sourceBuilder.query(boolQueryBuilder); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查询匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //输出每条查询的结果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
6)范围查询
public class EsTestDocRangeQuery { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); // 创建搜索请求对象 SearchRequest request = new SearchRequest(); request.indices("student"); // 构建查询的请求体 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("age"); // 大于等于 rangeQuery.gte("30"); // 小于等于 rangeQuery.lte("40"); sourceBuilder.query(rangeQuery); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查询匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //输出每条查询的结果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); } }
7)模糊查询
// 创建搜索请求对象 SearchRequest request = new SearchRequest(); request.indices("student"); // 构建查询的请求体 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.fuzzyQuery("name","zhangsan").fuzziness(Fu zziness.ONE)); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); // 查询匹配 SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { //输出每条查询的结果信息 System.out.println(hit.getSourceAsString()); } System.out.println("<<========");
8)高亮查询
// 高亮查询 SearchRequest request = new SearchRequest().indices("student"); //2.创建查询请求体构建器 SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); //构建查询方式:高亮查询 TermsQueryBuilder termsQueryBuilder = QueryBuilders.termsQuery("name","zhangsan"); //设置查询方式 sourceBuilder.query(termsQueryBuilder) //构建高亮字段 HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.preTags("<font color='red'>");//设置标签前缀 highlightBuilder.postTags("</font>");//设置标签后缀 highlightBuilder.field("name");//设置高亮字段 //设置高亮构建对象 sourceBuilder.highlighter(highlightBuilder); //设置请求体 request.source(sourceBuilder); //3.客户端发送请求,获取响应对象 SearchResponse response = client.search(request, RequestOptions.DEFAULT); //4.打印响应结果 SearchHits hits = response.getHits(); System.out.println("took::"+response.getTook()); System.out.println("time_out::"+response.isTimedOut()); System.out.println("total::"+hits.getTotalHits()); System.out.println("max_score::"+hits.getMaxScore()); System.out.println("hits::::>>"); for (SearchHit hit : hits) { String sourceAsString = hit.getSourceAsString(); System.out.println(sourceAsString); //打印高亮结果 Map<String, HighlightField> highlightFields = hit.getHighlightFields(); System.out.println(highlightFields); } System.out.println("<<::::");
9)聚合查询
① 最大值
public class EsTestDocMaxAggregation { public static void main(String[] args) throws IOException { //创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest request = new SearchRequest(); request.indices("user"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.aggregation(AggregationBuilders.max("maxAge").field("age")); //设置请求体 request.source(sourceBuilder); //3.客户端发送请求,获取响应对象 SearchResponse response = client.search(request, RequestOptions.DEFAULT); //4.打印响应结果 SearchHits hits = response.getHits(); System.out.println(response); client.close(); } }
② 分组聚合
public class EsTestDocMaxAggregation { public static void main(String[] args) throws IOException { //1创建Es客户端 RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("127.0.0.1", 9200, "http"))); SearchRequest request = new SearchRequest(); request.indices("user"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.aggregation(AggregationBuilders.terms("age_groupby").field("age")); //设置请求体 request.source(sourceBuilder); //3.客户端发送请求,获取响应对象 SearchResponse response = client.search(request, RequestOptions.DEFAULT); //4.打印响应结果 SearchHits hits = response.getHits(); System.out.println(response); client.close(); } }