Elasticsearch 系列（五）- 数据聚合
本章将和大家分享 Elasticsearch 中的数据聚合功能，通过聚合（aggregations）可以实现对文档数据的统计、分析、运算。
一、数据聚合-聚合的分类
聚合（aggregations）可以实现对文档数据的统计、分析、运算。聚合的官方文档地址：https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html
聚合常见的有三类：
1）桶（Bucket）聚合：用来对文档做分组
- TermAggregation：按照文档字段值分组
- Date Histogram：按照日期阶梯分组，例如：一周为一组，或者一月为一组。
2）度量（Metric）聚合：用于计算一些值，比如：最大值、最小值、平均值等。
- Avg：求平均值
- Max：求最大值
- Min：求最小值
- Stats：同时求max、min、avg、sum等。
3）管道（Pipeline）聚合：以其它聚合的结果为基础做聚合。
总结：
1）什么是聚合？
- 聚合是对文档数据的统计、分析、计算
2）聚合的常见种类有哪些？
- Bucket：对文档数据分组，并统计每组数量
- Metric：对文档数据做计算，例如：avg
- Pipeline：基于其它聚合结果再做聚合
3）参与聚合的字段类型不能是 text（可分词的文本）类型，可以是：keyword、数值、日期、布尔类型。
二、数据聚合-DSL实现Bucket聚合
1、DSL实现Bucket聚合
现在，我们要统计所有数据中的酒店品牌有几种，此时可以根据酒店品牌的名称做聚合。
类型为term类型，DSL示例：
# 聚合功能 GET /hotel/_search { "size": 0, //设置size为0，结果中不包含文档，只包含聚合结果 "aggs": { //定义聚合 "brandAgg": { //给聚合起个名字 "terms": { //聚合的类型，按照品牌值聚合，所以选择term "field": "brand", //参与聚合的字段 "size": 10 //希望获取的聚合结果数量 } } } }
运行结果如下：
{ "took" : 1, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 201, "relation" : "eq" }, "max_score" : null, "hits" : [ ] }, "aggregations" : { "brandAgg" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 39, "buckets" : [ { "key" : "7天酒店", "doc_count" : 30 }, { "key" : "如家", "doc_count" : 30 }, { "key" : "皇冠假日", "doc_count" : 17 }, { "key" : "速8", "doc_count" : 15 }, { "key" : "万怡", "doc_count" : 13 }, { "key" : "华美达", "doc_count" : 13 }, { "key" : "和颐", "doc_count" : 12 }, { "key" : "万豪", "doc_count" : 11 }, { "key" : "喜来登", "doc_count" : 11 }, { "key" : "希尔顿", "doc_count" : 10 } ] } } }
2、Bucket聚合-聚合结果排序
默认情况下，Bucket聚合会统计Bucket内的文档数量，记为_count，并且按照_count降序排序。
我们可以修改结果排序方式：
# 聚合功能，自定义排序规则 GET /hotel/_search { "size": 0, "aggs": { "brandAgg": { "terms": { "field": "brand", "size": 10, "order": { "_count": "asc" //按照_count升序排列 } } } } }
运行结果如下：
{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 201, "relation" : "eq" }, "max_score" : null, "hits" : [ ] }, "aggregations" : { "brandAgg" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 130, "buckets" : [ { "key" : "万丽", "doc_count" : 2 }, { "key" : "丽笙", "doc_count" : 2 }, { "key" : "君悦", "doc_count" : 4 }, { "key" : "豪生", "doc_count" : 6 }, { "key" : "维也纳", "doc_count" : 7 }, { "key" : "凯悦", "doc_count" : 8 }, { "key" : "希尔顿", "doc_count" : 10 }, { "key" : "汉庭", "doc_count" : 10 }, { "key" : "万豪", "doc_count" : 11 }, { "key" : "喜来登", "doc_count" : 11 } ] } } }
3、Bucket聚合-限定聚合范围
默认情况下，Bucket聚合是对索引库的所有文档做聚合，我们可以限定要聚合的文档范围，只要添加query条件即可。
示例：
# 聚合功能，限定聚合范围 GET /hotel/_search { "query": { "range": { "price": { "lte": 200 //只对200元以下的文档聚合 } } }, "size": 0, "aggs": { "brandAgg": { "terms": { "field": "brand", "size": 10, "order": { "_count": "asc" } } } } }
运行结果如下：
{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 17, "relation" : "eq" }, "max_score" : null, "hits" : [ ] }, "aggregations" : { "brandAgg" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "7天酒店", "doc_count" : 1 }, { "key" : "汉庭", "doc_count" : 1 }, { "key" : "速8", "doc_count" : 2 }, { "key" : "如家", "doc_count" : 13 } ] } } }
4、总结
1）aggs代表聚合，与query同级，此时query的作用是？
- 限定聚合的的文档范围
2）聚合必须的三要素是什么？
- 聚合名称
- 聚合类型
- 聚合字段
3）聚合可配置属性有哪些？
- size：指定聚合结果数量
- order：指定聚合结果排序方式
- field：指定聚合字段
三、数据聚合-DSL实现Metric聚合
例如：我们要求获取每个品牌的用户评分的min、max、avg等值。
我们可以利用stats聚合：
# 嵌套聚合Metric GET /hotel/_search { "size": 0, "aggs": { "brandAgg": { "terms": { "field": "brand", "size": 10, "order": { "scoreAgg.avg": "desc" //对桶里面的数据做排序 } }, "aggs": { //是brandAgg聚合的子聚合，也就是分组后对每组分别计算 "scoreAgg": { //聚合名称 "stats": { //聚合类型，这里stats可以计算min、max、avg等 "field": "score" //聚合字段，这里是score } } } } } }
运行结果如下所示：
{ "took" : 0, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 201, "relation" : "eq" }, "max_score" : null, "hits" : [ ] }, "aggregations" : { "brandAgg" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 111, "buckets" : [ { "key" : "万丽", "doc_count" : 2, "scoreAgg" : { "count" : 2, "min" : 46.0, "max" : 47.0, "avg" : 46.5, "sum" : 93.0 } }, { "key" : "凯悦", "doc_count" : 8, "scoreAgg" : { "count" : 8, "min" : 45.0, "max" : 47.0, "avg" : 46.25, "sum" : 370.0 } }, { "key" : "和颐", "doc_count" : 12, "scoreAgg" : { "count" : 12, "min" : 44.0, "max" : 47.0, "avg" : 46.083333333333336, "sum" : 553.0 } }, { "key" : "丽笙", "doc_count" : 2, "scoreAgg" : { "count" : 2, "min" : 46.0, "max" : 46.0, "avg" : 46.0, "sum" : 92.0 } }, { "key" : "喜来登", "doc_count" : 11, "scoreAgg" : { "count" : 11, "min" : 44.0, "max" : 48.0, "avg" : 46.0, "sum" : 506.0 } }, { "key" : "皇冠假日", "doc_count" : 17, "scoreAgg" : { "count" : 17, "min" : 44.0, "max" : 48.0, "avg" : 46.0, "sum" : 782.0 } }, { "key" : "万豪", "doc_count" : 11, "scoreAgg" : { "count" : 11, "min" : 43.0, "max" : 47.0, "avg" : 45.81818181818182, "sum" : 504.0 } }, { "key" : "万怡", "doc_count" : 13, "scoreAgg" : { "count" : 13, "min" : 44.0, "max" : 48.0, "avg" : 45.69230769230769, "sum" : 594.0 } }, { "key" : "君悦", "doc_count" : 4, "scoreAgg" : { "count" : 4, "min" : 44.0, "max" : 47.0, "avg" : 45.5, "sum" : 182.0 } }, { "key" : "希尔顿", "doc_count" : 10, "scoreAgg" : { "count" : 10, "min" : 37.0, "max" : 48.0, "avg" : 45.4, "sum" : 454.0 } } ] } } }
四、数据聚合-多条件聚合
需求：搜索页面中的城市、星级、品牌等信息不应该是在页面写死，而是通过聚合索引库中的酒店数据得来的。
示例：
# 多条件聚合 GET /hotel/_search { "query": { "bool": { "must": [ { "match": { "all": "酒店" } } ], "should": [ { "term": { "brand": "皇冠假日" } }, { "term": { "brand": "华美达" } } ], "must_not": [ { "range": { "price": { "lte": 500 } } } ], "filter": [ { "range": { "score": { "gte": 45 } } } ], "minimum_should_match": 1, "boost": 1 } }, "size": 0, "aggs": { "cityAgg": { "terms": { "field": "city", "size": 10, "order": { "_count": "desc" } } }, "starNameAgg": { "terms": { "field": "starName", "size": 10, "order": { "_count": "desc" } } }, "brandAgg": { "terms": { "field": "brand", "size": 10, "order": { "scoreAgg.avg": "desc" } }, "aggs": { "scoreAgg": { "stats": { "field": "score" } } } } } }
运行结果如下：
{ "took" : 1, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : { "value" : 17, "relation" : "eq" }, "max_score" : null, "hits" : [ ] }, "aggregations" : { "brandAgg" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "皇冠假日", "doc_count" : 12, "scoreAgg" : { "count" : 12, "min" : 45.0, "max" : 48.0, "avg" : 46.416666666666664, "sum" : 557.0 } }, { "key" : "华美达", "doc_count" : 5, "scoreAgg" : { "count" : 5, "min" : 45.0, "max" : 46.0, "avg" : 45.2, "sum" : 226.0 } } ] }, "starNameAgg" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "五钻", "doc_count" : 7 }, { "key" : "五星级", "doc_count" : 5 }, { "key" : "四星级", "doc_count" : 3 }, { "key" : "四钻", "doc_count" : 2 } ] }, "cityAgg" : { "doc_count_error_upper_bound" : 0, "sum_other_doc_count" : 0, "buckets" : [ { "key" : "上海", "doc_count" : 10 }, { "key" : "北京", "doc_count" : 4 }, { "key" : "深圳", "doc_count" : 3 } ] } } }
至此本文就全部介绍完了，如果觉得对您有所启发请记得点个赞哦！！！
此文由博主精心撰写转载请保留此原文链接：https://www.cnblogs.com/xyh9039/p/18093166
版权声明：如有雷同纯属巧合，如有侵权请及时联系本人修改，谢谢！！！
