博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
ElasticSearch java API - 聚合查询-聚合多字段聚合demo
阅读量:7117 次
发布时间:2019-06-28

本文共 7096 字,大约阅读时间需要 23 分钟。

hot3.png

以球员信息为例,player索引的player type包含5个字段,姓名,年龄,薪水,球队,场上位置。

index的mapping为:

"mappings": {	"player": {		"properties": {			"name": {				"index": "not_analyzed",				"type": "string"			},			"age": {				"type": "integer"			},			"salary": {				"type": "integer"			},			"team": {				"index": "not_analyzed",				"type": "string"			},			"position": {				"index": "not_analyzed",				"type": "string"			}		},		"_all": {			"enabled": false		}	}}

索引中的全部数据:
 

 
首先,初始化Builder:

SearchRequestBuilder sbuilder = client.prepareSearch("player").setTypes("player");

接下来举例说明各种聚合操作的实现方法,因为在es的api中,多字段上的聚合操作需要用到子聚合(subAggregation),初学者可能找不到方法(网上资料比较少,笔者在这个问题上折腾了两天,最后度了源码才彻底搞清楚T_T),后边会特意说明多字段聚合的实现方法。另外,聚合后的排序也会单独说明。

  • group by/count

例如要计算每个球队的球员数,如果使用SQL语句,应表达如下:

select team, count(*) as player_count from player group by team;

ES的java api:

TermsBuilder teamAgg= AggregationBuilders.terms("player_count ").field("team");sbuilder.addAggregation(teamAgg);SearchResponse response = sbuilder.execute().actionGet();

 

  • group by多个field

例如要计算每个球队每个位置的球员数,如果使用SQL语句,应表达如下:

select team, position, count(*) as pos_count from player group by team, position;

ES的java api:

TermsBuilder teamAgg= AggregationBuilders.terms("player_count ").field("team");TermsBuilder posAgg= AggregationBuilders.terms("pos_count").field("position");sbuilder.addAggregation(teamAgg.subAggregation(posAgg));SearchResponse response = sbuilder.execute().actionGet();

 

  • max/min/sum/avg

例如要计算每个球队年龄最大/最小/总/平均的球员年龄,如果使用SQL语句,应表达如下:

select team, max(age) as max_age from player group by team;

ES的java api:

TermsBuilder teamAgg= AggregationBuilders.terms("player_count ").field("team");MaxBuilder ageAgg= AggregationBuilders.max("max_age").field("age");sbuilder.addAggregation(teamAgg.subAggregation(ageAgg));SearchResponse response = sbuilder.execute().actionGet();

 

  • 对多个field求max/min/sum/avg

例如要计算每个球队球员的平均年龄,同时又要计算总年薪,如果使用SQL语句,应表达如下:

select team, avg(age)as avg_age, sum(salary) as total_salary from player group by team;

ES的java api:

TermsBuilder teamAgg= AggregationBuilders.terms("team");AvgBuilder ageAgg= AggregationBuilders.avg("avg_age").field("age");SumBuilder salaryAgg= AggregationBuilders.avg("total_salary ").field("salary");sbuilder.addAggregation(teamAgg.subAggregation(ageAgg).subAggregation(salaryAgg));SearchResponse response = sbuilder.execute().actionGet();

 

  • 聚合后对Aggregation结果排序

例如要计算每个球队总年薪,并按照总年薪倒序排列,如果使用SQL语句,应表达如下:

select team, sum(salary) as total_salary from player group by team order by total_salary desc;

ES的java api:

TermsBuilder teamAgg= AggregationBuilders.terms("team").order(Order.aggregation("total_salary ", false);SumBuilder salaryAgg= AggregationBuilders.avg("total_salary ").field("salary");sbuilder.addAggregation(teamAgg.subAggregation(salaryAgg));SearchResponse response = sbuilder.execute().actionGet();

需要特别注意的是,排序是在TermAggregation处执行的,Order.aggregation函数的第一个参数是aggregation的名字,第二个参数是boolean型,true表示正序,false表示倒序。 

  • Aggregation结果条数的问题

默认情况下,search执行后,仅返回10条聚合结果,如果想反悔更多的结果,需要在构建TermsBuilder 时指定size:

TermsBuilder teamAgg= AggregationBuilders.terms("team").size(15);

 

  • Aggregation结果的解析/输出

得到response后:

Map
aggMap = response.getAggregations().asMap();StringTerms teamAgg= (StringTerms) aggMap.get("keywordAgg");Iterator
teamBucketIt = teamAgg.getBuckets().iterator();while (teamBucketIt .hasNext()) {Bucket buck = teamBucketIt .next();//球队名String team = buck.getKey();//记录数long count = buck.getDocCount();//得到所有子聚合Map subaggmap = buck.getAggregations().asMap();//avg值获取方法double avg_age= ((InternalAvg) subaggmap.get("avg_age")).getValue();//sum值获取方法double total_salary = ((InternalSum) subaggmap.get("total_salary")).getValue();//...//max/min以此类推}

 

  • 总结

综上,聚合操作主要是调用了SearchRequestBuilder的addAggregation方法,通常是传入一个TermsBuilder,子聚合调用TermsBuilder的subAggregation方法,可以添加的子聚合有TermsBuilder、SumBuilder、AvgBuilder、MaxBuilder、MinBuilder等常见的聚合操作。

 
从实现上来讲,SearchRequestBuilder在内部保持了一个私有的 SearchSourceBuilder实例, SearchSourceBuilder内部包含一个List<AbstractAggregationBuilder>,每次调用addAggregation时会调用 SearchSourceBuilder实例,添加一个AggregationBuilder。
同样的,TermsBuilder也在内部保持了一个List<AbstractAggregationBuilder>,调用addAggregation方法(来自父类addAggregation)时会添加一个AggregationBuilder。有兴趣的读者也可以阅读源码的实现。
 
如果有什么问题,欢迎一起讨论,如果文中有什么错误,欢迎批评指正。
 
注:文中使用的Elastic Search API版本为2.3.2

 

 

public List
> queryAggregationsByAttr(BoolQueryBuilder boolQueryBld){ List
> result = new ArrayList<>(); NestedBuilder nestedBuilder= AggregationBuilders.nested("negstedAttr").path("spuAttrList"); //属性名称分组 TermsBuilder tbName= AggregationBuilders.terms("attrNameAgg").field("spuAttrList.name"); //嵌套查询的子查询中分组count TermsBuilder tb= AggregationBuilders.terms("attrvIdAgg").field("spuAttrList.attrvId"); //属性值字段 TermsBuilder tbVal= AggregationBuilders.terms("attrValAgg").field("spuAttrList.value"); NestedBuilder all = nestedBuilder.subAggregation(tbName.subAggregation(tb.subAggregation(tbVal))); NativeSearchQueryBuilder nativeQueryBuilderAgg = new NativeSearchQueryBuilder() .withQuery(boolQueryBld) .withIndices("skus").withTypes("skus") .addAggregation(all); SearchQuery searchQueryAgg = nativeQueryBuilderAgg.build(); Aggregations aggregations = elasticsearchTemplate.query(searchQueryAgg, new ResultsExtractor
() { @Override public Aggregations extract(SearchResponse response) { return response.getAggregations(); } }); Map
map=aggregations.asMap(); for(String s:map.keySet()){ if("negstedAttr".equals(s)) { InternalNested internalNested = (InternalNested)map.get(s); //属性名称 StringTerms nameTerms=(StringTerms) internalNested.getAggregations().get("attrNameAgg"); //属性子表id for(org.elasticsearch.search.aggregations.bucket.terms.Terms.Bucket tbket:nameTerms.getBuckets()){ //对应一组属性值 Map
categoryIdsMapTerms = new HashMap
(); categoryIdsMapTerms.put("typeId", "attrValueIds"); categoryIdsMapTerms.put("typeName", tbket.getKeyAsString()); LongTerms attrvIdTerms=(LongTerms)tbket.getAggregations().asMap().get("attrvIdAgg"); if(attrvIdTerms == null || CollectionUtils.isEmpty(attrvIdTerms.getBuckets())) { continue; } List
> dataList = new ArrayList<>(); //属性子表val for(org.elasticsearch.search.aggregations.bucket.terms.Terms.Bucket attrIdB : attrvIdTerms.getBuckets()) { //dataListMap Map
dataListMap = new HashMap
(); Long attrvId = (Long) attrIdB.getKeyAsNumber(); StringTerms valTerms=(StringTerms) attrIdB.getAggregations().asMap().get("attrValAgg"); if(valTerms == null || CollectionUtils.isEmpty(valTerms.getBuckets())) { continue; } String attrValStr = valTerms.getBuckets().get(0).getKeyAsString(); dataListMap.put("id", attrvId); dataListMap.put("name", attrValStr); dataList.add(dataListMap); } if(!CollectionUtils.isEmpty(dataList)) { categoryIdsMapTerms.put("dataList", dataList); } result.add(categoryIdsMapTerms); } } } return result; }

 

转载于:https://my.oschina.net/xiaominmin/blog/1845353

你可能感兴趣的文章
步步为营 .NET 设计模式学习笔记 十七、Flyweight(享元模式)
查看>>
增量式 爬虫
查看>>
备份整个schema的数据
查看>>
DAY11-MYSQL索引原理与慢查询优化
查看>>
第三次实验第二项
查看>>
JavaScript 开发的技巧
查看>>
mac 常用命令
查看>>
001——搭建OpenCV实验环境
查看>>
Windows Server 2008更改SID
查看>>
使用AOP的方式监测方法执行耗时
查看>>
cryptopunks的代码解释
查看>>
Linux中查看你的用户是否为root用户
查看>>
正则表达式
查看>>
js 常用的功能
查看>>
C语言和sh脚本的杂交代码
查看>>
oracle常用加解密函数
查看>>
linux中redis安装
查看>>
2018-2019 Exp3 免杀原理与实践
查看>>
linux install mysql
查看>>
CF871D Paths
查看>>