基于百度地图SDK和Elasticsearch GEO查询的地理围栏分析系统（2）-查询实现
在上一篇博客中，我们准备好了数据。现在数据已经以我们需要的格式，存放在Elasticsearch中了。
本文讲述如何在Elasticsearch中进行空间GEO查询和聚合查询，以及如何准备ajax接口。
平台的服务端部分使用的springboot+mybatis的基本开发模式。工程结构如下。
可以看到本工程有三个module：
1）moonlight-web是controller和service层的实现；
2）moonlight-dsl封装了ES空间索引查询和聚合查询的方法；
3）moonlight-dao封装了持久化地理围栏的方法。
我们以客户端请求的处理顺序为例进行讲解。
1、controller
在controller层中，我们实现了4个接口，分别是circle、box、polygon、heatmap，也就是圆形圈选，矩形圈选，多边形圈选和热力图。
先看一下代码的具体实现。
@RestController @RequestMapping("/moonlight") public class MoonlightController { protected final Logger logger = LoggerFactory.getLogger(this.getClass()); @Autowired private MoonlightService moonlightService; @RequestMapping(value = "/circle", method = RequestMethod.GET) public ResponseEntity<Response> circle(HttpServletRequest request, HttpServletResponse response) { String point = request.getParameter("point"); String radius = request.getParameter("radius"); try { Map<String, Object> result = moonlightService.circle(point, radius); logger.info("circle圈选成功, points={}, radius={}, result={}", point, radius, result); return new ResponseEntity<>( new Response(ResultCode.SUCCESS, "circle圈选成功", result), HttpStatus.OK); } catch (Exception e) { logger.error("circle圈选失败, points={}, radius={}, result={}", point, radius, null, e); return new ResponseEntity<>( new Response(ResultCode.EXCEPTION, "circle圈选失败", null), HttpStatus.INTERNAL_SERVER_ERROR); } } @RequestMapping(value = "/box", method = RequestMethod.GET) public ResponseEntity<Response> box(HttpServletRequest request, HttpServletResponse response) { String point1 = request.getParameter("point1"); String point2 = request.getParameter("point2"); String point3 = request.getParameter("point3"); String point4 = request.getParameter("point4"); try { Map<String, Object> result = moonlightService.boundingBox(point1, point2, point3, point4); logger.info("box圈选成功, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, result); return new ResponseEntity<>( new Response(ResultCode.SUCCESS, "box圈选成功", result), HttpStatus.OK); } catch (Exception e) { logger.error("box圈选失败, point1={}, point2={}, point3={}, point4={}, result={}", point1, point2, point3, point4, null, e); return new ResponseEntity<>( new Response(ResultCode.EXCEPTION, "box圈选失败", null), HttpStatus.INTERNAL_SERVER_ERROR); } } @RequestMapping(value = "/polygon", method = RequestMethod.GET) public ResponseEntity<Response> polygon(HttpServletRequest request, HttpServletResponse response) { List<String> points = new ArrayList<>(); Enumeration<String> paramNames = request.getParameterNames(); while (paramNames.hasMoreElements()) { String paramName = paramNames.nextElement(); if (paramName.startsWith("point")) { points.add(request.getParameter(paramName)); } } try { Map<String, Object> result = moonlightService.polygon(points); logger.info("polygon圈选成功, points={}, result={}", points, result); return new ResponseEntity<>( new Response(ResultCode.SUCCESS, "polygon圈选成功", result), HttpStatus.OK); } catch (Exception e) { logger.error("polygon圈选失败, points={}, result={}", points, null, e); return new ResponseEntity<>( new Response(ResultCode.EXCEPTION, "polygon圈选失败", null), HttpStatus.INTERNAL_SERVER_ERROR); } } @RequestMapping(value = "/heatMap", method = RequestMethod.GET) public ResponseEntity<Response> heatMap(HttpServletRequest request, HttpServletResponse response) { try { List<Map<String, Object>> result = moonlightService.heatMap(); logger.info("heatMap请求成功, result={}", result); return new ResponseEntity<>( new Response(ResultCode.SUCCESS, "heatMap请求成功", result), HttpStatus.OK); } catch (Exception e) { logger.error("heatMap请求失败, result={}", null, e); return new ResponseEntity<>( new Response(ResultCode.EXCEPTION, "heatMap请求失败", null), HttpStatus.INTERNAL_SERVER_ERROR); } } }
我们以圆形圈选（circle接口）为例，circle接口传入两个参数，一个是point，也就是中心点坐标，一个是radius，也就是半径，它干的事情就是圈选出，point点周围radius长度内的所有订单数据，具体实现是调用了service层的方法，controller得到圈选的数据后就返回了。
下面我们来看一下service层。
2、service
service层是具体业务的实现。我们这里的service仍然比较简单，可以看到只是初始化了esDao的句柄，然后进行es的geo查询。
先看一下具体代码。
@Service public class MoonlightService { protected final Logger logger = LoggerFactory.getLogger(this.getClass()); @Autowired private ESDao esDao; public Map<String, Object> circle(String point, String radius) { POI center = new POI(point); return esDao.circle(center, Double.parseDouble(radius)); } public Map<String, Object> boundingBox(String point1, String point2, String point3, String point4) { POI poi1 = new POI(point1); POI poi2 = new POI(point2); POI poi3 = new POI(point3); POI poi4 = new POI(point4); POI topLeft = getTopLeft(poi1, poi2, poi3, poi4); POI bottomRight = getBottomRight(poi1, poi2, poi3, poi4); logger.info("topLeft - lat={}, lng={}, bottomRight - lat={}, lng={}", topLeft.getLat(), topLeft.getLng(), bottomRight.getLat(), bottomRight.getLng()); return esDao.boundingBox(topLeft, bottomRight); } public Map<String, Object> polygon(List<String> points) { List<POI> poiList = new ArrayList<>(); for (String point : points) { POI poi = new POI(point); poiList.add(poi); } return esDao.polygon(poiList); } public List<Map<String, Object>> heatMap() { return esDao.heatMap(); } private POI getTopLeft(POI poi1, POI poi2, POI poi3, POI poi4) { POI topLeft = new POI(); List<Double> latList = new ArrayList<>(); List<Double> lngList = new ArrayList<>(); latList.add(poi1.getLat()); latList.add(poi2.getLat()); latList.add(poi3.getLat()); latList.add(poi4.getLat()); Collections.sort(latList); Double minLat = latList.get(0); Double maxLat = latList.get(3); lngList.add(poi1.getLng()); lngList.add(poi2.getLng()); lngList.add(poi3.getLng()); lngList.add(poi4.getLng()); Collections.sort(lngList); Double minLng = lngList.get(0); Double maxLng = lngList.get(3); topLeft.setLat(maxLat); topLeft.setLng(minLng); return topLeft; } private POI getBottomRight(POI poi1, POI poi2, POI poi3, POI poi4) { POI bottomRight = new POI(); List<Double> latList = new ArrayList<>(); List<Double> lngList = new ArrayList<>(); latList.add(poi1.getLat()); latList.add(poi2.getLat()); latList.add(poi3.getLat()); latList.add(poi4.getLat()); Collections.sort(latList); Double minLat = latList.get(0); Double maxLat = latList.get(3); lngList.add(poi1.getLng()); lngList.add(poi2.getLng()); lngList.add(poi3.getLng()); lngList.add(poi4.getLng()); Collections.sort(lngList); Double minLng = lngList.get(0); Double maxLng = lngList.get(3); bottomRight.setLat(minLat); bottomRight.setLng(maxLng); return bottomRight; } }
我们仍然是以圆形圈选为例，可以看到，service代码的逻辑就是，创建出圈选需要的数据接口，然后调用Dao层进行查询就是了。
circle圈选需要的是一个中心点POI类型，和一个Double半径。
box矩形查询需要的是左上坐标点和右下坐标点，里面有两个函数getTopLeft、getBottomRight分别可以求出矩形的左上点和右下点。
polygon多边形查询需要的是一系列点，这些点顺序的连接所绘制出来的图形就是目标多边形。
heatmap热力图什么参数也不要，将返回一定精度的经纬度计数值，后面我们会详述。
之后所有的service都调用了Dao层的es查询逻辑。所以最重要的一部分是esDao的实现，下面我们就来看一看。
3、Dao
Dao层代码是整个项目的核心，包括对Elasticsearch数据进行圈选和聚合两部分，此外就是热力图数据的准备。
先看一下代码。
@Component public class ESDao { protected final Logger logger = LoggerFactory.getLogger(this.getClass()); @Autowired private ESClient esClient; public Map<String, Object> circle(POI center, Double radius) { TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4}); GeoDistanceRangeQueryBuilder geoDistanceRangeQuery = QueryBuilders.geoDistanceRangeQuery("location") .point(center.getLat(), center.getLng()) .from("0m") .to(String.format("%fm", radius)) .includeLower(true) .includeUpper(true) .optimizeBbox("memory") .geoDistance(GeoDistance.SLOPPY_ARC); QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery); SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj") .setSearchType(SearchType.DFS_QUERY_AND_FETCH) .setQuery(queryBuilder); return agg(search); } public Map<String, Object> boundingBox(POI topLeft, POI bottomRight) { TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4}); GeoBoundingBoxQueryBuilder geoBoundingBoxQuery = QueryBuilders.geoBoundingBoxQuery("location") .topLeft(topLeft.getLat(), topLeft.getLng()) .bottomRight(bottomRight.getLat(), bottomRight.getLng()); QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoBoundingBoxQuery); SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj") .setSearchType(SearchType.DFS_QUERY_AND_FETCH) .setQuery(queryBuilder); return agg(search); } public Map<String, Object> polygon(List<POI> poiList) { TermsQueryBuilder termsQuery = termsQuery("product_id", new double[]{3, 4}); GeoPolygonQueryBuilder geoPolygonQuery = QueryBuilders.geoPolygonQuery("location"); for (POI poi : poiList) { geoPolygonQuery.addPoint(poi.getLat(), poi.getLng()); } QueryBuilder queryBuilder = QueryBuilders.boolQuery().must(termsQuery).must(geoPolygonQuery); SearchRequestBuilder search = esClient.getClient().prepareSearch("moon").setTypes("bj") .setSearchType(SearchType.DFS_QUERY_AND_FETCH) .setQuery(queryBuilder); return agg(search); } public List<Map<String, Object>> heatMap() { TermQueryBuilder queryBuilder = termQuery("date", "2017-11-24"); SearchRequestBuilder searchRequestBuilder = esClient.getClient() .prepareSearch("moon").setTypes("bj"); SearchResponse response = searchRequestBuilder .setQuery(queryBuilder) .setFrom(0).setSize(10000) .setExplain(true).execute().actionGet(); SearchHits hits = response.getHits(); Map<String, Integer> countMap = new HashMap<>(); for (SearchHit hit : hits) { Map<String, Object> source = hit.getSource(); Map<String, Double> locationMap = (Map<String, Double>) source.get("location"); DecimalFormat df = new DecimalFormat("#.000"); String lat = df.format(locationMap.get("lat")); String lon = df.format(locationMap.get("lon")); String key = lat+"-"+lon; if (countMap.containsKey(key)) { countMap.put(key, countMap.get(key) + 1); } else { countMap.put(key, 1); } } List<Map<String, Object>> result = new ArrayList<>(); for (Map.Entry<String, Integer> entry : countMap.entrySet()) { String lat = entry.getKey().split("-")[0]; String lon = entry.getKey().split("-")[1]; Integer count = entry.getValue(); Map<String, Object> map = new HashMap<>(); map.put("lat", Double.parseDouble(lat)); map.put("lng", Double.parseDouble(lon)); map.put("count", count); result.add(map); } return result; } private Map<String, Object> agg(SearchRequestBuilder search) { Map<String, Object> resultMap = new HashMap<>(); GroupBy groupBy = new GroupBy(search, "date_group", "date", true); groupBy.addSumAgg("pre_total_fee_sum", "pre_total_fee"); groupBy.addCountAgg("order_id_count", "order_id"); groupBy.addSumAgg("cancel_count", "type"); List<String> xAxis = new ArrayList<>(); List<String> profits = new ArrayList<>(); List<String> totals = new ArrayList<>(); List<String> cancelRatios = new ArrayList<>(); List<Map<String, Object>> details = new ArrayList<>(); Map<String, Object> groupbyResponse = groupBy.getGroupbyResponse(); for (Map.Entry<String, Object> entry : groupbyResponse.entrySet()) { String date = entry.getKey(); xAxis.add(date); Map<String, String> subAggMap = (Map<String, String>) entry.getValue(); String profit = subAggMap.get("pre_total_fee_sum"); profits.add(profit); String total = subAggMap.get("order_id_count"); totals.add(total); String cancelRatioDouble = new DecimalFormat("#.0000").format( Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count")) ); String cancelRatio = new DecimalFormat("0.00%").format( Double.parseDouble(subAggMap.get("cancel_count")) / Double.parseDouble(subAggMap.get("order_id_count")) ); cancelRatios.add(cancelRatioDouble); Map<String, Object> tempMap = new HashMap<>(); tempMap.put("profit", profit); tempMap.put("total", total); tempMap.put("cancelRatio", cancelRatio); tempMap.put("date", date); details.add(tempMap); } resultMap.put("xAxis", xAxis); resultMap.put("profit", profits); resultMap.put("total", totals); resultMap.put("cancelRatio", cancelRatios); resultMap.put("detail", details); return resultMap; } }
es圈选部分
circle为例，我们构造了一个geoDistanceRangeQuery查询，这个查询到上一篇博客准备好的moon索引，bj type中去将数据圈选出来。
类似的我们有矩形geoBoundingBoxQuery查询，多边形geoPolygonQuery查询，具体构造查询的方式可以参照代码，这个代码还是很简单的，熟悉es的同学很快可以上手并且实现这样的查询，不熟悉的话可以自行百度一下。如果还有其他的查询条件，可以通过QueryBuilders.boolQuery().must(termsQuery).must(geoDistanceRangeQuery)加入，例如我这里在圈选之外加入了一个terms查询，这个查询相当于sql中的where product_id in (3,4) and ...。
es聚合部分
es聚合部分做的事情是，对查询出的订单进行了聚合运算，例如求和和计数，是两个最常见的运算，这部分在这里不详细叙述了，请参见这篇博客。
热力图
这里要额外说明的是，热力图heatmap，和圈选不一样，他是查询了最近一天type=bj分区里的所有数据，按照坐标进行了计数，可以看到的是，计数的时候，我们指定了精度，这里是小数点后三位有效数字
DecimalFormat df = new DecimalFormat("#.000"); String lat = df.format(locationMap.get("lat")); String lon = df.format(locationMap.get("lon")); String key = lat+"-"+lon;
然后将计数结果返回。百度地图SDK会将计数结果绘制成热力图，这个不用我们管，我会在另一篇博客中讲述这个过程。
到这里，整个工程的基本功能就介绍完了。
