使用elasticsearch做分页查询时,当查询记录超过10000时报错
使用elasticsearch做分页查询时,当查询记录超过10000时,比如我每页100条数据,第100页没问题,101页就报错了;比如每页50条数据,第200页就没问题,201页就报错了,会报如下错误:
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
at org.elasticsearch.action.search.AbstractSearchAsyncAction.onInitialPhaseResult(AbstractSearchAsyncAction.java:223) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.access$100(AbstractSearchAsyncAction.java:58) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:148) ~[na:na]
at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1032) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1134) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1112) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.transport.TransportService$7.onFailure(TransportService.java:629) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:623) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) ~[elasticsearch-5.6.8.jar:5.6.8]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_201]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_201]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_201]
Caused by: org.elasticsearch.transport.RemoteTransportException: [es-xh-1][192.168.5.57:9300][indices:data/read/search[phase/dfs]]
Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: Result window is too large, from + size must be less than or equal to: [10000] but was [195900]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.
at org.elasticsearch.search.DefaultSearchContext.preProcess(DefaultSearchContext.java:202) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.search.query.QueryPhase.preProcess(QueryPhase.java:90) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.search.SearchService.createContext(SearchService.java:480) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:444) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.search.SearchService.executeDfsPhase(SearchService.java:221) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.action.search.SearchTransportService$5.messageReceived(SearchTransportService.java:320) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.action.search.SearchTransportService$5.messageReceived(SearchTransportService.java:317) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:618) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.6.8.jar:5.6.8]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_201]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_201]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_201]
解决方案:
第一种:curl -XPUT http://127.0.0.1:9200/_settings -d '{ "index" : { "max_result_window" : 100000000}}‘
第二种:在config/elasticsearch.yml文件中的最后加上index.max_result_window: 100000000,但是这种方法要注意在最前面加上空格