异常记录___处理方法待记录

 

 

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
	at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
	at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:638)
	at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
	at cn.crawler.mft_first.HotItems2.main(HotItems2.java:144)
Caused by: AsynchronousException{java.lang.Exception: Could not materialize checkpoint 14 for operator Window(SlidingEventTimeWindows(3600000, 180000), EventTimeTrigger, CountAgg, WindowResultFunction) (1/1).}
	at org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointExceptionHandler.tryHandleCheckpointException(StreamTask.java:1153)
	at org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.handleExecutionException(StreamTask.java:947)
	at org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.run(StreamTask.java:884)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Could not materialize checkpoint 14 for operator Window(SlidingEventTimeWindows(3600000, 180000), EventTimeTrigger, CountAgg, WindowResultFunction) (1/1).
	at org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.handleExecutionException(StreamTask.java:942)
	... 6 more
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Size of the state is larger than the maximum permitted memory-backed state. Size=14068058 , maxSize=5242880 . Consider using a different state backend, like the File System State backend.
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:394)
	at org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.<init>(OperatorSnapshotFinalizer.java:47)
	at org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.run(StreamTask.java:853)
	... 5 more
Caused by: java.io.IOException: Size of the state is larger than the maximum permitted memory-backed state. Size=14068058 , maxSize=5242880 . Consider using a different state backend, like the File System State backend.
	at org.apache.flink.runtime.state.memory.MemCheckpointStreamFactory.checkSize(MemCheckpointStreamFactory.java:64)
	at org.apache.flink.runtime.state.memory.MemCheckpointStreamFactory$MemoryCheckpointOutputStream.closeAndGetBytes(MemCheckpointStreamFactory.java:145)
	at org.apache.flink.runtime.state.memory.MemCheckpointStreamFactory$MemoryCheckpointOutputStream.closeAndGetHandle(MemCheckpointStreamFactory.java:126)
	at org.apache.flink.runtime.state.CheckpointStreamWithResultProvider$PrimaryStreamOnly.closeAndFinalizeCheckpointStreamResult(CheckpointStreamWithResultProvider.java:77)
	at org.apache.flink.runtime.state.heap.HeapSnapshotStrategy$1.callInternal(HeapSnapshotStrategy.java:199)
	at org.apache.flink.runtime.state.heap.HeapSnapshotStrategy$1.callInternal(HeapSnapshotStrategy.java:158)
	at org.apache.flink.runtime.state.AsyncSnapshotCallable.call(AsyncSnapshotCallable.java:75)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:391)
	... 7 more

Process finished with exit code 1

 

04-09 16:14:27.748 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":3747017,"itemId":2639035,"timestamp":1511663037,"userId":574109}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.748 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2748804,"itemId":4022764,"timestamp":1511678146,"userId":117095}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.748 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2748804,"itemId":4022764,"timestamp":1511678146,"userId":117095}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

=============={"behavior":"pv","categoryId":3673501,"itemId":2805556,"timestamp":1511663429,"userId":465435}
=============={"behavior":"pv","categoryId":4804883,"itemId":4192527,"timestamp":1511663430,"userId":1009325}
=============={"behavior":"pv","categoryId":84264,"itemId":2371984,"timestamp":1511660547,"userId":34710}
2020-04-09 16:14:27.749 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":171529,"itemId":4179432,"timestamp":1511660139,"userId":460188}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.749 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":171529,"itemId":4179432,"timestamp":1511660139,"userId":460188}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

=============={"behavior":"pv","categoryId":1575622,"itemId":4031722,"timestamp":1511680995,"userId":782671}
2020-04-09 16:14:27.750 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"cart","categoryId":154040,"itemId":4465640,"timestamp":1511678146,"userId":77488}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.750 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"cart","categoryId":154040,"itemId":4465640,"timestamp":1511678146,"userId":77488}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.750 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":285583,"itemId":2439102,"timestamp":1511663037,"userId":597163}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.750 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":285583,"itemId":2439102,"timestamp":1511663037,"userId":597163}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

=============={"behavior":"pv","categoryId":550660,"itemId":4996544,"timestamp":1511666041,"userId":568375}
=============={"behavior":"pv","categoryId":2735466,"itemId":3504554,"timestamp":1511659292,"userId":703365}
2020-04-09 16:14:27.752 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":3158249,"itemId":5126546,"timestamp":1511660139,"userId":425679}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.752 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":3158249,"itemId":5126546,"timestamp":1511660139,"userId":425679}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

=============={"behavior":"pv","categoryId":1168771,"itemId":4881184,"timestamp":1511663430,"userId":164193}
2020-04-09 16:14:27.753 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":4801426,"itemId":5070496,"timestamp":1511663037,"userId":423481}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.753 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":4801426,"itemId":5070496,"timestamp":1511663037,"userId":423481}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.753 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2465336,"itemId":1240615,"timestamp":1511678147,"userId":845531}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.753 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2465336,"itemId":1240615,"timestamp":1511678147,"userId":845531}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

=============={"behavior":"pv","categoryId":383749,"itemId":2567216,"timestamp":1511660547,"userId":645841}
=============={"behavior":"pv","categoryId":3002561,"itemId":2317157,"timestamp":1511680995,"userId":42844}
2020-04-09 16:14:27.754 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":1851156,"itemId":4133598,"timestamp":1511660139,"userId":72695}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.754 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":1851156,"itemId":4133598,"timestamp":1511660139,"userId":72695}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.754 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":3381583,"itemId":1955230,"timestamp":1511663037,"userId":940892}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.754 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":3381583,"itemId":1955230,"timestamp":1511663037,"userId":940892}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

=============={"behavior":"pv","categoryId":772629,"itemId":1741298,"timestamp":1511666041,"userId":368855}
=============={"behavior":"pv","categoryId":4170419,"itemId":715934,"timestamp":1511659292,"userId":34416}
2020-04-09 16:14:27.755 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2212343,"itemId":3024985,"timestamp":1511678147,"userId":231896}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

2020-04-09 16:14:27.755 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2212343,"itemId":3024985,"timestamp":1511678147,"userId":231896}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

=============={"behavior":"pv","categoryId":448398,"itemId":2691009,"timestamp":1511663430,"userId":982185}
2020-04-09 16:14:27.756 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2885642,"itemId":1738745,"timestamp":1511660139,"userId":304626}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation

.....................................................
2020-04-09 16:14:29.825 ERROR 9120 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='{"behavior":"pv","categoryId":2520771,"itemId":3221838,"timestamp":1511663089,"userId":297686}' to topic userTest:

org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for userTest-0:120000 ms has passed since batch creation


Process finished with exit code -1