入口类CliFrontend
LinkedHashMap accessOrder default=false=insertion-order
System.setOut 重定向输出流
getManiFest get main class
-d detach模式, false 可以运行任务后无需再控制台保持连接
JobID implement Comparable 内部实现compareTo
parallelism默认传参是-1 ,但是会default为1,区分正在传入的1吧
JobGraph -> JobVertex, ScheduleMode(LAZY/EAGER)
WindowWordCount 启动报错:
Connected to the target VM, address: '127.0.0.1:50957', transport: 'socket' Executing WindowWordCount example with default input data set. Use --input to specify file input. Printing result to stdout. Use --output to specify output path. Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146) at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:627) at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:117) at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1507) at org.apache.flink.streaming.examples.windowing.WindowWordCount.main(WindowWordCount.java:92) Caused by: java.util.concurrent.TimeoutException: Heartbeat of TaskManager with id 281188bf-2ff7-4bb1-9128-b8e0e728e1d9 timed out. at org.apache.flink.runtime.jobmaster.JobMaster$TaskManagerHeartbeatListener.notifyHeartbeatTimeout(JobMaster.java:1149) at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl$HeartbeatMonitor.run(HeartbeatManagerImpl.java:318) at java.util.concurrent.Executors$RunnableAdapter.call$$$capture(Executors.java:511) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java) at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266) at java.util.concurrent.FutureTask.run(FutureTask.java) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:397) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:190) at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at akka.actor.Actor$class.aroundReceive$$$capture(Actor.scala:517) at akka.actor.Actor$class.aroundReceive(Actor.scala) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) at akka.actor.ActorCell.invoke(ActorCell.scala:561) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) at akka.dispatch.Mailbox.run(Mailbox.scala:225) at akka.dispatch.Mailbox.exec(Mailbox.scala:235) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Disconnected from the target VM, address: '127.0.0.1:50957', transport: 'socket' Process finished with exit code 1
错误很明显是因为某种原因导致超时了,若是线上可能需要更改flink-conf.yaml参数akka.ask.timeout和web.timeout都从10s(默认) 修改为100s,但是我本地的运行的模式不会是因为网络什么的导致超时,真正的问题是我的断点没有释放......
查看源码发现最后是在HeartBeatManagerImpl的run()中发出超时(AtomicReference<T>的compareAndSet的经典使用方式):
private final AtomicReference<State> state = new AtomicReference<>(State.RUNNING); ......//忽略部分代码 @Override public void run() { // The heartbeat has timed out if we're in state running if (state.compareAndSet(State.RUNNING, State.TIMEOUT)) { heartbeatListener.notifyHeartbeatTimeout(resourceID); } }