fxjwind

摘要： 读accumlator JobManager 在job finish的时候会汇总accumulator的值， newJobStatus match { case JobStatus.FINISHED => try { val accumulatorResults = executionGraph.getAccumulatorsSerialized() va... 阅读全文

摘要： http://wuchong.me/blog/2016/04/26/flink-internals-how-to-handle-backpressure/ https://ci.apache.org/projects/flink/flink-docs-master/internals/back_pressure_monitoring.html 反压之所以会是需要的，因为当源端流量过大，而... 阅读全文

摘要： 先看例子， final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); DataStream> stream = env.addSource(...); stream .keyBy(0) .timeWindow(Time.of(2500, MILLI... 阅读全文

摘要： PS参数详解 http://blog.csdn.net/hanner_cheung/article/details/6081440 JVM 参数 JVM调优总结 -Xms -Xmx -Xmn –Xss JStack JStack分析cpu消耗过高问题 JMap http://my.oschina.n 阅读全文

摘要： http://apex.apache.org/docs.html https://apex.apache.org/docs/apex/application_development/ 阅读全文

摘要： 先看 AbstractFetcher 这个可以理解就是，consumer中具体去kafka读数据的线程，一个fetcher可以同时读多个partitions的数据来看看 /** * Base class for all fetchers, which implement the connections to Kafka brokers and * pull records f... 阅读全文

摘要： 2.1 Producer API We encourage all new development to use the new Java producer. This client is production tested and generally both faster and more fu 阅读全文

摘要： https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/state.html#custom-serialization-for-managed-state All transformations in Flink may look like functions (in the functional pro... 阅读全文

摘要： https://cloud.google.com/dataflow/blog/dataflow-beam-and-spark-comparison https://github.com/apache/incubator-beam https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101 https://www.ore... 阅读全文

摘要： https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/event_timestamps_watermarks.html To work with Event Time, streaming programs need to set the time characteristic accordin... 阅读全文