摘要：读accumlator JobManager 在job finish的时候会汇总accumulator的值， newJobStatus match { case JobStatus.FINISHED => try { val accumulatorResults = executionGraph.getAccumulatorsSerialized() va... 阅读全文

摘要：http://wuchong.me/blog/2016/04/26/flink-internals-how-to-handle-backpressure/ https://ci.apache.org/projects/flink/flink-docs-master/internals/back_pressure_monitoring.html 反压之所以会是需要的，因为当源端流量过大，而... 阅读全文

摘要：先看例子， final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); DataStream> stream = env.addSource(...); stream .keyBy(0) .timeWindow(Time.of(2500, MILLI... 阅读全文

摘要：先看 AbstractFetcher 这个可以理解就是，consumer中具体去kafka读数据的线程，一个fetcher可以同时读多个partitions的数据来看看 /** * Base class for all fetchers, which implement the connections to Kafka brokers and * pull records f... 阅读全文

摘要：https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/state.html#custom-serialization-for-managed-state All transformations in Flink may look like functions (in the functional pro... 阅读全文

摘要：https://cloud.google.com/dataflow/blog/dataflow-beam-and-spark-comparison https://github.com/apache/incubator-beam https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101 https://www.ore... 阅读全文

摘要：https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/event_timestamps_watermarks.html To work with Event Time, streaming programs need to set the time characteristic accordin... 阅读全文

摘要：Example Program The following program is a complete, working example of streaming window word count application, that counts the words coming from a w 阅读全文

摘要：https://cwiki.apache.org/confluence/display/FLINK/Flink+Internals Memory Management (Batch API) Introduction Memory management in Flink serves the purpose to control how much memory certain runt... 阅读全文

摘要：https://flink.apache.org/news/2015/09/16/off-heap-memory.html Running data-intensive code in the JVM and making it well-behaved is tricky. Systems that put billions of data objects naively onto th... 阅读全文

摘要：http://flink.apache.org/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html Join Processing in Apache Flink In this blog post, we cut through Apache Flink’s layered architecture and take ... 阅读全文

摘要：http://www.36dsj.com/archives/33650 http://flink.apache.org/news/2015/05/11/Juggling-with-Bits-and-Bytes.html http://www.bigsynapse.com/addressing-big-data-performance ，addressing-big-data-performan... 阅读全文

摘要：https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/programming_guide.html Example Program 编程的风格和spark很类似， ExecutionEnvironment -- SparkContext DataSet – RDD Transformations 这... 阅读全文