【flink】同一份流可以sink多个

结论

同一个sourceStream可以进行多个处理输出多个sink
每次算子操作都会产生新的对象,对之前的DataStream不会影响

  // 创建数据流
        DataStream<TaskNodeInstance> sourceStream = StreamCommon.getKafkaSourceStream(parameters, env);

        // 对消息进行过滤
        DataStream<TaskNodeInstance> filterStream = sourceStream
                .filter(new NotNullFilter())
                .filter(new BasicFilter());
                

        // 业务1.失败响应码统计
        filterStream.filter(new RespCodeFilter())
                .flatMap(new RespCodeCountMap())
                .keyBy("taskId")
                .timeWindow(Time.seconds(10))
                .reduce(new RespCodeCountReduce(), new RespCodeCountWindowFunction())
                .addSink(new RespCodeStatSink(parameters)).name("RespCodeStatSink:" + profile);

        // 业务2.智能推荐位结果统计
        filterStream.filter(new RecommendFilter())
                .map(new RecommendCountMap())
                .keyBy("taskId")
                .timeWindow(Time.seconds(10))
                .reduce(new RecommendCountReduce(), new RecommendCountWindow())
                .addSink(new RecommendSink(parameters)).name("RecommendCountSink:" + profile)
                ;



参考:https://blog.csdn.net/qq_44112474/article/details/122717134

posted @ 2022-10-26 16:01  xiaoyu_jane  阅读(2124)  评论(0编辑  收藏  举报