2021.06.04 刘一辰的软件工程随笔

今日目标
：mapToPair


今日总结：.

JavaPairDStream<String, Integer> WordPairDStream = 
                WordDSteram.mapToPair(
                        
                        new PairFunction<String, String, Integer>() {

                            private static final long serialVersionUID = 2101884706537316002L;

                            @Override
                            public Tuple2<String, Integer> call(String word) throws Exception {

                                return new Tuple2<String, Integer>(word, 1);
                            }
                        });
        
        // reduceByKeyAndWindow
        JavaPairDStream<String, Integer> WindowWordDStream = 
                WordPairDStream.reduceByKeyAndWindow(
                        new Function2<Integer, Integer, Integer>() {
                    
                            private static final long serialVersionUID = -358144101893232390L;

                            @Override
                            public Integer call(Integer v1, Integer v2) throws Exception {

                                return v1 + v2;
                            }
                        }, 
                        Durations.seconds(10), 
                        Durations.seconds(60)
                        );
        
        // sort
        JavaPairDStream<String, Integer> ResultSortDStream = 
                WindowWordDStream.transformToPair(
                        
                    new Function<JavaPairRDD<String,Integer>, JavaPairRDD<String,Integer>>() {

                        private static final long serialVersionUID = 1441798634812792342L;

                        @Override
                        public JavaPairRDD<String, Integer> call(JavaPairRDD<String, Integer> unSortRDD)
                                throws Exception {
                            
                            JavaPairRDD<String, Integer> sortRDD = unSortRDD
                            .mapToPair(
                                    
                                new PairFunction<Tuple2<String,Integer>, Integer, String>() {

                                    private static final long serialVersionUID = -3715362497048144520L;

                                    @Override
                                    public Tuple2<Integer, String> call(Tuple2<String, Integer> tuple) throws Exception {

                                        return new Tuple2<Integer, String>(tuple._2, tuple._1);
                                    }
                                })

明日目标：

sortByKey