Storm Trident示例batchGlobal

batchGlobal把同属于一个batch的tuples分配到相同的partition当中。

省略部分代码,省略部分可参考:https://blog.csdn.net/nickta/article/details/79666918

FixedBatchSpout spout = new FixedBatchSpout(new Fields("user", "score"), 3,   
                new Values("nickt1", 4),  
                new Values("nickt2", 7),   
                new Values("nickt3", 8),  
                new Values("nickt4", 9),   
                new Values("nickt5", 7),  
                new Values("nickt6", 11),  
                new Values("nickt7", 5)  
                );  
        spout.setCycle(false);  
        TridentTopology topology = new TridentTopology();  
        topology.newStream("spout1", spout)  
                .batchGlobal()  
                .each(new Fields("user"),new Debug("print:"))  
                .parallelismHint(5);  

  

分析以上代码:使用FixedBatchSpout发射数据,每个batch设置的size为3,对于如上的nickt1到nickt7共7条数据,则

batch1的内容为:

nickt1 4
nickt2 7

nickt3 8

batch2的内容为:

nickt4 9

nickt5 7

nickt6 11

batch3的内容为:

nickt7 5

因为batchGlobal,把相同batch的tuples分配到同一个partition,可以看如下输出:

输出结果:

<Fri Mar 23 17:31:03 CST 2018[partition1-Thread-116-b-0-executor[34 34]]> DEBUG(print:): [nickt1]
<Fri Mar 23 17:31:03 CST 2018[partition1-Thread-116-b-0-executor[34 34]]> DEBUG(print:): [nickt2]
<Fri Mar 23 17:31:03 CST 2018[partition1-Thread-116-b-0-executor[34 34]]> DEBUG(print:): [nickt3]
<Fri Mar 23 17:31:03 CST 2018[partition2-Thread-55-b-0-executor[35 35]]> DEBUG(print:): [nickt4]
<Fri Mar 23 17:31:03 CST 2018[partition2-Thread-55-b-0-executor[35 35]]> DEBUG(print:): [nickt5]
<Fri Mar 23 17:31:03 CST 2018[partition2-Thread-55-b-0-executor[35 35]]> DEBUG(print:): [nickt6]
<Fri Mar 23 17:31:03 CST 2018[partition3-Thread-108-b-0-executor[36 36]]> DEBUG(print:): [nickt7]

posted @ 2018-03-23 18:35  nickt  阅读(94)  评论(0编辑  收藏  举报