Storm Trident示例broadcast

下代码使用broadcast做repartition, 广播,会把tuples分配到所有的partitions当中, 如果有5个partition,则会把原tuples复制5份,分配到5个partition去

省略部分代码,省略部分可参考:https://blog.csdn.net/nickta/article/details/79666918

FixedBatchSpout spout = new FixedBatchSpout(new Fields("user", "score"), 3,   
                new Values("nickt1", 4),  
                new Values("nickt2", 7),   
                new Values("nickt3", 8),  
                new Values("nickt4", 9),   
                new Values("nickt5", 7),  
                new Values("nickt6", 11),  
                new Values("nickt7", 5)  
                );  
        spout.setCycle(false);  
        TridentTopology topology = new TridentTopology();  
        topology.newStream("spout1", spout)  
                .broadcast()  
                .each(new Fields("user"),new Debug("print:"))  
                .parallelismHint(5);  

  

输出:源tuples从nickt1到nickt7,一共7条,因为parallelismHint为5,所以有5个partition,

 

7条数据,复制5份,最终输出7*5=35条数据,而且从结果可以看出,每个partition输出的内容是相同的,都是nickt1到nickt7.

<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt7]

posted @ 2018-03-23 16:11  nickt  阅读(103)  评论(0编辑  收藏  举报