Storm Trident示例broadcast
下代码使用broadcast做repartition, 广播,会把tuples分配到所有的partitions当中, 如果有5个partition,则会把原tuples复制5份,分配到5个partition去
省略部分代码,省略部分可参考:https://blog.csdn.net/nickta/article/details/79666918
FixedBatchSpout spout = new FixedBatchSpout(new Fields("user", "score"), 3, new Values("nickt1", 4), new Values("nickt2", 7), new Values("nickt3", 8), new Values("nickt4", 9), new Values("nickt5", 7), new Values("nickt6", 11), new Values("nickt7", 5) ); spout.setCycle(false); TridentTopology topology = new TridentTopology(); topology.newStream("spout1", spout) .broadcast() .each(new Fields("user"),new Debug("print:")) .parallelismHint(5);
输出:源tuples从nickt1到nickt7,一共7条,因为parallelismHint为5,所以有5个partition,
7条数据,复制5份,最终输出7*5=35条数据,而且从结果可以看出,每个partition输出的内容是相同的,都是nickt1到nickt7.
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt1]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt2]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt3]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt4]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt5]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt6]
<Fri Mar 23 16:05:09 CST 2018[partition4-Thread-142-b-0-executor[37 37]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition2-Thread-99-b-0-executor[35 35]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition0-Thread-106-b-0-executor[33 33]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition3-Thread-69-b-0-executor[36 36]]> DEBUG(print:): [nickt7]
<Fri Mar 23 16:05:09 CST 2018[partition1-Thread-65-b-0-executor[34 34]]> DEBUG(print:): [nickt7]