RocketMQ<一>API使用
消息队列的种类有很多,这里以rocketmq为例讲一下rocketmq的简单使用,安装步骤这里不详细讲述,如果有对消息队列不太了解的可以先看一下上一篇文章《消息队列一》。
一、前提
1、需要添加响应的pom依赖,这里不需要整合spring boot,直接使用 rocketmq-client ,【注意】这里pom文件的版本必须要和你安装的版本一致,否者启动会报错。
<dependencies> <dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-client</artifactId> <version>4.3.0</version> </dependency> </dependencies>
2、这里提示一下新手可能会出现的问题
- No route info of this topic:在启动broker时加上 autoCreateTopicEnable=true,如果是用broker.conf启动在配置文件中添加,默认自动创建没有的topic。
- Producer没有正确连接到Name Server:查看是否为防火墙原因 关闭防火墙(生产环境不建议),开放rocketmq的相应端口,或者加入相应的可以访问rocketmq的ip。
- connect to <ip:port> failed:rocketmq默认开启了vip通道 producer.setVipChannelEnabled(false); consumer.setVipChannelEnabled(false);
- Send [3] times, still failed, cost [575]ms, Topic: Name, BrokersSent: [broker-a, broker-a, null] :如果出现开始发送消息成功,消费者接收消息后,再发送消息的话,出现上诉异常,则有很大概率是机器磁盘空间不足。
二、简单的API使用
1、发送同步消息
这个消息适用于对消息丢失忍受力比较低的,对性能要求没那么高的
生产者:
/** * @author D-L * @version 1.0.0 * @ClassName SyncProducer.java * @Description 发送同步消息 * @createTime 2021-06-22 23:51:00 */ public class SyncProducer { public static void main(String[] args) throws Exception { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.启动producer producer.start(); for (int i = 0; i < 5; i++) { //4.创建消息对象,指定主题Topic、Tag和消息体 /** * 参数一:消息主题Topic * 参数二:消息Tag 用来过滤消息使用 * 参数三:消息key 用来过滤消息使用 * 参数四:消息体(内容) */ Message msg = new Message("base", "Tag2", "key2",("同步消息" + i).getBytes()); //5.发送消息 SendResult result = producer.send(msg); //发送状态 SendStatus status = result.getSendStatus(); System.out.println("发送结果:" + result); //线程睡1秒 TimeUnit.SECONDS.sleep(1); } //6.关闭生产者producer producer.shutdown(); } }
消费者:
/** * @author D-L * @version 1.0.0 * @ClassName Consumer.java * @Description 消息的接受者 * @createTime 2021-06-22 23:51:00 */ public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group2"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("base", "*"); //设定消费模式:负载均衡:CLUSTERING || 广播模式:BROADCASTING consumer.setMessageModel(MessageModel.CLUSTERING); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("consumeThread=" + Thread.currentThread().getName() + "," + new String(msg.getBody())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("消费者启动-----------------------------"); } }
2、发送异步消息
生产者:发送异步消息时会有两个回调,成功和失败后可以在对应的回调里对发送的消息进行相应的处理。
public class AsyncProducer { public static void main(String[] args) throws Exception { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.启动producer producer.start(); for (int i = 0; i < 10; i++) { //4.创建消息对象,指定主题Topic、Tag和消息体 Message msg = new Message("async", "Tag2","key2", ("异步消息" + i).getBytes()); //5.发送异步消息 producer.send(msg, new SendCallback() { /** * 发送成功回调函数 * @param sendResult */ public void onSuccess(SendResult sendResult) { System.out.println("发送结果:" + sendResult); } /** * 发送失败回调函数 * @param e */ public void onException(Throwable e) { System.out.println("发送异常:" + e); } }); //线程睡1秒 TimeUnit.SECONDS.sleep(1); } //6.关闭生产者producer producer.shutdown(); } }
消费者:
public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group2"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("async", "*"); //设定消费模式:负载均衡:CLUSTERING || 广播模式:BROADCASTING consumer.setMessageModel(MessageModel.CLUSTERING); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("consumeThread=" + Thread.currentThread().getName() + "," + new String(msg.getBody())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("消费者启动-----------------------------"); } }
3、发送单向消息
发送单向消息,因为是单向消息,所有没有返回值,也不会阻塞线程,单向发送消息这种方式主要用在不特别关心发送结果的场景,例如日志发送。
生产者:
public class OneWayProducer { public static void main(String[] args) throws Exception, MQBrokerException { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); //消息发送失败重试次数 producer.setRetryTimesWhenSendFailed(3); //关闭vip通道,阿里,腾讯上部署时建议关闭,不然可能发送失败 //在producer和consumer连接时连接的是vip端口,导致发送、启动失败;建议在MQProducerAutoConfiguration 和 // MQConsumerAutoConfiguration中对producer和consumer进行配置参数读取并设置VipChannelEnabled的,以便通过配置即可对低版本服务端的支持 producer.setVipChannelEnabled(false); //3.启动producer producer.start(); for (int i = 0; i < 3; i++) { //4.创建消息对象,指定主题Topic、Tag和消息体 Message msg = new Message("OneWay", "Tag2", ("单向消息" + i).getBytes()); //5.发送单向消息 producer.sendOneway(msg); //线程睡1秒 TimeUnit.SECONDS.sleep(5); } //6.关闭生产者producer producer.shutdown(); } }
消费者:
public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group2"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("OneWay", "*"); //设定消费模式:负载均衡:CLUSTERING || 广播模式:BROADCASTING consumer.setMessageModel(MessageModel.CLUSTERING); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("consumeThread=" + Thread.currentThread().getName() + "," + new String(msg.getBody())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("消费者启动-----------------------------"); } }
4、批量发送消息
生产者:
public class Producer { public static void main(String[] args) throws Exception { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); producer.setVipChannelEnabled(false); producer.setRetryTimesWhenSendFailed(3); //3.启动producer producer.start(); List<Message> msgs = new ArrayList<Message>(); //4.创建消息对象,指定主题Topic、Tag和消息体 /** * 参数一:消息主题Topic * 参数二:消息Tag * 参数三:消息内容 */ Message msg1 = new Message("BatchTopic", "Tag1", ("Hello World" + 1).getBytes()); Message msg2 = new Message("BatchTopic", "Tag1", ("Hello World" + 2).getBytes()); Message msg3 = new Message("BatchTopic", "Tag1", ("Hello World" + 3).getBytes()); msgs.add(msg1); msgs.add(msg2); msgs.add(msg3); //5.发送消息 SendResult result = producer.send(msgs); //发送状态 SendStatus status = result.getSendStatus(); System.out.println("发送结果:" + result); //线程睡1秒 TimeUnit.SECONDS.sleep(1); //6.关闭生产者producer producer.shutdown(); } }
消费者:
public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("BatchTopic", "*"); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("consumeThread=" + Thread.currentThread().getName() + "," + new String(msg.getBody())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("消费者启动"); } }
5、发送延迟消息
等待一定时间后投递消息,默认延迟等级 : 1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h, 传入1代表1s, 2代表5s, 以此类推
生产者:
public class Producer { public static void main(String[] args) throws InterruptedException, RemotingException, MQClientException, MQBrokerException { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); producer.setRetryTimesWhenSendFailed(3); producer.setVipChannelEnabled(false); //3.启动producer producer.start(); for (int i = 0; i < 10; i++) { //4.创建消息对象,指定主题Topic、Tag和消息体 /** * 参数一:消息主题Topic * 参数二:消息Tag * 参数三:消息内容 * 默认延迟等级 : 1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h, 传入1代表1s, 2代表5s, 以此类推 */ Message msg = new Message("DelayTopic", "Tag1", ("延迟消息发送" + i).getBytes()); //设定延迟时间 msg.setDelayTimeLevel(3); //5.发送消息 SendResult result = producer.send(msg); //发送状态 SendStatus status = result.getSendStatus(); System.out.println("发送结果:" + result); //线程睡1秒 TimeUnit.SECONDS.sleep(1); } //6.关闭生产者producer producer.shutdown(); } }
消费者:
public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); consumer.setVipChannelEnabled(false); //3.订阅主题Topic和Tag consumer.subscribe("DelayTopic", "*"); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("消息ID:【" + msg.getMsgId() + "】,延迟时间:" + (System.currentTimeMillis() - msg.getStoreTimestamp())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("消费者启动"); } }
6、消息过滤
消息过滤通过两种:tag过滤、sql过滤
<1.>tag过滤
生产者:
public class Producer { public static void main(String[] args) throws Exception { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.启动producer producer.start(); for (int i = 0; i < 3; i++) { //4.创建消息对象,指定主题Topic、Tag和消息体 /** * 参数一:消息主题Topic * 参数二:消息Tag * 参数三:消息内容 */ Message msg = new Message("FilterTagTopic", "Tag1", ("Hello World" + i).getBytes()); //5.发送消息 SendResult result = producer.send(msg); //发送状态 SendStatus status = result.getSendStatus(); System.out.println("发送结果:" + result); //线程睡1秒 TimeUnit.SECONDS.sleep(1); } //6.关闭生产者producer producer.shutdown(); } }
消费者:
public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("FilterTagTopic", "Tag1 || Tag2 ||Tag3"); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("consumeThread=" + Thread.currentThread().getName() + "," + new String(msg.getBody())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("消费者启动"); } }
<2.>sql过滤 【注意】这里要想通过sql过滤必须要在 broker.conf中添加 : enablePropertyFilter=true
生产者:
public class Producer { public static void main(String[] args) throws Exception { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.启动producer producer.start(); for (int i = 0; i < 10; i++) { //4.创建消息对象,指定主题Topic、Tag和消息体 /** * 参数一:消息主题Topic * 参数二:消息Tag * 参数三:消息内容 */ Message msg = new Message("FilterSQLTopic", "Tag1", ("消息过滤sql" + i).getBytes()); msg.putUserProperty("i", String.valueOf(i)); //5.发送消息 SendResult result = producer.send(msg); //发送状态 SendStatus status = result.getSendStatus(); System.out.println("发送结果:" + result); //线程睡1秒 TimeUnit.SECONDS.sleep(2); } //6.关闭生产者producer producer.shutdown(); } }
消费者:
public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("FilterSQLTopic", MessageSelector.bySql("i>5")); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("consumeThread=" + Thread.currentThread().getName() + "," + new String(msg.getBody())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("消费者启动"); } }
7、发送顺序消息
顺序消息可以通过 new MessageQueueSelector() 消息选择器放入同一个队列 。【注意】这里的顺序只能保证局部的顺序,而不是全局的顺序性。
构建订单集合:
/** * 订单构建者 */ public class OrderStep { private long orderId; private String desc;
//省略get set 方法
public static List<OrderStep> buildOrders() { // 1039L : 创建 付款 推送 完成 // 1065L : 创建 付款 // 7235L :创建 付款 List<OrderStep> orderList = new ArrayList<OrderStep>(); OrderStep orderDemo = new OrderStep(); orderDemo.setOrderId(1L); orderDemo.setDesc("创建"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(2L); orderDemo.setDesc("创建"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(1L); orderDemo.setDesc("付款"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(3L); orderDemo.setDesc("创建"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(2L); orderDemo.setDesc("付款"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(3L); orderDemo.setDesc("付款"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(2L); orderDemo.setDesc("完成"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(1L); orderDemo.setDesc("推送"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(3L); orderDemo.setDesc("完成"); orderList.add(orderDemo); orderDemo = new OrderStep(); orderDemo.setOrderId(1L); orderDemo.setDesc("完成"); orderList.add(orderDemo); return orderList; } }
生产者:
public class Producer { public static void main(String[] args) throws Exception { //1.创建消息生产者producer,并制定生产者组名 DefaultMQProducer producer = new DefaultMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.启动producer producer.start(); //构建消息集合 List<OrderStep> orderSteps = OrderStep.buildOrders(); //发送消息 for (int i = 0; i < orderSteps.size(); i++) { String body = orderSteps.get(i) + ""; Message message = new Message("OrderTopic", "Order", "i" + i, body.getBytes()); /** * 参数一:消息对象 * 参数二:消息队列的选择器 * 参数三:选择队列的业务标识(订单ID) */ SendResult sendResult = producer.send(message, new MessageQueueSelector() { /** * * @param mqs:队列集合 * @param msg:消息对象 * @param arg:业务标识的参数 * @return */ @Override public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) { long orderId = (long) arg; long index = orderId % mqs.size(); return mqs.get((int) index); } }, orderSteps.get(i).getOrderId()); System.out.println("发送结果:" + sendResult); } producer.shutdown(); } }
消费者:
public class Consumer { public static void main(String[] args) throws MQClientException { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("OrderTopic", "*"); //4.注册消息监听器 consumer.registerMessageListener(new MessageListenerOrderly() { @Override public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs, ConsumeOrderlyContext context) { for (MessageExt msg : msgs) { System.out.println("QueueId : "+ msg.getQueueId() + "------"+ new String(msg.getBody())); } return ConsumeOrderlyStatus.SUCCESS; } }); //5.启动消费者 consumer.start(); System.out.println("消费者启动"); } }
8、事务消息
先发送一个
,然后根据本地事务逻辑确定半消息的 提交(COMMIT_MESSAGE)、回滚(ROLLBACK_MESSAGE),不确定(UNKNOW)等状态。如果消息为UNKNOW会进行定期回查,如果多次回查还是不能确定,会进行消息的清除。通过事务监听器中的回调方法,处理对应的本地事务,消息回查,保证数据的一致性。生产者:
/** * 发送同步消息 */ public class Producer { public static void main(String[] args) throws Exception { //1.创建消息生产者producer,并制定生产者组名 TransactionMQProducer producer = new TransactionMQProducer("group1"); //2.指定Nameserver地址 producer.setNamesrvAddr("xx.xx.xx.xx:9876"); //添加事务监听器 producer.setTransactionListener(new TransactionListener() { /** * 在该方法中执行本地事务 * @param msg * @param arg * @return */ @Override public LocalTransactionState executeLocalTransaction(Message msg, Object arg) { if (StringUtils.equals("TAGA", msg.getTags())) { return LocalTransactionState.COMMIT_MESSAGE; } else if (StringUtils.equals("TAGB", msg.getTags())) { return LocalTransactionState.ROLLBACK_MESSAGE; } else if (StringUtils.equals("TAGC", msg.getTags())) { return LocalTransactionState.UNKNOW; } return LocalTransactionState.UNKNOW; } /** * 该方法时MQ进行消息事务状态回查 * @param msg * @return */ @Override public LocalTransactionState checkLocalTransaction(MessageExt msg) { System.out.println("没有获得消息ack ----- 进行消息回查 消息的Tag:" + msg.getTags()); return LocalTransactionState.COMMIT_MESSAGE; } }); //3.启动producer producer.start(); String[] tags = {"TAGA", "TAGB", "TAGC"}; for (int i = 0; i < 3; i++) { //4.创建消息对象,指定主题Topic、Tag和消息体 /** * 参数一:消息主题Topic * 参数二:消息Tag * 参数三:消息内容 */ Message msg = new Message("TransactionTopic", tags[i], ("Hello World" + i).getBytes()); //5.发送消息 SendResult result = producer.sendMessageInTransaction(msg, null); //发送状态 SendStatus status = result.getSendStatus(); System.out.println("发送结果:" + result); //线程睡1秒 TimeUnit.SECONDS.sleep(2); } //6.关闭生产者producer //producer.shutdown(); } }
消费者:
/** * 消息的接受者 */ public class Consumer { public static void main(String[] args) throws Exception { //1.创建消费者Consumer,制定消费者组名 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1"); //2.指定Nameserver地址 consumer.setNamesrvAddr("xx.xx.xx.xx:9876"); //3.订阅主题Topic和Tag consumer.subscribe("TransactionTopic", "*"); //4.设置回调函数,处理消息 consumer.registerMessageListener(new MessageListenerConcurrently() { //接受消息内容 @Override public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) { for (MessageExt msg : msgs) { System.out.println("consumeThread=" + Thread.currentThread().getName() + "," + new String(msg.getBody())); } return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; } }); //5.启动消费者consumer consumer.start(); System.out.println("生产者启动"); } }