RocketMQ单机服务搭建
1.服务端
准备依赖环境
-
建议使用Linux / Unix / Mac等64位操作系统;
-
64位JDK 1.8+;
-
Maven 3.2.x;
-
适用于Broker服务器的4G +可用磁盘
下载RocketMQ服务
https://archive.apache.org/dist/rocketmq/
或者git下载
> git clone https://github.com/apache/rocketmq.git
source release需要自己编译
> unzip rocketmq-all-4.7.1-source-release.zip > cd rocketmq-all-4.7.1/ > mvn -Prelease-all -DskipTests clean install -U > cd distribution/target/rocketmq-4.7.1/rocketmq-4.7.1
如果想更改Name Server的日志路径, 默认是在user.home下
源码ClientLogger类中:
修改bin文件夹下的runserver.sh, 指定user.home路径,broker日志同理。
JAVA_OPT="${JAVA_OPT} -Duser.home=/Users/yue/Downloads/rocketmq-all-4.7.1"
在user.home目录下创建文件夹(可忽略):
/Users/yue/Downloads/rocketmq-all-4.7.1/store /Users/yue/Downloads/rocketmq-all-4.7.1/store/commitlog /Users/yue/Downloads/rocketmq-all-4.7.1/store/consumequeue /Users/yue/Downloads/rocketmq-all-4.7.1/store/index
调整内存:修改bin文件夹下的runserver.sh,runbroker.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"
启动Name Server
> nohup sh bin/mqnamesrv &
> tail -f ~/logs/rocketmqlogs/namesrv.log
The Name Server boot success...
启动 Broker
> nohup sh bin/mqbroker -n localhost:9876 autoCreateTopicEnable=true & > tail -f ~/logs/rocketmqlogs/broker.log The broker[%s, 172.30.30.233:10911] boot success...
关闭服务
> sh bin/mqshutdown broker The mqbroker(36695) is running... Send shutdown request to mqbroker(36695) OK > sh bin/mqshutdown namesrv The mqnamesrv(36664) is running... Send shutdown request to mqnamesrv(36664) OK
配置项
如果不知道有哪些配置项,可在 /Users/yue/Downloads/rocketmq-all-4.7.1/logs/rocketmqlogs/文件夹下查看 namesrv.log,broker.log 日志文件。(前提是启动过 Name Server 和 Broker,有生成日志文件)。
例:Broker部分配置
#集群名字 brokerClusterName = DefaultCluster #broker名字,不同的配置文件填写的不一样 brokerName = broker-a #0 表示 Master,>0 表示 Slave brokerId = 0 # brokerIp=127.0.0.1 #nameServer地址,分号分割 namesrvAddr=127.0.0.1:9876 #ASYNC_MASTER 异步复制Master, SYNC_MASTER 同步双写Master brokerRole = ASYNC_MASTER #ASYNC_FLUSH 异步刷盘, SYNC_FLUSH 同步刷盘 flushDiskType = ASYNC_FLUSH #默认创建的队列数TOPIC defaultTopicQueueNums=4 #是否允许 Broker 自动创建Topic,建议线下开启,线上关闭 autoCreateTopicEnable=true #是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭 autoCreateSubscriptionGroup=true #Broker 对外监听端口 listenPort=10911 #回查事务消息 checkTransactionMessageEnable=false #发消息线程池数量 sendMessageTreadPoolNums=128 #拉消息线程池数量 pullMessageTreadPoolNums=128 #commitlog目录所在的分区使用比例大于该值,则触发过期文件删除,默认75 diskMaxUsedSpaceRatio=80 #存储路径 storePathRootDir=/data/rocketmq/store #commitLog 存储路径 storePathCommitLog=/data/rocketmq/store/commitlog #消费队列存储路径存储路径 storePathConsumeQueue=/data/rocketmq/store/consumequeue #消息索引存储路径 storePathIndex=/data/rocketmq/store/index #checkpoint 文件存储路径 storeCheckpoint=/data/rocketmq/store/checkpoint #abort 文件存储路径 abortFile=/data/rocketmq/store/abort #限制的消息大小 maxMessageSize=65536 #一次刷盘至少需要脏页的数量,针对Commitlog文件,默认4 flushCommitLogLeastPages=4 #一次刷盘至少需要脏页的数量,针对Consume文件,默认2 flushConsumeQueueLeastPages=2 #Commitlog两次刷盘的最大间隔,如果超过该间隔,将fushCommitLogLeastPages要求直接执行刷盘操作,默认10000 flushCommitLogThoroughInterval=10000 #Consume两次刷盘的最大间隔,如果超过该间隔,将忽略,默认60000 flushConsumeQueueThoroughInterval=60000 #删除文件时间点,默认凌晨 4点 deleteWhen = 04 #文件保留时间,默认 48 小时 fileReservedTime = 48 #commitLog每个文件的大小默认1G mapedFileSizeCommitLog=1073741824 #ConsumeQueue每个文件默认存30W条,根据业务情况调整 mapedFileSizeConsumeQueue=300000 #销毁MappedFile被拒绝的最大存活时间,默认120s destroyMapedFileIntervalForcibly=120000 #重试删除文件间隔,配合destorymapedfileintervalforcibly,默认120s redeleteHangedFileInterval=120000
2.可视化
下载rocketmq-externals
git clone https://github.com/apache/rocketmq-externals.git
修改rocketmq-externals/rocketmq-console/src/main/resources/application.properties
server.address=0.0.0.0 server.port=8088 rocketmq.config.namesrvAddr=127.0.0.1:9876
修改rocketmq-externals/rocketmq-console/pom.xml,与服务端版本一致
<rocketmq.version>4.7.1</rocketmq.version>
打包, 打包期间报错修改pom.xml中的jar 版本
> cd rocketmq-externals/rocketmq-console > mvn clean package -Dmaven.test.skip=true
启动 java -jar target/rocketmq-console-ng-2.0.0.jar
访问 http://localhost:8088
3.SpringBoot整合
添加依赖
//RocketMQ compile 'org.apache.rocketmq:rocketmq-spring-boot-starter:2.1.1'
配置项
rocketmq: name-server: 127.0.0.1:9876 # 生产者配置 producer: # 发送同一类消息的设置为同一个group,保证唯一 group: rocketGroup
生产者
@Component public class Producer { @Resource private RocketMQTemplate mqTemplate; public void send() { Map<String, Object> resultMap = new HashMap<String, Object>(); resultMap.put("name", "testName"); resultMap.put("id", "1"); resultMap.put("sex", "1"); //发送消息 mqTemplate.convertAndSend("topic-test:TagA", resultMap); //发送spring的Message mqTemplate.send("topic-test:TagA", MessageBuilder.withPayload(resultMap).build()); //发送异步消息 mqTemplate.asyncSend("topic-test:TagA", resultMap, new SendCallback() { @Override public void onSuccess(SendResult sendResult) { System.out.println("发送成功"); } @Override public void onException(Throwable throwable) { System.out.println("发送失败"); } }); //发送顺序消息 mqTemplate.syncSendOrderly("topic-test", "1,创建", "3"); mqTemplate.syncSendOrderly("topic-test", "2,支付", "2"); mqTemplate.syncSendOrderly("topic-test", "3,完成", "1"); } }
业务代码
@Resource Producer producer; @PostMapping("/test") public void test() { producer.send(); }
消费者
@Component @RocketMQMessageListener(topic = "topic-test", consumerGroup = "my-consumer_test-topic") public class Consumer implements RocketMQListener<String> { @Override public void onMessage(String message) { System.out.println("接受到消息:"+message.toString()); } }
常见错误
org.apache.rocketmq.remoting.exception.RemotingTooMuchRequestException: send DefaultImpl call timeout 原因:多网卡多IP情况下,rocketmq会根据当前网卡选择一个IP使用,当你的机器有多块网卡时,很可能会有问题,因此需要配置broker.conf指定当前公网的ip,然后重启broker。 修改配置:/rocketmq-4.7.1/conf/broker.conf 新增这个配置:brokerIP1=xxx.xxx.xxx.xxx 启动命令:nohup sh bin/mqbroker -n localhost:9876 -c ./conf/broker.conf &
还存在另外一种情况:
Mac系统下每次启动只有在发送第一条消息报send DefaultImpl call timeout错误:归根结底是程序寻找HostName引起的超时问题
执行下面命令即可:
scutil --set HostName 'localhost'
RocketMQ Broker 无法启动:删除store 后重新启动Broker
MQClientException: No route info of this topic, TopicTest: 通过启动命令 nohup sh bin/mqbroker -n localhost:9876 autoCreateTopicEnable=true 或 broker.conf中添加配置属性 autoCreateTopicEnable=true;默认是true
无法自动创建topic: 客户端版本要和服务端版本保持一致