php rabbitmq的开发体验(二)
一、前言
在上一篇rabbitmq开发体验,我们大致介绍和安装上了rabbitmq和php扩展和界面操作rabbitmq的方法,下面正是正式的用我们php来操作消息队列的生产和消费。附上参考的网站:
- rabbitmq官网 https://www.rabbitmq.com/ 全英文看起来吃力 官方入门文档:https://www.rabbitmq.com/getstarted.html
- 官方入门文档 https://www.cnblogs.com/grimm/p/5728736.html 此博客主对官方php案例的解释
- RabbitMQ发布订阅实战-实现延时重试队列 代码项目:https://github.com/mylxsw/rabbitmq-pubsub-php
- rabbitmq高级特性 B站视频 https://www.bilibili.com/video/BV1S5411H7ef?from=search&seid=8055368004001009131
- RabbitMQ的ack或nack机制使用不当导致的队列堵塞或死循环问题 https://blog.csdn.net/lisheng19870305/article/details/112849495
- RabbitMQ 延时消息队列
- Rabbitmq之高级特性——百分百投递消息&消息确认模式&消息返回模式实现
- RabbitMQ:消费者优先级的介绍和使用 https://blog.csdn.net/weixin_45492007/article/details/106189961
- RabbitMQ消息模式(消息100%的投递、幂等性概念) https://blog.csdn.net/weixin_42687829/article/details/104327711
- 简介Rabbitmq的几种消费模式 https://www.jianshu.com/p/7ac733b7481b
二、开发经历
对于rabbitmq的php类库,我开发是使用PHP amqplib,composer解决依赖管理。
添加composer.json:
{ "require": { "php-amqplib/php-amqplib": ">=2.6.1" } } composer install# 或者 直接运行包引入
composer require php-amqplib/php-amqplib
我的开发框架是yii1.1,核心代码如下,有错误请指正。
1.rabbitmq的连接底层类
<?php
include_once(ROOT_PATH . 'protected/extensions/rabbitmq/autoload.php');
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
/
-
rabbitmq工具类
*/
class RabbitMq
{
protected $connection;
protected $channel;
protected $exchange_name;
protected $query_name;
protected $route_key_name;
/
-
构造器
*/
public function __construct()
{
//读取文件会导致并发性高时连接失败故写在配置文件
$config = $GLOBALS['rabbitmq_config'];
if (!$config)
throw new \AMQPConnectionException('config error!');
$this->connection = new AMQPStreamConnection($config['host'], $config['port'], $config['username'], $config['password'], $config['vhost']);
if (!$this->connection) {
throw \AMQPConnectionException("Cannot connect to the broker!\n");
}
$this->channel = $this->connection->channel();
}
/
- 日志写入
- @param $file log文件路径
- @param $dataStr 报错字符串
*/
protected function writeLog($file,$dataStr)
{
file_put_contents(ROOT_PATH.$file, date('Y-m-d H:i:s').' '.$dataStr .PHP_EOL, FILE_APPEND);
}
/
- close link
*/
public function close()
{
$this->channel->close();
$this->connection->close();
}
/
- RabbitMQ destructor
*/
public function __destruct()
{
$this->close();
}
}
2.消息的封装类
<?php
include_once(ROOT_PATH . 'protected/extensions/rabbitmq/autoload.php');
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
use PhpAmqpLib\Wire\AMQPTable;
class SubMessage
{
public $message;
private $routingKey;
private $params;
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* SubMessage constructor.
*
* @param AMQPMessage $message
* @param string $routingKey
* @param array $params
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> __construct(AMQPMessage <span style="color: #800080;">$message</span>, <span style="color: #800080;">$routingKey</span>, <span style="color: #800080;">$params</span> =<span style="color: #000000;"> [])
{
</span><span style="color: #800080;">$this</span>->params = <span style="color: #800080;">$params</span>; <span style="color: #008000;">//</span><span style="color: #008000;">额外的参数这里主要存储重试的次数</span>
<span style="color: #800080;">$this</span>->message = <span style="color: #800080;">$message</span><span style="color: #000000;">;
</span><span style="color: #800080;">$this</span>->routingKey = <span style="color: #800080;">$routingKey</span><span style="color: #000000;">;
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* Get AMQP Message
*
* @return AMQPMessage
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> getAMQPMessage()
{
</span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$this</span>-><span style="color: #000000;">message;
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* Get original Message
*
* @return Message
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> getMessage()
{
</span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$this</span>->message-><span style="color: #000000;">body;
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* Get meta params
*
* @return array
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> getParams()
{
</span><span style="color: #0000ff;">return</span> <span style="color: #008080;">is_array</span>(<span style="color: #800080;">$this</span>->params) ? <span style="color: #800080;">$this</span>->params :<span style="color: #000000;"> [];
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* Get meta param
*
* @param string $key
*
* @return mixed|null
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> getParam(<span style="color: #0000ff;">string</span> <span style="color: #800080;">$key</span><span style="color: #000000;">)
{
</span><span style="color: #0000ff;">return</span> <span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$this</span>->params[<span style="color: #800080;">$key</span>]) ? <span style="color: #800080;">$this</span>->params[<span style="color: #800080;">$key</span>] : <span style="color: #0000ff;">null</span><span style="color: #000000;">;
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* Get routing key
*
* @return string
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> getRoutingKey()
{
</span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$this</span>-><span style="color: #000000;">routingKey;
}
}
3.消息的核心类
<?php use PhpAmqpLib\Connection\AMQPStreamConnection; use PhpAmqpLib\Message\AMQPMessage; use PhpAmqpLib\Wire\AMQPTable; /** * vm独立站推送 * 简介Rabbitmq的几种消费模式 https://www.jianshu.com/p/7ac733b7481b */ class VmMq extends RabbitMq { protected $exchange_name = 'master'; //主Exchange,发布消息时发布到该Exchange protected $exchange_retry_name = 'master.retry'; //重试Exchange,消息处理失败时(3次以内),将消息重新投递给该Exchange protected $exchange_failed_name = 'master.failed'; //失败Exchange,超过三次重试失败后,消息投递到该Exchange protected $query_name = 'query_vm'; //消费服务需要declare三个队列[queue_name] 队列名称,格式符合 [服务名称]@订阅服务标识 protected $query_retry_name = 'query_vm@retry'; protected $query_failed_name = 'query_vm@fail'; protected $route_key_name = 'route_key_vm'; //路由键名 /** * 构造器 */ public function __construct() { parent::__construct();</span><span style="color: #008000;">//</span><span style="color: #008000;">第2个参数:rabbitmq将给一个消费者一次只发布十条消息。或者说,只有收到消费者上十个消息的完成应答,才给它发布新的消息。如果消费者很忙,消费过慢,队列可能会被填满,这时你需要增加消费者,或者使用其他策略。</span> <span style="color: #800080;">$this</span>->channel->basic_qos(<span style="color: #0000ff;">null</span>, 100, <span style="color: #0000ff;">false</span><span style="color: #000000;">); </span><span style="color: #008000;">/*</span><span style="color: #008000;"> * 第1个参数:交换机名 * 第2个参数:声明topic类型交换器 * 第3个参数:passive消极的,如果该交换机已存在,则不会创建;如果不存在,创建新的交换机。 * 第4个参数:durable持久的,指定交换机持久 * 第5个参数:auto_delete,通道关闭后是否删除交换机,自删除的前提是以前有队列连接这个交换器,后来所有与这个交换器绑定的队列或者交换器都与此解绑, * Topic交换器非常强大,可以像其他类型的交换器一样工作:
* 当一个队列的绑定键是"#"是,它将会接收所有的消息,而不再考虑所接收消息的路由键,就像是fanout发布与订阅交换器一样;
* 当一个队列的绑定键没有用到”#“和”“时,它又像direct交换一样工作。
* routing-key是模糊匹配,可以只替换一个单词,#可以代替零个或多个单词。
* https://www.cnblogs.com/wuhenzhidu/p/10802749.html
*/
$this->channel->exchange_declare($this->exchange_name, 'topic', false, true, false);
$this->channel->exchange_declare($this->exchange_retry_name, 'topic', false, true, false);
$this->channel->exchange_declare($this->exchange_failed_name, 'topic', false, true, false);
}</span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 生产消息 </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> product(<span style="color: #800080;">$data</span>,<span style="color: #800080;">$priority</span> = 10<span style="color: #000000;">){ </span><span style="color: #800080;">$unique_messageId</span> = <span style="color: #800080;">$this</span>->create_guid(); <span style="color: #008000;">//</span><span style="color: #008000;">生成消息的唯一标识,用来幂等性</span> <span style="color: #0000ff;">if</span>(!<span style="color: #008080;">is_array</span>(<span style="color: #800080;">$data</span><span style="color: #000000;">)){ </span><span style="color: #800080;">$data</span> = <span style="color: #0000ff;">array</span>('msg' => <span style="color: #800080;">$data</span><span style="color: #000000;">); } </span><span style="color: #800080;">$uid</span>=<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$data</span>['uid'])?<span style="color: #800080;">$data</span>['uid']:0<span style="color: #000000;">; </span><span style="color: #800080;">$langid</span> = <span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$data</span>['langid'])?<span style="color: #800080;">$data</span>['langid']:1<span style="color: #000000;">; </span><span style="color: #800080;">$data</span>['unique_messageId'] = <span style="color: #800080;">$unique_messageId</span><span style="color: #000000;">; </span><span style="color: #800080;">$data</span> = json_encode(<span style="color: #800080;">$data</span>,<span style="color: #000000;">JSON_UNESCAPED_UNICODE); </span><span style="color: #008000;">//</span><span style="color: #008000;">存入到表中,保证生产的消息100%到mq队列</span> <span style="color: #800080;">$newModel</span> = DynamicAR::model('nt_vm_message_idempotent'<span style="color: #000000;">); </span><span style="color: #800080;">$newModel</span>->message_id = <span style="color: #800080;">$unique_messageId</span><span style="color: #000000;">; </span><span style="color: #800080;">$newModel</span>->uid = <span style="color: #800080;">$uid</span><span style="color: #000000;">; </span><span style="color: #800080;">$newModel</span>->message_content = <span style="color: #800080;">$data</span><span style="color: #000000;">; </span><span style="color: #800080;">$newModel</span>->product_status = 0<span style="color: #000000;">; </span><span style="color: #800080;">$newModel</span>->consume_status = 0<span style="color: #000000;">; </span><span style="color: #800080;">$newModel</span>->create_time = <span style="color: #800080;">$newModel</span>->update_time = <span style="color: #008080;">time</span><span style="color: #000000;">(); </span><span style="color: #800080;">$newModel</span>->langid = <span style="color: #800080;">$langid</span><span style="color: #000000;">; </span><span style="color: #800080;">$newModel</span>->priority = <span style="color: #800080;">$priority</span><span style="color: #000000;">; </span><span style="color: #800080;">$newModel</span>->isNewRecord = <span style="color: #0000ff;">true</span><span style="color: #000000;">; </span><span style="color: #0000ff;">if</span>(!<span style="color: #800080;">$newModel</span>-><span style="color: #000000;">save()){ </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_product_failed.log','数据库保存失败' . json_encode(<span style="color: #800080;">$newModel</span>->getErrors()).<span style="color: #800080;">$data</span><span style="color: #000000;">); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } </span><span style="color: #008000;">//</span><span style="color: #008000;">推送成功的ack回调</span> <span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">set_ack_handler( </span><span style="color: #0000ff;">function</span>(AMQPMessage <span style="color: #800080;">$msg</span><span style="color: #000000;">){ </span><span style="color: #800080;">$msgBody</span> = json_decode(<span style="color: #800080;">$msg</span>->getBody(),<span style="color: #0000ff;">true</span><span style="color: #000000;">); </span><span style="color: #0000ff;">if</span>(!<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$msgBody</span>['unique_messageId']) || !<span style="color: #800080;">$msgBody</span>['unique_messageId'<span style="color: #000000;">]){ </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_product_failed.log','获取消费ID为空!' . <span style="color: #800080;">$msg</span>-><span style="color: #000000;">getBody()); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } </span><span style="color: #800080;">$unique_messageId</span> = <span style="color: #800080;">$msgBody</span>['unique_messageId'<span style="color: #000000;">]; </span><span style="color: #800080;">$criteria</span> = <span style="color: #0000ff;">new</span><span style="color: #000000;"> CDbCriteria; </span><span style="color: #800080;">$criteria</span>->addCondition("message_id = '".<span style="color: #800080;">$unique_messageId</span>."'"<span style="color: #000000;">); </span><span style="color: #800080;">$messageIdempotent</span> = DynamicAR::model('nt_vm_message_idempotent')->find(<span style="color: #800080;">$criteria</span><span style="color: #000000;">); </span><span style="color: #0000ff;">if</span> (!<span style="color: #800080;">$messageIdempotent</span><span style="color: #000000;">) { </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_product_failed.log','该消息数据库里不存在' . <span style="color: #800080;">$msg</span>-><span style="color: #000000;">getBody()); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; }</span><span style="color: #0000ff;">else</span><span style="color: #000000;">{ </span><span style="color: #800080;">$connection</span> = Yii::app()-><span style="color: #000000;">db; </span><span style="color: #800080;">$command</span> = <span style="color: #800080;">$connection</span>->createCommand("<span style="color: #000000;"> UPDATE nt_vm_message_idempotent SET product_status=1 WHERE message_id = '</span><span style="color: #800080;">$unique_messageId</span><span style="color: #000000;">' </span>"<span style="color: #000000;">); </span><span style="color: #800080;">$re</span> = <span style="color: #800080;">$command</span>-><span style="color: #000000;">execute(); </span><span style="color: #0000ff;">if</span>(<span style="color: #800080;">$re</span><span style="color: #000000;">) { </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_product_log.log',<span style="color: #800080;">$messageIdempotent</span>->message_id . <span style="color: #800080;">$msg</span>-><span style="color: #000000;">getBody()); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; }</span><span style="color: #0000ff;">else</span><span style="color: #000000;">{ </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_product_failed.log','数据库保存失败' . <span style="color: #800080;">$msg</span>-><span style="color: #000000;">getBody()); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } } } ); </span><span style="color: #008000;">//</span><span style="color: #008000;">推送失败的nack回调</span> <span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">set_nack_handler( </span><span style="color: #0000ff;">function</span>(AMQPMessage <span style="color: #800080;">$message</span><span style="color: #000000;">){ </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_product_failed.log',"消息生产到mq nack ".<span style="color: #800080;">$message</span>-><span style="color: #000000;">body); } ); </span><span style="color: #008000;">//</span><span style="color: #008000;">监听交换机或者路由键是否存在</span> <span style="color: #800080;">$returnListener</span> = <span style="color: #0000ff;">function</span><span style="color: #000000;"> ( </span><span style="color: #800080;">$replyCode</span>, <span style="color: #800080;">$replyText</span>, <span style="color: #800080;">$exchange</span>, <span style="color: #800080;">$routingKey</span>, <span style="color: #800080;">$message</span><span style="color: #000000;"> ) { </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm.log','replyCode ='.<span style="color: #800080;">$replyCode</span>.';replyText='.<span style="color: #800080;">$replyText</span>.';exchange='.<span style="color: #800080;">$exchange</span>.';routingKey='.<span style="color: #800080;">$routingKey</span>.';body='.<span style="color: #800080;">$message</span>-><span style="color: #000000;">body); }; </span><span style="color: #008000;">//</span><span style="color: #008000;">开启发送消息的return机制</span> <span style="color: #800080;">$this</span>->channel->set_return_listener(<span style="color: #800080;">$returnListener</span><span style="color: #000000;">); </span><span style="color: #008000;">//</span><span style="color: #008000;">开启发送消息的ack回调</span> <span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">confirm_select(); </span><span style="color: #800080;">$msg</span> = <span style="color: #0000ff;">new</span> AMQPMessage(<span style="color: #800080;">$data</span>,<span style="color: #0000ff;">array</span><span style="color: #000000;">( </span>'delivery_mode' => AMQPMessage::DELIVERY_MODE_PERSISTENT, <span style="color: #008000;">//</span><span style="color: #008000;">设置消息持久化</span> 'priority' => <span style="color: #800080;">$priority</span>, <span style="color: #008000;">//</span><span style="color: #008000;">消息的优先级 优先级越大越优先</span>
));
$msg->set('application_headers', new AMQPTable([]));
//推送消息到某个交换机,第三个是路由键为方便即是队列名
$this->channel->basic_publish($msg, $this->exchange_name,$this->query_name,true);
//等待发送消息的ack回调消息
$this->channel->wait_for_pending_acks();
$this->close();
}</span><span style="color: #0000ff;">protected</span> <span style="color: #0000ff;">function</span> create_guid(<span style="color: #800080;">$namespace</span> = ''<span style="color: #000000;">) { </span><span style="color: #0000ff;">static</span> <span style="color: #800080;">$guid</span> = ''<span style="color: #000000;">; </span><span style="color: #800080;">$uid</span> = <span style="color: #008080;">uniqid</span>("", <span style="color: #0000ff;">true</span><span style="color: #000000;">); </span><span style="color: #800080;">$data</span> = <span style="color: #800080;">$namespace</span><span style="color: #000000;">; </span><span style="color: #800080;">$data</span> .= <span style="color: #800080;">$_SERVER</span>['REQUEST_TIME'<span style="color: #000000;">]; </span><span style="color: #800080;">$data</span> .= <span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$_SERVER</span>['HTTP_USER_AGENT'])?<span style="color: #800080;">$_SERVER</span>['HTTP_USER_AGENT']:''<span style="color: #000000;">; </span><span style="color: #800080;">$data</span> .= <span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$_SERVER</span>['SERVER_ADDR'])?<span style="color: #800080;">$_SERVER</span>['SERVER_ADDR']:''<span style="color: #000000;">; </span><span style="color: #800080;">$data</span> .= <span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$_SERVER</span>['SERVER_PORT'])?<span style="color: #800080;">$_SERVER</span>['SERVER_PORT']:''<span style="color: #000000;">; </span><span style="color: #800080;">$data</span> .= <span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$_SERVER</span>['REMOTE_ADDR'])?<span style="color: #800080;">$_SERVER</span>['REMOTE_ADDR']:''<span style="color: #000000;">; </span><span style="color: #800080;">$data</span> .= <span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$_SERVER</span>['REMOTE_PORT'])?<span style="color: #800080;">$_SERVER</span>['REMOTE_PORT']:''<span style="color: #000000;">; </span><span style="color: #800080;">$hash</span> = <span style="color: #008080;">strtoupper</span>(hash('ripemd128', <span style="color: #800080;">$uid</span> . <span style="color: #800080;">$guid</span> . <span style="color: #008080;">md5</span>(<span style="color: #800080;">$data</span><span style="color: #000000;">))); </span><span style="color: #800080;">$guid</span> = '{' . <span style="color: #008080;">substr</span>(<span style="color: #800080;">$hash</span>, 0, 8) . '-' . <span style="color: #008080;">substr</span>(<span style="color: #800080;">$hash</span>, 8, 4) . '-' . <span style="color: #008080;">substr</span>(<span style="color: #800080;">$hash</span>, 12, 4) . '-' . <span style="color: #008080;">substr</span>(<span style="color: #800080;">$hash</span>, 16, 4) . '-' . <span style="color: #008080;">substr</span>(<span style="color: #800080;">$hash</span>, 20, 12) . '}'<span style="color: #000000;">; </span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$guid</span><span style="color: #000000;">; } </span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 消费消息 </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> consume(\Closure <span style="color: #800080;">$callback</span>,\Closure <span style="color: #800080;">$shouldExitCallback</span> = <span style="color: #0000ff;">null</span>,<span style="color: #800080;">$priority</span> = 5<span style="color: #000000;">){ </span><span style="color: #800080;">$this</span>-><span style="color: #000000;">declareRetryQueue(); </span><span style="color: #800080;">$this</span>->declareConsumeQueue(<span style="color: #800080;">$priority</span><span style="color: #000000;">); </span><span style="color: #800080;">$this</span>-><span style="color: #000000;">declareFailedQueue(); </span><span style="color: #008000;">//</span><span style="color: #008000;">执行上面的步骤主要是为保证这些目标交换机和队列已经存在</span> <span style="color: #800080;">$queueName</span> = <span style="color: #800080;">$this</span>-><span style="color: #000000;">query_name; </span><span style="color: #800080;">$exchangeRetryName</span> = <span style="color: #800080;">$this</span>-><span style="color: #000000;">exchange_retry_name; </span><span style="color: #800080;">$exchangeFailedName</span> = <span style="color: #800080;">$this</span>-><span style="color: #000000;">exchange_failed_name; </span><span style="color: #008000;">//</span><span style="color: #008000;"> 发起延时重试的回调</span> <span style="color: #800080;">$publishRetry</span> = <span style="color: #0000ff;">function</span> (<span style="color: #800080;">$msg</span>) <span style="color: #0000ff;">use</span> (<span style="color: #800080;">$queueName</span>,<span style="color: #800080;">$exchangeRetryName</span><span style="color: #000000;">) { </span><span style="color: #008000;">/*</span><span style="color: #008000;">* @var AMQPTable $headers </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">if</span> (<span style="color: #800080;">$msg</span>->has('application_headers'<span style="color: #000000;">)) { </span><span style="color: #800080;">$headers</span> = <span style="color: #800080;">$msg</span>->get('application_headers'<span style="color: #000000;">); } </span><span style="color: #0000ff;">else</span><span style="color: #000000;"> { </span><span style="color: #800080;">$headers</span> = <span style="color: #0000ff;">new</span><span style="color: #000000;"> AMQPTable(); } </span><span style="color: #800080;">$headers</span>->set('x-orig-routing-key', <span style="color: #800080;">$this</span>->getOrigRoutingKey(<span style="color: #800080;">$msg</span><span style="color: #000000;">)); </span><span style="color: #800080;">$properties</span> = <span style="color: #800080;">$msg</span>-><span style="color: #000000;">get_properties(); </span><span style="color: #800080;">$properties</span>['application_headers'] = <span style="color: #800080;">$headers</span><span style="color: #000000;">; </span><span style="color: #800080;">$newMsg</span> = <span style="color: #0000ff;">new</span> AMQPMessage(<span style="color: #800080;">$msg</span>->getBody(), <span style="color: #800080;">$properties</span><span style="color: #000000;">); </span><span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">basic_publish( </span><span style="color: #800080;">$newMsg</span>, <span style="color: #800080;">$exchangeRetryName</span>, <span style="color: #800080;">$queueName</span><span style="color: #000000;"> ); </span><span style="color: #008000;">//</span><span style="color: #008000;"> 发送ack信息应答当前消息处理完成</span> <span style="color: #800080;">$msg</span>->delivery_info['channel']->basic_ack(<span style="color: #800080;">$msg</span>->delivery_info['delivery_tag'<span style="color: #000000;">]); }; </span><span style="color: #008000;">//</span><span style="color: #008000;"> 将消息发送到失败队列的回调</span> <span style="color: #800080;">$publishFailed</span> = <span style="color: #0000ff;">function</span> (<span style="color: #800080;">$msg</span>) <span style="color: #0000ff;">use</span> (<span style="color: #800080;">$queueName</span>,<span style="color: #800080;">$exchangeFailedName</span><span style="color: #000000;">) { </span><span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">basic_publish( </span><span style="color: #800080;">$msg</span>, <span style="color: #800080;">$exchangeFailedName</span>, <span style="color: #800080;">$queueName</span><span style="color: #000000;"> ); </span><span style="color: #800080;">$msg</span>->delivery_info['channel']->basic_ack(<span style="color: #800080;">$msg</span>->delivery_info['delivery_tag'<span style="color: #000000;">]); }; </span><span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">basic_consume( </span><span style="color: #800080;">$this</span>->query_name, '', <span style="color: #008000;">//</span><span style="color: #008000;">customer_tag 消费者标签,用来区分多个消费者</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">no_local 若设置为true,表示不能将同一个Conenction中生产者发送的消息传递给这个Connection中的消费者</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">no_ack 是否自动确认消息,true自动确认,false不自动要消费脚本手动调用ack,避免消费异常系统反而自动ack完成</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">exclusive 排他消费者,即这个队列只能由一个消费者消费.适用于任务不允许进行并发处理的情况下.比如系统对接</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">nowait // 消费主回调函数</span> <span style="color: #0000ff;">function</span>(AMQPMessage <span style="color: #800080;">$msg</span>) <span style="color: #0000ff;">use</span> (<span style="color: #800080;">$callback</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">) { </span><span style="color: #800080;">$retry</span> = <span style="color: #800080;">$this</span>->getRetryCount(<span style="color: #800080;">$msg</span><span style="color: #000000;">); </span><span style="color: #0000ff;">try</span><span style="color: #000000;">{ </span><span style="color: #008000;">/*</span><span style="color: #008000;"> * 需要注意的是:在消费消息之前,先获取消息ID,然后根据ID去数据库中查询是否存在主键为消息ID的记录,如果存在的话, * 说明这条消息之前应该是已经被消费过了,那么就不处理这条消息;如果不存在消费记录的话,则消费者进行消费,消费完成发送确认消息, * 并且将消息记录进行入库。 </span><span style="color: #008000;">*/</span> <span style="color: #800080;">$msgBody</span> = json_decode(<span style="color: #800080;">$msg</span>->getBody(),<span style="color: #0000ff;">true</span><span style="color: #000000;">); </span><span style="color: #0000ff;">if</span>(!<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$msgBody</span>['unique_messageId']) || !<span style="color: #800080;">$msgBody</span>['unique_messageId'<span style="color: #000000;">]){ </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_consume_failed.log','获取消费ID为空!' . <span style="color: #800080;">$msg</span>-><span style="color: #000000;">getBody()); </span><span style="color: #008000;">//</span><span style="color: #008000;">发送ack信息应答当前消息处理完成</span> <span style="color: #800080;">$msg</span>->delivery_info['channel']->basic_ack(<span style="color: #800080;">$msg</span>->delivery_info['delivery_tag'<span style="color: #000000;">]); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } </span><span style="color: #800080;">$unique_messageId</span> = <span style="color: #800080;">$msgBody</span>['unique_messageId'<span style="color: #000000;">]; </span><span style="color: #800080;">$criteria</span> = <span style="color: #0000ff;">new</span><span style="color: #000000;"> CDbCriteria; </span><span style="color: #800080;">$criteria</span>->addCondition("message_id = '".<span style="color: #800080;">$unique_messageId</span>."'"<span style="color: #000000;">); </span><span style="color: #800080;">$messageIdempotent</span> = DynamicAR::model('nt_vm_message_idempotent')->find(<span style="color: #800080;">$criteria</span><span style="color: #000000;">); </span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$messageIdempotent</span><span style="color: #000000;">) { </span><span style="color: #0000ff;">if</span>(<span style="color: #800080;">$messageIdempotent</span>->consume_status == 0<span style="color: #000000;">){ </span><span style="color: #008000;">//</span><span style="color: #008000;">如果找不到,则进行消费此消息</span> <span style="color: #800080;">$callback</span>(<span style="color: #800080;">$msg</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">); }</span><span style="color: #0000ff;">else</span><span style="color: #000000;">{ </span><span style="color: #008000;">//</span><span style="color: #008000;">如果根据消息ID(作为主键)查询出有已经消费过的消息,那么则不进行消费;</span> <span style="color: #800080;">$this</span>->writeLog('runtime/vm_consume_failed.log','该消息已消费,无须重复消费!' . <span style="color: #800080;">$msg</span>-><span style="color: #000000;">getBody()); </span><span style="color: #008000;">//</span><span style="color: #008000;">发送ack信息应答当前消息处理完成</span> <span style="color: #800080;">$msg</span>->delivery_info['channel']->basic_ack(<span style="color: #800080;">$msg</span>->delivery_info['delivery_tag'<span style="color: #000000;">]); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } } </span><span style="color: #0000ff;">else</span><span style="color: #000000;"> { </span><span style="color: #008000;">//</span><span style="color: #008000;">插入太快</span> <span style="color: #800080;">$this</span>->writeLog('runtime/vm_consume_failed.log','插入太快'. <span style="color: #800080;">$msg</span>-><span style="color: #000000;">getBody()); </span><span style="color: #800080;">$this</span>->retryFail(<span style="color: #800080;">$retry</span>,<span style="color: #800080;">$msg</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">); } }</span><span style="color: #0000ff;">catch</span> (<span style="color: #0000ff;">Exception</span> <span style="color: #800080;">$e</span><span style="color: #000000;">){ </span><span style="color: #800080;">$this</span>->writeLog('runtime/vm_exception.log',<span style="color: #800080;">$e</span>-><span style="color: #000000;">getMessage()); </span><span style="color: #800080;">$this</span>->retryFail(<span style="color: #800080;">$retry</span>,<span style="color: #800080;">$msg</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">); } } ); </span><span style="color: #008000;">//</span><span style="color: #008000;">监听通道消息 快递员看有没有信,有就立马寄</span> <span style="color: #0000ff;">while</span> (<span style="color: #008080;">count</span>(<span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">callbacks)) { </span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$shouldExitCallback</span><span style="color: #000000;">()) { </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">try</span><span style="color: #000000;"> { </span><span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">wait(); } </span><span style="color: #0000ff;">catch</span> (AMQPTimeoutException <span style="color: #800080;">$e</span><span style="color: #000000;">) { } </span><span style="color: #0000ff;">catch</span> (AMQPIOWaitException <span style="color: #800080;">$e</span><span style="color: #000000;">) { } } </span><span style="color: #800080;">$this</span>-><span style="color: #000000;">close(); } </span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 重试失败的消息 * 注意: 该方法会堵塞执行 * @param \Closure $callback 回调函数,可以为空,返回true则重新发布,false则丢弃 </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> retryFailed(<span style="color: #800080;">$callback</span> = <span style="color: #0000ff;">null</span><span style="color: #000000;">) { </span><span style="color: #800080;">$this</span>-><span style="color: #000000;">declareConsumeQueue(); </span><span style="color: #800080;">$this</span>-><span style="color: #000000;">declareFailedQueue(); </span><span style="color: #800080;">$queueName</span> = <span style="color: #800080;">$this</span>-><span style="color: #000000;">query_name; </span><span style="color: #800080;">$exchangeName</span> = <span style="color: #800080;">$this</span>-><span style="color: #000000;">exchange_name; </span><span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">basic_consume( </span><span style="color: #800080;">$this</span>->query_failed_name, '', <span style="color: #008000;">//</span><span style="color: #008000;">customer_tag</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">no_local</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">no_ack</span> <span style="color: #0000ff;">true</span>, <span style="color: #008000;">//</span><span style="color: #008000;">exclusive</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">nowait</span> <span style="color: #0000ff;">function</span> (<span style="color: #800080;">$msg</span>) <span style="color: #0000ff;">use</span> (<span style="color: #800080;">$queueName</span>, <span style="color: #800080;">$exchangeName</span>, <span style="color: #800080;">$callback</span><span style="color: #000000;">) { </span><span style="color: #0000ff;">if</span> (<span style="color: #008080;">is_null</span>(<span style="color: #800080;">$callback</span>) || <span style="color: #800080;">$callback</span>(<span style="color: #800080;">$msg</span><span style="color: #000000;">)) { </span><span style="color: #008000;">//</span><span style="color: #008000;"> 重置header中的x-orig-routing-key属性</span> <span style="color: #800080;">$msg</span>->set('application_headers', <span style="color: #0000ff;">new</span><span style="color: #000000;"> AMQPTable([ </span>'x-orig-routing-key' => <span style="color: #800080;">$this</span>->getOrigRoutingKey(<span style="color: #800080;">$msg</span>),<span style="color: #000000;"> ])); </span><span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">basic_publish( </span><span style="color: #800080;">$msg</span>, <span style="color: #800080;">$exchangeName</span>, <span style="color: #800080;">$queueName</span><span style="color: #000000;"> ); } </span><span style="color: #800080;">$msg</span>->delivery_info['channel']->basic_ack(<span style="color: #800080;">$msg</span>->delivery_info['delivery_tag'<span style="color: #000000;">]); } ); </span><span style="color: #0000ff;">while</span> (<span style="color: #008080;">count</span>(<span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">callbacks)) { </span><span style="color: #0000ff;">try</span><span style="color: #000000;"> { </span><span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">wait(); } </span><span style="color: #0000ff;">catch</span> (AMQPTimeoutException <span style="color: #800080;">$e</span><span style="color: #000000;">) { </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } </span><span style="color: #0000ff;">catch</span> (AMQPIOWaitException <span style="color: #800080;">$e</span><span style="color: #000000;">) { } } } </span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 获取绑定queue与exchange时的routingkey </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">protected</span> <span style="color: #0000ff;">function</span> getOrigRoutingKey(<span style="color: #800080;">$msg</span><span style="color: #000000;">){ </span><span style="color: #800080;">$retry</span> = <span style="color: #0000ff;">null</span><span style="color: #000000;">; </span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$msg</span>->has('application_headers'<span style="color: #000000;">)) { </span><span style="color: #800080;">$headers</span> = <span style="color: #800080;">$msg</span>->get('application_headers')-><span style="color: #000000;">getNativeData(); </span><span style="color: #0000ff;">if</span> (<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$headers</span>['x-orig-routing-key'<span style="color: #000000;">])) { </span><span style="color: #800080;">$retry</span> = <span style="color: #800080;">$headers</span>['x-orig-routing-key'<span style="color: #000000;">]; } } </span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$retry</span>?<span style="color: #800080;">$retry</span>:<span style="color: #800080;">$msg</span>->get('routing_key'<span style="color: #000000;">); } </span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 获取消息重试次数 * @param AMQPMessage $msg * @return int </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">protected</span> <span style="color: #0000ff;">function</span> getRetryCount(<span style="color: #800080;">$msg</span><span style="color: #000000;">) { </span><span style="color: #800080;">$retry</span> = 0<span style="color: #000000;">; </span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$msg</span>->has('application_headers'<span style="color: #000000;">)) { </span><span style="color: #800080;">$headers</span> = <span style="color: #800080;">$msg</span>->get('application_headers')-><span style="color: #000000;">getNativeData(); </span><span style="color: #0000ff;">if</span> (<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$headers</span>['x-death'][0]['count'<span style="color: #000000;">])) { </span><span style="color: #800080;">$retry</span> = <span style="color: #800080;">$headers</span>['x-death'][0]['count'<span style="color: #000000;">]; } } </span><span style="color: #0000ff;">return</span> (int)<span style="color: #800080;">$retry</span><span style="color: #000000;">; } </span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 消息重试 </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> retryFail(<span style="color: #800080;">$retry</span>,AMQPMessage <span style="color: #800080;">$msg</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">){ </span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$retry</span> >= 3<span style="color: #000000;">) { </span><span style="color: #008000;">//</span><span style="color: #008000;"> 超过最大重试次数,消息无法处理</span> <span style="color: #800080;">$publishFailed</span>(<span style="color: #800080;">$msg</span><span style="color: #000000;">); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } </span><span style="color: #008000;">//</span><span style="color: #008000;"> 消息处理失败,稍后重试</span> <span style="color: #800080;">$publishRetry</span>(<span style="color: #800080;">$msg</span><span style="color: #000000;">); </span><span style="color: #0000ff;">return</span><span style="color: #000000;">; } </span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 声明重试队列 </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">private</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> declareRetryQueue() { </span><span style="color: #800080;">$this</span>->channel->queue_declare(<span style="color: #800080;">$this</span>->query_retry_name, <span style="color: #0000ff;">false</span>, <span style="color: #0000ff;">true</span>, <span style="color: #0000ff;">false</span>, <span style="color: #0000ff;">false</span>, <span style="color: #0000ff;">false</span>,<span style="color: #0000ff;">new</span> AMQPTable(<span style="color: #0000ff;">array</span><span style="color: #000000;">( </span>'x-dead-letter-exchange' => <span style="color: #800080;">$this</span>->exchange_name, 'x-dead-letter-routing-key' => <span style="color: #800080;">$this</span>->query_name, 'x-message-ttl' => 3 * 1000,<span style="color: #000000;"> ))); </span><span style="color: #800080;">$this</span>->channel->queue_bind(<span style="color: #800080;">$this</span>->query_retry_name, <span style="color: #800080;">$this</span>->exchange_retry_name, <span style="color: #800080;">$this</span>-><span style="color: #000000;">query_name); } </span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 声明消费队列 * @param $priority 消息队列优先级 暂时取个中间值 1-10 </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">private</span> <span style="color: #0000ff;">function</span> declareConsumeQueue(<span style="color: #800080;">$priority</span> = 5<span style="color: #000000;">) { </span><span style="color: #008000;">//</span><span style="color: #008000;">声明队列</span> <span style="color: #800080;">$this</span>->channel-><span style="color: #000000;">queue_declare( </span><span style="color: #800080;">$this</span>->query_name, <span style="color: #008000;">//</span><span style="color: #008000;">队列名称</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">passive消极的,如果该队列已存在,则不会创建;如果不存在,创建新的队列。</span> <span style="color: #0000ff;">true</span>, <span style="color: #008000;">//</span><span style="color: #008000;">durable持久的,指定队列持久</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">exclusive独占的,是否能被其他队列访问true排他的。如果一个队列声明为排他队列,该队列仅对首次声明它的连接可见,并在连接断开时自动删除。 //排它是基于连接可见的,同一个连接不同信道是可以访问同一连接创建的排它队列,“首次”是指如果一个连接已经声明了一个排他队列, //其他连接是不允许建立同名的排他队列,即使这个队列是持久化的,一旦连接关闭或者客户端退出,该排它队列会被自动删除,这种队列适用于一个客户端同时发送与接口消息的场景。</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">设置是否自动删除。当所有消费者都与这个队列断开连接时,这个队列会自动删除。注意: 不是说该队列没有消费者连接时该队列就会自动删除,因为当生产者声明了该队列且没有消费者连接消费时,该队列是不会自动删除的。</span> <span style="color: #0000ff;">false</span>, <span style="color: #008000;">//</span><span style="color: #008000;">nowait</span> <span style="color: #0000ff;">new</span> AMQPTable(<span style="color: #0000ff;">array</span><span style="color: #000000;">( </span>'x-max-priority' => <span style="color: #800080;">$priority</span>, <span style="color: #008000;">//</span><span style="color: #008000;">消息队列的优先级</span>
))
);
//绑定交换机和队列 参数:队列名,交换机名,路由键名
$this->channel->queue_bind($this->query_name, $this->exchange_name, $this->route_key_name);
$this->channel->queue_bind($this->query_name, $this->exchange_name, $this->query_name);
}</span><span style="color: #008000;">/*</span><span style="color: #008000;">* * 声明消费失败队列 </span><span style="color: #008000;">*/</span> <span style="color: #0000ff;">private</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> declareFailedQueue() { </span><span style="color: #800080;">$this</span>->channel->queue_declare(<span style="color: #800080;">$this</span>->query_failed_name, <span style="color: #0000ff;">false</span>, <span style="color: #0000ff;">true</span>, <span style="color: #0000ff;">false</span>, <span style="color: #0000ff;">false</span>, <span style="color: #0000ff;">false</span><span style="color: #000000;">); </span><span style="color: #800080;">$this</span>->channel->queue_bind(<span style="color: #800080;">$this</span>->query_failed_name, <span style="color: #800080;">$this</span>->exchange_failed_name, <span style="color: #800080;">$this</span>-><span style="color: #000000;">query_name); }
}
我们将会实现如下功能
- 结合RabbitMQ的Topic模式和Work Queue模式实现生产方产生消息,消费方按需订阅,消息投递到消费方的队列之后,多个worker同时对消息进行消费
- 结合RabbitMQ的 Message TTL 和 Dead Letter Exchange 实现消息的延时重试功能
- 消息达到最大重试次数之后,将其投递到失败队列,等待人工介入处理bug后,重新将其加入队列消费
具体流程见下图
- 生产者发布消息到主Exchange
- 主Exchange根据Routing Key将消息分发到对应的消息队列
- 多个消费者的worker进程同时对队列中的消息进行消费,因此它们之间采用“竞争”的方式来争取消息的消费
- 消息消费后,不管成功失败,都要返回ACK消费确认消息给队列,避免消息消费确认机制导致重复投递,同时,如果消息处理成功,则结束流程,否则进入重试阶段
- 如果重试次数小于设定的最大重试次数(3次),则将消息重新投递到Retry Exchange的重试队列
- 重试队列不需要消费者直接订阅,它会等待消息的有效时间过期之后,重新将消息投递给Dead Letter Exchange,我们在这里将其设置为主Exchange,实现延时后重新投递消息,这样消费者就可以重新消费消息
- 如果三次以上都是消费失败,则认为消息无法被处理,直接将消息投递给Failed Exchange的Failed Queue,这时候应用可以触发报警机制,以通知相关责任人处理
- 等待人工介入处理(解决bug)之后,重新将消息投递到主Exchange,这样就可以重新消费了
外部确认消息表结构
CREATE TABLE `nt_vm_message_idempotent` (
`message_id` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL DEFAULT '' COMMENT '消息ID',
`message_content` varchar(2000) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL DEFAULT '' COMMENT '消息内容',
`product_status` tinyint(1) UNSIGNED NOT NULL DEFAULT 0 COMMENT '是否生产成功到mq',
`consume_status` tinyint(1) NOT NULL COMMENT '是否消费成功',
`create_time` int(10) UNSIGNED NOT NULL DEFAULT 0,
`update_time` int(10) UNSIGNED NOT NULL DEFAULT 0,
`priority` tinyint(1) UNSIGNED NOT NULL DEFAULT 0 COMMENT '优先级属性',
PRIMARY KEY (`message_id`) USING BTREE,
UNIQUE INDEX `unique_message_id`(`message_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
4.消息的消费脚本
<?php
include_once(ROOT_PATH . 'protected/extensions/rabbitmq/autoload.php');
use PhpAmqpLib\Connection\AMQPStreamConnection;
use PhpAmqpLib\Message\AMQPMessage;
/
- Created by PhpStorm.
- User: tangkeji
- Date: 21-4-26
- Time: 下午3:31
*/
class VmMqCommand extends CConsoleCommand {
</span><span style="color: #0000ff;">private</span> <span style="color: #0000ff;">function</span> _Output(<span style="color: #800080;">$data</span>, <span style="color: #800080;">$isEnd</span> = 0<span style="color: #000000;">) {
</span><span style="color: #0000ff;">if</span> (<span style="color: #008080;">is_array</span>(<span style="color: #800080;">$data</span>) || <span style="color: #008080;">is_object</span>(<span style="color: #800080;">$data</span><span style="color: #000000;">)) {
</span><span style="color: #008080;">var_dump</span>(<span style="color: #800080;">$data</span><span style="color: #000000;">);
</span><span style="color: #0000ff;">echo</span> "\n"<span style="color: #000000;">;
} </span><span style="color: #0000ff;">else</span><span style="color: #000000;"> {
</span><span style="color: #0000ff;">echo</span> <span style="color: #800080;">$data</span> . "\n"<span style="color: #000000;">;
}
</span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$isEnd</span><span style="color: #000000;">) {
Yii</span>::app()-><span style="color: #008080;">end</span><span style="color: #000000;">();
}
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* 消息进程
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span><span style="color: #000000;"> actionRun(){
</span><span style="color: #0000ff;">if</span>(LibCommon::isRunCommand('vmmq run')===<span style="color: #0000ff;">true</span>)<span style="color: #0000ff;">return</span> <span style="color: #0000ff;">true</span><span style="color: #000000;">;
</span><span style="color: #800080;">$stopped</span> = <span style="color: #0000ff;">false</span><span style="color: #000000;">;
</span><span style="color: #008000;">//</span><span style="color: #008000;"> 自动退出计数器,当值小于1的时候退出
// 发生异常-20,正常执行每次-2 (是否开启判断关闭)</span>
<span style="color: #800080;">$autoExitCounter</span> = 200<span style="color: #000000;">;
</span><span style="color: #008000;">//</span><span style="color: #008000;"> 信号处理,接收到SIGUSR2信号的时候自动退出
// 注意:环境必须是php7.1+才支持
// if (function_exists('pcntl_async_signals')) {
// pcntl_async_signals(true);
// }
//
// if (function_exists('pcntl_signal')) {
// pcntl_signal(SIGUSR2, function ($sig) use (&$stopped) {
// $stopped = true;
// });
// }
<span style="color: #800080;">$mq</span> = <span style="color: #0000ff;">new</span><span style="color: #000000;"> VmMq();
</span><span style="color: #800080;">$callback</span> = <span style="color: #0000ff;">function</span> (AMQPMessage <span style="color: #800080;">$msg</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span>) <span style="color: #0000ff;">use</span><span style="color: #000000;">
(
</span>&<span style="color: #800080;">$autoExitCounter</span><span style="color: #000000;">
) {
</span><span style="color: #800080;">$retry</span> = <span style="color: #800080;">$this</span>->getRetryCount(<span style="color: #800080;">$msg</span><span style="color: #000000;">);
</span><span style="color: #0000ff;">try</span><span style="color: #000000;"> {
</span><span style="color: #800080;">$routingKey</span> = <span style="color: #800080;">$this</span>->getOrigRoutingKey(<span style="color: #800080;">$msg</span><span style="color: #000000;">);
</span><span style="color: #800080;">$subMessage</span> = <span style="color: #0000ff;">new</span> SubMessage(<span style="color: #800080;">$msg</span>, <span style="color: #800080;">$routingKey</span> ,<span style="color: #000000;"> [
</span>'retry_count' => <span style="color: #800080;">$retry</span>, <span style="color: #008000;">//</span><span style="color: #008000;"> 重试次数</span>
]);
</span><span style="color: #800080;">$this</span>->subscribe(<span style="color: #800080;">$subMessage</span>,<span style="color: #800080;">$retry</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">);
</span><span style="color: #008000;">//</span><span style="color: #008000;">$autoExitCounter = $autoExitCounter - 2;</span>
} catch (\Exception $ex) {
//$autoExitCounter = $autoExitCounter - 20; // 发生普通异常,退出计数器-20(关闭)
$this->writeLog('runtime/vm_consume_failed.log', '消费失败!' . $ex->getMessage() . $msg->getBody());
$this->retryFail($retry,$msg,$publishRetry,$publishFailed);
}
};
$mq->consume(
$callback,
function () use (&$stopped, &$autoExitCounter) {
return $stopped || $autoExitCounter < 1;
}
);
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* 消息重试
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> retryFail(<span style="color: #800080;">$retry</span>,AMQPMessage <span style="color: #800080;">$msg</span>, <span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">){
</span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$retry</span> >= 3<span style="color: #000000;">) {
</span><span style="color: #008000;">//</span><span style="color: #008000;"> 超过最大重试次数,消息无法处理</span>
<span style="color: #800080;">$publishFailed</span>(<span style="color: #800080;">$msg</span><span style="color: #000000;">);
</span><span style="color: #0000ff;">return</span><span style="color: #000000;">;
}
</span><span style="color: #008000;">//</span><span style="color: #008000;"> 消息处理失败,稍后重试</span>
<span style="color: #800080;">$publishRetry</span>(<span style="color: #800080;">$msg</span><span style="color: #000000;">);
</span><span style="color: #0000ff;">return</span><span style="color: #000000;">;
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* 获取消息重试次数
* @param AMQPMessage $msg
* @return int
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">protected</span> <span style="color: #0000ff;">function</span> getRetryCount(<span style="color: #800080;">$msg</span><span style="color: #000000;">)
{
</span><span style="color: #800080;">$retry</span> = 0<span style="color: #000000;">;
</span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$msg</span>->has('application_headers'<span style="color: #000000;">)) {
</span><span style="color: #800080;">$headers</span> = <span style="color: #800080;">$msg</span>->get('application_headers')-><span style="color: #000000;">getNativeData();
</span><span style="color: #0000ff;">if</span> (<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$headers</span>['x-death'][0]['count'<span style="color: #000000;">])) {
</span><span style="color: #800080;">$retry</span> = <span style="color: #800080;">$headers</span>['x-death'][0]['count'<span style="color: #000000;">];
}
}
</span><span style="color: #0000ff;">return</span> (int)<span style="color: #800080;">$retry</span><span style="color: #000000;">;
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* 订阅消息处理
* @param \Aicode\RabbitMQ\SubMessage $msg
* @param $retry
* @param $publishRetry
* @param $publishFailed
* @return bool 处理成功返回true(返回true后将会对消息进行处理确认),失败throw 异常
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">public</span> <span style="color: #0000ff;">function</span> subscribe(<span style="color: #800080;">$msg</span>,<span style="color: #800080;">$retry</span>,<span style="color: #800080;">$publishRetry</span>, <span style="color: #800080;">$publishFailed</span><span style="color: #000000;">)
{
</span><span style="color: #008000;">//</span><span style="color: #008000;"> TODO 业务逻辑实现
// throw new Exception("消费异常!!!");
echo sprintf(
"subscriber:<%s> %s %s\n",
$msg->getRoutingKey(),
$retry,
$msg->getMessage()
);
echo "----------------------------------------\n";
//存入到表中,标识该消息已消费
$msgBody = json_decode($msg->getMessage(),true);
if(!isset($msgBody['unique_messageId']) || !$msgBody['unique_messageId']){
$this->writeLog('runtime/vm_consume_failed.log','获取消费ID为空!' . $msg->getMessage());
//发送ack信息应答当前消息处理完成
$msg->message->delivery_info['channel']->basic_ack($msg->message->delivery_info['delivery_tag']);
return;
}
$unique_messageId = $msgBody['unique_messageId'];
$criteria = new CDbCriteria;
$criteria->addCondition("message_id = '".$unique_messageId."'");
$messageIdempotent = DynamicAR::model('nt_vm_message_idempotent')->find($criteria);
//如果找到,则更新数据库消费状态
if ($messageIdempotent && $messageIdempotent->consume_status == 0) {
try {
LibCommon::doMqPull($msgBody);
</span><span style="color: #800080;">$update_time</span> = <span style="color: #008080;">time</span><span style="color: #000000;">();
</span><span style="color: #800080;">$connection</span> = Yii::app()-><span style="color: #000000;">db;
</span><span style="color: #800080;">$command</span> = <span style="color: #800080;">$connection</span>->createCommand("<span style="color: #000000;">
UPDATE nt_vm_message_idempotent SET consume_status=1,update_time='</span><span style="color: #800080;">$update_time</span>' WHERE message_id = '<span style="color: #800080;">$unique_messageId</span><span style="color: #000000;">'
</span>"<span style="color: #000000;">);
</span><span style="color: #800080;">$re</span> = <span style="color: #800080;">$command</span>-><span style="color: #000000;">execute();
</span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$re</span><span style="color: #000000;">) {
// $this->writeLog('runtime/vm_consume_log.log', $messageIdempotent->message_id . $msg->getMessage());
//发送ack信息应答当前消息处理完成
$msg->message->delivery_info['channel']->basic_ack($msg->message->delivery_info['delivery_tag']);
return;
} else {
$this->writeLog('runtime/vm_consume_failed.log', '数据库保存失败' . $msg->getMessage());
$this->retryFail($retry,$msg->message, $publishRetry, $publishFailed);
}
}catch (Exception $e) {
echo $e->getMessage();
$this->writeLog('runtime/vm_consume_failed.log', '消费失败!' . $e->getMessage() . $msg->getMessage());
$this->retryFail($retry,$msg->message, $publishRetry, $publishFailed);
}
} else {
//如果根据消息ID(作为主键)查询出有已经消费过的消息,那么则不进行消费;
$this->writeLog('runtime/vm_consume_failed.log','该消息已消费,无须重复消费!' . $msg->getMessage());
//发送ack信息应答当前消息处理完成
$msg->message->delivery_info['channel']->basic_ack($msg->message->delivery_info['delivery_tag']);
}
return true;
}
</span><span style="color: #0000ff;">private</span> <span style="color: #0000ff;">function</span> getOrigRoutingKey(AMQPMessage <span style="color: #800080;">$msg</span><span style="color: #000000;">)
{
</span><span style="color: #800080;">$retry</span> = <span style="color: #0000ff;">null</span><span style="color: #000000;">;
</span><span style="color: #0000ff;">if</span> (<span style="color: #800080;">$msg</span>->has('application_headers'<span style="color: #000000;">)) {
</span><span style="color: #800080;">$headers</span> = <span style="color: #800080;">$msg</span>->get('application_headers')-><span style="color: #000000;">getNativeData();
</span><span style="color: #0000ff;">if</span> (<span style="color: #0000ff;">isset</span>(<span style="color: #800080;">$headers</span>['x-orig-routing-key'<span style="color: #000000;">])) {
</span><span style="color: #800080;">$retry</span> = <span style="color: #800080;">$headers</span>['x-orig-routing-key'<span style="color: #000000;">];
}
}
</span><span style="color: #0000ff;">return</span> <span style="color: #800080;">$retry</span>?<span style="color: #800080;">$retry</span>:<span style="color: #800080;">$msg</span>->get('routing_key'<span style="color: #000000;">);
}
</span><span style="color: #008000;">/*</span><span style="color: #008000;">*
* 日志写入
* @param $file log文件路径
* @param $dataStr 报错字符串
</span><span style="color: #008000;">*/</span>
<span style="color: #0000ff;">protected</span> <span style="color: #0000ff;">function</span> writeLog(<span style="color: #800080;">$file</span>,<span style="color: #800080;">$dataStr</span><span style="color: #000000;">)
{
</span><span style="color: #008080;">file_put_contents</span>(ROOT_PATH.<span style="color: #800080;">$file</span>, <span style="color: #008080;">date</span>('Y-m-d H:i:s').' '.<span style="color: #800080;">$dataStr</span> .<span style="color: #ff00ff;">PHP_EOL</span>,<span style="color: #000000;"> FILE_APPEND);
}
}
三、总结
以上是我的rabbitmq从0到有的经历,可能里面有不完美或者错误请大家指出,必会好好纠正,主要我这个消息要保证消息的可靠性,不容许丢失。里面用到rabbitmq的高级特性如ack确认机制,幂等性,限流机制,重回机制,ttl,死信队列(相当于失败消息的回收站)。
RabbitMQ消息模式(消息100%的投递、幂等性概念) https://blog.csdn.net/weixin_42687829/article/details/104327711
RabbitMQ服务版本升/降级
https://blog.csdn.net/weixin_42687829/article/details/125800276