rabbitMQ消息队列
https://www.cnblogs.com/wupeiqi/articles/5132791.html
RabbitMQ是一个在AMQP基础上完整的,可复用的企业消息系统。他遵循Mozilla Public License开源协议。
MQ全称为Message Queue, 消息队列(MQ)是一种应用程序对应用程序的通信方法。应用程序通过读写出入队列的消息(针对应用程序的数据)来通信,而无需专用连接来链接它们。消 息传递指的是程序之间通过在消息中发送数据进行通信,而不是通过直接调用彼此来通信,直接调用通常是用于诸如远程过程调用的技术。排队指的是应用程序通过 队列来通信。队列的使用除去了接收和发送应用程序同时执行的要求。
1.rabbitMQ安装步骤
服务端安装
1. rpm -ivh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm 2. rpm -ivh https://dl.bintray.com/rabbitmq/all/rabbitmq-server/3.7.7/rabbitmq-server-3.7.7-1.el7.noarch.rpm 3. yum -y install erlang 4. yum -y install rabbitmq-server 5. service rabbitmq-server start 6. 用户名和密码配置: sudo rabbitmqctl add_user wupeiqi 123 7. 设置用户为administrator角色 sudo rabbitmqctl set_user_tags wupeiqi administrator 8. 设置权限 sudo rabbitmqctl set_permissions -p "/" wupeiqi ".*" ".*" ".*"
客户端安装
pip install pika or easy_install pika or 源码 https://pypi.python.org/pypi/pika
使用API操作RabbitMQ
1. 基于Queue实现生产者消费者模型
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
#!/usr/bin/env python # -*- coding:utf-8 -*- import Queue import threading message = Queue.Queue(10) def producer(i): while True: message.put(i) def consumer(i): while True: msg = message.get() for i in range(12): t = threading.Thread(target=producer, args=(i,)) t.start() for i in range(10): t = threading.Thread(target=consumer, args=(i,)) t.start()
一、RabbitMQ来说,生产和消费不再针对内存里的一个Queue对象,服务器上的RabbitMQ Server实现的消息队列。
生产者
import pika credentials = pika.PlainCredentials("wupeiq","123") connection =pika.BlockingConnection(pika.ConnectionParameters(host="132.232.55.209")) #定义一个队列 channel =connection.channel() channel.queue_declare(queue="hello") channel.basic_publish(exchange="",routing_key="hello", body="HELLO, WORLD!!!") #要向指定队列中放入数据. print("[x] Send ‘hello world’") connection.close()
消费者
import pika # credentials = pika.PlainCredentials("wupeiq","123") connection =pika.BlockingConnection(pika.ConnectionParameters(host="132.232.55.209")) channel =connection.channel() #定义一个队列 channel.queue_declare(queue="hello") def callback(ch,method,properties,body): print("[x]Received %r" %body) #对名字叫做hello的队列进行消费(获取队列中的数据) channel.basic_consume(callback,queue='hello',no_ack=True) print("[*]Waiting for Message. to Exit press CTRL + C") channel.start_consuming()
二、acknowledgment 消息不丢失
no-ack = False,如果消费者遇到情况(its channel is closed, connection is closed, or TCP connection is lost)挂掉了,那么,RabbitMQ会重新将该任务添加到队列中。
生产者
import pika credentials = pika.PlainCredentials("wupeiq","123") connection =pika.BlockingConnection(pika.ConnectionParameters(host="132.232.55.209")) #定义一个队列 channel =connection.channel() channel.queue_declare(queue="s2") channel.basic_publish(exchange="",routing_key="s2", body="6666!!") #要向指定队列中放入数据. print("[x] Send ‘hello world’") connection.close()
消费者
import pika # credentials = pika.PlainCredentials("wupeiq","123") connection =pika.BlockingConnection(pika.ConnectionParameters(host="132.232.55.209")) channel =connection.channel() #定义一个队列 channel.queue_declare(queue="hello") def callback(ch,method,properties,body): print("[x]Received %r" %body) ch.basic_ack(delivery_tag=method.delivery_tag) #对名字叫做hello的队列进行消费(获取队列中的数据) channel.basic_consume(callback,queue='s2',no_ack=False) print("[*]Waiting for Message. to Exit press CTRL + C") channel.start_consuming()
三、durable 消息不丢失
生产者
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() # make message persistent channel.queue_declare(queue='s3', durable=True) channel.basic_publish(exchange='', routing_key='s3', body='Hello World!', properties=pika.BasicProperties( delivery_mode=2, # make message persistent )) print(" [x] Sent 'Hello World!'") connection.close()
消费者
#!/usr/bin/env python # -*- coding:utf-8 -*- import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() # make message persistent channel.queue_declare(queue='s3', durable=True) def callback(ch, method, properties, body): print(" [x] Received %r" % body) ch.basic_ack(delivery_tag = method.delivery_tag) channel.basic_consume(callback, queue='s3', no_ack=False) print(' [*] Waiting for messages. To exit press CTRL+C') channel.start_consuming()
四、消息获取顺序
默认消息队列里的数据是按照顺序被消费者拿走,例如:消费者1 去队列中获取 奇数 序列的任务,消费者1去队列中获取 偶数 序列的任务。
channel.basic_qos(prefetch_count=1) 表示谁来谁取,不再按照奇偶数排列
生产者:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() # make message persistent channel.queue_declare(queue='s4', durable=True) channel.basic_publish(exchange='', routing_key='s4', body='Hello World!', properties=pika.BasicProperties( delivery_mode=2, # make message persistent )) print(" [x] Sent 'Hello World!'") connection.close()
消费者1
#!/usr/bin/env python # -*- coding:utf-8 -*- import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() # make message persistent channel.queue_declare(queue='s4', durable=True) def callback(ch, method, properties, body): print(" [x] Received %r" % body) ch.basic_ack(delivery_tag = method.delivery_tag) channel.basic_qos(prefetch_count=1) channel.basic_consume(callback, queue='s4', no_ack=False) print(' [*] Waiting for messages. To exit press CTRL+C') channel.start_consuming()
生产者2
#!/usr/bin/env python # -*- coding:utf-8 -*- import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() # make message persistent channel.queue_declare(queue='s4', durable=True) def callback(ch, method, properties, body): print(" [x] Received %r" % body) ch.basic_ack(delivery_tag = method.delivery_tag) channel.basic_qos(prefetch_count=1) channel.basic_consume(callback, queue='s4', no_ack=False) print(' [*] Waiting for messages. To exit press CTRL+C') channel.start_consuming()
以上例子是官方文档中的前两个例子.即:
https://www.rabbitmq.com/getstarted.html
五、发布订阅
发布订阅和简单的消息队列区别在于,发布订阅会将消息发送给所有的订阅者,而消息队列中的数据被消费一次便消失。所以,RabbitMQ实现发布和订阅时,会为每一个订阅者创建一个队列,而发布者发布消息时,会将消息放置在所有相关队列中。
exchange type = fanout
发布者
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex1',exchange_type="fanout") message = "HELLO WORLD" channel.basic_publish(exchange='ex1',routing_key='',body=message,) #route_key 为队列名为空. print(" [x] Sent 'Hello World!'") connection.close()
订阅者1:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex1', exchange_type="fanout") #动态生成一个队列 result =channel.queue_declare(exclusive=True) queue_name =result.method.queue print(queue_name) #exchange 和队列进行绑定 channel.queue_bind(exchange="ex1",queue=queue_name) def callback(ch,method,properties,body): print("[x] %r" %body) channel.basic_consume(callback,queue=queue_name,no_ack=True) channel.start_consuming()
订阅者2:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex1',exchange_type="fanout") #动态生成一个队列 result =channel.queue_declare(exclusive=True) queue_name =result.method.queue print(queue_name) #exchange 和队列进行绑定 channel.queue_bind(exchange="ex1",queue=queue_name) def callback(ch,method,properties,body): print("[x] %r" %body) channel.basic_consume(callback,queue=queue_name,no_ack=True) channel.start_consuming()
六、关键字发送
exchange type = direct
之前事例,发送消息时明确指定某个队列并向其中发送消息,RabbitMQ还支持根据关键字发送,即:队列绑定关键字,发送者将数据根据关键字发送到消息exchange,exchange根据 关键字 判定应该将数据发送至指定队列。
生产者:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex2',exchange_type="direct") message = "HELLO WORLD" channel.basic_publish(exchange='ex2',routing_key='error',body=message,) #route_key 为队列名为空. connection.close()
消费者1:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex2', exchange_type="direct") #动态生成一个队列 result =channel.queue_declare(exclusive=True) queue_name =result.method.queue print(queue_name) #exchange 和队列进行绑定 channel.queue_bind(exchange="ex2",queue=queue_name,routing_key="info") channel.queue_bind(exchange="ex2",queue=queue_name,routing_key="error") channel.queue_bind(exchange="ex2",queue=queue_name,routing_key="warning") def callback(ch,method,properties,body): print("[x] %r" %body) channel.basic_consume(callback,queue=queue_name,no_ack=True) channel.start_consuming()
消费者2:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex2', exchange_type="direct") #动态生成一个队列 result =channel.queue_declare(exclusive=True) queue_name =result.method.queue print(queue_name) #exchange 和队列进行绑定 channel.queue_bind(exchange="ex2",queue=queue_name,routing_key="info") def callback(ch,method,properties,body): print("[x] %r" %body) channel.basic_consume(callback,queue=queue_name,no_ack=True) channel.start_consuming()
七、模糊匹配
exchange type = topic
在topic类型下,可以让队列绑定几个模糊的关键字,之后发送者将数据发送到exchange,exchange将传入”路由值“和 ”关键字“进行匹配,匹配成功,则将数据发送到指定队列。
- # 表示可以匹配 0 个 或 多个 单词
- * 表示只能匹配 一个 单词
1
2
3
|
发送者路由值 队列中 old.boy.python old. * - - 不匹配 old.boy.python old. # -- 匹配 |
生产者:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex3',exchange_type="topic") message = "HELLO WORLD" channel.basic_publish(exchange='ex3',routing_key='old.boy.python',body=message,) #route_key 为队列名为空. connection.close()
消费者1:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex3', exchange_type="topic") #动态生成一个队列 result =channel.queue_declare(exclusive=True) queue_name =result.method.queue print(queue_name) #exchange 和队列进行绑定 channel.queue_bind(exchange="ex3",queue=queue_name,routing_key="old.*") # “*”匹配一个单词 def callback(ch,method,properties,body): print("[x] %r" %body) channel.basic_consume(callback,queue=queue_name,no_ack=True) channel.start_consuming()
消费者2:
import pika connection = pika.BlockingConnection(pika.ConnectionParameters(host='132.232.55.209')) channel = connection.channel() #定义exchange channel.exchange_declare(exchange='ex3', exchange_type="topic") #动态生成一个队列 result =channel.queue_declare(exclusive=True) queue_name =result.method.queue print(queue_name) #exchange 和队列进行绑定 channel.queue_bind(exchange="ex3",queue=queue_name,routing_key="old.#") # "#" 匹配所有单词 def callback(ch,method,properties,body): print("[x] %r" %body) channel.basic_consume(callback,queue=queue_name,no_ack=True) channel.start_consuming()