23-Python-RabbitMQ
普通队列不能实现不同线程间通信，因此可以使用RabbitMQ。
安装RabbitMQ:http://www.rabbitmq.com/install-windows.html 。另外，还需要安装erlang和pika模块。
1、实现最简单的队列通信
send端代码如下：
1 import pika 2 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters("localhost")) # 创建一个连接 5 6 channel = connection.channel() # 声明管道，消息从管道发送 7 8 9 channel.queue_declare(queue="hello") # 声明queue的名称 10 11 12 channel.basic_publish(exchange="", 13 routing_key="hello", 14 body="hello, world!") 15 16 print("[x] Sent 'Hello World!") 17 18 connection.close()
received端代码如下：
1 import pika 2 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters("localhost")) 5 6 channel = connection.channel() 7 8 channel.queue_declare(queue="hello") 9 10 11 def callback(ch, method, properties, body): 12 print("--->", ch, method, properties) 13 print("[X] Received %r " % body) 14 15 16 channel.basic_consume(callback, queue="hello", no_ack=True) 17 18 print("[X] waiting for message.") 19 20 channel.start_consuming()
2、消息轮询
在这种模式下，RabbitMQ会默认把p发的消息依次分发给各个消费者(c),跟负载均衡差不多。
received端代码如下：
1 import pika 2 import time 3 4 5 connection = pika.BlockingConnection(pika.ConnectionParameters("localhost")) 6 7 channel = connection.channel() 8 9 channel.queue_declare(queue="hello") 10 11 12 def callback(ch, method, properties, body): 13 print("--->", ch, method, properties) 14 # time.sleep(30) 15 print("[X] Received %r " % body) 16 ch.basic_ack(delivery_tag=method.delivery_tag) # 确认消息已经被接收。否则rece端成功接收消息后，断开该rece连接，其余的rece端还会接收该消息。 17 18 19 channel.basic_consume( # 消费消息 20 callback, # 如果收到消息，就调用callback函数来处理消息 21 queue="hello", 22 ) 23 24 print("[X] waiting for message. Exit press CTRL+C") 25 26 channel.start_consuming()
3、消息持久化
send端代码如下：
1 import pika 2 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters("localhost")) # 创建一个连接 5 6 channel = connection.channel() # 声明管道，消息从管道发送 7 8 9 channel.queue_declare(queue="hello3", # 声明queue的名称 10 durable=True # 让队列持久化 11 ) 12 13 14 channel.basic_publish(exchange="", 15 routing_key="hello3", 16 body="hello, world!", 17 properties=pika.BasicProperties( 18 delivery_mode=2, # 让消息持久化 19 )) 20 21 print("[x] Sent 'Hello World!") 22 23 connection.close()
received端代码如下：
1 import pika 2 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters("localhost")) 5 6 channel = connection.channel() 7 8 channel.queue_declare(queue="hello3", durable=True) 9 10 11 def callback(ch, method, properties, body): 12 print("--->", ch, method, properties) 13 print("[X] Received %r " % body) 14 ch.basic_ack(delivery_tag=method.delivery_tag) # 手工确认消息。避免rece接收到消息后断开连接，其它的rece端重复接收该消息。 15 16 17 channel.basic_consume(callback, 18 queue="hello3") 19 20 print("[X] waiting for message.") 21 22 channel.start_consuming()
4、消息公平分发
如果Rabbit只管按顺序把消息发到各个消费者身上，不考虑消费者负载的话，很可能出现，一个机器配置不高的消费者那里堆积了很多消息处理不完，同时配置高的消费者却一直很轻松。为解决此问题，可以在各个消费者端，配置perfetch=1,意思就是告诉RabbitMQ在我这个消费者当前消息还没处理完的时候就不要再给我发新消息了。
1 import pika 2 import time 3 4 5 connection = pika.BlockingConnection(pika.ConnectionParameters("localhost")) 6 7 channel = connection.channel() 8 9 channel.basic_qos(prefetch_count=1) # 最多处理一条消息 10 channel.queue_declare(queue="hello3", durable=True) 11 12 13 def callback(ch, method, properties, body): 14 print("--->", ch, method, properties) 15 time.sleep(20) 16 print("[X] Received %r " % body) 17 ch.basic_ack(delivery_tag=method.delivery_tag) # 手工确认消息。避免rece接收到消息后断开连接，其它的rece端重复接收该消息。 18 19 20 channel.basic_consume(callback, 21 queue="hello3") 22 23 print("[X] waiting for message.") 24 25 channel.start_consuming()
5、Publish\Subscribe(消息发布\订阅)
之前的例子都基本都是1对1的消息发送和接收，即消息只能发送到指定的queue里，但有些时候你想让你的消息被所有的Queue收到，类似广播的效果，这时候就要用到exchange了。
Exchange在定义的时候是有类型的，以决定到底是哪些Queue符合条件，可以接收消息：
fanout: 所有bind到此exchange的queue都可以接收消息；
direct: 通过routingKey和exchange决定的那个唯一的queue可以接收消息；
topic:所有符合routingKey(此时可以是一个表达式)的routingKey所bind的queue可以接收消息；
表达式符号说明：#代表一个或多个字符，*代表任何字符
例：#.a会匹配a.a，aa.a，aaa.a等
*.a会匹配a.a，b.a，c.a等
注：使用RoutingKey为#，Exchange Type为topic的时候相当于使用fanout
headers: 通过headers来决定把消息发给哪些queue。
5.1 fanout模式
消息publisher：
1 import pika 2 import sys 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters(host="localhost")) 5 channel = connection.channel() 6 7 channel.exchange_declare(exchange="logs", 8 exchange_type="fanout") # 根据pika版本不同，来决定这里用type或者exchange_type 9 10 # message = " ".join(sys.argv[1:]) or "info: Hello World!" 11 message = "send info: Hello World!" 12 channel.basic_publish(exchange="logs", 13 routing_key="", 14 body=message) 15 print(" [x] Sent %r" % message) 16 connection.close()
消息subscriber：
1 import pika 2 3 connection = pika.BlockingConnection(pika.ConnectionParameters(host="localhost")) 4 channel = connection.channel() 5 6 channel.exchange_declare(exchange="logs", exchange_type="fanout") 7 8 result = channel.queue_declare(exclusive=True) # 不指定queue名字,rabbit会随机分配一个名字,exclusive=True会在使用此queue的消费者断开后,自动将queue删除 9 queue_name = result.method.queue 10 # print(queue_name) 11 12 channel.queue_bind(exchange="logs", 13 queue=queue_name) 14 15 print(" [*] Waiting for logs. To exit press CTRL+C") 16 17 18 def callback(ch, method, properties, body): 19 print(" [x] %r" % body) 20 21 22 channel.basic_consume(callback, 23 queue=queue_name, 24 no_ack=True) 25 26 channel.start_consuming()
5.2 direct模式
RabbitMQ direct模式支持根据关键字发送，即：队列绑定关键字，发送者将数据根据关键字发送到消息exchange，exchange根据 关键字判定应该将数据发送至指定队列。
消息publisher：
1 import pika 2 import sys 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters( 5 host="localhost")) 6 channel = connection.channel() 7 8 channel.exchange_declare(exchange="direct_logs", 9 exchange_type="direct") 10 11 severity = sys.argv[1] if len(sys.argv) > 1 else "info" 12 message = " ".join(sys.argv[2:]) or "Hello World!" 13 channel.basic_publish(exchange="direct_logs", 14 routing_key=severity, 15 body=message) 16 print(" [x] Sent %r:%r" % (severity, message)) 17 connection.close()
消息subscriber：
1 import pika 2 import sys 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters( 5 host="localhost")) 6 channel = connection.channel() 7 8 channel.exchange_declare(exchange="direct_logs", 9 exchange_type="direct") 10 11 result = channel.queue_declare(exclusive=True) 12 queue_name = result.method.queue 13 14 severities = sys.argv[1:] # severities为列表 15 # print(severities) 16 17 if not severities: 18 sys.stderr.write("Usage: %s [info] [warning] [error]\n" % sys.argv[0]) 19 sys.exit(1) 20 21 for severity in severities: 22 channel.queue_bind(exchange="direct_logs", 23 queue=queue_name, 24 routing_key=severity) 25 26 print(" [*] Waiting for logs. To exit press CTRL+C") 27 28 29 def callback(ch, method, properties, body): 30 print(" [x] %r:%r" % (method.routing_key, body)) 31 32 33 channel.basic_consume(callback, 34 queue=queue_name, 35 no_ack=True) 36 37 channel.start_consuming()
运行subscriber端时输入info参数，代表接收info类型的消息（其它类型依次类推）：
5.3 topic模式
消息publisher：
1 import pika 2 import sys 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters( 5 host="localhost")) 6 channel = connection.channel() 7 8 channel.exchange_declare(exchange="topic_logs", 9 exchange_type="topic") 10 11 routing_key = sys.argv[1] if len(sys.argv) > 1 else "anonymous.info" 12 message = " ".join(sys.argv[2:]) or "Hello World!" 13 channel.basic_publish(exchange="topic_logs", 14 routing_key=routing_key, 15 body=message) 16 print(" [x] Sent %r:%r" % (routing_key, message)) 17 connection.close()
消息subscriber：
1 import pika 2 import sys 3 4 connection = pika.BlockingConnection(pika.ConnectionParameters( 5 host="localhost")) 6 channel = connection.channel() 7 8 channel.exchange_declare(exchange="topic_logs", 9 exchange_type="topic") 10 11 result = channel.queue_declare(exclusive=True) 12 queue_name = result.method.queue 13 14 binding_keys = sys.argv[1:] 15 if not binding_keys: 16 sys.stderr.write("Usage: %s [binding_key]...\n" % sys.argv[0]) 17 sys.exit(1) 18 19 for binding_key in binding_keys: 20 channel.queue_bind(exchange="topic_logs", 21 queue=queue_name, 22 routing_key=binding_key) 23 24 print(" [*] Waiting for logs. To exit press CTRL+C") 25 26 27 def callback(ch, method, properties, body): 28 print(" [x] %r:%r" % (method.routing_key, body)) 29 30 31 channel.basic_consume(callback, 32 queue=queue_name, 33 no_ack=True) 34 35 channel.start_consuming()
To receive all the logs run:
python receive_logs_topic.py "#"
To receive all logs from the facility "kern":
python receive_logs_topic.py "kern.*"
Or if you want to hear only about "critical" logs:
python receive_logs_topic.py "*.critical"
You can create multiple bindings:
python receive_logs_topic.py "kern.*" "*.critical"
And to emit a log with a routing key "kern.critical" type:
python emit_log_topic.py "kern.critical" "A critical kernel error"
