【Rollo的Python之路】Python 协程 yield与gevent 学习笔记
Python 协程:
协程,又称微线程,纤程。英文名 Coroutine.一句话说明什么是协程: 协程是一种用户态的轻量级线程。
协程拥有自己的寄存器上下文和栈。协程调度切换时,将寄存器上下文和栈保存到其他地方,在切回来的时候,
恢复先前保存的寄存器上下文和栈。因此:协程能保留上一次调用时的状态(即所有局部状态的一个特定组合)
每次过程重入时,就相当于进入上一次调用的状态。换种说法,进入上一次离开时所处逻辑流的位置
协程的好处:
-
无需线程上下文切换的开销
- 无需原子操作锁定及同步的开始
- 方便切换控制流,简化编程模型
- 高并发+高扩展性+低成本: 一个CPU支持上万的协程都不是问题,所有很适合高并发处理
协程的缺点:
- 无法利用多核CUP,
- 进行阻塞(Blocking)操作会阻塞个整个程序
1.1 yield协程:
import time import queue def consumer(name): print('--------------start------------') while True: new_baozi = yield print('%s is eating baozi %s' %(name,new_baozi)) time.sleep(1) def producer(): r = con.__next__() #也可以是Next(con) r = con2.__next__() n =0 while n < 5: n += 1 con.send(n) con2.send(n) print("\033[32;1m[producer]\033[0m is making baozi %s" % n) if __name__ == "__main__": con = consumer('c1') con2 = consumer('c2') p = producer()
1.2 greanlet:
from greenlet import greenlet def test1(): print(12) gr2.switch() print(34) gr2.switch() def test2(): print(56) gr1.switch() print(78) gr1 = greenlet(test1) gr2 = greenlet(test2) gr1.switch()
1.3 gevent协程:
import gevent def foo(): print('Running in foo...') gevent.sleep(1) print('explicit context switch to foo again') def bar(): print('explicit context to bar') gevent.sleep(2) print('Implicit context switch back to bar') gevent.joinall([ gevent.spawn(foo), gevent.spawn(bar), ])
from urllib.request import urlopen from gevent import monkey import gevent import urllib def f(url): print('GET: %s' % url) resp = urllib.request.urlopen(url) data = resp.read() print('%d bytes received from %s.' % (len(data), url)) gevent.joinall([ gevent.spawn(f, 'https://www.python.org/'), gevent.spawn(f, 'https://www.yahoo.com/'), gevent.spawn(f, 'https://github.com/'), ])
加monkey.patch_all(),很快
from urllib.request import urlopen from gevent import monkey monkey.patch_all() import gevent def f(url): print('GET: %s' % url) resp = urlopen(url) data = resp.read() print('%d bytes received from %s.' % (len(data), url)) gevent.joinall([ gevent.spawn(f, 'https://www.python.org/'), gevent.spawn(f, 'https://www.yahoo.com/'), gevent.spawn(f, 'https://github.com/'), ])