协程及其原理和应用
1.协程
1 import gevent 2 3 4 def foo(): 5 print('Running in foo') 6 gevent.sleep(0) 7 print('Explicit context switch to foo again') 8 9 10 def bar(): 11 print('Explicit context to bar') 12 gevent.sleep(0) 13 print('Implicit context switch back to bar') 14 ''' 15 遇到sleep就切换到下一个 16 ''' 17 18 gevent.joinall([ 19 gevent.spawn(foo), 20 gevent.spawn(bar), 21 ])
结果显示:如注释所写,遇到sleep就切换。
Running in foo Explicit context to bar Explicit context switch to foo again Implicit context switch back to bar
2.协程原理
1 def consumer(name): 2 print("--->starting eating baozi...") 3 while True: 4 new_baozi = yield 5 print("[%s] is eating baozi %s" % (name, new_baozi)) 6 # time.sleep(1) 7 8 9 def producer(): 10 r = con.__next__() 11 r = con2.__next__() 12 n = 0 13 while n < 5: 14 n += 1 15 con.send(n) 16 con2.send(n) 17 print("\033[32;1m[producer]\033[0m is making baozi %s" % n) 18 19 20 if __name__ == '__main__': 21 con = consumer("c1") 22 con2 = consumer("c2") 23 p = producer()
上述的代码是之前所学到的吃包子。其中几点注意:
1.只有含有yield的函数才能有next方法。现在一般为next(conn)
2.执行next方法后函数才会运行(函数加括号不能启动函数),遇到yield后冻结,返回yield后的值给调用next的对象。
3.send也有next的效果,但是是将自己值送给yield前面的变量。但是不能先手send。send不能启动函数
结果显示:
1 --->starting eating baozi... 2 --->starting eating baozi... 3 [c1] is eating baozi 1 4 [c2] is eating baozi 1 5 [producer] is making baozi 1 6 [c1] is eating baozi 2 7 [c2] is eating baozi 2 8 [producer] is making baozi 2 9 [c1] is eating baozi 3 10 [c2] is eating baozi 3 11 [producer] is making baozi 3 12 [c1] is eating baozi 4 13 [c2] is eating baozi 4 14 [producer] is making baozi 4 15 [c1] is eating baozi 5 16 [c2] is eating baozi 5 17 [producer] is making baozi 5
3.爬虫
1 from gevent import monkey 2 import time 3 4 monkey.patch_all() # 监听IO口的程序 大量节约时间 5 import gevent 6 from urllib.request import urlopen 7 8 9 def f(url): 10 print('GET: %s' % url) 11 resp = urlopen(url) 12 data = resp.read() 13 print('%d bytes received from %s.' % (len(data), url)) 14 start = time.time() 15 16 gevent.joinall([ 17 gevent.spawn(f, 'https://www.python.org/'), 18 gevent.spawn(f, 'https://www.yahoo.com/'), 19 gevent.spawn(f, 'https://github.com/'), 20 ]) 21 22 print(time.time()-start)
只是简单讲主页的数据全爬了一遍没做处理。