python并发与web
python并发与web
python并发主要方式有:
- Thread(线程)
- Process(进程)
- 协程
python因为GIL的存在使得python的并发无法利用CPU多核的优势以至于性能比较差,下面我们将通过几个例子来介绍python的并发。
线程
我们通过一个简单web server程序来观察python的线程,首先写一个耗时的小函数
def fib(n):
if n <= 2:
return 1
else:
return fib(n - 1) + fib(n - 2)
然后写一个fib web server,程序比较简单就不解释了。
from socket import *
from fib import fib
def fib_server(address):
sock = socket(AF_INET, SOCK_STREAM)
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind(address)
sock.listen(5)
while True:
client, addr = sock.accept()
print('Connection', addr)
fib_handle(client)
def fib_handler(client):
while True:
req = client.recv(100)
if not req:
break
n = int(req)
result = fib(n)
resp = str(result).encode('ascii') + b'\n'
client.send(resp)
print('Closed')
fib_server(('', 25002))
运行shell命令可以看到计算结果
nc localhost 25002
10
55
由于服务段是单线程的,如果另外启动一个连接将得不到计算结果
nc localhost 25002
10
为了能让我们的server支持多个请求,我们对服务端代码加入多线程支持
#sever.py
#服务端代码
from socket import *
from fib import fib
from threading import Thread
def fib_server(address):
sock = socket(AF_INET, SOCK_STREAM)
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind(address)
sock.listen(5)
while True:
client, addr = sock.accept()
print('Connection', addr)
#fib_handler(client)
Thread(target=fib_handler, args=(client,), daemon=True).start() #需要在python3下运行
def fib_handler(client):
while True:
req = client.recv(100)
if not req:
break
n = int(req)
result = fib(n)
resp = str(result).encode('ascii') + b'\n'
client.send(resp)
print('Closed')
fib_server(('', 25002)) #在25002端口启动程序
运行shell命令可以看到计算结果
nc localhost 25002
10
55
由于服务端是多线程的,启动一个新连接将得到计算结果
nc localhost 25002
10
55
性能测试
我们加入一段性能测试代码
#perf1.py
from socket import *
from threading import Thread
import time
sock = socket(AF_INET, SOCK_STREAM)
sock.connect(('localhost', 25002))
n = 0
def monitor():
global n
while True:
time.sleep(1)
print(n, 'reqs/sec')
n = 0
Thread(target=monitor).start()
while True:
start = time.time()
sock.send(b'1')
resp = sock.recv(100)
end = time.time()
n += 1
#代码非常简单,通过全局变量n来统计qps(req/sec 每秒请求数)
在shell中运行perf1.py可以看到结果如下:
- 106025 reqs/sec
- 109382 reqs/sec
- 98211 reqs/sec
- 105391 reqs/sec
- 108875 reqs/sec
平均每秒请求数大概是10w左右
如果我们另外启动一个进程来进行性能测试就会发现python的GIL对线程造成的影响
python3 perf1.py
- 74677 reqs/sec
- 78284 reqs/sec
- 72029 reqs/sec
- 81719 reqs/sec
- 82392 reqs/sec
- 84261 reqs/sec
并且原来的shell中的qps也是类似结果
- 96488 reqs/sec
- 99380 reqs/sec
- 84918 reqs/sec
- 87485 reqs/sec
- 85118 reqs/sec
- 78211 reqs/sec
如果我们再运行
nc localhost 25002
40
来完全占用服务器资源一段时间,就可以看到shell窗口内的rqs迅速下降到
- 99 reqs/sec
- 99 reqs/sec
这也反映了Python的GIL的一个特点,会优先处理占用CPU资源大的任务
具体原因我也不知道,可能需要阅读GIL实现源码才能知道。
线程池在web编程的应用
python有个库叫做cherrypy,最近用到,大致浏览了一下其源代码,其内核使用的是python线程池技术。
cherrypy通过Python线程安全的队列来维护线程池,具体实现为:
class ThreadPool(object):
"""A Request Queue for an HTTPServer which pools threads.
ThreadPool objects must provide min, get(), put(obj), start()
and stop(timeout) attributes.
"""
def __init__(self, server, min=10, max=-1,
accepted_queue_size=-1, accepted_queue_timeout=10):
self.server = server
self.min = min
self.max = max
self._threads = []
self._queue = queue.Queue(maxsize=accepted_queue_size)
self._queue_put_timeout = accepted_queue_timeout
self.get = self._queue.get
def start(self):
"""Start the pool of threads."""
for i in range(self.min):
self._threads.append(WorkerThread(self.server))
for worker in self._threads:
worker.setName('CP Server ' + worker.getName())
worker.start()
for worker in self._threads:
while not worker.ready:
time.sleep(.1)
....
def put(self, obj):
self._queue.put(obj, block=True, timeout=self._queue_put_timeout)
if obj is _SHUTDOWNREQUEST:
return
def grow(self, amount):
"""Spawn new worker threads (not above self.max)."""
if self.max > 0:
budget = max(self.max - len(self._threads), 0)
else:
# self.max <= 0 indicates no maximum
budget = float('inf')
n_new = min(amount, budget)
workers = [self._spawn_worker() for i in range(n_new)]
while not all(worker.ready for worker in workers):
time.sleep(.1)
self._threads.extend(workers)
....
def shrink(self, amount):
"""Kill off worker threads (not below self.min)."""
[...]
def stop(self, timeout=5):
# Must shut down threads here so the code that calls
# this method can know when all threads are stopped.
[...]
可以看出来,cherrypy的线程池将大小初始化为10,每当有一个httpconnect进来时就将其放入任务队列中,然后WorkerThread会不断从任务队列中取出任务执行,可以看到这是一个非常标准的线程池模型。
进程
由于Python的thread无法利用多核,为了充分利用多核CPU,Python可以使用了多进程来模拟线程以提高并发的性能。Python的进程代价比较高可以看做是另外再启动一个python进程。
#server_pool.py
from socket import *
from fib import fib
from threading import Thread
from concurrent.futures import ProcessPoolExecutor as Pool #这里用的python3的线程池,对应python2的threadpool
pool = Pool(4) #启动一个大小为4的进程池
def fib_server(address):
sock = socket(AF_INET, SOCK_STREAM)
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind(address)
sock.listen(5)
while True:
client, addr = sock.accept()
print('Connection', addr)
Thread(target=fib_handler, args=(client,), daemon=True).start()
def fib_handler(client):
while True:
req = client.recv(100)
if not req:
break
n = int(req)
future = pool.submit(fib, n)
result = future.result()
resp = str(result).encode('ascii') + b'\n'
client.send(resp)
print('Closed')
fib_server(('', 25002))
性能测试
可以看到新的server的qps为:
- 4613 reqs/sec
- 4764 reqs/sec
- 4619 reqs/sec
- 4393 reqs/sec
- 4768 reqs/sec
- 4846 reqs/sec
这个结果远低于前面的10w qps主要原因是进程启动速度较慢,进程池内部逻辑比较复杂,涉及到了数据传输,队列等问题。
但是通过多进程我们可以保证每一个链接相对独立,不会受其他请求太大的影响。
即使我们使用以下耗时的命令也不会影响到性能测试
nc localhost 25502
40
协程
协程简介
协程是一个古老的概念,最早出现在早期的os中,它出现的时间甚至比线程进程还要早。
协程也是一个比较难以理解和运用的并发方式,用协程写出来的代码比较难以理解。
python中使用yield和next来实现协程的控制。
def count(n):
while(n > 0):
yield n #yield起到的作用是blocking,将代码阻塞在这里,生成一个generator,然后通过next调用。
n -= 1
for i in count(5):
print(i)
#可以看到运行结果:
5
4
3
2
1
下面我们通过例子来介绍如何书写协程代码。首先回到之前的代码。首先我们要想到我们为什么要用线程,当然是为了防止阻塞,
这里的阻塞来自socket的IO和cpu占用2个方面。协程的引入也是为了防止阻塞,因此我们先将代码中的阻塞点标记出来。
#sever.py
#服务端代码
from socket import *
from fib import fib
def fib_server(address):
sock = socket(AF_INET, SOCK_STREAM)
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind(address)
sock.listen(5)
while True:
client, addr = sock.accept() #blocking
print('Connection', addr)
fib_handler(client)
def fib_handler(client):
while True:
req = client.recv(100) #blocking
if not req:
break
n = int(req)
result = fib(n)
resp = str(result).encode('ascii') + b'\n'
client.send(resp) #blocking
print('Closed')
fib_server(('', 25002)) #在25002端口启动程序
上面标记了3个socket IO阻塞点,我们先忽略CPU占用。
- 首先我们在blocking点插入yield语句,这样做的原因就是,通过yield标记出blocking点以及blocking的原因,这样我们就可以在调度的时候实现noblocking,我们调度的时候遇到yield语句并且block之后就可以直接去执行其他的请求而不用阻塞在这里,这里我们也将实现一个简单的noblocking调度方法。
#sever.py
#服务端代码
from socket import *
from fib import fib
def fib_server(address):
sock = socket(AF_INET, SOCK_STREAM)
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind(address)
sock.listen(5)
while True:
yield 'recv', sock
client, addr = sock.accept() #blocking
print('Connection', addr)
fib_handler(client)
def fib_handler(client):
while True:
yield 'recv', client
req = client.recv(100) #blocking
if not req:
break
n = int(req)
result = fib(n)
resp = str(result).encode('ascii') + b'\n'
yield 'send', client
client.send(resp) #blocking
print('Closed')
fib_server(('', 25002)) #在25002端口启动程序
- 上述程序无法运行,因为我们还没有一个yield的调度器,程序只是单纯的阻塞在了yield所标记的地方,这也是协程的一个好处,可以人为来调度,不像thread一样乱序执行。下面是包含了调度器的代码。
from socket import *
from fib import fib
from threading import Thread
from collections import deque
from concurrent.futures import ProcessPoolExecutor as Pool
from select import select
tasks = deque()
recv_wait = {}
send_wait = {}
def run():
while any([tasks, recv_wait, send_wait]):
while not tasks:
can_recv, can_send, _ = select(recv_wait, send_wait, [])
for s in can_recv:
tasks.append(recv_wait.pop(s))
for s in can_send:
tasks.append(send_wait.pop(s))
task = tasks.popleft()
try:
why, what = next(task)
if why == 'recv':
recv_wait[what] = task
elif why == 'send':
send_wait[what] = task
else:
raise RuntimeError("ARG!")
except StopIteration:
print("task done")
def fib_server(address):
sock = socket(AF_INET, SOCK_STREAM)
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind(address)
sock.listen(5)
while True:
yield 'recv', sock
client, addr = sock.accept()
print('Connection', addr)
tasks.append(fib_handler(client))
def fib_handler(client):
while True:
yield 'recv', client
req = client.recv(100)
if not req:
break
n = int(req)
result = fib(n)
resp = str(result).encode('ascii') + b'\n'
yield 'send', client
client.send(resp)
print('Closed')
tasks.append(fib_server(('', 25003)))
run()
- 我们通过轮询+select来控制协程,核心是用一个task queue来维护程序运行的流水线,用recv_wait和send_wait两个字典来实现任务的分发。
性能测试
可以看到新的server的qps为:
- (82262, 'reqs/sec')
- (82915, 'reqs/sec')
- (82128, 'reqs/sec')
- (82867, 'reqs/sec')
- (82284, 'reqs/sec')
- (82363, 'reqs/sec')
- (82954, 'reqs/sec')
与之前的thread模型性能比较接近,协程的好处是异步的,但是协程 仍然只能使用到一个CPU
当我们让服务器计算40的fib从而占满cpu时,qps迅速下降到了0。
tornado 基于协程的 python web框架
tornado是facebook出品的异步web框架,tornado中协程的使用比较简单,利用coroutine.gen装饰器可以将自己的异步函数注册进tornado的ioloop中,tornado异步方法一般的书写方式为:
@gen.coroutime
def post(self):
resp = yield GetUser()
self.write(resp)
tornado异步原理
def start(self):
"""Starts the I/O loop.
The loop will run until one of the I/O handlers calls stop(), which
will make the loop stop after the current event iteration completes.
"""
self._running = True
while True:
[ ... ]
if not self._running:
break
[ ... ]
try:
event_pairs = self._impl.poll(poll_timeout)
except Exception, e:
if e.args == (4, "Interrupted system call"):
logging.warning("Interrupted system call", exc_info=1)
continue
else:
raise
# Pop one fd at a time from the set of pending fds and run
# its handler. Since that handler may perform actions on
# other file descriptors, there may be reentrant calls to
# this IOLoop that update self._events
self._events.update(event_pairs)
while self._events:
fd, events = self._events.popitem()
try:
self._handlers[fd](fd, events)
except KeyboardInterrupt:
raise
except OSError, e:
if e[0] == errno.EPIPE:
# Happens when the client closes the connection
pass
else:
logging.error("Exception in I/O handler for fd %d",
fd, exc_info=True)
except:
logging.error("Exception in I/O handler for fd %d",fd, exc_info=True)
这是tornado异步调度的核心主循环,poll()方法返回一个形如(fd: events)的键值对,并赋值给event_pairs变量,在内部的while循环中,event_pairs中的内容被一个一个的取出,然后相应的处理器会被调用,tornado通过下面的函数讲socket注册进epoll中。tornado在linux默认选择epoll,在windows下默认选择select(只能选择select)。
def add_handler(self, fd, handler, events):
"""Registers the given handler to receive the given events for fd."""
self._handlers[fd] = handler
self._impl.register(fd, events | self.ERROR)
cherrypy线程池与tornado协程的比较
我们通过最简单程序运行在单机上进行性能比较
测试的语句为:
ab -c 100 -n 1000 -k localhost:8080/ | grep "Time taken for tests:"
其中cherrypy的表现为:
- Completed 100 requests
- Completed 200 requests
- Completed 300 requests
- Completed 400 requests
- Completed 500 requests
- Completed 600 requests
- Completed 700 requests
- Completed 800 requests
- Completed 900 requests
- Completed 1000 requests
- Finished 1000 requests
Time taken for tests: 10.773 seconds
tornado的表现为:
- Completed 100 requests
- Completed 200 requests
- Completed 300 requests
- Completed 400 requests
- Completed 500 requests
- Completed 600 requests
- Completed 700 requests
- Completed 800 requests
- Completed 900 requests
- Completed 1000 requests
- Finished 1000 requests
Time taken for tests: 0.377 seconds
可以看出tornado的性能还是非常惊人的,当应用程序涉及到异步IO还是要尽量使用tornado
总结
本文主要介绍了python的线程、进程和协程以及其应用,并对这几种模型进行了简单的性能分析,python由于GIL的存在,不管是线程还是协程都不能利用到多核。
- 对于计算密集型的web app线程模型与协程模型的性能大致一样,线程由于调度受操作系统管理,其性能略好。
- 对于IO密集型的web app协程模型性能会有很大的优势。
参考文献
略