webpy/flask/bottle性能测试
这三个都是Python WSGI的web开发框架，到底用哪个呢？单纯从性能的角度而言，可能哪个快就用哪个，但是这也不是绝对的。比如我就比较喜欢webpy的router配置放在一个文件中，而flask和bottle的配置分散到各个文件中，从开发角度，写在哪里无所谓，但是从阅读的角度，webpy就比较方便了。因为在工作中本着拿来主义的角度使用了webpy，但是看起来flask也很火，而bottle据说更是一个精简的文件，所以本文简单测试了三者的性能，供参考。
测试方法：用三者分别写一个URL API，这个API直接返回“hello world!”字符串，用uwsgi启动python应用。客户端采用apache benchmark工具：ab，对server执行500000次请求，并发为1000，然后比较ab的测试结果，同时用统计机器的CPU消耗。
测试脚本：http://pan.baidu.com/s/1URGJ0，提取密码：6thp
测试结果：
CPU曲线图（测试顺序：webpy，sleep 120秒，flask，sleep 120秒，bottle）
ab的测试结果：
- webpy
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
……
Completed 50000 requests
Completed …… requests
Finished 500000 requests
……
Server Port: 8010
Document Path: /
Document Length: 12 bytes
Concurrency Level: 1000
Time taken for tests: 1533.489 seconds
Complete requests: 500000
Failed requests: 24246
(Connect: 0, Receive: 0, Length: 24246, Exceptions: 0)
Write errors: 0
Total transferred: 14748374 bytes
HTML transferred: 5709048 bytes
Requests per second: 326.05 [#/sec] (mean)
Time per request: 3066.978 [ms] (mean)
Time per request: 3.067 [ms] (mean, across all concurrent requests)
Transfer rate: 9.39 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 10.1 1 3000
Processing: 111 2962 7581.0 645 69933
Waiting: 0 1629 3949.2 642 68942
Total: 218 2963 7581.6 646 69933
Percentage of the requests served within a certain time (ms)
50% 646
......
100% 69933 (longest request)
- flask
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
......
Completed 50000 requests
......
Completed 500000 requests
Finished 500000 requests
......
Server Port: 8020
Document Path: /
Document Length: 12 bytes
Concurrency Level: 1000
Time taken for tests: 862.997 seconds
Complete requests: 500000
Failed requests: 17067
(Connect: 0, Receive: 0, Length: 17067, Exceptions: 0)
Write errors: 0
Total transferred: 43946903 bytes
HTML transferred: 5795196 bytes
Requests per second: 579.38 [#/sec] (mean)
Time per request: 1725.993 [ms] (mean)
Time per request: 1.726 [ms] (mean, across all concurrent requests)
Transfer rate: 49.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 42.4 1 3002
Processing: 40 1635 6311.8 196 69819
Waiting: 0 725 3301.8 195 69030
Total: 71 1636 6312.6 197 69819
Percentage of the requests served within a certain time (ms)
50% 197
......
100% 69819 (longest request)
- bottle
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
......
Completed 50000 requests
......
Completed 500000 requests
Finished 500000 requests
......
Server Port: 8030
Document Path: /
Document Length: 12 bytes
Concurrency Level: 1000
Time taken for tests: 419.576 seconds
Complete requests: 500000
Failed requests: 4111
(Connect: 0, Receive: 0, Length: 4111, Exceptions: 0)
Write errors: 0
Total transferred: 45125899 bytes
HTML transferred: 5950668 bytes
Requests per second: 1191.68 [#/sec] (mean)
Time per request: 839.153 [ms] (mean)
Time per request: 0.839 [ms] (mean, across all concurrent requests)
Transfer rate: 105.03 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 21 256.5 1 9015
Processing: 9 709 3949.7 117 69845
Waiting: 0 460 2441.9 116 67652
Total: 29 730 3957.6 119 69846
Percentage of the requests served within a certain time (ms)
50% 119
......
100% 69846 (longest request)
结果分析：
（1）从平均的response time：bottle(0.839) < flask(1.726) < webpy(3.067)
（2）从TPS：bottle(1191.68) > flask(579.38) > webpy(326.05)
（3）从Failed请求数：bottle(1191.68) > flask(579.38) > webpy(326.05)
[注]：为什么会有fail？从uwsgi的log看来，报了一些（Write IO Error），这个错误的原因是当uwsgi处理完请求返回结果时，发现客户端已经断开了连接，结果无处可送，则Writer IO Error。一般情况下，这是由于客户端响应太慢，导致了客户端timeout所致。看起来响应还是足够快的，并且并发只有1000，这里为何会timeout，不是十分确定根本原因。
（4）从CPU：
- webpy最耗CPU，其中黄色部分是（CPU system time），看起来系统调用比其余两个高很多。
- bottle比flask略高一些，总体是上，相差不算过大。
结论：性能排序结果为webpy < flask < bottle。
那是不是就意味着应该选性能最好的bottle？这也不应太绝对，因为在开发项目时，不同的人可能对不同的项目的熟悉程度不一样，另外flask可能有广泛的插件支持，所以选择最适合自己的，而不是一昧追求性能最快。
说明：本文试图比较三者之间的性能，而不是测试三个模块的性能极限（例如我都没有说明测试机的配置，因为这里测试的是同一配置下的相对值）。如果测试三个模块的性能极限，可能server配置、以及uwsgi的配置均需要根据测试进行优化，在这里不是重点，不再赘述。
