web服务器都提供长连接的方式,所谓长连接就是客户端一次请求完后,不关闭连接,保持一段时间的连接,下次此客户端再次请求时,不用创建新连接,复用所保持的连接即可。从理论上,长连接可以免去大量建立和关闭连接的资源消耗,但同时也有大量连接被占用的代价。因此可以初步判断长连接比短连接能带来更高的TPS,更低的CPU消耗,更少的IO,更高的内存占用,下面通过实战来验证。
服务器环境和测试工具可以见工具和环境准备篇
本次web服务器选用apache prefork模式,apache长短连接的选择可以配置httpd.conf里的KeepAlive选项,如:
KeepAlive On:长连接
KeepAlive Off:短连接
另外如果选择长连接还需配置KeepAliveTimeout和MaxKeepAliveRequests,其中KeepAliveTimeout为每个长连接服务端保持时长,默认配置为15秒,MaxKeepAliveRequests为每个长连接服务请求最大数,默认配置为100次,本次试验保持默认配置。
使用ab来压apache,如:
短连接环境下: /usr/alibaba/install/httpd-2.0.63-prefork/bin/ab -c 100 -n 1000000 http://localhost/
长连接环境下: /usr/alibaba/install/httpd-2.0.63-prefork/bin/ab -c 100 -n 1000000 -k http://localhost/
同时都已100个并发请求apache默认首页1000000次
然后通过ab测试结果、nmon收集资源消耗和strace跟踪实际调用三个维度来考察短连接和长连接的区别。
1)ab测试结果
短连接:
- Concurrency Level: 100
- Time taken for tests: 190.754776 seconds
- Complete requests: 1000000
- Failed requests: 0
- Write errors: 0
- Total transferred: 1891115351 bytes
- HTML transferred: 1456088816 bytes
- Requests per second: 5242.33 [#/sec] (mean)
- Time per request: 19.075 [ms] (mean)
- Time per request: 0.191 [ms] (mean, across all concurrent requests)
- Transfer rate: 9681.50 [Kbytes/sec] received
- Connection Times (ms)
- min mean[+/-sd] median max
- Connect: 0 8 3.7 8 44
- Processing: 1 10 3.8 9 79
- Waiting: 0 7 3.0 7 61
- Total: 4 18 5.7 17 101
长连接:
- Concurrency Level: 100
- Time taken for tests: 59.509558 seconds
- Complete requests: 1000000
- Failed requests: 0
- Write errors: 0
- Keep-Alive requests: 990148
- Total transferred: 1927566346 bytes
- HTML transferred: 1456007280 bytes
- Requests per second: 16804.02 [#/sec] (mean)
- Time per request: 5.951 [ms] (mean)
- Time per request: 0.060 [ms] (mean, across all concurrent requests)
- Transfer rate: 31631.71 [Kbytes/sec] received
- Connection Times (ms)
- min mean[+/-sd] median max
- Connect: 0 0 0.1 0 12
- Processing: 0 5 22.5 1 1406
- Waiting: 0 5 22.4 1 1405
- Total: 0 5 22.5 1 1409
从中不然发现,在其他参数和环境相同的情况下,长连接比短连接的TPS高很多,16804.02/sec vs 5242.33/sec,另外也不难发现长连接在connection上花的时间几乎为0
2)nmon 的测试结果
cpu消耗:
短连接
长连接
以上数据表明长连接比短连接消耗CPU较少
IO占用:
短连接
长连接
以上数据表明长连接比短连接IO占用更少
内存空闲:
短连接
长连接
以上数据表明长连接比短连接占用更多内存
3)strace结果
apache的prefork模式是每个请求由单独的子进程来响应,因此通过对其中的一个子进程跟踪来比较调用系统资源的次数
短连接:
- % time seconds usecs/call calls errors syscall
- ------ ----------- ----------- --------- --------- ----------------
- 44.24 0.187941 19 9997 accept
- 40.22 0.170887 10 17738 poll
- 2.58 0.010976 0 67716 17737 read
- 2.49 0.010583 0 59964 9994 lstat
- 2.19 0.009319 0 49970 9994 stat
- 1.74 0.007388 0 39976 setsockopt
- 1.42 0.006045 1 9997 shutdown
- 1.25 0.005312 0 29988 close
- 1.06 0.004499 0 19989 open
- 0.71 0.003003 0 19994 fcntl
- 0.57 0.002426 0 9994 write
- 0.45 0.001911 0 9994 writev
- 0.38 0.001598 0 9994 sendfile
- 0.35 0.001503 0 9997 getsockname
- 0.34 0.001439 0 9997 gettimeofday
- 0.00 0.000002 1 2 fstat
- 0.00 0.000001 1 1 lseek
- 0.00 0.000001 1 1 mmap
- 0.00 0.000001 1 1 munmap
- ------ ----------- ----------- --------- --------- ----------------
- 100.00 0.424835 375310 37725 total
长连接:
- % time seconds usecs/call calls errors syscall
- ------ ----------- ----------- --------- --------- ----------------
- 37.05 0.032997 3 9919 write
- 21.90 0.019503 2 9940 poll
- 10.38 0.009248 0 39676 setsockopt
- 7.86 0.007000 0 49595 9919 stat
- 7.46 0.006642 0 59514 9919 lstat
- 5.35 0.004764 0 49720 9941 read
- 3.54 0.003156 0 19839 open
- 2.27 0.002018 0 9919 sendfile
- 1.95 0.001735 0 19941 close
- 1.28 0.001143 0 9919 writev
- 0.92 0.000816 0 9921 gettimeofday
- 0.02 0.000014 0 200 fcntl
- 0.01 0.000007 0 100 accept
- 0.01 0.000007 0 100 getsockname
- 0.01 0.000006 0 100 1 shutdown
- 0.00 0.000002 1 2 fstat
- 0.00 0.000001 1 1 lseek
- 0.00 0.000001 1 1 mmap
- 0.00 0.000001 1 1 munmap
- ------ ----------- ----------- --------- --------- ----------------
- 100.00 0.089061 288408 29780 total
以上数据表明,长连接accept和shutdown次数仅为100次,而短连接为9997次,近100倍的差距,从这里就不难发现为什么长连接的TPS那么高了,省了这么多次系统调用,不快才怪啊。
本次试验得出验证来开始的理论分析:长连接比短连接能带来更高的TPS,更低的CPU消耗,更少的IO,更高的内存占用,更少的系统调用