服务器性能测试利器之sysbench

前言

sysbench是一个开源的、模块化的、跨平台的多线程性能测试工具，可以用来进行CPU、内存、磁盘I/O、线程、数据库的性能测试。sysbench是基于LuaJIT的可编写脚本的多线程基准测试工具。

sysbench附带以下捆绑的基准测试：

oltp_*.lua：类似OLTP的数据库基准测试的集合
fileio：文件系统级基准
cpu：简单的CPU基准测试
memory：内存访问基准
threads：基于线程的调度程序基准
mutex：POSIX互斥基准测试

特征

提供有关速率和延迟的大量统计信息，包括延迟百分位数和直方图；
即使有数千个并发线程，开销也很低。sysbench每秒能够生成和跟踪数亿个事件。
通过在用户提供的Lua脚本中实现预定义的挂钩，可以轻松创建新的基准测试；
可以也可以作为一个通用的Lua解释，只需更换#!/usr/bin/lua用#!/usr/bin/sysbench在你的脚本。

安装

安装以Centos为例，其他版本或者mac请参考sysbench说明

1.下载安转

wget https://github.com/akopytov/sysbench/archive/1.0.zip -O "sysbench-1.0.zip"
  
unzip sysbench-1.0.zip
 
 
cd sysbench-1.0

//安装依赖
yum install -y automake libtool

2.编译安装

./autogen.sh
./configure
export LD_LIBRARY_PATH=/usr/include/mysql
make
make install

3.监测是否安装成功

[root@guanbin-k8s-master ~]# sysbench --version
sysbench 1.0.19

测试

1. cpu测试

cpu测试主要是进行素数的加法运算，我们的例子中，指定了最大的质数发生器数量为 20000

[root@guanbin-k8s-master ~]# sysbench --test=cpu --cpu-max-prime=20000 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Prime numbers limit: 20000

Initializing worker threads...

Threads started!

CPU speed:
    events per second:   168.59

General statistics:
    total time:                          10.0027s
    total number of events:              1687

Latency (ms):
         min:                                    5.63
         avg:                                    5.93
         max:                                   66.09
         95th percentile:                        6.43
         sum:                                 9999.10

Threads fairness:
    events (avg/stddev):           1687.0000/0.00
    execution time (avg/stddev):   9.9991/0.00

执行时间大约为10秒（这个测试最好是两台机器进行对比测试，才能发现那个机器有问题）

2.对内存（memory）进行测试

测试是在内存中传输 10G 的数据量，每个 block 大小为 8K

[root@guanbin-k8s-master ~]# sysbench --test=memory --memory-block-size=8k --memory-total-size=10G run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time


Running memory speed test with the following options:
  block size: 8KiB
  total size: 10240MiB
  operation: write
  scope: global

Initializing worker threads...

Threads started!

Total operations: 1310720 (724931.70 per second)

10240.00 MiB transferred (5663.53 MiB/sec)


General statistics:
    total time:                          1.8045s
    total number of events:              1310720

Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                   12.44
         95th percentile:                        0.00
         sum:                                 1441.61

Threads fairness:
    events (avg/stddev):           1310720.0000/0.00
    execution time (avg/stddev):   1.4416/0.00

totaltime为1.8045s

3.对disk 的IO测试

新建一个临时目录，并确保磁盘剩余空间足够（--file-total-size不要超过剩余容量）

测试场景：16个线程，创建128个文件，总共占用磁盘空间10GB，测试磁盘读写效率

运行结束后要删除临时测试文件： rm -rf test_file.*

root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw prepare
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)

128 files, 8192Kb each, 1024Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
Creating file test_file.3
Creating file test_file.4
Creating file test_file.5
Creating file test_file.6
Creating file test_file.7
Creating file test_file.8
Creating file test_file.9
Creating file test_file.10
Creating file test_file.11
Creating file test_file.12
Creating file test_file.13
Creating file test_file.14
Creating file test_file.15
Creating file test_file.16
Creating file test_file.17
Creating file test_file.18
Creating file test_file.19
Creating file test_file.20
Creating file test_file.21
Creating file test_file.22
Creating file test_file.23
Creating file test_file.24
Creating file test_file.25
Creating file test_file.26
Creating file test_file.27
Creating file test_file.28
Creating file test_file.29
Creating file test_file.30
Creating file test_file.31
Creating file test_file.32
Creating file test_file.33
Creating file test_file.34
Creating file test_file.35
Creating file test_file.36
Creating file test_file.37
Creating file test_file.38
Creating file test_file.39
Creating file test_file.40
Creating file test_file.41
Creating file test_file.42
Creating file test_file.43
Creating file test_file.44
Creating file test_file.45
Creating file test_file.46
Creating file test_file.47
Creating file test_file.48
Creating file test_file.49
Creating file test_file.50
Creating file test_file.51
Creating file test_file.52
Creating file test_file.53
Creating file test_file.54
Creating file test_file.55
Creating file test_file.56
Creating file test_file.57
Creating file test_file.58
Creating file test_file.59
Creating file test_file.60
Creating file test_file.61
Creating file test_file.62
Creating file test_file.63
Creating file test_file.64
Creating file test_file.65
Creating file test_file.66
Creating file test_file.67
Creating file test_file.68
Creating file test_file.69
Creating file test_file.70
Creating file test_file.71
Creating file test_file.72
Creating file test_file.73
Creating file test_file.74
Creating file test_file.75
Creating file test_file.76
Creating file test_file.77
Creating file test_file.78
Creating file test_file.79
Creating file test_file.80
Creating file test_file.81
Creating file test_file.82
Creating file test_file.83
Creating file test_file.84
Creating file test_file.85
Creating file test_file.86
Creating file test_file.87
Creating file test_file.88
Creating file test_file.89
Creating file test_file.90
Creating file test_file.91
Creating file test_file.92
Creating file test_file.93
Creating file test_file.94
Creating file test_file.95
Creating file test_file.96
Creating file test_file.97
Creating file test_file.98
Creating file test_file.99
Creating file test_file.100
Creating file test_file.101
Creating file test_file.102
Creating file test_file.103
Creating file test_file.104
Creating file test_file.105
Creating file test_file.106
Creating file test_file.107
Creating file test_file.108
Creating file test_file.109
Creating file test_file.110
Creating file test_file.111
Creating file test_file.112
Creating file test_file.113
Creating file test_file.114
Creating file test_file.115
Creating file test_file.116
Creating file test_file.117
Creating file test_file.118
Creating file test_file.119
Creating file test_file.120
Creating file test_file.121
Creating file test_file.122
Creating file test_file.123
Creating file test_file.124
Creating file test_file.125
Creating file test_file.126
Creating file test_file.127
1073741824 bytes written in 110.77 seconds (9.24 MiB/sec).
[root@guanbin-k8s-master tmp]# sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 16
Initializing random number generator from current time


Extra file open flags: (none)
128 files, 8MiB each
1GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...

Threads started!


File operations:
    reads/s:                      157.44
    writes/s:                     104.96
    fsyncs/s:                     480.79

Throughput:
    read, MiB/s:                  2.46
    written, MiB/s:               1.64

General statistics:
    total time:                          13.3355s
    total number of events:              7865

Latency (ms):
         min:                                    0.00
         avg:                                   20.61
         max:                                 1115.85
         95th percentile:                      142.39
         sum:                               162071.40

Threads fairness:
    events (avg/stddev):           491.5625/141.20
    execution time (avg/stddev):   10.1295/0.03

吞吐量：读2.46MiB/s，写1.64MiB/s; 95%延迟 142.39ms

4.测试mysql性能

测试时使用的脚本为lua脚本，可以使用sysbench自带脚本，也可以自己开发。对于大多数应用，使用sysbench自带的脚本就足够了。不同版本的sysbench中，lua脚本的位置可能不同，可以自己在sysbench路径下使用find命令搜索oltp.lua。

P.S.：大多数数据服务都是oltp类型的，如果你不了解什么是oltp，那么大概率你的数据服务就是oltp类型的。MySQL OLTP（On-Line Transaction Processing联机事务处理过程）；

（1）尽量不要在MySQL服务器运行的机器上进行测试，一方面可能无法体现网络（哪怕是局域网）的影响，另一方面，sysbench的运行（尤其是设置的并发数较高时）会影响MySQL服务器的表现。
（2）可以逐步增加客户端的并发连接数（--thread参数），观察在连接数不同情况下，MySQL服务器的表现；如分别设置为10,20,50,100等。
（3）一般执行模式选择complex即可，如果需要特别测试服务器只读性能，或不使用事务时的性能，可以选择simple模式或nontrx模式。
（4）如果连续进行多次测试，注意确保之前测试的数据已经被清理干净。

下面是sysbench使用的一个例子：

注意：先准备数据库

   mysql> create database sbtest;

其中，执行模式为complex，使用了10个表，每个表有10万条数据，客户端的并发线程数为10，执行时间为120秒，每10秒生成一次报告。

sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --mysql-table-engine=innodb --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 prepare

其中的hostip、username、password请自行切换为自己真实的环境数据

查看执行结果

sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-test-mode=complex --oltp-tables-count=10 --oltp-table-size=100000 --threads=10 --time=120 --report-interval=10 run

Running the test with following options:
Number of threads: 10
Report intermediate results every 10 second(s)
Initializing random number generator from current time


Initializing worker threads...

Threads started!

[ 10s ] thds: 10 tps: 17.19 qps: 362.78 (r/w/o: 254.64/72.76/35.38) lat (ms,95%): 1589.90 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 10 tps: 33.50 qps: 670.08 (r/w/o: 469.06/134.02/67.01) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 10 tps: 39.30 qps: 786.00 (r/w/o: 550.20/157.20/78.60) lat (ms,95%): 458.96 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 10 tps: 38.80 qps: 775.45 (r/w/o: 543.16/154.69/77.59) lat (ms,95%): 634.66 err/s: 0.00 reconn/s: 0.00
[ 50s ] thds: 10 tps: 30.10 qps: 602.54 (r/w/o: 421.43/120.91/60.20) lat (ms,95%): 590.56 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 10 tps: 33.90 qps: 678.00 (r/w/o: 474.60/135.60/67.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00
[ 70s ] thds: 10 tps: 29.50 qps: 589.98 (r/w/o: 412.99/118.00/59.00) lat (ms,95%): 787.74 err/s: 0.00 reconn/s: 0.00
[ 80s ] thds: 10 tps: 39.90 qps: 798.02 (r/w/o: 558.62/159.60/79.80) lat (ms,95%): 569.67 err/s: 0.00 reconn/s: 0.00
[ 90s ] thds: 10 tps: 38.10 qps: 761.99 (r/w/o: 533.40/152.40/76.20) lat (ms,95%): 484.44 err/s: 0.00 reconn/s: 0.00
[ 100s ] thds: 10 tps: 43.90 qps: 877.99 (r/w/o: 614.60/175.60/87.80) lat (ms,95%): 383.33 err/s: 0.00 reconn/s: 0.00
[ 110s ] thds: 10 tps: 37.80 qps: 756.00 (r/w/o: 529.20/151.20/75.60) lat (ms,95%): 530.08 err/s: 0.00 reconn/s: 0.00
[ 120s ] thds: 10 tps: 44.40 qps: 888.01 (r/w/o: 621.60/177.60/88.80) lat (ms,95%): 419.45 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            59836
        write:                           17096
        other:                           8548
        total:                           85480
    transactions:                        4274   (35.59 per sec.)
    queries:                             85480  (711.77 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          120.0919s
    total number of events:              4274

Latency (ms):
         min:                                   50.07
         avg:                                  280.96
         max:                                 2486.17
         95th percentile:                      559.50
         sum:                              1200810.10

Threads fairness:
    events (avg/stddev):           427.4000/4.20
    execution time (avg/stddev):   120.0810/0.01

其中，对于我们比较重要的信息包括：

queries：查询总数及qps

transactions：事务总数及tps

Latency-95th percentile：前95%的请求的最大响应时间，本例中是344毫秒，这个延迟非常大，是因为我用的MySQL服务器性能很差；在正式环境中这个数值是绝对不能接受的。

最后清理数据：

sysbench ./tests/include/oltp_legacy/oltp.lua --mysql-host={yourhostip} --mysql-port=3306 --mysql-user={username} --mysql-password={password} --oltp-tables-count=10  cleanup

清理的日志输出

sysbench 1.0.19 (using bundled LuaJIT 2.1.0-beta2)

Dropping table 'sbtest1'...
Dropping table 'sbtest2'...
Dropping table 'sbtest3'...
Dropping table 'sbtest4'...
Dropping table 'sbtest5'...
Dropping table 'sbtest6'...
Dropping table 'sbtest7'...
Dropping table 'sbtest8'...
Dropping table 'sbtest9'...
Dropping table 'sbtest10'...

posted @ 2020-04-24 00:44 纵码万水千山阅读(970) 评论(0) 编辑收藏举报

刷新页面返回顶部

服务器性能测试利器之sysbench

前言

特征

安装

测试

公告