HBase读写性能测试[YCSB]

1 YCSB工具介绍

YCSB全称“Yahoo！Cloud Serving Benchmark”，是雅虎开发的用来对云服务进行基础测试的工具，内部涵盖了常见的NoSQL数据库产品，如Cassandra、MongoDB、HBase、Redis等。在运行YCSB时，可以配置不同的workload和DB，也可以指定线程数、并发数等参数。YCSB不仅安装使用简单，测试报告详细，比较具有说服力。

2 YCSB工具使用

压力测试的步骤：

• 配置需要测试的数据库
• 选择数据库接口
• 选择workload
• 选择运行参数
• 加载数据（loading phase）
• 执行压力测试（transaction phase）

2.1 配置需要测试的数据库

这里使用YCSB工具对HBase读写能力测试，需要启动HBase Shell先创建一个usertable。

2.2 选择数据库接口

使用./bin/ycsb shell hbase12 -cp /etc/hbase/conf/ -p columnfamily=A测试HBase数据库接口层配置是否正确。连接成功后，可以对HBase表进行操作。

2.3 选择workload

Workload 定义了如何向数据库中加载测试数据，YCSB自带了6中压力测试场景，文件和Workload的对应关系如下：

Workload 文件	相应压力测试场景
workloada	50%的读和50%的写；
workloadb	Read mostly workload，95%的读和5%的写，该workload侧重于测试集群的读能力；
workloadc	Read only，100%只读;
workloadd	Read latest workload读最近更新，会插入数据，接着就读取这些新插入的数据;
workloade	Short ranges，小范围查询，不同于随机读，每个测试线程都会去scan一段数据，而不是单个纪录;
workloadf	Read-modiy-wirte，读改写，客户端读出一个记录，修改它并将被修改的记录返回;

2.4 选择运行参数

YCSB 支持以下运行参数：

• threads：客户端线程数，默认为 1
• target：每秒的目标操作数，默认为无限制（尽可能快地完成操作）。例如一个操作需要 100 ms，那么一个线程 1s 内可以完成 10 个操作，通过 -target 参数可以将操作放缓，控制在 10 个以下
• s：每 10s 打印一次客户端状态，用于调试

2.5 加载数据

Workload 包含两个阶段：装载阶段和事务阶段。在装载阶段向数据库中插入测试数据。
默认的A类型方案，默认是1000条记录数和1000条操作数，这里可以使用-p recordcount=100000 -p operationcount=100000来覆盖原有属性。加载加载1,000,000条记录测试。
参数解释如下:

• load 表示这是一次加载数据操作；
• hbase12指明了所用的数据库类型；
• -P指明了所用的配置文件的路径；
• -p 可以显示修改YCSB内置的默认配置，例如这里配置recordcount=10000来覆盖之前说过的workloada中默认的recordcount=1000；
• -s一次加载数据或执行测试的过程实践很长，YCSB客户端需要定时(默认10s)发送状态信息；> logs/load.log表示结果日志写入位置（若没有logs文件夹就自己新建一个，也可以不加此参数，则运行结果直接打印到屏幕上）；

(base) root@node2:/opt/YCSB/ycsb-hbase12-binding-0.14.0/bin/logs# sh ycsb.sh load hbase12 -P /opt/YCSB/ycsb-hbase12-binding-0.14.0/workloads/workloadb -threads 30 -p table=usertable -p columnfamily=family -p recordcount=100000 -s > logs/load.log

/opt/java/jdk1.8.0_181/bin/java  -classpath /opt/YCSB/ycsb-hbase12-binding-0.14.0/conf:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/HdrHistogram-2.1.4.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/commons-logging-1.2.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/core-0.14.0.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/findbugs-annotations-1.3.9-1.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/hbase-shaded-client-1.2.5.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/hbase10-binding-0.14.0.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/hbase12-binding-0.14.0.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/htrace-core-3.1.0-incubating.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/htrace-core4-4.1.0-incubating.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/jackson-core-asl-1.9.4.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/jackson-mapper-asl-1.9.4.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/log4j-1.2.17.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/slf4j-api-1.7.25.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/slf4j-log4j12-1.6.1.jar com.yahoo.ycsb.Client -load -db com.yahoo.ycsb.db.hbase12.HBaseClient12 -P /opt/YCSB/ycsb-hbase12-binding-0.14.0/workloads/workloadb -threads 30 -p table=usertable -p columnfamily=family -p recordcount=100000 -s
[OVERALL], RunTime(ms), 10588
[OVERALL], Throughput(ops/sec), 9444.654325651682
[TOTAL_GCS_PS_Scavenge], Count, 7
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 58
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.5477899508877975
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 7
[TOTAL_GC_TIME], Time(ms), 58
[TOTAL_GC_TIME_%], Time(%), 0.5477899508877975
[CLEANUP], Operations, 60
[CLEANUP], AverageLatency(us), 673.7333333333333
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 39679
[CLEANUP], 95thPercentileLatency(us), 38
[CLEANUP], 99thPercentileLatency(us), 375
[INSERT], Operations, 100000
[INSERT], AverageLatency(us), 2937.82403
[INSERT], MinLatency(us), 732
[INSERT], MaxLatency(us), 190591
[INSERT], 95thPercentileLatency(us), 4327
[INSERT], 99thPercentileLatency(us), 12687
[INSERT], Return=OK, 100000

2.6 执行压力测试

当装载完测试数据后，就可以运行 workload 测试。以workloada测试场景为例，切换到$ YCSB_HOME目录，执行以下命令进行压力测试。
参数解释如下:

• -threads 10配置了并发线程个数
• -p operationcount=1000000 覆盖了workloada中operationcount=1000表示本次测试操作记录数为1000000
• -p measurementtype=timeseries -p timeseries.granularity=2000指明了YCSB客户端多长时间汇总一次延时，timeseries.granularity属性值的单位为1000milliseconds。所以这里的配置就表示2000s记录一次本2000s的平均延时。

(base) root@node2:/opt/YCSB/ycsb-hbase12-binding-0.14.0/bin/logs# sh ycsb.sh run hbase12 -P /opt/YCSB/ycsb-hbase12-binding-0.14.0/workloads/workloada -threads 10 -p operationcount=1000000 -p table=usertable -p columnfamily=family -p measurementtype=timeseries -p timeseries.granularity=2000 -s > logs/transaction-workloadAA.log

/opt/java/jdk1.8.0_181/bin/java  -classpath /opt/YCSB/ycsb-hbase12-binding-0.14.0/conf:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/HdrHistogram-2.1.4.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/commons-logging-1.2.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/core-0.14.0.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/findbugs-annotations-1.3.9-1.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/hbase-shaded-client-1.2.5.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/hbase10-binding-0.14.0.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/hbase12-binding-0.14.0.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/htrace-core-3.1.0-incubating.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/htrace-core4-4.1.0-incubating.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/jackson-core-asl-1.9.4.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/jackson-mapper-asl-1.9.4.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/log4j-1.2.17.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/slf4j-api-1.7.25.jar:/opt/YCSB/ycsb-hbase12-binding-0.14.0/lib/slf4j-log4j12-1.6.1.jar com.yahoo.ycsb.Client -t -db com.yahoo.ycsb.db.hbase12.HBaseClient12 -P /opt/YCSB/ycsb-hbase12-binding-0.14.0/workloads/workloada -threads 10 -p operationcount=1000000 -p table=usertable -p columnfamily=family -p measurementtype=timeseries -p timeseries.granularity=2000 -s
2020-03-05 15:30:34:984 0 sec: 0 operations; est completion in 0 second
2020-03-05 15:30:44:958 10 sec: 83374 operations; 8337.4 current ops/sec; est completion in 1 minute [READ AverageLatency(us)=763.55] [UPDATE AverageLatency(us)=1446.53]
2020-03-05 15:30:54:958 20 sec: 181223 operations; 9784.9 current ops/sec; est completion in 1 minute [READ AverageLatency(us)=707.89] [UPDATE AverageLatency(us)=1332.18]
2020-03-05 15:31:04:958 30 sec: 274005 operations; 9278.2 current ops/sec; est completion in 1 minute [READ AverageLatency(us)=750.17] [UPDATE AverageLatency(us)=1400.65]
2020-03-05 15:31:14:958 40 sec: 367391 operations; 9338.6 current ops/sec; est completion in 1 minute [READ AverageLatency(us)=755.36] [UPDATE AverageLatency(us)=1382.87]
2020-03-05 15:31:24:958 50 sec: 455210 operations; 8781.9 current ops/sec; est completion in 1 minute [READ AverageLatency(us)=763.14] [UPDATE AverageLatency(us)=1507.92]
2020-03-05 15:31:34:958 60 sec: 548540 operations; 9333 current ops/sec; est completion in 50 seconds [READ AverageLatency(us)=760.83] [UPDATE AverageLatency(us)=1378.63]
2020-03-05 15:31:44:958 70 sec: 636895 operations; 8835.5 current ops/sec; est completion in 40 seconds [READ AverageLatency(us)=794.56] [UPDATE AverageLatency(us)=1466.25]
2020-03-05 15:31:54:958 80 sec: 731496 operations; 9460.1 current ops/sec; est completion in 30 seconds [READ AverageLatency(us)=739.29] [UPDATE AverageLatency(us)=1369.51]
2020-03-05 15:32:04:958 90 sec: 820120 operations; 8862.4 current ops/sec; est completion in 20 seconds [READ AverageLatency(us)=817.48] [UPDATE AverageLatency(us)=1433.91]
2020-03-05 15:32:14:958 100 sec: 907433 operations; 8731.3 current ops/sec; est completion in 11 seconds [READ AverageLatency(us)=819.79] [UPDATE AverageLatency(us)=1462.1]
2020-03-05 15:32:24:958 110 sec: 991899 operations; 8446.6 current ops/sec; est completion in 1 second [READ AverageLatency(us)=878.54] [UPDATE AverageLatency(us)=1486.24]
2020-03-05 15:32:26:018 111 sec: 1000000 operations; 7649.67 current ops/sec; [READ AverageLatency(us)=809.84] [CLEANUP AverageLatency(us)=2390.95] [UPDATE AverageLatency(us)=1346.01]
[OVERALL], RunTime(ms), 111059
[OVERALL], Throughput(ops/sec), 9004.222980577892
[TOTAL_GCS_PS_Scavenge], Count, 227
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 513
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.46191663890364587
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 227
[TOTAL_GC_TIME], Time(ms), 513
[TOTAL_GC_TIME_%], Time(%), 0.46191663890364587
[READ], Operations, 500160
[READ], AverageLatency(us), 776.0126479526551
[READ], MinLatency(us), 337
[READ], MaxLatency(us), 104082
[READ], Return=OK, 500160
[READ], 0, 1276.6995282251505
[READ], 2000, 649.2255352429368
[READ], 4000, 678.6151322534345
[READ], 6000, 689.1046949791221
[READ], 8000, 697.6631187351452
[READ], 10000, 696.6432106096595
[READ], 12000, 694.0623793558874
[READ], 14000, 704.836350834842
[READ], 16000, 728.077892325315
[READ], 18000, 720.3161686798051
[READ], 20000, 751.959930502769
[READ], 22000, 730.324644303666
[READ], 24000, 771.3616493916179
[READ], 26000, 765.4120380376054
[READ], 28000, 743.7830276655269
[READ], 30000, 735.0576027107159
[READ], 32000, 783.482898289829
[READ], 34000, 740.5353734439834
[READ], 36000, 775.5729718051205
[READ], 38000, 748.442555870318
[READ], 40000, 746.5480135967707
[READ], 42000, 857.5666501894875
[READ], 44000, 747.9915065293555
[READ], 46000, 746.8779106356199
[READ], 48000, 758.7511555412233
[READ], 50000, 749.19970890945
[READ], 52000, 771.9811688311688
[READ], 54000, 756.14262990456
[READ], 56000, 756.4904629232748
[READ], 58000, 767.1301332755445
[READ], 60000, 760.4330708661417
[READ], 62000, 757.5231950517223
[READ], 64000, 762.1128894579599
[READ], 66000, 805.3563124854685
[READ], 68000, 923.7562767094017
[READ], 70000, 774.668242178708
[READ], 72000, 731.8520678685047
[READ], 74000, 700.3442723196086
[READ], 76000, 730.2195071868583
[READ], 78000, 748.3515947864787
[READ], 80000, 776.6867721379916
[READ], 82000, 785.2448049783309
[READ], 84000, 908.4006651065403
[READ], 86000, 840.6130624426079
[READ], 88000, 820.8384752962218
[READ], 90000, 807.3594118307329
[READ], 92000, 841.4906301241508
[READ], 94000, 817.9093775545954
[READ], 96000, 809.1530436776338
[READ], 98000, 854.6974949542919
[READ], 100000, 882.0022943673282
[READ], 102000, 883.7747460429955
[READ], 104000, 872.9479387514723
[READ], 106000, 848.1641962541652
[READ], 108000, 871.6533301099963
[READ], 110000, 701.7737226277372
[CLEANUP], Operations, 20
[CLEANUP], AverageLatency(us), 2390.95
[CLEANUP], MinLatency(us), 2
[CLEANUP], MaxLatency(us), 43992
[CLEANUP], 0, 2390.95
[UPDATE], Operations, 499840
[UPDATE], AverageLatency(us), 1421.4756862195902
[UPDATE], MinLatency(us), 592
[UPDATE], MaxLatency(us), 413007
[UPDATE], Return=OK, 499840
[UPDATE], 0, 2082.2369496855345
[UPDATE], 2000, 1299.0737151248165
[UPDATE], 4000, 1362.0576982097186
[UPDATE], 6000, 1334.1392353716662
[UPDATE], 8000, 1343.1511391375102
[UPDATE], 10000, 1312.182779302633
[UPDATE], 12000, 1330.7014030093535
[UPDATE], 14000, 1311.3423240721963
[UPDATE], 16000, 1337.775634413039
[UPDATE], 18000, 1368.5339654094603
[UPDATE], 20000, 1422.6674296926096
[UPDATE], 22000, 1350.59671169756
[UPDATE], 24000, 1478.6751915277152
[UPDATE], 26000, 1401.5775307834804
[UPDATE], 28000, 1360.3254526091587
[UPDATE], 30000, 1356.0114643043253
[UPDATE], 32000, 1440.6150516150517
[UPDATE], 34000, 1351.6456377254985
[UPDATE], 36000, 1424.361296791444
[UPDATE], 38000, 1338.594216515294
[UPDATE], 40000, 1357.5026799789805
[UPDATE], 42000, 2399.6707872478855
[UPDATE], 44000, 1369.6318972835313
[UPDATE], 46000, 1366.3809067688378
[UPDATE], 48000, 1373.7106188092448
[UPDATE], 50000, 1344.1879746835443
[UPDATE], 52000, 1405.8942760205757
[UPDATE], 54000, 1380.139823199655
[UPDATE], 56000, 1374.5373547905788
[UPDATE], 58000, 1386.807153630683
[UPDATE], 60000, 1365.0507679180887
[UPDATE], 62000, 1372.8914645871168
[UPDATE], 64000, 1381.0190794437858
[UPDATE], 66000, 1541.132995043663
[UPDATE], 68000, 1751.9204194112112
[UPDATE], 70000, 1451.740411339633
[UPDATE], 72000, 1362.6300541892456
[UPDATE], 74000, 1314.8973506598168
[UPDATE], 76000, 1352.960918571579
[UPDATE], 78000, 1360.7175331509156
[UPDATE], 80000, 1363.995857233907
[UPDATE], 82000, 1411.9140256607084
[UPDATE], 84000, 1539.7952101661779
[UPDATE], 86000, 1482.3387342961137
[UPDATE], 88000, 1429.3741365643755
[UPDATE], 90000, 1414.1459603236174
[UPDATE], 92000, 1502.5730403102598
[UPDATE], 94000, 1532.652647790546
[UPDATE], 96000, 1407.9624577103568
[UPDATE], 98000, 1513.354842543078
[UPDATE], 100000, 1457.4831061055127
[UPDATE], 102000, 1466.6979178920128
[UPDATE], 104000, 1462.9405582155787
[UPDATE], 106000, 1444.2828305998394
[UPDATE], 108000, 1507.9255580083395
[UPDATE], 110000, 1040.9772727272727

3 各属性含义说明

属性	含义
[OVERALL], RunTime(ms)	数据加载所用时间(毫秒)
[OVERALL], Throughput(ops/sec)	加载操作的吞吐量(ops/sec)
[INSERT], Operations	执行insert操作的总数
[INSERT], AverageLatency(us)	每次insert操作的平均延时(微秒)
[INSERT], MinLatency(us)	每次insert操作的最小延时(微秒)
[INSERT], MaxLatency(us)	每次insert操作的最大延时(微秒)
[INSERT], 95thPercentileLatency(us)	95%的insert操作延时在多少微秒以内
[INSERT], 99thPercentileLatency(us)	99%的insert操作延时在多少微秒以内
[INSERT], Return=OK	操作成功返回数