基于Avocado 的自动化测试.md

qdata-mysql 自动化测试概要设计

│   ├── 1. 依赖环境
│   │   ├
│   │   └───── 
│   ├── 2. 配置文件
│   │   ├
│   │   └───── 
│   ├
│   ├── 3. 测试流程
│   │   ├
│   │   └───── 
│   ├
│   ├── 4. 输出格式
│   │   ├
│   │   └───── 
│   │ 
│   ├── 5. 命令行参数
│   │   ├
│   │   └─────

依赖环境

python2.7.9
avocado-framework (52.0)
ssh

avocado需要插件：

        avocado-framework-plugin-result-html (52.0)
        avocado-framework-plugin-runner-remote (52.0)
        avocado-framework-plugin-varianter-yaml-to-mux (52.0)

前提条件
- qdata_mysql 版本 1.2.2
- 集群正常
- sysbench 压测工具
- ntp 服务正常（ntp.conf 配置正常 pool.ntp.org）
- ssh 长连接
- master 配置数据库sbtest的权限
- 配置文件 qdata.conf, qdata_for_mysql.conf, qdata_conf_default, 需要备份一份，在恢复集群时，恢复配置文件，再gen_conf
```
grant all on sbtest.* to sbtest identified by 'sbtest';
flush privileges;
```

配置文件

用户配置
- yaml参数配置文件，配置集群master_ip
```
#cat data/cluster1.yaml
master_ip: '10.10.30.101'
```

测试流程

集群测试
- 支持多套集群测试
- 在data目录下新建参数配置文件，配置每套集群的master_ip,配置方法如下：
```
配置文件名称依照cluster1，cluster2，cluster3.....等以此类推
参数配置集群的master_ip即可，例如master_ip: '10.10.30.101'
```
指定第一套配置文件执行集群测试后，可新开窗口，再次指定其他配置文件进行测试
- 通过指定配置文件来进行集群测试：
  avocado run --mux-yaml data/cluster1.yaml -- cls.py ha.py lb.py db.py
- 若测试中断，可以从中断的案例开始执行（指定中断任务的job id）
  avocado run --replay 56b4aab --replay-test-status INTERRUPTED
- 支持只执行测试失败的案例,会跳过执行成功的案例（指定刚执行过的任务job id）
  avocado run --replay 56b4aab --replay-test-status FAIL
- 测试cancel的案例
  avocado run --replay 56b4aab --replay-test-status CANCEL
屏幕输出

	#avocado run --mux-yaml data/cluster2.yaml -- ha.py db.py lb.py
JOB ID     : eec9fe46c4aee730ee0e4433967244d7781e5c8e
JOB LOG    : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/job.log
 (1/9) ha.py:HATest.test_qmcls_start;run-7599: PASS (0.00 s)
 (2/9) ha.py:HATest.test_qmcls_stop;run-7599: FAIL (0.04 s)
 (3/9) ha.py:HATest.test_qmcls_show;run-7599: PASS (0.00 s)
 (4/9) db.py:DBTest.test_db_stop;run-7599: PASS (0.00 s)
 (5/9) db.py:DBTest.test_db_start;run-7599: FAIL (0.04 s)
 (6/9) db.py:DBTest.test_db_backup;run-7599: PASS (0.00 s)
 (7/9) lb.py:LBTest.test_lb_join;run-7599: PASS (0.00 s)
 (8/9) lb.py:LBTest.test_lb_kickout;run-7599: PASS (15.02 s)
 (9/9) lb.py:LBTest.test_set_kickout_time;run-7599: PASS (0.00 s)
RESULTS    : PASS 7 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 16.65 s
JOB HTML   : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/results.html

定位FAIL案例的具体失败信息
- 打开/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/results.json，根据id，2和5是FAIL，找到对应信息如下：

	    {
            "end": 1501139986.080578,
            "fail_reason": "qmcls stop failed in 10.10.30.102",
            "id": "2-ha.py:HATest.test_qmcls_stop;run-7599",
            "logdir": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/2-ha.py:HATest.test_qmcls_stop;run-7599",
            "logfile": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/2-ha.py:HATest.test_qmcls_stop;run-7599/debug.log",
            "start": 1501139986.039281,
            "status": "FAIL",
            "test": "2-ha.py:HATest.test_qmcls_stop;run-7599",
            "time": 0.041297197341918945,
            "url": "2-ha.py:HATest.test_qmcls_stop;run-7599",
            "whiteboard": ""
        },
        
       {
            "end": 1501139986.487331,
            "fail_reason": "qmcls db start all failed 10.10.30.102",
            "id": "5-db.py:DBTest.test_db_start;run-7599",
            "logdir": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/5-db.py:DBTest.test_db_start;run-7599",
            "logfile": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/5-db.py:DBTest.test_db_start;run-7599/debug.log",
            "start": 1501139986.447326,
            "status": "FAIL",
            "test": "5-db.py:DBTest.test_db_start;run-7599",
            "time": 0.04000496864318848,
            "url": "5-db.py:DBTest.test_db_start;run-7599",
            "whiteboard": ""
        },

输出格式

屏幕打印格式

		#avocado run --mux-yaml data/cluster2.yaml -- ha.py db.py lb.py
JOB ID     : eec9fe46c4aee730ee0e4433967244d7781e5c8e
JOB LOG    : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/job.log
 (1/9) ha.py:HATest.test_qmcls_start;run-7599: PASS (0.00 s)
 (2/9) ha.py:HATest.test_qmcls_stop;run-7599: FAIL (0.04 s)
 (3/9) ha.py:HATest.test_qmcls_show;run-7599: PASS (0.00 s)
 (4/9) db.py:DBTest.test_db_stop;run-7599: PASS (0.00 s)
 (5/9) db.py:DBTest.test_db_start;run-7599: FAIL (0.04 s)
 (6/9) db.py:DBTest.test_db_backup;run-7599: PASS (0.00 s)
 (7/9) lb.py:LBTest.test_lb_join;run-7599: PASS (0.00 s)
 (8/9) lb.py:LBTest.test_lb_kickout;run-7599: PASS (15.02 s)
 (9/9) lb.py:LBTest.test_set_kickout_time;run-7599: PASS (0.00 s)
RESULTS    : PASS 7 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 16.65 s
JOB HTML   : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/results.html

结果导出文件格式：
- 查看详细测试结果:
  /root/avocado/job-results/job-2017-07-25T09.14-56b4aab/results.json
- 查看日志信息：
  /root/avocado/job-results/job-2017-07-25T09.14-56b4aab/job.log
- HTML格式结果：
  /root/avocado/job-results/job-2017-07-26T14.49-020f341/results.html

命令行参数

avocado run／list / variants / config
- run 执行测试
- list 列出函数包含的测试案例
- variants 查看参数配置信息
- config 查看配置路径信息
avocado run --mux-yaml
- --mux-yaml 指定参数配置文件
执行命令可指定测试案例函数，即可执行测试案例，例如：
avocado run --mux-yaml data/cluster1.yaml -- ha.py lb.py db.py 或
avocado run ha.py lb.py db.py --mux-yaml data/cluster1.yaml
查看配置文件参数详细信息
avocado variants -m --mux-yaml data/cluster1.yaml --summary 2 --variants 2

查看日志和结果文件位置
avocado config --datadir

#avocado config --datadir
Config files read (in order):
/usr/local/pyenv/product/etc/avocado/avocado.conf
/usr/local/pyenv/product/etc/avocado/conf.d/gdb.conf
/root/.config/avocado/avocado.conf

Avocado replaces config dirs that can't be accessed
with sensible defaults. Please edit your local config
file to customize values

Avocado Data Directories:
base     /var/lib/avocado
tests    /usr/share/avocado/tests
data     /var/lib/avocado/data
logs     /root/avocado/job-results

avocado list nvmtest.py 列出具体的测试用例函数有哪些
avocado list -V/--verbose nvmtest.py 查看详细信息
or avocado list nvmtest.py -V/--verbose

#avocado list ha.py --verbose
Type         Test                          Tag(s)
INSTRUMENTED ha.py:HATest.test_qmcls_start
INSTRUMENTED ha.py:HATest.test_qmcls_stop
INSTRUMENTED ha.py:HATest.test_qmcls_show

TEST TYPES SUMMARY
==================
ACCESS_DENIED: 0
BROKEN_SYMLINK: 0
EXTERNAL: 0
INSTRUMENTED: 3
MISSING: 0
NOT_A_TEST: 0
SIMPLE: 0

posted on 2018-03-19 10:01 logicalsky 阅读(709) 评论(0) 编辑收藏举报

刷新页面返回顶部