iOS自动化探索（四）自动化测试框架pytest - 安装和使用
自动化测试框架 - pytest
pytest是Python最流行的单元测试框架之一, 帮助更便捷的编写测试脚本， 并支持多种功能复杂的测试场景， 能用来做app测试也能用作函数测试
官方文档: https://docs.pytest.org/en/latest/
pytest具有以下优点:
- 允许使用assert进行断言
- 自动识别测试脚本、类、函数
- 可用于管理小型或者参数类型的测试数据或资源
- 兼容unittest和nose测试框架
- 支持Python2.7/Python3.4+
- 丰富的插件支持，超过315个插件支持
pytest安装
pip install -U pytest
如果提示下面的错误，说明是pip的版本太老了， 要更新下：
Could not find a version that satisfies the requirement pytest (from versions: )
No matching distribution found for pytest
更新方式:
easy_install --upgrade pip
官方示例
准备一个test_sample.py, 内容如下：
def inc(x): return x + 1 def test_answer(): assert inc(3) == 5
在文件所在目录执行:
pytest
这里我们做下说明：
pytest脚本都以test_xxx.py为文件名；
inc方法是我们定义的一个自增函数，该函数将传递进来的参数加1后返回；
test_answer是我们编写的一个测试函数，其中我们使用基本的断言语句assert来对结果进行验证，测试函数以test_xxx作为命名
执行结果如下：
============================================================ test session starts ============================================================ platform darwin -- Python 2.7.15, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 rootdir: /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test, inifile: collected 1 item test_sample.py F [100%] ================================================================= FAILURES ================================================================== ________________________________________________________________ test_answer ________________________________________________________________ def test_answer(): > assert inc(3) == 5 E assert 4 == 5 E + where 4 = inc(3) test_sample.py:5: AssertionError ========================================================= 1 failed in 0.05 seconds ========================================================== (wda_python) bash-3.2$
当执行到assert inc(3) == 5时，报错
执行pytest会在当前目录和子目录中寻找test_xx.py的测试文件，并进入到测试文件中寻找test_xx开头的测试函数开始执行
执行pytest -q test_xxx.py是执行执行的脚本
在看一个例子,测试指定错误： (Assert that a certain exception is raised)
import pytest def f(): raise SystemExit(1) def test_mytest(): with pytest.raises(SystemExit): f()
执行指令：
pytest -q test_sysexit.py
输出:
(wda_python) bash-3.2$ pytest -q test_sysexit.py . [100%] 1 passed in 0.04 seconds (wda_python) bash-3.2$
如果要开发多个测试方法，可以把方法写进一个class中
class TestClass(object): def test_one(self): x = 'this' assert 'h' in x def test_two(self): x = 'hello' assert hasattr(x, 'check')
pytest能够自动识别类中的测试方法， 也不用我们去创建子类或者实实例, 运行结果如下：
(wda_python) bash-3.2$ pytest -q test_sample.py .F [100%] ================================================================== FAILURES ================================================================== _____________________________________________________________ TestClass.test_two _____________________________________________________________ self = <test_sample.TestClass object at 0x102e151d0> def test_two(self): x = 'hello' > assert hasattr(x, 'check') E AssertionError: assert False E + where False = hasattr('hello', 'check') test_sample.py:8: AssertionError 1 failed, 1 passed in 0.08 seconds (wda_python) bash-3.2$
除了直接在脚本路径执行pytest外， 还可以用以下方式
python -m pytest xxx.py
出现第一个（或第N个）错误时停止
pytest -x # stop after first failure pytest --maxfail=2 # stop after two failures
运行执行测试脚本
pytest test_mod.py
运行指定目录下的所有脚本
pytest testing/
运行包含指定关键字的测试方法, 可以是文件名、类名、测试函数名
pytest -k "MyClass and not method"
执行node id运行测试脚本，每一个被收集的测试方法都会分配一个指定的id， 我们可以用一下方式运行执行的测试方法:
# To run a specific test within a module pytest test_mod.py::test_func # To run a test within a class pytest test_mod.py::TestClass::test_method
日志打印的不同方式
pytest --showlocals # show local variables in tracebacks pytest -l # show local variables (shortcut) pytest --tb=auto # (default) 'long' tracebacks for the first and last # entry, but 'short' style for the other entries pytest --tb=long # exhaustive, informative traceback formatting pytest --tb=short # shorter traceback format pytest --tb=line # only one line per failure pytest --tb=native # Python standard library formatting pytest --tb=no # no traceback at all
测试报告
pytest默认是完整的测试报告， 我们可以加上-r标签显示简短测试报告，可再搭配一下参数
Here is the full list of available characters that can be used: f - failed E - error s - skipped x - xfailed X - xpassed p - passed P - passed with output a - all except pP
可以多个参数一起使用
Debug模式
pytest --pdb
示例:
(wda_python) bash-3.2$ pytest --pdb ========================================================== test session starts =========================================================== platform darwin -- Python 2.7.15, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 rootdir: /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test, inifile: collected 3 items test_sample.py .F >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> self = <test_sample.TestClass object at 0x10e928610> def test_two(self): x = 'hello' > assert hasattr(x, 'check') E AssertionError: assert False E + where False = hasattr('hello', 'check') test_sample.py:8: AssertionError >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test/test_sample.py(8)test_two() -> assert hasattr(x, 'check') (Pdb) print x hello (Pdb) print hasattr(x,'check') False (Pdb)
还可以指定第几次失败开始进入debug:
pytest -x --pdb # drop to PDB on first failure, then end test session pytest --pdb --maxfail=3 # drop to PDB for first three failures
任何失败的异常信息都会存储在sys.last_value，sys.last_type 以及 sys_last_traceback
在debug中可以通过以下方式获取最后报错的内容
(Pdb) import sys (Pdb) sys.last_traceback.tb_lineno 1357 (Pdb) sys.last_value AssertionError(u"assert False\n + where False = hasattr('hello', 'check')",) (Pdb)
在执行一开始就进入到debug模式
pytest --trace
输入next执行下一步, exit退出
脚本中设置断点
import pdb
pdb.set_trace()
例如：
import pdb class TestClass(object): def test_one(self): x = 'this' pdb.set_trace() assert 'h' in x def test_two(self): x = 'hello' assert hasattr(x, 'check')
获取执行最慢的n个测试步骤
pytest --durations=10
======================================================= slowest 10 test durations ======================================================== (0.00 durations hidden. Use -vv to show these durations.)
但如果所有脚本的运行时间都小于0.01s, 就不显示了， 除非带上-vv参数
pytest --durations=10 -vv
输出结果:
======================================================= slowest 10 test durations ======================================================== 0.00s call test_sample.py::TestClass::test_two 0.00s setup test_sysexit.py::test_mytest 0.00s setup test_sample.py::TestClass::test_two 0.00s setup test_sample.py::TestClass::test_one 0.00s teardown test_sample.py::TestClass::test_two 0.00s teardown test_sample.py::TestClass::test_one 0.00s call test_sysexit.py::test_mytest 0.00s teardown test_sysexit.py::test_mytest 0.00s call test_sample.py::TestClass::test_one =================================================== 1 failed, 2 passed in 0.06 seconds =================================================== (wda_python) bash-3.2$
将日志保存到指定文件
pytest --resultlog=path
Disabling plugins
To disable loading specific plugins at invocation time, use the -p option together with the prefix no:.
Example: to disable loading the plugin doctest, which is responsible for executing doctest tests from text files, invoke pytest like this:
pytest -p no:doctest
我们也可以在pytestdemo脚本中去启动pytest:
import pytest
pytest.main()
执行python pytestdemo.py就可以执行pytest
main()不会抛出SystemExit的异常, 但会返回exitcode, 一共有6种exitcode
Exit code 0: All tests were collected and passed successfully Exit code 1: Tests were collected and run but some of the tests failed Exit code 2: Test execution was interrupted by the user Exit code 3: Internal error happened while executing tests Exit code 4: pytest command line usage error Exit code 5: No tests were collected
我们试着加上打印
import pytest
print pytest.main()
输出：
(wda_python) bash-3.2$ python pytestDemo.py ========================================================== test session starts =========================================================== platform darwin -- Python 2.7.15, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 rootdir: /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test, inifile: collected 3 items test_sample.py .F [ 66%] test_sysexit.py . [100%] ================================================================ FAILURES ================================================================ ___________________________________________________________ TestClass.test_two ___________________________________________________________ self = <test_sample.TestClass object at 0x1038ba650> def test_two(self): x = 'hello' > assert hasattr(x, 'check') E AssertionError: assert False E + where False = hasattr('hello', 'check') test_sample.py:11: AssertionError =================================================== 1 failed, 2 passed in 0.05 seconds =================================================== 1 (wda_python) bash-3.2$
我们还可以在main中传递参数:
pytest.main(['-q','test_sample.py'])
给pytest.main添加plugin, 如下示例在执行的开头和结尾， 添加打印信息
import pytest
class MyPlugin(object):
def pytest_sessionfinish(self):
print '*** Test run reporting finishing'
def pytest_sessionstart(self):
print '*** Test run report beginning'
pytest.main(['-q','test_sample.py'], plugins=[MyPlugin()])
输出：
(wda_python) bash-3.2$ python pytestDemo.py
*** Test run report beginning
.F [100%]*** Test run reporting finishing
================================================================ FAILURES ================================================================
___________________________________________________________ TestClass.test_two ___________________________________________________________
self = <test_sample.TestClass object at 0x1090843d0>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_sample.py:11: AssertionError
1 failed, 1 passed in 0.05 seconds
