doctest 库介绍
doctest 库介绍
链接：http://pymotw.com/2/doctest/
doctest的目的： 通过模块的doc部分写自动化测试case
１. 开始
使用doctest的第一步是，在python的交互式解释器中写case，然后将其copy到模块的doc中。下面是my_function()的两个case介绍
# cat test1.py
def my_function(a, b):
'''
>>>my_function(2, 3)
6
>>>my_function('a', 3)
'aaa'
'''
return a * b
运行doctest，可以通过python -m doctest youcode.py
# python -m doctest test1.py
Traceback (most recent call last):
File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
exec code in run_globals
File "/usr/lib64/python2.6/doctest.py", line 2704, in <module>
sys.exit(_test())
File "/usr/lib64/python2.6/doctest.py", line 2693, in _test
failures, _ = testmod(m)
File "/usr/lib64/python2.6/doctest.py", line 1847, in testmod
for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
File "/usr/lib64/python2.6/doctest.py", line 864, in find
self._find(tests, obj, name, module, source_lines, globs, {})
File "/usr/lib64/python2.6/doctest.py", line 918, in _find
globs, seen)
File "/usr/lib64/python2.6/doctest.py", line 906, in _find
test = self._get_test(obj, name, module, globs, source_lines)
File "/usr/lib64/python2.6/doctest.py", line 990, in _get_test
filename, lineno)
File "/usr/lib64/python2.6/doctest.py", line 609, in get_doctest
return DocTest(self.get_examples(string, name), globs,
File "/usr/lib64/python2.6/doctest.py", line 623, in get_examples
return [x for x in self.parse(string, name)
File "/usr/lib64/python2.6/doctest.py", line 585, in parse
self._parse_example(m, name, lineno)
File "/usr/lib64/python2.6/doctest.py", line 643, in _parse_example
self._check_prompt_blank(source_lines, indent, name, lineno)
File "/usr/lib64/python2.6/doctest.py", line 730, in _check_prompt_blank
line[indent:indent+3], line))
ValueError: line 2 of the docstring for test1.my_function lacks blank after >>>: '>>>my_function(2, 3)'
扫噶 报错了，这个是因为>>>的后面需要有个空格，否则就会报错。因为在python交互式中都会有一个空格的，所以case最好还是在交互式写完之后copy过去。好，修改一下，再来一次。
# python -m doctest test1.py
什么也没有输出，说明所有的case，都通过，如果要看详细的信息，可以加-v。
# python -m doctest -v test1.py
Trying:
my_function(2, 3)
Expecting:
6
ok
Trying:
my_function('a', 3)
Expecting:
'aaa'
ok
1 items had no tests:
test1
1 items passed all tests:
2 tests in test1.my_function
2 tests in 2 items.
2 passed and 0 failed.
Test passed.
可以清晰的看到，两个case都通过了。
当然，只有case代码是不行的，还需要说明文档，说明文档只要和case保持一个空行开外即可，这样就会被doctest自动过滤的。如下：
# cat test1.py
def my_function(a, b):
'''
Returns a * b
Works with numbers:
>>> my_function(2, 3)
6
Works with strings:
>>> my_function('a', 3)
'aaa'
'''
return a * b
最后的结果还是一样的：
# python -m doctest -v test1.py
Trying:
my_function(2, 3)
Expecting:
6
ok
Trying:
my_function('a', 3)
Expecting:
'aaa'
ok
1 items had no tests:
test1
1 items passed all tests:
2 tests in test1.my_function
2 tests in 2 items.
2 passed and 0 failed.
Test passed.
2. 处理不确定的输出
有些情况下，输出不是确定，没有明确的输出，但是又不能不测。例如本地时间，时间变量，或者对象的ID，这些每次测试都会变化，还有浮点数的默认精确度是基于编译器的。虽然这些东西对于你来说是失控的，实际上有一种技术可以来对付他们。
在Cpython中，对象描述符是基于拥有对象的数据结构的内存地址。例如：
class MyClass(object):
pass
def unpredictable(obj):
"""Returns a new list containing obj.
>>> unpredictable(MyClass())
[<doctest_unpredictable.MyClass object at 0x10055a2d0>]
"""
return [obj]
这个ID values 没运行一次程序都会变化一次，因为程序载入了不同的内存模块。
$ python -m doctest -v /tmp/doctest_unpredictable.py
Trying:
unpredictable(MyClass())
Expecting:
[<doctest_unpredictable.MyClass object at 0x10055a2d0>]
**********************************************************************
File "/tmp/doctest_unpredictable.py", line 7, in doctest_unpredictable.unpredictable
Failed example:
unpredictable(MyClass())
Expected:
[<doctest_unpredictable.MyClass object at 0x10055a2d0>]
Got:
[<doctest_unpredictable.MyClass object at 0x1855c90>]
2 items had no tests:
doctest_unpredictable
doctest_unpredictable.MyClass
**********************************************************************
1 items had failures:
1 of 1 in doctest_unpredictable.unpredictable
1 tests in 3 items.
0 passed and 1 failed.
***Test Failed*** 1 failures.
在写测试的过程中，如果碰到值不确定，同时测试对象对测试结果不知很重要的时候，可以通过ELLIPSIS（省略号的意思）选项来过滤，告诉doctest忽略。
class MyClass(object):
pass
def unpredictable(obj):
"""Returns a new list containing obj.
>>> unpredictable(MyClass()) #doctest: +ELLIPSIS
[<doctest_unpredictable.MyClass object at 0x...>]
"""
return [obj]
使用方式是：
在command后面加comment, #doctest: +ELLIPSIS
在结果中，变化值的地方使用 ...
运行结果如下：
$ python -m doctest -v /tmp/doctest_unpredictable.py
Trying:
unpredictable(MyClass()) #doctest: +ELLIPSIS
Expecting:
[<doctest_unpredictable.MyClass object at 0x...>]
ok
2 items had no tests:
doctest_unpredictable
doctest_unpredictable.MyClass
1 items passed all tests:
1 tests in doctest_unpredictable.unpredictable
1 tests in 3 items.
1 passed and 0 failed.
Test passed.
有些情况下，你将不得不去测试变化的值。例如，在处理字符表示会不一致的数据类型时，一些简单的case会瞬间变的很复杂。字典的字符串表示（string representations）会随着key的加入而不一样。例如：
keys = [ 'a', 'aa', 'aaa' ]
d1 = dict( (k,len(k)) for k in keys )
d2 = dict( (k,len(k)) for k in reversed(keys) )
print
print 'd1:', d1
print 'd2:', d2
print 'd1 == d2:', d1 == d2
s1 = set(keys)
s2 = set(reversed(keys))
print
print 's1:', s1
print 's2:', s2
print 's1 == s2:', s1 == s2
由于cache collision，两个字典的内部key的顺序是不一样的，尽管他们包含相同的key，并且最后是相等的，集合使用的是相同的hash算法，最后获得了相同的结果。
$ python doctest_hashed_values.py
d1: {'a': 1, 'aa': 2, 'aaa': 3}
d2: {'aa': 2, 'a': 1, 'aaa': 3}
d1 == d2: True
s1: set(['a', 'aa', 'aaa'])
s2: set(['aa', 'a', 'aaa'])
s1 == s2: True
实际上处理这些潜在不同的最好方式就是使用不常变化的值。在上面集合和字典的case中，可能要分别寻找特定keys，产生一个有序list。
3. 回溯
回溯是改变数据中的一种特殊case。因为回溯中的路径是基于模块安装时的路径的。所以会很难写通配的case。
def this_raises():
"""This function always raises an exception.
>>> this_raises()
Traceback (most recent call last):
File "/tmp/test.py", line 13, in <module>
this_raises()
File "/tmp/test.py", line 11, in this_raises
raise RuntimeError('here is the error')
RuntimeError: here is the error
"""
raise RuntimeError('here is the error')
运行一下：
Trying:
this_raises()
Expecting:
Traceback (most recent call last):
File "/tmp/test.py", line 13, in <module>
this_raises()
File "/tmp/test.py", line 11, in this_raises
raise RuntimeError('here is the error')
RuntimeError: here is the error
ok
1 items had no tests:
test
1 items passed all tests:
1 tests in test.this_raises
1 tests in 2 items.
1 passed and 0 failed.
Test passed.
其实在doctest中，会自动识别回溯（Traceback）,然后过滤掉会随着系统变化而变化的东西。不会去检查。例如上面的代码修改为如下：
def this_raises():
"""This function always raises an exception.
>>> this_raises()
Traceback (most recent call last):
RuntimeError: here is the error
"""
raise RuntimeError('here is the error')
最后运行出来的结果和上面的是一样的。
4. 处理空格
在实际写代码的过程中，输出常常包含了空格。特别是空行，会导致doctest出问题。
def double_space(lines):
"""Prints a list of lines double-spaced.
>>> double_space(['Line one.', 'Line two.'])
Line one.
Line two.
"""
for l in lines:
print l
print
return
运行一下
# python -m doctest -v /tmp/test.py
Trying:
double_space(['Line one.', 'Line two.'])
Expecting:
Line one.
**********************************************************************
File "/tmp/test.py", line 4, in test.double_space
Failed example:
double_space(['Line one.', 'Line two.'])
Expected:
Line one.
Got:
Line one.
<BLANKLINE>
Line two.
<BLANKLINE>
1 items had no tests:
test
**********************************************************************
1 items had failures:
1 of 1 in test.double_space
1 tests in 2 items.
0 passed and 1 failed.
***Test Failed*** 1 failures.
这个case失败的原因是在doctest中，空行会被认为是一个case的结束，难道没辙了吗，不是，可以通过<BLACKLINE>来代替空行，这样就不会认为是case的结束。
# cat /tmp/test.py
def double_space(lines):
"""Prints a list of lines double-spaced.
>>> double_space(['Line one.', 'Line two.'])
Line one.
<BLACKLINE>
Line two.
<BLACKLINE>
"""
for l in lines:
print l
print
return
运行一下：
# python -m doctest -v /tmp/test.py
Trying:
double_space(['Line one.', 'Line two.'])
Expecting:
Line one.
<BLACKLINE>
Line two.
<BLACKLINE>
**********************************************************************
File "/tmp/test.py", line 4, in test.double_space
Failed example:
double_space(['Line one.', 'Line two.'])
Expected:
Line one.
<BLACKLINE>
Line two.
<BLACKLINE>
Got:
Line one.
<BLANKLINE>
Line two.
<BLANKLINE>
1 items had no tests:
test
**********************************************************************
1 items had failures:
1 of 1 in test.double_space
1 tests in 2 items.
0 passed and 1 failed.
***Test Failed*** 1 failures.
运行错误了，这么大一坨，怎么看出来错误呢，有没有可以指示出不同点的地方。当然有了，在case后面加上 #doctest: +REPORT_NDIFF,即可，我们来试试。
# python -m doctest -v /tmp/test.py
Trying:
double_space(['Line one.', 'Line two.']) #doctest: +REPORT_NDIFF
Expecting:
Line one.
<BLACKLINE>
Line two.
<BLACKLINE>
**********************************************************************
File "/tmp/test.py", line 4, in test.double_space
Failed example:
double_space(['Line one.', 'Line two.']) #doctest: +REPORT_NDIFF
Differences (ndiff with -expected +actual):
Line one.
- <BLACKLINE>
? ^
+ <BLANKLINE>
? ^
Line two.
- <BLACKLINE>
? ^
+ <BLANKLINE>
? ^
1 items had no tests:
test
**********************************************************************
1 items had failures:
1 of 1 in test.double_space
1 tests in 2 items.
0 passed and 1 failed.
***Test Failed*** 1 failures.
这下可以清晰的看到，是错误的问题所在，原来是有一个字母写错了，修改过来即可。
再写一个case，如下
def my_function(a, b):
"""Returns a * b.
>>> my_function(['A', 'B', 'C'], 3) #doctest: +NORMALIZE_WHITESPACE
['A', 'B', 'C',
'A', 'B', 'C',
'A', 'B', 'C']
This does not match because of the extra space after the [ in the list
>>> my_function(['A', 'B', 'C'], 2) #doctest: +NORMALIZE_WHITESPACE
[ 'A', 'B', 'C',
'A', 'B', 'C' ]
"""
return a * b
上面的NORMALIZE_WHITESPAC会把空格当做必须要匹配的东西。即当做正常的字符。运行一下；
# python -m doctest -v /tmp/test.py
Trying:
my_function(['A', 'B', 'C'], 3) #doctest: +NORMALIZE_WHITESPACE
Expecting:
['A', 'B', 'C',
'A', 'B', 'C',
'A', 'B', 'C']
ok
Trying:
my_function(['A', 'B', 'C'], 2) #doctest: +NORMALIZE_WHITESPACE
Expecting:
[ 'A', 'B', 'C',
'A', 'B', 'C' ]
**********************************************************************
File "/tmp/test.py", line 11, in test.my_function
Failed example:
my_function(['A', 'B', 'C'], 2) #doctest: +NORMALIZE_WHITESPACE
Expected:
[ 'A', 'B', 'C',
'A', 'B', 'C' ]
Got:
['A', 'B', 'C', 'A', 'B', 'C']
1 items had no tests:
test
**********************************************************************
1 items had failures:
1 of 2 in test.my_function
2 tests in 2 items.
1 passed and 1 failed.
***Test Failed*** 1 failures.
为了能够看到详细的不一样的，我们再加上+REPORT_NDIFF参数。
# cat /tmp/test.py
def my_function(a, b):
"""Returns a * b.
>>> my_function(['A', 'B', 'C'], 3) #doctest: +NORMALIZE_WHITESPACE +REPORT_NDIFF
['A', 'B', 'C',
'A', 'B', 'C',
'A', 'B', 'C']
This does not match because of the extra space after the [ in the list
>>> my_function(['A', 'B', 'C'], 2) #doctest: +NORMALIZE_WHITESPACE +REPORT_NDIFF
[ 'A', 'B', 'C',
'A', 'B', 'C' ]
"""
return a * b
结果如下：
# python -m doctest -v /tmp/test.py
Trying:
my_function(['A', 'B', 'C'], 3) #doctest: +NORMALIZE_WHITESPACE +REPORT_NDIFF
Expecting:
['A', 'B', 'C',
'A', 'B', 'C',
'A', 'B', 'C']
ok
Trying:
my_function(['A', 'B', 'C'], 2) #doctest: +NORMALIZE_WHITESPACE +REPORT_NDIFF
Expecting:
[ 'A', 'B', 'C',
'A', 'B', 'C' ]
**********************************************************************
File "/tmp/test.py", line 11, in test.my_function
Failed example:
my_function(['A', 'B', 'C'], 2) #doctest: +NORMALIZE_WHITESPACE +REPORT_NDIFF
Differences (ndiff with -expected +actual):
+ ['A', 'B', 'C', 'A', 'B', 'C']
- [ 'A', 'B', 'C',
- 'A', 'B', 'C' ]
1 items had no tests:
test
**********************************************************************
1 items had failures:
1 of 2 in test.my_function
2 tests in 2 items.
1 passed and 1 failed.
***Test Failed*** 1 failures.
5. 测试位置
上面的case都是写在函数的docstring的，实际上也可以写在其他地方，那我们最开始的例子，
# cat /tmp/test.py
"""
>>> my_function(2, 3)
6
>>> my_function('a', 3)
'aaa'
"""
def my_function(a, b):
return a * b
还可以文件中，写在另外一个文件中
# cat test_case.py
from test import my_function
__test__ = {
'test1': """
>>> my_function(2, 3)
6
>>> my_function('a', 3)
'aaa'
"""
}
需要载入响应的函数，同时要定义__test__
6. 在说明文档中写case
出了将case放在代码中以外，还可以将case放在module的说明文档中
# cat doctest_in_help.rst
w to Use test.py
===============================
This library is very simple, since it only has one function called
``my_function()``.
Numbers
=======
``my_function()`` returns the product of its arguments. For numbers,
that value is equivalent to using the ``*`` operator.
::
>>> from test import my_function
>>> my_function(2, 3)
6
It also works with floating point values.
::
>>> my_function(2.0, 3)
6.0
Non-Numbers
===========
Because ``*`` is also defined on data types other than numbers,
``my_function()`` works just as well if one of the arguments is a
string, list, or tuple.
::
>>> my_function('a', 3)
'aaa'
>>> my_function(['A', 'B', 'C'], 2)
['A', 'B', 'C', 'A', 'B', 'C']
运行方式：
# python -m doctest -v doctest_in_help.rst
Trying:
from test import my_function
Expecting nothing
ok
Trying:
my_function(2, 3)
Expecting:
6
ok
Trying:
my_function(2.0, 3)
Expecting:
6.0
ok
Trying:
my_function('a', 3)
Expecting:
'aaa'
ok
Trying:
my_function(['A', 'B', 'C'], 2)
Expecting:
['A', 'B', 'C', 'A', 'B', 'C']
ok
1 items passed all tests:
5 tests in doctest_in_help.rst
5 tests in 1 items.
5 passed and 0 failed.
Test passed.
7. 运行case
（1） 通过module
# cat test.py
def my_function(a, b):
"""
>>> my_function(2, 3)
6
>>> my_function('a', 3)
'aaa'
"""
return a * b
if __name__ == '__main__':
import doctest
doctest.testmod()
运行方式：
# python test.py # 没有详情
# python test.py -v # 有详情
（2） 通过文件
import doctest
if __name__ == '__main__':
doctest.testfile('doctest_in_help.rst')
运行：
# python test_case.py
# python test_case.py -v
（3）单元测试
If you use both unittest and doctest for testing the same code in different situations, you may find the unittest integration in doctest useful for running the tests together. Two classes, DocTestSuite and DocFileSuite create test suites compatible with the test-runner API of unittest.
import doctest
import unittest
import doctest_simple
suite = unittest.TestSuite()
suite.addTest(doctest.DocTestSuite(doctest_simple))
suite.addTest(doctest.DocFileSuite('doctest_in_help.rst'))
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)
The tests from each source are collapsed into a single outcome, instead of being reported individually.
$ python doctest_unittest.py
my_function (doctest_simple)
Doctest: doctest_simple.my_function ... ok
doctest_in_help.rst
Doctest: doctest_in_help.rst ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.003s
OK
