python testing fundamentals
DESCRIPTION
Slides from PyOhio, a brief tour of unittest and doctest.TRANSCRIPT
magePython Testing Fundamentals
Saturday, July 28, 2012
PyOhio
Ohio State University
Columbus, OH
Chris Calloway
University of North Carolina
Department of Marine Sciences
Python Testing Fundamentals
PyCamp™ Programming For The People
Copyright © 2012
Trizpug
mage
http://drunkenpython.org/pytestfund.pdf
http://drunkenpython.org/pyohio.zip
http://drunkenpython.org/pyohio.tgz
Python Testing Fundamentals
PyCamp™ Programming For The People
Copyright © 2012
Trizpug
•Fundamental
•Python 3.2.3
•Assertion
•Unittest
•Doctest
Programming For The People
About This Tutorial
Copyright © 2012
TrizpugPyCamp™
“Untested Code is Broken Code”
- Phillip von Weitershausen
Programming For The People
Why Test?
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Why Test?
•Tests help you design good code
•Test help you find bugs
•Tests help you document code
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
•Assert statements use the assert keyword
•Assert statements raise AssertionError
•Based on bool test expressions
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
>>> assert 1 == 1
keyword
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
>>> assert 1 == 1
expression
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
>>> assert expression
is almost the same as:
>>> if not expression:
... raise AssertionError()
>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
>>> assert 1 == 1
>>> assert 1 == 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError
>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
•A second optional expression on the assert statement provides a message for the AssertionError
•This helps you distinguish one assertion from another
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
>>> assert 1 == 2, "Reality check"
second expression
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
assert expression1, expression2
is almost the same as:
if not expression1:
raise AssertionError(expression2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
>>> assert 1 == 2, "Reality check"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AssertionError: Reality check
>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
•Assertions may be sprinkled liberally throughout your code to check that your code is running as expected
•Think of assertions as Python's reality check
•You decide what "reality" is
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
profit = bottom_line(today)
assert profit > 0, "Unacceptable!"
projection = growth(tomorrow)
assert projection > profit, "UR fired!"
Copyright © 2012
TrizpugPyCamp™
bigbiz.py:
Programming For The People
Assert Statements
•Assertions usually generate unhandled exceptions which halt your program
Copyright © 2012
TrizpugPyCamp™
PyCamp™ Programming For The People
Assert Statements
bigbiz.py:
Text
def bottom_line(timestamp):
"""Compute the profit on date."""
return -1e6
Copyright © 2012
Trizpug
Programming For The People
Assert Statements
> python bigbiz.py
Traceback (most recent call last):
File "bigbiz.py", line 20, in <module>
assert profit > 0, "Unacceptable!"
AssertionError: Unacceptable!
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
•By using Python's -i command line switch, you may interactively inspect what went wrong where the assertion was raised
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
> python -i bigbiz.py Traceback (most recent call last): File "bigbiz.py", line 20, in <module> assert profit > 0, "Unacceptable!"AssertionError: Unacceptable!>>> profit-1000000.0>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
•By using Python's -i command line switch, you may use also Python's debugger to see what went wrong at the point of assertion
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
> python -i bigbiz.pyTraceback (most recent call last): File "bigbiz.py", line 20, in <module> assert profit > 0, "Unacceptable!"AssertionError: Unacceptable!>>> import pdb>>> pdb.pm()> /Users/cbc/pyohio/bigbiz.py(20)<module>()-> assert profit > 0, "Unacceptable!"(Pdb) profit-1000000.0(Pdb)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
•Assertions may be turned off by "optimizing" Python with the -O command line switch
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
> python -O
>>> assert 1 == 2
>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
> python -O bigbiz.py
Profit is -1000000.00 USD.
Projected profit is -2000000.00 USD.
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Assert Statements
•It's a fine line whether assert statements are testing or debugging
•Assert statements are slightly more sophisticated than using print
•But assert statements form the basis for testing in Python
•In Python, a test is an assertion of an expected result
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Unittest Module
•Unittest is "batteries included" in Python
•Unittest helps you separate test code from the code under test
•Unittest helps you write tests before code
•Unittest helps you organize and discover all your tests
•Unittest hooks into many other Python tools
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Unittest Module
>>> import unittest
>>> dir(unittest)
['BaseTestSuite', 'FunctionTestCase', 'SkipTest', 'TestCase', 'TestLoader', 'TestProgram', 'TestResult', 'TestSuite', 'TextTestResult', 'TextTestRunner', '_TextTestResult', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__unittest', 'case', 'defaultTestLoader', 'expectedFailure', 'findTestCases', 'getTestCaseNames', 'installHandler', 'loader', 'main', 'makeSuite', 'registerResult', 'removeHandler', 'removeResult', 'result', 'runner', 'signals', 'skip', 'skipIf', 'skipUnless', 'suite', 'util']
>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
•unittest.TestCase is a class
•You create TestCase subclasses
•You add methods whose names start with "test" to your TestCase subclass
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
•Each test method you supply in your subclass executes a TestCase supplied method whose name starts with "assert"
•A TestCase provides what is known as a "test fixture" in testing parlance
•The unittest module provides many ways to run the test methods of your text fixture
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Text
"""Demonstrate the unittest module."""
import operator
import unittest
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Textclass TestOperator(unittest.TestCase):
"""Test the operator module."""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Text
def test_add(self):
"""Test the add function."""
self.assertEqual(operator.add(2, 2),
2 + 2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Text
def test_sub(self):
"""Test the sub function."""
self.assertEqual(operator.sub(4, 2),
4 - 2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Text
def test_mul(self):
"""Test the mul function."""
self.assertEqual(operator.mul(2, 2),
2 * 2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Programming For The People
main Class
•unittest.main is also a class
•unittest.main is normally only used within a script containing test fixtures
•When unittest.main is instantiated, all of the tests in the script's namespace are loaded and run
Copyright © 2012
TrizpugPyCamp™
Programming For The People
main Class
> python test_operator.py
...
--------------------------------------Ran 3 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestLoader Class
•Notice that nowhere does it appear that your script has instantiated your TestCase subclass or execute any of its methods
•unittest.main instead instantiates a special unittest.TestLoader class which has methods to search your module for TestCase classes and instantiate them
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestRunner Class
•Once the TestLoader instance created by unittest.main discovers and instantiates the TestCase in your script, unittest.main instantiates a special unittest.TestRunner class which has methods to run the methods of your TestCase instances
•unittest.main takes care of handling TestLoaders and TestRunners for you!
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestRunner Class
Copyright © 2012
TrizpugPyCamp™
TestCase(superclass)
TestOperator(subclass)
unittest.main(instance)
TestLoader(instance)
TestRunner(instance)
TestOperator(instance)
createTests()
runTests()
discover()
run()
Programming For The People
main Class
test_operator.py:
Textif __name__ == "__main__":
unittest.main(verbosity=2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
main Class
> python test_operator.py test_add (__main__.TestOperator)Test the add function. ... oktest_mul (__main__.TestOperator)Test the mul function. ... oktest_sub (__main__.TestOperator)Test the sub function. ... ok
--------------------------------------Ran 3 tests in 0.001s
OK>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
main Class
•Notice that your tests did not run in the same order in which they were defined
•unittest.main loads the test methods from your TestCase instance's __dict__ attribute
•Dictionaries are unordered
•unittest.main() runs the test methods in your Python's built-in order for strings
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Text
def test_add(self):
"""Test the add function."""
self.assertEqual(operator.add(2, 2),
2 + 3)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
> python test_operator.py F..======================================FAIL: test_add (__main__.TestOperator)Test the add function.--------------------------------------Traceback (most recent call last): File "test_operator.py", line 13, in test_add self.assertEqual(operator.add(2, 2), 2 + 3)AssertionError: 4 != 5
--------------------------------------Ran 3 tests in 0.082s
FAILED (failures=1)>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operator.py:
Text
def test_add(self):
"""Test the add function."""
self.assertEqual(operator.add(2, 2),
2 + "2")
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
> python test_operator.py E..================================================ERROR: test_add (__main__.TestOperator)Test the add function.------------------------------------------------Traceback (most recent call last): File "test_operator.py", line 13, in test_add self.assertEqual(operator.add(2, 2), 2 + "2")TypeError: unsupported operand type(s) for +:int' and 'str'
------------------------------------------------Ran 3 tests in 0.001s
FAILED (errors=1)>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
•Running a test results in one of three outcomes:
★ Success (expected result)
★ Failure (unexpected result)
★ Error (error running the test)
•The outcomes of all tests are accumulated in a unittest.TestResult instance
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
•Most of the time you will not need to create your own TestResult instances
•Most of the ways you will run tests will instantiate and report a TestResult for you
•But to run a test always requires a TestResult instance somewhere
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
> python -m unittest test_operator
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
> python -m unittest \
test_operator.TestOperator
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
> python -m unittest \
test_operator.TestOperator.test_add
.
--------------------------------------
Ran 1 test in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
> python -m unittest test_operator.py
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
> python -m unittest -v test_operator test_add (test_operator.TestOperator)Test the add function. ... oktest_mul (test_operator.TestOperator)Test the mul function. ... oktest_sub (test_operator.TestOperator)Test the sub function. ... ok
--------------------------------------Ran 3 tests in 0.001s
OK>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestResult Class
> python -m unittest -hUsage: python -m unittest [options] [tests]
Options: -h, --help Show this message -v, --verbose Verbose output -q, --quiet Minimal output -f, --failfast Stop on first failure -c, --catch Catch control-C and display results -b, --buffer Buffer stdout and stderr during test runs[tests] can be a list of any number of testmodules, classes and test methods.
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
> python -m unittest
.......
--------------------------------------
Ran 7 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
Alternative Usage: python -m unittest \ discover [options]
Options: -s directory Directory to start discovery ('.' default) -p pattern Pattern to match test files ('test*.py' default) -t directory Top level directory of project (default to start directory)For test discovery all test modules must beimportable from the top level directory.
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
•Notice that TestCase.assertEqual does not appear to raise an unhandled AssertionError
•The TestRunner instance handles the AssertionError for failing tests and updates the TestResult instance
•TestCase.assertEqual is but one of many test methods you may use in your tests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Tests If
assertEqual(a, b) a == b
assertNotEqual(a, b) a != b
assertTrue(x) bool(x) is True
assertFalse(x) bool(x) is False
assertIs(a, b) a is b
assertIsNot(a, b) a is not b
assertIsNone(x) x is None
assertIsNotNone(x) x is not None
assertIn(a, b) a in b
assertNotIn(a, b) a not in b
assertIsInstance(a, b) isinstance(a, b)
assertNotIsInstance(a, b) not isinstance(a, b)
Programming For The People
TestCase Class
•You not only need to test if your code produces expected results, you also need to test if your code handles unexpected results in an expected manner!
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
test_operatorNG.py:
Text
def test_add_str(self):
"""Test bad args for add."""
with self.assertRaises(TypeError):
operator.add(2, "2")
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
> python -m unittest test_operatorNG
....
--------------------------------------
Ran 4 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Tests If
assertRaises(exception) exception raised
assertRaisesRegex(exception, regex) exception raised and message matches regex
assertWarns(warning) warning raised
assertWarnsRegex(warning, regex) warning raised and message matches regex
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Tests If
assertAlmostEqual(a, b) round(a-b, 7) == 0
assertNotAlmostEqual(a, b) round(a-b, 7) != 0
assertGreater(a, b) a > b
assertGreaterEqual(a, b) a >= b
assertLess(a, b) a < b
assertLessEqual(a, b) a <= b
assertRegex(s, re) s matches regex
assertNotRegex(s, re) s does not match regex
assertCountEqual(a, b)a and b have the same elements in the same number, regardless of order
Programming For The People
TestCase Class
Copyright © 2012
TrizpugPyCamp™
Method Compares
assertMultiLineEqual(a, b) Strings
assertSequenceEqual(a, b) Sequences
assertListEqual(a, b) Lists
assertTupleEqual(a, b) Tuples
assertSetEqual(a, b) Sets and Frozensets
assertDictEqual(a, b) Dictionaries
Programming For The People
TestCase Class
>>> [attr for attr
... in dir(unittest.TestCase)
... if attr.startswith('assert')]
Copyright © 2012
TrizpugPyCamp™
•Plus many more!
Programming For The People
TestCase Class
>>> help(unittest.TestCase.
... assertDictContainsSubset)
Help on function assertDictContainsSubset in module unittest.case:
assertDictContainsSubset(self, subset,
dictionary, msg=None)
Checks whether dictionary is a superset of subset.
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
"""Demonstrate the unittest module."""
import operator
import unittest
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textclass TestAdd(unittest.TestCase):
"""Test the add function."""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_add_int(self):
"""Test with ints."""
self.assertEqual(operator.add(2, 2),
2 + 2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_add_str(self):
"""Test with strs."""
with self.assertRaises(TypeError):
operator.add(2, "2")
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textclass TestSub(unittest.TestCase):
"""Test the sub function."""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_sub_int(self):
"""Test with ints."""
self.assertEqual(operator.sub(4, 2),
4 - 2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_sub_str(self):
"""Test with strs."""
with self.assertRaises(TypeError):
operator.sub(4, "2")
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textclass TestMul(unittest.TestCase):
"""Test the mul function."""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_mul_int(self):
"""Test with ints."""
self.assertEqual(operator.mul(2, 2),
2 * 2)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
def test_mul_str(self):
"""Test with strs."""
self.assertEqual(operator.mul(2, "2"),
"22")
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Text
str_suite = unittest.TestSuite()
str_suite.addTest(TestAdd("test_add_str"))
str_suite.addTest(TestSub("test_sub_str"))
str_suite.addTest(TestMul("test_mul_str"))
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
test_operatorNG2.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
•Objects you have bound in the test_operatorNG2 namespace:
•TestAdd class
•TestSub class
•TestMul class
•str_suite instance
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
> python test_operatorNG2.py
......
--------------------------------------
Ran 6 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
> python -m unittest \
test_operatorNG2.str_suite
...
--------------------------------------
Ran 3 tests in 0.000s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestSuite Class
> python -m unittest -v \ test_operatorNG2.str_suitetest_add_str (test_operatorNG2.TestAdd)Test with strs. ... oktest_sub_str (test_operatorNG2.TestSub)Test with strs. ... oktest_mul_str (test_operatorNG2.TestMul)Test with strs. ... ok
--------------------------------------Ran 3 tests in 0.001sOK>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
Copyright © 2012
TrizpugPyCamp™
pyohio-+ bin/python pycamp-+ __init__.py setup.py fibonacci.py triangle.py tests-+ __init__.py test_fibonacci.py test_triangular.py
Programming For The People
Organizing Tests
> python pycamp/fibonacci.py 100
1 2 3 5 8 13 21 34 55 89
> python
>>> from pycamp.fibonacci import fib
>>> fib(100)
[1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
> python pycamp/triangular.py 100
1 3 6 10 15 21 28 36 45 55 66 78 91
> python
>>> from pycamp.triangular import tri
>>> tri(100)
[1, 3, 6, 10, 15, 21, 28, 36, 45, 55, 66, 78, 91]
>>>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
"""Tests for the fibonacci module."""
from pycamp import fibonacci
import unittest
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Textclass TestFibonacci(unittest.TestCase):
"""Test fibonacci's functions."""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
•TestCase also supplies methods you override
•TestCase.setUp() is called before every test method you supply in your subclass
•TestCase.tearDown() is called after every test method you supply in your subclass
Copyright © 2012
TrizpugPyCamp™
Programming For The People
TestCase Class
•Your TestCase subclasses, along with all the test methods you supply, and all the TestCase supplied methods you override, possibly all bundled up into a TestSuite, are collectively known as a "test fixture" in testing parlance
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
def setUp(self):
"""Test fixture build."""
self.lessThan100 = [1, 2, 3, 5, 8,
13, 21, 34, 55,
89, ]
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
def test_fib_100(self):
"""Test fib for numbers < 100."""
self.assertEqual(fibonacci.fib(100),
self.lessThan100)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Text
def test_fib_10(self):
"""Test fib for numbers < 10."""
self.assertEqual(fibonacci.fib(10),
self.lessThan100[:5])
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_fibonacci.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
"""Tests for the triangular module."""
from pycamp import triangular
import unittest
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Textclass TestTriangular(unittest.TestCase):
"""Test triangular's functions."""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
def setUp(self):
"""Test fixture build."""
self.lessThan100 = [1, 3, 6, 10, 15,
21, 28, 36, 45,
55, 66, 78, 91, ]
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
def test_tri_100(self):
"""Test tri for numbers < 100."""
self.assertEqual(triangular.tri(100),
self.lessThan100)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Text
def test_tri_10(self):
"""Test tri for numbers < 10."""
self.assertEqual(triangular.tri(10),
self.lessThan100[:3])
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/tests/test_triangular.py:
Textif __name__ == "__main__":
unittest.main()
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
> python -m unittest discover \
-s pycamp -t .
....
--------------------------------------Ran 4 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/setup.py:
Text
"""Setup for pycamp package."""
from setuptools import setup, \
find_packages
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/setup.py:
Text
setup(
name="pycamp",
version="1.0",
packages=find_packages(),
author="Chris Calloway",
author_email="[email protected]",
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
pycamp/setup.py:
Text
description="Testing Fundamentals",
license="PSF",
keywords="testing pycamp",
url="http://pycamp.org",
test_suite="pycamp.tests",
)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
> python pycamp/setup.py test
running test
..lots of output omitted for brevity..
--------------------------------------
Ran 4 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
> python pycamp/setup.py test
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
> python pycamp/setup.py install
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Organizing Tests
> python pycamp/setup.py register
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Unittest Module
•unittest provides nice separation of tests from code
•One person can write tests, while another writes code to make tests pass
•unittest provides fine grain control over what tests to run when
•unittest conforms to industry standard testing
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Unittest Module
•However, it can be difficult to keep tests and code in sync
•Also, writing code to test code can also be more difficult than the code under test
•What about testing and debugging the test code?
•Reading unittest tests is a poor way to figure out how code works
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
•Informally, even without written tests, you probably already test your code by simply using it as it was meant to be used
•You've probably imported your code at a Python prompt and inspect how it works manually
•You just don't yet have a way of repeating that informal testing in an automated manner
•What you need is the doctest module
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
Copyright © 2012
TrizpugPyCamp™
pyohio-+ bin/python pycampNG-+ __init__.py setup.py fibonacci.py triangle.py tests-+ __init__.py test_pycamp.py
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
"""A module of functions about non-zero Fibonacci numbers.
>>> import fibonacci
>>>
"""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
def fib(n):
"""Return the sequence of non-zero
Fibonacci numbers less than n.
fib(n) -> [0 < fibonacci numbers < n]
where n in an int.
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
>>> lessThan100 = [1, 2, 3, 5, 8, 13,
... 21, 34, 55, 89]
>>> fib(100) == lessThan100
True
>>> fib(10) == lessThan100[:5]
True
"""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
"""A module of functions about non-zero triangular numbers.
>>> import triangular
>>>
"""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
def tri(n):
"""Return the sequence of non-zero
triangular numbers less than n.
tri(n) -> [0 < triangular numbers < n]
where n in an int.
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
>>> lessThan100 = [1, 3, 6, 10, 15,
21, 28, 36, 45,
55, 66, 78, 91]
>>> tri(100) == lessThan100
True
>>> tri(10) == lessThan100[:3]
True
"""
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
> python -m doctest pycampNG/fibonacci.py
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
> python -m doctest -v \
pycampNG/fibonacci.py
Trying:
import fibonacci
Expecting nothing
ok
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
Trying:
lessThan100 = [1, 2, 3, 5, 8, 13,
21, 34, 55, 89]
Expecting nothing
ok
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
Trying:
fib(100) == lessThan100
Expecting:
True
ok
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
Trying:
fib(10) == lessThan100[:5]
Expecting:
True
ok
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
2 items passed all tests:
1 tests in fibonacci
3 tests in fibonacci.fib
4 tests in 2 items.
4 passed and 0 failed.
Test passed.
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
> python -m doctest pycampNG/triangular.py
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
pycampNG/fibonacci.py:
Text
if __name__ == '__main__':
if sys.argv[1].lower() == 'test':
import doctest
doctest.testmod()
else:
print(" ".join([str(x) for x in
fib(int(sys.argv[1]))]))
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
pycampNG/triangular.py:
Text
if __name__ == '__main__':
if sys.argv[1].lower() == 'test':
import doctest
doctest.testmod()
else:
print(" ".join([str(x) for x in
tri(int(sys.argv[1]))]))
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
> python pycampNG/fibonacci.py test
> python pycampNG/fibonacci.py 100
1 2 3 5 8 13 21 34 55 89
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
> python pycampNG/triangular.py test
> python pycampNG/triangular.py 100
1 3 6 10 15 21 28 36 45 55 66 78 91
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
•But what about setuptools.setup?
•The test_suite argument of setup triggers unittest discovery, not doctest discovery
•What you need is a way to turn doctests into unittests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
•The doctest.DocTestSuite() function searches a module for doctests and converts them into a unittest.TestSuite instance
•Now all you need is a way to communicate your TestSuite instance(s) to unittest discovery
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
unittest.TestLoader
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
unittest.TestSuite
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
"test*.py"
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
def load_tests(loader,
tests,
pattern):
...
return tests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Textimport doctest
from pycampNG import fibonacci
from pycampNG import triangular
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
pycampNG/tests/test_pycamp.py:
Text
def load_tests(loader, tests, pattern):
tests.addTests(
doctest.DocTestSuite(fibonacci))
tests.addTests(
doctest.DocTestSuite(triangular))
return tests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
pycamp/setup.py:
Text
setup(
name="pycampNG",
...
test_suite="pycampNG.tests",
)
Copyright © 2012
TrizpugPyCamp™
Programming For The People
load_tests Protocol
> python pycampNG/setup.py test
running test
..lots of output omitted for brevity..
--------------------------------------
Ran 4 tests in 0.001s
OK
>
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Doctest Module
Together, the doctest.DocTestSuite() function and the load_tests protocol from the unittest module enable you to use all the tools available for unittests with doctests
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Python Testing Tools Taxonomy
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Test Driven Development
•Doctests enable you to do Test Driven Development (TDD)
•TDD is where you write tests for your code before you write code
•Then you write code to make your tests pass
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Test Driven Development
•When all your tests pass, then you have finished coding
•You can develop your code incrementally, writing one test at a time, then getting that one test to pass
•That means you can stop coding at any time
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Test Driven Development
•If your code needs more features, then what you really need is more tests, and code which makes those tests pass
•Writing tests lets you see what the API for your code is up front, instead of having it designed haphazardly
•Writing tests can provide the documentation for your code
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Test Driven Development
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Tomorrow 12:15pm in Barbie Tootle
Copyright © 2012
TrizpugPyCamp™
Programming For The People
Thank You
Copyright © 2012
TrizpugPyCamp™