py.test; unittest alternative

Ian Bicking

Imaginary Landscape

What is it?

What is a test?

A test is a runnable thing:

Functions

def test_addition():
    assert 1 + 1 == 2
    assert 1 + -1 == 0

Classes

class TestStuff:
    numbers = [(1, 1, 2)]
    def test_me(self):
        for n1, n2, addition in self.numbers:
            assert n1 + n2 == addition

Tests with parameters

def test_lots():
    numbers = [(1, 1, 2), (2, 2, 4), (5, 5, 9)]
    for args in numbers:
        yield (adder,) + args
def adder(n1, n2, addition):
    assert n1 + n2 == addition

Running

  • Use py.test; finds all tests under current directory
  • Use --session and it tracks modified date of all test files it finds, does constant updates

Reporting

  • Shows tracebacks for each failed test
  • Re-evaluates assertions (no need for assertEqual)
  • Can show local variables in each frame
  • Can enter debugger on exception

Why not unittest?

  • There's lots of extensions to unittest, but they are all mutually incompatible and usually fragile. They all enforce a specific code layout (among other things). None of them are well documented.
  • It's surprisingly hard to collect unittest tests from a large project.
  • Lots of boilerplate in unittest.
  • Really hard to do data-based testing.
  • Smells of Java.

Why change to py.test?

  • It's (relatively) easy to change test systems: your unit tests are self-testing.
  • Your tests contain very few py.test-related idioms; it's just bare code.
  • Still under development; will keep getting better.
  • Lots of debugging aids.
  • Smells of Python.

Test fixtures

  • unittest provides a fixture (TestCase)
  • With py.test you must provide your own fixtures
  • You don't need to use inheritance

Decorator fixtures

  • Decorators wrap functions; that's all they do
  • Easy to add more decorators and compose decorators

Decorator fixtures

Setup, teardown, can be achieved with decorators: 
def setup(func):
    setup_stuff()
    func(value_that_was_set_up)
    teardown_stuff()

@setup
def test_here(some_value): ...

param

Dissecting the param decorator; here's the original: 
def test_to_roman():
    for integer, numeral in known_values:
        assert roman.toRoman(integer) == numeral

param: generators

def test_to_roman():
    for integer, numeral in known_values:
        yield inner_test_to_roman, integer, numeral
def inner_test_to_roman(integer, numeral):
    assert roman.toRoman(integer) == numeral

param: decorator

def param(param_list):
    def make_param_func(func):
        def yielder():
            for args in param_list:
                yield (func,) + args
        return yielder
    return decorator

@param(known_values)
def test_to_roman(integer, numeral):
    assert roman.toRoman(integer) == numeral

param: decorator 2

You must pass in a param_list like [(arg1, arg2), (arg1, arg2)]. We might want to be able to pass in [arg, arg, arg] for functions that take one argument: 
    def yielder():
        for args in param_list:
            if not isinstance(args, tuple):
                yield (func, args)
            else:
                yield (func,) + args

param: decorator 3

Sometimes we don't want to unpack list of arguments into multiple tests. 
    if expand:
        return yielder
    else:
        def runner():
            for args in yielder():
                args[0](*args[1:])
        return runner

py.test

Find it at http://codespeak.net/py
Find this at http://svn.colorstudy.com/home/ianb/pytest_roman