Running tests¶

Launchpad has a large test suite. The normal pattern when developing is to locally run the tests you think your change affects, and rely on post-merge testing by buildbot for full coverage.

There are two ways to run the tests:

bin/test — the standard test runner. Start here.
testr (testrepository) - an alternative wrapper you can use to iterate on failing tests.

Note

Before you can run the tests, build the test runner once with make build. This creates the bin/test script. See Setting up and running Launchpad (advanced) if you have not set up Launchpad yet.

bin/test (recommended)

Run the whole suite

From the top of the Launchpad tree:

xvfb-run bin/test -vvc

The complete suite can take hours, depending on your machine, so you will usually want to narrow it down (see below).

The -vvc options give verbose, coloured output. Run bin/test --help to see everything the runner accepts.

Run specific tests

Use the -t option to select tests by pattern. The pattern is a regular expression matched against test names, and you can pass -t several times:

bin/test -vvc -t test-pattern-1 -t test-pattern-2

You can also point the runner to a dotted module, which is the fastest way to iterate on a single module:

bin/test -vvc -m lp.path.to.test_module

Select test layers

Tests are grouped into layers. Use --layer to run only a given layer, and prefix a layer name with ! to run everything except that layer:

# Run only the tests in LAYER.
bin/test -vvc --layer 'LAYER'

# Run everything except the tests in LAYER.
bin/test -vvc --layer '!LAYER'

Speed up repeated runs

The Librarian and memcached services are slow to start and stop and rarely change between runs. Set LP_PERSISTENT_TEST_SERVICES=1 to leave them running between test runs:

LP_PERSISTENT_TEST_SERVICES=1 bin/test <test arguments>

When you are done, stop the persistent services with:

bin/kill-test-services

Run browser tests without a display

Some tests drive a browser and need an X server. On a machine without a display, run them under xvfb-run, which starts a virtual display for the duration of the command:

xvfb-run bin/test <test arguments>

testr (testrepository)

testr is an alternative runner that some developers prefer for working through failing tests. You do not need to create a .testr.conf file; the defaults work.

Install and initialise it once:

sudo apt install testrepository
testr init

Run the tests:

# Run all the tests.
testr run

# Run tests matching a pattern.
testr run -- -t foo

# Re-run only the tests that are currently failing.
testr run --failing

Inspect failures:

# List the currently known failures.
testr failing