Python Testing and Continuous Integration: Glossary

Key Points

Basics of Testing
  • Tests check whether the observed result, from running the code, is what was expected ahead of time.

  • Tests should ideally be written before the code they are testing is written, however some tests must be written after the code is written.

  • Assertions and exceptions are like alarm systems embedded in the software, guarding against exceptional behaviour.

  • Unit tests try to test the smallest pieces of code possible, usually functions and methods.

  • Integration tests make sure that code units work together properly.

  • Regression tests ensure that everything works the same today as it did yesterday.

Exceptions, Status Values and Tracebacks
  • Exceptions are one way exposing errors.

  • There are many types of exceptions.

Using Exceptions
  • Exceptions are effectively specialized runtime tests

  • Exceptions can be caught and handled with a try-except block

  • Many built-in Exception types are available

Assertions
  • Assertions are one line tests embedded in code.

  • The assert keyword is used to set an assertion.

  • Assertions halt execution if the argument is false.

  • Assertions do nothing if the argument is true.

  • The numpy.testing module provides tools numeric testing.

  • Assertions are the building blocks of tests.

Unit Tests
  • Functions are the atomistic unit of software.

  • Simpler units are easier to test than complex ones.

  • A single unit test is a function containing assertions.

  • Such a unit test is run just like any other function.

  • Running tests one at a time is pretty tedious, so we will use a framework instead.

Running Tests with pytest
  • The pytest command collects and runs tests starting with Test or test_.

  • . means the test passed

  • F means the test failed or erred

  • x is a known failure

  • s is a purposefully skipped test

Fixtures
  • It may be necessary to set up “fixtures” composing the test environment.

Edge and Corner Cases
  • Functions often fail at the edge of their range of validity

  • Edge case tests query the limits of a function’s behavior

  • Corner cases are where two edge cases meet

Integration and Regression Tests
  • Integration tests interrogate the coopration of pieces of the software

  • Regression tests use past behavior as the expected result

Continuous Integration
  • Servers exist for automatically running your tests

  • Running the tests can be triggered by a GitHub pull request

  • CI allows cross-platform build testing

  • A .travis.yml file configures a build on the travis-ci servers

  • Many free CI servers are available

Test Driven Development
  • Test driven development is a common software development technique

  • By writing the tests first, the function requirements are very explicit

  • TDD is not for everyone

  • TDD requires vigilance for success

Glossary

assert
A keyword that raises a AssertionError (a type of exception) when its argument is false. Used by pytest to define tests.
continuous integration
Automatically checking the building and testing process across platforms.
exceptions
A python type which contains information about an unusual occurrence or problem.
except
A keyword used to catch and carefully handle that exception. Used with try.
integration test
Tests that check that various pieces of the software work together as expected.
pytest
A Python package with testing utilities.
pytest
A command-line program that collects and runs unit tests.
regression test
Tests that defend against new bugs, or regressions, which might appear due to new software and updates.
test-driven development
A software development strategy in which the tests are written before the code.
try
A keyword that guards a piece of code which may throw an exception.
unit test
Tests that investigate the behavior of units of code (such as functions, classes, or data structures).