Python Testing and Continuous Integration: Glossary

Key Points

Basics of Testing	Tests check whether the observed result, from running the code, is what was expected ahead of time. Tests should ideally be written before the code they are testing is written, however some tests must be written after the code is written. Assertions and exceptions are like alarm systems embedded in the software, guarding against exceptional behaviour. Unit tests try to test the smallest pieces of code possible, usually functions and methods. Integration tests make sure that code units work together properly. Regression tests ensure that everything works the same today as it did yesterday.
Exceptions, Status Values and Tracebacks	Exceptions are one way exposing errors. There are many types of exceptions.
Using Exceptions	Exceptions are effectively specialized runtime tests Exceptions can be caught and handled with a try-except block Many built-in Exception types are available
Assertions	Assertions are one line tests embedded in code. The `assert` keyword is used to set an assertion. Assertions halt execution if the argument is false. Assertions do nothing if the argument is true. The `numpy.testing` module provides tools numeric testing. Assertions are the building blocks of tests.
Unit Tests	Functions are the atomistic unit of software. Simpler units are easier to test than complex ones. A single unit test is a function containing assertions. Such a unit test is run just like any other function. Running tests one at a time is pretty tedious, so we will use a framework instead.
Running Tests with pytest	The `pytest` command collects and runs tests starting with `Test` or `test_`. `.` means the test passed `F` means the test failed or erred `x` is a known failure `s` is a purposefully skipped test
Fixtures	It may be necessary to set up “fixtures” composing the test environment.
Edge and Corner Cases	Functions often fail at the edge of their range of validity Edge case tests query the limits of a function’s behavior Corner cases are where two edge cases meet
Integration and Regression Tests	Integration tests interrogate the coopration of pieces of the software Regression tests use past behavior as the expected result
Continuous Integration	Servers exist for automatically running your tests Running the tests can be triggered by a GitHub pull request CI allows cross-platform build testing A `.travis.yml` file configures a build on the travis-ci servers Many free CI servers are available
Test Driven Development	Test driven development is a common software development technique By writing the tests first, the function requirements are very explicit TDD is not for everyone TDD requires vigilance for success

Glossary

assert: A keyword that raises a AssertionError (a type of exception) when its argument is false. Used by pytest to define tests.
continuous integration: Automatically checking the building and testing process across platforms.
exceptions: A python type which contains information about an unusual occurrence or problem.
except: A keyword used to catch and carefully handle that exception. Used with try.
integration test: Tests that check that various pieces of the software work together as expected.
pytest: A Python package with testing utilities.
pytest: A command-line program that collects and runs unit tests.
regression test: Tests that defend against new bugs, or regressions, which might appear due to new software and updates.
test-driven development: A software development strategy in which the tests are written before the code.
try: A keyword that guards a piece of code which may throw an exception.
unit test: Tests that investigate the behavior of units of code (such as functions, classes, or data structures).