Library/Software Engineering/Unit Test Structure

From Athile

Test Hierarchy

The unit tests are broken down into the following structure:

• module
  • group
    • test set
      • test
        • check

Starting with a test, a test should encompass ensuring that one logical premise holds. It is usually composed of one, but sometimes multiple checks. Every check within a particular test must pass successfully for the test itself to be considered passing. A check is single boolean expression: for example, the equality of an integer value or checking for a non-null return. A example of a test with multiple checks might be a bounding box initialization: for the initialization test to pass correctly, the bounding box must return zero from its volume() method and return true from its empty() method. The line between what is considered a check and what is considered a test can be fuzzy, but in general, whenever possible a test should have one check or at least as few as possible.

A test set is a group of closely related tests. For example, a single method may have a host of pre-conditions, post-conditions, and variations of input parameters: it may make sense to create a test set for that single method. On the other hand, if there several distinct uses of that method, perhaps each use pattern has its own test set? The test set is merely an organizational tool, so it may be used however is deemed best. As a general guideline, a test set should likely contain between one dozen and a couple hundred tests (these numbers seem to keep the unit test organization most manageable.

A group is the next level of organization, grouping together related test sets. These groups generally should attempt to get complete coverage of one particular aspect of the module. The intention is to develop a good set of groups within a module so that a programmer when a programmer modifies a piece of code, they know what groups logically should be retested. The organization of test sets into groups should largely be guided by the architecture of the system so that closely dependent code ends up in the same group whenever possible.

A module is the whole set of tests for an entire application, library, plug-in, or other single target.

Test Lists

Lastly is a test list. A test list is simply a list of groups, test sets, and/or tests that should be run when the test list is passed to the unit test executable. This represents cross-sectional functionality which may represent the components of a particular user feature, a set of related interrelated tests that programmers often unintentionally break due to an un-obvious correlation between the subsystems, a quick full-coverage smoke-test, tests that do not require internet access / an advanced GPU / etc., etc. While the intention for the module > group > test set organization to largely follow the architecture of the code, test lists can pick and choose from by whatever rationale best meets the objective of the test lists.