Test composer reference

The test composer is designed to make it easy to write powerful autonomous tests for the Antithesis platform. It does this by letting you define test commands, which it runs in appropriate situations during testing.

The test composer takes care of varying parallelism, test length, and command order with an API that tries to balance simplicity with expressiveness.

The test composer works by running executables or scripts that you provide in your containers (called test commands), which are organized into test directories. The filename of each test command starts with a prefix, which tells test composer how the command should be run.

Test directory

/opt/antithesis/test/v1/<test_name>

A directory in your container that represents a collection of test commands that are compatible with one another. In any particular branch of the Antithesis multiverse, test commands will all be selected from the same test directory. If test directories with the same name appear in multiple containers, they will be treated as belonging to the same test, even if they define different test commands.

<test_name> can be any valid directory name.

Test command

<test_name>/<prefix>_<command>

An executable for the test composer to run. It can be a binary or script and must have an appropriate shebang in the first line, e.g. #!/usr/bin/env bash

It may have any or no extension, but must be marked executable by the container’s default user. It will typically call the APIs of the software under test in order to change the state of the system, check properties, or both.

<prefix> specifies the type of test command. See options below. <command> can be any valid unix filename.

Test Commands

Driver Commands

Driver commands run after the first command (if provided) and during a period where Antithesis may be adding faults to the test environment. A test should have at least one driver command. The types of driver command vary in their approach to parallelism.

Parallel driver command

<test_name>/parallel_driver_<command>

Parallel_driver commands are the core of a typical Antithesis test. The test composer may start a parallel_driver command while other parallel_driver commands (including earlier copies of the same one) are running.

Examples include:

• A script in a “client” container that writes rows to a database, queries for those rows, and asserts that it finds the rows it wrote
• A “low consistency” availability check that runs on a database node and asserts that it is possible to make a read without timing out
• A “data source” that publishes to a queue

Singleton driver command

<test_name>/singleton_driver_<command>

A singleton_driver command will run as the only driver command in a particular branch of history. Non-driver commands like first and finally will still run.

Singleton_driver commands are the most straightforward way to port existing integration tests to Antithesis, or to use an existing Antithesis workload in the test composer, but they don’t take full advantage of the test composer.

Serial driver command

<test_name>/serial_driver_<command>

Serial_driver commands run at any point after the first command (if any) when there are no other driver commands running. This means that they may be run before or after parallel_driver commands or other serial_driver commands, but they will not be run alongside either. Non-driver commands like anytime will still run alongside.

Common examples of serial_driver fragments include:

• “Do full failover”
• A self-validating step that checks “when I’m done, exactly the rows I’ve tried to write got added”
• Any other instruction that would be hindered by other things running in parallel

Serial_driver commands may be helpful in adapting singleton_driver commands to use the test composer.

Quiescent Commands

First command

test_name>/first_<command>

First commands are optional commands used to leave the system in an appropriate state for the driver commands. If your test directory includes first commands, exactly one will be selected for a branch of history, and it will run after the Setup Complete lifecycle event but before any driver commands (see below).

First commands are often appropriate for setting up data or database schemas, or bootstrapping other components for later commands to depend on, but they are not required and may not be useful for every testing approach.

Eventually command

<test_name>/eventually<command>

Eventually commands run after at least one driver command (see below) has started. Eventually commands create a new alternate branch of the multiverse in which all driver commands are killed and Antithesis no longer injects new faults.

Any ongoing faults will continue to run till their predetermined end. An eventually command should therefore include checks for whether faults are occuring, and time for the system to recover, if appropriate.

Since Antithesis will not resume testing on this branch of history, Eventually commands may take destructive actions.

Eventually commands are ideal for testing “eventual” properties of a system, like availability or replica consistency.

Advanced Commands

The test command types above work well for most Antithesis users, but the test composer recognizes a few more in order to support specific situations.

Finally command

<test_name>/finally_<command>

Finally commands are quiescent commands that closely resemble eventually commands except that the test composer will only run a finally command in a branch where every driver command that was started has also completed successfully.

Finally commands may be useful for testing subtle invariants that are disrupted when drivers are killed partway through.

Common examples of finally commands include:

• “Data is eventually consistent”
• “The database contains $X lines”

Anytime command

<test_name>/anytime_<command>

Anytime commands will run at any time after the first command (if any), including during singleton or serial driver commands.

Anytime commands enable you to run many of the same invariant checks that are typically expressed as parallel_driver commands alongside serial_driver and/or singleton_driver commands.

A common example of an anytime command is:

• “A read always reflects the previous write” (causal consistency)
• A “low consistency” availability check that runs on a database node and asserts that it is possible to make a read without timing out

For a more detailed explanation of the concepts underpinning the test composer, see Principles of test composition.