Test composer reference
The test composer is designed to make it easy to write powerful autonomous tests for the Antithesis platform. It does this by letting you define test commands, which it runs in appropriate situations during testing.
The test composer takes care of varying parallelism, test length, and command order with an API that tries to balance simplicity with expressiveness.
The test composer works by running executables or scripts that you provide in your containers (called test commands), which are organized into test directories. The filename of each test command starts with a prefix, which tells test composer how the command should be run.
This feature is stable but still in beta. Your Professional Services representative can help you adapt your existing integration test or Antithesis workload, or contact us for more details.
Test directory
/opt/antithesis/test/v1/<test_name>
A directory in your container that represents a collection of test commands that are compatible with one another. In any particular branch of the Antithesis multiverse, test commands will all be selected from the same test directory. If test directories with the same name appear in multiple containers, they will be treated as belonging to the same test, even if they define different test commands.
<test_name>
can be any valid directory name.
Test command
<test_name>/<prefix>_<command>
An executable for the test composer to run. It can be a binary or script and must have an appropriate shebang in the first line, e.g. #!/usr/bin/env bash
It may have any or no extension, but must be marked executable by the container’s default user. It will typically call the APIs of the software under test in order to change the state of the system, check properties, or both.
<prefix>
specifies the type of test command. See options below.
<command>
can be any valid unix filename.
Test Commands
Driver Commands
Driver commands run after the first
command (if provided) and during a period where Antithesis may be adding faults to the test environment. A test should have at least one driver command. The types of driver command vary in their approach to parallelism.
Parallel driver command
<test_name>/parallel_driver_<command>
Parallel_driver
commands are the core of a typical Antithesis test. The test composer may start a parallel_driver
command while other parallel_driver
commands (including earlier copies of the same one) are running.
Examples include:
- A script in a “client” container that writes rows to a database, queries for those rows, and asserts that it finds the rows it wrote
- A “low consistency” availability check that runs on a database node and asserts that it is possible to make a read without timing out
- A “data source” that publishes to a queue
Singleton driver command
<test_name>/singleton_driver_<command>
A singleton_driver
command will run as the only driver command in a particular branch of history. Non-driver commands like first
and finally
will still run.
Singleton_driver
commands are the most straightforward way to port existing integration tests to Antithesis, or to use an existing Antithesis workload in the test composer, but they don’t take full advantage of the test composer.
Serial driver command
<test_name>/serial_driver_<command>
Serial_driver
commands run at any point after the first
command (if any) when there are no other driver commands running. This means that they may be run before or after parallel_driver
commands or other serial_driver
commands, but they will not be run alongside either. Non-driver commands like anytime
will still run alongside.
Common examples of serial_driver
fragments include:
- “Do full failover”
- A self-validating step that checks “when I’m done, exactly the rows I’ve tried to write got added”
- Any other instruction that would be hindered by other things running in parallel
Serial_driver
commands may be helpful in adapting singleton_driver
commands to use the test composer.
Quiescent Commands
First command
test_name>/first_<command>
First
commands are optional commands used to leave the system in an appropriate state for the driver commands. If your test directory includes first
commands, exactly one will be selected for a branch of history, and it will run after the Setup Complete lifecycle event but before any driver commands (see below).
First
commands are often appropriate for setting up data or database schemas, or bootstrapping other components for later commands to depend on, but they are not required and may not be useful for every testing approach.
Eventually command
<test_name>/eventually_<command>
Eventually
commands run after at least one driver command (see below) has started. Eventually
commands create a new alternate branch of the multiverse in which all driver commands are killed and Antithesis no longer injects new faults.
Any ongoing faults will continue to run till their predetermined end. An eventually command should therefore include checks for whether faults are occuring, and time for the system to recover, if appropriate.
Since Antithesis will not resume testing on this branch of history, Eventually
commands may take destructive actions.
Eventually
commands are ideal for testing “eventual” properties of a system, like availability or replica consistency.
Advanced Commands
The test command types above work well for most Antithesis users, but the test composer recognizes a few more in order to support specific situations.
Finally command
<test_name>/finally_<command>
Finally
commands are quiescent commands that closely resemble eventually
commands except that the test composer will only run a finally
command in a branch where every driver command that was started has also completed successfully.
Finally
commands may be useful for testing subtle invariants that are disrupted when drivers are killed partway through.
Common examples of finally
commands include:
- “Data is eventually consistent”
- “The database contains $X lines”
Anytime command
<test_name>/anytime_<command>
Anytime
commands will run at any time after the first
command (if any), including during singleton
or serial
driver commands.
Anytime
commands enable you to run many of the same invariant checks that are typically expressed as parallel_driver
commands alongside serial_driver
and/or singleton_driver
commands.
A common example of an anytime
command is:
- “A read always reflects the previous write” (causal consistency)
- A “low consistency” availability check that runs on a database node and asserts that it is possible to make a read without timing out
For a more detailed explanation of the concepts underpinning the test composer, see Principles of test composition.