Checking Test Templates Locally

When you’re developing a test template for the Antithesis Test Composer, whether you’re starting from scratch or just tweaking a command, it can be convenient to check your work locally before you kick off a full Antithesis test run.

These guidelines should help you try out your commands in circumstances that resemble what the Test Composer might do in one branch of the Antithesis multiverse. They will not get you to an exhaustive test of your commands or your software – for that, you need a real Antithesis test run – but they may be useful for development or troubleshooting.

We’re assuming…

You’re a little bit familiar with the Antithesis [Test Composer], including the idea of a test directory and the existence of different types of test commands. If not, please start with our basics or reference.
You’re on a computer with Docker, the software images and a docker-compose.yaml file that you’ll be using in Antithesis.
At least one of the containers in your docker-compose.yaml has at least one test command (which should be located at /opt/antithesis/test/v1/<test_dir>/<command> in a container).
You know which of your containers have which test commands.

How to use this guide

Every type of test command has a section below, which tells you what must be true before you run a command of that type, and what must not be true when you run it.

Before you run a command, start your cluster according to the setup guide. Then, make sure you’ve done everything in the “must” section for the relevant command type, and nothing in the “must not” section.

Since we don’t have a time machine when we run outside of the Antithesis Environment, if you want to run a command whose “must not” section has been violated, you’ll need to tear down your cluster and start over with setup.

Run your commands with:

docker compose -f </path/to/docker-compose.yaml> exec <container_with_command> <path/to/command>

(Our examples assume a test directory named my_test in a service called my_test_service.)

To make it easy to cancel your commands, we recommend running everything in the foreground and using multiple terminals. (Note that the Test Composer will not cancel commands except before running an eventually command.)

In general, remember that you should:

Never mix commands from different test directories.
Always start with a first command if your test directory has any.

Setup

Run docker compose -f </path/to/docker-compose.yaml> up This will run in the foreground, and you’ll be able to cancel it when you want to take down the cluster.
Wait for the SetupComplete event
Open a new terminal to run your commands (with docker compose -f </path/to/docker-compose.yaml> exec <container_with_command> <path/to/command> as above)

docker exec and docker compose exec are both valid ways to run something in a container. We suggest docker compose exec because it accepts service names, but you can use whichever you’re comfortable with.

Guidelines by Command

First command

docker compose -f ./docker-compose.yaml exec my_test_service /opt/antithesis/test/v1/my_test/first_example.sh

Must not

Must not have run any other commands.

Singleton driver command

docker compose -f ./docker-compose.yaml exec my_test_service /opt/antithesis/test/v1/my_test/singleton_driver_example.sh

Must

Must have run a first command to completion, if this test directory contains any.

Must not

Must not have run any commands other than an applicable first command.

Serial driver command

docker compose -f ./docker-compose.yaml exec my_test_service /opt/antithesis/test/v1/my_test/serial_driver_example.sh

Must

Must have run a first command to completion, if this test directory contains any.

Must not

Must not have run commands from any other test directory.
Must not have run a singleton driver, an eventually, or a finally.
Must not be currently running a serial driver or a parallel driver (it’s fine to have run one or more to completion).

Parallel driver command

docker compose -f ./docker-compose.yaml exec my_test_service /opt/antithesis/test/v1/my_test/parallel_driver_example.sh

Must

Must have run a first command to completion, if this test directory contains any

Must not

Must not have run commands from any other test directory.
Must not have run a singleton driver or an eventually.

It’s fine to run more than one parallel driver at a time. As we mentioned in how to, we suggest running test commands in the foreground so that you can tell when and whether they complete.

Anytime command

docker compose -f ./docker-compose.yaml exec my_test_service /opt/antithesis/test/v1/my_test/anytime_example.sh

Must

Must have run a first command to completion, if this test directory contains any.

Must not

Must not have run commands from any other test directory.

Finally command

docker compose -f ./docker-compose.yaml exec my_test_service /opt/antithesis/test/v1/my_test/finally_example.sh

Must

Must have run a first command to completion, if this test directory contains any.
Must have run at least one (singleton, serial, or parallel) driver command to completion.

Must not

Must not have run commands from any other test directory.
Must not be currently running any driver commands (but you may let them run to completion).

Eventually command

docker compose -f ./docker-compose.yaml exec my_test_service /opt/antithesis/test/v1/my_test/eventually_example.sh

Must

Must have run a first command to completion, if this test directory contains any.
Must have run at least one (singleton, serial, or parallel) driver command to completion.

Must not

Must not have run commands from any other test directory.
Must not be currently running any driver commands (but you may cancel them or let them run to completion).