Product FAQs

1. How can I validate my setup?

You can manually validate your setup by running docker-compose up locally and verifying that all services are running as expected. If this happens, you should be able to call basic_test webhook successfully.

To streamline this process, you can automate it using a GitHub Action. The example workflow includes steps to build the project, validate the local setup, run and validate test commands, and finally call the basic_test endpoint.

We recommend setting up either this or a similar workflow in your CI pipeline.

2. How can I validate test composer commands locally?

To validate your test composer commands locally:

• Run docker-compose up to bring up your system.
• Run the individual test commands using docker exec <container_with_command> path/to/test-file/. <container_name> is the container containing the test command.

If the test commands run successfully then you’re good to kick off an actual test. This page has more details.

3. How often should we test?

Most Antithesis customers run tests nightly to catch bugs and regressions as early in development as possible, and some also run shorter tests on every commit to help reduce the feedback cycle for active development.

• Nightly tests:

  • Run for 6-8hrs.
  • Uncover bugs that are hard to find. The longer a test runs, the more states are explored.

• Short tests:

  • Run on a commit to the main development branch.
  • Run for ~30min - 1hr.
  • Catch simple bugs that slipped through conventional unit and end-to-end tests.

4. Is there a way to bias test runs? If I believe there’s a bug in subsystem X can I get Antithesis to apply more effort to a subset of our tests?

Yes, there are several ways to focus on a particular subsystem:

• Write more test drivers that call the subsystem.
• Put more assertions in the subsystem or the clients that exercise the subsystem.
• Talk to our solutions engineers to help tune the configuration: support@antithesis.com.

5. Which S3 mocks do you support?

We support MinIO and Localstack services. This page has more details, and a long list of mocks for other dependencies.

6. Why isn’t MY question covered here?

Probably because we don’t know about it! Please drop us a line at support@antithesis.com or ask us on Discord.