How to Run a Test#

Antithesis provides webhook endpoints that allow you to run tests on demand. Generally, a customer will have a webhook for each distinct product that they test. Each webhook supports optional parameters that configure the test and how its results are reported.

Calling a webhook first fetches the relevant container images, which may be precisely specified by tag or digest with the antithesis.images parameter. After fetching the images, our system will build the test environment, run your test for a configured duration, and generate a triage report. A link to this report is emailed to the addresses specified with the antithesis.report.recipients parameter. Contact us if you prefer to receive your results in Slack, an issue tracker, or somewhere else.

Here is an example of calling a webhook with curl:

curl --fail -u 'user:password' /
-X POST https://<tenant>.antithesis.com/api/v1/launch_experiment/<endpoint> /
-d '{"params": { "antithesis.integrations.type":"github",
    "antithesis.integrations.callback_url":"https://api.github.com/repos/<org>/<repo>/statuses/<sha1>",
    "antithesis.integrations.token":"my_github_token",
    "antithesis.images":"my_images_with_version_list" } }'

Webhook workflow#

Antithesis tests are high-throughput but also high-latency. They should therefore be triggered by an automated system (for example with your CI server) rather than by a human being.

Your automated system should be configued to run tests at least nightly. Frequent testing has many benefits. Among them are better bisection capability around when bugs were introduced, richer data around whether we can find the bug every time, and lower risk of a bug going unnoticed and being released in production.

You should configure your CI system so that each night it:

  1. builds all of the containers associated with your Antithesis setup

  2. pushes them to our registry

  3. triggers the webhook.

If you are unable to configure your CI server to do this, or are having any other issues running your test, please contact us.

Webhook syntax#

Webhooks (basic)#

The following command runs a webhook:
curl --fail -u 'user:password' -X POST /
https://<tenant>.antithesis.com/api/v1/launch_experiment/<endpoint>

The webhook returns an HTTP status code to indicate success or failure at starting the test. This code does not contain your test results, which will be delivered asynchronously via email.

Warning

Curl (and similar tools) will exit successfully so long as the webhook returns anything at all, even if it returns a failure code. To check for success of the underlying webhook, you must ensure that a 200 or 202 code was returned. The examples on this page use the --fail flag in curl to check for success.

The following are the variables for the webhook call.

user:password

The basic auth user and password for your tenant.

<tenant>

Your tenant’s name.

<endpoint>

The endpoint to run.

Webhooks (with parameters)#

Webhook take optional parameters that control test behavior. There are default parameters, which affect all webhooks, and custom parameters that your Antithesis consultant can create for your needs.

Parameters are passed as JSON according to the following template:

{
    "params": {
        "parameter1_name": "parameter1_value",
        "parameter2_name": "parameter2_value",
    }
}

Here is an example of using curl to call a webhook with parameters:

curl --fail -u 'user:password' -X POST /
https://<tenant>.antithesis.com/api/v1/launch_experiment/<endpoint> /
-d '{"params": { "param1_name": "param1_value", "param2_name":"param2_value" } }'

The webhook returns an HTTP status code to indicate success or failure at starting the test. This code does not contain your test results, which will be delivered asynchronously both via email and in a format appropriate for the integration type. For example, a "github" integration type will deliver an asyncronous response in the Github REST API for commit statuses to your Github callback URL.

Default webhook parameters#

The following default parameters can be used to configure any Antithesis webhook. Your consultant may create additional custom parameters if you want additional ways to parametrize your tests. All parameters are optional.

antithesis.report.recipients

Recipients who will receive emailed links to the triage report produced by this test run. If this parameter is not specified, emails will be sent to the default users set up for the endpoint.

A ‘;’ delimited list. Each entry is an email address.

E.g. u1@site.com;u2@site.com

antithesis.images

The image versions that Antithesis will use to build your test environment. The images are specified as an optional registry, a container name and either a digest (which is recommended) or a tag.

If a registry is not specified, the default registry configured for your tenant will be used (us-central1-docker.pkg.dev/molten-verve-216720/$TENANT_NAME-repository/). Images will by default appear in our simulated environment with tag latest, but you can reach out to your consultant if you require something else.

A ‘;’ delimited list. Each entry is in this format: [REGISTRY/]NAME[:TAG|@DIGEST].

E.g. container_1@sha25612341234;registry/container_2:latest_tag

antithesis.config_image

The image version that Antithesis will pull from the container registry for the config image. This should be a single image version formatted in the same way as those in the antithesis.images parameter.

antithesis.description

A string description of your test run. The description will be in the headers of the generated report and of any emails triggered by the test run.

antithesis.source

A source control descriptor used to filter history. (It is most common to filter based on branch.) The generated report will show the history of properties for runs with the same antithesis.source parameter, allowing you to (for example) easily compare tests run on a single branch.

The parameter is any text, but you will likely provide a source control descriptor.

The following parameters are automatically configured by integrations such as the Antithesis Github action. You will probably not set these parameters manually.

antithesis.integrations.type

The CI / Source Control integration type.

Values: ["none", "github"]. Default value: "none".

antithesis.integrations.callback_url

The URL Antithesis will call back to post the test results.

antithesis.integrations.token

A security token used to authenticate when calling back with results.