Assertions in Antithesis
This page offers an overview of how assertions work in Antithesis, and high level guidelines on how to add them to your system and test templates. Languages with first-class assertion support are listed below.
Assertions express properties
Properties describe how your software should behave, in terms a person would use.
You express properties in software by adding assertions to the code, using one of our SDKs.
Evaluating assertions
Antithesis evaluates each assertion as it’s encountered at runtime, to see if the assertion holds in that particular execution history. Each such evaluation generates an example or counterexample, which ultimately serves to demonstrate whether the property being asserted passes or fails. The triage report summarizes your software’s behavior as a list of properties which either passed or failed on the relevant test run.
The message parameter
Assertions aggregate to properties via the message parameter.
All Antithesis assertions require a message parameter, which appears in the triage report as the name of a property – so each of the “Control service” properties in the example below is a message in an assertion.
Antithesis generates one test property per unique message. If you have multiple assertions with the same message, Antithesis labels the property as passing or failing by aggregating every assertion with the same message.
We therefore strongly recommend that message should be a human-readable description of the property being expressed.
If you change the message in a given assertion between test runs, you’ll lose the history of that assertion. Conversely, if you retain the message but move the assertion, you’ll still be able to track its history across runs (modulo the effects of your moving it).
Assertions are clues
In addition, assertions provide hints to our platform about which states should be explored in order to search for bugs. This is an important form of guidance for our platform and makes its exploration of your software much more efficient – in other words, using assertions ensures you get the best bang for your buck in testing.
Performance and failures
All the Antithesis assertions you add to your code (and indeed all our SDKs) are designed to be able to run safely in production with minimal impact on your software’s performance.
Unlike many assertions libraries, failed Antithesis assertions do not cause your program to exit. This is useful in test, because we may be able to escalate an assertion failure into a more severe bug. It also reduces the friction to running the exact same code in test and production.
Under the hood
Under the hood, the assertions added using the SDK each send two JSONL messages to Antithesis:
- One message for assertion declaration in the assertion catalog at startup,
- One message for assertion evaluation when encountered during execution.
Assertion Cataloging
The assertion catalog is the collection of all assertion declarations made at startup.
The assertion catalog initializes every assertion in your software and test template, and informs Antithesis that they exist. Remember that in any given execution history, Antithesis may not encounter every assertion you make. Since many types of assertions are supposed to fail if they’re never reached, Antithesis must be informed that that these assertions exist so it can mark them as failing when this happens.
Our SDKs handle assertion cataloging for you, though the precise way this happens differs from language to language – some SDKs include an instrumentor package. Please read the section on instrumentation or our SDK documentation for details.