Pausing faults

There are times during a test when you need a window of normal operation, without faults. For example, you might want to test a system’s liveness properties, check an invariant, or check the system’s ability to recover from a failure. To allow this, Antithesis provides ways to pause fault injection during a test timeline or at the end of a test timeline.

Pause during a test timeline

Antithesis injects a ready-to-use binary on every container or pod path in a test run that lets you stop faults for a given duration in seconds. Invoke it from any test command or script:

[ "${ANTITHESIS_STOP_FAULTS}" ] && "${ANTITHESIS_STOP_FAULTS}" <duration_seconds>

The example given is in bash, but you can use any programming language.

A request to stop faults affects all fault types (except clock skips, which continue normally during a quiet period), and all containers or pods, so the entire system is granted a recovery period.

When you invoke the binary:

1. All fault injection stops and no new faults are scheduled.
2. The simulated network and killed containers are restored, but just like any container restart operation, restored containers will take some time to be fully operational.
3. Fault injection will automatically resume after the requested duration_seconds has elapsed.
4. Any overlapping quiet period requests will be merged to reflect the biggest interval.

Pattern: mid-run liveness check

A common workload pattern uses ANTITHESIS_STOP_FAULTS to assert an invariant in the middle of a timeline without giving up the rest of the test budget:

1. Run your workload operations while faults are active.
2. Call ANTITHESIS_STOP_FAULTS <SECONDS> with enough time for the system to stabilize.
3. Poll for health (retry reads until they succeed, wait for replicas to converge, etc.).
4. Assert your liveness property — for example, “all replicas converge to the same value”, “queued work eventually drains”, “every committed write is readable”.
5. Resume the workload. Faults will restart automatically when the quiet period elapses.

This pattern is particularly useful during rolling operations — upgrades, schema migrations, config rollouts — where you want to verify the system is healthy at each step before continuing.

Anti-patterns

• Don’t use a quiet period to hide flakiness. If a property only passes during quiet periods but fails during normal operation, that’s a real bug.
• Don’t assume restarted containers are immediately reachable. A quiet period restores killed containers but they need time to come up. Add retry loops to ensure successful restoration before your liveness check.

Pause at the end of a test timeline

To pause fault injection at the end of a test timeline, use an eventually or finally test command.

These commands create a terminal pause at the end of a test timeline, giving the system under test time to recover before final validation checks.