I found a bug, now what?

If you see a property failure, or an error line in your logs, how do you triage and debug it?

Antithesis offers three tools to help with debugging.

• The logs explorer gives you full access to every execution path.
• Causality analysis tells you which events made the bug more likely.
• The Multiverse Debugger lets you test your theories about the bug’s root cause against the actual system state at the time of the failure.

This page helps you decide which one to reach for, and when.

Start with the failure

Open the failing property in the triage report and pick one counterexample. This gives you a concrete moment in the test run when a failure happened.

Read the logs

Start as you would with any bug, by reading the logs around the failure. You’ll find them in the logs explorer, or by using the event set search endpoint or with a Snouty command.

You can query the logs to understand the failure in the context of events in the environment (such as faults), assertion evaluations, and more.

Often this is enough to identify a probable root cause — an error a few seconds before the failure, or a particular fault event. If the cause isn’t obvious from just looking at the logs, you can choose one of the two paths below.

No obvious suspects?

When the logs don’t narrow things down, you can run a causality analysis to search for events that increase the probability of the failure happening are presented in the analysis.

This might be enough information to pinpoint the root cause, but if you need to dig deeper into system state, the Multiverse Debugger’s got you.

Check your theories in the simulation

With multiverse debugging, you can jump to a given moment in the simulation, where you have access to the exact system state at any point leading up to the failure. You can pull core dumps, change events, or run other destructive commands to test your theories.

Check your fix

Once you’ve made a fix, run another test to verify it. When checking a fix, we suggest not considering the ticket closed until you’ve gotten a clean nightly run, not just a quick test.

The decision tree

Here’s a quick visual representation of the decision tree to use when you find a failure and don’t know what to do next.