Instrumentation and code coverage

Introduction

The Antithesis Platform relies on two kinds of information about the software under test:

1. Coverage instrumentation tells the Platform which lines of code are reached over the course of testing. Our instrumentors can add coverage callbacks at every basic block in the program’s control flow and write the transformed source files to a separate directory. These callbacks enable the Antithesis platform to test your program more effectively.

2. Assertion cataloging emits a catalog entry for every assertion you add to your code using the Antithesis SDKs (like Always(), Sometimes(), Unreachable(), etc.). No additional code is being added to your software beyond the assertions themselves, but this does provide a vital source of information about your system’s behavior.

The way we do this is a little different for every language, but each way produces a common output formatting, enabling us to provide unified reporting across your entire architecture.

Some of our SDKs include an instrumentor package that facilitates this instrumentation, and we provide some filtering capabilities that enable you to control which portions of your code the instrumentor acts on. Please read the relevant language support documentation for details.

Be aware that when you instrument your code, you may need to install additional runtime dependencies into your containers, but the benefits are well worth it.

Benefits of instrumentation

• Instrumentation allows us to support assertions that refer to specific lines of code or conditions in your code.
• Instrumentation provides much richer data to the bug report, allowing us to automatically discover which lines of code were most and least associated with the bug.
• Instrumentation enables a powerful fault-injection technique called thread pausing which is useful for finding subtle race conditions and other concurrency bugs.
• Instrumentation sends continuous feedback to the Antithesis Platform which enables it to find bugs faster and more reliably.

Supported languages

The following is a list of languages that are supported for instrumentation and code coverage.

• Go
• C/C++
• Java instrumentation
• JavaScript instrumentation
• Rust instrumentation
• .NET instrumentation
• Other LLVM instrumentation

If your language is not on this list, it may still be possible to enable instrumentation. Contact us for more details.

Symbolization

Symbolization is the process of translating between locations in your compiled code and locations in your source files. For some languages, the Antithesis Platform handles all of this automatically. For other languages, we require your build/CI system to put symbol files into a specific location in order for symbolization to work.

Antithesis will search the root of every container image you push for a directory named /symbols and automatically extract any symbol files it finds there. When searching for symbol files, Antithesis will follow symlinks. In general, symbols are placed in the image containing the instrumented software. Howevever, on rare occasion the resulting images will be so large as to impact test performance. In this case, you can instead place the symbols in the config image.

Please consult the language-specific documentation for detailed instructions on enabling symbolization. Symbol files from multiple programs, containers, or microservices can be combined into the same config image.

Validation

You can confirm that the instrumentation process worked correctly by looking at your triage report. Specifically, under the Setup property group, there is a default property called Software was instrumented. If this property is passing, then your code was instrumented successfully and we were able to symbolize it as well. Each example listed under this property is a different binary, program, or module that was successfully instrumented. The details section for the examples will list the name of the module and the number of locations within it that were instrumented.