Spend less time worrying about bugs. Leave that to us.
Current approaches to software testing are outdated, inefficient, and costly, leading to reliability problems
and wasted engineering effort.
Antithesis is a completely new approach to testing and debugging.
Your team will spend less time testing and more time building your software.
Our platform continuously searches your software for problems, within a simulated
environment where every problem can be perfectly reproduced, enabling
efficient debugging of the most complex issues.
Find bugs right away
Antithesis is a cloud platform that continuously tests the reliability of software systems. As your developers write code, they push their software (as container images) to Antithesis. We run your software under a variety of conditions, and report any unintended behavior. We test your system against a multitude of predefined properties, and also offer an SDK for defining additional test properties specific to your system.
Catch bugs most tests miss
Each time your developers push new code, Antithesis starts running it and searching for problems. As your system runs, faults (such as network retries, thread hangs, or node restarts) are artificially injected, and your software's behavior is analyzed. When Antithesis notices interesting or rare behavior, it makes a copy of the system state and explores many possible futures from that point.
Any path that increases code coverage, or produces rare log messages, is more intensely explored. This branching happens many thousands of times in each test run.
Production systems are safe
Antithesis performs each test run in a simulated environment, which we provision and manage automatically. This environment contains your entire service architecture, and uses virtualization to simulate all hardware and network components in the system. Customer production systems are never involved, so intense fault-injection can be performed with no risk of downtime.
No more flaky reproductions
The Antithesis simulation environment is fully deterministic (meaning any state in a test run can always be reached again), so all problems it finds can always be reproduced. This means that there are no flaky tests, and that rare production mysteries can be pinned down and diagnosed. Not only is the moment of the problem recorded, every moment leading up to the problem can also be replayed. This remains true even weeks or months after the test run.
Painless debugging (you might even find it fun)
Antithesis's perfect determinism allows for truly unique debugging methods. These include the ability to arbitrarily rewind time from the moment of an incident, change inputs, examine system state, or enable additional logging. Moreover, counterfactual histories of the issue can be explored, like modifying the circumstances and seeing if the issue still occurs. This massively accelerates root-cause analysis and increases developer velocity.
You hate testing. You'll love us.
Bugs are hard to reproduce
Problems in complex systems often depend on particular network, timing, or hardware conditions. The same tests performed on the same system, under seemingly similar conditions, might never reproduce the problem, because not all contributing factors are known or controllable.
Perfect reproducibility
Our testing environment is deterministic, so all bugs found in Antithesis are always reproducible, no matter how complex or rare. This means your engineers don't have to spend weeks trying to reproduce a flaky bug or a rare production issue.
Unknown unknowns can't be found
Manually written tests (unit tests, integration tests, etc.) mostly find problems anticipated by the authors of the tests. But software systems have many problems not anticipated by their designers.
Unknown unknowns are discovered
Antithesis explores your software in novel ways, so it finds bugs that your developers didn't go looking for. This makes your software more reliable, because it won't be brought down by unforeseen situations.
Debugging is hard
Diagnosing and fixing bugs in complex systems is difficult, because of the non-deterministic nature of networks and hardware. Standard approaches require lots of skill (and luck) to reproduce a bug, and to identify its cause among the many components of the system.
Debugging is easy
Antithesis performs all testing in a deterministic environment, so the exact path that led to a bug always can be reproduced and analyzed. Instrumentation can be added at interesting points, and artifacts such as log files, core dumps, and backtraces can be retrieved at any point, from any component of the system. Even the most complex bugs can be rapidly diagnosed, leading to quick fixes.
Old bugs are hard to diagnose
A lack of proper testing can let bugs enter production, but sometimes they don't cause a problem for months or years. Once they act up, these latent bugs are notoriously expensive to diagnose, because so much has changed since they were introduced.
New bugs are easy to fix
Antithesis continuously tests new versions of your software, and bugs are found soon after they're introduced. Finding bugs right away makes them easier to fix, because you know what code is responsible. That means your team spends less time chasing obscure bugs, and more time building.
Fixes are uncertain
Even when a bug has been diagnosed and a fix applied, it can be difficult to verify whether the fix has really eliminated the bug, and difficult to be sure it hasn't introduced a new one.
Confident fixes
Antithesis can recreate scenarios similar to the one that led to the discovery of a bug. Once a fix has been applied, it can be tested in these scenarios to verify whether the fix actually eliminated the bug. We can also exhaustively test the fixed version of the software, to make sure no new problems were introduced.
Testing is time-intensive
Traditional software testing relies on valuable software engineers spending 20-40% of their time manually writing, maintaining and running tests.
Less time testing is more time building
Antithesis autonomously searches your software for problems, so your developers can spend less time writing and maintaining tests. Our powerful debugging tools make fixing the bugs we find much faster as well. This means your team can spend more of their time improving your software, and less of their time testing it.
End-to-end testing is infrequent
The most important test of a complex system is that the entire system functions together properly. Tests of entire systems are difficult and expensive to set up, so teams frequently test individual components instead.
End-to-end testing is the default
Antithesis tests your entire software system together, so all tests are end-to-end tests by default. This finds problems that only arise when multiple components interact. We find these problems early in the development process, saving time and averting outages.
Isn't this just like ___?
No. Here’s how Antithesis compares with traditional alternatives:
Automated testing
vs.
Antithesis
"Automated" tests are manually-written tests that get run automatically on a set schedule or on demand by a developer. Each test covers a fixed set of code paths, and running it more than once on the same software doesn't provide any additional benefit. If more coverage is wanted, new tests must be written. Automated tests are frequently brittle: when the underlying software changes, the tests must be updated.
Antithesis autonomous testing explores a different set of code paths every time it's run. If more coverage is needed, simply run the tests for longer, spending more computer time instead of more human time. The tests will cover situations that were not anticipated by any developer, increasing the chance that they uncover "unknown unknowns" before they're encountered in production. Since the test assertions cover high-level properties of the system, they usually don't need to be manually updated with new versions of the software.
Unit testing
vs.
Antithesis
Unit tests are tests of self-contained software functionality. Unit tests run quickly and are usually reproducible and easy to debug. Unit tests do not check whether an entire system works properly together and cannot find the vast majority of bugs.
Antithesis tests your whole system under a wide variety of environmental conditions and use patterns. It resembles end-to-end testing, but with the perfect reproducibility and easy debugging of unit testing. Existing unit tests can contribute to an Antithesis test template.
End-to-end testing
vs.
Antithesis
End-to-end tests validate that all of the components of your system work together. However, they are frequently flaky or brittle, because low-level network and timing conditions can violate implicit assumptions built into the test. This causes developers to neglect them, which is unfortunate because they are the most powerful kind of traditional test. Moreover, this same flakiness leads developers to design end-to-end tests of the "happy" path, without network faults, retries, or other real-world complications.
Antithesis build an ever-evolving collection of end-to-end tests that are perfectly reproducible (never flaky), and always finding new conditions or situations to test. Antithesis combines the coverage of end-to-end tests, the easy debugging of unit tests, and the flexibility and power of property-based testing.
Chaos testing
vs.
Antithesis
Chaos testing consists of testing software systems by intentionally introducing hardware and network failures in production environments. The goal is to identify problems that may occur, and to harden the system against common failure scenarios.
Antithesis testing resembles chaos testing, in that it injects faults to trigger and identify problems. But Antithesis runs these tests in a fully deterministic simulated environment, rather than in production. This means Antithesis testing never risks real-world downtime. This in turn allows for much more aggressive fault injection, which finds more bugs, and finds them faster. Antithesis can also test new builds before they roll out to production, meaning you find the bugs before your customer does.
Finally, Antithesis can perfectly reproduce any problem it finds, enabling quick debugging. While chaos testing can discover problems in production, it is then unable to replicate them, because the real world is not deterministic.
Fuzzing
vs.
Antithesis
Fuzzing consists of sending random inputs into a software system in order to trigger undesirable behavior. Guided fuzzing implicitly assumes that the software-under-test is deterministic or close to deterministic. This means that fuzzing is very difficult to apply to concurrent, stateful, or interactive systems. Fuzzing is generally only practical for self-contained libraries and processes, making it useful for testing simple programs for security vulnerabilities, but limiting its applicability to the real world.
Antithesis is uniquely able to use fuzzing-like techniques to test complex distributed or interactive systems. This is possible because Antithesis can force any collection of software to act like a deterministic pure function. This means that autonomous testing techniques, including those used by conventional fuzzers, can be applied to many more systems.
Property-based testing
vs.
Antithesis
Property-based testing (PBT) uses random inputs to check individual data structures, procedures, or occasionally whole programs for high-level invariants or properties. Property-based testing has much in common with fuzzing—the main differences are heritage (PBT comes from the functional programming world, while fuzzing comes from the security/systems programming world) and focus (program functionality vs. security issues). Like fuzzing, PBT is generally only applicable to self-contained libraries and processes.
Antithesis is analogous to applying PBT to an entire interacting software system—including systems that are concurrent, stateful, and interactive. Antithesis can randomly vary the inputs to a software program, and also the environment within which it runs. Like a PBT system, Antithesis is designed to check high-level properties and invariants of the system under test, but it can do so with many more types of software.
Best of all, you can see it today
We built a toy distributed database integrated with ourSDK,
and use our
GitHub Action
to run Antithesis on it every night.
We put a bunch of bugs in our demo system on purpose. The good news is Antithesis found all of those. The
better news is it also found other ones we didn't know about.
Antithesis automatically generates reports summarizing all the bugs it found.
Check out what it looks like for the demo system.
Antithesis is an exciting and unique tool for debugging blockchains and distributed systems. We used it extensively when testing the Merge. Antithesis was able to deterministically explore and find bugs in very exotic states and scenarios, ones that would have been nearly impossible to hand-code and unlikely to be hit in less stateful, traditional fuzzing.
This investigation has exceeded my expectations and has had a concrete influence on our release schedule for our annual release. This is the kind of thing we'd often take a month+ to solve with all hands on deck, getting there in a week improves our release confidence and saves me 3+ weeks of some of my most expensive engineers. I'm impressed, really well done.
Testing Stardog with Antithesis gives us increased confidence that unanticipated execution paths are being explored and cleared of bugs before they impact our customers, and their ability to perfectly reproduce every bug found has made diagnosing and debugging issues much faster for our team.
We could hire 100 distributed systems engineers and make them write integration tests for an entire year, and they probably wouldn’t be able to trigger all the interesting states and behavior that a single Antithesis run covered in 6 hours of wall clock time.
Antithesis's deterministic testing framework, featuring state space exploration and sophisticated fuzzing techniques, empowers us to rigorously examine and challenge uninstrumented programs in a methodical and predictable manner. As we embark on this collaboration, our aim is to significantly expedite the process of identifying and resolving bugs, ultimately enhancing the dependability and performance of our platforms.
If you want to get in a flame war with us, mention @AntithesisHQ on Twitter𝕏.
And if you want pictures of our beautiful office dog 🐕 Apollo, see our Instagram.
We have offices in Northern Virginia (US) and Cambridge (UK). If you're in our neck of the woods, contact us with one of the above methods and we can try to set up an office visit. We love chatting in person 😊
