About Antithesis POCs
The goal of our POCs is to prove that Antithesis can help you find and fix the bugs that give you nightmares – the kind that cause outages and correctness issues and tie up your best engineers for weeks.
We structure them to require minimal effort from your team, and we want to be up front about who’ll need to be involved on your side to make this maximally informative and effective for you.
Do I need a POC?
No!
Antithesis will strengthen almost any testing regime, by verifying correctness and availability under really strenuous conditions.
Many customers already trust Antithesis to improve software reliability and boost engineering velocity, and you can start a 60-day risk free onboarding program today.
How Antithesis POCs work
Our POCs are 2-week sprints.
We’ll work with your team to find, debug and regression test a known bug that your team had a hard time solving in the past – what we call a “brown M&M.” This will give you an accurate picture of what it’s like to use Antithesis, and how much we can boost your development velocity.
Most companies that engage in a POC not only find their known bug, but discover other, previously unknown issues in their software as well.
POC Structure
Two weeks, two phases, no complications.
System setup (2 to 4 days)
Most of this can be completed ahead of time.
- Before we start, pick a known bug to look for. See what makes a good one.
- At or before kick off, provide a containerized build of your system that includes your known bug.
- Lift-and-shift one or more of your integration tests or end-to-end (e2e) tests into Antithesis.
- Start testing!
Testing & debugging (the rest of the time)
- Start testing in Antithesis.
- Iterate on your testing until we find the bug (and probably some other issues as well).
- Use our advanced debugging tools to triage and debug.
- Upload a new version of the software with your fix.
- Re-test the new build to confirm your fix has worked.
Who needs to be involved?
You should make sure you have the right team lined up to make this happen. If this expertise already resides within your team, or if you can be assured of this support, you’re off to a great start.
| Expertise needed | What’s needed from them |
|---|---|
| Build, run, deploy | A lot of the setup work will fall on the build team; securing their bandwidth is crucial. Overall, you should plan 2-4 days of developer time depending on your system. |
| Cybersecurity | Provision SSO, check policies. You’ll likely need no more than a few hours from 1 person, but it’s essential. |
| Knowledge of the SUT (this is typically your team’s SWEs) | Design and iterate on testing. Triage test results and fix bugs discovered. You’ll need 1-2 dev-days to do the initial test, but then you can invest more time in triaging, fixing, and iterating. |
| Dependency owners | If your system has dependencies owned by other teams at the company, you’ll likely need their help. If they have pre-built mocks for their APIs (and they should!), they’ll just be doing basic support. |
What kind of known bug should we use?
The best place to look is your production bug tracker. Pick the worst, hardest-to-find, hardest-to-debug issue you can find. The more miserable it made you, the better it works as a “brown M&M”.
The bug you pick should have several of the following qualities:
- Finding the issue took a long time, because:
- The root cause was very far from the symptom, or
- It showed up very rarely in production.
- Reproducing/triaging the issue was difficult, because:
- It relied on specific timings that were hard to reproduce, or
- There were multiple potential causes.
- The issue re-appeared after being “fixed.”
- A clear indicator of the issue exists, such as a unique log line.
General classes of issue that tend to meet these criteria are:
- Race conditions
- Concurrency issues
- Execution timing issues
- Data corruption issues
What’s with the “brown M&Ms”?
Van Halen was notorious for requiring concert venues to sign a lengthy contract before they’d agree to play. The contract defined precise technical specifications vital to the safety of the band’s pyrotechnics-laden shows. Buried within it was a line specifying a bowl of M&M candies in the dressing room with all of the brown ones removed.
The “no brown M&Ms” clause was a quick test of whether the venue had actually read the contract carefully. A glance at the bowl of candy immediately told the band whether they needed to take a thorough look at the rest of the setup as well.
We’re not trying to stage a rock concert, but we believe this is a great, low-effort way to establish whether Antithesis will help you resolve bugs that you’ve previously found difficult or time-consuming to resolve.