About Antithesis POCs
This guide helps you understand the effort required for an effective proof-of-concept (POC) with Antithesis. The effort needed varies depending on your system and your goals, but this should get you a reasonably solid estimate.
First things first
Antithesis will strengthen almost any testing regime, by verifying correctness and availability under really strenuous conditions.
What’s your goal for this POC?
Most teams considering a POC with Antithesis want to accomplish one of three things:
- Proof-of-technology to see if your system works in Antithesis. Once that’s established, you can expand and iterate on testing at your own pace. This can be done with little effort from your team.
- Investigate a specific bug or property. Teams often come to us because they’re bothered by a flaky bug. If we know what properties or bugs you’re looking for, we can scope network topology, system architecture, versions to test, and the kind of exercise the system will need. This will require some effort from your team.
- Find unknown unknowns. Antithesis is a great way to do this, but you’ll need to be prepared to go through several cycles of iteration, develop a strong set of tests, and integrate our SDK with your code. Teams using Antithesis routinely do all these things, but to get this far during a 1 month POC requires committed effort.
Who needs to be involved?
You should make sure you have the right team lined up to make this happen. If this expertise already resides within your team, or if you can be assured of this support, you’re off to a great start.
| Expertise needed | What’s needed from them |
|---|---|
| Build, run, deploy | A lot of the setup work will fall on the build team, securing their bandwidth is crucial. Overall, you should plan between 2-3 days to 2-3 weeks of their time depending on your system. We explain how to estimate this below. |
| Cybersecurity | Provision SSO, check policies. You’ll likely need no more than 1 dev/day of time, but it’s essential. |
| Knowledge of the SUT (this is typically your team’s SWEs) | Design and iterate on testing. Triage test results and fix bugs discovered. You’ll need 1-2 dev/days to do the initial test, but then you can invest more time in triaging, fixing, and iterating. |
| Dependency owners | If your system has dependencies owned by other teams at the company, you’ll likely need their help. If they have pre-built mocks for their APIs, then they’re just doing basic support. Otherwise, you’ll need to work with them to design and build mocks. This takes time but is a real investment in the company’s ability to iterate faster. |
What will it take to make this happen?
You’ll need three things to bring your system under test in Antithesis:
- System setup
- A client to exercise the system
- Sample data (probably)
You can work on these in sequence or in parallel.
System setup
Broadly speaking, you should expect effort required to scale with the complexity of your system and dependencies.
If both your software and dependencies are containerized, setup will take one developer with the necessary expertise less than a week. If a lot of dependencies need to be mocked or the system itself needs to be containerized, plan on setup taking 2-3 weeks. Detailed documentation on containerization and container orchestration in Antithesis can be found here.
If you can share a list of your dependencies, we can give a more solid estimate of the effort required. Detailed documentation on dependency handling is here.
| System characteristics | Effort to expect |
|---|---|
| Fully containerized system, no dependencies | 1 day of work from someone who has the necessary expertise. |
| If all services are packaged in a single container | Add 2-5 developer/days to separate them into individual containers |
| A few open source dependencies, already available on DockerHub | Add 1-2 dev/days of work. |
| Dependencies that need to be emulated e.g. AWS, GCS, Azure (through use of technologies like LocalStack, MinIO, etc.) | Add 1-2 dev/days of work. |
| Dependencies that need to be mocked or stubbed, e.g. an API service provider, whether internal or external | Varies with complexity. |
Client
You’ll need a client that exercises the functionality of your system. How complex this needs to be depends on what your goal for the POC is.
| Your goal | Effort to expect |
|---|---|
| Proof of technology | Minimal effort post-setup. Hours at most. |
| Investigate a specific bug or property | We’ll start with a “hello world” client, then work with you to adapt it to bring the relevant part of your system under test. You’ll use our SDKs in the client but probably not in your codebase. Generally 1-3 dev/days from someone who knows the system well. |
| Find unknown unknowns | Requires a deeper understanding of system architecture so that we can optimize how Antithesis explores your state space. Necessitates using our SDKs in your codebase as well as in the client. 1-2 weeks of work from someone who knows the system well. |
Sample data
If you’re testing system correctness, you’ll need sample data. Your team likely wants to use one of three approaches:
| Desired approach | Effort to expect |
|---|---|
| Dummy data from an internal DB | Test DBs under 1GB can simply be imported to our system, which should take an hour at most. Larger test DBs will need trimming, with effort depending on the complexity of the DB. |
| Use an existing random data generator | Adapting an existing data generator to use Antithesis’ sources of randomness will take 1-2 dev-days. |
| New, Antithesis-native, data generator | Will depend on desired complexity. |