Build and run an etcd cluster

Before you start, please make sure you’ve acquired a container registry and credentials. If you get stuck, please don’t hesitate to contact us at support@antithesis.com or on Discord.

For this tutorial, you’ll need Docker and Helm installed.

You’ll also need a local Kubernetes cluster. We recommend K3s for parity with our environment. Kind or minikube also work for local testing.

Here’s the source code for this tutorial.

What is etcd?

Etcd is a strongly consistent, distributed key-value datastore.

We’ll set up a 3-node etcd cluster in a local kubernetes environment, then deploy and test it in Antithesis.

Create manifests

Create a working directory

$ mkdir etcd-antithesis && cd etcd-antithesis

Add the Bitnami Helm repo

$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo update

Create a `values.yaml` file

These values configure a 3-node etcd cluster for the Antithesis environment. We’ll use this file to template the Helm chart.

fullnameOverride: "etcd" # Ensures pods have predictable names like etcd-0, etcd-1, etcd-2
replicaCount: 3
image:
  repository: bitnamilegacy/etcd
  tag: 3.5
preUpgradeJob:
  enabled: false
persistence:
  size: 1Gi
auth:
  rbac:
    create: false
  client:
    enableAuthentication: false
  token:
    enabled: false

Template the helm chart

Use the values.yaml file to render the kubernetes manifests.

$ mkdir -p manifests &&
helm template etcd bitnami/etcd -f values.yaml > manifests/etcd.yaml

Start the cluster locally

Apply the generated manifests to your local Kubernetes cluster to confirm they start a healthy etcd cluster. This approximates how the setup will behave in Antithesis.

Apply your manifests

$ kubectl apply -f manifests/etcd.yaml

note

We use kapp to deploy manifests reliably without having to worry about ordering or dependencies. You can use the following to apply your manifests with kapp:

$ kapp deploy -a etcd -f manifests/ --yes

Check rollout status

$ kubectl rollout status statefulset/etcd --timeout=3m
$ kubectl get sts etcd
$ kubectl get pods

You should see 3/3 pods become ready, named etcd-0, etcd-1, etcd-2.

Package your manifests/ directory into a minimal scratch image called the config image. Before tests run, Antithesis pulls the image, extracts /manifests, and applies the YAML files to run your system.

Create a Dockerfile

Create a Dockerfile.config alongside the manifests/ folder:

FROM scratch
COPY ./manifests/ /manifests/

Build the config image

$ docker build -f Dockerfile.config -t us-central1-docker.pkg.dev/molten-verve-216720/$TENANT_NAME-repository/etcd-config:k8s .

Push to the registry

When you become a customer, we configure a container registry for you and send you a credential file $TENANT_NAME.key.json.

To authenticate to your container registry, run the following command:

$ cat $TENANT_NAME.key.json | docker login -u _json_key https://us-central1-docker.pkg.dev --password-stdin

Now you’re locally authenticated to the registry and can run all other Docker commands as normal.

Push your custom and non-public images (including your config image) to: us-central1-docker.pkg.dev/molten-verve-216720/$TENANT_NAME-repository/:

$ docker push us-central1-docker.pkg.dev/molten-verve-216720/$TENANT_NAME-repository/etcd-config:k8s

Images that are publicly available (e.g. docker.io/bitnamilegacy/etcd:3.5) can be referenced directly in your config files, you do not need to copy them into the Antithesis registry.

Run your first test

You can now run your first test in Antithesis!

In the web app, go to the “Test launchers” page at https://$TENANT_NAME.antithesis.com/test-launchers, and select Basic_k8s_test. Enter a duration of “15” and click “launch.”

You can also use our webhook to do this from command line.

Since you’re just learning the ropes here, we’ll set Antithesis up to test for 15 minutes, but once you’re up and running, you’ll want to do longer test runs.

View your report

To check on the progress of your run, go to your Runs page at https://$TENANT_NAME.antithesis.com/runs. You should see your current run along with its status.

When your run completes, click the Triage results button to see your triage report, or click the link in the email you receive. You can also get results through Slack or Discord as well as email.

You’re now ready to exercise your cluster in part 2 of the tutorial!