Querying with event sets
Event sets allow you to ask powerful questions about a moment’s history or future. An event set is an unevaluated query about the Antithesis multiverse.
These queries analyze a single timeline and return data in the form of events (see below) and include things like stdout
messages, SDK assertion statuses, and journald
messages.
An example
Here’s the scenario: we’re testing Chromium and believe it has a memory leak. We’ve already set up a naive memory monitor that logs percent memory usage by process name every few seconds as JSON.
First, we want to know when in the simulation Chromium’s memory usage is unacceptably high. To do this, we define the set of events where Chromium’s memory usage is above a threshold.
With this event set in hand, we can ask questions about both the past and future of a timeline in the multiverse.
In this example we use the Antithesis Notebook to ask:
A) In both the past and future of this timeline, what is the memory usage of Chromium? B) If we let the simulation continue forward, how long will it take for Chromium’s memory usage to pass our threshold?
To access the Notebook, see the workflow document.
// Step 1
memory_events = environment.events.filter(
(event) => event.source.includes('your_custom_memory_monitor')
)
parsed_set = memory_events.map((event) => {
const parsed = JSON.parse(event.output_text)
return {
process_name: parsed.process_name,
mem_usage: parsed.mem_usage,
}
})
chromium_memory_events = parsed_set.filter((event) => event.process_name === 'chromium')
// Step 2
high_memory_usage_events = chromium_memory_events.filter((event) => event.mem_usage > 50)
high_memory_in_past = high_memory_usage_events.up_to(moment)
print(high_memory_in_past)
// Step 3
future_branch = moment.branch()
future_branch.wait_until(high_memory_usage_events)
memory_events_in_future = chromium_memory_events.up_to(future_branch)
print(memory_events_in_future)
Let’s break this down:
Step 1: Parsing our data
Many things will be happening in the Antithesis simulation, but only some of them are interesting. We’re interested in a memory leak, so let’s focus on the relevant messages emitted by our memory monitor.
These messages are printed to stdout
and are therefore events. Most events have a field source
that contains information about the emitter of an event (including the container name). The source
field can be manually set by your event emitter. Let’s assume we used the $ANTITHESIS_OUTPUT_DIR
functionality (documented here) to set the source name for this memory monitor.
We can narrow our event set to relevant events by looking only at events whose source includes the string “your_custom_memory_monitor”:
memory_events = environment.events.filter(
(event) => event.source?.name?.includes('your_custom_memory_monitor')
)
We know that our memory monitor writes logs as JSON. These can be parsed:
parsed_set = memory_events.map((event) => {
const parsed = JSON.parse(event.output_text)
return {
process_name: parsed.process_name,
mem_usage: parsed.mem_usage,
}
})
Now we can access process_name
and mem_usage
as fields in our events and filter down further to only the relevant Chromium ones.
If you were doing this in a real-world scenario. It would be preferable to use our built-in JSONL support rather than parsing this yourself.
chromium_memory_events = parsed_set.filter((event) => event.process_name === 'chromium')
Step 2: Setting up conditions
After this setup, we can now ask a key question: has Chromium broken the memory threshold yet?
high_memory_usage_events = chromium_memory_events.filter((event) => event.mem_usage > 50)
high_memory_in_past = high_memory_usage_events.up_to(moment)
print(high_memory_in_past)
Here up_to
(see reference below) takes our abstract EventSet query and executes it. It returns a concrete list of Events (we call this an EventSequence) that we can render nicely with our sequence viewer.
Step 3: Execute and Analyze
It seems like it hasn’t yet, so we ask our second question: will we pass the memory threshold in the future?
We spawn a new branch to run a wait_until campaign. This tells Antithesis to keep running our software until an event that matches our query is found. That is, the simulation will run until it encounters the first event where Chromium is using lots of memory.
future_branch = moment.branch()
future_branch.wait_until(high_memory_usage_events)
We can watch Chromium’s memory usage grow over time on this branch, by printing all of this branch’s Chromium memory events:
memory_events_in_future = chromium_memory_events.up_to(future_branch)
print(memory_events_in_future)
Event set reference
There are a couple of common event sets:
environment.events
, which is a set of all commonly useful eventsProcess.stdout
, which is all text printed tostdout
by some process.Process.stderr
, which is all text printed tostderr
by some process.- And there are more event sets available as part of the environment.
Querying
up_to
is currently the only available query plan for event sets.
up_to
is an instance method on EventSet. It accepts a single positional argument. For simple queries this argument is a branch or moment.
up_to
returns an EventSequence containing events from the beginning of the simulation leading ‘up to’ the provided branch end or end moment. This EventSequence can be rendered by our sequence viewer.
If you’d like your results to start from a moment that is not the beginning of the simulation, you can provide a begin moment via an object of the shape {begin?: Moment, end: Branch | Moment}
as the sole positional argument to up_to
.
Here are a few examples:
print(environment.events.up_to(bug_branch))
// or to grab only part of a branch's history
print(environment.events.up_to({begin: start_moment, end: bug_branch})
// or to grab only particular kinds of events
print(environment.events.filter(event => event.output_text.includes("[WARN]")).up_to(bug_branch))
Like most values in the Notebook, this a reactive value that will update when the underlying answer updates, e.g. if a branch end is moving over time (due to a campaign), then streaming results are returned.
Creating other event sets
All event sets start from a root event set. In multiverse debugging this will typically be environment.events
.
From here you can build derivative event sets by chaining together the following instance methods
EventSet::filter
environment.events.filter( event => event.moment.vtime > 20 )
Accepts an arbitrary function from event to any value. Returns the EventSet containing events where the function returns a truthy value.
EventSet::map
// The function returns an object with two new fields
// So events_with_new_fields is environment.events with these two new fields
events_with_new_fields = environment.events.map((event) => {
const parsed = JSON.parse(event.output_text)
return {
process_name: parsed.process_name,
mem_usage: parsed.mem_usage,
}
})
Accepts an arbitrary function from event to object. Returns the originating EventSet with the object’s fields added to each event.
Contrary to what you might expect, map
will create these custom fields in addition to the existing fields in order to preserve canon fields.
EventSet::contains
worker1_warnings = environment.events.contains({output_text: 'warning', source: 'worker1'})
Accepts an object with optional keys output_text
, source
, or stream
and string values. Returns the EventSet representing events whose fields contain the string for every present key. This is case-sensitive.
EventSet::matches
server_events = environment.events.matches({source: 'my_db_server'})
Similar to contains. Accepts an object with optional keys output_text
, source
, or stream
and string values. Returns the EventSet representing events whose fields are exactly equal to the provided string for every present key. This is case-sensitive.
EventSet::union
concerning_events = sev1_events.union(sev2_events, sev3_events)
Accepts arbitrarily many event sets. Returns the EventSet containing events that are contained in any of the given event sets.
A full list can be seen in the notebook with help(environment.events)
.
Events
Event emitters
An event represents anything that happened within the simulation. The following will emit events:
- stdout/stderr: Standard output of processes inside your containers will emit events
- SDK events: Our SDK assertions communicate with the rest of our system using events. You may also generate custom events using the SDK’s
send_event()
method. - SDK capture directory: Files written to a special location will emit events.
- Antithesis environment: Systems like our fault injector, journald, and others will emit events. See their documentation for details.
Event fields
Events have the following fields (optional where noted):
-
event.moment
: The moment at which an event happened. -
event.moment.vtime
: The time that passed within the simulation along this timeline, measured in seconds. -
event.source
: The source that emitted an event. -
event.source.name
: The human-readable name of the system or process that emitted the event.- For
stdout
/stderr
and SDK assertion events it is your container name. - For the SDK capture directory, it is effectively the path into that directory and is documented here.
- For Environment event emitters
fault_injector
,journal
, andbackground_monitor
it is the that emitter name.
- For
-
event.source.pid
: Optional. The Linux process ID of the process that generated this event. PIDs are available for events emitted by the Antithesis SDK, or via writes to the SDK capture directory. -
event.source.stream
: Optional. Current values include"error"
,"info"
,"internal"
.- For
stdout
/stderr
this maps to"info"
and"error"
respectively.
- For
-
event.output_text
: Optional. The text-payload of the event.- For
stdout
events this will be the verbatim text, with one event per newline.
- For
-
event.exit_code
: Optional. If present, means that the Antithesis environment has terminated with an exit code.- This is an error in our setup of your system, contact your Professional Services representative or email
support@antithesis.com
.
- This is an error in our setup of your system, contact your Professional Services representative or email