Hi, it’s Carl again. You may remember me as the guy who taught Claude to use Antithesis.
Earlier this year, SQLite released (3.51.3), which fixed a longstanding bug in their Write-Ahead Logging (WAL) subsystem called the WAL-Reset bug. The bug had been hanging around since 2010, but the SQLite team had apparently been unaware of its existence until earlier this year (more on this below). As they wrote at the time:
“The bug is a data race with tight timing constraints. It is unlikely to occur in common use. The developers have never been able to reproduce the bug organically and had to add special testing logic to SQLite that deliberately triggers the circumstances of the bug in order to verify that the issue has been fixed.”
I was actually on a road trip with my girlfriend when I read about this, but I’m also a giant database nerd, so I was immediately nerd-sniped, hard. Bugs in SQLite, after all, are legendarily rare. Moreover, this sounded like just a perfect brown M&M: a known, challenging bug that we could track down with Antithesis (we’ve done this a lot in our POCs). On top of that, I’d recently shipped our skills for Claude.
So, sitting on a hillside on the Sunshine Coast, I whipped out my phone, and asked Claude to get to work. I had it get SQL 3.51.2 – still buggy – set up in Antithesis, and then instrument the code with a bunch of Antithesis assertions. You can see the instrumented version here.
Shoutout here to my incomparably beautiful home province of British Columbia.
Then I asked it to write a simple workload which exercised the WAL insert and checkpoint code. Notably, this is a completely generic workload. It just runs writes and checkpoints concurrently – things you’d expect to actually happen in production, all the time. The assertions are also generic to the bug, they’re all standard assertions you’d add to any database, things like “no lost committed writes” and “database is not corrupt” (called integrity check in sqlite).
We actually find that often, the simplest workloads find the hardest bugs.
On my first run, Antithesis caught the bug in 15 mins. Here’s the report. The part you’re looking for is:
Then I repeated the exercise with 3.51.3, with the same workload and Antithesis instrumentation. Sure enough, the run came back green.
I thought about this today because Tailscale just wrote an excellent blog post about resolving the uptime issues they’d experienced in 2025. Those issues were how the SQLite team discovered the WAL-Reset bug. Tailscale suffered 6 months of shaky uptime, then they and the SQLite team spent weeks hunting the bug, rolled out and rolled back a fix that broke something else, then had to wait two more months to see if the “real” fix (3.51.3) worked.
To root cause the issue, they had to write a new transaction logging pipeline in Tailscale, then shim in a new debugging tool for the virtual filesystem layer in SQLite. In Antithesis, this process isn’t quite down to a single click, but one click will give you a causality analysis that pinpoints the issue to within a fraction of a second, and deterministic, time-travel debugging that allows you to do what-ifs and destructive analysis.
As the Tailscale team wrote, “nobody wanted us to spend six months looking for bugs in SQLite. This was an immensely frustrating experience for both our customers and staff”.
Finding bugs like the WAL-Reset bug is excruciatingly difficult (perhaps even like crawling over broken glass) – but with rare and difficult bugs, the real torture can come when you’re waiting to see if your fix actually worked. I’ve worked on enough databases, and have experienced this myself many, many times.
So it was both sobering and uplifting to realize just how painful this bug had been in the wild. By giving agents the skills to use Antithesis, I’d just found and verified it in like an hour, from my phone, sitting under spruce trees in the sunshine. I knew our agent skills worked, but I had no idea they worked this well. If you have a gnarly database issue, call me.