If your code calls a paid API, say an LLM, you can't have those calls run in your automated tests. They cost money and they're slow. The common trick is to record the real network call once and replay it forever after. The tool saves the request and response to a file (a "cassette"), and your tests read from that file instead of hitting the network. No API key, no cost, and it's fast.
That's the plan. In practice it cost me three debugging sessions, because the recording tool has three behaviors that never raise an error. They just quietly do the wrong thing, and you only find out when you open the file and look.
Trap 1: the record command made zero calls
I ran the record command with a real API key set. It finished, no errors. But the test still failed on replay with a connection error, the same error you get when there's no recording at all. So... it didn't record? With a valid key right there?
The cause was one line in my config: record_mode="none". I expected the command-line flag (--record-mode=once) to win. But with this tool, it's the other way around: the setting in the config file overrides the command-line flag. So my flag did nothing. The tests had been stuck in "replay only" mode the whole time. They refused to make any call, then failed because the recording they wanted didn't exist yet.
The fix was to remove that line from the config and let the command-line flag decide. The worst kind of "winner" is the one nothing tells you about. Nothing logged "I'm ignoring your flag."
There was an earlier trap, too. A line at the top of the test file told the test runner to skip these tests entirely. So even after I fixed the flag, the record command still recorded nothing: you can't record a test that never runs. I had to remove the skip first. "It recorded nothing" had two different causes, and neither one raised an error.
Trap 2: the cassette recorded traffic I never asked for
Once recording finally worked, the cassette came out bigger than expected. Something else in my program was quietly making its own network calls in the background: a tracing tool sending usage data to its own server. The recorder can't tell your calls from its calls. It records every request the program makes. So my real calls were now mixed in with this tool's background traffic, all saved into a file I was about to commit.
The fix was to turn that background traffic off, early, before the tool starts up. Most tools like this have an "off" switch you can set with an environment variable. Set it at the very top of your test setup, before anything else loads. (If you set it later, it's too late: by then the tool has already read its settings and started sending.)
The wider lesson: a cassette records what your program does, not what you meant to do. Any library that quietly calls home in the background ends up on the recording unless you mute it first.
Trap 3: re-recording adds to the file, it doesn't replace it
This one looks like file corruption. Every time you re-record, the tool adds the new calls to whatever is already in the file. It doesn't clear it out and start fresh. After a couple of record runs, I had a cassette with six saved calls in a file that should have held two: an old leftover pair from an early run, plus the good ones piled on top.
The instinct is to just re-record cleanly, but that means another paid call, and it doesn't fix the "add instead of replace" behavior anyway. The real fix needed zero API calls. The cassette is just a text file. So I opened it, kept only the two calls I wanted, and saved it. You can edit it by hand like any other file.
The takeaway
Three failures, one root cause: recording tools have hidden behaviors that don't show up as errors. The config quietly beats the command line. Background traffic gets recorded alongside the calls you meant to capture. Re-recording piles on instead of replacing. None of these raised an exception. Each one just produced a cassette that was empty, polluted, or bloated, and let the test fail somewhere else, far from the real cause.
In every case, the fix started the same way: open the file and read it. Count the saved calls. Check whose web addresses are in there. Make sure it isn't stuck in "replay only." The cassette is the source of truth, and it's plain text. When a recording tool surprises you, the answer is usually sitting right there in the file, waiting to be read instead of guessed at.