Engineering Patterns
Reusable techniques for Rails, React, and the bits between.
The eval graded the answer key, not the document
An extraction eval stayed green for weeks while users watched their data get mangled. The cause: it graded output against our labels, not against the source.
Charge the quota when you deliver value, not when you accept the request
A visitor gets two free runs. Two dead links later they have used both and received nothing. Where you decrement a quota is a product decision in disguise.
The eval was grading a prompt we had deleted
Our record/replay eval could not run for two days and reported green throughout, because the scorer kept grading cached results from a prompt we had deleted.
Restorable but not shareable: hiding a token in history.state
A result that survives refresh and the back button but cannot be shared by copying the URL. A forgotten browser API, a header, and one deliberate deletion.
Three Ways VCR Cassette Recording Lied to Me
Three vcr recording behaviors that never raise an error: the config beats the CLI flag, ambient traffic gets taped, and re-records append instead of replacing.
Cross-provider LLM failover without losing observability
Every LLM provider goes down eventually, so the obvious move is to retry against another one. The naive rescue-and-retry works until your bill spikes and the logs can't tell you how often failover fired or what it cost. Here's the version that keeps the receipts.
