Charge the quota when you deliver value, not when you accept the request

Our public AI feature is free but limited: two runs per person, for life. The limit exists to keep a free thing affordable, and it is the first thing a returning visitor bumps into, so the message on the way out is friendly and points at the signup.

Here is a run that costs a person one of those two:

1. They paste their document and a link to a job posting.
2. We accept the request, decrement their allowance, and enqueue a job.
3. The job fetches the link. The site behind it blocks scrapers, which is most of the big job boards. The fetch returns nothing.
4. The job marks the run failed with "we couldn't read that link, paste the description instead".

The visitor did what we asked, got nothing, and is now one dead link away from "That's both of your free runs."

A reviewer flagged this as a conversion bug, and they were right, but I think the framing undersells it. It is a design bug, and it comes from a choice that is easy to make without noticing you made it: when to charge.

Charge-at-accept is the default because it is easy

Decrementing on accept is the natural place. You have the request, you have validated it, you know who the visitor is, and the counter is right there. It also protects you: nobody can enqueue a thousand jobs by racing the check before the charge lands (well, mostly; that is a different post).

The problem is that "accepted" and "delivered" are different events with a gap between them, and everything that goes wrong in that gap is charged to the visitor. In our case the gap contained:

• fetching a third-party URL that might be dead, paywalled or bot-walled;
• a content check on the fetched text (is this actually a job posting?);
• and only then the expensive model call.

Every failure before the model call is a run that cost us nothing and gave the visitor nothing. Charging for it is charging for the privilege of hitting our error path.

Charge-at-value, or refund-on-no-value

The clean version of the rule is charge when you deliver value. In practice that is hard to do literally, because the moment of value (the model returned a result) is inside a background job that has no request context, and moving the charge there reopens the racing problem at accept.

So we kept the charge at accept and added the mirror image: refund when the job fails before spending anything.

def perform(token)
  input = consume_input(token)
  description = resolve_description(input)   # fetch the link, or use pasted text
  return refund_and_fail(input, :link_unreadable) if description.blank?

  result = Scorer.call(input[:text], description)
  return refund_and_fail(input, :bad_request) if result.code == :bad_request

  complete(token, result)
end

Two failure classes get a refund, and the line between them and everything else is deliberate:

• Dead link. No fetch result, no model call, no cost to us. Refund.
• Pre-model content rejection. The fetched text fails the "is this a job posting" gate. Also before any spend. Refund.
• Model-side failure (rate limit, provider outage, malformed response). We may or may not have been billed, the visitor may retry, and the failure is ours to absorb, so we do not refund the per-person allowance for it; the daily backstops handle those.

The refund itself is a conditional decrement on the per-person row, with a floor at zero, and a small extra: if the refund empties the row, delete it. Otherwise a person who was refunded to zero still occupies a slot in the "first five hundred people" launch batch for six months, which is its own quiet unfairness.

Why not refund everything?

Because the charge is also a defense. If every failure refunded, an attacker could hammer the endpoint with inputs designed to fail late (a link to a slow host, say) and never spend a run. The rule "refund only when we spent nothing" keeps the incentive aligned: the visitor is never charged for our side's failures before cost, and never uncharged for something that already cost us.

That is also why the refund lives in the job, keyed by the same identity the charge used, rather than being a client-visible "undo" endpoint. The visitor cannot claim a refund; the system grants one when it can prove nothing was delivered.

The general rule

Whenever you decrement something scarce on behalf of a user (a free-tier allowance, a monthly credit, a rate-limit token), draw the timeline of the request and mark two points on it: accepted and value delivered. Then list what can fail between them and ask, for each failure, "who should eat this?"

Failures before you incurred any cost belong to you, and the honest response is to give the unit back. Failures after you incurred cost are a judgement call, and "the user retries, we absorb it" is usually the right one for small units. Failures the user could engineer to dodge the charge should never refund.

Most quota code charges at accept because that is where the code is easiest to write. That is a fine default. Just make sure someone has read the timeline and decided, on purpose, that a dead link should cost a person one of their two chances. We had not, and the fix was fifteen lines once we looked.