Layered bot defense for a public form: honeypot, Turnstile, email verification, and the pieces between

The form

We shipped a public data-request form. It lets users exercise their rights under data-protection law: access, correction, deletion, portability, and withdrawal of consent. The form is public and unauthenticated by design. A user might want to delete their data without first creating an account to ask for the deletion.

Public unauthenticated POST endpoints are exactly the kind of thing bots find within hours of landing on the public internet. Our submit handler:

• Persists a row to the database.
• Issues a one-time email-verification token.
• Sends a confirmation email to the requester.
• After verification, sends a notification to the team responsible for handling the request.

Each submit is a database write and at minimum one outgoing email. A motivated bot could trivially turn that into a denial-of-service against our team's mailbox, our email provider's sending reputation, and our database index churn. None of those are catastrophic individually, but all of them are bad weeks of cleanup if they ramp up unobserved.

Rails has a built-in toolbox for exactly this. There is a well-maintained open-source rate-limiting middleware for the Rack stack, well-known patterns for honeypots, and drop-in CAPTCHA replacements that do not require selling your soul to a large ad-network. The question was not whether to defend the form. It was how to layer the defense so the failure mode of any single layer does not take the whole thing down.

This is the layered defense we landed on. Each layer is cheap in different ways and catches a different class of attacker. The combination is the point.

Layer 1: Honeypot field

The cheapest layer goes first. We render a field bots will fill in but humans never see:

<div aria-hidden="true" className="absolute left-[-10000px] top-auto h-px w-px overflow-hidden">
  <label htmlFor="website_url">If you are human, leave this field blank.</label>
  <input
    type="text"
    id="website_url"
    name="website_url"
    tabIndex={-1}
    autoComplete="off"
  />
</div>

Three things that matter here:

1. The name looks like a real field, not like "do_not_fill_this". Naive bots auto-fill anything that looks like a real URL or email field. A field literally called "honeypot" is one a careful bot would skip.
2. It is positioned off-screen with absolute positioning, not display: none. Bots that respect display: none will skip the field; bots that do not will fill it. We want the latter group to fall in.
3. autoComplete="off" stops the browser's autofill from helping the user accidentally fail the check. The screenreader text "If you are human, leave this field blank" is a fallback for assistive technology, in case our off-screen positioning ever leaks into accessibility tooling.

The server side:

def create
  if params[:website_url].present?
    Rails.logger.warn("[form_submit] honeypot tripped ip_hash=#{ip_hash}")
    return head :ok
  end
  # ... rest of the create flow ...
end

Note the response: a 200 OK with no body. We do not render an error. We do not leave a row in the database. We do not send the verification email. From the bot's perspective, the form worked. It just does not get the email follow-up. That is a feature: a bot that gets 422 Unprocessable Content knows the honeypot is there and tries again with a smarter strategy. A bot that gets 200 OK and never sees an email may give up before it figures out why.

This catches roughly the bottom tier of automated form submissions: the ones that scrape <input> tags and POST whatever they find. It catches a surprising percentage. It also catches zero of the bots that bother to render the page in a real browser and skip non-visible fields. We need the next layer for those.

Layer 2: Min-fill-time

A real human takes time to read a form, type their request, and click submit. A bot that is already past Layer 1 (one that renders the page and skips off-screen fields) is still likely going to POST in milliseconds, not seconds.

On render, we capture the timestamp:

const mountedAt = useMemo(() => Date.now(), [])

On submit, we send it to the server alongside the form data:

await apiPost('/api/form_submissions', {
  ...formFields,
  form_mounted_at: mountedAt,
})

The server checks the elapsed time:

def mounted_long_enough?
  raw = params[:form_mounted_at]
  return false if raw.blank?

  mounted_at =
    if raw.to_s.match?(/\A\d+\z/)
      Time.zone.at(raw.to_i / 1000.0)
    else
      Time.zone.parse(raw.to_s) rescue nil
    end
  return false if mounted_at.nil?

  (Time.current - mounted_at) >= MIN_FILL_SECONDS
end

If a request arrives implausibly soon after the claimed mount time, it is rejected as too fast. The threshold is set conservatively low (well under the time a real user spends) but is enough to filter direct-POSTs that do not render the page at all. A determined bot would render the page, wait out the threshold, then submit. That is the kind of bot Layer 4 (CAPTCHA verification) is for.

The min-fill-time check has two failure modes worth thinking about:

• The clock can be tampered with. form_mounted_at is client-supplied, so a bot can simply send a mount time far enough in the past that the check passes. That makes this a coarse filter, not a real barrier. Layer 4 is what catches scripted bots, not Layer 2.
• The check is on the client too. We do an early client-side reject so a real user spamming submit-on-page-load gets a friendly rejection instead of a round-trip for the same answer. The server check is the security boundary; the client check is a UX shortcut.

This layer takes about 30 seconds to add and catches another 5 to 15 percent of bots, mostly lazy headless-script ones.

Layer 3: Rate limits

Rate limits are about graceful degradation under attack. Not about catching individual bad requests, but about putting a ceiling on how much damage any single attacker can do.

Per-IP rate limits are the obvious starting point but they are poor at this:

• They are trivial to defeat. Anyone with a residential proxy network can rotate through thousands of IPs.
• They false-positive on shared networks. Offices, universities, mobile carriers, VPNs.

We use a multi-key throttle that combines IP and user-agent, with a small per-identity cap per throttle window:

throttle("form_submit/identity", limit: PER_IDENTITY_CAP, period: PER_IDENTITY_PERIOD) do |req|
  if form_submit_path?(req)
    ua_digest = Digest::SHA256.hexdigest(req.user_agent.to_s.first(500))
    "#{req.ip}:#{ua_digest}"
  end
end

The discriminator is IP + SHA256(user_agent). A real user submits from one IP/UA pair. A bot rotating user-agents to hide is also rotating its discriminator key, but each unique pair is still capped at the per-identity limit. To push its total volume up, the bot has to manufacture a proportional number of distinct user-agents. That is not impossible, but it is a quantifiable cost we can monitor for rather than the unbounded "one IP can submit forever" of a UA-blind throttle.

We also have a global cap that catches coordinated wide-IP floods that no per-key throttle can see:

throttle("form_submit/global", limit: GLOBAL_CAP, period: GLOBAL_CAP_PERIOD) do |req|
  "global" if form_submit_path?(req)
end

One ceiling on total submissions per throttle window, across the entire surface area. A real product has bursts (a viral post, a press hit) but a sustained flood of requests inside a single throttle window is "we are clearly under coordinated attack" territory. When this throttle trips we emit a structured log line and fire an admin alert so a human can decide whether to flip a kill-switch:

unless defined?(@global_cap_subscribed)
  @global_cap_subscribed = true
  ActiveSupport::Notifications.subscribe("throttle.rack_attack") do |_name, _start, _finish, _id, payload|
    req = payload[:request]
    matched_name = req&.env&.dig("rack.attack.matched")

    if matched_name == "form_submit/global"
      cache_key = "admin_alert:form_submit_global:#{Time.current.utc.strftime('%Y%m%d%H')}"
      unless Rails.cache.read(cache_key)
        Rails.cache.write(cache_key, true, expires_in: 1.hour)
        AppMailer.alert(
          name: "FormSubmitFlood",
          severity: :critical,
          summary: "Global cap tripped on the public form submit endpoint (likely coordinated bot flood)",
          details: { ip: req&.ip, ua: req&.user_agent.to_s.first(200) }
        ).deliver_later
      end
    end
  end
end

Two things worth noting:

1. The unless defined?(@...subscribed) guard. Rails's dev reloader can re-evaluate the initializer; without the guard you end up with N copies of the listener firing N admin alerts per global-cap trip. Subscriptions do not auto-clean.
2. The cache-keyed dedupe. The rate-limiter notifies on every throttled request. If a flood is in progress, the global cap might trip 10,000 times in an hour. We do not want 10,000 admin emails. Tagging by hour keeps it to one alert per hour per fired-condition.

Layer 4: Bot-defense challenge (CAPTCHA)

The previous three layers catch about 95 percent of bot traffic without ever showing a human a CAPTCHA. The remaining 5 percent (bots that render the page in a real browser, fill the form correctly, wait realistic amounts of time, and rotate user-agents and IPs) needs a different approach.

One caveat up front: the CAPTCHA layer is feature-flagged. We switch it on in response to observed abuse rather than running it unconditionally, so the integration below describes the layer as it behaves when enabled, not a defense that is always live.

We chose a bot-defense provider whose widget is invisible by default. It only shows a challenge if its signals suspect a bot. The main reasons to pick a privacy-respecting provider over a legacy CAPTCHA service:

• UX: invisible by default, challenge only when signals warrant it.
• Privacy: no cross-site fingerprinting, no data sharing with an ad network.
• Cost: our provider's tier is free.

The integration is server-side only:

class BotChallengeVerifier < ApplicationService
  SITEVERIFY_URL = "https://bot-challenge-provider.example.com/verify"

  def call
    return failure_result(:missing_token, "Challenge token is missing.") if @token.blank?
    return failure_result(:misconfigured, "Secret key not configured.") if secret_key.blank?

    body = post_to_siteverify
    return body if body.is_a?(ServiceResult)  # network/parse failure

    if body["success"] != true
      log_failure(body)
      return failure_result(:invalid_token, friendly_failure_reason(body), data: { error_codes: body["error-codes"] })
    end

    if @expected_action.present? && body["action"].present? && body["action"] != @expected_action
      log_failure(body, reason: "action_mismatch")
      return failure_result(:action_mismatch, "Token was issued for a different action.")
    end

    ServiceResult.success(action: body["action"], hostname: body["hostname"])
  end
end

Three details that took longer to figure out than they should have:

Verification has to be server-side. Always.

The CAPTCHA widget gives the client a token. The token is meaningless on its own. The client could send any string. The real verification is a server-to-server POST to the provider's endpoint with the token plus the server-side secret key. The provider returns { success: true, action: "...", hostname: "...", ... }. The server is the one that decides the request is legitimate.

This is obvious in retrospect but it is surprisingly common to see "client-validated" CAPTCHA implementations in the wild: the client gets a token and the server trusts that the token's existence proves humanity. It does not. The whole point of the round-trip is the server verifying with the provider.

Action mismatch is a real attack vector.

Many bot-defense providers let you label tokens with an "action" (for example, login, register, data_request). The client widget mints a token for a specific action; the server verifies the token and checks that the action matches what it expected. Without the action check, an attacker could grab a token from a low-friction form (where the challenge is invisible and rarely triggers) and replay it against a high-impact form.

We keep the action labels in one constant per language instead of scattering string literals across call sites:

module BotChallengeActions
  DATA_REQUEST = "data_request"
  LOGIN        = "login"
  REGISTER     = "register"
  # ... one constant per protected action
end

And a TypeScript mirror with a comment cross-referencing the Ruby file. A rename forces both sides to update in the same commit.

Dev and test bypass via documented test keys, not environment detection.

The wrong way to handle bot-challenge verification in tests is if Rails.env.test? then return success. That makes integration tests skip the real server round-trip and silently diverge from production behavior. The right way is to use the provider's published test keys. Most providers document a pair:

• An "always pass" site/secret pair for use in dev and the success path of tests.
• An "always fail" site/secret pair for exercising the rejection path.

Both are real keys that round-trip to the provider's endpoint and return a deterministic response. Your integration tests run the same code path as production; you just pre-arrange the answer. CI exercises both keys to make sure the rejection path actually rejects.

Failed verifications go to logs, not the database.

When you ship a protected endpoint and the bots find it, the failure rate dwarfs the success rate. If you persist every failed verification to a database table, that table will balloon. Indexes become expensive to maintain. Backups get larger. Any slow query on an unrelated table starts taking longer because the database is busy with this one.

Logs are the right place. We emit a structured key=value line:

[bot_challenge_failed] ip_hash=ab12... action="data_request" error_codes=["timeout-or-duplicate"]

It is grep-able, parseable by any log aggregator, and costs nothing to drop on the floor when we are not actively investigating an attack.

Layer 5: Email verification round-trip

The last layer is probably the most underrated.

When a user submits the form, we do not immediately notify the responsible team. Instead, we:

1. Persist a row with status: "pending_email_verification".
2. Generate a one-time, short-lived verification token.
3. Email the requester a confirmation link.
4. Only after they click the link does the row transition to a received state, and only then does the team get notified.

This catches the class of bots that have cleared every previous layer (fully rendered the form, waited the right amount of time, passed the CAPTCHA, evaded rate limits) but used a fake or unreachable email address. There is a lot of this. Bots will plausibly fill in attacker+abc123@disposable-email-service.com. The email round-trip silently filters them all out.

It also doubles as anti-impersonation. A request to delete the data of alice@example.com should not fire just because someone typed that email into the form. Requiring a click-back from the actual alice@example.com mailbox is the minimum viable proof that the person submitting the request controls the mailbox. We do additional verification for high-impact requests, but the email round-trip is the floor.

The token lives on the model:

class SubmittedRequest < ApplicationRecord
  def self.generate_verification_token
    SecureRandom.urlsafe_base64(32)  # 256 bits of entropy
  end

  def verification_token_expired?
    return true if verified_at.present?    # already used
    created_at < TOKEN_TTL.ago
  end

  def mark_verified!
    return false if verification_token_expired?
    return true  if verified_at.present?

    update!(
      status: "received",
      verified_at: Time.current,
      received_at: Time.current,
      verification_token: nil
    )
    true
  end
end

urlsafe_base64(32) gives 256 bits of entropy. That is more than strictly necessary for a short-lived token, but it matches the entropy of every other one-shot token in the app, which keeps the security review simple. The token is stored in plain form (not a digest) because the email link has to be clickable. The single-use property combined with the rate limits on the verify endpoint bounds the replay risk.

The verified-page route returns Cache-Control: no-store so an intermediate proxy cannot cache the URL with the token attached:

def verify
  response.set_header("Cache-Control", "no-store")
  response.set_header("Pragma", "no-cache")
  # ... lookup + transition ...
end

And on the frontend, the route segment config nails it down:

export const dynamic = 'force-dynamic'
export const fetchCache = 'force-no-store'

The token is single-use server-side anyway, but defense-in-depth means you do not want the URL with a still-fresh token sitting in someone's CDN cache.

What each layer catches

Tabulating to make the design choice explicit:

Layer	Latency	Catches
Honeypot	Free	Naive bots that scrape <input> tags
Min-fill-time	Free	Headless scripts that POST without rendering
Per-identity throttle	One cache write per request	Bursts from a single IP/UA pair
Global cap	Same	Coordinated wide-IP floods (with admin alert)
Bot-defense challenge	One HTTPS round-trip to provider	Real-browser bots that pass everything else
Email verification	One outgoing email + a click	Bots with fake addresses; impersonation

No single row of that table is sufficient on its own. A motivated attacker can defeat any one of them. The cost of clearing all five (render the page in a real browser, wait out the min-fill threshold, present a real and reachable email address per request, respect the rate limits, and pass the bot-defense signals) is high enough that it stops being attractive compared to other targets.

What we deliberately did not do

• No legacy reCAPTCHA-style service. Privacy is a product value here; we would rather not have a large ad-network grading every page visit.
• No IP allowlists or blocklists. Too coarse, too easy to defeat with proxies, hostile to mobile users on shared carriers.
• No database table for failed verifications. Logs scale, database tables do not under attack.
• No CAPTCHA on high-volume analytics endpoints. Volume is high enough that even an invisible challenge would hurt legitimate traffic. We use per-visitor-ID plus IP rate limits, event-shape validation, and batching. Different threat model, different defense.

Lessons

1. No single layer is sufficient. Stop trying to find one.

The temptation when shipping a defended endpoint is to pick "the" CAPTCHA solution and call it done. That is how you ship rules that catch one class of attacker but miss four others. Layered defenses are uglier but they actually work: each layer catches a different class, and the combined cost-to-defeat is what stops the attack.

2. Each layer should be cheap to defeat individually, and that is fine.

Honeypot is trivial to defeat. Min-fill-time is trivial. Rate limits are trivial with proxies. Bot-defense challenges are trivial against well-resourced attackers. Email verification is trivial against attackers with disposable mailboxes. None of those facts argue against using the layer; they argue for combining them.

3. Failed checks should be silent, not informative.

When a layer rejects a request, the response should look as close to "everything worked" as possible. Honeypot: 200 OK, no row created, no email sent. Rate-limited: 429 with a Retry-After, no detail. Bot-challenge failure: a generic "verification failed" message, never the specific provider error code. Bots learn from informative responses. Humans do not see them.

4. Test keys beat environment detection.

if Rails.env.test? then bypass is a security anti-pattern. Your tests do not run the real code path, so a regression in the real code path goes uncaught until it is in production. Use the bypass mechanism your provider designs, the way they design it. Published test keys are exactly this.

5. Logs scale, tables do not under attack.

Whenever you are tempted to persist "every failure of X" to a database table, ask what happens during a sustained attack. If your failure rate is 10,000 per hour for a week, your table has 1.7 million rows of mostly-noise that you are paying to maintain forever. Logs handle this gracefully; tables do not.

6. The cheapest and most underrated layer is email verification.

It costs you one email per submission. It is invisible to humans. It silently filters out an enormous class of attacker (anyone using throwaway addresses) without a single false positive on legitimate users. If you are shipping any unauthenticated POST endpoint that triggers a notification or a state change, the email round-trip is almost always worth the latency.

Reading list

• OWASP: Automated Threats to Web Applications - the canonical taxonomy of bot-flavored threats, useful for thinking about which attacker class each defense layer is actually catching
• rack/rack-attack on GitHub - multi-key throttles and global caps
• PortSwigger: CAPTCHA bypass techniques - useful for understanding why a single CAPTCHA layer is insufficient and what attackers actually do