A new API let clients send up to 1,000 items (diamonds, jewelry, or gemstones) in one POST request. For each item it created a database record, and every record triggered extra callback work. On top of that, the query was pulling every column (SELECT *) instead of just the fields it needed. Together this made the API take about 7 minutes to respond, long enough to freeze the frontend.
Make the API fast enough to handle 1,000 items in one request without freezing the page.
I made four main changes: 1. inserted the records in batches instead of saving them one at a time 2. moved the per-record callback work into a background job using Sidekiq, so it no longer blocked the response 3. rewrote the query to select only the fields it actually needed instead of every column; and 4. added database indexes to speed up the lookups.
The API's response time dropped from about 7 minutes to about 7 seconds ( roughly 60x faster ) and the frontend no longer froze when submitting large batches.
Key takeaways
- Use batched inserts instead of one-by-one
- Move expensive operations to background jobs
- Selecte only required fields (no SELECT *)