The four ceilings an api rate limit integration has to respect

An api rate limit integration is one built with the explicit assumption that the far end will, at some point, refuse to answer. That refusal is not a fault. It is the vendor protecting a shared platform from a single tenant consuming more than its share. The mistake is to treat throttling as an exception path bolted on late, rather than as a property of the system that shapes the schedule, the queue design and the acceptance criteria.

Limits are not all the same shape, and the shape determines the remedy. A fixed-window limit forgives you at the top of the next minute. A concurrency limit does not care how many calls you make per hour, only how many are in flight at once. A daily quota cannot be waited out inside a working day. Confusing one for another produces retry logic that makes the situation worse rather than better.

In practice you will meet four families, often more than one on the same platform:

  • Fixed or rolling window — A count of calls permitted per unit of time. Xero, for example, publishes a per-minute ceiling per connected tenant alongside a daily ceiling per tenant and an application-wide per-minute ceiling. Exceeding the minute limit is recoverable in under sixty seconds; exceeding the daily one is not.
  • Token or leaky bucket — A bucket refills at a steady rate and each call costs one or more tokens, which permits short bursts and a sustained lower average. Amazon Seller applies this per operation per seller, and some operations sustain only a fraction of a request per second with a small burst allowance. Shopify applies a query-cost variant where the price of a call depends on what you asked for.
  • Concurrency — A cap on simultaneous in-flight requests rather than on their frequency. NetSuite governs web services this way at account level, so ten parallel workers will trip a limit that the same volume of work run sequentially would not.
  • Rolling daily or entitlement quota — A budget consumed over twenty-four hours, frequently tied to licence count or subscription tier, as with Salesforce org-level request allowances and the tiered daily caps that apply to HubSpot applications. These are capacity-planning problems, not retry problems.

Read the response properly before you write a single retry

The first discipline is to classify the response accurately. HTTP 429 means throttled and is almost always safe to retry. HTTP 503 with a Retry-After header often means the same thing. HTTP 500 may be a transient fault or a genuine data problem that will fail identically on every attempt. HTTP 400 and 422 are your fault and retrying them simply burns quota you will need later. An integration that retries everything uniformly will spend its remaining allowance re-sending requests that were never going to succeed.

Where a platform tells you what it is doing, believe it over your own arithmetic. Retry-After, whether expressed in seconds or as an HTTP date, should override any calculated delay. Amazon Seller returns the current rate for the operation in a response header, which is more reliable than the published documentation because limits are sometimes adjusted per seller. Shopify returns the state of the call bucket on each response, so a client can slow down before it is refused rather than after. Reading these signals turns throttling from a collision into a negotiation.

Retries and idempotency are inseparable. A timeout is ambiguous: the request may have been rejected at the edge, or it may have created an invoice and lost the response on the way back. Any retried write needs a deterministic key so the receiving system can recognise the repeat, whether that is a native idempotency key, an external reference field on the document, or a pre-flight lookup by your own order number. Without that, aggressive backoff produces duplicate postings, and duplicate postings in a finance system cost far more to unwind than a delayed sync.

Finally, cap the attempt count and the total elapsed time. An unbounded retry loop against a daily quota that reset six hours ago is not resilience, it is a queue that never drains. Five attempts over roughly ten minutes is a reasonable default for a per-minute limit. A daily quota breach should stop the worker, record the reason and schedule a resume, not keep hammering.

Backoff that behaves under load, not just in a unit test

Exponential backoff with jitter remains the correct default. Doubling the wait after each failure gives the far end room to recover; randomising the wait prevents a fleet of workers, all throttled at the same instant, from returning in perfect unison and repeating the collision. Full jitter, where the delay is a random value between zero and the current exponential ceiling, is the variant that behaves best when several processes share a quota. The distinction matters as soon as you run more than one instance, which is to say as soon as you deploy for real.

Backoff alone is reactive, however, and reacting is expensive because every 429 is a round trip you paid for and gained nothing from. The stronger pattern is a client-side limiter that refuses to exceed the known ceiling in the first place. A token bucket held in shared state, typically Redis, lets every worker draw from the same allowance and keeps aggregate throughput just under the published rate. Where the limit is a concurrency cap, as with NetSuite, the equivalent control is a semaphore that bounds in-flight requests regardless of how much work is queued behind it.

Add a circuit breaker for the case where the far end is not throttling but genuinely unwell. After a threshold of consecutive failures, open the circuit, stop calling for a defined cool-down, then admit a single probe request before resuming normal traffic. This protects your own infrastructure as much as theirs: a worker pool blocked on thirty-second timeouts consumes memory and connections that the rest of the estate needs.

Two further details separate a design that holds up from one that merely passes a demonstration. First, queue depth must be visible; a backlog that grows steadily during the working day is a capacity problem that backoff will never solve. Second, priority lanes should be explicit. A customer-facing stock update and an overnight historical backfill should not compete for the same tokens, and the backfill should be the one that yields.

Shaping the workload so the ceiling is rarely reached

Most throttling problems are really volume design problems. An integration that polls a Shopify store every minute for all orders, then fetches each order individually to check whether anything changed, can consume several hundred calls an hour to discover nothing. The same outcome is available from a webhook subscription plus a periodic reconciliation sweep, at a fraction of the cost. Where webhooks are unavailable or unreliable, filtered queries using a modified-since timestamp will cut the volume by an order of magnitude compared with a full enumeration.

Batching is the next lever. Many platforms accept composite or bulk operations that cost a single unit of quota for multiple records, and some offer an asynchronous bulk export that removes the work from the interactive limit entirely. Where a platform charges by query cost rather than call count, as Shopify does for its GraphQL surface, requesting only the fields you actually map is a direct reduction in consumption. Asking for the full object graph out of convenience is a habit that becomes expensive at scale.

Reference data should be cached with a deliberate lifetime. Tax rates, nominal codes, warehouse identifiers, currency lists and product attributes rarely change during a business day, yet they are commonly re-fetched on every transaction. Caching them for an hour, with an explicit invalidation path, frequently halves the call volume of an order-posting integration. The corollary is that the cache must be documented at handover, because an operations manager who changes a nominal code and sees no effect for fifty minutes deserves an explanation that exists in writing.

Finally, do the arithmetic during discovery rather than after go-live. If a daily quota is five thousand calls per tenant and the current order volume is four hundred orders a day, the budget is roughly twelve calls per order across every process touching that tenant, including retries, reconciliation and any reporting job someone added later. Write that figure down. It tells you immediately whether the design has headroom for the peak trading week, and it is the number that determines whether a second connected application can share the same tenant without either of them failing.

What to specify, test and hand over

Rate limit behaviour belongs in the scope document, not in the developer's head. A fixed scope should state the ceilings assumed for each platform, the expected peak volume, the calculated headroom, the retry policy with its maximum attempts and elapsed time, and the behaviour when the quota is genuinely exhausted. That last item is a business decision rather than a technical one: does the integration queue and resume, does it alert and hold, or does it degrade to a reduced set of fields? Operations and finance should answer that question before the build begins.

Acceptance testing must include throttling explicitly. It is straightforward to simulate 429 responses in a test harness and confirm that the connector waits, honours Retry-After, resumes without duplication and reports accurately. It is equally worth running a deliberate burst against a sandbox to observe real behaviour, because vendor documentation and vendor implementation occasionally diverge. A test that only exercises the happy path proves nothing about the day the warehouse uploads six thousand stock adjustments at once.

Monitoring should surface three figures to the people who run the process: the proportion of calls returning 429 over the last hour, the current queue depth, and the oldest unprocessed item. Those three tell an operations manager whether the integration is healthy without requiring any knowledge of HTTP. An alert on the oldest item exceeding an agreed threshold is usually more useful than an alert on individual errors, because it measures the thing the business actually cares about, which is latency of information.

At handover, the documentation should record where the limiter configuration lives, how to adjust it if the vendor changes a ceiling or the client upgrades a subscription tier, and which jobs may safely be paused to free capacity during a peak. Limits move. Platforms revise them, licence changes alter entitlements, and trading volumes grow. An integration whose throttling assumptions are written down can be adjusted in an afternoon; one whose assumptions are implicit has to be rediscovered by whoever inherits it, usually during the week it matters most.

Key points

  • Identify which kind of limit you face — window, bucket, concurrency or daily quota — because each requires a different remedy and only some can be waited out.
  • Combine a shared client-side limiter with jittered exponential backoff, honour Retry-After, and never retry a write that lacks a deterministic idempotency key.
  • Calculate the call budget per transaction during discovery, record it in the scope, and test throttled responses as part of acceptance rather than discovering the ceiling in production.