API Rate Limiting That Doesn't Break Your Clients

Most backend engineers can explain token bucket. Fewer can tell a client whether remaining: 12 means twelve requests, twelve cost points, or twelve requests on this endpoint before a different global limiter fires. That ambiguity survives every clean Redis implementation.

An API rate limiter has two products. One protects the service. The other tells clients how to stay out of its way. Teams spend weeks tuning the first and ship the second as 429, try later.

That is how a safety mechanism becomes an outage multiplier.

The Algorithm Is Not the API

429 Too Many Requests tells generic HTTP software that a request crossed a rate boundary. It does not define the boundary, the identity being counted, or the algorithm. The original specification leaves counting scope and client identification open on purpose.

So “we use token bucket” answers an internal question. A client needs different answers:

• What consumes quota?
• Who shares the quota?
• How much burst is allowed?
• Which policy is currently tight?
• When is another attempt useful?
• Can the failed operation be retried safely?

Two APIs can both use token buckets and expose incompatible behavior. One might refill ten tokens per second into a bucket of one hundred per tenant. Another might apply separate buckets per endpoint, plus an undocumented account-wide concurrency ceiling. The noun matches. The contract does not.

The bucket name is not the contract.

Pick the Traffic Shape — Then Pick the Bucket

Choose the algorithm from the traffic shape you want clients to produce.

Token bucket permits controlled bursts while capping sustained rate. Refill rate defines the long-run pace; capacity defines how much short-term work can arrive together. AWS API Gateway exposes exactly those two controls, though it also warns that throttles are best-effort targets rather than hard ceilings.

Use it when legitimate clients are naturally bursty: a mobile app syncing after reconnect, a CI runner uploading results, or an integration flushing a short queue. The burst is a feature you are promising to absorb.

Leaky bucket favors a steadier output rate. Shopify describes its limits through bucket capacity and restore rate, then charges GraphQL requests by calculated query cost rather than treating every call as equal. That combination protects expensive downstream work more honestly than “100 requests per minute” when one query can cost a thousand cheap ones.

Sliding-window counters fit policies phrased as “no more than N during any recent window.” They avoid the boundary spike of fixed windows without retaining every timestamp. Cloudflare's production design used current and previous counters because its storage primitives, latency budget, and distributed edge architecture made that approximation a better fit than a textbook-perfect log.

There is no universally mature choice. There is only a choice whose burst behavior, state cost, and error match your service.

The Partition Key — Who Shares the Pain

A rate number without a partition key is marketing.

“1,000 requests per minute” could mean per IP, user, API key, tenant, endpoint, region, or the whole account. Each produces a different failure mode.

Per-IP limits punish offices, NAT gateways, and mobile carriers that put many users behind one address. Per-user limits can let one tenant create thousands of users and multiply load. Per-tenant limits protect fairness but make a noisy internal job consume the same budget as interactive traffic. Per-endpoint limits protect expensive routes while leaving aggregate capacity exposed.

Real systems layer them.

AWS applies throttling at regional, account, stage, method, and client levels. GitHub exposes primary quota headers while keeping some secondary-limit state undisclosed; clients can cross a hidden policy even when the visible primary counter looks healthy.

That is why remaining quota is advisory, not a reservation. The current IETF draft says clients must not assume that a positive available quota guarantees another request will be served. A concurrent request, another node, or a tighter policy can consume capacity first.

Document the partition in human terms. If revealing its raw value would expose sensitive implementation detail, give the policy a stable public name: account-default, search, write-heavy, graphql-cost. Clients need an identity they can log and route on, not your Redis key.

The Header Contract — Tell Clients Before They Guess

A considerate API reports rate state on successful responses. Waiting until 429 means every client learns pacing by collision.

The current standards-track IETF draft uses two fields:

• RateLimit-Policy describes a policy's quota and window.
• RateLimit reports current available quota and the effective time window.

Cloudflare adopted that shape for its REST APIs, including named policies, remaining quota, and reset timing. The draft is not an RFC yet, so freezing a parser to one draft revision without a compatibility plan is its own contract risk.

Legacy APIs commonly expose X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset. Those names are conventions, not one consistent semantic package. GitHub's reset value is a UTC epoch timestamp. Other APIs use seconds until reset. A client that guesses wrong waits decades or retries immediately.

Pick one unit and state it.

On successful responses, expose enough information for proactive pacing:

• A stable policy name.
• The quota unit: requests, points, bytes, rows, or another declared cost.
• Available quota under that policy.
• The relevant window or restore rate.
• The partition scope in documentation.

Do not promise precision your distributed counter cannot deliver. If state is approximate, say so. Clients can pace against a conservative signal; they cannot pace against fictional certainty.

One Status, Several Failures — A 429 Needs a Reason

429 is a class of refusal, not a diagnosis.

Stripe uses the same status for global request rate, endpoint rate, global concurrency, endpoint concurrency, and resource-specific limits. It emits Stripe-Rate-Limited-Reason so clients and operators can tell those cases apart. A 429 without that header can instead be a lock timeout.

Those failures want different fixes. Request-rate pressure calls for slower pacing. Concurrency pressure calls for fewer in-flight operations. A resource-specific limit may require serializing work for one object while unrelated objects continue. Lock contention asks for coordination, not a larger quota.

Give the response body a stable machine-readable reason. HTTP Problem Details provides a standard application/problem+json envelope with extension fields for domain-specific data. A useful response can look like this:

HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 12
RateLimit: "write-heavy";r=0;t=12
RateLimit-Policy: "write-heavy";q=120;w=60
 
{
  "type": "https://api.example.com/problems/rate-limit",
  "title": "Write quota exhausted",
  "status": 429,
  "detail": "The tenant write quota is exhausted.",
  "reason": "tenant-write-rate",
  "policy": "write-heavy"
}

The type URI is an identifier clients can compare, not an invitation to scrape prose. reason and policy are versioned contract fields. detail is for humans.

A clean status without a reason still breaks clients.

Client Behavior — Pace, Back Off, Stop

The server contract is only half the job. Your SDK needs a bounded policy.

First, pace before failure when successful responses expose available quota and restore timing. This spreads load and preserves latency better than running at full speed until the first rejection.

After a 429, honor Retry-After when present. HTTP allows either an absolute date or delay seconds. Delta seconds are easier for clients because they reduce clock-skew mistakes. Add a small amount of jitter when many workers share a deadline, but never jitter earlier than the server's minimum.

If Retry-After is absent, use capped exponential backoff with randomness. Stripe recommends that pattern to avoid synchronized retry traffic. GitHub goes further: continued requests while limited can get an integration banned, and persistent secondary-limit failures should stop after a finite retry count.

Backoff without a retry budget is a slower infinite loop.

Mutating requests add one more gate: replay safety. A client should retry only when the operation is idempotent by method semantics, protected by an idempotency key, or explicitly documented as safe. The limiter may reject before application execution — but a timeout around the response can leave the client uncertain about whether execution started.

Bound attempts. Bound elapsed time. Return the last machine-readable reason to the caller.

The Contract You Keep

Before shipping a limiter, change the algorithm in your design document. Swap token bucket for a sliding-window counter. If the client contract changes with it, internal mechanics leaked into the API.

The server needs a ceiling. The client needs a clock. The system needs a reason both sides can name.