Distributed API Rate Limiting & Idempotency at Scale: Redis Keyspace Architecture & Lock Patterns

Protecting high-throughput public APIs requires robust mechanisms to shield downstream services from traffic spikes and ensure that duplicate client payloads execute exactly once. Implementing these guarantees at scale relies on atomic operations within a distributed cache, typically using Redis to manage state across microservice clusters.

To enforce predictable rate limiting, systems must move beyond simple fixed-window counters, which are susceptible to boundary burst vulnerabilities. The sliding window log pattern provides a superior approach by tracking request timestamps within a sorted set (ZSET). This method allows for precise pruning of expired entries and evaluation of request volume with O(log N + M) complexity. By executing this logic via Lua scripts, developers ensure atomicity, preventing race conditions during concurrent request processing.

Achieving exactly-once execution requires an idempotency layer to handle network timeouts and client-side retry logic. A common architectural pattern involves an interceptor that tracks the lifecycle of an Idempotency-Key header through defined states:

• IN_PROGRESS: A distributed lock (with a short TTL) is acquired to prevent concurrent execution of identical payloads.
• COMPLETED: Upon successful completion, the response body and status code are cached for a defined duration (typically 24 to 72 hours) to facilitate replays.
• FAILED: If processing errors occur, the lock is evicted, permitting future retries to attempt execution again.

Operational stability depends on rigorous memory management within the Redis cluster. Engineering teams should adopt the following practices to avoid resource exhaustion:

• Offload Payload Storage: For large response bodies, cache only a signed retrieval URL in Redis rather than the raw data, utilizing external object storage to prevent memory pressure.
• Implement TTL Policies: Explicitly define expiration intervals for all keys to prevent unbounded memory growth.
• Sync Clocks: Maintain strict NTP synchronization across Redis nodes to prevent inconsistencies in timestamp-based logic.

Fixed‑window rate limiters divide time into discrete intervals (e.g., one minute) and count requests per key within each interval. Because the counter resets only at the interval boundary, a client can issue the full allowance at the end of one window and immediately repeat the same amount at the start of the next. This “boundary burst” effectively doubles the permitted rate for a short period, violating the intended smooth traffic profile. Fixed windows also suffer from coarse granularity: a limit of 100 requests per minute cannot enforce sub‑second pacing, and the implementation typically requires a separate key per window, increasing memory churn in a distributed cache.

The sliding‑window log pattern eliminates these issues by tracking each request timestamp in a Redis sorted set (ZSET). The ZSET score is the request’s epoch time, allowing the algorithm to consider only the most recent window milliseconds regardless of when the window started. By pruning entries older than now - window and counting the remaining members, the limiter enforces a true sliding rate limit with O(log N + M) complexity, where N is the number of stored timestamps and M is the number removed.

• Store a ZSET per client or API key, using timestamps as scores.
• Execute a Lua script that (a) removes stale scores, (b) checks the current cardinality, and (c) adds the new timestamp if under the limit.
• Set an EXPIRE on the key equal to the window length to bound memory usage.
• Include a per‑request nonce (e.g., INCR global:nonce) to guarantee unique members when multiple requests share the same timestamp.
• Return the remaining allowance to the caller for client‑side back‑off logic.

local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])
local clearBefore = now - window

redis.call('ZREMRANGEBYSCORE', key, '-inf', clearBefore)
local current = redis.call('ZCARD', key)

if current < limit then
  redis.call('ZADD', key, now, now .. ':' .. redis.call('INCR', 'global:nonce'))
  redis.call('EXPIRE', key, math.ceil(window / 1000))
  return {1, limit - current - 1}
else
  return {0, 0}
end

Because the entire sequence runs inside a single Lua script, it is atomic across all Redis nodes, preventing race conditions that would otherwise allow excess requests. The pattern also aligns with operational best practices: explicit EXPIRE avoids unbounded memory growth, and synchronized clocks (via NTP) ensure consistent timestamp scoring across a clustered Redis deployment.

An idempotency layer must guarantee that a request identified by an Idempotency-Key header executes its business logic exactly once, even when clients retry after timeouts or network errors. The core of the solution is a three‑state finite state machine stored in Redis:

• IN_PROGRESS – a distributed lock is held on idempotency:{tenant_id}:{key} with a short TTL (e.g., 30 seconds). This prevents concurrent workers from invoking the same side‑effecting operation.
• COMPLETED – after successful processing, the lock is replaced by a record that contains the HTTP status code and a serialized response body. The record lives longer (24‑72 hours) so any subsequent retry can be answered from cache.
• FAILED – if the handler throws an unhandled error, the lock entry is deleted immediately, allowing the next retry to start a fresh IN_PROGRESS cycle.

Implementing this pattern in a Node.js service typically involves a middleware interceptor that performs three actions: acquire the lock, serve a cached response when the state is COMPLETED, and clean up on failure. The following excerpt illustrates the flow:

async function idempotencyMiddleware(req, res, next) {
  const key = req.headers['idempotency-key'];
  if (!key) return next();

  const lockKey = `idempotency:${req.tenant.id}:${key}`;
  const acquired = await redis.set(
    lockKey,
    JSON.stringify({status: 'IN_PROGRESS'}),
    'NX', 'EX', 30
  );

  if (!acquired) {
    const raw = await redis.get(lockKey);
    const state = JSON.parse(raw || '{}');

    if (state.status === 'COMPLETED') {
      res.set('X-Cache-Lookup', 'HIT-IDEMPOTENT');
      return res.status(state.statusCode).send(state.body);
    }
    return res.status(409).send({error: 'Request is already processing'});
  }

  const originalSend = res.send.bind(res);
  res.send = body => {
    if (res.statusCode >= 200 && res.statusCode < 300) {
      redis.set(
        lockKey,
        JSON.stringify({status: 'COMPLETED', statusCode: res.statusCode, body}),
        'EX', 86400
      );
    } else {
      redis.del(lockKey);
    }
    return originalSend(body);
  };
  next();
}

Key operational considerations include:

• Always set an explicit EXPIRE on both lock and cache entries to avoid unbounded memory growth in the Redis cluster.
• For large payloads, store the response body in object storage (e.g., S3) and keep only a signed URL in Redis to reduce memory pressure.
• Synchronize clocks across Redis nodes (NTP) when timestamps are used for related rate‑limiting logic, preventing drift‑induced lock inconsistencies.

By coupling the Idempotency-Key header with a Redis‑backed state machine and short‑lived distributed locks, engineers can reliably eliminate duplicate side effects while preserving high throughput for multi‑tenant APIs.

Implementing idempotency within a Fastify or Express architecture requires a middleware interceptor capable of managing distributed state to prevent duplicate side effects from retries. The middleware pattern relies on an Idempotency-Key header to track the lifecycle of a request, transitioning through three distinct states: IN_PROGRESS, COMPLETED, and FAILED.

The implementation follows a three-step progression within the request-response lifecycle:

• Atomic Lock Acquisition: Using redis.set(key, value, 'NX', 'EX', ttl), the middleware attempts to establish an IN_PROGRESS mutex lock. The NX flag ensures the lock is only created if the key does not exist, effectively serializing concurrent requests with the same key.
• Cache Retrieval: If the lock acquisition fails, the middleware fetches the existing state. If the status is COMPLETED, the system replays the cached HTTP response body and status code directly to the client, avoiding execution of the downstream business logic.
• Response Capture: By overriding the res.send method, the middleware monitors the response downstream. Upon a successful HTTP 2xx status, it caches the payload in Redis for a defined duration (e.g., 24 hours). If an application error occurs, the middleware evicts the key to allow for valid future retries.

Below is a functional representation of this pattern:

async function idempotencyMiddleware(req, res, next) {
  const idempotencyKey = req.headers['idempotency-key'];
  if (!idempotencyKey) return next();
  const lockKey = `idempotency:${req.tenant.id}:${idempotencyKey}`;
  const acquired = await redis.set(lockKey, JSON.stringify({ status: 'IN_PROGRESS' }), 'NX', 'EX', 30);
  
  if (!acquired) {
    const rawState = await redis.get(lockKey);
    const state = JSON.parse(rawState);
    if (state.status === 'COMPLETED') return res.status(state.statusCode).send(state.body);
    return res.status(409).send({ error: 'Request in progress' });
  }

  const originalSend = res.send.bind(res);
  res.send = (body) => {
    if (res.statusCode >= 200 && res.statusCode < 300) {
      redis.set(lockKey, JSON.stringify({ status: 'COMPLETED', statusCode: res.statusCode, body }), 'EX', 86400);
    } else {
      redis.del(lockKey);
    }
    return originalSend(body);
  };
  next();
}

To maintain performance, engineers should account for memory overhead. For high-volume environments, avoid storing large response blobs directly in Redis; instead, store large payloads in object storage and cache only the references.