Dynamic Rate Limiting with Redis Backends

Stateful, distributed rate limiting and throttling is mandatory for multi-tenant API gateways where request volume exceeds single-node capacity. In-process counters cannot be shared across gateway replicas — a tenant can trivially exceed its quota by routing requests round-robin across pods, because each pod maintains an independent count with no coordination. Offloading counter state to Redis provides a single authoritative store that all gateway instances read and write atomically, eliminating the split-quota problem and enabling dynamic per-tenant limits derived from JWT claims or API key metadata at request time.

Prerequisite Concepts

This page assumes familiarity with how the middleware chain sequences plugins, and with the sliding-window algorithm described in Rate Limiting & Throttling Strategies. You should also understand how authentication and token validation runs before the rate limiter in the request pipeline — the JWT must be validated before the gateway can extract a per-tenant quota claim and pass it to Redis.

Sliding-Window Counter Architecture

Redis implements sliding-window counters using sorted sets (ZADD) combined with ZREMRANGEBYSCORE to evict expired entries. For atomic execution and O(log N) complexity, every gateway replica dispatches a Lua script via a single EVALSHA call — no multi-step round trips, no race conditions between the prune and the increment.

Key naming must enforce tenant and route isolation to prevent cross-tenant quota bleeding:

rl:{tenant_id}:{route_id}

Fixed windows count requests per discrete interval (e.g., 00:00–00:01), causing burst doubling at interval boundaries. Sliding windows evaluate the exact trailing duration (e.g., the last 60 seconds from NOW()), smoothing traffic spikes without approximation.

Step-by-Step Gateway Configuration

Lua Script (load once at startup)

Pre-load via SCRIPT LOAD during gateway startup; store the returned SHA1 as lua_script_sha. This eliminates per-request script transmission overhead.

-- KEYS[1]: sorted set key, e.g., "rl:tenant1:route_orders"
-- ARGV[1]: current timestamp (milliseconds)
-- ARGV[2]: window duration (milliseconds)
-- ARGV[3]: max requests per window
local key    = KEYS[1]
local now    = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit  = tonumber(ARGV[3])
local cutoff = now - window

-- 1. Prune timestamps outside the sliding window
redis.call("ZREMRANGEBYSCORE", key, "-inf", cutoff)

-- 2. Add this request's timestamp (unique member via random suffix)
redis.call("ZADD", key, "NX", now, now .. "-" .. math.random(1e9))

-- 3. Count remaining entries
local total = redis.call("ZCARD", key)

-- 4. Set TTL so keys self-expire even if traffic stops
redis.call("EXPIRE", key, math.ceil(window / 1000) + 1)

if total > limit then
  return {0, total}  -- quota exhausted → 429
end
return {1, total}    -- allowed → pass through

Kong 3.x — Rate Limiting Advanced Plugin

Kong’s rate-limiting-advanced plugin (Kong 3.x) uses Redis as its shared storage back-end. Configure it at the service or route scope:

# kong.yaml — declarative (deck sync)
plugins:
  - name: rate-limiting-advanced
    config:
      limit:
        - 1000                       # requests per window
      window_size:
        - 60                         # seconds
      identifier: consumer           # or "ip", "credential", "header"
      sync_rate: -1                  # -1 = synchronous (exact), >0 = async flush interval
      namespace: "rl_orders"         # isolates counters per plugin instance
      strategy: redis
      redis:
        host: "redis-cluster.internal.svc.cluster.local"
        port: 6379
        timeout: 50                  # ms — keep ≤ 100 ms to avoid blocking worker threads
        max_connections: 64
      hide_client_headers: false
      error_code: 429
      error_message: "Rate limit exceeded. Retry-After header indicates wait time."

sync_rate: -1 blocks the request thread until Redis confirms the counter update — exact enforcement. A positive value (e.g., 0.5) flushes every 500 ms in background, reducing Redis IOPS by ~90% at the cost of approximate counting.

Envoy 1.32+ — External Rate Limit Service

Envoy delegates rate-limit decisions to an external gRPC service (e.g., lyft/ratelimit). The Envoy filter config defines the descriptor; the sidecar ratelimit service owns the Redis interaction.

# envoy.yaml — HTTP connection manager filter chain
http_filters:
  - name: envoy.filters.http.ratelimit
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
      domain: "api_gateway"
      timeout: 0.05s                  # 50 ms — fail open/closed beyond this
      failure_mode_deny: false        # false = fail_open; true = fail_closed
      rate_limit_service:
        grpc_service:
          envoy_grpc:
            cluster_name: rate_limit_cluster
        transport_api_version: V3
      request_headers_to_add_when_not_present:
        - header:
            key: x-tenant-id
            value: "%REQ(x-tenant-id)%"

The companion ratelimit service config maps descriptors to limits and Redis coordinates:

# ratelimit/config.yaml
domain: api_gateway
descriptors:
  - key: x-tenant-id
    rate_limit:
      unit: MINUTE
      requests_per_unit: 1000
redis_socket_type: tcp
redis_url: "redis-cluster.internal.svc.cluster.local:6379"
expiration_jitter_max_seconds: 5

NGINX + OpenResty — lua-resty-limit-traffic

NGINX with the OpenResty lua-resty-limit-traffic library implements the same sorted-set approach directly in Lua, without a sidecar service:

# nginx.conf — http block
http {
  lua_shared_dict rate_limit_locks 1m;

  server {
    location /api/ {
      access_by_lua_block {
        local redis = require "resty.redis"
        local r = redis:new()
        r:set_timeouts(50, 50, 50)          -- connect/send/recv ms

        local ok, err = r:connect(
          "redis-cluster.internal.svc.cluster.local", 6379
        )
        if not ok then
          -- fail_open: let request through on Redis error
          ngx.log(ngx.WARN, "Redis unavailable: ", err)
          return
        end

        local tenant_id = ngx.req.get_headers()["x-tenant-id"] or "anonymous"
        local route_id  = ngx.var.uri:match("^(/[^/]+)")       or "root"
        local key       = "rl:" .. tenant_id .. ":" .. route_id
        local now_ms    = ngx.now() * 1000
        local window_ms = 60000   -- 60 s
        local limit     = 1000

        local res, err = r:evalsha(
          ngx.shared.rate_limit_locks:get("lua_sha"),
          1, key,
          now_ms, window_ms, limit
        )
        if res and res[1] == 0 then
          ngx.header["Retry-After"]              = "60"
          ngx.header["X-RateLimit-Limit"]        = limit
          ngx.header["X-RateLimit-Remaining"]    = 0
          return ngx.exit(429)
        end
        ngx.header["X-RateLimit-Remaining"] = limit - (res and res[2] or 0)
      }
    }
  }
}

The Lua SHA is loaded at server start via init_by_lua_block calling redis:script("load", ...) and stored in lua_shared_dict rate_limit_locks.

Decision Matrix: Sync vs Async Counters, Fail-Open vs Fail-Closed

Dynamic Per-Tenant Quotas from JWT Claims

Static limits are insufficient for SaaS platforms where each tenant subscribes to a different tier. Both Kong and Envoy support resolving the effective limit at request time.

Kong 3.x — extract limit from jwt.claims.rate_limit:

plugins:
  - name: rate-limiting-advanced
    config:
      identifier: consumer
      limit:
        - 0                           # placeholder; overridden by consumer group
      dynamic_limits:
        enabled: true
        source: jwt.claims.rate_limit # extracts integer claim from validated JWT
        default: 500                  # fallback for tokens without the claim
      window_size:
        - 60
      strategy: redis
      redis:
        host: "redis-cluster.internal.svc.cluster.local"
        port: 6379
        timeout: 50

Envoy 1.32+ — inject tenant limit via metadata into descriptor:

The Envoy JWT authn filter validates the token and forwards claims as dynamic metadata. A Lua filter then writes the limit tier to a request header for the rate-limit descriptor.

-- Envoy Lua filter: translate JWT claim to rate-limit tier header
function envoy_on_request(request_handle)
  local meta   = request_handle:streamInfo():dynamicMetadata()
  local claims = meta:get("envoy.filters.http.jwt_authn")
  local limit  = (claims and claims["rate_limit"]) or "500"
  request_handle:headers():add("x-rate-limit-tier", tostring(limit))
end

Gotchas and Failure Signals

ZADD uniqueness collisions. If two requests arrive within the same millisecond from the same tenant, identical scores with different members (now .. "-" .. math.random(1e9)) prevent ZADD NX from silently dropping one. Omitting the random suffix causes under-counting under burst traffic.

Key expiry drift. Always set EXPIRE to ceil(window_ms / 1000) + 1. Without it, keys for dormant tenants accumulate indefinitely, growing Redis memory usage until an OOM condition forces eviction.

Connection pool exhaustion. Pool sizing formula:

max_connections = ceil(concurrent_rps × (redis_p99_latency_ms / 1000) × 1.25)

Example: 10,000 RPS with 2 ms P99 → ceil(10000 × 0.002 × 1.25) = 25 minimum. Allocate 2–3x this baseline to absorb connection churn during Redis primary failover.

redis_timeout_ms above 100 ms. Values above 100 ms risk cascading 504s during Redis background saves (BGSAVE) or AOF rewrite pauses. Set redis_timeout_ms: 50 and let the circuit breaker handle sustained timeouts.

Circuit breaker misconfiguration. If consecutive_failures is set too high (e.g., 50), the gateway threads saturate waiting for Redis timeouts before the circuit opens. Five consecutive failures is a practical threshold for 50 ms timeout budgets.

429 vs 503 under outage. When fallback_action: fail_open is active and Redis is reachable but slow, the gateway must still return 200 (pass-through), not 429. A 429 during a Redis latency spike misleads clients into backing off when the API is actually available.

Validation Steps

• SCRIPT LOAD the Lua sliding-window script at gateway startup; verify SHA1 is stored in lua_script_sha
• Confirm Redis key format includes tenant and route IDs: redis-cli KEYS "rl:*" shows isolated namespaces
• Verify ZADD NX + ZCARD atomicity: run redis-cli MONITOR | grep "rl:" under synthetic load and confirm no interleaved non-Lua calls
• Test fail_open path: stop Redis, send 10 requests, confirm all return 200 with no X-RateLimit-* headers (or last-known values)
• Test fail_closed path: stop Redis, confirm all requests return 503 with Retry-After: 30
• Run k6 at 110% of configured limit for 2 minutes; confirm rate_limit_hits metric climbs and no 5xx responses appear
• Check redis_latency_ms P99 < 5 ms under load; latency above 10 ms indicates memory fragmentation or network saturation
• Verify X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, and Retry-After headers are present on 429 responses
• Confirm EXPIRE is set on every key: redis-cli TTL "rl:tenant1:route_orders" returns a positive value

FAQ

Why use Redis sorted sets for sliding-window rate limiting instead of a simple counter?

A simple counter (INCR + EXPIRE) implements a fixed window: all requests in the same time slot share one bucket, which allows up to 2x the configured limit at window boundaries. Sorted sets track individual timestamps so the window slides continuously from the current moment, eliminating boundary spikes and providing sub-second precision without approximation.

What happens to rate limiting when Redis becomes unreachable?

The gateway’s configured fallback_action determines the behavior: fail_open passes all traffic (prefer for non-critical APIs), fail_closed rejects all traffic (prefer for financial or compliance endpoints), and cached_quota serves the last known counter value from gateway memory. A circuit breaker prevents every request from incurring the Redis timeout before the fallback activates.

How do I derive per-tenant rate limits from JWT claims at the gateway?

Configure dynamic_limits.source: jwt.claims.rate_limit (Kong 3.x) or inject the claim into a descriptor header via a Lua filter (Envoy 1.32+). The gateway extracts the claim after JWT validation, selects the effective limit, and passes it as ARGV[3] in the Lua EVALSHA call. Always set a default fallback for tokens that omit the claim.

Parent: Rate Limiting & Throttling Strategies

Related:

• Implementing JWT Validation in Kong Plugins — JWT validation that must complete before the rate limiter can extract per-tenant quota claims
• Caching & Response Optimization — reduce upstream load and Redis IOPS by serving cached responses before the rate-limit check fires
• Authentication & Token Proxying — plugin ordering, token forwarding, and how auth context propagates through the middleware chain to downstream counters