Implement rate limiting with Redis sliding window

Introduction

When you expose a GenAI API to the public internet without per-key rate limits, a single misbehaving client can drain your upstream LLM token budget, saturate your connection pool, and degrade latency for every other tenant in minutes. Fixed-window counters look tempting, but they leak: a client that fires its full quota at second 59 of one window and again at second 0 of the next achieves 2× the intended rate in a 2-second span — exactly the kind of boundary burst that triggers $0.03/request billing spikes and provider-side 429s on your account.

By the end of this lesson you'll be able to implement a Redis-backed sliding window rate limiter keyed on API key, wire it into FastAPI as a dependency, and return standards-compliant Retry-After / X-RateLimit-* headers so well-behaved clients self-throttle before they trip your limits.

Key Terminology

• Sliding window counter — a rate-limit algorithm that counts requests in the trailing N seconds from now, rather than within a fixed clock-aligned window; eliminates the boundary-burst exploit that breaks fixed-window counters.
• Redis sorted set (ZSET) — the data structure backing this lesson's counter; storing one member per request with the Unix timestamp as score lets us prune expired entries and count survivors in O(log N) per operation.
• ZREMRANGEBYSCORE — the Redis command that drops sorted-set members whose score falls in a range; we use it to delete every timestamp older than now − window_seconds on each request, keeping the set bounded.
• Rate-limit key — the Redis key under which a single API key's request timestamps live (e.g. ratelimit:<api_key>); namespacing per key is what makes the limiter per API key instead of global.
• Retry-After header — the RFC 7231 response header that tells a rejected client how many seconds to wait before retrying; computed from the oldest still-valid timestamp so the client retries the instant capacity frees up, not a generic interval later.

Concepts

Why fixed windows leak

A fixed-window counter aligned to clock minutes resets at :00, :01, :02. A client capped at 120 req/min can send 120 requests at 12:00:59.9 and another 120 at 12:01:00.1 — 240 requests in 200 ms, all approved. For GenAI traffic where each request costs real money and consumes a provider-side quota, this boundary burst is the difference between a healthy bill and an incident. The sliding window counter fixes it by anchoring the window to the current request, not to the wall clock.

Sliding window with Redis sorted sets

Every inbound request gets one sorted-set member, scored by its Unix timestamp. On each call we first pipeline two commands in a single round trip: ZREMRANGEBYSCORE prunes everything older than now − window_seconds, and ZCARD counts what remains. Only when that count is below the quota do we issue a second pipeline that ZADDs the new request and sets an EXPIRE so idle API keys don't leak memory forever. That check-then-act split keeps each request to two short round trips, so per-request overhead stays sub-millisecond even at thousands of req/s (see Code Walkthrough).

Per-key tiering and member uniqueness

In production each API key carries its own quota — free tier might get 10 req/min, paid 100, enterprise 1,000 — resolved from a lookup table and passed to the limiter at call time. Two implementation traps to know up front: sorted-set members must be unique per request, or Redis silently dedupes and the counter undercounts (concatenate the timestamp with a UUID fragment under heavy load to guarantee uniqueness); and EXPIRE must be set to window_seconds + 1 so quiet keys disappear from Redis after their window closes — without it, every key you ever rate-limited lives in memory forever.

Code Walkthrough

The snippets below implement the two concepts above: the RateLimiter class wraps the ZSET pipeline, and the FastAPI dependency wires it into the request lifecycle so a rejected request never touches your route handler.

Code snippetpython
1import time
2import redis.asyncio as redis
3from dataclasses import dataclass
4from uuid import uuid4
5
6@dataclass
7class RateLimitResult:
8    allowed: bool
9    remaining: int
10    reset_after: float
11    limit: int
12
13class RateLimiter:
14    def __init__(
15        self,
16        redis_client: redis.Redis,
17        max_requests: int = 60,
18        window_seconds: int = 60,
19    ):
20        self.redis = redis_client
21        self.max_requests = max_requests
22        self.window_seconds = window_seconds
23
24    async def check_rate_limit(self, api_key: str) -> RateLimitResult:
25        now = time.time()
26        window_start = now - self.window_seconds
27        redis_key = f"ratelimit:{api_key}"
28
29        pipe = self.redis.pipeline(transaction=True)
30        pipe.zremrangebyscore(redis_key, 0, window_start)
31        pipe.zcard(redis_key)
32        results = await pipe.execute()
33        current_count = results[1]
34
35        if current_count < self.max_requests:
36            member = f"{now}:{uuid4().hex[:8]}"
37            pipe2 = self.redis.pipeline(transaction=True)
38            pipe2.zadd(redis_key, {member: now})
39            pipe2.expire(redis_key, self.window_seconds + 1)
40            await pipe2.execute()
41            return RateLimitResult(
42                allowed=True,
43                remaining=self.max_requests - current_count - 1,
44                reset_after=self.window_seconds,
45                limit=self.max_requests,
46            )
47
48        oldest = await self.redis.zrange(redis_key, 0, 0, withscores=True)
49        reset_after = (
50            oldest[0][1] + self.window_seconds - now
51            if oldest
52            else self.window_seconds
53        )
54        return RateLimitResult(
55            allowed=False,
56            remaining=0,
57            reset_after=round(reset_after, 2),
58            limit=self.max_requests,
59        )

• RateLimitResult dataclass carries everything the route handler and response headers need: allowed for the gate decision, remaining to populate X-RateLimit-Remaining, reset_after to populate Retry-After, and limit for X-RateLimit-Limit.
• The pipeline (zremrangebyscore + zcard) is one round trip that simultaneously prunes expired timestamps and counts survivors. results[1] is the ZCARD reply — the count of requests still inside the trailing window.
• The accept branch adds the new request as f"{now}:{uuid4().hex[:8]}"; the UUID fragment guarantees member uniqueness so Redis can't silently dedupe two requests landing in the same microsecond. EXPIRE window_seconds + 1 prevents idle keys from accumulating in memory.
• The reject branch fetches the oldest surviving timestamp and computes when it'll fall out of the window — that's the exact second capacity opens up, which we hand the client via Retry-After instead of a guess.

Code snippetpython
1from fastapi import FastAPI, Request, HTTPException, Depends
2from fastapi.responses import JSONResponse
3
4app = FastAPI()
5rate_limiter: RateLimiter | None = None
6
7@app.on_event("startup")
8async def startup():
9    global rate_limiter
10    redis_client = redis.Redis(
11        host="localhost", port=6379, decode_responses=True
12    )
13    rate_limiter = RateLimiter(
14        redis_client, max_requests=100, window_seconds=60
15    )
16
17async def enforce_rate_limit(request: Request) -> RateLimitResult:
18    api_key = request.headers.get("X-API-Key")
19    if api_key is None:
20        raise HTTPException(status_code=401, detail="Missing API key")
21
22    result = await rate_limiter.check_rate_limit(api_key)
23    if not result.allowed:
24        raise HTTPException(
25            status_code=429,
26            detail="Rate limit exceeded",
27            headers={
28                "Retry-After": str(result.reset_after),
29                "X-RateLimit-Limit": str(result.limit),
30                "X-RateLimit-Remaining": "0",
31            },
32        )
33    return result
34
35@app.post("/v1/completions")
36async def create_completion(
37    rate_result: RateLimitResult = Depends(enforce_rate_limit),
38):
39    response = JSONResponse(
40        content={"message": "Completion generated", "model": "gpt-4"}
41    )
42    response.headers["X-RateLimit-Limit"] = str(rate_result.limit)
43    response.headers["X-RateLimit-Remaining"] = str(rate_result.remaining)
44    return response

• enforce_rate_limit as a Depends runs before the route handler. A 429 raised here short-circuits the request so a rate-limited call never reaches your LLM provider — that's the whole point of gating at the dependency layer.
• The 429 response headers (Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining) follow the IETF draft adopted by OpenAI and Anthropic, so any client using their SDKs' retry middleware will back off correctly without custom code.
• Successful responses also carry X-RateLimit-* headers so well-behaved clients can self-throttle as they approach the limit instead of waiting to be rejected.

You'll know it works when a script that fires 101 requests in under 60 seconds against /v1/completions (with the same X-API-Key) sees the first 100 return 200 and the 101st return 429 with a Retry-After value that counts down toward the moment your oldest in-window request expires.

Do's and Don'ts

Building on the implementation and discipline-specific framing above, here are the habits that keep this limiter correct under production load — and the traps that quietly break it.

Do's

1. ✓Do include a uniqueness suffix in the sorted-set member — f"{now}:{uuid4().hex[:8]}" prevents Redis from silently deduping two requests that arrive in the same microsecond and undercounting.
2. ✓Do compute Retry-After from the oldest in-window timestamp — it tells the client the exact second capacity reopens, so retries land at the right moment instead of stampeding early or wasting headroom.
3. ✓Do set EXPIRE to window_seconds + 1 — idle API keys disappear from Redis automatically; without it, every key you ever rate-limited lives in memory forever.

Don'ts

1. ✗Don't use a fixed-window counter for revenue-sensitive endpoints — the boundary burst lets a client get 2× the configured rate in a 2-second span at every window roll, which translates directly into bill shock and upstream 429s.
2. ✗Don't fire each Redis command as its own round trip — batch the prune-and-count (ZREMRANGEBYSCORE + ZCARD) into one transactional pipeline and the record-and-expire (ZADD + EXPIRE) into a second. Two pipelines instead of four independent calls cut per-request latency and keep each pair atomic.
3. ✗Don't apply one global quota to every API key — resolve the quota per key from a lookup table so free, paid, and enterprise tiers each get the right limit without code changes.