Monitor Redis performance and memory

Introduction

In production, a Redis cluster rarely crashes with a loud error — it degrades silently: the hit rate dips from 92% to 14%, session keys start evicting under memory pressure, and every LLM call begins stampeding the upstream API before anyone notices the dashboards. Knowing which four metrics to watch, how to scrape them into Prometheus, and which command patterns block the single-threaded server turns reactive firefighting into proactive operations. By the end of this lesson you will be able to read INFO output for hit rate, memory, and eviction signals; write Prometheus alert rules for each; and replace the blocking KEYS anti-pattern with non-blocking SCAN.

Key Terminology

• hit rate — keyspace_hits / (keyspace_hits + keyspace_misses) from INFO stats; the leading indicator that your cache is doing its job. A drop from 90% to 60% over an hour is a P1 signal because every miss now stampedes upstream.
• eviction — Redis dropping a key under memory pressure as driven by maxmemory-policy; on a cache it just costs upstream calls, on a noeviction session store it is silent data loss.
• slow log — Redis's record of any command exceeding slowlog-log-slower-than (default 10ms); the cheapest way to catch the KEYS * and unbounded-LRANGE patterns that block the single-threaded server.
• maxmemory-policy — the eviction algorithm Redis applies when memory fills (noeviction, allkeys-lru, volatile-lru, allkeys-lfu); chosen once per instance and rarely changeable without an outage.
• AOF (Append-Only File) — the persistence mode that logs every mutation and fsyncs every second; the floor for any session store or queue, since RDB-only loses 5–15 minutes of writes between snapshots.

Concepts

Memory Policy: Choose Once, Choose Right

maxmemory-policy is set in redis.conf and rarely changed afterwards. The choice is determined by what the data represents.

• noeviction — Writes return an OOM error when memory is full; reads still succeed. Use for session stores, queues, durable state.
• allkeys-lru — Evict least-recently-used across all keys. Use for pure caches where every key is regenerable from an upstream source.
• volatile-lru — Evict LRU only among keys with TTL. Use for mixed workloads where durable keys (no TTL) sit alongside cached keys (with TTL) in the same instance.
• allkeys-lfu — Evict least-frequently-used. Use for LLM response caches with hot-key skew: a few popular prompts dominate, and you want them sticky even when a long tail of one-shot queries floods the cache.

For LLM response caches with prompt clustering (top 1000 prompts → 80% of hits), allkeys-lfu outperforms allkeys-lru by 10–20 points on hit rate. For a vector embedding cache with uniform access, allkeys-lru is fine. For session state holding active chat conversations, noeviction is mandatory—you cannot silently drop a user's mid-conversation context.

Persistence: RDB vs AOF

• RDB snapshots (save 900 1) fork and dump every N seconds with M+ key changes. Cheap on steady state, but a crash between snapshots loses 5–15 minutes of writes.
• AOF with appendonly yes and appendfsync everysec logs every mutation, fsyncing every second. Worst-case loss is ~1 second; appendfsync always cuts that to zero at a 5–10x throughput cost.

For an LLM response cache, RDB is sufficient—a brief hit-rate dip while the cache rewarms is acceptable. For a session store or queue, AOF with everysec is the floor; run both (save + appendonly) so RDB serves as a fast-restart baseline and AOF replays the delta.

Operating discipline

• Disable KEYS *, FLUSHALL, FLUSHDB, DEBUG SLEEP, and MONITOR via rename-command in redis.conf. These commands have no safe production use case.
• Alert on eviction rate, not just used_memory. A cluster at 95% memory with zero evictions is healthy; one at 60% memory with 1000 evictions/minute is broken. The rate is the leading indicator; the gauge is the lagging one.
• Keep connected_clients under 30% of maxclients. Right-size your client max_connections to (workers × pools), not "as high as possible."
• Set slowlog-log-slower-than 10000 and review SLOWLOG GET 10 weekly. A clean slow log is the cheapest health check you have.
• Pick maxmemory-policy once at deploy time and document why. Switching noeviction → allkeys-lru later quietly enables data loss; the reverse starts rejecting writes. Both transitions cause outages without a runbook.
• Run redis-cli --latency from a sidecar before blaming Redis. P99 under 1ms is normal; if your app reports 50ms but --latency shows 0.4ms, the problem is in the client, network, or serialization—not in Redis.

Code Walkthrough

Now that you've chosen maxmemory-policy and persistence settings from the Concepts section, you need a runtime view that confirms those choices are holding — the four numbers that matter are hit rate, memory headroom, eviction rate, and slow-log depth.

Reading INFO in Python

Rather than querying redis-cli by hand, pull the INFO sections you need directly from the client. The redis library's info() method returns a parsed dictionary keyed by field name:

Code snippetpython
1import redis
2
3r = redis.Redis(host="localhost", port=6379, decode_responses=True)
4
5def redis_health_snapshot(r: redis.Redis) -> dict:
6    stats = r.info("stats")
7    memory = r.info("memory")
8    clients = r.info("clients")
9
10    hits = stats["keyspace_hits"]
11    misses = stats["keyspace_misses"]
12    total = hits + misses
13    hit_rate = hits / total if total > 0 else 0.0
14
15    used = memory["used_memory"]
16    maxmem = memory.get("maxmemory", 0)
17    mem_pct = used / maxmem if maxmem > 0 else 0.0
18
19    return {
20        "hit_rate": round(hit_rate, 4),
21        "memory_pct": round(mem_pct, 4),
22        "evicted_keys": stats["evicted_keys"],
23        "connected_clients": clients["connected_clients"],
24        "slowlog_depth": len(r.slowlog_get(128)),
25    }
26
27snapshot = redis_health_snapshot(r)
28print(snapshot)
29# {'hit_rate': 0.9001, 'memory_pct': 0.7234, 'evicted_keys': 0, 'connected_clients': 142, 'slowlog_depth': 0}

r.info("stats") fetches only the stats stanza rather than the full INFO all payload, keeping the call lightweight. evicted_keys is a cumulative counter — wrap it in a rate calculation before alerting, or let Prometheus handle the derivative. If maxmemory is 0, Redis is running unbounded; mem_pct returns 0.0, which should trigger a configuration alarm rather than a headroom alarm. slowlog_depth > 0 after steady-state startup points at a KEYS * scan or an unbounded LRANGE that is holding the single-threaded server hostage.

Prometheus alert rules

For continuous monitoring, the oliver006/redis_exporter sidecar translates the same INFO fields into Prometheus metrics on :9121/metrics. The four rules below map one-to-one to the signals the Python snapshot surfaces:

Code snippetyaml
1groups:
2- name: redis-genai
3  rules:
4  - alert: RedisHitRateLow
5    expr: |
6      rate(redis_keyspace_hits_total[5m])
7        / (rate(redis_keyspace_hits_total[5m])
8           + rate(redis_keyspace_misses_total[5m]))
9        < 0.70
10    for: 10m
11    labels: { severity: page }
12  - alert: RedisMemoryHigh
13    expr: redis_memory_used_bytes / redis_memory_max_bytes > 0.85
14    for: 5m
15    labels: { severity: page }
16  - alert: RedisEvictions
17    expr: rate(redis_evicted_keys_total[5m]) > 1
18    for: 5m
19    labels: { severity: page }
20  - alert: RedisSlowlogBurst
21    expr: increase(redis_slowlog_length[5m]) > 50
22    for: 0m
23    labels: { severity: warn }

for: 10m on RedisHitRateLow debounces cold-start dips after a deploy. RedisMemoryHigh fires before maxmemory is reached, giving you time to scale capacity before allkeys-lru silently drops keys or noeviction starts returning OOM errors to writers. RedisSlowlogBurst fires immediately (for: 0m) because a cluster of slow commands means a blocking operation is running right now, not trending toward a problem.

Confirm that redis_health_snapshot() returns evicted_keys: 0 and hit_rate > 0.85 against your local Redis instance, and that the four Prometheus rules load cleanly via promtool check rules alerts.yaml.

Discipline-specific monitoring playbook

Do's and Don'ts

Now that you have worked through the implementation, the practices below separate a durable approach from a fragile one.

Do's

1. ✓Do call r.info() with a specific section name ("stats", "memory", or "clients") rather than fetching INFO all — pulling only the stanza you need keeps the monitoring call lightweight and avoids deserializing several kilobytes of server state on every scrape interval.
2. ✓Do check whether maxmemory is 0 before computing mem_pct — a zero value means Redis is running unbounded, so mem_pct silently returns 0.0 and masks the missing safety net; route that case to a configuration alarm rather than a headroom alarm.
3. ✓Do match for: debounce windows to the failure mode's urgency — set for: 0m on RedisSlowlogBurst because a cluster of slow commands means a blocking operation (KEYS *, an unbounded LRANGE) is holding the single-threaded server right now, and set for: 10m on RedisHitRateLow to absorb the cold-start dip that follows every deploy before the cache warms.

Don'ts

1. ✗Don't use KEYS * to inspect the live key space — the command blocks Redis's single thread for the full scan duration, stalling every concurrent client; replace it with SCAN which iterates in small, non-blocking cursor steps at the cost of multiple round-trips.
2. ✗Don't alert on the raw evicted_keys cumulative counter — because it only ever increases, an alert like evicted_keys > 0 fires once and never re-arms; wrap it in rate(redis_evicted_keys_total[5m]) > 1 so Prometheus fires on active eviction pressure rather than on a counter that crossed zero during a long-ago incident.
3. ✗Don't set RedisMemoryHigh to fire at 100% of maxmemory — by the time used_memory reaches maxmemory, Redis is already silently dropping keys under allkeys-lru or returning OOM errors to writers under noeviction; the 0.85 threshold gives you time to scale capacity before either failure mode lands.