Cache Warming and Cold Starts

Replication and failover keep a cache serving through node failures, but when a new or replacement node comes online its memory is empty. Every request it receives is a miss, and the database suddenly sees load it hasn't seen since the cache was first deployed. This is the cold start problem, and cache warming is how you survive it.

The problem

A system serves 50,000 reads a second with a 95% hit rate, so the database handles about 2,500 queries a second and is sized for that. If the cache goes cold, all 50,000 become misses: a 20× spike on the database, instantly.

It isn't just slow, it can cascade: the database saturates, query latency spikes, application threads block waiting, connection pools run out, and users get errors. The cache was protecting the database; remove that protection even briefly and you can cause an outage.

When cold starts happen

• Fresh deployments of a new cache cluster.
• Node replacement after a failure: if the replacement is a fresh instance rather than a promoted replica, it starts empty.
• Scaling out: adding a shard to a Redis Cluster moves hash slots, and keys migrated to the new shard must be fetched from the database on first access.
• A cache flush after a bug: code wrote corrupt data into the cache, and the fix is flushing the affected keys, or everything.
• Mass TTL expiration: many keys written together with the same TTL (a batch import) expire together, and that key range goes cold at once. TTL jitter prevents this.

Warm-up strategies

1. Gradual warm-up

Let natural traffic fill the cache: the first request for each key goes to the database, later ones hit. The hit rate climbs from 0% to steady state over minutes as popular keys accumulate. It's the simplest approach, and fine if the database has headroom: in the example it must handle 50,000 queries a second for several minutes instead of 2,500. If it can't, the system degrades before the cache has time to warm.

2. Controlled ramp-up

Send traffic to the new cache node gradually: 10% first, let it warm for a few minutes, then 25%, 50% and finally 100%. Meanwhile the old cache (or the database) serves the rest, so the cold-start spike is limited to a fraction of total load. Load balancers or service meshes do this with weighted routing; the cost is operational tooling to manage the ramp and monitoring to decide when to step up.

3. Pre-loading (cache priming)

Before the node takes any traffic, load the most popular keys into it. That needs to know which keys are hot:

• Recent access logs: extract the most-requested keys from the last hour.
• A popularity index in the database: a counter table that tracks access frequency.
• A snapshot of an existing replica: copy the key-value pairs directly.

Pre-loading typically gets the hit rate to 70–80% of steady state before the node serves real traffic; the rest fills in through normal misses. It's common in large systems where a full cold start would overwhelm the database.

def prime_cache(cache, db, access_log, top_n=50_000, batch=500):
    """Load the hottest keys before the node joins the load balancer."""
    hot_ids = access_log.top_requested("product", window="1h", limit=top_n)
    for i in range(0, len(hot_ids), batch):
        rows = db.fetch_products(hot_ids[i:i + batch])          # batched, gentle on the DB
        with cache.pipeline() as pipe:
            for row in rows:
                pipe.set(f"product:v1:detail:{row['id']}", serialize(row), ex=ttl_with_jitter(900))
            pipe.execute()
    mark_ready()                                                # only now start taking traffic

Protecting the database during warm-up

Even with warm-up, some extra database load is unavoidable. Two mechanisms stop it from cascading:

• Rate limiting the miss path. Cap concurrent database queries triggered by misses. With a 5,000 queries-a-second limit and 50,000 misses arriving, 45,000 requests wait or get a degraded response (stale data from another source, or a brief error), and the database stays within capacity. See rate limiting.
• Request coalescing. When many requests miss on the same key at once, only one queries the database and the rest wait for its result: the single-flight pattern. It's especially effective during cold starts, because many users ask for the same popular keys. See cache failure modes.

Monitoring cold starts

The hit rate is the main signal: a sudden drop from steady state (say 95%) towards 0% means a cold start. Correlate it with deploys, node replacements and flushes to find the cause. Then watch the warm-up curve, meaning how fast the hit rate recovers. A healthy warm-up reaches 80% of steady state within minutes for most workloads; if recovery is slow, the working set is too large for natural traffic to refill quickly, and pre-loading would help.