Distributed Caching: Sharding, Redis Cluster and Hot Keys

A single Redis node with 64 GB of RAM serves small and medium workloads well. When the working set or the request rate outgrows one machine, you need several cache nodes, and the central question becomes: which node holds which key? A distributed cache splits the key space across nodes so that, to the application, it still behaves like one big cache.

Why shard the cache

• Memory limit. One machine has finite RAM. A 200 GB working set can't fit on one node; spread over 4 nodes it's 50 GB each.
• Throughput limit. One Redis node handles roughly 100,000–200,000 operations a second, depending on value size, command type and hardware. If the workload needs more, you must shard even if memory would fit.

Sharding gives each node a subset of the keys. For each key, the client works out which node owns it, sends the request there and gets the answer.

Hash-based routing (modulo)

The simplest scheme: hash the key and take the remainder by the number of nodes.

node = hash(key) % num_nodes

Keys spread evenly, but adding or removing a node changes num_nodes, which changes the result for nearly every key. Going from 4 to 5 nodes remaps about 80% of keys (a key stays put only if h % 4 == h % 5). Every remapped key is now a miss on its new node, so the database takes a burst of traffic until the cache re-warms.

Consistent hashing

Consistent hashing fixes the remapping problem. Both nodes and keys are placed on a circular hash space (a ring), and each key belongs to the first node clockwise from its position. Adding a node between two others only takes keys from its clockwise neighbor; every other assignment stays put. Adding or removing a node moves only about 1/N of the keys; the other (N−1)/N stay where they are.

In practice each physical node is placed at many points on the ring (virtual nodes) so load stays even and a departing node's keys spread over all the others. The full algorithm is in consistent hashing; the same technique is used for sharding databases.

Redis Cluster: hash slots

Redis Cluster uses a fixed set of 16,384 hash slots instead of a continuous ring. Each key maps to a slot with CRC16(key) % 16384, and each node owns a range of slots. To scale from 3 to 4 nodes, you (or your managed service) move some slots from the existing nodes to the new one. Only keys in moved slots migrate; clients may see brief redirections during the move, and all other slots are unaffected.

Redis Cluster has tooling to assign and rebalance slots, and managed services often automate it. Client libraries learn the slot map from any node and send each request straight to the right node, with no proxy.

Hash tags put related keys on the same node: only the part inside braces is hashed, so {user:123}:profile and {user:123}:settings land in the same slot. That's required for multi-key operations such as MGET or transactions, which only work on co-located keys. See cache key design.

Client-side vs proxy-based routing

• Client-side routing: the client library holds the shard map (in Redis Cluster, the slot-to-node mapping) and sends each command straight to the right node. No extra hop, but every application needs a cluster-aware client.
• Proxy-based routing: a proxy such as Twemproxy or Envoy sits between the application and the cache nodes. The application uses a simple client and talks only to the proxy, which routes each request. One extra network hop, but simpler clients and a central place for access control.

Redis Cluster deployments usually use client-side routing, because direct connections save latency. Choose a proxy when applications can't use a cluster-aware client or you want centralized control.

The hot key problem

Sharding spreads keys evenly by hash, not by popularity. If one key gets 50% of all reads (a viral product page, a breaking news story), its shard handles half the total load while the others idle. That shard becomes the bottleneck no matter how many nodes you add. See hot partitions.

• Read replicas per shard: give each shard one or more replicas and spread the hot key's reads over the primary and its replicas. Helps, at the cost of complexity and memory.
• Key splitting: store the hot value under several suffixed keys (product:viral:1 … product:viral:10) and have clients read a random one. Ten copies can spread load over up to 10 shards (depending on slot placement). The write path must update all copies: write amplification.
• Local caching: keep the hot key in each application server's memory (an in-process LRU with a short TTL). This can cut Redis traffic for that key dramatically, depending on the local hit rate and TTL, but adds a second layer to keep consistent.

import random

HOT_COPIES = 10

def write_hot(key, value, ttl=60):
    for i in range(1, HOT_COPIES + 1):                     # write amplification: 10 writes
        redis.set(f"{key}:{i}", value, ex=ttl)

def read_hot(key):
    return redis.get(f"{key}:{random.randint(1, HOT_COPIES)}")   # spreads reads across shards

Next: keeping the distributed cache serving when nodes fail, in cache high availability.