Key-Value Databases: Redis, DynamoDB and When to Use Them

A key-value database is the simplest NoSQL store: it works like a giant hash map. You put a value under a key, get it back by key, and delete it by key. That's the whole interface. Many features need nothing more (sessions, caches, shopping carts, user preferences), and in return a key-value store gives very fast lookups and almost effortless horizontal scaling.

Why key-value databases exist

Amazon's engineers spelled out the pattern in the original Dynamo paper: many services on Amazon's platform (best-seller lists, shopping carts, customer preferences, session management, sales rank, the product catalog) only ever access their data by primary key. Running them on relational databases used those databases inefficiently and limited scale and availability. Dynamo offered a simple primary-key-only interface built for exactly those needs, and it inspired DynamoDB, Cassandra and Riak. See Dynamo-style key-value stores.

How key-value databases work

Each record has two parts, a unique key and a value, and the database offers three operations: get(key), put(key, value) and delete(key). No schemas, no relationships, no joins.

• Primary-key lookups are fast. The database hashes the key to find its exact location: no table scans, no index traversals. In-memory stores answer in microseconds.
• Horizontal scaling is straightforward. Partition keys by range or by hash and add servers for capacity; each server owns a subset of keys. Because no query ever spans partitions, the partitioning logic stays simple. See consistent hashing.
• The API matches programming-language dictionaries. Developers already know get and put: no SQL to learn, no ORM, and the application's model lines up with the database.

import json
import boto3
import redis

# In-memory key-value: a session store with a 30-minute expiry
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
r.set("session:abc123", json.dumps({"user_id": 456, "role": "member"}), ex=1800)
session = json.loads(r.get("session:abc123") or "null")
r.delete("session:abc123")                                  # logout

# Durable key-value: a DynamoDB table "file_metadata" keyed by "file_id"
table = boto3.resource("dynamodb", region_name="us-east-1").Table("file_metadata")
table.put_item(Item={"file_id": "f_789", "owner": "u456", "size": 52480, "path": "/docs/plan.pdf"})
item = table.get_item(Key={"file_id": "f_789"}).get("Item")
table.delete_item(Key={"file_id": "f_789"})

Designing keys

With lookup by key as the only query, the key is the design. Namespacing keys keeps different kinds of data apart and makes them predictable to build:

Anything beyond lookup by key ("all carts containing SKU 9", "sessions older than a day") needs a second structure you maintain yourself, or a different kind of database. Some stores relax this: DynamoDB adds a sort key and secondary indexes, and Redis offers sorted sets and other data structures. See cache key design.

When to use a key-value database

• High-performance reads and writes: reading and writing by key is about as fast as storage gets, so applications that need high-speed operations choose key-value stores.
• Simple data structures: customer details, web pages keyed by URL, shopping-cart contents, product categories and e-commerce product details, where each entity is accessed independently with no relationships.
• Basic query requirements: no complex queries and no updates that must span several records together; plain storage and retrieval.

Avoid them when you need ad-hoc queries, joins, or transactions across many records; use a relational database or a document database instead.

Popular key-value databases

Thanks to their speed and simplicity, key-value stores rank high in popularity. The DB-Engines ranking of key-value stores in March 2023 (popularity score, not performance):

Further down the same list: Riak KV, ArangoDB, RocksDB, OrientDB, GemFire, Oracle NoSQL, LevelDB, Infinispan, Oracle Berkeley DB, InterSystems IRIS, LMDB, InterSystems Caché and Amazon SimpleDB.

Real-world examples

• Dropbox keeps file metadata in DynamoDB through its Alki system: each file's metadata maps to a unique file ID, relationships between files don't matter, and simple key lookups give the speed it needs. It saved millions in infrastructure costs compared with the previous on-premises storage.
• Snapchat stores Stories in DynamoDB: a story ID maps to its content metadata, stories don't relate to each other, and their short-lived nature suits key-value simplicity. Snap handles over 10 million queries per second and cut median latency by 20%.
• Disney+ uses DynamoDB for content metadata and user interactions: a video ID maps to its title, description and thumbnail URL, and each video loads independently. It ingests billions of customer actions a day for watchlists and personalized recommendations.