Non-Functional Requirements: Availability, Scalability, Latency, Consistency

Functional requirements say what a system does: post a photo, send a message, book a seat. Non-functional requirements say how well it must do it: how often it may be down, how much traffic it must absorb, how fast it must answer, and how accurate its data must be. In an interview they are what turns a toy design into a real one, because each requirement forces specific components into the architecture.

Scale is what drives complexity. A chat app for 100 users works fine with plain request–response calls to one database. The same app for 100 million users needs caching layers, message queues and distributed databases. Twitter's timeline shows the same thing: a simple database query works for thousands of users, but millions need precomputed feeds cached in Redis.

Four come up in nearly every design. Others matter in specific problems: reliability (correct results), durability (data is never lost), efficiency (no wasted work), security and privacy, and cost.

Availability: staying online

Availability is the share of time a system is up, written as a percentage. The decimals matter enormously: 99.9% allows about 8.77 hours of downtime a year, while 99.99% allows only 52.6 minutes. Those two targets lead to completely different architectures. See availability, SLAs and the nines for the full table.

• Redundancy removes single points of failure. Production systems run several servers spread over different data centers, and traffic moves automatically to healthy ones when one fails. Netflix runs each service across three AWS availability zones; if a whole zone goes down, people keep streaming.
• Health checks enable fast recovery. Load balancers probe their backends every few seconds, and a failed check reroutes traffic within roughly 10–30 seconds. Slack's load balancers detect failed servers and move WebSocket connections elsewhere before most users notice. See health checks and failover.
• Geographic distribution limits the blast radius. Critical systems replicate across regions and continents. When AWS us-east-1 has an outage, globally distributed services like Route 53 keep answering from other regions, so one region's failure doesn't become a global outage. See geo-replication.

Scalability: handling growth

Scalability means keeping performance steady as load grows. In practice, scaling out with many small servers usually beats scaling up to ever-bigger machines: you can grow and shrink with demand, and one failed server is a small loss. See scalability.

• Stateless services scale horizontally. Each server handles requests on its own, keeping no sessions or temporary data in memory, so servers can be added or removed freely. Uber's ride-matching runs hundreds of identical stateless containers that autoscaling adds at peak hours. See stateless vs stateful.
• Caching takes pressure off the database. Hot data lives in memory instead of on disk. Reddit keeps post rankings and user profiles in Redis so millions of requests don't overwhelm PostgreSQL; hit rates above 90% are normal for read-heavy workloads. See caching.
• Async processing smooths spikes. Queues separate accepting a request from doing the work. When you upload a video to YouTube the upload completes at once and transcoding happens in the background, so encoding delays never slow the upload during a surge. See message queues.

Latency: speed matters

Latency is response time: the delay between sending a request and getting the answer. People expect web pages in under a second and API calls in milliseconds, and higher latency measurably hurts engagement and revenue. See latency vs throughput.

• Get close to users. Light travels at a fixed speed, so distance adds unavoidable delay. CDNs serve static files from locations near users; Cloudflare serves images and CSS from 270+ cities, cutting typical latency for distant users from around 200 ms to 20 ms.
• Fix slow queries. Slow database queries dominate many response times. Instagram brought feed queries from about 1 second to 50 ms by adding indexes and removing joins. Profile queries to find where the time goes. See database optimization.
• Compute at the edge. Instead of sending every request to a central region, edge nodes run code near users. Fastly runs customer code at its edge locations so many API requests never go to the origin, cutting latency by up to 10× for global apps.

Consistency: data accuracy

Consistency means every replica shows the same data. In a distributed system this is hard, because network partitions and failed servers can leave nodes temporarily disagreeing. See consistency models and the CAP theorem.

• Strong consistency costs availability. A bank balance must be identical everywhere: you can't withdraw the same $100 twice. Banks use synchronous replication, where every replica confirms a write before the transaction completes. Accuracy is guaranteed, but writes stall when a replica is slow or down.
• Eventual consistency enables scale. Social networks accept brief disagreement for speed. When you post on Facebook, people in different regions may see it at slightly different moments while the update propagates; within seconds everyone converges.
• Mix models by operation. Most systems use both. Amazon's shopping cart is eventually consistent (an item might briefly vanish and reappear), but charging your card is strongly consistent. Each operation gets the level it needs.

Patterns for each requirement

Trade-offs shape the architecture. These requirements pull against each other. Strong consistency lowers availability during network partitions. Lower latency usually means more complexity and cost. Good systems rank the requirements by what the business needs rather than trying to maximize all of them.

How each building block affects the requirements

Every standard component helps some requirements and can hurt others. Knowing both sides is what lets you justify a component in an interview.

Load balancing

A load balancer spreads traffic over many servers. Availability: if one server dies, traffic goes to the others, so no single server is a point of failure. Scalability: more servers can serve more requests in parallel. Latency: because no server is swamped, each one keeps answering quickly.

Caching

A cache keeps a copy of data in fast temporary storage so later requests are served quickly. Availability: it indirectly prevents overloads and crashes by cutting load. Scalability: the database handles fewer reads, so the system serves more users. Latency: memory reads are far faster than the source. Consistency is the cost: when the database changes but the cached copy doesn't, users get stale data. Managing that is the job of cache invalidation.

Partitioning

Partitioning (sharding) splits a database into smaller pieces on different nodes.

• Availability: a failure in one partition leaves the others serving; combined with replication, even the failed partition's data stays available.
• Scalability: data is divided among nodes, so the system can store and process more than a single database server (DBMS) could, and new partitions can be added as it grows.
• Latency: keeping related data in one partition lets queries go straight to it instead of scanning everything.
• Throughput: queries on different partitions run in parallel, so the system handles more operations per second.
• Consistency: harder, because data spread over nodes must be kept in sync. Protocols like two-phase commit make strong consistency possible, and the simplest fix is to keep data that must change together in the same partition.

Replication

Replication keeps several copies of the same data. Availability: if the primary fails, a copy takes over. Scalability: reads spread across copies. Consistency: the copies must change together, and if one is updated before the others, users can see different answers depending on which copy they read. Consistency models (strong, eventual, causal) define what readers are guaranteed.

Message queues

A message queue lets sender and receiver work asynchronously: they never need to be active at the same moment. Availability and reliability: messages are stored until processed, so work isn't lost when a component fails. Scalability: tasks move off the request path and are processed in the background, so larger volumes can be handled.

Batch processing

Batch processing runs many jobs together instead of one at a time, often off-peak. It improves throughput and scalability for non-interactive work such as analytics or backups. The cost is latency: each item waits for its batch.

Stating requirements in an interview

Vague requirements ("it should be fast and scalable") don't constrain anything. Turn each one into a target the design can be checked against: