Capacity Planning and Load Testing

Capacity planning makes sure a system has enough resources (servers, database throughput, network, storage, third-party quotas) to meet demand with acceptable performance, at a reasonable cost, and before the demand arrives. It combines forecasting how load will grow, measuring how much load each component can actually handle through load testing, and keeping headroom for spikes and failures.

The capacity planning loop

1. Measure current demand in the units that drive cost: requests per second per endpoint, messages per second, storage growth per day, concurrent connections.
2. Forecast growth from trends and known events: organic growth, launches, marketing campaigns, seasonal peaks such as festival sales.
3. Find each component's limit with load tests, and identify the bottleneck that saturates first.
4. Plan headroom: target utilisation that leaves room for spikes and for losing a zone.
5. Provision and verify, then repeat on a schedule, because both demand and the system change.

Headroom and utilisation targets

Running at 100% utilisation means no room for spikes, and latency rises sharply well before that because queues grow as utilisation approaches its limit. Typical targets are 50-70% at expected peak for latency-sensitive services. Also plan for N+1 or N+2 redundancy: if traffic is spread across three zones and one fails, the other two must absorb its share.

Load testing

• Model real traffic: the same mix of endpoints, payload sizes, cache hit rates and user behaviour as production. A test that only hits a cached endpoint proves nothing.
• Test in a production-like environment, or carefully in production with synthetic traffic, since staging rarely has the same data size or dependencies.
• Watch the whole path: database connections, thread pools, queues and downstream services often saturate before CPU does. See connection pooling.
• Measure latency percentiles, not averages. See latency vs throughput.

Finding the bottleneck

Every system has one resource that saturates first, and adding capacity anywhere else does nothing. Use the USE method (utilisation, saturation, errors) per resource and look for the first one to hit its limit: CPU, memory, disk I/O, network, database connections, lock contention, a single-threaded component, or a third-party rate limit. Fix or scale that one, then test again, because the bottleneck moves.

Back-of-the-envelope maths gets you a first estimate before any test. See capacity estimation.

Planning for spiky and unpredictable demand

• Autoscaling handles gradual changes but lags by minutes; pre-warm before known events. See autoscaling.
• Queues absorb bursts for work that can be asynchronous. See message queues.
• Load shedding and rate limits protect the system when demand exceeds any plan. See load shedding.
• Quotas and limits with cloud providers and third parties must be raised ahead of time.