Load Balancing: L4 vs L7 and Algorithms

A load balancer sits between clients and a pool of servers and decides which server handles each request or connection. It works as a reverse proxy: clients talk to the balancer as if it were the server, and it forwards each request to a backend chosen by an algorithm. The goal is to use every server well, keep response times low and never overload any single one. It is the component that turns "one server" into "a fleet of interchangeable servers": it spreads load, hides failed instances, and lets you add or remove capacity without clients noticing.

Why you need one

Once servers are stateless, scaling out needs two things: a load balancer in front of the servers to decide which one handles each request, and a mechanism to launch more servers when traffic is high, remove them when it's low, and replace servers that fail. That mechanism is autoscaling; the balancer is the other half.

• Horizontal scaling. Adding servers only helps if traffic actually reaches them. The load balancer makes a pool of stateless servers behave like one big server. See scalability.
• Availability. With health checks, a crashed or overloaded instance is taken out of rotation in seconds, so users see a slightly slower site instead of errors. See availability.
• Zero-downtime deploys. You drain an instance (stop sending it new requests, let in-flight ones finish), upgrade it, and put it back. Rolling and canary releases depend on this. See deployment strategies.
• One stable address. Clients talk to one hostname. Which machines sit behind it can change every minute.

Load balancers come as hardware appliances (F5, Citrix), virtual appliances, or software (Nginx, HAProxy, Envoy) and managed cloud services (AWS ALB/NLB, Google Cloud Load Balancing).

Load balancing algorithms

Round robin

Requests go to each server in turn: 1, 2, 3, 1, 2, 3… Simple and fair when servers are identical and requests cost about the same.

Least connections

Each new request goes to the server with the fewest active connections. Better than round robin when requests vary in length, or for long-lived connections such as WebSockets, because a server stuck with slow requests stops receiving new ones.

Least response time

Requests go to the server that is answering fastest right now (usually combined with fewest connections). It adapts automatically when one server is slow because of a noisy neighbor, garbage collection or a bad disk.

IP hash

The balancer hashes the client's IP address to pick a server, so the same client always lands on the same server. Useful for simple affinity, but adding or removing a server reshuffles most clients (consistent hashing reduces that), and many users behind one corporate NAT all hit the same server.

More variants

• Weighted round robin: like round robin, but each server gets traffic in proportion to its capacity. A 16-core server can take twice the share of an 8-core one, or a canary can get 5%.
• URL hash: like IP hash, but on the request URL. Each server ends up handling a fixed set of URLs, which is great when servers cache content locally: each URL is cached on exactly one server, so the cache hit rate is maximized.
• Power of two choices: pick two servers at random and send to the less loaded one.
• Consistent hashing: map keys onto a ring so adding a server moves only a small share of keys. See consistent hashing.

Power of two choices is worth knowing. Always sending to the least-loaded server sounds best, but many balancers acting on slightly stale data all pick the same server at once and overload it. Picking the better of two random servers gets almost all of the benefit with none of the herding.

Layer 4 vs layer 7 load balancing

Load balancers differ mainly in how much of the request they look at, named after the layers of the OSI networking model. Layer 4 (transport) balancing works on TCP/UDP connections and packets; layer 7 (application) balancing works on whole HTTP requests. Application developers mostly deal with layer 7, because that's where web apps live; layer 4 is more often the concern of network engineers and SREs.

Most web systems use a layer 7 load balancer at the edge because routing by path and host is so useful. Layer 4 balancers appear where raw throughput matters or the protocol isn't HTTP: database proxies, game servers, and the tier in front of a fleet of L7 proxies at very large scale.

Health checks

The balancer periodically calls each server, for example GET /healthz every 5 seconds, and removes a server after a few consecutive failures (often 2 or 3), then adds it back after a few successes. Two kinds matter:

• Active checks: the balancer probes each server on a timer. Simple, but a server can pass the probe while failing real traffic.
• Passive checks (outlier detection): the balancer watches real responses and ejects a server that returns too many 5xx errors or times out.

Keep the health endpoint honest but shallow: it should fail if the process can't serve, not because a shared database is slow, otherwise one database blip removes every server at once. More in health checks and failover.

Hands-on: load balancing with Nginx

Nginx is an open-source web server, reverse proxy and load balancer, popular because it handles huge numbers of concurrent connections with little memory and CPU. Here is a small demo that spreads traffic across three web servers. You need Python with Flask, and Nginx installed.

1. A tiny web server that replies with its own port number:

# demo.py
import sys
from flask import Flask

app = Flask("demo")

@app.route("/")
def hello():
    return f"Hello, {app.config['PORT']}!\n"

if __name__ == "__main__":
    port = int(sys.argv[1]) if len(sys.argv) > 1 else 8000
    app.config["PORT"] = port
    app.run(host="127.0.0.1", port=port)

Start three copies, each in its own terminal:

python demo.py 8001
python demo.py 8002
python demo.py 8003

2. Configure Nginx as the load balancer. Create /etc/nginx/conf.d/demo.conf. The upstream block lists the pool; proxy_pass sends every request to it.

upstream demo {
    server localhost:8001;
    server localhost:8002;
    server localhost:8003;
}

server {
    listen 8080;
    location / {
        proxy_pass http://demo;
    }
}

Reload Nginx with sudo systemctl restart nginx, then open http://localhost:8080/ in a browser or call it a few times from a terminal (this works the same on Debian, Ubuntu or macOS):

$ for i in 1 2 3 4; do curl -s localhost:8080/; done
Hello, 8001!
Hello, 8002!
Hello, 8003!
Hello, 8001!

Nginx uses round robin by default, so the replies rotate through the three servers. To try other algorithms, add least_conn; or ip_hash; as the first line inside the upstream block, or give a server a weight with server localhost:8001 weight=3;.

3. Watch it survive failures. Stop the servers on 8002 and 8003 and keep calling port 8080. The site is still up and every reply is Hello, 8001!. Two of three servers are down and users notice nothing. That is load balancing's contribution to high availability, and it's also how you scale: put more servers behind the balancer.

Open-source Nginx detects failures passively: when a request to a server fails, it retries on the next one and marks the failed server unavailable for a while (max_fails and fail_timeout control this). Managed balancers and Nginx Plus add active health checks.

Sticky sessions and why to avoid them

Session affinity pins a user to one server, usually with a cookie, so in-memory session data stays usable. It works, but it fights everything a load balancer is for: load becomes uneven, and a server failure logs out everyone pinned to it. The better design is stateless servers with session state in a shared store like Redis or in a signed token, so any server can handle any request. See stateless vs stateful.

Avoiding a single point of failure

A single load balancer would itself be a single point of failure. Real deployments run balancers in pairs or fleets: managed cloud balancers are already redundant across zones, and self-hosted ones use a floating virtual IP (keepalived/VRRP) or anycast so several machines answer on one address. At the very top, DNS spreads users across regions, each with its own balancers. See how DNS works.