Service Mesh and the Sidecar Pattern
What a service mesh does: sidecar proxies, mTLS, retries, traffic shifting and observability, and when it is worth the complexity.
A service mesh moves the networking concerns of service-to-service communication (encryption, retries, timeouts, load balancing, traffic shifting and telemetry) out of application code and into a dedicated infrastructure layer. Usually that layer is a sidecar proxy deployed next to every service instance, configured centrally by a control plane.
How it works
- Data plane: the proxies (usually Envoy, or Linkerd's Rust proxy) that intercept all inbound and outbound traffic of each service.
- Control plane: distributes routing rules, policies and certificates to the proxies and collects their telemetry.
- Some newer meshes avoid per-pod sidecars (Istio ambient mode, Cilium with eBPF) by using per-node proxies, reducing resource overhead.
What a mesh gives you
| Capability | Without a mesh | With a mesh |
|---|---|---|
| Encryption between services | TLS configured in every service | Automatic mutual TLS with rotated certificates and service identities |
| Retries, timeouts, circuit breaking | A library per language, configured per service | Uniform policies set centrally. See retries |
| Load balancing | Client libraries or central load balancers | Per-request balancing with outlier detection in every proxy |
| Traffic shifting | Deploy-time changes | Canary 5% → 50% → 100% by config; header-based routing. See deployment strategies |
| Access control | Custom auth between services | Policy: "only orders may call payments" |
| Observability | Instrument every service | Golden metrics (rate, errors, latency) and trace spans for every hop. See observability |
The costs
- Latency: each call passes through two extra proxies, typically adding around a millisecond or less per hop, which compounds in deep call chains.
- Resources: a sidecar per pod costs CPU and memory across the whole fleet.
- Operational complexity: the control plane is critical infrastructure; misconfiguration can break all traffic; upgrades must be managed carefully.
- Debugging: failures can now come from the proxy layer as well as the application.
A mesh pays off with many services, many languages, and requirements like mTLS everywhere or fine-grained traffic control. With a handful of services in one language, a good RPC library and an API gateway are usually simpler.
Mesh vs API gateway
An API gateway handles north-south traffic: clients entering the system, with concerns like API keys, public rate limits and request aggregation. A mesh handles east-west traffic between internal services. Many architectures use both, often with Envoy powering each.
Where it’s used
The most widely deployed mesh on Kubernetes, built on Envoy, offering mTLS, traffic management and policy; now also in sidecar-less ambient mode.
A lightweight mesh focused on simplicity and low overhead, with automatic mTLS and golden metrics.
Lyft built Envoy to handle service-to-service networking uniformly across its polyglot microservices.
Use meshes for zero-trust networking (mTLS everywhere) and gradual traffic shifting during migrations.
In an interview
- Bring up a mesh when you have many services needing mTLS, consistent retries and traffic shifting.
- Explain sidecar proxies and the control plane in a sentence each.
- Acknowledge latency and operational costs, and when a library or gateway is enough.
- Distinguish east-west (mesh) from north-south (gateway) traffic.