Serverless Architecture
Serverless architecture explained: functions as a service, cold starts, scaling to zero, limits, pricing, and the workloads it suits.
Serverless computing lets you run code without managing servers: you deploy functions or containers, and the platform runs them on demand, scales them automatically (including to zero), and bills only for actual usage. AWS Lambda, Google Cloud Functions and Cloud Run, Azure Functions, Cloudflare Workers and Vercel Functions are common examples.
How functions as a service work
- Each instance typically handles one request at a time (Lambda) or several (Cloud Run, Vercel's concurrency options); the platform adds instances as concurrency rises.
- Instances are ephemeral: no local state survives between invocations reliably. Keep state in databases, caches and object storage.
- Billing is per request plus compute time (often per millisecond × memory), so idle costs nothing.
Cold starts
When no warm instance is available, the platform must create one: start a micro-VM or container, load the runtime and your code, run initialisation. This cold start adds anywhere from tens of milliseconds (lightweight runtimes, V8 isolates at the edge) to seconds (large Java or .NET apps, big dependencies, VPC networking).
- Keep deployment packages small and initialisation light; move work out of the startup path.
- Use provisioned concurrency or minimum instances for latency-sensitive endpoints (you pay for them while idle).
- Reuse connections across invocations by creating clients outside the handler.
Limits to design around
| Limit | Typical values | Design implication |
|---|---|---|
| Max execution time | Lambda: 15 min; HTTP functions often less | Long jobs need queues, step functions or containers |
| Concurrency limits | Account-level caps (e.g. 1,000 by default on Lambda) | A traffic spike can be throttled; reserve concurrency for critical functions |
| Database connections | Thousands of instances × connections | Use connection poolers or proxies (RDS Proxy, PgBouncer) or HTTP-based databases |
| Payload sizes | A few MB per request/response | Large files go directly to object storage via presigned URLs |
| Local disk and memory | Limited, ephemeral | No big local caches; use Redis or similar |
Serverless data pipelines
Pipelines can be built entirely from event-triggered functions, orchestrated by a managed workflow service (AWS Step Functions, Google Workflows, Azure Durable Functions). A file landing in S3 triggers a function that validates it, another transforms it, and a workflow coordinates retries and branching. You pay only while functions run.
- Good fits: spiky or infrequent events (file drops, webhooks, IoT messages), small transformations, glue between managed services. Coca-Cola's connected vending machines are a well-known example of this API Gateway + function shape.
- Limits: a maximum run time per invocation (15 minutes on AWS Lambda), so large batch transformations belong in Spark or a warehouse; per-invocation and state-transition costs grow quickly at sustained high volume.
- A cautionary tale: Amazon's Prime Video team moved a video-quality monitoring service from a Step Functions-based serverless design to a single process and reported about a 90% infrastructure cost reduction. Choose serverless for its fit, not because it's newer.
See data engineering design patterns for how serverless pipelines compare with the other architectures.
Pros and cons at a glance
| Pros | Cons |
|---|---|
| Cost-effective: you pay only for actual processing time, which saves money for sporadic or event-driven work | Cold starts: a function that has been idle takes longer to respond the first time |
| Automatic scaling: capacity follows demand, ideal for unpredictable traffic | Less control over the infrastructure and the runtime environment |
| No servers to manage: developers focus on code while the provider handles infrastructure | Platform limits: execution time, concurrency, payload size and vendor lock-in |
Serverless is the last step in the evolution from physical servers to VMs to containers; see containers vs virtual machines for how the four compare.
When serverless fits
- Spiky or unpredictable traffic, and workloads that are idle much of the time: scale to zero saves money and effort.
- Event-driven glue: processing uploads, reacting to queue messages, webhooks, scheduled jobs. See message queues.
- APIs and web backends for small teams that want zero server management.
- Edge logic (redirects, auth checks, personalisation) with edge functions close to users. See CDN.
It fits less well for steady high-throughput services (always-on containers are cheaper), latency-critical paths that can't tolerate cold starts, long-running or stateful workloads (WebSocket servers, streaming), and heavy local caching.
Where it’s used
Uploads to object storage trigger functions that create thumbnails, extract metadata or start transcoding.
Functions receive Stripe, GitHub or Slack webhooks and push work onto queues. See webhooks.
Pages and API routes run as serverless functions with automatic scaling and preview deployments.
Built large parts of their backends on Lambda to avoid managing servers for spiky, event-driven workloads.
In an interview
- Suggest serverless for spiky, event-driven or low-traffic workloads, and justify with cost and operations.
- Address cold starts, execution time limits and concurrency limits explicitly.
- Keep functions stateless; put state in managed stores and use a connection proxy for relational databases.
- Prefer containers for steady high-throughput or long-lived connection workloads.