Serverless Cold Starts & Limits in Production

Serverless Cold Starts and Operational Limits

Serverless platforms — AWS Lambda, Google Cloud Run functions, Azure Functions, and others — run your code on demand, scale automatically, and bill per use. The model hides servers, not physics. When no warm instance exists, the platform must create an execution environment, load your code, and run its initialization: a cold start that can add tens of milliseconds to several seconds. And every platform imposes limits — concurrency quotas, scaling rates, timeouts, payload sizes — that surface as throttling, dropped requests, or exhausted databases exactly when traffic spikes.

Production-ready serverless systems treat these as design inputs: keep initialization lean, reserve or pre-warm capacity where latency matters, reuse connections, size concurrency so downstream systems survive, make every handler idempotent for retries, and watch the metrics that reveal cold starts and throttles.

TL;DR

A cold start = new environment + runtime start + code load + your initialization; you control the last two most.
Reduce cold starts with small packages, lazy loading, more memory (more CPU), and lighter runtimes; remove them where needed with provisioned concurrency, SnapStart, or minimum instances.
Concurrency limits and scaling rates cause throttling; use reserved concurrency to protect critical functions and downstream systems.
Respect timeouts and payload limits: pass large data by reference to object storage.
Create clients and connections outside the handler; use connection poolers or proxies for relational databases.
Assume retries: make handlers idempotent, report partial batch failures, and route poison messages to dead-letter destinations.

Quick Example

A Lambda handler that reuses its client across invocations and reports partial failures for an SQS batch:

Measuring cold starts from Lambda logs with CloudWatch Logs Insights:

Core Concepts

Anatomy of a Cold Start

Cold starts happen on first invocation, when traffic grows beyond warm capacity, after deployments, and after idle periods. They matter most for synchronous, user-facing requests; for queue consumers, they are usually irrelevant.

Reducing and Removing Cold Starts

Scheduled "keep-warm" pings keep only one environment warm and do not help with concurrent bursts; prefer platform features.

Concurrency, Scaling, and Throttling

Platforms limit how many invocations run at once (Lambda has a regional account concurrency quota, 1,000 by default and raisable) and how fast new environments are added. When limits are reached, synchronous callers receive throttling errors (HTTP 429), and asynchronous and stream sources retry with backoff. Reserved concurrency guarantees capacity for critical functions and also caps a function, which protects downstream databases and APIs from being overwhelmed. Cloud Run additionally lets one instance serve many concurrent requests, which reduces cold starts and connection counts.

Timeouts and Payload Limits

Functions have maximum durations (15 minutes on Lambda), and fronting services often impose shorter limits — API gateways commonly time out after about 30 seconds by default. Request and response payloads are limited to a few megabytes for synchronous calls and less for asynchronous events. Move long work to queues and workflows (see AWS Step Functions), and pass large data as references to object storage with signed URLs.

Connections and Databases

Each execution environment holds its own connections. A spike to 1,000 concurrent functions can open 1,000 database connections and exhaust a PostgreSQL server. Mitigations: initialize clients outside the handler for reuse; use a connection proxy or pooler (such as RDS Proxy or PgBouncer); use HTTP-based database APIs designed for serverless; cap function concurrency to what the database can handle. See PostgreSQL Connection Pooling.

Retries and Idempotency

Without idempotency, retries create duplicate orders, emails, and charges. See Idempotency.

Diagnostics

Logs: Lambda's REPORT line includes duration, billed duration, memory used, and Init Duration on cold starts.
Metrics: invocations, errors, throttles, concurrent executions, duration percentiles, iterator age (streams), and queue age.
Tracing: X-Ray or OpenTelemetry traces show initialization, downstream calls, and retries.
Deployments: use versions and aliases with gradual traffic shifting (canary or linear) and automatic rollback on alarms; correlate latency changes with deployments, since a new version starts cold.

Best Practices

Keep Initialization Lean and Measured

Track Init Duration and package size in CI, and treat regressions like performance bugs.

Pre-Warm Only What Needs It

Use provisioned concurrency or minimum instances for latency-sensitive synchronous paths, not for background consumers.

Size Concurrency for Downstream Systems

Set reserved concurrency and batch sizes so databases and third-party APIs stay within their limits.

Design Every Handler for Retries

Make processing idempotent, report partial failures, and route poison messages to dead-letter destinations with alerts.

Common Mistakes

Creating Clients Inside the Handler

Unlimited Concurrency Against a Database

Failing Whole Batches

FAQ

What causes serverless cold starts?

Creating a new execution environment, starting the language runtime, loading your code, and running your initialization code when no warm instance is available — for example after a deployment, an idle period, or a traffic increase.

How do I eliminate cold starts on AWS Lambda?

Use provisioned concurrency for latency-sensitive functions, SnapStart for supported runtimes, and reduce initialization work; for most asynchronous workloads, cold starts do not need to be eliminated.

Why is my Lambda function being throttled?

It reached a concurrency limit — the account's regional quota, the function's reserved concurrency, or the scaling rate. Check concurrency metrics and request quota increases or adjust reserved concurrency.

How should serverless functions connect to relational databases?

Reuse connections across invocations, use a connection proxy or pooler, consider HTTP-based database APIs, and cap function concurrency to what the database can handle.

Are serverless invocations retried automatically?

Asynchronous, queue, and stream invocations are retried on failure according to the platform's rules; synchronous retries depend on the caller. Design handlers to be idempotent.