High Availability (HA)
High availability is designing systems to stay operational even when individual components fail. The mindset shift is crucial: HA isn't about preventing failures (they're inevitable) — it's about surviving them so users barely notice.
You achieve it by removing single points of failure with redundancy, detecting failures with health checks, and recovering automatically through failover. And because "available" also means "recoverable," tested backups are part of HA, not separate from it.
TL;DR
- HA is about surviving failures, not preventing them.
- Remove single points of failure with redundancy at every layer.
- Make failures detectable (health checks) and recoverable (failover + automation).
- Define SLOs, identify SPOFs, and rehearse failover (game days).
Quick Example
The foundational pattern — redundant instances behind a health-checking load balancer, spread across zones:
Core Concepts
- Redundancy across failure domains — multiple instances across availability zones; never put all critical components in one zone or host.
- Health checks — liveness ("is the process alive?") and readiness ("can it safely take traffic?") drive routing.
- Stateless services — keep state in durable, replicated systems so any instance can serve any request.
- Graceful degradation — shed non-critical features under stress (serve cached pages, disable extras) rather than failing entirely.
Active-Active vs Active-Passive
HA and Databases
The database is often the hardest part. Replication improves read availability and enables failover, but introduces lag — and during a network partition you trade consistency against availability (the CAP tension). Design for it explicitly. See Database Replication.
Best Practices
- Define SLOs — what "available" means for your product (e.g. 99.9%).
- Inventory single points of failure — DB, cache, auth provider, storage, DNS.
- Health checks + alerts on customer-impacting symptoms, not just host metrics.
- Run game days — controlled failover exercises so failover works when it's real.
Common Mistakes
A single availability zone
Never testing failover
FAQ
What's the difference between HA and disaster recovery?
HA keeps a service running through component failures (a dead instance or zone) with little or no downtime, usually automatically. Disaster recovery is about recovering from large-scale disasters (region loss, data corruption) using backups and runbooks, typically with a defined RPO/RTO. You need both.
Active-active or active-passive?
Active-active serves traffic from all nodes (near-zero failover, full capacity use) but adds state/consistency complexity and split-brain risk. Active-passive keeps a standby that takes over on failure — simpler, but with some failover delay and idle capacity. Choose by how much downtime and complexity you can accept.
How do I find single points of failure?
Trace every dependency in the request path — load balancer, app instances, database, cache, auth provider, DNS, storage — and ask "what happens if this one thing dies?" Anything whose failure takes down the service is a SPOF to make redundant or degrade gracefully around.
How many nines do I actually need?
Match availability targets to product needs and cost: 99.9% (~8.7h/year of downtime) is a common baseline; each additional nine is dramatically more expensive and complex. Set an SLO users would actually feel, not the highest number you can imagine.
Related Topics
- Infrastructure Architecture — Designing for resilience
- Database Replication — HA for data
- Monitoring & Alerting — Detecting failures
- Deployment Strategies — Zero-downtime releases
- Caching Layers — Serving through degradation