PostgreSQL High Availability & Failover

High availability (HA) for PostgreSQL means the database keeps serving even when a server fails — no single machine is a fatal single point of failure. Because a standard Postgres instance runs on one primary node, HA is built by adding replicas (standbys kept in sync via streaming replication) and a mechanism to fail over — promote a replica to primary — automatically when the primary dies.

The hard parts aren't running a replica; they're the decisions around it: how much data loss you'll accept on failover (synchronous vs asynchronous replication), how failover is triggered without a human awake at 3am, and how to avoid the catastrophic split-brain scenario where two nodes both think they're primary. This page is the Postgres-specific view of HA; it builds on the general high availability and database replication concepts.

TL;DR

Quick Example

The core HA topology: a primary streams WAL to standbys; on failure, a replica is promoted and clients are redirected.

Applications connect through a stable endpoint (a virtual IP, a proxy like HAProxy/PgBouncer, or DNS) so that when failover repoints it to the new primary, clients reconnect without config changes.

Core Concepts

Streaming Replication and Hot Standbys

Postgres HA rests on physical streaming replication: the primary continuously ships its WAL to standby servers, which apply it to stay nearly identical. A standby configured as a hot standby can also serve read-only queries, so replicas do double duty as read-scaling capacity. On primary failure, a standby is promoted to become the new primary. This is the same physical replication used for read replicas — HA is about adding the failover machinery on top.

Synchronous vs Asynchronous

The central tradeoff, and it's about data loss:

Asynchronous replication (the default) commits on the primary without waiting for replicas, so a sudden primary loss can drop the last few not-yet-shipped transactions. Synchronous replication makes the primary wait until a replica has the transaction before confirming the commit — guaranteeing durability across failover, but adding latency to every write. Choose per your tolerance for data loss vs write speed; some setups use synchronous to one nearby replica and async to others.

Failover and Orchestration

Detecting that the primary is really down (not just a transient blip) and promoting a replica safely is not something to do by hand under pressure. Orchestration tools automate it:

The orchestrator watches health, coordinates promotion through consensus (so only one node becomes primary), fences the old primary, and updates the connection endpoint.

Split-Brain: The Failure to Prevent

Split-brain is when a network partition leaves two nodes each believing they're primary and both accepting writes — producing divergent, conflicting data that's extremely painful to reconcile. It's the worst HA failure. Prevention relies on quorum/consensus (a majority must agree who's primary, so a minority partition can't self-promote) and fencing (forcibly stopping or isolating the old primary before promoting a new one). This is the reason to use a proven orchestrator rather than a hand-rolled promotion script.

Best Practices

Route Clients Through a Stable Endpoint

Applications shouldn't hardcode the primary's address. Put a stable indirection in front — a virtual IP, a proxy (HAProxy, PgBouncer), or service discovery — so failover repoints that endpoint and clients simply reconnect. Without it, every failover is a manual reconfiguration scramble.

Use a Proven Failover Orchestrator

Don't hand-roll failover. Use Patroni (or an equivalent, or a managed service) with a consensus store, so promotion is coordinated, quorum-gated, and fenced against split-brain. Automatic, correct failover is subtle; battle-tested tooling exists for good reason.

Choose Sync/Async by Data-Loss Tolerance

Decide explicitly: if losing the last few transactions on failover is unacceptable (payments, orders of record), use synchronous replication to at least one replica and accept the write latency. If some loss is tolerable and latency matters more, asynchronous is fine. Don't leave it to default without a decision.

Remember Replicas Are Not Backups

A replica faithfully replicates mistakes — a DELETE FROM or a DROP TABLE propagates to the standbys instantly. HA protects against hardware/node failure, not human error or corruption. You still need backups and PITR to recover from bad data. HA and backups are complementary, not substitutes.

Test Failover Regularly

Practice failover in a controlled setting (game days). Verify promotion works, clients reconnect, no split-brain occurs, and your monitoring catches it. An HA setup that's never been failed over on purpose is an HA setup you don't actually trust.

Common Mistakes

Treating Replicas as Backups

Hand-Rolled Failover Scripts

A custom "if primary down, promote replica" script without quorum and fencing eventually causes split-brain during a network partition. Use a consensus-based orchestrator (Patroni) that only promotes with majority agreement and fences the old primary.

Clients Hardcoding the Primary

If applications connect directly to a specific host, every failover requires reconfiguring and redeploying them. Front the database with a stable endpoint that failover updates automatically.

FAQ

How is high availability different from replication?

Replication is the mechanism — streaming WAL to keep standby copies in sync. High availability is the goal — staying up through failures — which uses replication plus automatic failover, health detection, split-brain prevention, and client redirection. You can have replication (read replicas) without HA; HA adds the machinery to promote a replica and reroute traffic when the primary dies.

Synchronous or asynchronous replication — which should I use?

It's a data-loss vs latency decision. Asynchronous (the default) is faster because the primary doesn't wait for replicas, but a sudden primary loss can drop the last few transactions. Synchronous makes commits wait until a replica has the data, guaranteeing no committed data is lost on failover, at the cost of higher write latency. Use synchronous (to at least one replica) when zero committed-data loss is required; asynchronous when some loss is tolerable and latency matters.

What is split-brain and how do I prevent it?

Split-brain is when a network partition leaves two nodes both acting as primary and accepting writes, creating divergent, conflicting data that's very hard to reconcile — the worst HA failure. Prevent it with quorum/consensus (a majority must agree on the primary, so a minority partition can't self-promote) and fencing (isolating the old primary before promoting a new one). This is why you use a proven orchestrator like Patroni rather than a custom promotion script.

Do I still need backups if I have replicas?

Yes, absolutely. Replicas protect against hardware and node failure, but they replicate everything — including an accidental DELETE or DROP, which propagates to all replicas instantly. Backups and point-in-time recovery are what let you recover from human error and corruption. HA and backups address different failure modes; you need both.

How do applications keep working through a failover?

By connecting through a stable indirection — a virtual IP, a proxy (HAProxy/PgBouncer), or service discovery — rather than the primary's raw address. When the orchestrator promotes a new primary, it updates that endpoint, and clients reconnect to the right node automatically. The application's connection string never changes. Set this up before you need it, or every failover becomes a manual reconfiguration.

Related Topics

References