Skip to content

RTO Trade-offs

Trade-off analysis for RTO (Recovery Time Objective), finding the optimal balance between recovery speed and false failover risk.

RTO (Recovery Time Objective) defines the maximum time required for the system to restore write capability when the primary fails.

Critical transaction systems typically need the shortest practical RTO, often below one minute.

Shorter RTO increases false failover risk. Network jitter can appear to be a failure. Cross-datacenter or cross-region deployments often allow 1–2 minutes to reduce this risk.


Trade-offs

pg_rto controls the failover unavailability limit. Presets fast, norm, safe, and wide support different network and deployment conditions. The default is norm, approximately 45 seconds.

Recovery includes detection, DCS lock expiration, election, promotion, and HAProxy detection of the new primary. Shorter phase timeouts reduce RTO. They also increase sensitivity to network jitter and false failover risk.

Choose a preset that balances recovery speed and false failover risk for your network. Poor networks need longer timeouts. Stable networks can use shorter timeouts.


Four Modes

Select one of four RTO presets for the network conditions.

Namefastnormsafewide
Use CaseSame rackSame datacenter (default)Same region, cross-DCCross-region/continent
Network< 1ms, very stable1-5ms, normal10-50ms, cross-DC100-200ms, public network
Target RTO30s45s90s150s
False Failover RiskHigherMediumLowerVery Low
Configurationpg_rto: fastpg_rto: normpg_rto: safepg_rto: wide
fast: Same Rack/Switch
  • Use for stable networks below 1ms latency, such as one rack or switch.
  • Average RTO: 14s; worst case: 29s. TTL is 20s; check interval is 5s.
  • This needs the best network quality. Jitter can trigger failover, so false failover risk is higher.
norm: Same Datacenter (Default)
  • Default mode for normal same-datacenter networks, 1–5ms latency, and reasonable packet loss.
  • Average RTO: 21s; worst case: 43s. TTL is 30s, allowing some network variation.
  • This balances recovery speed and stability for most production environments.
safe: Same Region, Cross-Datacenter
  • Use for same-region or same-area datacenters with 10–50ms latency and occasional jitter.
  • Average RTO: 43s; worst case: 91s. TTL is 60s.
  • The 60s primary restart wait allows more local recovery time and reduces false failover risk.
wide: Cross-Region/Continent
  • Use across regions or continents with 100–200ms latency and possible public-network packet loss.
  • Average RTO: 92s; worst case: 207s. TTL is 120s.
  • Slower recovery provides a very low false failover rate for geographic disaster recovery.

RTO Timeline

Patroni HA has two main paths. Active detection finds a PostgreSQL crash and attempts a restart. Passive lease expiration waits for the TTL after node failure, then starts an election.


Implementation

Each preset selects values for these 10 Patroni and HAProxy parameters:

ComponentParameterfastnormsafewideDescription
patronittl203060120Leader lock TTL (seconds)
loop_wait551020HA loop check interval (seconds)
retry_timeout5102030DCS operation retry timeout (seconds)
primary_start_timeout15254595Primary restart wait time (seconds)
safety_margin551015Watchdog safety margin (seconds)
haproxyinter1s2s3s4sNormal state check interval
fastinter500ms1s1500ms2sState transition check interval
downinter1s2s3s4sDOWN state check interval
rise3333Consecutive successes to mark UP
fall3333Consecutive failures to mark DOWN

Patroni Parameters

  • ttl: Leader lease lifetime. Failure to renew triggers expiration and election. It sets passive detection delay.
  • loop_wait: Main loop interval. Each cycle checks health and synchronizes state.
  • retry_timeout: DCS retry period. After partition-related retries fail, the primary demotes to prevent split brain.
  • primary_start_timeout: Time allowed for local restart after a PostgreSQL crash. On expiration, release the Leader lock and fail over.
  • safety_margin: Watchdog time allowance for restarting the system before split brain can occur.

HAProxy Parameters

  • inter: Check interval while service state is stable.
  • fastinter: Shorter check interval during state changes.
  • downinter: Check interval for recovery after a service is marked DOWN.
  • rise: Consecutive successes required for UP. A new primary must pass this many checks before receiving traffic.
  • fall: Consecutive failures required for DOWN.

Key Constraint

Patroni constraint: The primary must complete demotion before TTL expiration to prevent split brain.

loop_wait+2×retry_timeout≤ttlloop\_wait + 2 \times retry\_timeout \leq ttl

Data Summary


Recommendations

fast requires latency below 1ms and very low packet loss. Use it only within one rack or switch for strict RTO targets. Test it thoroughly in production before enabling it.

norm is the default for most same-datacenter deployments. This page’s model gives passive and active averages of about 34 and 35 seconds. Its timeout allowance reduces false failovers from jitter.

safe suits same-city datacenters with higher latency or occasional jitter. Longer timeouts reduce false failovers. It is recommended for cross-datacenter recovery.

wide suits high-latency, cross-region or cross-continent networks, including public-network packet loss. It favors stability over speed. Its long timeouts keep false failover rates very low.

ModeTarget RTOPassive RTOActive RTOScenario
fast3016 / 23 / 291 / 24 / 29Same switch, high-quality network
norm4527 / 34 / 412 / 35 / 41Default, same DC, standard network
safe9053 / 66 / 783 / 61 / 73Same-city active-active / cross-DC DR
wide150104 / 127 / 1504 / 122 / 145Geo-DR / cross-country
default32622 / 34 / 462 / 314 / 326Patroni default params

Set pg_rto to a preset name to configure Patroni and HAProxy automatically. The template checks pg_rto in pg_rto_plan. A numeric or unknown key falls back to norm. This fallback does not support an RTO expressed as seconds.

Presets come from pg_rto_plan. Override this mapping for a custom RTO strategy.

pg_rto_plan:  # [ttl, loop, retry, start, margin, inter, fastinter, downinter, rise, fall]
  fast: [ 20  ,5  ,5  ,15 ,5  ,'1s' ,'500ms'  ,'1s' ,3 ,3 ]  # rto < 30s
  norm: [ 30  ,5  ,10 ,25 ,5  ,'2s' ,'1s'     ,'2s' ,3 ,3 ]  # rto < 45s
  safe: [ 60  ,10 ,20 ,45 ,10 ,'3s' ,'1500ms' ,'3s' ,3 ,3 ]  # rto < 90s
  wide: [ 120 ,20 ,30 ,95 ,15 ,'4s' ,'2s'     ,'4s' ,3 ,3 ]  # rto < 150s

Was this page helpful?