Patroni 4.1.5 Documentation

• 1: Introduction
• 2: Installation
• 3: Patroni configuration
  • 3.1: Dynamic Configuration Settings
  • 3.2: YAML Configuration Settings
  • 3.3: Environment Configuration Settings
• 4: Patroni REST API
• 5: patronictl
• 6: Replica imaging and bootstrap
• 7: Replication modes
• 8: Standby cluster
• 9: Watchdog support
• 10: Pause/Resume mode for the cluster
• 11: DCS Failsafe Mode
• 12: Using Patroni with Kubernetes
• 13: Citus support
• 14: Convert a Standalone to a Patroni Cluster
• 15: Integration with other tools
• 16: Security Considerations
• 17: HA multi datacenter
• 18: FAQ
• 19: Release notes
• 20: Contributing guidelines

Source: https://patroni.readthedocs.io/en/latest/index.html

Warning

Running Patroni on memory-restricted systems with Python 3.11+

Unexpected behavior can occur with Python 3.11 or newer on systems with strict memory limits. For example, this includes vm.overcommit_memory=2, which is recommended for PostgreSQL. You may observe these symptoms:

• Patroni appears healthy
• PostgreSQL continues to run
• Patroni REST API becomes unresponsive
• The operating system reports that Patroni is listening on the REST API port
• Patroni logs look normal; however, following messages may appear once: Exception ignored in thread started by: <object repr() failed>, MemoryError
• Kernel logs may contain messages such as not enough memory for the allocation

A bug in Python 3.11+ causes this behavior. Under strict memory conditions, a new thread may hang indefinitely if free memory is insufficient.

Recommended solution

Recent Patroni releases (4.1.1+, 4.0.8+) start all required threads early, before the system has memory pressure. This reduces the impact of the issue.

Additional recommendations (Linux, glibc)

When you use vm.overcommit_memory=2 (recommended for PostgreSQL), we also recommend these environment variables before you start Patroni:

• MALLOC_ARENA_MAX=1 - reduces the amount of virtual memory allocated by glibc for multi-threaded applications
• PG_MALLOC_ARENA_MAX= - resets the value of MALLOC_ARENA_MAX for PostgreSQL processes started by Patroni.

In addition, you may tune the following Patroni configuration parameters:

• thread_stack_size - Stack size for threads that Patroni starts. A lower value reduces Patroni memory usage. The default is 512kB. Increase thread_stack_size if Patroni has stack-related crashes. Otherwise, the default is sufficient.
• thread_pool_size - Thread pool size for asynchronous tasks and REST API communication with other members during leader races or failsafe checks. The default is 5, which is sufficient for three-node clusters.
• restapi.thread_pool_size - Thread pool size for REST API requests. The default is 5, which allows up to five parallel requests. Requests with SQL queries effectively run in sequence because they use one database connection. A higher value typically gives no benefit.

---

Patroni is a Python template for high availability (HA) PostgreSQL solutions. It supports distributed configuration stores such as ZooKeeper, etcd, Consul, and Kubernetes. Database engineers, DBAs, DevOps engineers, and SREs can use it to deploy HA PostgreSQL quickly in datacenters or other environments.

Patroni is a template. Its replication system needs configuration for your requirements and has limitations that you must consider. PostgreSQL supports many high-availability approaches. See the PostgreSQL Documentation for a list.

Currently supported PostgreSQL versions: 9.3 to 18.

Note to Citus users: Patroni 3.0 and later integrate with the Citus database extension to Postgres. See the Citus support page for Patroni high availability with a Citus distributed cluster.

Note to Kubernetes users: Patroni can run natively on Kubernetes. See the Kubernetes chapter.

1 - Introduction

Source: https://patroni.readthedocs.io/en/latest/README.html

Patroni is a Python template for high availability (HA) PostgreSQL solutions. It started as a fork of Governor, a Compose project, and adds many features.

For additional background info, see:

• PostgreSQL HA with Kubernetes and Patroni, talk by Josh Berkus at KubeCon 2016 (video)
• Feb. 2016 Zalando Tech blog post

Development Status

Patroni is in active development and accepts contributions. See Contributing for details.

See release information for new releases.

Technical Requirements/Installation

See the installation guide to install and upgrade Patroni on different platforms.

Planning the Number of PostgreSQL Nodes

Patroni/PostgreSQL nodes are independent of DCS nodes, except when Patroni implements RAFT itself. Thus, there is no minimum node count for Patroni/PostgreSQL. A cluster can have one primary and one standby. You can add more standby nodes later.

2-node clusters (primary + standby) are common and provide automatic failover with high availability. During failover, the cluster temporarily has no redundancy until the failed node rejoins.

DCS requirements: Your DCS (etcd, ZooKeeper, Consul) must have 3 or 5 nodes for consensus and fault tolerance. One DCS cluster can store data for hundreds or thousands of Patroni clusters. Use different namespace/scope combinations for these clusters.

Running and Configuring

This section assumes you cloned the Patroni repository from https://github.com/patroni/patroni. You need the example configuration files postgres0.yml and postgres1.yml. If you installed Patroni with pip, get these files from the git repository. Then replace ./patroni.py below with the patroni command.

To start, run these commands in separate terminals:

> etcd --data-dir=data/etcd --enable-v2=true
> ./patroni.py postgres0.yml
> ./patroni.py postgres1.yml

These commands start a high-availability cluster. Test different YAML settings to see how they change cluster behavior. Stop some components to test how the system responds.

Add more postgres*.yml files to create an even larger cluster.

Patroni provides an HAProxy configuration. It gives your application one endpoint to connect to the cluster leader. To configure it, run:

> haproxy -f haproxy.cfg

> psql --host 127.0.0.1 --port 5000 postgres

YAML Configuration

See YAML settings for details about etcd, consul, and ZooKeeper settings. See postgres0.yml for an example.

Environment Configuration

See environment settings to configure or override settings with environment variables.

Replication Choices

Patroni uses Postgres streaming replication, which is asynchronous by default. Set maximum_lag_on_failover to prevent failover when a replica is more than the specified number of bytes behind the leader. Adjust this setting for your business requirements. You can also use synchronous replication for stronger durability guarantees. See replication modes documentation for details.

Applications Should Not Use Superusers

Always use a non-superuser for application connections. Patroni needs database access to operate correctly. An application that uses a superuser can consume all connections, including those reserved by superuser_reserved_connections. If no connections remain, Patroni cannot access the primary and can behave incorrectly.

Testing Your HA Solution

HA testing takes time and involves many variables, especially for a cross-platform application. A trained system administrator or consultant must do this work. These documents do not cover the process in detail.

Test these parts of your infrastructure:

• Network (the network in front of your system as well as the NICs $$physical or virtual$$themselves)
• Disk IO
• file limits (nofile in Linux)
• RAM. Even if you have oomkiller turned off, the unavailability of RAM could cause issues.
• CPU
• Virtualization Contention (overcommitting the hypervisor)
• Any cgroup limitation (likely to be related to the above)
• kill -9 of any postgres process (except postmaster!). This is a decent simulation of a segfault.

Do not run kill -9 on a postmaster process. This does not simulate a real-world failure. If an attacker can run kill -9, an HA process cannot resolve the security problem. The attacker can kill the process again or cause other damage.

2 - Installation

Source: https://patroni.readthedocs.io/en/latest/installation.html

Pre-requirements for Mac OS

On a Mac, install the requirements:

brew install postgresql etcd haproxy libyaml python

Psycopg

Since psycopg2-2.8, the binary package is not installed by default. A source installation requires a C compiler and Postgres and Python development packages. Python dependencies cannot express psycopg2 OR psycopg2-binary. Select an installation method:

1. Use the package manager from your distro

sudo apt-get install python3-psycopg2  # install psycopg2 module on Debian/Ubuntu
sudo yum install python3-psycopg2      # install psycopg2 on RedHat/Fedora/CentOS

2. Specify one of psycopg, psycopg2, or psycopg2-binary in the list of dependencies when installing Patroni with pip.

General installation for pip

Patroni can be installed with pip:

pip install patroni[dependencies]

Leave dependencies empty, or specify one or more of these dependencies:

etcd or etcd3
python-etcd module to use Etcd as the Distributed Configuration Store (DCS)

consul
py-consul module to use Consul as the DCS

zookeeper
kazoo module to use Zookeeper as the DCS

exhibitor
kazoo module to use Exhibitor as the DCS (same dependencies as Zookeeper)

kubernetes
kubernetes module to use Kubernetes as the DCS

raft
pysyncobj module to use the Python Raft implementation as the DCS

aws
boto3 to use AWS callbacks

jsonlogger
python-json-logger module for JSON logging

systemd
systemd-python for sd_notify integration

all
all of the above (except psycopg family)

psycopg3
psycopg\[binary\]\>=3.0.0 module

psycopg2
psycopg2\>=2.5.4 module

psycopg2-binary
psycopg2-binary module

For example, install Patroni with psycopg3, Etcd DCS dependencies, and AWS callbacks:

pip install patroni[psycopg3,etcd3,aws]

Install external tools, such as WAL-E, separately if your replica creation or custom bootstrap scripts call them.

Package installation on Linux

The Postgres community may provide Patroni packages for these operating systems:

• RHEL, RockyLinux, AlmaLinux;
• Debian and Ubuntu;
• SUSE Enterprise Linux.

Packages may also provide direct dependencies, such as Python modules absent from the operating system repositories.

For more information see the PGDG repository documentation.

On RedHat Enterprise Linux derivatives, you may also need EPEL packages. See the EPEL repository documentation.

After you configure the PGDG repository for your operating system, install Patroni.

Installing on Debian derivatives

After you configure the PGDG repository, install Patroni with apt:

apt-get install patroni

Installing on RedHat derivatives

On RHEL 9 and derivatives, first configure the PGDG repository. Then install Patroni with an etcd DCS:

dnf install patroni patroni-etcd

If your RedHat derivative does not provide etcd packages, install them from PGDG. On each DCS node, run:

dnf install 'dnf-command(config-manager)'
dnf config-manager --enable pgdg-rhel9-extras
dnf install etcd

For RHEL 8, replace the version with 8 to use pgdg-rhel8-extras. RockyLinux, AlmaLinux, Oracle Linux, and other derivatives also use the pgdg-rhelN-extras repository name.

Installing on SUSE Enterprise Linux

Some dependencies may require SUSE PackageHub repositories. See the SUSE PackageHub documentation.

On SLES 15, configure the PGDG repository. Then install Patroni:

zypper install patroni patroni-etcd

With the SUSE PackageHub repo enabled you can also install etcd:

SUSEConnect -p PackageHub/15.5/x86_64
zypper install etcd

Upgrading

To upgrade Patroni, update the software and restart the Patroni daemon on each cluster node.

A Patroni daemon restart also restarts the Postgres database. This can cause primary failover. We recommend maintenance mode until all Patroni daemon restarts finish.

On one Patroni node, put the cluster in maintenance mode:

patronictl pause --wait

Then on each node in the cluster, perform the package upgrade required for your OS:

apt-get update && apt-get install patroni patroni-etcd

Restart the patroni daemon process on each node:

systemctl restart patroni

Resume Patroni monitoring and leave maintenance mode:

patronictl resume --wait

The cluster now operates fully with the new Patroni version.

3 - Patroni configuration

Source: https://patroni.readthedocs.io/en/latest/patroni_configuration.html

There are 3 types of Patroni configuration:

• Global dynamic configuration.
  These options are stored in the DCS (Distributed Configuration Store) and apply to all nodes. Change them at any time with patronictl_edit_config or the REST API. Changes outside startup configuration apply asynchronously on each node’s next wake-up cycle, followed by a reload. Changed PostgreSQL parameters with postmaster context require a restart. Patroni sets pending_restart in members.data JSON for these nodes. Node status also shows "restart_pending": true.

• Local configuration file (patroni.yml).
  Local options override dynamic configuration. To reload patroni.yml without restarting Patroni, send SIGHUP, send POST /reload, or run patronictl_reload. Local configuration can be one YAML file or a directory. Patroni loads all YAML files in a directory in sorted order. If several files define a key, the last file takes precedence.

• Environment configuration.
  Environment variables can set or override some local parameters. Use them in dynamic environments where some values are unknown in advance. For example, the external IP address may be unknown inside docker.

Important rules

PostgreSQL parameters controlled by Patroni

These PostgreSQL parameters must have the same values on the primary and replicas. Local configuration and environment values have no effect. Set them through shared DCS configuration. The list shows their defaults and minimum values:

• max_connections: default value 100, minimal value 25
• max_locks_per_transaction: default value 64, minimal value 32
• max_worker_processes: default value 8, minimal value 2
• max_prepared_transactions: default value 0, minimal value 0
• wal_level: default value hot_standby, accepted values: hot_standby, replica, logical
• track_commit_timestamp: default value off

PostgreSQL permits different primary and replica values for the parameters below. However, a replica can become primary at any time. Different values are therefore unsuitable, and Patroni permits changes only through dynamic configuration.

• max_wal_senders: default value 10, minimal value 3
• max_replication_slots: default value 10, minimal value 4
• wal_keep_segments: default value 8, minimal value 1
• wal_keep_size: default value 128MB, minimal value 16MB
• wal_log_hints: on

Patroni validates these values for correctness or a required minimum.

There are some other Postgres parameters controlled by Patroni:

• listen_addresses - is set either from postgresql.listen or from PATRONI_POSTGRESQL_LISTEN environment variable
• port - is set either from postgresql.listen or from PATRONI_POSTGRESQL_LISTEN environment variable
• cluster_name - is set either from scope or from PATRONI_SCOPE environment variable
• hot_standby: on

Patroni writes the parameters above to postgresql.conf. Except for wal_keep_segments and wal_keep_size, it also passes them as postgres arguments. These arguments take precedence over all other settings, including ALTER SYSTEM.

Parameters such as postgresql.listen and postgresql.data_dir can only be set locally. Use the configuration file or environment variables. Local configuration overrides dynamic configuration in most cases.

When applying the local or dynamic configuration options, the following actions are taken:

• The node checks for postgresql.base.conf and custom_conf.
• If custom_conf is set, its file supplies the base configuration. Patroni ignores postgresql.base.conf and postgresql.conf as base files.
• Otherwise, an existing postgresql.base.conf supplies the base configuration. It contains the renamed original configuration.
• If neither exists, Patroni renames the original postgresql.conf to postgresql.base.conf and uses it as the base.
• Patroni writes dynamic options to postgresql.conf, subject to the exceptions above. It adds an include for postgresql.base.conf or the custom_conf file. New options can then apply without rereading the file to check for the include.
• Command-line arguments override parameters essential for Patroni cluster management.
• If a changed value requires restart according to pg_settings context, Patroni sets pending_restart on the node. Any restart resets this flag.

The parameters would be applied in the following order (run-time are given the highest priority):

1. load parameters from file postgresql.base.conf (or from a custom_conf file, if set)
2. load parameters from file postgresql.conf
3. load parameters from file postgresql.auto.conf
4. run-time parameter using -o --name=value

This order supports shared node configuration (2) and node-specific ALTER SYSTEM settings (3). It enforces essential Patroni parameters (4). It also permits external tools to manage the base configuration directly (1).

PostgreSQL parameters that touch shared memory

PostgreSQL has some parameters that determine the size of the shared memory used by them:

• max_connections
• max_prepared_transactions
• max_locks_per_transaction
• max_wal_senders
• max_worker_processes

Changes require a PostgreSQL restart. Standby shared memory structures must not be smaller than the corresponding primary structures.

Change these values through dynamic configuration, as Patroni requires. The usual sequence is:

1. Applying changes through patronictl_edit_config (or via REST API /config endpoint)
2. Restarting nodes through patronictl_restart (or via REST API /restart endpoint)

Note: Restart PostgreSQL through patronictl_restart or the REST API /restart endpoint. Restarting the Patroni daemon with systemctl restart patroni can trigger failover if the node is primary.

These shared memory settings require a specific restart order:

• If you want to increase the value of any of those settings:

  1. Restart all standbys first
  2. Restart the primary after that

• If you want to decrease the value of any of those settings:

  1. Restart the primary first
  2. Restart all standbys after that

Note: If you restart all nodes together after a decrease, Patroni restarts standbys with the original value. You must restart them again later. This prevents an endless crash loop: PostgreSQL exits with FATAL if a value is below that recorded in standby pg_controldata. A standby value can decrease only after its pg_controldata reflects the primary change.

More information about that can be found at PostgreSQL Administrator’s Overview.

Patroni configuration parameters

Also the following Patroni configuration options can be changed only dynamically:

• ttl: 30
• loop_wait: 10
• retry_timeout: 10
• maximum_lag_on_failover: 1048576
• max_timelines_history: 0
• check_timeline: false
• postgresql.use_slots: true

After a change, Patroni reads the relevant DCS configuration section and updates its runtime values.

After each configuration change, Patroni writes DCS options to patroni.dynamic.json in the Postgres data directory. Only the leader can restore these options from disk. It can do so only when DCS options are absent or invalid.

Configuration generation and validation

The patroni executable can generate and validate local configuration. Use it to:

• Create a sample local Patroni configuration;
• Create a Patroni configuration file for the locally running PostgreSQL instance (e.g. as a preparation step for the Patroni integration);
• Validate a given Patroni configuration file.

Sample Patroni configuration

patroni --generate-sample-config [configfile]

Description

Generate a sample Patroni configuration in yaml format. Values come from Environment configuration. For unset parameters, use Patroni defaults or #FIXME for values that the user must supply.

Some default values are defined based on the local setup:

• postgresql.listen: the IP address returned by gethostname call for the current machine’s hostname and the standard 5432 port.
• postgresql.connect_address: the IP address returned by gethostname call for the current machine’s hostname and the standard 5432 port.
• postgresql.authentication.rewind: is only defined if the PostgreSQL version can be defined from the binary and the version is 11 or later.
• restapi.listen: IP address returned by gethostname call for the current machine’s hostname and the standard 8008 port.
• restapi.connect_address: IP address returned by gethostname call for the current machine’s hostname and the standard 8008 port.

Parameters

configfile - full path to the configuration file used to store the result. If not provided, the result is sent to stdout.

Patroni configuration for a running instance

patroni --generate-config [--dsn DSN] [configfile]

Description

Generate a yaml configuration for the local running PostgreSQL instance. Connection settings use the supplied DSN first, then PostgreSQL environment variables. If no password is supplied, enter it at the prompt.

All non-internal GUCs from the source instance supply these Patroni parameters. This includes GUCs set through files, postmaster command-line arguments, or environment variables:

• scope: cluster_name GUC value;
• postgresql.listen: listen_addresses and port GUC values;
• postgresql.datadir: data_directory GUC value;
• postgresql.parameters: archive_command, restore_command, archive_cleanup_command, recovery_end_command, ssl_passphrase_command, hba_file, ident_file, config_file GUC values;
• bootstrap.dcs: all other gathered PostgreSQL GUCs.

If Postgres GUCs do not supply scope, postgresql.listen, or postgresql.datadir, Patroni uses the corresponding Environment configuration value.

Other rules applied for the values definition:

• name: PATRONI_NAME environment variable value if set, otherwise the current machine’s hostname.
• postgresql.bin_dir: path to the Postgres binaries gathered from the running instance.
• postgresql.connect_address: the IP address returned by gethostname call for the current machine’s hostname and the port used for the instance connection or the port GUC value.
• postgresql.authentication.superuser: the configuration used for the instance connection;
• postgresql.pg_hba: the lines gathered from the source instance’s hba_file.
• postgresql.pg_ident: the lines gathered from the source instance’s ident_file.
• restapi.listen: IP address returned by gethostname call for the current machine’s hostname and the standard 8008 port.
• restapi.connect_address: IP address returned by gethostname call for the current machine’s hostname and the standard 8008 port.

The generated file also includes other parameters from Environment configuration.

Parameters

configfile
Full path to the configuration file used to store the result. If not provided, result is sent to stdout.

dsn
Optional DSN string for the local PostgreSQL instance to get GUC values from.

Validate Patroni configuration

patroni --validate-config [configfile] [--ignore-listen-port | -i]

Description

Validate the Patroni configuration and print failed checks.

Parameters

configfile
Full path to the file to check. If absent or nonexistent, read PATRONI_CONFIG_VARIABLE. If that variable is unset, use Patroni environment variables.

--ignore-listen-port | -i
Optional flag to ignore bind failures for occupied listen ports during configfile validation.

--print | -p
Optional flag to print local configuration, including environment overrides, after successful validation.

3.1 - Dynamic Configuration Settings

Source: https://patroni.readthedocs.io/en/latest/dynamic_configuration.html

Dynamic configuration is stored in the DCS (Distributed Configuration Store) and applied on all cluster nodes.

Change dynamic configuration with patronictl_edit_config or the Patroni REST API.

• loop_wait: the number of seconds the loop will sleep. Default value: 10, minimum possible value: 1
• ttl: leader lock TTL, in seconds. This is the interval before automatic failover can start. Default: 30. Minimum: 20.
• retry_timeout: timeout for DCS and PostgreSQL operation retries (in seconds). DCS or network issues shorter than this will not cause Patroni to demote the leader. Default value: 10, minimum possible value: 3

loop_wait + 2 * retry_timeout <= ttl

• maximum_lag_on_failover: the maximum bytes a follower may lag to be able to participate in leader election.
• primary_race_backoff: delays standby leader races by primary_race_backoff seconds while WAL replication from the primary advances. This reduces unnecessary failovers when Patroni is briefly unresponsive. Default: 0 (disabled).
• maximum_lag_on_syncnode: maximum synchronous replica lag in bytes before replacement by a healthy asynchronous replica. With multiple replicas, Patroni uses the maximum replica LSN. Otherwise, it uses the leader’s current WAL LSN. Default: -1. Values of 0 or less disable replacement. Use a sufficiently high value to avoid frequent replacements during high transaction volume.
• max_timelines_history: maximum number of timeline history items kept in DCS. Default value: 0. When set to 0, it keeps the full history in DCS.
• primary_start_timeout: seconds allowed for primary recovery before failover. Default: 300. A value of 0 permits immediate failover after crash detection. Asynchronous failover can lose transactions. Worst-case failover time is loop_wait + primary_start_timeout + loop_wait, or only loop_wait when primary_start_timeout is zero. Select the value for your durability and availability requirements.
• primary_stop_timeout: seconds Patroni can wait for Postgres shutdown when synchronous_mode is enabled. With synchronous_mode enabled and a value > 0, Patroni sends SIGKILL to postmaster if shutdown exceeds this interval. Select the value for your durability and availability requirements. Unset values or values <= 0 disable this timeout.
• synchronous_mode: synchronous replication mode: off, on, or quorum. The leader manages synchronous_standby_names. Only the last known leader or synchronous replicas can enter a leader race. This protects successful commits during failover but can block writes when Patroni cannot guarantee durability. See replication modes documentation.
• synchronous_mode_strict: keeps synchronous replication enabled when no synchronous replica is available, blocking client writes. Without an eligible streaming replica, Patroni uses the last synchronous nodes from /sync for synchronous_standby_names. If no prior state exists, it uses __patroni_strict_sync_replica_placeholder__. Do not use that reserved value for the node name in patroni.yaml. See replication modes documentation.
• synchronous_node_count: required synchronous standby count when synchronous_mode is enabled. Patroni updates the DCS and PostgreSQL synchronous_standby_names as members join or leave. It reduces the count if it exceeds the number of eligible nodes. Default: 1.
• failsafe_mode: Enables DCS Failsafe Mode. Defaults to false.
• postgresql:
  • use_pg_rewind: enables pg_rewind. Default: false. For correct operation, pg_rewind requires data page checksums and/or wal_log_hints set to on. Enable checksums with the --data-checksums option for initdb.
  • use_slots: whether or not to use replication slots. Defaults to true on PostgreSQL 9.4+.
  • recovery_conf: extra recovery.conf settings for replicas. PostgreSQL 12 no longer uses recovery.conf. You can still use this section because Patroni handles the version difference.
  • parameters: configuration parameters (GUCs) for Postgres in format {max_connections: 100, wal_level: "replica", max_wal_senders: 10, wal_log_hints: "on"}. Many of these are required for replication to work.
  • parameters_primary: (optional) role-specific parameter overrides for primary. These values are merged with and override the base parameters.
  • parameters_replica: (optional) role-specific parameter overrides for replica. These values are merged with and override the base parameters.
  • parameters_standby_leader: (optional) role-specific parameter overrides for standby_leader. These values are merged with and override the base parameters.
  • pg_hba: list of lines that Patroni will use to generate pg_hba.conf. Patroni ignores this parameter if hba_file PostgreSQL parameter is set to a non-default value.
    • - host all all 0.0.0.0/0 md5
    • - host replication replicator 127.0.0.1/32 md5: A line like this is required for replication.
  • pg_hba_primary: (optional) role-specific pg_hba entries for primary. These completely replace pg_hba (no merging). If not defined, pg_hba is used.
  • pg_hba_replica: (optional) role-specific pg_hba entries for replica. These completely replace pg_hba (no merging). If not defined, pg_hba is used.
  • pg_hba_standby_leader: (optional) role-specific pg_hba entries for standby_leader. These completely replace pg_hba (no merging). If not defined, pg_hba is used.
  • pg_ident: list of lines that Patroni will use to generate pg_ident.conf. Patroni ignores this parameter if ident_file PostgreSQL parameter is set to a non-default value.
    • - mapname1 systemname1 pguser1
    • - mapname1 systemname2 pguser2
  • pg_ident_primary: (optional) role-specific pg_ident entries for primary. These completely replace pg_ident (no merging). If not defined, pg_ident is used.
  • pg_ident_replica: (optional) role-specific pg_ident entries for replica. These completely replace pg_ident (no merging). If not defined, pg_ident is used.
  • pg_ident_standby_leader: (optional) role-specific pg_ident entries for standby_leader. These completely replace pg_ident (no merging). If not defined, pg_ident is used.
• standby_cluster: if this section is defined, we want to bootstrap a standby cluster.
  • host: an address of remote node
  • port: a port of remote node
  • primary_slot_name: replication slot on the remote node. Optional. The default comes from the instance name; see slot_name_from_member_name.
  • create_replica_methods: ordered methods to bootstrap the standby leader from the remote primary. This list can differ from postgresql_settings.
  • restore_command: command to restore remote primary WAL records to standby cluster nodes. It can differ from postgresql_settings.
  • archive_cleanup_command: cleanup command for standby leader
  • recovery_min_apply_delay: how long to wait before actually apply WAL records on a standby leader
• member_slots_ttl: physical slot retention after replicas shut down. Default: 30min. A value of 0 restores immediate removal when the member key expires from the DCS. Requires PostgreSQL 11 or later.
• slots: permanent replication slots, retained through switchover and failover. Patroni creates missing slots. On PostgreSQL 11 and later, it creates physical slots on all nodes and advances them every loop_wait seconds. Earlier versions maintain permanent physical slots only on the primary. Logical slots are copied from the primary to a standby with restart. Their positions then advance every loop_wait seconds if necessary. Patroni copies logical slot files through libpq with rewind or superuser credentials; see postgresql.authentication. A replica slot can lag behind the old primary. Applications must handle repeated messages after failover, for example by tracking confirmed_flush_lsn. Permanent slots require postgresql.use_slots set to true. Permanent logical slots automatically enable hot_standby_feedback. Logical slot failover is unsafe on PostgreSQL 9.6 and older, and PostgreSQL 10 lacks required functions. This feature requires PostgreSQL 11+.
  • my_slot_name: permanent slot name. Patroni does not create a slot on the node with the same name. A permanent physical slot named after a member remains even when that member becomes unresponsive. Normally, Patroni would remove it. This preserves member slots during temporary failures or migration to Patroni; see Convert a Standalone to a Patroni Cluster. Remove these matching names from DCS configuration when the slots are no longer required. Otherwise, they affect normal Patroni operation.
    • type: physical or logical. Logical slots also require database and plugin. Physical slots optionally accept cluster_type.
    • database: the database name where logical slots should be created.
    • plugin: the plugin name for the logical slot.
    • cluster_type: create the slot only on this cluster type: primary or standby. On other types, Patroni skips creation or drops an existing slot.
• ignore_slots: sets of slot properties that Patroni must ignore. Use this for slots managed outside Patroni. Any matching subset of properties causes a slot to be ignored.
  • name: the name of the replication slot.
  • type: slot type. Can be physical or logical. If the slot is logical, you may additionally define database and/or plugin.
  • database: the database name (when matching a logical slot).
  • plugin: the logical decoding plugin (when matching a logical slot).

Note: slots is a hashmap while ignore_slots is an array. For example:

slots:
  permanent_logical_slot_name:
    type: logical
    database: my_db
    plugin: test_decoding
  permanent_physical_slot_name:
    type: physical
  ...
ignore_slots:
  - name: ignored_logical_slot_name
    type: logical
    database: my_db
    plugin: test_decoding
  - name: ignored_physical_slot_name
    type: physical
  ...

Note: On PostgreSQL v11 or later, Patroni maintains physical slots on all nodes eligible to become leader. Replicas therefore retain WAL segments that other nodes may need. After an absent node’s DCS member key expires, Patroni drops its slot after member_slots_ttl, which defaults to 30min. Adjust retention for your requirements.

For a fixed topology with stable node names, you can configure permanent physical slots named after the nodes. This prevents slot removal and WAL recycling while a replica is temporarily down:

slots:
  node_name1:
    type: physical
  node_name2:
    type: physical
  node_name3:
    type: physical
  ...

3.2 - YAML Configuration Settings

Source: https://patroni.readthedocs.io/en/latest/yaml_configuration.html

Global/Universal

• thread_pool_size: thread pool size for asynchronous tasks and REST API communication with other members during leader races or failsafe checks. Minimum: 5. Default: 5.
• thread_stack_size: stack size for Patroni threads. Use a multiple of 64kB. Minimum: 64kB. Patroni default: 512kB.
• name: the name of the host. Must be unique for the cluster. The value __patroni_strict_sync_replica_placeholder__ is reserved for internal use by Patroni and cannot be used as a node name.
• namespace: path within the configuration store where Patroni will keep information about the cluster. Default value: “/service”
• scope: cluster name

Log

• type: sets the format of logs. Can be either plain or json. To use json format, you must have the jsonlogger installed. The default value is plain.
• level: sets the general logging level. Default value is INFO (see the docs for Python logging)
• traceback_level: sets the level where tracebacks will be visible. Default value is ERROR. Set it to DEBUG if you want to see tracebacks only if you enable log.level=DEBUG.
• format: log format. For plain logs, use a string. For json, use a string or a list of LogRecord attributes. In lists, use field names without %( and ). To rename a field, use a dictionary with the original field as key and the output name as value. Default: %(asctime)s %(levelname)s: %(message)s.
• dateformat: sets the datetime formatting string. (see the formatTime() documentation)
• static_fields: add additional fields to the log. This option is only available when the log type is set to json.
• max_queue_size: maximum internal log queue size. Patroni writes records to an in-memory queue. A separate thread reads them and writes to stderr or a file. The default 1000 records retain approximately 1h20m of logs.
• dir: application log directory. It must exist and be writable by the Patroni user. By default, Patroni retains four 25MB files. Adjust retention with file_num and file_size.
• mode: Permissions for log files (for example, 0644). If not specified, permissions will be set based on the current umask value.
• file_num: The number of application logs to retain.
• file_size: Size of patroni.log file (in bytes) that triggers a log rolling.
• loggers: This section allows redefining logging level per python module
  • patroni.postmaster: WARNING
  • urllib3: DEBUG
• deduplicate_heartbeat_logs: suppresses consecutive identical heartbeat messages when true. Default: false.

For JSON logging, use this example:

log:
   type: json
   format:
      - message
      - module
      - asctime: '@timestamp'
      - levelname: level
   static_fields:
      app: patroni

Bootstrap configuration

• bootstrap:

  • dcs: global dynamic cluster configuration. Put parameters from Dynamic Configuration settings under bootstrap.dcs. After bootstrap, Patroni writes this section to /<namespace>/<scope>/config in the configuration store.

  • method: custom script to use for bootstrapping this cluster.

    See custom bootstrap methods. Set method to initdb, or omit it, to use the default initdb command.

  • initdb: (optional) list options to be passed on to initdb.

    • - data-checksums: Must be enabled when pg_rewind is needed on 9.3.
    • - encoding: UTF8: default encoding for new databases.
    • - locale: UTF8: default locale for new databases.

  • post_bootstrap or post_init: script to run after cluster initialization. It receives a connection URL with the cluster superuser name. PGPASSFILE points to the pgpass file.

Citus

Enable Citus integration. Patroni then registers workers on the coordinator. See Citus support.

• group: the Citus group id, integer. Use 0 for coordinator and 1, 2, etc… for workers
• database: database for the citus extension. Use the same database on the coordinator and all workers. Currently, only one database is supported.

Consul

Most of the parameters are optional, but you have to specify one of the host or url

• host: the host:port for the Consul local agent.
• url: url for the Consul local agent, in format: http(s)://host:port.
• port: (optional) Consul port.
• scheme: (optional) http or https, defaults to http.
• token: (optional) ACL token.
• verify: (optional) whether to verify the SSL certificate for HTTPS requests.
• cacert: (optional) The ca certificate. If present it will enable validation.
• cert: (optional) file with the client certificate.
• key: (optional) file with the client key. Can be empty if the key is part of cert.
• dc: (optional) Datacenter to communicate with. By default the datacenter of the host is used.
• consistency: (optional) Select consul consistency mode. Possible values are default, consistent, or stale (more details in consul API reference)
• checks: (optional) list of Consul health checks used for the session. By default an empty list is used.
• register_service: (optional) registers a service named by scope. Its role tag is master, primary, replica, or standby-leader. Default: false.
• service_tags: (optional) additional static tags to add to the Consul service apart from the role (primary/replica/standby-leader). By default an empty list is used.
• service_check_interval: (optional) how often to perform health check against registered url. Defaults to ‘5s’.
• service_check_tls_server_name: (optional) override SNI host when connecting via TLS, see also consul agent check API reference.

The token needs to have the following ACL permissions:

service_prefix "${scope}" {
    policy = "write"
}
key_prefix "${namespace}/${scope}" {
    policy = "write"
}
session_prefix "" {
    policy = "write"
}

Etcd

Most of the parameters are optional, but you have to specify one of the host, hosts, url, proxy or srv

• host: the host:port for the etcd endpoint.
• hosts: list of etcd endpoint in format host1:port1,host2:port2,etc… Could be a comma separated string or an actual yaml list.
• use_proxies: if true, treats hosts as proxies. Patroni does not discover etcd topology.
• url: url for the etcd.
• proxy: proxy url for the etcd. If you are connecting to the etcd using proxy, use this parameter instead of url.
• srv: domain for SRV-based cluster discovery. Patroni queries these services in order until one succeeds: _etcd-client-ssl, _etcd-client, _etcd-ssl, _etcd, _etcd-server-ssl, _etcd-server. For _etcd-server-ssl or _etcd-server, it queries available members through the ETCD peer protocol. Otherwise, it uses hosts from the SRV records.
• srv_suffix: suffix for discovery SRV names. Requires srv. Use it to distinguish etcd clusters in one domain. For example, srv_suffix: foo and srv: example.org produce _etcd-client-ssl-foo._tcp.example.com. Patroni applies this suffix to each possible ETCD SRV service name.
• protocol: (optional) http or https, if not specified http is used. If the url or proxy is specified - will take protocol from them.
• username: (optional) username for etcd authentication.
• password: (optional) password for etcd authentication.
• cacert: (optional) The ca certificate. If present it will enable validation.
• cert: (optional) file with the client certificate.
• key: (optional) file with the client key. Can be empty if the key is part of cert.

Etcdv3

To use Etcd protocol version 3, configure the etcd3 section. Its parameters are the same as etcd.

ZooKeeper

• hosts: List of ZooKeeper cluster members in format: $$'host1:port1', 'host2:port2', 'etc...'$$.
• use_ssl: (optional) Whether SSL is used or not. Defaults to false. If set to false, all SSL specific parameters are ignored.
• cacert: (optional) The CA certificate. If present it will enable validation.
• cert: (optional) File with the client certificate.
• key: (optional) File with the client key.
• key_password: (optional) The client key password.
• verify: (optional) Whether to verify certificate or not. Defaults to true.
• set_acls: (optional) default ACL that Kazoo applies to each new ZNode. Use x509 (default) or another supported ZooKeeper scheme, such as digest. Supply a dictionary: each key is the full principal, optionally prefixed with its scheme. Each value lists one or more permissions: CREATE, READ, WRITE, DELETE, ADMIN, or ALL. Example: set_acls: {CN=principal1: [CREATE, READ], digest:principal2:+pjROuBuuwNNSujKyH8dGcEnFPQ=: [ALL]}.
• auth_data: (optional) connection authentication dictionary. Use scheme as the key and credential as the value. Default: empty dictionary.

Exhibitor

• hosts: initial list of Exhibitor (ZooKeeper) nodes in format: ‘host1,host2,etc…’. This list updates automatically whenever the Exhibitor (ZooKeeper) cluster topology changes.
• poll_interval: how often the list of ZooKeeper and Exhibitor nodes should be updated from Exhibitor.
• port: Exhibitor port.

Kubernetes

• bypass_api_service: (optional) normally, Patroni contacts the kubernetes service at KUBERNETES_SERVICE_HOST. With bypass_api_service set to true, it discovers the API nodes behind the service and contacts them directly.
• namespace: (optional) Kubernetes namespace where Patroni pod is running. Default value is default.
• labels: labels in {label1: value1, label2: value2} format. Patroni uses them to find cluster Pods and Endpoints or ConfigMaps. It also adds them to each Endpoint or ConfigMap it creates.
• scope_label: (optional) name of the label containing cluster name. Default value is cluster-name.
• bootstrap_labels: (optional) labels in {label1: value1, label2: value2} format. Patroni assigns them to pods in these states: initializing new cluster, running custom bootstrap script, starting after custom bootstrap, or creating replica.
• role_label: (optional) name of the label containing role (primary, replica, or other custom value). Patroni will set this label on the pod it runs in. Default value is role.
• leader_label_value: (optional) value of the pod label when Postgres role is primary. Default value is primary.
• follower_label_value: (optional) value of the pod label when Postgres role is replica. Default value is replica.
• standby_leader_label_value: (optional) value of the pod label when Postgres role is standby_leader. Default value is primary.
• tmp_role_label: (optional) name of the temporary label containing role (primary or replica). Value of this label will always use the default of corresponding role. Set only when necessary.
• use_endpoints: (optional) if set to true, Patroni will use Endpoints instead of ConfigMaps to run leader elections and keep cluster state.
• pod_ip: (optional) Patroni pod IP address. Required when use_endpoints is enabled. On promotion, Patroni uses this value to populate leader endpoint subsets.
• ports: (optional) named Service ports require identical Endpoint port names. Otherwise, the service cannot work. For {Kind: Service, spec: {ports: [{name: postgresql, port: 5432, targetPort: 5432}]}}, set kubernetes.ports: [{"name": "postgresql", "port": 5432}]. Patroni uses this value to update leader Endpoint subsets. Applies only with kubernetes.use_endpoints.
• cacert: (optional) CA_BUNDLE file with trusted CA certificates for Kubernetes API SSL verification. Defaults to the ServiceAccount secret value.
• retriable_http_codes: (optional) Kubernetes API status codes to retry. By default, Patroni retries 500, 503, and 504, and responses with a retry-after HTTP header.

Raft (deprecated)

• self_addr: ip:port to listen on for Raft connections. The self_addr must be accessible from other nodes of the cluster. If not set, the node will not participate in consensus.

• bind_addr: (optional) ip:port to listen on for Raft connections. If not specified the self_addr will be used.

• partner_addrs: list of other Patroni nodes in the cluster in format:

  $$'ip1:port', 'ip2:port', 'etc...'$$

• data_dir: directory where to store Raft log and snapshot. If not specified the current working directory is used.

• password: (optional) Encrypt Raft traffic with a specified password, requires cryptography python module.

• min_timeout: (optional) minimum election timeout in seconds for the underlying pysyncobj Raft implementation. Must be greater than 3 * append_entries_period. Default: 0.4.

• max_timeout: (optional) maximum election timeout in seconds for the underlying pysyncobj Raft implementation. Must be greater than min_timeout. Default: 1.4.

• connection_timeout: (optional) time in seconds after which a connection with no data received is considered dead. Must be greater than or equal to max_timeout. Default: 3.5.

• append_entries_period: (optional) interval in seconds for sending heartbeat (append_entries) commands. Must be less than one-third of min_timeout. Default: 0.1.

• connection_retry_time: (optional) interval in seconds between reconnection attempts to offline nodes. Default: 5.0.

• leader_fallback_timeout: (optional) time in seconds after which a leader with no response from the majority falls back to follower state. Must be greater than append_entries_period. Default: 30.0.

Short FAQ about Raft implementation

• Q: How to list all the nodes providing consensus?

  A: syncobj_admin -conn host:port -status where the host:port is the address of one of the cluster nodes

• Q: Node that was a part of consensus and has gone and I can’t reuse the same IP for other node. How to remove this node from the consensus?

  A: syncobj_admin -conn host:port -remove host2:port2 where the host2:port2 is the address of the node you want to remove from consensus.

• Q: Where to get the syncobj_admin utility?

  A: It is installed together with pysyncobj module (python RAFT implementation), which is Patroni dependency.

• Q: it is possible to run Patroni node without adding in to the consensus?

  A: Yes, just comment out or remove raft.self_addr from Patroni configuration.

• Q: It is possible to run Patroni and PostgreSQL only on two nodes?

  A: Yes, on the third node you can run patroni_raft_controller (without Patroni and PostgreSQL). In such a setup, one can temporarily lose one node without affecting the primary.

PostgreSQL

• postgresql:

  • authentication:

    • superuser:
      • username: name for the superuser, set during initialization (initdb) and later used by Patroni to connect to the postgres.
      • password: password for the superuser, set during initialization (initdb).
      • sslmode: (optional) maps to sslmode. Specifies the TLS negotiation mode. See the PostgreSQL documentation for each mode. Default: prefer.
      • sslkey: (optional) maps to sslkey. Path to the secret key for the client certificate.
      • sslpassword: (optional) maps to sslpassword. Password for the secret key in sslkey.
      • sslcert: (optional) maps to sslcert. Path to the client certificate.
      • sslrootcert: (optional) maps to sslrootcert. File containing one or more trusted CA certificates. The client uses them to verify the server certificate.
      • sslcrl: (optional) maps to sslcrl. Certificate revocation list file. The client rejects servers whose certificates appear in the list.
      • sslcrldir: (optional) maps to sslcrldir. Directory containing certificate revocation list files. The client rejects servers whose certificates appear in these lists.
      • sslnegotiation: (optional) maps to sslnegotiation. Controls server SSL negotiation when SSL is used.
      • gssencmode: (optional) maps to gssencmode. Controls whether to negotiate secure GSS TCP/IP and its priority.
      • channel_binding: (optional) maps to channel_binding. Controls client channel binding.
    • replication:
      • username: replication username; the user will be created during initialization. Replicas will use this user to access the replication source via streaming replication
      • password: replication password; the user will be created during initialization.
      • sslmode: (optional) maps to sslmode. Specifies the TLS negotiation mode. See the PostgreSQL documentation for each mode. Default: prefer.
      • sslkey: (optional) maps to sslkey. Path to the secret key for the client certificate.
      • sslpassword: (optional) maps to sslpassword. Password for the secret key in sslkey.
      • sslcert: (optional) maps to sslcert. Path to the client certificate.
      • sslrootcert: (optional) maps to sslrootcert. File containing one or more trusted CA certificates. The client uses them to verify the server certificate.
      • sslcrl: (optional) maps to sslcrl. Certificate revocation list file. The client rejects servers whose certificates appear in the list.
      • sslcrldir: (optional) maps to sslcrldir. Directory containing certificate revocation list files. The client rejects servers whose certificates appear in these lists.
      • sslnegotiation: (optional) maps to sslnegotiation. Controls server SSL negotiation when SSL is used.
      • gssencmode: (optional) maps to gssencmode. Controls whether to negotiate secure GSS TCP/IP and its priority.
      • channel_binding: (optional) maps to channel_binding. Controls client channel binding.
    • rewind:
      • username: (optional) name for the user for pg_rewind; the user will be created during initialization of postgres 11+ and all necessary permissions will be granted.
      • password: (optional) password for the user for pg_rewind; the user will be created during initialization.
      • sslmode: (optional) maps to sslmode. Specifies the TLS negotiation mode. See the PostgreSQL documentation for each mode. Default: prefer.
      • sslkey: (optional) maps to sslkey. Path to the secret key for the client certificate.
      • sslpassword: (optional) maps to sslpassword. Password for the secret key in sslkey.
      • sslcert: (optional) maps to sslcert. Path to the client certificate.
      • sslrootcert: (optional) maps to sslrootcert. File containing one or more trusted CA certificates. The client uses them to verify the server certificate.
      • sslcrl: (optional) maps to sslcrl. Certificate revocation list file. The client rejects servers whose certificates appear in the list.
      • sslcrldir: (optional) maps to sslcrldir. Directory containing certificate revocation list files. The client rejects servers whose certificates appear in these lists.
      • sslnegotiation: (optional) maps to sslnegotiation. Controls server SSL negotiation when SSL is used.
      • gssencmode: (optional) maps to gssencmode. Controls whether to negotiate secure GSS TCP/IP and its priority.
      • channel_binding: (optional) maps to channel_binding. Controls client channel binding.

  • callbacks: callback scripts to run on certain actions. Patroni will pass the action, role and cluster name. (See scripts/aws.py as an example of how to write them.)

    • on_reload: run this script when configuration reload is triggered.
    • on_restart: run this script when the postgres restarts (without changing role).
    • on_role_change: run this script when the postgres is being promoted or demoted.
    • on_start: run this script when the postgres starts.
    • on_stop: run this script when the postgres stops.

  • connect_address: IP address + port through which Postgres is accessible from other nodes and applications.

  • proxy_address: address and port of the connection pool beside Postgres, such as pgbouncer. Patroni stores it as proxy_url in the DCS member key for service discovery.

  • create_replica_methods: ordered methods for creating replicas. The default is “basebackup”. Other methods refer to scripts, each with its own configuration section. See custom replica creation methods.

  • data_dir: The location of the Postgres data directory, either existing or to be initialized by Patroni.

  • config_dir: The location of the Postgres configuration directory, defaults to the data directory. Must be writable by Patroni.

  • bin_dir: (optional) Path to PostgreSQL binaries (pg_ctl, initdb, pg_controldata, pg_basebackup, postgres, pg_isready, pg_rewind). If not provided or is an empty string, PATH environment variable will be used to find the executables.

  • bin_name: (optional) Make it possible to override Postgres binary names, if you are using a custom Postgres distribution:

    • pg_ctl: (optional) Custom name for pg_ctl binary.
    • initdb: (optional) Custom name for initdb binary.
    • pgcontroldata: (optional) Custom name for pg_controldata binary.
    • pg_basebackup: (optional) Custom name for pg_basebackup binary.
    • postgres: (optional) Custom name for postgres binary.
    • pg_isready: (optional) Custom name for pg_isready binary.
    • pg_rewind: (optional) Custom name for pg_rewind binary.

  • listen: Postgres listen addresses and port. With streaming replication, other cluster nodes must be able to reach this address. Separate multiple addresses with commas and append the port after the last address: listen: 127.0.0.1,127.0.0.2:5432. Patroni uses the first address for local connections.

  • use_unix_socket: prefer Unix sockets for cluster connections. Default: false. If unix_socket_directories is set, use its first suitable value, or TCP if none is suitable. If absent from postgresql.parameters, assume the default socket directory and omit host from connection parameters.

  • use_unix_socket_repl: prefer Unix sockets for replication user connections. Default: false. If unix_socket_directories is set, use its first suitable value, or TCP if none is suitable. If absent from postgresql.parameters, assume the default socket directory and omit host from connection parameters.

  • pgpass: path to the .pgpass password file. Patroni creates this file before executing pg_basebackup, the post_init script and under some other circumstances. The location must be writable by Patroni.

  • recovery_conf: additional configuration settings written to recovery.conf when configuring follower.

  • custom_conf : optional custom postgresql.conf file to use instead of postgresql.base.conf. It must exist on all nodes and be readable by PostgreSQL. The actual postgresql.conf includes it from its location. Patroni neither monitors nor backs up this file. Patroni configuration can still override its settings; see dynamic configuration.

  • parameters: configuration parameters (GUCs) for Postgres in format {ssl: "on", ssl_cert_file: "cert_file"}.

  • parameters_primary: (optional) role-specific parameter overrides for primary. These values are merged with and override the base parameters.

  • parameters_replica: (optional) role-specific parameter overrides for replica. These values are merged with and override the base parameters.

  • parameters_standby_leader: (optional) role-specific parameter overrides for standby_leader. These values are merged with and override the base parameters.

  • pg_hba: list of lines that Patroni will use to generate pg_hba.conf. Patroni ignores this parameter if hba_file PostgreSQL parameter is set to a non-default value. Together with dynamic configuration this parameter simplifies management of pg_hba.conf.

    • - host all all 0.0.0.0/0 md5
    • - host replication replicator 127.0.0.1/32 md5: A line like this is required for replication.

  • pg_hba_primary: (optional) role-specific pg_hba entries for primary. These completely replace pg_hba (no merging). If not defined, pg_hba is used.

  • pg_hba_replica: (optional) role-specific pg_hba entries for replica. These completely replace pg_hba (no merging). If not defined, pg_hba is used.

  • pg_hba_standby_leader: (optional) role-specific pg_hba entries for standby_leader. These completely replace pg_hba (no merging). If not defined, pg_hba is used.

  • pg_ident: list of lines that Patroni will use to generate pg_ident.conf. Patroni ignores this parameter if ident_file PostgreSQL parameter is set to a non-default value. Together with dynamic configuration this parameter simplifies management of pg_ident.conf.

    • - mapname1 systemname1 pguser1
    • - mapname1 systemname2 pguser2

  • pg_ident_primary: (optional) role-specific pg_ident entries for primary. These completely replace pg_ident (no merging). If not defined, pg_ident is used.

  • pg_ident_replica: (optional) role-specific pg_ident entries for replica. These completely replace pg_ident (no merging). If not defined, pg_ident is used.

  • pg_ident_standby_leader: (optional) role-specific pg_ident entries for standby_leader. These completely replace pg_ident (no merging). If not defined, pg_ident is used.

  • pg_ctl_timeout: How long should pg_ctl wait when doing start, stop or restart. Default value is 60 seconds.

  • use_pg_rewind: try pg_rewind when the old leader rejoins as a replica. For correct operation, pg_rewind requires data page checksums and/or wal_log_hints set to on. Enable checksums with the --data-checksums option for initdb.

  • rewind: (optional) custom options to pass to the pg_rewind command. Can be specified as a list of strings and/or single key-value dictionaries. Not allowed options include: target-pgdata, source-pgdata, source-server, write-recovery-conf, dry-run, restore-target-wal, config-file, no-ensure-shutdown, version, and help. Example usage:

    postgresql:
      rewind:
        - debug
        - progress
        - sync-method: fsync

  • remove_data_directory_on_rewind_failure: If this option is enabled, Patroni will remove the PostgreSQL data directory and recreate the replica. Otherwise it will try to follow the new leader. Default value is false.

  • remove_data_directory_on_diverged_timelines: remove the data directory and recreate the replica if timelines diverge and the old primary cannot stream from the new primary. Use this when pg_rewind is unavailable. On PostgreSQL v10 and older, Patroni checks divergence by connecting to the “postgres” database with replication credentials. Permit this access in pg_hba.conf. Default: false.

  • replica_method: add a section for each create_replica_methods entry other than basebackup. Use the method name as the section name. At minimum, set “command” to the full script path. Patroni passes other parameters to the script as “parameter=value”.

  • pre_promote: fencing script run after leader lock acquisition and before replica promotion during failover. A non-zero exit prevents promotion and removes the DCS leader key.

  • before_stop: script run immediately before Postgres stops. It runs synchronously and blocks shutdown until completion. Shutdown then proceeds regardless of its exit code.

REST API

• restapi:
  • thread_pool_size: size of thread pool used by Patroni to process REST API requests. Minimal value is 5, default value is 5.
  • connect_address: address or host name and port for the REST API. Every cluster member must reach it. Use a non-loopback address except for a local demonstration; “localhost” and “127.0.0.1” are unsuitable for other deployments. It supports HTTP health checks, user queries, and member checks during leader elections. These checks determine whether the leader runs or another node has a later WAL position. Patroni stores connect_address in the DCS member key so member names resolve to REST API addresses.
  • listen: address or host name and port for REST API health checks and communication between cluster members. HAProxy and other load balancers can use HTTP “OPTION” or “GET” checks.
  • authentication: (optional)
    • username: Basic-auth username to protect unsafe REST API endpoints.
    • password: Basic-auth password to protect unsafe REST API endpoints.
  • certfile: (optional): Specifies the file with the certificate in the PEM format. If the certfile is not specified or is left empty, the API server will work without SSL.
  • keyfile: (optional): Specifies the file with the secret key in the PEM format.
  • keyfile_password: (optional): Specifies a password for decrypting the keyfile.
  • cafile: (optional): Specifies the file with the CA_BUNDLE with certificates of trusted CAs to use while verifying client certs.
  • ciphers: (optional): Specifies the permitted cipher suites (e.g. “ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:!SSLv1:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1”)
  • verify_client: (optional) client certificate verification mode: none (default), optional, or required. none disables verification. required requires certificates for all calls; successful signature verification authenticates the client. optional requires and checks certificates only for unsafe PUT, POST, PATCH, and DELETE requests.
  • allowlist: (optional) hosts permitted to call unsafe endpoints. Each entry can be a host name, IP address, or CIDR network. Default: allow all. With allowlist or allowlist_include_members set, all unlisted sources are rejected.
  • allowlist_include_members: (optional) if true, permits unsafe endpoint access from cluster members registered in the DCS. It uses each member’s api_url host name or IP address. The operating system may use a different address for outgoing connections.
  • http_extra_headers: (optional): HTTP headers let the REST API server pass additional information with an HTTP response.
  • https_extra_headers: (optional): HTTPS headers let the REST API server pass additional information with an HTTP response when TLS is enabled. This will also pass additional information set in http_extra_headers.
  • request_queue_size: (optional): Sets request queue size for TCP socket used by Patroni REST API. Once the queue is full, further requests get a “Connection denied” error. The default value is 5.
  • server_tokens: (optional): Configures the value of the Server HTTP header.
    • Minimal: The header will contain only the Patroni version, e.g. Patroni/4.0.0.
    • ProductOnly: The header will contain only the product name, e.g. Patroni.
    • Original (default): The header will expose the original behaviour and display the BaseHTTP and Python versions, e.g. BaseHTTP/0.6 Python/3.12.3.

Here is an example of both http_extra_headers and https_extra_headers:

restapi:
  listen: <listen>
  connect_address: <connect_address>
  authentication:
    username: <username>
    password: <password>
  http_extra_headers:
    'X-Frame-Options': 'SAMEORIGIN'
    'X-XSS-Protection': '1; mode=block'
    'X-Content-Type-Options': 'nosniff'
  cafile: <ca file>
  certfile: <cert>
  keyfile: <key>
  https_extra_headers:
    'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'

CTL

• ctl: (optional)
  • authentication:
    • username: Basic-auth username for accessing protected REST API endpoints. If not provided patronictl will use the value provided for REST API “username” parameter.
    • password: Basic-auth password for accessing protected REST API endpoints. If not provided patronictl will use the value provided for REST API “password” parameter.
  • insecure: Allow connections to REST API without verifying SSL certs.
  • cacert: CA_BUNDLE file or directory with trusted CAs for REST API SSL verification. If absent, patronictl uses the REST API “cafile” value.
  • certfile: Specifies the file with the client certificate in the PEM format.
  • keyfile: Specifies the file with the client secret key in the PEM format.
  • keyfile_password: Specifies a password for decrypting the client keyfile.

Watchdog

• mode: off, automatic, or required. off disables the watchdog. automatic uses it when available and otherwise ignores it. required prevents leadership unless watchdog activation succeeds.
• device: Path to watchdog device. Defaults to /dev/watchdog.
• safety_margin: Number of seconds of safety margin between watchdog triggering and leader key expiration.

Tags

• clonefrom: true or false. If true, other nodes may prefer this node as their bootstrap pg_basebackup source. With multiple nodes tagged clonefrom, selection is random. Default: false.
• noloadbalance: true or false. If set to true the node will return HTTP Status Code 503 for the GET /replica REST API health-check and therefore will be excluded from the load-balancing. Defaults to false.
• replicatefrom: The name of another replica to replicate from. Used to support cascading replication.
• nosync: true or false. If set to true the node will never be selected as a synchronous replica.
• sync_priority: integer priority for synchronous standby selection with synchronous_mode set to on. Higher values are preferred. Values of 0 or less exclude the node from synchronous_standby_names, like nosync: true. This is the opposite of sync_priority in pg_stat_replication.
• nofailover: true or false, controls whether this node is allowed to participate in the leader race and become a leader. Defaults to false, meaning this node can_ participate in leader races.
• failover_priority: integer failover priority. Higher values are preferred when nodes have received/replayed the same WAL amount. A higher receive/replay LSN takes precedence over priority. Values of 0 or less prevent leader election participation and promotion, like nofailover: true. Currently, failover_priority does not work with quorum-based synchronous replication.
• nostream: true or false. If true, disable WAL streaming through the replication protocol. Use archive recovery, if restore_command is configured, and pg_wal/pg_xlog polling instead. Also disable permanent logical slot copying and synchronization on this node and its cascading replicas. This tag has no effect on a primary.

In addition to these predefined tags, you can also add your own ones:

• key1: true
• key2: false
• key3: 1.4
• key4: "RandomString"

Tags appear in the REST API and patronictl_list. Use them in instance health checks. An undefined tag or a value different from the query value returns HTTP 503.

3.3 - Environment Configuration Settings

Source: https://patroni.readthedocs.io/en/latest/ENVIRONMENT.html

System environment variables can override some Patroni configuration file parameters. This page lists all supported variables. Environment values always take precedence over file values.

Global/Universal

• PATRONI_CONFIGURATION: sets the complete Patroni configuration through PATRONI_CONFIGURATION. Patroni then ignores all other environment variables.
• PATRONI_THREAD_POOL_SIZE: thread pool size for asynchronous tasks and REST API communication with other members during leader races or failsafe checks. Minimum: 5. Default: 5.
• PATRONI_THREAD_STACK_SIZE: stack size for Patroni threads. Use a multiple of 64kB. Minimum: 64kB. Patroni default: 512kB.
• PATRONI_NAME: name of the node where the current instance of Patroni is running. Must be unique for the cluster. The value __patroni_strict_sync_replica_placeholder__ is reserved for internal use by Patroni and cannot be used as a node name.
• PATRONI_NAMESPACE: path within the configuration store where Patroni will keep information about the cluster. Default value: “/service”
• PATRONI_SCOPE: cluster name
• PG_MALLOC_ARENA_MAX: custom value for MALLOC_ARENA_MAX environment variable for postmaster process. If not set, postmaster will inherit MALLOC_ARENA_MAX value.

Log

• PATRONI_LOG_TYPE: sets the format of logs. Can be either plain or json. To use json format, you must have the jsonlogger installed. The default value is plain.
• PATRONI_LOG_LEVEL: sets the general logging level. Default value is INFO (see the docs for Python logging)
• PATRONI_LOG_TRACEBACK_LEVEL: sets the level where tracebacks will be visible. Default value is ERROR. Set it to DEBUG if you want to see tracebacks only if you enable PATRONI_LOG_LEVEL=DEBUG.
• PATRONI_LOG_FORMAT: log format. For plain logs, use a string. For json, use a string or a list of LogRecord attributes. In lists, use field names without %( and ). To rename a field, use a dictionary with the original field as key and the output name as value. Default: %(asctime)s %(levelname)s: %(message)s.
• PATRONI_LOG_DATEFORMAT: sets the datetime formatting string. (see the formatTime() documentation)
• PATRONI_LOG_STATIC_FIELDS: add additional fields to the log. This option is only available when the log type is set to json. Example PATRONI_LOG_STATIC_FIELDS="{app: patroni}"
• PATRONI_LOG_MAX_QUEUE_SIZE: maximum internal log queue size. Patroni writes records to an in-memory queue. A separate thread reads them and writes to stderr or a file. The default 1000 records retain approximately 1h20m of logs.
• PATRONI_LOG_DIR: application log directory. It must exist and be writable by the Patroni user. By default, Patroni retains four 25MB files. Adjust retention with PATRONI_LOG_FILE_NUM and PATRONI_LOG_FILE_SIZE.
• PATRONI_LOG_MODE: Permissions for log files (for example, 0644). If not specified, permissions will be set based on the current umask value.
• PATRONI_LOG_FILE_NUM: The number of application logs to retain.
• PATRONI_LOG_FILE_SIZE: Size of patroni.log file (in bytes) that triggers a log rolling.
• PATRONI_LOG_LOGGERS: Redefine logging level per python module. Example PATRONI_LOG_LOGGERS="{patroni.postmaster: WARNING, urllib3: DEBUG}"
• PATRONI_LOG_DEDUPLICATE_HEARTBEAT_LOGS: suppresses consecutive identical heartbeat messages when true. Default: false.

Citus

Enable Citus integration. Patroni then registers workers on the coordinator. See Citus support.

• PATRONI_CITUS_GROUP: the Citus group id, integer. Use 0 for coordinator and 1, 2, etc… for workers
• PATRONI_CITUS_DATABASE: database for the citus extension. Use the same database on the coordinator and all workers. Currently, only one database is supported.

Consul

• PATRONI_CONSUL_HOST: the host:port for the Consul local agent.
• PATRONI_CONSUL_URL: url for the Consul local agent, in format: http(s)://host:port
• PATRONI_CONSUL_PORT: (optional) Consul port
• PATRONI_CONSUL_SCHEME: (optional) http or https, defaults to http
• PATRONI_CONSUL_TOKEN: (optional) ACL token
• PATRONI_CONSUL_VERIFY: (optional) whether to verify the SSL certificate for HTTPS requests
• PATRONI_CONSUL_CACERT: (optional) The ca certificate. If present it will enable validation.
• PATRONI_CONSUL_CERT: (optional) File with the client certificate
• PATRONI_CONSUL_KEY: (optional) File with the client key. Can be empty if the key is part of certificate.
• PATRONI_CONSUL_DC: (optional) Datacenter to communicate with. By default the datacenter of the host is used.
• PATRONI_CONSUL_CONSISTENCY: (optional) Select consul consistency mode. Possible values are default, consistent, or stale (more details in consul API reference)
• PATRONI_CONSUL_CHECKS: (optional) list of Consul health checks used for the session. By default an empty list is used.
• PATRONI_CONSUL_REGISTER_SERVICE: (optional) registers a service named by scope. Its role tag is master, primary, replica, or standby-leader. Default: false.
• PATRONI_CONSUL_SERVICE_TAGS: (optional) additional static tags to add to the Consul service apart from the role (primary/replica/standby-leader). By default an empty list is used.
• PATRONI_CONSUL_SERVICE_CHECK_INTERVAL: (optional) how often to perform health check against registered url
• PATRONI_CONSUL_SERVICE_CHECK_TLS_SERVER_NAME: (optional) override SNI host when connecting via TLS, see also consul agent check API reference.

Etcd

• PATRONI_ETCD_PROXY: proxy url for the etcd. If you are connecting to the etcd using proxy, use this parameter instead of PATRONI_ETCD_URL
• PATRONI_ETCD_URL: url for the etcd, in format: http(s)://(username:password@)host:port
• PATRONI_ETCD_HOSTS: list of etcd endpoints in format ‘host1:port1’,‘host2:port2’,etc…
• PATRONI_ETCD_USE_PROXIES: if true, treats hosts as a fixed proxy list. Patroni does not discover etcd topology.
• PATRONI_ETCD_PROTOCOL: http or https, if not specified http is used. If the url or proxy is specified - will take protocol from them.
• PATRONI_ETCD_HOST: the host:port for the etcd endpoint.
• PATRONI_ETCD_SRV: domain for SRV-based cluster discovery. Patroni queries these services in order until one succeeds: _etcd-client-ssl, _etcd-client, _etcd-ssl, _etcd, _etcd-server-ssl, _etcd-server. For _etcd-server-ssl or _etcd-server, it queries available members through the ETCD peer protocol. Otherwise, it uses hosts from the SRV records.
• PATRONI_ETCD_SRV_SUFFIX: suffix for discovery SRV names. Requires PATRONI_ETCD_SRV. Use it to distinguish etcd clusters in one domain. For example, PATRONI_ETCD_SRV_SUFFIX=foo and PATRONI_ETCD_SRV=example.org produce _etcd-client-ssl-foo._tcp.example.com. Patroni applies this suffix to each possible ETCD SRV service name.
• PATRONI_ETCD_USERNAME: username for etcd authentication.
• PATRONI_ETCD_PASSWORD: password for etcd authentication.
• PATRONI_ETCD_CACERT: The ca certificate. If present it will enable validation.
• PATRONI_ETCD_CERT: File with the client certificate.
• PATRONI_ETCD_KEY: File with the client key. Can be empty if the key is part of certificate.

Etcdv3

For Etcdv3, replace ETCD with ETCD3 in Etcd variable names. Examples include PATRONI_ETCD3_HOST and PATRONI_ETCD3_CACERT.

ZooKeeper

• PATRONI_ZOOKEEPER_HOSTS: Comma separated list of ZooKeeper cluster members: “‘host1:port1’,‘host2:port2’,’etc…’”. It is important to quote every single entity!
• PATRONI_ZOOKEEPER_USE_SSL: (optional) Whether SSL is used or not. Defaults to false. If set to false, all SSL specific parameters are ignored.
• PATRONI_ZOOKEEPER_CACERT: (optional) The CA certificate. If present it will enable validation.
• PATRONI_ZOOKEEPER_CERT: (optional) File with the client certificate.
• PATRONI_ZOOKEEPER_KEY: (optional) File with the client key.
• PATRONI_ZOOKEEPER_KEY_PASSWORD: (optional) The client key password.
• PATRONI_ZOOKEEPER_VERIFY: (optional) Whether to verify certificate or not. Defaults to true.
• PATRONI_ZOOKEEPER_SET_ACLS: (optional) default ACL that Kazoo applies to each new ZNode. Use x509 (default) or another supported ZooKeeper scheme, such as digest. Supply a dictionary: each key is the full principal, optionally prefixed with its scheme. Each value lists one or more permissions: CREATE, READ, WRITE, DELETE, ADMIN, or ALL. Example: set_acls: {CN=principal1: [CREATE, READ], digest:principal2:+pjROuBuuwNNSujKyH8dGcEnFPQ=: [ALL]}.
• PATRONI_ZOOKEEPER_AUTH_DATA: (optional) connection authentication dictionary. Use scheme as the key and credential as the value. Default: empty dictionary.

Exhibitor

• PATRONI_EXHIBITOR_HOSTS: initial list of Exhibitor (ZooKeeper) nodes in format: ‘host1,host2,etc…’. This list updates automatically whenever the Exhibitor (ZooKeeper) cluster topology changes.
• PATRONI_EXHIBITOR_PORT: Exhibitor port.

Kubernetes

• PATRONI_KUBERNETES_BYPASS_API_SERVICE: (optional) normally, Patroni contacts the kubernetes service at KUBERNETES_SERVICE_HOST. With PATRONI_KUBERNETES_BYPASS_API_SERVICE set to true, it discovers the API nodes behind the service and contacts them directly.
• PATRONI_KUBERNETES_NAMESPACE: (optional) Kubernetes namespace where the Patroni pod is running. Default value is default.
• PATRONI_KUBERNETES_LABELS: labels in {label1: value1, label2: value2} format. Patroni uses them to find cluster Pods and Endpoints or ConfigMaps. It also adds them to each Endpoint or ConfigMap it creates.
• PATRONI_KUBERNETES_SCOPE_LABEL: (optional) name of the label containing cluster name. Default value is cluster-name.
• PATRONI_KUBERNETES_BOOTSTRAP_LABELS: (optional) labels in {label1: value1, label2: value2} format. Patroni assigns them to pods in these states: initializing new cluster, running custom bootstrap script, starting after custom bootstrap, or creating replica.
• PATRONI_KUBERNETES_ROLE_LABEL: (optional) name of the label containing role (primary, replica or other custom value). Patroni will set this label on the pod it runs in. Default value is role.
• PATRONI_KUBERNETES_LEADER_LABEL_VALUE: (optional) value of the pod label when Postgres role is primary. Default value is primary.
• PATRONI_KUBERNETES_FOLLOWER_LABEL_VALUE: (optional) value of the pod label when Postgres role is replica. Default value is replica.
• PATRONI_KUBERNETES_STANDBY_LEADER_LABEL_VALUE: (optional) value of the pod label when Postgres role is standby_leader. Default value is primary.
• PATRONI_KUBERNETES_TMP_ROLE_LABEL: (optional) name of the temporary label containing role (primary or replica). Value of this label will always use the default of corresponding role. Set only when necessary.
• PATRONI_KUBERNETES_USE_ENDPOINTS: (optional) if set to true, Patroni will use Endpoints instead of ConfigMaps to run leader elections and keep cluster state.
• PATRONI_KUBERNETES_POD_IP: (optional) Patroni pod IP address. Required when PATRONI_KUBERNETES_USE_ENDPOINTS is enabled. On promotion, Patroni uses this value to populate leader endpoint subsets.
• PATRONI_KUBERNETES_PORTS: (optional) named Service ports require identical Endpoint port names. Otherwise, the service cannot work. For {Kind: Service, spec: {ports: [{name: postgresql, port: 5432, targetPort: 5432}]}}, set PATRONI_KUBERNETES_PORTS='[{"name": "postgresql", "port": 5432}]'. Patroni uses this value to update leader Endpoint subsets. Applies only with PATRONI_KUBERNETES_USE_ENDPOINTS.
• PATRONI_KUBERNETES_CACERT: (optional) CA_BUNDLE file with trusted CA certificates for Kubernetes API SSL verification. Defaults to the ServiceAccount secret value.
• PATRONI_RETRIABLE_HTTP_CODES: (optional) Kubernetes API status codes to retry. By default, Patroni retries 500, 503, and 504, and responses with a retry-after HTTP header.

Raft (deprecated)

• PATRONI_RAFT_SELF_ADDR: ip:port to listen on for Raft connections. The self_addr must be accessible from other nodes of the cluster. If not set, the node will not participate in consensus.
• PATRONI_RAFT_BIND_ADDR: (optional) ip:port to listen on for Raft connections. If not specified the self_addr will be used.
• PATRONI_RAFT_PARTNER_ADDRS: list of other Patroni nodes in the cluster in format "'ip1:port1','ip2:port2'". It is important to quote every single entity!
• PATRONI_RAFT_DATA_DIR: directory where to store Raft log and snapshot. If not specified the current working directory is used.
• PATRONI_RAFT_PASSWORD: (optional) Encrypt Raft traffic with a specified password, requires cryptography python module.
• PATRONI_RAFT_MIN_TIMEOUT: (optional) minimum election timeout in seconds for the underlying pysyncobj Raft implementation. Must be greater than 3 * PATRONI_RAFT_APPEND_ENTRIES_PERIOD. Default: 0.4.
• PATRONI_RAFT_MAX_TIMEOUT: (optional) maximum election timeout in seconds for the underlying pysyncobj Raft implementation. Must be greater than PATRONI_RAFT_MIN_TIMEOUT. Default: 1.4.
• PATRONI_RAFT_CONNECTION_TIMEOUT: (optional) time in seconds after which a connection with no data received is considered dead. Must be greater than or equal to PATRONI_RAFT_MAX_TIMEOUT. Default: 3.5.
• PATRONI_RAFT_APPEND_ENTRIES_PERIOD: (optional) interval in seconds for sending heartbeat commands. Must be less than one-third of PATRONI_RAFT_MIN_TIMEOUT. Default: 0.1.
• PATRONI_RAFT_CONNECTION_RETRY_TIME: (optional) interval in seconds between reconnection attempts to offline nodes. Default: 5.0.
• PATRONI_RAFT_LEADER_FALLBACK_TIMEOUT: (optional) time in seconds after which a leader with no response from the majority falls back to follower state. Must be greater than PATRONI_RAFT_APPEND_ENTRIES_PERIOD. Default: 30.0.

PostgreSQL

• PATRONI_POSTGRESQL_LISTEN: Postgres listen addresses and port. Separate multiple addresses with commas and append the port after the last address: listen: 127.0.0.1,127.0.0.2:5432. Patroni uses the first address for local PostgreSQL connections.
• PATRONI_POSTGRESQL_CONNECT_ADDRESS: IP address + port through which Postgres is accessible from other nodes and applications.
• PATRONI_POSTGRESQL_PROXY_ADDRESS: address and port of the connection pool beside Postgres, such as pgbouncer. Patroni stores it as proxy_url in the DCS member key for service discovery.
• PATRONI_POSTGRESQL_DATA_DIR: The location of the Postgres data directory, either existing or to be initialized by Patroni.
• PATRONI_POSTGRESQL_CONFIG_DIR: The location of the Postgres configuration directory, defaults to the data directory. Must be writable by Patroni.
• PATRONI_POSTGRESQL_BIN_DIR: PostgreSQL binary directory: pg_ctl, initdb, pg_controldata, pg_basebackup, postgres, pg_isready, and pg_rewind. The default empty string uses PATH to find executables.
• PATRONI_POSTGRESQL_BIN_PG_CTL: (optional) Custom name for pg_ctl binary.
• PATRONI_POSTGRESQL_BIN_INITDB: (optional) Custom name for initdb binary.
• PATRONI_POSTGRESQL_BIN_PG_CONTROLDATA: (optional) Custom name for pg_controldata binary.
• PATRONI_POSTGRESQL_BIN_PG_BASEBACKUP: (optional) Custom name for pg_basebackup binary.
• PATRONI_POSTGRESQL_BIN_POSTGRES: (optional) Custom name for postgres binary.
• PATRONI_POSTGRESQL_BIN_IS_READY: (optional) Custom name for pg_isready binary.
• PATRONI_POSTGRESQL_BIN_PG_REWIND: (optional) Custom name for pg_rewind binary.
• PATRONI_POSTGRESQL_PGPASS: path to the .pgpass password file. Patroni creates this file before executing pg_basebackup and under some other circumstances. The location must be writable by Patroni.
• PATRONI_REPLICATION_USERNAME: replication username; the user will be created during initialization. Replicas will use this user to access the replication source via streaming replication
• PATRONI_REPLICATION_PASSWORD: replication password; the user will be created during initialization.
• PATRONI_REPLICATION_SSLMODE: (optional) maps to sslmode. Specifies the TLS negotiation mode. See the PostgreSQL documentation for each mode. Default: prefer.
• PATRONI_REPLICATION_SSLKEY: (optional) maps to sslkey. Path to the secret key for the client certificate.
• PATRONI_REPLICATION_SSLPASSWORD: (optional) maps to sslpassword. Password for the secret key in PATRONI_REPLICATION_SSLKEY.
• PATRONI_REPLICATION_SSLCERT: (optional) maps to sslcert. Path to the client certificate.
• PATRONI_REPLICATION_SSLROOTCERT: (optional) maps to sslrootcert. File containing one or more trusted CA certificates. The client uses them to verify the server certificate.
• PATRONI_REPLICATION_SSLCRL: (optional) maps to sslcrl. Certificate revocation list file. The client rejects servers whose certificates appear in the list.
• PATRONI_REPLICATION_SSLCRLDIR: (optional) maps to sslcrldir. Directory containing certificate revocation list files. The client rejects servers whose certificates appear in these lists.
• PATRONI_REPLICATION_SSLNEGOTIATION: (optional) maps to sslnegotiation. Controls server SSL negotiation when SSL is used.
• PATRONI_REPLICATION_GSSENCMODE: (optional) maps to gssencmode. Controls whether to negotiate secure GSS TCP/IP and its priority.
• PATRONI_REPLICATION_CHANNEL_BINDING: (optional) maps to channel_binding. Controls client channel binding.
• PATRONI_SUPERUSER_USERNAME: name for the superuser, set during initialization (initdb) and later used by Patroni to connect to the postgres. Also this user is used by pg_rewind.
• PATRONI_SUPERUSER_PASSWORD: password for the superuser, set during initialization (initdb).
• PATRONI_SUPERUSER_SSLMODE: (optional) maps to sslmode. Specifies the TLS negotiation mode. See the PostgreSQL documentation for each mode. Default: prefer.
• PATRONI_SUPERUSER_SSLKEY: (optional) maps to sslkey. Path to the secret key for the client certificate.
• PATRONI_SUPERUSER_SSLPASSWORD: (optional) maps to sslpassword. Password for the secret key in PATRONI_SUPERUSER_SSLKEY.
• PATRONI_SUPERUSER_SSLCERT: (optional) maps to sslcert. Path to the client certificate.
• PATRONI_SUPERUSER_SSLROOTCERT: (optional) maps to sslrootcert. File containing one or more trusted CA certificates. The client uses them to verify the server certificate.
• PATRONI_SUPERUSER_SSLCRL: (optional) maps to sslcrl. Certificate revocation list file. The client rejects servers whose certificates appear in the list.
• PATRONI_SUPERUSER_SSLCRLDIR: (optional) maps to sslcrldir. Directory containing certificate revocation list files. The client rejects servers whose certificates appear in these lists.
• PATRONI_SUPERUSER_SSLNEGOTIATION: (optional) maps to sslnegotiation. Controls server SSL negotiation when SSL is used.
• PATRONI_SUPERUSER_GSSENCMODE: (optional) maps to gssencmode. Controls whether to negotiate secure GSS TCP/IP and its priority.
• PATRONI_SUPERUSER_CHANNEL_BINDING: (optional) maps to channel_binding. Controls client channel binding.
• PATRONI_REWIND_USERNAME: (optional) name for the user for pg_rewind; the user will be created during initialization of postgres 11+ and all necessary permissions will be granted.
• PATRONI_REWIND_PASSWORD: (optional) password for the user for pg_rewind; the user will be created during initialization.
• PATRONI_REWIND_SSLMODE: (optional) maps to sslmode. Specifies the TLS negotiation mode. See the PostgreSQL documentation for each mode. Default: prefer.
• PATRONI_REWIND_SSLKEY: (optional) maps to sslkey. Path to the secret key for the client certificate.
• PATRONI_REWIND_SSLPASSWORD: (optional) maps to sslpassword. Password for the secret key in PATRONI_REWIND_SSLKEY.
• PATRONI_REWIND_SSLCERT: (optional) maps to sslcert. Path to the client certificate.
• PATRONI_REWIND_SSLROOTCERT: (optional) maps to sslrootcert. File containing one or more trusted CA certificates. The client uses them to verify the server certificate.
• PATRONI_REWIND_SSLCRL: (optional) maps to sslcrl. Certificate revocation list file. The client rejects servers whose certificates appear in the list.
• PATRONI_REWIND_SSLCRLDIR: (optional) maps to sslcrldir. Directory containing certificate revocation list files. The client rejects servers whose certificates appear in these lists.
• PATRONI_REWIND_SSLNEGOTIATION: (optional) maps to sslnegotiation. Controls server SSL negotiation when SSL is used.
• PATRONI_REWIND_GSSENCMODE: (optional) maps to gssencmode. Controls whether to negotiate secure GSS TCP/IP and its priority.
• PATRONI_REWIND_CHANNEL_BINDING: (optional) maps to channel_binding. Controls client channel binding.

REST API

• PATRONI_RESTAPI_THREAD_POOL_SIZE: size of thread pool used by Patroni to process REST API requests. Minimal value is 5, default value is 5.
• PATRONI_RESTAPI_CONNECT_ADDRESS: IP address and port to access the REST API.
• PATRONI_RESTAPI_LISTEN: IP address and port that Patroni will listen to, to provide health-check information for HAProxy.
• PATRONI_RESTAPI_USERNAME: Basic-auth username to protect unsafe REST API endpoints.
• PATRONI_RESTAPI_PASSWORD: Basic-auth password to protect unsafe REST API endpoints.
• PATRONI_RESTAPI_CERTFILE: Specifies the file with the certificate in the PEM format. If the certfile is not specified or is left empty, the API server will work without SSL.
• PATRONI_RESTAPI_KEYFILE: Specifies the file with the secret key in the PEM format.
• PATRONI_RESTAPI_KEYFILE_PASSWORD: Specifies a password for decrypting the keyfile.
• PATRONI_RESTAPI_CAFILE: Specifies the file with the CA_BUNDLE with certificates of trusted CAs to use while verifying client certs.
• PATRONI_RESTAPI_CIPHERS: (optional) Specifies the permitted cipher suites (e.g. “ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:!SSLv1:!SSLv2:!SSLv3:!TLSv1:!TLSv1.1”)
• PATRONI_RESTAPI_VERIFY_CLIENT: (optional) client certificate verification mode: none (default), optional, or required. none disables verification. required requires certificates for all calls; successful signature verification authenticates the client. optional requires and checks certificates only for unsafe PUT, POST, PATCH, and DELETE requests.
• PATRONI_RESTAPI_ALLOWLIST: (optional) hosts permitted to call unsafe endpoints. Each entry can be a host name, IP address, or CIDR network. Default: allow all. With allowlist or allowlist_include_members set, all unlisted sources are rejected.
• PATRONI_RESTAPI_ALLOWLIST_INCLUDE_MEMBERS: (optional) if true, permits unsafe endpoint access from cluster members registered in the DCS. It uses each member’s api_url host name or IP address. The operating system may use a different address for outgoing connections.
• PATRONI_RESTAPI_HTTP_EXTRA_HEADERS: (optional) HTTP headers let the REST API server pass additional information with an HTTP response.
• PATRONI_RESTAPI_HTTPS_EXTRA_HEADERS: (optional) HTTPS headers let the REST API server pass additional information with an HTTP response when TLS is enabled. This will also pass additional information set in http_extra_headers.
• PATRONI_RESTAPI_REQUEST_QUEUE_SIZE: (optional): Sets request queue size for TCP socket used by Patroni REST API. Once the queue is full, further requests get a “Connection denied” error. The default value is 5.
• PATRONI_RESTAPI_SERVER_TOKENS: (optional) Configures the value of the Server HTTP header. Original (default) will expose the original behaviour and display the BaseHTTP and Python versions, e.g. BaseHTTP/0.6 Python/3.12.3. Minimal: The header will contain only the Patroni version, e.g. Patroni/4.0.0. ProductOnly: The header will contain only the product name, e.g. Patroni.

CTL

• PATRONICTL_CONFIG_FILE: (optional) location of the configuration file.
• PATRONI_CTL_USERNAME: (optional) Basic-auth username for accessing protected REST API endpoints. If not provided patronictl will use the value provided for REST API “username” parameter.
• PATRONI_CTL_PASSWORD: (optional) Basic-auth password for accessing protected REST API endpoints. If not provided patronictl will use the value provided for REST API “password” parameter.
• PATRONI_CTL_INSECURE: (optional) Allow connections to REST API without verifying SSL certs.
• PATRONI_CTL_CACERT: (optional) CA_BUNDLE file or directory with trusted CAs for REST API SSL verification. If absent, patronictl uses the REST API “cafile” value.
• PATRONI_CTL_CERTFILE: (optional) Specifies the file with the client certificate in the PEM format.
• PATRONI_CTL_KEYFILE: (optional) Specifies the file with the client secret key in the PEM format.
• PATRONI_CTL_KEYFILE_PASSWORD: (optional) Specifies a password for decrypting the client keyfile.

4 - Patroni REST API

Source: https://patroni.readthedocs.io/en/latest/rest_api.html

Patroni uses its REST API during leader races. patronictl uses it for failovers, switchovers, reinitialization, restarts, and reloads. HAProxy and other load balancers use it for HTTP health checks. Monitoring systems can also use it. This page lists the available endpoints.

Health check endpoints

Health check GET requests return an HTTP status code and a JSON document with node status. Use HEAD or OPTIONS instead of GET if you do not need JSON.

• The following requests to Patroni REST API will return HTTP status code 200 only when the Patroni node is running as the primary with leader lock:

  • GET /
  • GET /primary
  • GET /read-write

• GET /standby-leader: returns HTTP status code 200 only when the Patroni node is running as the leader in a standby cluster.

• GET /leader: returns HTTP 200 if the node holds the leader lock. Unlike the preceding endpoints, it does not distinguish PostgreSQL roles primary and standby_leader.

• GET /replica: returns HTTP 200 only if the node is running, its role is replica, and noloadbalance is unset.

• GET /replica?replication_state=<required state>: adds a replication state check to replica. Use replication_state=streaming to exclude replicas still catching up through archive recovery.

• GET /replica?lag=<max-lag>: adds a lag check to replica. HTTP 200 requires lag below the limit. For performance, Patroni uses DCS cluster.last_leader_operation as the leader WAL position to calculate lag. Specify max-lag as an integer byte count or a readable value, such as 16kB, 64MB, or 1GB.

  • GET /replica?lag=1048576
  • GET /replica?lag=1024kB
  • GET /replica?lag=10MB
  • GET /replica?lag=1GB

• GET /replica?tag_key1=value1&tag_key2=value2: also checks custom key1 and key2 tags in the YAML tags section. An undefined tag or a value different from the query value returns HTTP 503.

  The following requests check leader or standby leader status. They ignore custom tags:

  • GET /?tag_key1=value1&tag_key2=value2
  • GET /leader?tag_key1=value1&tag_key2=value2
  • GET /primary?tag_key1=value1&tag_key2=value2
  • GET /read-write?tag_key1=value1&tag_key2=value2
  • GET /standby_leader?tag_key1=value1&tag_key2=value2
  • GET /standby-leader?tag_key1=value1&tag_key2=value2

• GET /read-only: like the above endpoint, but also includes the primary.

• GET /synchronous or GET /sync: returns HTTP status code 200 only when the Patroni node is running as a synchronous standby.

• GET /read-only-sync: like the above endpoint, but also includes the primary.

• GET /quorum: returns HTTP status code 200 only when this Patroni node is listed as a quorum node in synchronous_standby_names on the primary.

• GET /read-only-quorum: like the above endpoint, but also includes the primary.

• GET /asynchronous or GET /async: returns HTTP status code 200 only when the Patroni node is running as an asynchronous standby.

• GET /asynchronous?lag=<max-lag> or GET /async?lag=<max-lag>: adds a lag check to asynchronous or async. HTTP 200 requires lag below the limit. For performance, Patroni uses DCS cluster.last_leader_operation as the leader WAL position to calculate lag. Specify max-lag as an integer byte count or a readable value, such as 16kB, 64MB, or 1GB.

  • GET /async?lag=1048576
  • GET /async?lag=1024kB
  • GET /async?lag=10MB
  • GET /async?lag=1GB

• GET /health: returns HTTP status code 200 only when PostgreSQL is up and running.

• GET /liveness: returns HTTP 200 if the heartbeat loop operates correctly. Returns 503 if the last run was more than ttl seconds ago on the primary or 2*ttl on a replica. Use this for livenessProbe.

• GET /readiness?lag=<max-lag>&mode=apply|write: returns HTTP 200 for a leader or a running, replicating node within the lag limit. The lag parameter limits standby lag and defaults to maximum_lag_on_failover. Use bytes or readable values, such as 16kB, 64MB, or 1GB. Mode specifies whether WAL must be replayed (apply) or only received (write). The default is apply.

  As a Kubernetes readinessProbe, this prevents new pods from becoming ready before they catch up to the leader. With a PodDisruptionBudget, it prevents premature leader termination during rolling restarts. It also excludes replicas that cannot keep up from read-only traffic. Use this endpoint for readinessProbe where Kubernetes endpoints cannot support leader elections, such as OpenShift.

The liveness endpoint is lightweight and runs no SQL. Configure probes to start failing around leader key expiration. With the default ttl of 30s, use this example:

readinessProbe:
  httpGet:
    scheme: HTTP
    path: /readiness
    port: 8008
  initialDelaySeconds: 3
  periodSeconds: 10
  timeoutSeconds: 5
  successThreshold: 1
  failureThreshold: 3
livenessProbe:
  httpGet:
    scheme: HTTP
    path: /liveness
    port: 8008
  initialDelaySeconds: 3
  periodSeconds: 10
  timeoutSeconds: 5
  successThreshold: 1
  failureThreshold: 3

Monitoring endpoint

Patroni uses GET /patroni during leader races. Monitoring systems can also use it. Its JSON structure matches the health check endpoints.

Example: A healthy cluster

$ curl -s http://localhost:8008/patroni | jq .
{
  "state": "running",
  "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00",
  "role": "primary",
  "server_version": 160004,
  "xlog": {
    "location": 67395656
  },
  "timeline": 1,
  "replication": [
    {
      "usename": "replicator",
      "application_name": "patroni2",
      "client_addr": "10.89.0.6",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    },
    {
      "usename": "replicator",
      "application_name": "patroni3",
      "client_addr": "10.89.0.2",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    }
  ],
  "dcs_last_seen": 1692356718,
  "tags": {
    "clonefrom": true
  },
  "database_system_identifier": "7268616322854375442",
  "patroni": {
    "version": "4.0.0",
    "scope": "demo",
    "name": "patroni1"
  }
}

Example: An unlocked cluster

$ curl -s http://localhost:8008/patroni  | jq .
{
  "state": "running",
  "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00",
  "role": "replica",
  "server_version": 160004,
  "xlog": {
    "received_location": 67419744,
    "replayed_location": 67419744,
    "replayed_timestamp": null,
    "paused": false
  },
  "timeline": 1,
  "replication": [
    {
      "usename": "replicator",
      "application_name": "patroni2",
      "client_addr": "10.89.0.6",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    },
    {
      "usename": "replicator",
      "application_name": "patroni3",
      "client_addr": "10.89.0.2",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    }
  ],
  "cluster_unlocked": true,
  "dcs_last_seen": 1692356928,
  "tags": {
    "clonefrom": true
  },
  "database_system_identifier": "7268616322854375442",
  "patroni": {
    "version": "4.0.0",
    "scope": "demo",
    "name": "patroni1"
  }
}

Example: An unlocked cluster with DCS failsafe mode enabled

$ curl -s http://localhost:8008/patroni  | jq .
{
  "state": "running",
  "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00",
  "role": "replica",
  "server_version": 160004,
  "xlog": {
    "location": 67420024
  },
  "timeline": 1,
  "replication": [
    {
      "usename": "replicator",
      "application_name": "patroni2",
      "client_addr": "10.89.0.6",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    },
    {
      "usename": "replicator",
      "application_name": "patroni3",
      "client_addr": "10.89.0.2",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    }
  ],
  "cluster_unlocked": true,
  "failsafe_mode_is_active": true,
  "dcs_last_seen": 1692356928,
  "tags": {
    "clonefrom": true
  },
  "database_system_identifier": "7268616322854375442",
  "patroni": {
    "version": "4.0.0",
    "scope": "demo",
    "name": "patroni1"
  }
}

Example: A cluster with the pause mode enabled

$ curl -s http://localhost:8008/patroni  | jq .
{
  "state": "running",
  "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00",
  "role": "replica",
  "server_version": 160004,
  "xlog": {
    "location": 67420024
  },
  "timeline": 1,
  "replication": [
    {
      "usename": "replicator",
      "application_name": "patroni2",
      "client_addr": "10.89.0.6",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    },
    {
      "usename": "replicator",
      "application_name": "patroni3",
      "client_addr": "10.89.0.2",
      "state": "streaming",
      "sync_state": "async",
      "sync_priority": 0
    }
  ],
  "pause": true,
  "dcs_last_seen": 1724874295,
  "tags": {
    "clonefrom": true
  },
  "database_system_identifier": "7268616322854375442",
  "patroni": {
    "version": "4.0.0",
    "scope": "demo",
    "name": "patroni1"
  }
}

Retrieve the Patroni metrics in Prometheus format through the GET /metrics endpoint.

$ curl http://localhost:8008/metrics

# HELP patroni_version Patroni semver without periods. \
# TYPE patroni_version gauge
patroni_version{scope="batman",name="patroni1"} 040000
# HELP patroni_postgres_running Value is 1 if Postgres is running, 0 otherwise.
# TYPE patroni_postgres_running gauge
patroni_postgres_running{scope="batman",name="patroni1"} 1
# HELP patroni_postmaster_start_time Epoch seconds since Postgres started.
# TYPE patroni_postmaster_start_time gauge
patroni_postmaster_start_time{scope="batman",name="patroni1"} 1724873966.352526
# HELP patroni_primary Value is 1 if this node is the leader, 0 otherwise.
# TYPE patroni_primary gauge
patroni_primary{scope="batman",name="patroni1"} 1
# HELP patroni_xlog_location Current location of the Postgres transaction log, 0 if this node is not the leader.
# TYPE patroni_xlog_location counter
patroni_xlog_location{scope="batman",name="patroni1"} 22320573386952
# HELP patroni_standby_leader Value is 1 if this node is the standby_leader, 0 otherwise.
# TYPE patroni_standby_leader gauge
patroni_standby_leader{scope="batman",name="patroni1"} 0
# HELP patroni_replica Value is 1 if this node is a replica, 0 otherwise.
# TYPE patroni_replica gauge
patroni_replica{scope="batman",name="patroni1"} 0
# HELP patroni_sync_standby Value is 1 if this node is a sync standby replica, 0 otherwise.
# TYPE patroni_sync_standby gauge
patroni_sync_standby{scope="batman",name="patroni1"} 0
# HELP patroni_quorum_standby Value is 1 if this node is a quorum standby replica, 0 otherwise.
# TYPE patroni_quorum_standby gauge
patroni_quorum_standby{scope="batman",name="patroni1"} 0
# HELP patroni_xlog_received_location Current location of the received Postgres transaction log, 0 if this node is not a replica.
# TYPE patroni_xlog_received_location counter
patroni_xlog_received_location{scope="batman",name="patroni1"} 0
# HELP patroni_xlog_replayed_location Current location of the replayed Postgres transaction log, 0 if this node is not a replica.
# TYPE patroni_xlog_replayed_location counter
patroni_xlog_replayed_location{scope="batman",name="patroni1"} 0
# HELP patroni_xlog_replayed_timestamp Current timestamp of the replayed Postgres transaction log, 0 if null.
# TYPE patroni_xlog_replayed_timestamp gauge
patroni_xlog_replayed_timestamp{scope="batman",name="patroni1"} 0
# HELP patroni_xlog_paused Value is 1 if the Postgres xlog is paused, 0 otherwise.
# TYPE patroni_xlog_paused gauge
patroni_xlog_paused{scope="batman",name="patroni1"} 0
# HELP patroni_postgres_streaming Value is 1 if Postgres is streaming, 0 otherwise.
# TYPE patroni_postgres_streaming gauge
patroni_postgres_streaming{scope="batman",name="patroni1"} 1
# HELP patroni_postgres_in_archive_recovery Value is 1 if Postgres is replicating from archive, 0 otherwise.
# TYPE patroni_postgres_in_archive_recovery gauge
patroni_postgres_in_archive_recovery{scope="batman",name="patroni1"} 0
# HELP patroni_postgres_server_version Version of Postgres (if running), 0 otherwise.
# TYPE patroni_postgres_server_version gauge
patroni_postgres_server_version{scope="batman",name="patroni1"} 160004
# HELP patroni_cluster_unlocked Value is 1 if the cluster is unlocked, 0 if locked.
# TYPE patroni_cluster_unlocked gauge
patroni_cluster_unlocked{scope="batman",name="patroni1"} 0
# HELP patroni_failsafe_mode_is_active Value is 1 if failsafe mode is active, 0 otherwise.
# TYPE patroni_failsafe_mode_is_active gauge
patroni_failsafe_mode_is_active{scope="batman",name="patroni1"} 0
# HELP patroni_failsafe_mode_enabled Value is 1 if failsafe_mode is enabled, 0 otherwise.
# TYPE patroni_failsafe_mode_enabled gauge
patroni_failsafe_mode_enabled{scope="batman",name="patroni1"} 0
# HELP patroni_failsafe_member Value is 1 if this node is a member of failsafe, 0 otherwise.
# TYPE patroni_failsafe_member gauge
patroni_failsafe_member{scope="batman",name="patroni1"} 0
# HELP patroni_postgres_timeline Postgres timeline of this node (if running), 0 otherwise.
# TYPE patroni_postgres_timeline gauge
patroni_postgres_timeline{scope="batman",name="patroni1"} 24
# HELP patroni_dcs_last_seen Epoch timestamp when DCS was last contacted successfully by Patroni.
# TYPE patroni_dcs_last_seen gauge
patroni_dcs_last_seen{scope="batman",name="patroni1"} 1724874235
# HELP patroni_pending_restart Value is 1 if the node needs a restart, 0 otherwise.
# TYPE patroni_pending_restart gauge
patroni_pending_restart{scope="batman",name="patroni1"} 1
# HELP patroni_is_paused Value is 1 if auto failover is disabled, 0 otherwise.
# TYPE patroni_is_paused gauge
patroni_is_paused{scope="batman",name="patroni1"} 1
# HELP patroni_postgres_state Numeric representation of Postgres state.
# Values: 0=initdb, 1=initdb_failed, 2=custom_bootstrap, 3=custom_bootstrap_failed, 4=creating_replica, 5=running, 6=starting, 7=bootstrap_starting, 8=start_failed, 9=restarting, 10=restart_failed, 11=stopping, 12=stopped, 13=stop_failed, 14=crashed
# TYPE patroni_postgres_state gauge
patroni_postgres_state{scope="batman",name="patroni1"} 5
# HELP patroni_failover_priority Failover priority of this node.
# TYPE patroni_failover_priority gauge
patroni_failover_priority{scope="batman",name="patroni1"} 1

PostgreSQL State Values

The patroni_postgres_state metric represents the current PostgreSQL state numerically. Monitoring and alerting systems can use it to track state changes. The PostgresqlState.get_metrics_description() static method generates these values.

PostgreSQL State Values

Cluster status endpoints

• The GET /cluster endpoint generates a JSON document describing the current cluster topology and state:

$ curl -s http://localhost:8008/cluster | jq .
{
  "members": [
    {
      "name": "patroni1",
      "role": "leader",
      "state": "running",
      "api_url": "http://10.89.0.4:8008/patroni",
      "host": "10.89.0.4",
      "port": 5432,
      "timeline": 5,
      "tags": {
        "clonefrom": true
      }
    },
    {
      "name": "patroni2",
      "role": "replica",
      "state": "streaming",
      "api_url": "http://10.89.0.6:8008/patroni",
      "host": "10.89.0.6",
      "port": 5433,
      "timeline": 5,
      "tags": {
        "clonefrom": true
      },
      "receive_lag": 0,
      "receive_lsn": "0/4000060",
      "replay_lag": 0,
      "replay_lsn": "0/4000060",
      "lag": 0,
      "lsn": "0/4000060"
    }
  ],
  "scope": "demo",
  "scheduled_switchover": {
    "at": "2023-09-24T10:36:00+02:00",
    "from": "patroni1",
    "to": "patroni3"
  }
}

• GET /history returns cluster switchover and failover history. Its format matches history files in pg_wal, with one addition: the timestamp when each timeline was created.

$ curl -s http://localhost:8008/history | jq .
[
  [
    1,
    25623960,
    "no recovery target specified",
    "2019-09-23T16:57:57+02:00"
  ],
  [
    2,
    25624344,
    "no recovery target specified",
    "2019-09-24T09:22:33+02:00"
  ],
  [
    3,
    25624752,
    "no recovery target specified",
    "2019-09-24T09:26:15+02:00"
  ],
  [
    4,
    50331856,
    "no recovery target specified",
    "2019-09-24T09:35:52+02:00"
  ]
]

Config endpoint

GET /config: Get the current version of the dynamic configuration:

$ curl -s http://localhost:8008/config | jq .
{
  "ttl": 30,
  "loop_wait": 10,
  "retry_timeout": 10,
  "maximum_lag_on_failover": 1048576,
  "postgresql": {
    "use_slots": true,
    "use_pg_rewind": true,
    "parameters": {
      "hot_standby": "on",
      "wal_level": "hot_standby",
      "max_wal_senders": 5,
      "max_replication_slots": 5,
      "max_connections": "100"
    }
  }
}

PATCH /config: Change the existing configuration.

$ curl -s -XPATCH -d \
    '{"loop_wait":5,"ttl":20,"postgresql":{"parameters":{"max_connections":"101"}}}' \
    http://localhost:8008/config | jq .
{
  "ttl": 20,
  "loop_wait": 5,
  "maximum_lag_on_failover": 1048576,
  "retry_timeout": 10,
  "postgresql": {
    "use_slots": true,
    "use_pg_rewind": true,
    "parameters": {
      "hot_standby": "on",
      "wal_level": "hot_standby",
      "max_wal_senders": 5,
      "max_replication_slots": 5,
      "max_connections": "101"
    }
  }
}

The above REST API call patches the existing configuration and returns the new configuration.

Check that the node applied the configuration. With loop_wait=5, it should log every 5 seconds. Changing “max_connections” requires restart, so the output should show “pending_restart”:

$ curl -s http://localhost:8008/patroni | jq .
{
  "database_system_identifier": "6287881213849985952",
  "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00",
  "xlog": {
    "location": 2197818976
  },
  "timeline": 1,
  "dcs_last_seen": 1724874545,
  "database_system_identifier": "7408277255830290455",
  "pending_restart": true,
  "pending_restart_reason": {
    "max_connections": {
      "old_value": "100",
      "new_value": "101"
    }
  },
  "patroni": {
    "version": "4.0.0",
    "scope": "batman",
    "name": "patroni1"
  },
  "state": "running",
  "role": "primary",
  "server_version": 160004
}

Removing parameters:

To remove or reset a setting, patch it with null:

$ curl -s -XPATCH -d \
    '{"postgresql":{"parameters":{"max_connections":null}}}' \
    http://localhost:8008/config | jq .
{
  "ttl": 20,
  "loop_wait": 5,
  "retry_timeout": 10,
  "maximum_lag_on_failover": 1048576,
  "postgresql": {
    "use_slots": true,
    "use_pg_rewind": true,
    "parameters": {
      "hot_standby": "on",
      "unix_socket_directories": ".",
      "wal_level": "hot_standby",
      "max_wal_senders": 5,
      "max_replication_slots": 5
    }
  }
}

The above call removes postgresql.parameters.max_connections from the dynamic configuration.

PUT /config: unconditionally replace the complete dynamic configuration:

$ curl -s -XPUT -d \
    '{"maximum_lag_on_failover":1048576,"retry_timeout":10,"postgresql":{"use_slots":true,"use_pg_rewind":true,"parameters":{"hot_standby":"on","wal_level":"hot_standby","unix_socket_directories":".","max_wal_senders":5}},"loop_wait":3,"ttl":20}' \
    http://localhost:8008/config | jq .
{
  "ttl": 20,
  "maximum_lag_on_failover": 1048576,
  "retry_timeout": 10,
  "postgresql": {
    "use_slots": true,
    "parameters": {
      "hot_standby": "on",
      "unix_socket_directories": ".",
      "wal_level": "hot_standby",
      "max_wal_senders": 5
    },
    "use_pg_rewind": true
  },
  "loop_wait": 3
}

Switchover and failover endpoints

Switchover

The /switchover endpoint requires a healthy cluster with a leader. It can also schedule a future switchover.

A candidate is optional for /switchover but required for /failover. Without a candidate, all eligible nodes enter the leader race after the leader steps down.

The POST JSON body must include leader. Optional fields are candidate and scheduled_at. Use scheduled_at to schedule a future switchover.

HTTP 200 means switchover or failover completed successfully. HTTP 202 means Patroni scheduled the switchover. Errors return 400, 412, or 503, with details in the response body.

DELETE /switchover can be used to delete the currently scheduled switchover.

Example: perform a switchover to any healthy standby

$ curl -s http://localhost:8008/switchover -XPOST -d '{"leader":"postgresql1"}'
Successfully switched over to "postgresql2"

Example: perform a switchover to a specific node

$ curl -s http://localhost:8008/switchover -XPOST -d \
    '{"leader":"postgresql1","candidate":"postgresql2"}'
Successfully switched over to "postgresql2"

Example: schedule a switchover from the leader to any other healthy standby in the cluster at a specific time.

$ curl -s http://localhost:8008/switchover -XPOST -d \
    '{"leader":"postgresql0","scheduled_at":"2019-09-24T12:00+00"}'
Switchover scheduled

Failover

Use /failover for manual failover when no sufficiently healthy node is available. For example, promote an asynchronous standby when all synchronous standbys are unsuitable. A missing leader is not required: failover also works on a healthy cluster.

In the JSON body of the POST request you must specify the candidate field. If the leader field is specified, a switchover is triggered instead.

Example:

$ curl -s http://localhost:8008/failover -XPOST -d '{"candidate":"postgresql1"}'
Successfully failed over to "postgresql1"

POST /switchover and POST /failover endpoints are used by patronictl_switchover and patronictl_failover, respectively.

DELETE /switchover is used by patronictl flush cluster-name switchover.

Failover/Switchover comparison

Healthy standby

A member must pass these checks to enter a switchover leader race or become the selected failover/switchover leader:

• be reachable via Patroni API;
• not have nofailover tag set to true;
• have watchdog fully functional (if required by the configuration);
• in case of a switchover in a healthy cluster or an automatic failover, not exceed maximum replication lag (maximum_lag_on_failover configuration parameter);
• For a healthy-cluster switchover or automatic failover, check the check_timeline configuration parameter. If it is true, the member timeline must not be smaller than the cluster timeline;
• in synchronous mode:
  • In case of a switchover (both with and without a candidate): be listed in the /sync key members;
  • For a failover in both healthy and unhealthy clusters, this check is omitted.

Restart endpoint

• POST /restart: restart Postgres on the specified node. The POST JSON body can contain these optional conditions:
  • restart_pending: if true, restart only when configuration changes require it.
  • role: restart only if the current node role matches the request.
  • postgres_version: restart only if the current Postgres version is lower than the requested version.
  • timeout: maximum wait for PostgreSQL to accept connections. Overrides primary_start_timeout.
  • schedule: future restart timestamp with time zone.
• DELETE /restart: delete the scheduled restart.

POST /restart and DELETE /restart endpoints are used by patronictl_restart and patronictl flush cluster-name restart respectively.

Reload endpoint

POST /reload makes Patroni reread and apply its configuration file. This is equivalent to SIGHUP. Changes to parameters such as shared_buffers still require an explicit Postgres restart. Use POST /restart or patronictl_restart.

The reload endpoint is used by patronictl_reload.

Reinitialize endpoint

POST /reinitialize: recreate the data directory on the specified replica. This operation is not allowed on a primary. It removes the data directory and runs pg_basebackup or another replica creation method.

This call may fail while Patroni repeatedly tries to restart a failed Postgres instance. Set {"force":true} in the request body to override this condition.

Set {“from-leader”:true} in the request body to take the basebackup directly from the leader. Use this for reinitialization when all replicas have failed.

The reinitialize endpoint is used by patronictl_reinit.

5 - patronictl

Source: https://patroni.readthedocs.io/en/latest/patronictl.html

The patronictl command-line interface communicates with the Patroni REST API and the DCS. Use it for cluster operations, interactively or in scripts.

Configuration

patronictl uses 3 sections of the configuration:

• ctl: REST API authentication and server identity verification. See ctl settings.
• restapi: fallback settings when ctl does not supply REST API authentication or server verification. patronictl uses restapi.authentication if ctl.authentication is absent, and restapi.cafile if ctl.cacert is absent. See REST API settings.
• DCS, such as etcd: DCS connection and authentication settings.

Supply these options through environment variables or a configuration file. See the corresponding sections in Environment Configuration Settings and YAML Configuration Settings.

Patronictl reads environment variables and uses their values directly.

By default, patronictl looks for patronictl.yaml in one of these system-specific directories. You can also specify a different file.

• Mac OS X: ~/Library/Application Support/patroni
• Mac OS X (POSIX): ~/.patroni
• Unix: ~/.config/patroni
• Unix (POSIX): ~/.patroni
• Windows (roaming): C:\Users\<user>\AppData\Roaming\patroni
• Windows (not roaming): C:\Users\<user>\AppData\Local\patroni

You can override that behavior either by:

• Setting the environment variable PATRONICTL_CONFIG_FILE with the path to a custom configuration file;
• Using the -c / --config-file command-line argument of patronictl with the path to a custom configuration file.

Usage

This section describes the operations available through patronictl.

The patronictl command accepts these global arguments:

-c / --config-file
As explained before, used to provide a path to a configuration file for patronictl.

-d / --dcs-url / --dcs
Provide a connection string to the DCS used by Patroni.

This argument sets or overrides the DCS and namespace in patronictl configuration.

Use DCS://HOST:PORT/NAMESPACE. For example, etcd3://localhost:2379/service connects to etcd v3 on localhost and stores cluster data under the service namespace. Missing components use configuration values or defaults.

-k / --insecure
Flag to bypass validation of REST API server SSL certificate.

This is the synopsis for running a command from the patronictl:

patronictl [ { -c | --config-file } CONFIG_FILE ]
  [ { -d | --dcs-url | --dcs } DCS_URL ] 
  [ { -k | --insecure } ]
  SUBCOMMAND

The following sections describe each patronictl command. Examples use postgres0.yml, postgres1.yml, and postgres2.yml from the Patroni GitHub repository.

patronictl demote-cluster

Synopsis

demote-cluster
  [ CLUSTER_NAME ]
  [ --host HOST ]
  [ --port PORT ]
  [ --restore-command RESTORE_COMMAND ]
  [ --primary-slot-name PRIMARY_SLOT_NAME ]
  [ --force ]

Description

patronictl demote-cluster converts a regular Patroni cluster into a standby cluster.

The command builds standby_cluster from the supplied remote primary connection options and adds it to dynamic configuration. It waits until the leader becomes a standby leader. Before the change, it prints the topology and requests confirmation unless --force is set.

At least one of --host, --port or --restore-command must be specified.

Parameters

CLUSTER_NAME: Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--host: Address of the remote node.

--port: Port of the remote node.

--restore-command: Command to restore WAL records from the remote primary.

--primary-slot-name: Name of the replication slot on the remote node to use for replication.

--force: Flag to skip confirmation prompts when demoting the cluster.

Useful for scripts.

Examples

Demote the cluster to a standby cluster that follows a remote primary endpoint:

$ patronictl -c postgres0.yml demote-cluster batman --host 192.0.2.10 --port 5432 --primary-slot-name batman --force

patronictl dsn

Synopsis

dsn
  [ CLUSTER_NAME ]
  [ { { -r | --role } { leader | primary | standby-leader | replica | standby | any } | { -m | --member } MEMBER_NAME } ]
  [ --group CITUS_GROUP ]

Description

patronictl dsn gets the connection string for one member of the Patroni cluster.

If several members match, the command selects one and gives priority to the primary.

Parameters

CLUSTER_NAME: Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

-r / --role
Choose a member that has the given role.

Role can be one of:

• leader: the leader of either a regular Patroni cluster or a standby Patroni cluster; or
• primary: the leader of a regular Patroni cluster; or
• standby-leader: the leader of a standby Patroni cluster; or
• replica: a replica of a Patroni cluster; or
• standby: same as replica; or
• any: any role. Same as omitting this parameter; or

-m / --member
Choose a member of the cluster with the given name.

MEMBER_NAME is the name of the member.

--group
Choose a member that is part of the given Citus group.

CITUS_GROUP is the ID of the Citus group.

Examples

Get DSN of the primary node:

$ patronictl -c postgres0.yml dsn batman -r primary
host=127.0.0.1 port=5432

Get DSN of the node named postgresql1:

$ patronictl -c postgres0.yml dsn batman --member postgresql1
host=127.0.0.1 port=5433

patronictl edit-config

Synopsis

edit-config
  [ CLUSTER_NAME ]
  [ --group CITUS_GROUP ]
  [ { -q | --quiet } ]
  [ { -s | --set } CONFIG="VALUE" [, ... ] ]
  [ { -p | --pg } PG_CONFIG="PG_VALUE" [, ... ] ]
  [ { --apply | --replace } CONFIG_FILE ]
  [ --force ]

Description

patronictl edit-config changes cluster dynamic configuration and writes it to the DCS.

Parameters

CLUSTER_NAME: Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--group
Change dynamic configuration of the given Citus group.

If omitted, patronictl uses citus.group if configured.

CITUS_GROUP is the ID of the Citus group.

-q / --quiet
Flag to skip showing the configuration diff.

-s / --set
Set a given dynamic configuration option with a given value.

CONFIG is the name of the dynamic configuration path in the YAML tree, with levels joined by . .

VALUE is the value for CONFIG. If it is null, then CONFIG will be removed from the dynamic configuration.

-p / --pg
Set a given dynamic Postgres configuration option with the given value.

This is shorthand for --s / --set with postgresql.parameters. before CONFIG.

PG_CONFIG is the name of the Postgres configuration to be set.

PG_VALUE is the value for PG_CONFIG. If it is null, then PG_CONFIG will be removed from the dynamic configuration.

--apply
Apply dynamic configuration from the given file.

It is similar to specifying multiple -s / --set options, one for each configuration from CONFIG_FILE.

CONFIG_FILE is the YAML file containing changes to apply. Use - to read from stdin.

--replace
Replace the dynamic configuration in the DCS with the dynamic configuration specified in the given file.

CONFIG_FILE is the YAML file containing replacement configuration. Use - to read from stdin.

--force
Flag to skip confirmation prompts when changing the dynamic configuration.

Useful for scripts.

Examples

Change max_connections Postgres GUC:

patronictl -c postgres0.yml edit-config batman --pg max_connections="150" --force
---
+++
@@ -1,6 +1,8 @@
loop_wait: 10
maximum_lag_on_failover: 1048576
postgresql:
+  parameters:
+    max_connections: 150
  pg_hba:
  - host replication replicator 127.0.0.1/32 md5
  - host all all 0.0.0.0/0 md5

Configuration changed

Change loop_wait and ttl settings:

patronictl -c postgres0.yml edit-config batman --set loop_wait="15" --set ttl="45" --force
---
+++
@@ -1,4 +1,4 @@
-loop_wait: 10
+loop_wait: 15
maximum_lag_on_failover: 1048576
postgresql:
  pg_hba:
@@ -6,4 +6,4 @@
  - host all all 0.0.0.0/0 md5
  use_pg_rewind: true
retry_timeout: 10
-ttl: 30
+ttl: 45

Configuration changed

Remove maximum_lag_on_failover setting from dynamic configuration:

patronictl -c postgres0.yml edit-config batman --set maximum_lag_on_failover="null" --force
---
+++
@@ -1,5 +1,4 @@
loop_wait: 10
-maximum_lag_on_failover: 1048576
postgresql:
  pg_hba:
  - host replication replicator 127.0.0.1/32 md5

Configuration changed

patronictl failover

Synopsis

failover
  [ CLUSTER_NAME ]
  [ --group CITUS_GROUP ]
  --candidate CANDIDATE_NAME
  [ --force ]

Description

patronictl failover performs a manual failover in the cluster.

Use this command when the cluster is unhealthy, for example:

• There is no leader; or
• There is no synchronous standby available in a synchronous cluster.

With synchronous mode enabled, this command can also fail over to an asynchronous node.

Parameters

CLUSTER_NAME: Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--group
Perform a failover in the given Citus group.

CITUS_GROUP is the ID of the Citus group.

--candidate
The node to be promoted on failover.

CANDIDATE_NAME is the name of the node to be promoted.

--force
Flag to skip confirmation prompts when performing the failover.

Useful for scripts.

Examples

Fail over to node postgresql2:

$ patronictl -c postgres0.yml failover batman --candidate postgresql2 --force
Current cluster topology
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  3 |             |     |            |     |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  3 |   0/40004E8 |   0 |  0/40004E8 |   0 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  3 |   0/40004E8 |   0 |  0/40004E8 |   0 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
2023-09-12 11:52:27.50978 Successfully failed over to "postgresql2"
+ Cluster: batman (7277694203142172922) -+---------+----+-------------+---------+------------+---------+
| Member      | Host           | Role    | State   | TL | Receive LSN |     Lag | Replay LSN |     Lag |
+-------------+----------------+---------+---------+----+-------------+---------+------------+---------+
| postgresql0 | 127.0.0.1:5432 | Replica | stopped |    |     unknown | unknown |    unknown | unknown |
| postgresql1 | 127.0.0.1:5433 | Replica | running |  3 |   0/4000188 |       0 |  0/4000188 |       0 |
| postgresql2 | 127.0.0.1:5434 | Leader  | running |  3 |             |         |            |         |
+-------------+----------------+---------+---------+----+-------------+---------+------------+---------+

patronictl flush

Synopsis

flush
  CLUSTER_NAME
  [ MEMBER_NAME [, ... ] ]
  { restart | switchover }
  [ --group CITUS_GROUP ]
  [ { -r | --role } { leader | primary | standby-leader | replica | standby | any } ]
  [ --force ]

Description

patronictl flush discards scheduled events, if any.

Parameters

CLUSTER_NAME
Name of the Patroni cluster.

MEMBER_NAME
Discard scheduled events for the given Patroni member(s).

Specify one or more members. If omitted, the command considers all members.

restart
Discard scheduled restart events.

switchover
Discard scheduled switchover event.

--group
Discard scheduled events from the given Citus group.

CITUS_GROUP is the ID of the Citus group.

-r / --role
Discard scheduled events for members that have the given role.

Role can be one of:

• leader: the leader of either a regular Patroni cluster or a standby Patroni cluster; or
• primary: the leader of a regular Patroni cluster; or
• standby-leader: the leader of a standby Patroni cluster; or
• replica: a replica of a Patroni cluster; or
• standby: same as replica; or
• any: any role. Same as omitting this parameter.

--force
Flag to skip confirmation prompts when performing the flush.

Useful for scripts.

Examples

Discard a scheduled switchover event:

$ patronictl -c postgres0.yml flush batman switchover --force
Success: scheduled switchover deleted

Discard scheduled restart of all standby nodes:

$ patronictl -c postgres0.yml flush batman restart -r replica --force
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+---------------------------+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag | Scheduled restart         |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+---------------------------+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     | 2025-03-23T18:00:00-03:00 |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/4000400 |   0 |  0/4000400 |   0 | 2025-03-23T18:00:00-03:00 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/4000400 |   0 |  0/4000400 |   0 | 2025-03-23T18:00:00-03:00 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+---------------------------+
Success: flush scheduled restart for member postgresql1
Success: flush scheduled restart for member postgresql2

Discard scheduled restart of nodes postgresql0 and postgresql1:

$ patronictl -c postgres0.yml flush batman postgresql0 postgresql1 restart --force
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+---------------------------+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag | Scheduled restart         |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+---------------------------+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     | 2025-03-23T18:00:00-03:00 |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/4000400 |   0 |  0/4000400 |   0 | 2025-03-23T18:00:00-03:00 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/4000400 |   0 |  0/4000400 |   0 | 2025-03-23T18:00:00-03:00 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+---------------------------+
Success: flush scheduled restart for member postgresql0
Success: flush scheduled restart for member postgresql1

patronictl history

Synopsis

history
  [ CLUSTER_NAME ]
  [ --group CITUS_GROUP ]
  [ { -f | --format } { pretty | tsv | json | yaml } ]

Description

patronictl history shows a history of failover and switchover events from the cluster, if any.

The following information is included in the output:

TL
Postgres timeline at which the event occurred.

LSN
Postgres LSN at which the event occurred.

Reason
Reason fetched from the Postgres .history file.

Timestamp
Time when the event occurred.

New Leader
Patroni member that has been promoted during the event.

Parameters

CLUSTER_NAME
Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--group
Show history of events from the given Citus group.

CITUS_GROUP is the ID of the Citus group.

If omitted, patronictl uses citus.group if configured.

-f / --format
How to format the list of events in the output.

Format can be one of:

• pretty: prints history as a pretty table; or
• tsv: prints history as tabular information, with columns delimited by \t; or
• json: prints history in JSON format; or
• yaml: prints history in YAML format.

The default is pretty.

--force
Flag to skip confirmation prompts when performing the flush.

Useful for scripts.

Examples

Show the history of events:

$ patronictl -c postgres0.yml history batman
+----+----------+------------------------------+----------------------------------+-------------+
| TL |      LSN | Reason                       | Timestamp                        | New Leader  |
+----+----------+------------------------------+----------------------------------+-------------+
|  1 | 24392648 | no recovery target specified | 2023-09-11T22:11:27.125527+00:00 | postgresql0 |
|  2 | 50331864 | no recovery target specified | 2023-09-12T11:34:03.148097+00:00 | postgresql0 |
|  3 | 83886704 | no recovery target specified | 2023-09-12T11:52:26.948134+00:00 | postgresql2 |
|  4 | 83887280 | no recovery target specified | 2023-09-12T11:53:09.620136+00:00 | postgresql0 |
+----+----------+------------------------------+----------------------------------+-------------+

Show the history of events in YAML format:

$ patronictl -c postgres0.yml history batman -f yaml
- LSN: 24392648
  New Leader: postgresql0
  Reason: no recovery target specified
  TL: 1
  Timestamp: '2023-09-11T22:11:27.125527+00:00'
- LSN: 50331864
  New Leader: postgresql0
  Reason: no recovery target specified
  TL: 2
  Timestamp: '2023-09-12T11:34:03.148097+00:00'
- LSN: 83886704
  New Leader: postgresql2
  Reason: no recovery target specified
  TL: 3
  Timestamp: '2023-09-12T11:52:26.948134+00:00'
- LSN: 83887280
  New Leader: postgresql0
  Reason: no recovery target specified
  TL: 4
  Timestamp: '2023-09-12T11:53:09.620136+00:00'

patronictl list

Synopsis

list
  [ CLUSTER_NAME [, ... ] ]
  [ --group CITUS_GROUP ]
  [ { -e | --extended } ]
  [ { -t | --timestamp } ]
  [ { -f | --format } { pretty | tsv | json | yaml } ]
  [ { -W | { -w | --watch } TIME } ]

Description

patronictl list shows information about Patroni cluster and its members.

The following information is included in the output:

Cluster
Name of the Patroni cluster.

Member
Name of the Patroni member.

Host
Host where the member is located.

Role
Current role of the member.

Can be one among:

• Leader: the current leader of a regular Patroni cluster; or
• Standby Leader: the current leader of a Patroni standby cluster; or
• Sync Standby: a synchronous standby of a Patroni cluster with synchronous mode enabled; or
• Replica: a regular standby of a Patroni cluster.

State
Current state of Postgres in the Patroni member.

Some examples among the possible states:

• running: if Postgres is currently up and running;
• streaming: if a replica and Postgres is currently streaming WALs from the primary node;
• in archive recovery: if a replica and Postgres is currently fetching WALs from the archive;
• stopped: if Postgres had been shut down;
• crashed: if Postgres has crashed.

TL
Current Postgres timeline in the Patroni member.

Receive LSN
Last WAL location received through streaming replication and synchronized to disk: pg_catalog.pg_last_(xlog|wal)_receive_(location|lsn)().

Receive Lag
Replication lag between the Receive LSN position of the member and its upstream in MB.

Replay LSN
Last WAL location replayed during member recovery: pg_catalog.pg_last_(xlog|wal)_replay_(location|lsn)().

Replay Lag
Replication lag between the Replay LSN position of the member and its upstream in MB.

Besides that, the following information may be included in the output:

System identifier
Postgres system identifier.

Group
Citus group ID.

Pending restart
* indicates that the node needs a restart for some Postgres configuration to take effect. An empty value indicates the node does not require a restart.

Scheduled restart
Scheduled restart time for the member’s Postgres instance. Empty if no restart is scheduled.

Tags
Member tags. Empty if no tags are set or all tags have default values.

Scheduled switchover
Timestamp at which a switchover has been scheduled for the Patroni cluster, if any.

Maintenance mode

If the cluster monitoring is currently paused.

Note

Shown in the table footer.

Only shown if the cluster is paused, and output format is pretty.

Parameters

CLUSTER_NAME
Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--group
Show information about members from the given Citus group.

CITUS_GROUP is the ID of the Citus group.

-e / --extended
Show extended information.

Force showing Pending restart, Scheduled restart and Tags attributes, even if their value is empty.

-t / --timestamp
Print timestamp before printing information about the cluster and its members.

-f / --format
How to format the list of events in the output.

Format can be one of:

• pretty: prints history as a pretty table; or
• tsv: prints history as tabular information, with columns delimited by \t; or
• json: prints history in JSON format; or
• yaml: prints history in YAML format.

The default is pretty.

-W
Automatically refresh information every 2 seconds.

-w / --watch
Automatically refresh information at the specified interval.

TIME is the interval between refreshes, in seconds.

Examples

Show information about the cluster in pretty format:

$ patronictl -c postgres0.yml list batman
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+

Show information about the cluster in pretty format with extended columns:

$ patronictl -c postgres0.yml list batman -e
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+-----------------+------------------------+-------------------+------+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag | Pending restart | Pending restart reason | Scheduled restart | Tags |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+-----------------+------------------------+-------------------+------+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     |                 |                        |                   |      |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |                 |                        |                   |      |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |                 |                        |                   |      |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+-----------------+------------------------+-------------------+------+

Show information about the cluster in YAML format, with timestamp of execution:

$ patronictl -c postgres0.yml list batman -f yaml -t
2023-09-12 13:30:48
- Cluster: batman
  Host: 127.0.0.1:5432
  Member: postgresql0
  Role: Leader
  State: running
  TL: 5
- Cluster: batman
  Host: 127.0.0.1:5433
  Receive LSN: 0/40004E8
  Receive Lag: 0
  Replay LSN: 0/40004E8
  Replay Lag: 0
  Member: postgresql1
  Role: Replica
  State: streaming
  TL: 5
- Cluster: batman
  Host: 127.0.0.1:5434
  Receive LSN: 0/40004E8
  Receive Lag: 0
  Replay LSN: 0/40004E8
  Replay Lag: 0
  Member: postgresql2
  Role: Replica
  State: streaming
  TL: 5

patronictl pause

Synopsis

pause
  [ CLUSTER_NAME ]
  [ --group CITUS_GROUP ]
  [ --wait ]

Description

patronictl pause temporarily puts the Patroni cluster in maintenance mode and disables automatic failover.

Parameters

CLUSTER_NAME
Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--group
Pause the given Citus group.

CITUS_GROUP is the ID of the Citus group.

If omitted, patronictl uses citus.group if configured.

--wait
Wait until all Patroni members are paused before returning control to the caller.

Examples

Put the cluster in maintenance mode, and wait until all nodes have been paused:

$ patronictl -c postgres0.yml pause batman --wait
'pause' request sent, waiting until it is recognized by all nodes
Success: cluster management is paused

patronictl promote-cluster

Synopsis

promote-cluster
  [ CLUSTER_NAME ]
  [ --force ]

Description

patronictl promote-cluster converts a standby cluster into a regular Patroni cluster.

The command removes standby_cluster from dynamic configuration and waits until the leader becomes primary. Before the change, it prints the topology and requests confirmation unless --force is set.

Parameters

CLUSTER_NAME: Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--force: Flag to skip confirmation prompts when promoting the cluster.

Useful for scripts.

Examples

Promote the standby cluster to run as a regular Patroni cluster:

$ patronictl -c postgres0.yml promote-cluster batman --force

patronictl query

Synopsis

query
  [ CLUSTER_NAME ]
  [ --group CITUS_GROUP ]
  [ { { -r | --role } { leader | primary | standby-leader | replica | standby | any } | { -m | --member } MEMBER_NAME } ]
  [ { -d | --dbname } DBNAME ]
  [ { -U | --username } USERNAME ]
  [ --password ]
  [ --format { pretty | tsv | json | yaml } ]
  [ { { -f | --file } FILE_NAME | { -c | --command } SQL_COMMAND } ]
  [ --delimiter ]
  [ { -W | { -w | --watch } TIME } ]

Description

patronictl query executes a SQL command or script against a member of the Patroni cluster.

Parameters

CLUSTER_NAME
Name of the Patroni cluster.

If omitted, patronictl uses scope if configured.

--group
Query the given Citus group.

CITUS_GROUP is the ID of the Citus group.

-r / --role
Choose a member that has the given role.

Role can be one of:

• leader: the leader of either a regular Patroni cluster or a standby Patroni cluster; or
• primary: the leader of a regular Patroni cluster; or
• standby-leader: the leader of a standby Patroni cluster; or
• replica: a replica of a Patroni cluster; or
• standby: same as replica; or
• any: any role. Same as omitting this parameter.

-m / --member
Choose a member that has the given name.

MEMBER_NAME is the name of the member to be picked.

-d / --dbname
Database to connect and run the query.

DBNAME is the name of the database. If not given, defaults to USERNAME.

-U / --username
User to connect to the database.

USERNAME name of the user. If not given, defaults to the operating system user running patronictl query.

--password
Prompt for the password of the connecting user.

Patroni uses libpq. You can therefore supply a password through ~/.pgpass or PGPASSWORD.

--format
How to format the output of the query.

Format can be one of:

• pretty: prints query output as a pretty table; or
• tsv: prints query output as tabular information, with columns delimited by \t; or
• json: prints query output in JSON format; or
• yaml: prints query output in YAML format.

The default is tsv.

-f / --file
Use a file as source of commands to run queries.

FILE_NAME is the path to the source file.

-c / --command
Run the given SQL command in the query.

SQL_COMMAND is the SQL command to be executed.

--delimiter
The delimiter when printing information in tsv format, or \t if omitted.

-W
Automatically re-run the query every 2 seconds.

-w / --watch
Automatically re-run the query at the specified interval.

TIME is the interval between re-runs, in seconds.

Examples

Run a SQL command as postgres user, and ask for its password:

$ patronictl -c postgres0.yml query batman -U postgres --password -c "SELECT now()"
Password:
now
2023-09-12 18:10:53.228084+00:00

Run a SQL command as postgres user, and take password from libpq environment variable:

$ PGPASSWORD=patroni patronictl -c postgres0.yml query batman -U postgres -c "SELECT now()"
now
2023-09-12 18:11:37.639500+00:00

Run a SQL command and print in pretty format every 2 seconds:

$ patronictl -c postgres0.yml query batman -c "SELECT now()" --format pretty -W
+----------------------------------+
| now                              |
+----------------------------------+
| 2023-09-12 18:12:16.716235+00:00 |
+----------------------------------+
+----------------------------------+
| now                              |
+----------------------------------+
| 2023-09-12 18:12:18.732645+00:00 |
+----------------------------------+
+----------------------------------+
| now                              |
+----------------------------------+
| 2023-09-12 18:12:20.750573+00:00 |
+----------------------------------+

Run a SQL command on database test and print the output in YAML format:

$ patronictl -c postgres0.yml query batman -d test -c "SELECT now() AS column_1, 'test' AS column_2" --format yaml
- column_1: 2023-09-12 18:14:22.052060+00:00
  column_2: test

Run a SQL command on member postgresql2:

$ patronictl -c postgres0.yml query batman -m postgresql2 -c "SHOW port"
port
5434

Run a SQL command on any of the standbys:

$ patronictl -c postgres0.yml query batman -r replica -c "SHOW port"
port
5433

patronictl reinit

Synopsis

reinit
  CLUSTER_NAME
  [ MEMBER_NAME [, ... ] ]
  [ --group CITUS_GROUP ]
  [ --wait ]
  [ --force ]
  [ --from-leader ]

Description

patronictl reinit rebuilds a Postgres standby instance managed by a replica member of the Patroni cluster.

Parameters

CLUSTER_NAME
Name of the Patroni cluster.

MEMBER_NAME
Name of the replica member for which the Postgres instance will be rebuilt.

Multiple replica members can be specified. If no members are specified, the command does nothing.

--group
Rebuild a replica member of the given Citus group.

CITUS_GROUP is the ID of the Citus group.

--wait
Wait until the reinitialization of the Postgres standby node(s) is finished.

--force
Flag to skip confirmation prompts when rebuilding Postgres standby instances.

--from-leader
Flag to get basebackup from leader directly.

Useful for scripts.

Examples

Request a rebuild of all replica members of the Patroni cluster and immediately return control to the caller:

$ patronictl -c postgres0.yml reinit batman postgresql1 postgresql2 --force
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
Success: reinitialize for member postgresql1
Success: reinitialize for member postgresql2

Request a rebuild of postgresql2 and wait for it to complete:

$ patronictl -c postgres0.yml reinit batman postgresql2 --wait --force
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
Success: reinitialize for member postgresql2
Waiting for reinitialize to complete on: postgresql2
Reinitialize is completed on: postgresql2

Request a rebuild of postgresql2 and get basebackup from leader directly:

$ patronictl -c postgres0.yml reinit batman postgresql2 --from-leader
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
Success: reinitialize for member postgresql2

patronictl reload

Synopsis

reload
  CLUSTER_NAME
  [ MEMBER_NAME [, ... ] ]
  [ --group CITUS_GROUP ]
  [ { -r | --role } { leader | primary | standby-leader | replica | standby | any } ]
  [ --force ]

Description

patronictl reload requests a reload of local configuration for one or more Patroni members.

It also triggers pg_ctl reload on the managed Postgres instance, even if nothing has changed.

Parameters

CLUSTER_NAME
Name of the Patroni cluster.

MEMBER_NAME
Request a reload of local configuration for the given Patroni member(s).

Specify one or more members. If omitted, the command considers all members.

--group
Request a reload of members of the given Citus group.

CITUS_GROUP is the ID of the Citus group.

-r / --role
Select members that have the given role.

Role can be one of:

• leader: the leader of either a regular Patroni cluster or a standby Patroni cluster; or
• primary: the leader of a regular Patroni cluster; or
• standby-leader: the leader of a standby Patroni cluster; or
• replica: a replica of a Patroni cluster; or
• standby: same as replica; or
• any: any role. Same as omitting this parameter.

--force
Flag to skip confirmation prompts when requesting a reload of the local configuration.

Useful for scripts.

Examples

Request a reload of the local configuration of all members of the Patroni cluster:

$ patronictl -c postgres0.yml reload batman --force
+ Cluster: batman (7277694203142172922) -+-----------+----+-------------+-----+------------+-----+
| Member      | Host           | Role    | State     | TL | Receive LSN | Lag | Replay LSN | Lag |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
| postgresql0 | 127.0.0.1:5432 | Leader  | running   |  5 |             |     |            |     |
| postgresql1 | 127.0.0.1:5433 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
| postgresql2 | 127.0.0.1:5434 | Replica | streaming |  5 |   0/40004E8 |   0 |  0/40004E8 |   0 |
+-------------+----------------+---------+-----------+----+-------------+-----+------------+-----+
Reload request received for member postgresql0 and will be processed within 10 seconds
Reload request received for member postgresql1 and will be processed within 10 seconds
Reload request received for member postgresql2 and will be processed within 10 seconds