Pigsty v4.5: 575 Extensions, Kafka, MySQL, Valkey, and Silo

Pigsty v4.5.0 is a feature release spanning data services, the PostgreSQL ecosystem, automation safety, and the software supply chain. It adds Kafka KRaft and MySQL 8.4 pilot modules, adds a Valkey engine to REDIS, converges the MINIO module on Silo, and grows the packaged PostgreSQL extension catalog from 531 to 575.

This full review covers 100 commits, 416 files, 42,058 insertions, and 15,984 deletions from v4.4.0 (2026-07-10) to v4.5.0 (dab5dba3, 2026-08-14). See the complete source diff at v4.4.0...v4.5.0. A substantial share of the structured diff comes from re-exporting the entire Grafana dashboard set to Dashboard API v2.

Release Highlights

• 575 PostgreSQL extensions: 46 entries added and 2 removed, for a net gain of 44. Many Rust extensions move to pgrx 0.19.1, with synchronized RPM/DEB coverage and package-name mappings.
• Kafka KRaft module: Native Kafka orchestration with multi-cluster support, dynamic enrollment and retirement, SCRAM-SHA-512/TLS, credential rotation, health gates, monitoring, alerting, and four Grafana dashboards.
• MySQL 8.4 module: Standalone and three-node InnoDB Cluster deployments with MySQL Router, XtraBackup, account and database provisioning, parameter protection, monitoring, alerting, and conservative retirement workflows.
• Valkey and Silo: REDIS adds redis_type: valkey; MINIO now accepts only minio_type: silo. The RustFS backend developed during this cycle was withdrawn before the final baseline.
• Safer cluster orchestration: PGSQL, REDIS, MINIO, KAFKA, and MYSQL select members by explicit cluster identity. etcd delegation, DBSU key exchange, PITR, backup markers, and removal workflows are tightened as well.
• Observability upgrades: New Kafka and MySQL dashboards, MinIO/Silo Metrics V3, and pg_exporter 1.4-series configuration with PG19, transaction-age, lock, replication, and cleanup-pressure coverage.
• Repository and supply-chain upgrades: SOW now generates local repositories atomically without fabricated ModuleMD. Offline bundles can carry versioned source archives. Infra repackaging standardizes SHA-256-pinned inputs, SPDX licenses, and the 1PGSTY release suffix.
• 51 standalone configuration templates: Adds demo/kafka, demo/mysql, and the eight-node ha/octo simulation, while updating ha/trio to a compact three-node Silo HA topology.

Kafka KRaft Pilot Module

The new Kafka module uses node state as the source of truth and manages brokers and controllers with dynamic KRaft. It supports multiple Kafka clusters in one inventory and unbounded kafka.yml runs; incomplete --limit selections are rejected to avoid updating only part of the quorum.

• Nodes persist authoritative manifests and secrets, supporting dynamic controller joins, broker admission, member retirement, and three-stage failed-member replacement.
• SCRAM-SHA-512, TLS, certificate rotation, and credential rotation are supported, followed by partition-health and protocol self-tests.
• kafka-rm.yml requires a non-empty -l/--limit and validates absolute data paths, target identity, and the surviving anchors needed for partial retirement before stopping services.
• Adds JMX Exporter, Kafka Exporter, Victoria scrape and alert rules, plus Overview, Instance, Topic, and Consumer Grafana dashboards.

Kafka remains a pilot module. Clients must resolve and reach every broker directly; do not put the Kafka data plane behind HAProxy, a VIP, or a conventional L4 load balancer.

MySQL 8.4 Pilot Module

The new MySQL module targets a fixed MySQL 8.4 LTS platform. It supports a standalone instance or a three-node InnoDB Cluster and includes MySQL Shell, MySQL Router, TLS, user/database provisioning, scheduled full XtraBackup backups, and unified monitoring.

• Initialization validates exactly one or three members, creates the cluster, coordinates Router, and handles existing members and partial states idempotently.
• mysql_parameters protects replication, TLS, and platform-reserved settings; loose_, skip_, disable_, and enable_ prefixes cannot bypass the guardrails.
• mysql_databases accepts only name, encoding, and collate; databases are created with DEFAULT ENCRYPTION='N'.
• Removal validates the target, quorum, and member state before isolation, shutdown, and cleanup. Unlike other modules, mysql-rm.yml fails closed when a selected host has no MySQL identity.
• Adds Overview, Cluster, Instance, Replication, and Alert Grafana dashboards plus mysqld_exporter alerts.

Valkey, Silo, and Mongo Mode

Redis / Valkey

REDIS keeps Redis as the default engine and can select Valkey explicitly with redis_type: valkey. Valkey uses the valkey-server and valkey-cli packages, while configuration paths, monitoring jobs, service interfaces, and module parameters retain redis names for compatibility with existing inventories and dashboards.

Redis/Valkey systemd units now use Type=notify with a 1,800-second startup timeout. Topology validation, primary/replica relationships, password handling, tag-scoped removal, and rebuild protection are also strengthened.

MINIO / Silo

MINIO remains the compatibility module name, but the v4.5 role deploys Silo and only Silo; silo is the sole valid minio_type. Silo preserves the S3/Admin APIs, /minio/* routes, MINIO_* environment variables, and existing disk format, while the package, binary, and systemd service are now named silo.

• Startup checks the systemd Invocation ID, ActiveState=active, and Silo cluster health, waiting up to roughly 600 seconds so stale process state is not mistaken for a successful restart.
• Object-storage members are grouped by minio_cluster, independent of the inventory group name. Multi-cluster inventories should use distinct minio_alias, minio_domain, and minio_endpoint values.
• ha/trio changes from single-node object storage to three single-disk Silo members with EC:1, exposed on port 9002 through VIP and HAProxy. The template declares all three default users explicitly so configure -g produces consistent S3 credentials.
• Distributed Silo requires /data/minio to be a separate filesystem; Silo rejects an ordinary directory on the root filesystem.
• A RustFS backend was added during development and fully withdrawn from the final source. Infra repositories may still carry RustFS or MinIO packages, but neither is a supported v4.5 MINIO backend.

PostgreSQL Mongo Mode

The standalone FERRET role and mongo.yml playbook are removed. Their function is split between PostgreSQL Mongo mode and the FerretDB Docker APP: PostgreSQL/DocumentDB provides the data layer, and FerretDB in Docker Compose provides the MongoDB protocol layer. Pigsty no longer ships the old FerretDB systemd service, dedicated scrape configuration, or Mongo dashboards.

Orchestration, Security, and Lifecycle

• deploy.yml, slim.yml, and the PGSQL, REDIS, MINIO, KAFKA, and MYSQL initialization playbooks skip unrelated hosts by their respective *_cluster identities. PGSQL, REDIS, MINIO, and KAFKA removal playbooks apply the same rule.
• PGSQL configuration, PITR, and removal delegate only when a canonical, non-empty etcd group exists. DBSU SSH keys are exchanged among actual pg_cluster_members, including cross-inventory-group topologies such as Citus.
• PITR and removal delete only etcd subtrees bounded by /<cluster>/, avoiding accidental deletion when one cluster name prefixes another. The initial pgBackRest backup writes initial.done only after the command succeeds.
• Removal workflows stop services before cleaning data. Kafka, MySQL, and object storage add directory, quorum, surviving-member, and cluster-identity checks.
• A full pgsql.yml run is now explicitly described as initialization-only: it restarts Patroni/PostgreSQL and reapplies managed configuration and bootstrap SQL, so it is not a routine convergence command for initialized clusters.
• Pigsty-rendered systemd units live under /etc/systemd/system; permissions are tightened for sensitive configuration, environment files, keys, and privileged scripts. Debian/Ubuntu package installation suppresses premature starts of Silo, Redis/Valkey, and legacy logging services.
• HAProxy uses a fixed /etc/haproxy/haproxy.cfg plus /etc/haproxy/conf.d layout, upstream master-worker mode, a master socket, and Type=notify. dnsmasq uses dynamic binding, answers private PTR locally, and handles node addresses that appear only after INFRA initialization.
• Debian/Ubuntu time sync writes the native /etc/chrony/chrony.conf, while EL continues to use /etc/chrony.conf. During initial synchronization, offsets greater than ten seconds may step within the first 120 updates.

PostgreSQL Kernels and Extensions

• Default PostgreSQL updates to 18.6 with the repositories. The four standard Patroni templates add an output_plugin_libraries allowlist for pgoutput, test_decoding, and wal2json; Patroni filters unsupported settings on older versions.
• The PostgreSQL 19 beta template gains pgBackRest 2.59 and scheduled backup support. It still installs no extensions by default and remains a PG19 trial template.
• Percona PostgreSQL 18 TDE uses cluster mode and keeps Pigsty’s private installation prefix to avoid collisions with native PostgreSQL.
• IvorySQL gains default-database initialization and compatible WAL compression in the standard workload templates.
• The PostgreSQL fact loader, per-platform package_map, and default extension groups are refreshed together, fixing PGDG/YUM names and missing packages.

Comparing the embedded catalog snapshots in Pig v1.5.1 (531 entries) and v1.7.0 (575 entries), this cycle adds 46 extensions and removes 2, for a net gain of 44:

• 32 primary extensions added: argm, cat_tools, cron_utils, fbsql, oidc_validator, online_advisor, pg_cjk_parser, pg_column_tetris, pg_describe, pg_disorder, pg_fts, pg_jieba, pg_kpart, pg_lake, pg_local_cache, pg_mentat, pg_oidc_validator, pg_policy, pg_roast, pg_tiktoken_c, pg_turbovec, pg_vault_tde, pgcontext, pgfr_record, pgmemento, pgmonitor, pgsqlmock, pgwasm, plruby, plx, postbis, and qdgc.
• 13 subextensions added: hstore_plruby, jsonb_plruby, ltree_plruby, pg_extension_base, pg_extension_updater, pg_lake_copy, pg_lake_engine, pg_lake_iceberg, pg_lake_table, pg_map, pgcontext_pgvector, pgfr_analyze, and qdgc_postgis.
• One PGDG extension added: pg_statviz. It remains in the online catalog but is not part of the default installation groups.
• pg_analytics and spat removed: the former is archived, and the latter is a deprecated alpha project.

In addition, emailaddr, explain_ui, the Rust oidc_validator, pg_summarize, and smlar leave the default installation groups because upstream does not provide a distributable license; packages and catalog records may remain. pg_relation_sql is a standalone SQL toolkit without CREATE EXTENSION support, so it appears in the packaging table but is not counted among the 575 extensions.

Extension Package Update Matrix

The table consolidates RPM and DEB batches after v4.4.0 into one final “old version → final version” row per extension: 207 packaged entries plus two catalog removals. Where RPM and DEB differ, both are shown. An unchanged version can still represent a rebuild or a change in package naming, license metadata, or platform coverage.

The first packaging log after July 10 covers July 7–24 without per-entry dates, so the whole batch is included to avoid omitting post-release pgrx 0.19.1 rebuilds and matrix fixes. The final repository indexes remain authoritative for versions and platform availability.

Observability

• The complete Grafana dashboard set is re-exported through the Pig/Grafana tooling as Dashboard API v2. Four Kafka and five MySQL dashboards are added, and links, variables, and layouts are refreshed for Node, PostgreSQL, Redis, and Infra dashboards.
• MinIO/Silo Overview and Instance dashboards move to Metrics V3. Victoria scrapes /minio/metrics/v3 and drops high-cardinality samples with a non-empty bucket label.
• pg_exporter configuration updates to 1.4.0 and fixes the duplicate time series produced by the 1.4.1 pg_subrel query.
• PG19 gains pg_sub_19, pg_recovery_state, pg_wal_19, pg_lock_stat, and pg_vacuum_score collectors. PG10+ gains a pg_xact_age transaction-age histogram, replication-slot idle_timeout, and WAL Receiver connecting state encoding.
• Kafka JMX/protocol exporters and the MySQL exporter join unified service discovery, Victoria scraping, and alert rules. Journald records without a PRIORITY field are handled safely.

Repositories, Offline Bundles, and Supply Chain

• REPO and CACHE use sow create --pigsty to generate DNF/APT metadata and the SHA-256 repo_complete marker atomically, without injecting fabricated ModuleMD.
• module_hotfixes becomes an explicit per-repository choice and is enabled only for EL repositories that must replace system module streams, avoiding indiscriminate DNF module-filter bypasses.
• If dist/<version>/pigsty-<version>.tgz already exists, cache.yml includes it in the offline repository. It does not implicitly package the current worktree or download source code from the internet.
• RPM exporter package names change from underscores to hyphens, for example node_exporter → node-exporter; unversioned Provides/Obsoletes entries bridge the old names.
• Local Infra repackaging standardizes the PGSTY vendor, SPDX license expressions, 1PGSTY release suffix, /etc/default/<service> environment files, vendor units under /usr/lib/systemd/system, and legal files under /usr/share/doc/<package>.
• Simple binary packages use reproducible upstream archives pinned by SHA-256; vendor-native RPM/DEB files retain their original bytes and metadata. Direct artifacts such as Silo, Grafana, Code, Code Server, and Vector gain pinned URLs and checksums.
• China-region routing is refreshed systematically: operating systems, Docker, Grafana, Percona, MongoDB APT, and uv/PyPI prefer Tencent Cloud; EL/Docker retain Huawei Cloud and Alibaba Cloud fallbacks; MySQL/Kubernetes use USTC; ClickHouse uses Huawei Cloud.

Infra Package Update Matrix

The table consolidates this cycle’s Infra batches into one row per package, showing only the v4.4.0 value, final v4.5.0 value, and essential notes. Intermediate versions are omitted. “Built” or “verified” describes packaging records only; the signed repository index at release time remains authoritative for installable versions.

Platforms, Templates, and Release Engineering

• Recommended platform baselines update to Rocky Linux 9.8/10.2, Debian 12.15/13.6, and Ubuntu 22.04.5/24.04.4/26.04.0. Terraform cloud images and pinned Vagrant box versions are refreshed with them.
• Vagrant’s root disk becomes configurable through root_disk: 64 GiB by default and 128 GiB for the all, rpm, deb, oss, and pro build matrices. The existing disk setting still means the extra /data disk and defaults to 128 GiB. The vagrant user’s login shell is standardized on Bash.
• Adds the eight-node ha/octo simulation. ha/trio becomes a three-node Silo deployment behind VIP/HAProxy; demo/minio explicitly selects Silo and narrows its local repository package groups.
• The safe and demo templates use valid extension aliases. Tuned-profile paths adapt to the operating system, cluster-size comparisons remain compatible with Ansible versions before 2.19, and unused internal flags plus MINIO/INFRA variables are removed.
• The Docker base image updates to Debian 13.6 and is marked v4.5.0. Image publishing becomes manually triggered and builds from tags.
• docker/Makefile fixes its data directory at repository-local ./data; make purge no longer accepts an external DATA override and has no countdown.
• GitHub Actions updates checkout, CodeQL, Docker build/login, and Cosign in bulk. Release, bootstrap, installation, checksum, and build templates tighten parameter and file handling.
• Release archives generate top-level pigsty.yml from conf/meta.yml, include Kafka/MySQL playbooks, and remove the old Mongo playbook. The signing workflow creates Cosign signatures only for the pigsty-<tag>.tgz source archive, not multi-gigabyte offline bundles.
• README, SECURITY, CONTRIBUTING, AGENTS, and NOTICE align on v4.5, 575 extensions, Silo, Redis/Valkey, and Mongo mode, with updated 2026 copyright and trademark language.

Compatibility Changes and Upgrade Notes

Before upgrading from v4.4, review these changes carefully:

1. FERRET split: Remove the old ferretdb systemd service. Deploy the DocumentDB data layer through Mongo configuration mode, then deploy the FerretDB protocol layer with docker.yml / app.yml. The old mongo.yml, mongo_* parameters, and dedicated dashboards are gone.
2. Silo replaces the MinIO server: minio_type accepts only silo. Protocol and disk-format compatibility do not constitute migration acceptance; validate backups, in-place compatibility, rollback, and real read/write traffic before switching existing object storage.
3. ha/trio object-storage topology changes: The new template uses three single-disk Silo members. Do not expand an existing single-node pool in place by simply adding two members; create a new cluster and migrate objects.
4. HAProxy unit changes: Pigsty no longer renders /etc/default/haproxy, though the unit can read it when present. Use EXTRAOPTS only, keep -S /run/haproxy-master.sock, and do not add -f.
5. Explicit cluster identity: Custom inventories must define pg_cluster, redis_cluster, minio_cluster, kafka_cluster, or mysql_cluster for target hosts. Older inventories that relied only on fixed group names must add these identities.
6. Full PGSQL playbook is initialization-only: Do not run the complete pgsql.yml against initialized nodes. Use precise tags for routine maintenance; for a full rebuild, validate backups and remove the target member through the formal workflow first.
7. Valkey is opt-in: Valkey is installed only with redis_type: valkey; module parameters, configuration paths, and monitoring interfaces keep Redis names.
8. SOW is a core dependency: REPO/CACHE requires SOW 0.3.0. If an old offline bundle or local repository lacks sow, add it from the Pigsty Infra repository first.
9. Exporter RPM renames: External automation and private repositories should move from old names such as node_exporter and redis_exporter to node-exporter and redis-exporter.
10. Default extension groups tighten licensing: Environments that depend on emailaddr, explain_ui, oidc_validator, pg_summarize, or smlar must install them explicitly and assess licensing and redistribution boundaries independently.
11. Docker purge semantics change: make purge directly deletes repository-local ./data; confirm that it contains no container data you need to retain.
12. Vagrant disk semantics change: root_disk controls the system disk, while disk controls the extra /data disk. Distributed Silo requires the latter to be a real independent mount.

Get v4.5.0

Install on a fresh, supported Linux node:

curl -fsSL https://repo.pigsty.io/get | bash -s v4.5.0
cd ~/pigsty
./bootstrap
./configure
./install.yml

A production upgrade is not an overwrite installation. Read the compatibility changes above, resolve the exact target cluster, backup, inventory identity, and live state, then use an explicit -l scope for check mode or maintenance operations.