MinIO Cluster Model

The MinIO module organizes MinIO in production as clusters—logical entities composed of a group of distributed MinIO instances, collectively providing highly available object storage services.

Each cluster is an autonomous S3-compatible object storage unit consisting of at least one MinIO instance, exposing service capabilities through the S3 API port.

There are three core entities in Pigsty’s MinIO module:

• Cluster: An autonomous MinIO service unit serving as the top-level namespace for other entities.
• Instance: A single MinIO server process running on a node, managing local disk storage.
• Node: A hardware resource abstraction running Linux + Systemd environment, implicitly declared.

Additionally, MinIO has the concept of Storage Pool, used for smooth cluster scaling. A cluster can contain multiple storage pools, each composed of a group of nodes and disks.

Deployment Modes

MinIO supports three main deployment modes for different scenarios:

SNSD mode can use any directory as storage for quick experimentation; SNMD and MNMD modes require real disk mount points, otherwise startup is refused.

Examples

Let’s look at a concrete multi-node multi-drive example with a four-node MinIO cluster:

minio:
  hosts:
    10.10.10.10: { minio_seq: 1 }
    10.10.10.11: { minio_seq: 2 }
    10.10.10.12: { minio_seq: 3 }
    10.10.10.13: { minio_seq: 4 }
  vars:
    minio_cluster: minio
    minio_data: '/data{1...4}'
    minio_node: '${minio_cluster}-${minio_seq}.pigsty'

The above config fragment defines a four-node MinIO cluster with four disks per node:

Identity Parameters

Pigsty uses the MINIO parameter group to assign deterministic identities to each MinIO module entity. Two parameters are required:

With cluster name defined at cluster level and instance number assigned at instance level, Pigsty automatically generates unique identifiers for each entity based on rules:

The MinIO module does not assign additional identity to host nodes; nodes are identified by their existing hostname or IP address. The minio_node parameter generates node names for MinIO cluster internal use (written to /etc/hosts for cluster discovery), not host node identity.

Core Configuration Parameters

Beyond identity parameters, the following parameters are critical for MinIO cluster configuration:

These parameters together determine MinIO’s core config MINIO_VOLUMES:

• SNSD: Direct minio_data value, e.g., /data/minio
• SNMD: Expanded minio_data directories, e.g., /data{1...4}
• MNMD: Combined minio_node and minio_data, e.g., https://minio-{1...4}.pigsty:9000/data{1...4}

Ports & Services

Each MinIO instance listens on the following ports:

MinIO enables HTTPS encrypted communication by default (controlled by minio_https). This is required for backup tools like pgBackREST to access MinIO.

Multi-node MinIO clusters can be accessed through any node. Best practice is to use a load balancer (e.g., HAProxy + VIP) for unified access point.

Resource Provisioning

After MinIO cluster deployment, Pigsty automatically creates the following resources (controlled by minio_provision):

Default Buckets (defined by minio_buckets):

Default Users (defined by minio_users):

pgbackrest is used for PostgreSQL cluster backups; s3user_meta and s3user_data are reserved users not actively used.

Monitoring Label System

Pigsty provides an out-of-box monitoring system that uses the above identity parameters to identify various MinIO entities.

minio_up{cls="minio", ins="minio-1", ip="10.10.10.10", job="minio"}
minio_up{cls="minio", ins="minio-2", ip="10.10.10.11", job="minio"}
minio_up{cls="minio", ins="minio-3", ip="10.10.10.12", job="minio"}
minio_up{cls="minio", ins="minio-4", ip="10.10.10.13", job="minio"}

For example, the cls, ins, ip labels correspond to cluster name, instance name, and node IP—the identifiers for these three core entities. They appear along with the job label in all MinIO monitoring metrics collected by VictoriaMetrics. The job name for collecting MinIO metrics is fixed as minio.