pgspider_ext

foreign-data wrapper for remote PGSpider servers

Overview

PackageVersionCategoryLicenseLanguage
pgspider_ext1.3.0FDWPostgreSQLC
IDExtensionBinLibLoadCreateTrustRelocSchema
8540pgspider_extNoYesNoYesNoYes-
Relatedplproxy wrappers multicorn postgres_fdw citus mysql_fdw oracle_fdw mongo_fdw

Version

TypeRepoVersionPG VerPackageDeps
EXTPGDG1.3.01817161514pgspider_ext-
RPMPGDG1.3.01817161514pgspider_ext_$v-
DEBPIGSTY1.3.01817161514postgresql-$v-pgspider-ext-
OS / PGPG18PG17PG16PG15PG14
el8.x86_64PGDG MISSPGDG MISS
el8.aarch64PGDG MISSPGDG MISS
el9.x86_64PGDG MISSPGDG MISS
el9.aarch64PGDG MISSPGDG MISS
el10.x86_64PGDG MISSPGDG MISS
el10.aarch64PGDG MISSPGDG MISS
d12.x86_64PIGSTY MISSPIGSTY MISS
d12.aarch64PIGSTY MISS
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY MISS
d13.x86_64PIGSTY MISS
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY MISS
d13.aarch64PIGSTY MISS
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY MISS
u22.x86_64PIGSTY MISS
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY MISS
u22.aarch64PIGSTY MISS
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY MISS
u24.x86_64PIGSTY MISS
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY MISS
u24.aarch64PIGSTY MISS
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY 1.3.0
PIGSTY MISS

Build

You can build the DEB packages for pgspider_ext using pig build:

pig build pkg pgspider_ext         # build DEB packages

Install

You can install pgspider_ext directly. First, make sure the PGDG repository is added and enabled:

pig repo add pgdg -u          # Add PGDG repo and update cache

Install the extension using pig or apt/yum/dnf:

pig install pgspider_ext;          # Install for current active PG version
pig ext install -y pgspider_ext -v 17  # PG 17
pig ext install -y pgspider_ext -v 16  # PG 16
pig ext install -y pgspider_ext -v 15  # PG 15
dnf install -y pgspider_ext_17       # PG 17
dnf install -y pgspider_ext_16       # PG 16
dnf install -y pgspider_ext_15       # PG 15
apt install -y postgresql-17-pgspider-ext   # PG 17
apt install -y postgresql-16-pgspider-ext   # PG 16
apt install -y postgresql-15-pgspider-ext   # PG 15

Create Extension:

CREATE EXTENSION pgspider_ext;

Usage

pgspider_ext: Foreign data wrapper for remote PGSpider servers

PGSpider Extension turns PostgreSQL into a distributed query engine by creating virtual partitioned tables that transparently query data across multiple remote nodes in parallel.

Setup Child Servers

First, create servers for each data source using their respective FDWs:

CREATE EXTENSION pgspider_ext;
CREATE EXTENSION postgres_fdw;

CREATE SERVER pgsrv1 FOREIGN DATA WRAPPER postgres_fdw
  OPTIONS (host '127.0.0.1', port '5433', dbname 'postgres');
CREATE SERVER pgsrv2 FOREIGN DATA WRAPPER postgres_fdw
  OPTIONS (host '127.0.0.1', port '5434', dbname 'postgres');

CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv1
  OPTIONS (user 'user', password 'pass');
CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv2
  OPTIONS (user 'user', password 'pass');

Create Child Foreign Tables

CREATE FOREIGN TABLE t1_pg1_child (i int, t text)
  SERVER pgsrv1 OPTIONS (table_name 't1');
CREATE FOREIGN TABLE t1_pg2_child (i int, t text)
  SERVER pgsrv2 OPTIONS (table_name 't1');

Create PGSpider Partitioned Table

Create a PGSpider server and a partitioned parent table with an extra partition key column:

CREATE SERVER spdsrv FOREIGN DATA WRAPPER pgspider_ext;
CREATE USER MAPPING FOR CURRENT_USER SERVER spdsrv;

CREATE TABLE t1 (i int, t text, node text)
  PARTITION BY LIST (node);

CREATE FOREIGN TABLE t1_pg1 PARTITION OF t1
  FOR VALUES IN ('node1') SERVER spdsrv;
CREATE FOREIGN TABLE t1_pg2 PARTITION OF t1
  FOR VALUES IN ('node2') SERVER spdsrv
  OPTIONS (child_name 't1_pg2_child');

By default, PGSpider finds child tables using the pattern [parent_table_name]_child. Use child_name to specify a different name.

Query Across All Nodes

SELECT * FROM t1;
 i  | t | node
----+---+-------
 10 | a | node1
 11 | b | node1
 20 | c | node2
 21 | d | node2

Queries automatically fan out to all child nodes in parallel. WHERE clauses and aggregate functions are pushed down to child nodes:

SET enable_partitionwise_aggregate TO on;
SELECT count(*), node FROM t1 GROUP BY node;

Note: Only SELECT operations are supported; INSERT, UPDATE, and DELETE are not supported.


Last Modified 2026-03-12: add pg extension catalog (95749bf)