pgspider_ext
Overview
| Package | Version | Category | License | Language |
|---|---|---|---|---|
pgspider_ext | 1.3.0 | FDW | PostgreSQL | C |
| ID | Extension | Bin | Lib | Load | Create | Trust | Reloc | Schema |
|---|---|---|---|---|---|---|---|---|
| 8540 | pgspider_ext | No | Yes | No | Yes | No | Yes | - |
| Related | plproxy wrappers multicorn postgres_fdw citus mysql_fdw oracle_fdw mongo_fdw |
|---|
Version
| Type | Repo | Version | PG Ver | Package | Deps |
|---|---|---|---|---|---|
| EXT | PGDG | 1.3.0 | 1817161514 | pgspider_ext | - |
| RPM | PGDG | 1.3.0 | 1817161514 | pgspider_ext_$v | - |
| DEB | PIGSTY | 1.3.0 | 1817161514 | postgresql-$v-pgspider-ext | - |
| OS / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
| el8.x86_64 | PGDG MISS | PGDG 1.3.0 el8.x86_64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel8.x86_64.rpm
| PGDG 1.3.0 el8.x86_64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel8.x86_64.rpm
| PGDG 1.3.0 el8.x86_64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel8.x86_64.rpm
| PGDG MISS |
| el8.aarch64 | PGDG MISS | PGDG 1.3.0 el8.aarch64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel8.aarch64.rpm
| PGDG 1.3.0 el8.aarch64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel8.aarch64.rpm
| PGDG 1.3.0 el8.aarch64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel8.aarch64.rpm
| PGDG MISS |
| el9.x86_64 | PGDG MISS | PGDG 1.3.0 el9.x86_64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel9.x86_64.rpm
| PGDG 1.3.0 el9.x86_64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel9.x86_64.rpm
| PGDG 1.3.0 el9.x86_64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel9.x86_64.rpm
| PGDG MISS |
| el9.aarch64 | PGDG MISS | PGDG 1.3.0 el9.aarch64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel9.aarch64.rpm
| PGDG 1.3.0 el9.aarch64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel9.aarch64.rpm
| PGDG 1.3.0 el9.aarch64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel9.aarch64.rpm
| PGDG MISS |
| el10.x86_64 | PGDG MISS | PGDG 1.3.0 el10.x86_64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel10.x86_64.rpm
| PGDG 1.3.0 el10.x86_64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel10.x86_64.rpm
| PGDG 1.3.0 el10.x86_64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel10.x86_64.rpm
| PGDG MISS |
| el10.aarch64 | PGDG MISS | PGDG 1.3.0 el10.aarch64.pg17 : pgspider_ext_17 pgspider_ext_17-1.3.0-1PGDG.rhel10.aarch64.rpm
| PGDG 1.3.0 el10.aarch64.pg16 : pgspider_ext_16 pgspider_ext_16-1.3.0-1PGDG.rhel10.aarch64.rpm
| PGDG 1.3.0 el10.aarch64.pg15 : pgspider_ext_15 pgspider_ext_15-1.3.0-1PGDG.rhel10.aarch64.rpm
| PGDG MISS |
| d12.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 d12.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~bookworm_amd64.deb
| PIGSTY 1.3.0 d12.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~bookworm_amd64.deb
| PIGSTY 1.3.0 d12.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~bookworm_amd64.deb
| PIGSTY MISS |
| d12.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 d12.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~bookworm_arm64.deb
| PIGSTY 1.3.0 d12.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~bookworm_arm64.deb
| PIGSTY 1.3.0 d12.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~bookworm_arm64.deb
| PIGSTY MISS |
| d13.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 d13.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~trixie_amd64.deb
| PIGSTY 1.3.0 d13.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~trixie_amd64.deb
| PIGSTY 1.3.0 d13.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~trixie_amd64.deb
| PIGSTY MISS |
| d13.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 d13.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~trixie_arm64.deb
| PIGSTY 1.3.0 d13.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~trixie_arm64.deb
| PIGSTY 1.3.0 d13.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~trixie_arm64.deb
| PIGSTY MISS |
| u22.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 u22.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~jammy_amd64.deb
| PIGSTY 1.3.0 u22.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~jammy_amd64.deb
| PIGSTY 1.3.0 u22.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~jammy_amd64.deb
| PIGSTY MISS |
| u22.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 u22.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~jammy_arm64.deb
| PIGSTY 1.3.0 u22.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~jammy_arm64.deb
| PIGSTY 1.3.0 u22.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~jammy_arm64.deb
| PIGSTY MISS |
| u24.x86_64 | PIGSTY MISS | PIGSTY 1.3.0 u24.x86_64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~noble_amd64.deb
| PIGSTY 1.3.0 u24.x86_64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~noble_amd64.deb
| PIGSTY 1.3.0 u24.x86_64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~noble_amd64.deb
| PIGSTY MISS |
| u24.aarch64 | PIGSTY MISS | PIGSTY 1.3.0 u24.aarch64.pg17 : postgresql-17-pgspider-ext postgresql-17-pgspider-ext_1.3.0-1PIGSTY~noble_arm64.deb
| PIGSTY 1.3.0 u24.aarch64.pg16 : postgresql-16-pgspider-ext postgresql-16-pgspider-ext_1.3.0-1PIGSTY~noble_arm64.deb
| PIGSTY 1.3.0 u24.aarch64.pg15 : postgresql-15-pgspider-ext postgresql-15-pgspider-ext_1.3.0-1PIGSTY~noble_arm64.deb
| PIGSTY MISS |
Build
You can build the DEB packages for pgspider_ext using pig build:
pig build pkg pgspider_ext # build DEB packages
Install
You can install pgspider_ext directly. First, make sure the PGDG repository is added and enabled:
pig repo add pgdg -u # Add PGDG repo and update cache
Install the extension using pig or apt/yum/dnf:
pig install pgspider_ext; # Install for current active PG version
pig ext install -y pgspider_ext -v 17 # PG 17
pig ext install -y pgspider_ext -v 16 # PG 16
pig ext install -y pgspider_ext -v 15 # PG 15
dnf install -y pgspider_ext_17 # PG 17
dnf install -y pgspider_ext_16 # PG 16
dnf install -y pgspider_ext_15 # PG 15
apt install -y postgresql-17-pgspider-ext # PG 17
apt install -y postgresql-16-pgspider-ext # PG 16
apt install -y postgresql-15-pgspider-ext # PG 15
Create Extension:
CREATE EXTENSION pgspider_ext;
Usage
pgspider_ext: Foreign data wrapper for remote PGSpider servers
PGSpider Extension turns PostgreSQL into a distributed query engine by creating virtual partitioned tables that transparently query data across multiple remote nodes in parallel.
Setup Child Servers
First, create servers for each data source using their respective FDWs:
CREATE EXTENSION pgspider_ext;
CREATE EXTENSION postgres_fdw;
CREATE SERVER pgsrv1 FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (host '127.0.0.1', port '5433', dbname 'postgres');
CREATE SERVER pgsrv2 FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (host '127.0.0.1', port '5434', dbname 'postgres');
CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv1
OPTIONS (user 'user', password 'pass');
CREATE USER MAPPING FOR CURRENT_USER SERVER pgsrv2
OPTIONS (user 'user', password 'pass');
Create Child Foreign Tables
CREATE FOREIGN TABLE t1_pg1_child (i int, t text)
SERVER pgsrv1 OPTIONS (table_name 't1');
CREATE FOREIGN TABLE t1_pg2_child (i int, t text)
SERVER pgsrv2 OPTIONS (table_name 't1');
Create PGSpider Partitioned Table
Create a PGSpider server and a partitioned parent table with an extra partition key column:
CREATE SERVER spdsrv FOREIGN DATA WRAPPER pgspider_ext;
CREATE USER MAPPING FOR CURRENT_USER SERVER spdsrv;
CREATE TABLE t1 (i int, t text, node text)
PARTITION BY LIST (node);
CREATE FOREIGN TABLE t1_pg1 PARTITION OF t1
FOR VALUES IN ('node1') SERVER spdsrv;
CREATE FOREIGN TABLE t1_pg2 PARTITION OF t1
FOR VALUES IN ('node2') SERVER spdsrv
OPTIONS (child_name 't1_pg2_child');
By default, PGSpider finds child tables using the pattern [parent_table_name]_child. Use child_name to specify a different name.
Query Across All Nodes
SELECT * FROM t1;
i | t | node
----+---+-------
10 | a | node1
11 | b | node1
20 | c | node2
21 | d | node2
Queries automatically fan out to all child nodes in parallel. WHERE clauses and aggregate functions are pushed down to child nodes:
SET enable_partitionwise_aggregate TO on;
SELECT count(*), node FROM t1 GROUP BY node;
Note: Only SELECT operations are supported; INSERT, UPDATE, and DELETE are not supported.
Feedback
Was this page helpful?
Thanks for the feedback! Please let us know how we can improve.
Sorry to hear that. Please let us know how we can improve.