PGML 构建指南

构建库内机器学习扩展 PostgresML 需要一些额外的配置

构建 PGML

构建 PostgresML 是相当麻烦的工作,下面是在 EL8 与 EL9 上构建 PGML 的教程。

首先,根据 RPM 构建环境 的说明,配置好 环境与代理,安装 rustpgrx,特别注意 pgml 使用的 pgrx 版本号

准备Python3

安装 Python,并设置为默认版本:

sudo yum install python3.11 python3.11-devel python3-virtualenv openssl openssl-devel cmake pkg-config libomp libomp-devel openblas* llvm llvm-devel lld openblas*
sudo alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1
sudo alternatives --set python3 /usr/bin/python3.11
sudo alternatives --set python /usr/bin/python3.11

准备代码仓库

克隆 pgml,并 检出指定版本

cd ~; git clone --recursive [email protected]:postgresml/postgresml.git; 
cd ~/postgresml && git checkout v2.9.3
cd ~/postgresml/pgml-extension

EL8专用编译说明

本节修改在 EL8 上进行,EL9 无需执行此操作。

sudo dnf install gcc-toolset-13
source /opt/rh/gcc-toolset-13/enable
source /opt/rh/gcc-toolset-13/enable
export CC=/opt/rh/gcc-toolset-13/root/usr/bin/gcc
export CXX=/opt/rh/gcc-toolset-13/root/usr/bin/g++
export LD_LIBRARY_PATH=/opt/rh/gcc-toolset-13/root/usr/lib64:$LD_LIBRARY_PATH

在 EL8 上构建时,需要修改 build.rs 文件,在合适的位置添加两行编译选项:

println!("cargo:rustc-link-lib=static=stdc++fs");
println!("cargo:rustc-link-search=native=/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13");

整个文件应该是这样的:

fn main() {
    #[cfg(target_os = "macos")]
    {
        println!("cargo:rustc-link-search=/opt/homebrew/opt/openblas/lib");
        println!("cargo:rustc-link-search=/opt/homebrew/opt/libomp/lib");
    }

    // PostgreSQL is using dlopen(RTLD_GLOBAL). this will parse some
    // of symbols into the previous opened .so file, but the others will use a
    // relative offset in pgml.so, and will cause a null-pointer crash.
    //
    // hide all symbol to avoid symbol conflicts.
    //
    // append mode (link-args) only works with clang ld (lld)
    println!(
        "cargo:link-args=-Wl,--version-script={}/ld.map",
        std::env::current_dir().unwrap().to_string_lossy(),
    );

    println!("cargo:rustc-link-lib=static=stdc++fs");
    println!("cargo:rustc-link-search=native=/opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13");

    vergen::EmitBuilder::builder().all_git().emit().unwrap();
}

接着修改 Cargo.toml

[build-dependencies] 一节中添加:cc = "1.0"

[build-dependencies]
+++ cc = "1.0"

开始构建

针对 PostgreSQL 16, 15, 14 构建 PGML:

cd ~/postgresml/pgml-extension; pg16 build; pg15 build; pg14 build;

将构建产物放置于 ~/rpmbuild/SOURCES 目录备用

rm -rf ~/rpmbuild/SOURCES/pgml_16; cp -r ~/postgresml/pgml-extension/target/release/pgml-pg16 ~/rpmbuild/SOURCES/pgml_16;
rm -rf ~/rpmbuild/SOURCES/pgml_15; cp -r ~/postgresml/pgml-extension/target/release/pgml-pg15 ~/rpmbuild/SOURCES/pgml_15;
rm -rf ~/rpmbuild/SOURCES/pgml_14; cp -r ~/postgresml/pgml-extension/target/release/pgml-pg14 ~/rpmbuild/SOURCES/pgml_14;

使用 pgml.spec 执行 RPM 打包:

cd ~/rpmbuild/SPECS && make pgml

# 或手工进行构建:
rm -rf ~/rpmbuild/RPMS/x86_64/pgml*.rpm;
rpmbuild --without debuginfo --define "pgmajorversion 16" -ba ~/rpmbuild/SPECS/pgml.spec
rpmbuild --without debuginfo --define "pgmajorversion 15" -ba ~/rpmbuild/SPECS/pgml.spec
rpmbuild --without debuginfo --define "pgmajorversion 14" -ba ~/rpmbuild/SPECS/pgml.spec

Last modified 2024-08-04: add extensions (25d8a2b6)