Offline Install

How to install pigsty without Internet access? How to make your own offline packages.

Module:

PIGSTY

Short Version

You have to download the Pigsty source tarball and the Offline Package in addition to the Standard Installation.

Package	GitHub Release	CDN Mirror
Pigsty Source Tarball	pigsty-v2.7.0.tgz	pigsty-v2.7.0.tgz
EL8 Offline Package	pigsty-pkg-2.7.0.el8.x86_64.tgz	pigsty-pkg-2.7.0.el8.x86_64.tgz
Debian 12 Offline Package	pigsty-pkg-v2.7.0.debian12.x86_64.tgz	pigsty-pkg-v2.7.0.debian12.x86_64.tgz
Ubuntu 22.04 Offline Package	pigsty-pkg-v2.7.0.ubuntu22.x86_64.tgz	pigsty-pkg-v2.7.0.ubuntu22.x86_64.tgz

You can also download them with curl:

VERSION=v2.6.0   # pigsty version
DISTRO=el8       # available distro: el7, el8, el9, debian11, debian12, ubuntu20, ubuntu22
curl "https://get.pigsty.cc/${VERSION}/pigsty-${VERSION}.tgz"                      -o ~/pigsty.tgz  # source tarball : ~/pigsty.tgz
curl "https://get.pigsty.cc/${VERSION}/pigsty-pkg-${VERSION}.${distro}.x86_64.tgz" -o /tmp/pkg.tgz  # offline package: /tmp/pkg.tgz

Then upload them to the isolated admin node, put the offline package at /tmp/pkg.tgz, then enable it th through the bootstrap procedure.

./bootstrap                 # you can install the pre-packed offline-package with '-y' flag, if you have Internet access
./configure; ./install.yml  # then continue with the standard configuration and installation tasks

What is offline installation?

Pigsty will download the required rpm/deb packages from the upstream yum/apt repo during the installation procedure and build a local software repo (located at /www/pigsty by default).

The local repo is served by Nginx and serves all nodes in this environment/deployment, including itself.

There are three main benefits to using a local repo:

It can avoid repetitive download requests and traffic consumption, significantly speeding up the installation and improving its reliability.
It will take a snapshot of available software versions, ensuring the consistency of the software versions installed across nodes in the deployment environment.
The built local software repo can be packaged as a whole tarball and copied to an isolated environment with the same operating system for offline installation.

The principle of offline installation is: first, complete the Standard Installation process on a node with the same operating system and internet access. Then, take a snapshot of the built local repo (/www/pigsty) and pack it to make the Offline Package, and then deliver it to the isolated environment for use.

When the Pigsty Installation procedure finds that the local repo already exists, It will enter offline install mode, which will install from the built local repo rather than upstream.

Pigsty will skip the process of downloading and building the local software source from the internet and complete the entire installation process using the local software repo without the need for internet access.

Criteria for Offline Install Mode

The criteria for the existence of a local repo is the presence of a marker file located by default at `/www/pigsty/repo_complete`.

This marker file is automatically generated after the download is complete during the standard installation procedure, indicating a usable local software repo is done.

Deleting the repo_complete marker file of the local repo will mark the procedure for re-download missing packages from upstream.

Offline Package

The offline package is a tarball made by gzip and tar, placed under /tmp/pkg.tgz, and extract to /www/pigsty for use.

Pigsty has pre-packed offline packages for primary supported 3 major OS distros.

Which can skip downloading & repo building if you are using the exact same operating system.

https://github.com/Vonng/pigsty/releases/download/v2.7.0/pigsty-pkg-v2.7.0.el8.x86_64.tgz       # EL 8.9 (Green Obsidian)
https://github.com/Vonng/pigsty/releases/download/v2.7.0/pigsty-pkg-v2.7.0.debian12.x86_64.tgz  # Debian 12    (bookworm)
https://github.com/Vonng/pigsty/releases/download/v2.7.0/pigsty-pkg-v2.7.0.ubuntu22.x86_64.tgz  # Ubuntu 22.04 (jammy)

https://get.pigsty.cc/v2.7.0/pigsty-pkg-v2.7.0.el8.x86_64.tgz      # (Green Obsidian)
https://get.pigsty.cc/v2.7.0/pigsty-pkg-v2.7.0.debian12.x86_64.tgz # (bookworm, 12.4)
https://get.pigsty.cc/v2.7.0/pigsty-pkg-v2.7.0.ubuntu22.x86_64.tgz # (jammy, 22.04.3)

Minor Version Matters!

When installing with offline packages, please ensure that the OS major version matches exactly, and try to keep the system minor version as close as possible.

Offline installation can usually succeed even if the OS minor version does not match, but there’s a chance that some packages might have missing or conflicting dependencies.

In such cases, please make your own offline package, or refer to the Frequently Asked Questions for solutions.

Bootstrap

Pigsty needs ansible to run the playbooks. You have to install Ansible even without Internet access.

Luckily, ansible and its dependencies are already included in Pigsty’s offline package. The process of extracting and installing ansible from the offline package is known as Bootstrap.

You have to download the offline package and place it under /tmp/pkg.tgz, then run the bootstrap command:

./bootstrap       # extract /tmp/pkg.tgz and install ansible

The bootstrap script will extract /tmp/pkg.tgz into /www/pigsty, setup a local file repo, and install ansible from it.

Bootstrap Procedure Logic

Check preconditions
Check local repo exists ?
- Y -> Extract to /www/pigsty and create repo file to enable it
- N -> Download offline package from the Internet? (if applicable)
  - Y -> Download from GitHub / CDN and extract & enable it
  - N -> Add basic os upstream repo file manually ?
    - Y -> add according to region / version
    - N -> leave it to user’s default configuration

Now we have an available repo for installing Ansible
- Precedence: local pkg.tgz > downloaded pkg.tgz > upstream > user provide

install boot utils from the available repo
- el7,8,9: ansible createrepo_c unzip wget yum-utils sshpass
- el8 extra: ansible python3.11-jmespath python3-cryptography createrepo_c unzip wget dnf-utils sshpass modulemd-tools
- el9 extra: ansible python3-jmespath python3.11-jmespath createrepo_c unzip wget dnf-utils sshpass modulemd-tools
- ubuntu/debian: ansible python3-jmespath dpkg-dev unzip wget sshpass acl
Check ansible availability.

Example: Download offline package from the Internet (EL8)

On a RockyLinux 8.9 node with the internet access, during the ./bootstrap process, if local repo /www/pigsty or offline package /tmp/pkg.tgz nor available, the system will prompt the user to download the offline package. Simply reply y to proceed, or you can use bootstrap -y to automatically reply “yes” by default.

[vagrant@build-el8 pigsty]$ ./bootstrap
bootstrap pigsty v2.7.0 begin
[ OK ] region = china
[ OK ] kernel = Linux
[ OK ] machine = x86_64
[ OK ] package = rpm,dnf
[ OK ] vendor = rocky (Rocky Linux)
[ OK ] version = 8 (8.9)
[ OK ] sudo = vagrant ok
[ OK ] EL 8.9 has pre-packed offline package available:
       https://get.pigsty.cc/v2.7.0/pigsty-pkg-v2.7.0.el8.x86_64.tgz
[ IN ] offline package not exist on /tmp/pkg.tgz, download? (y/n):
=> y
[ OK ] $ curl https://get.pigsty.cc/v2.7.0/pigsty-pkg-v2.7.0.el8.x86_64.tgz -o /tmp/pkg.tgz

... (yum install progress)

[ OK ] repo = extract from /tmp/pkg.tgz
[ OK ] repo file = use /etc/yum.repos.d/pigsty-local.repo
[WARN] rpm cache = updating, make take a while
[ OK ] repo cache = created
[ OK ] install el8 utils

...(yum install output)

Installed:
  createrepo_c-0.17.7-6.el8.x86_64  createrepo_c-libs-0.17.7-6.el8.x86_64 drpm-0.4.1-3.el8.x86_64 modulemd-tools-0.7-8.el8.noarch python3-createrepo_c-0.17.7-6.el8.x86_64 python3-libmodulemd-2.13.0-1.el8.x86_64
  python3-pyyaml-3.12-12.el8.x86_64 sshpass-1.09-4.el8.x86_64             unzip-6.0-46.el8.x86_64

...(yum install output)                                                                                                                18/18

Installed:
  ansible-8.3.0-1.el8.noarch                      ansible-core-2.15.3-1.el8.x86_64                          git-core-2.39.3-1.el8_8.x86_64                        mpdecimal-2.5.1-3.el8.x86_64
  python3-cffi-1.11.5-6.el8.x86_64                python3-cryptography-3.2.1-7.el8_9.x86_64                 python3-jmespath-0.9.0-11.el8.noarch                  python3-pycparser-2.14-14.el8.noarch
  python3.11-3.11.5-1.el8_9.x86_64                python3.11-cffi-1.15.1-1.el8.x86_64                       python3.11-cryptography-37.0.2-5.el8.x86_64           python3.11-jmespath-1.0.1-1.el8.noarch
  python3.11-libs-3.11.5-1.el8_9.x86_64           python3.11-pip-wheel-22.3.1-4.el8_9.1.noarch              python3.11-ply-3.11-1.el8.noarch                      python3.11-pycparser-2.20-1.el8.noarch
  python3.11-pyyaml-6.0-1.el8.x86_64              python3.11-setuptools-wheel-65.5.1-2.el8.noarch

Complete!
[ OK ] ansible = ansible [core 2.15.3]
[ OK ] boostrap pigsty complete
proceed with ./configure

Example: Bootstrap from local downloaded offline package (Debian 12)

A Debian 12 node without the Internet access, the user has pre-downloaded offline package and uploaded it to a specified path on the node.

If the offline package exists at /tmp/pkg.tgz, the bootstrap process will use it directly, and the output will be like:

vagrant@debian12:~/pigsty$ ./bootstrap
bootstrap pigsty v2.7.0 begin
[ OK ] region = china
[ OK ] kernel = Linux
[ OK ] machine = x86_64
[ OK ] package = deb,apt
[ OK ] vendor = debian (Debian GNU/Linux)
[ OK ] version = 12 (12)
[ OK ] sudo = vagrant ok
[ OK ] cache = /tmp/pkg.tgz exists
[ OK ] repo = extract from /tmp/pkg.tgz
[ OK ] repo file = use /etc/apt/sources.list.d/pigsty-local.list
[WARN] apt cache = updating, make take a while

...(apt install output)

[ OK ] ansible = ansible [core 2.14.3]
[ OK ] boostrap pigsty complete
proceed with ./configure

Example: Bootstrap from the Internet (Ubuntu 20.04)

在一台有互联网访问的 Ubuntu 20.04 节点上，因为 Pigsty 官方没有提供离线软件包，因此选择在线安装。在这种情况下，Bootstrap 会使用可用的源，使用 yum/apt 安装 ansible 与依赖：

On an Ubuntu 20.04 node with internet access, since Pigsty does not provide an official offline package for this OS major version, the online installation is performed. In this case, the bootstrap process will use the available upstream repo to install ansible and its dependencies using yum or apt:

vagrant@ubuntu20:~/pigsty$ ./bootstrap
bootstrap pigsty v2.7.0 begin
[ OK ] region = china
[ OK ] kernel = Linux
[ OK ] machine = x86_64
[ OK ] package = deb,apt
[ OK ] vendor = ubuntu (Ubuntu)
[ OK ] version = 20 (20.04)
[ OK ] sudo = vagrant ok
[WARN] ubuntu 20 focal does not have corresponding offline package, use online install
[WARN] cache = missing and skip download
[WARN] repo = skip (/tmp/pkg.tgz not exists)
[ OK ] repo file = add ubuntu focal china upstream
[WARN] apt cache = updating, make take a while

...(apt update/install output)

[ OK ] ansible = ansible 2.9.6
[ OK ] boostrap pigsty complete
proceed with ./configure

Make Offline Package

Pigsty has a bin/cache script to make offline packages.

Run this script on installed nodes; it will compress the /www/pigsty repo dir into /tmp/pkg.tgz tarball.

bin/cache [version=v2.6.0]
          [pkg_path=/tmp/pkg.tgz]
          [repo_dir=/www/pigsty]

You can upload that /tmp/pkg.tgz to isolated production nodes for using with the same bootstrap procedure.

sudo rm -rf /www/pigsty            # remove obsolete files
sudo tar -xf /tmp/pkg.tgz -C /www  # extract to nginx dir

Make Offline Package Example Output

Example: make an offline package on freshly installed Ubuntu 22.04 with bin/cache:

vagrant@ubuntu22:~/pigsty$ bin/cache
[ OK ] create offline package on ubuntu22
[ OK ] pkg type = deb
[ OK ] repo dir = /www/pigsty
[ OK ] copy /www/pigsty to /tmp/pigsty-build/pigsty
[ OK ] grafana plugins = found, overwrite /tmp/pigsty-build/pigsty/plugins.tgz
knightss27-weathermap-panel  marcusolsson-dynamictext-panel	marcusolsson-json-datasource	marcusolsson-treemap-panel  volkovlabs-form-panel	 volkovlabs-image-panel
marcusolsson-calendar-panel  marcusolsson-hourly-heatmap-panel	marcusolsson-static-datasource	volkovlabs-echarts-panel    volkovlabs-grapi-datasource  volkovlabs-variable-panel
[ OK ] ubuntu 22 = no packages needs to be cleansed
[ OK ] package = making pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz

pigsty/
pigsty/liblua5.2-0_5.2.4-2_amd64.deb
.................. # Lot's of packages
pigsty/distro-info-data_0.52ubuntu0.6_all.deb

[ OK ] package = finish pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz 48f9cb2dd2cabb61b115bfe2ac9e002d
-rw-rw-r-- 1 vagrant vagrant 1.2G Feb 29 03:09 /tmp/pkg.tgz
scp ubuntu22:/tmp/pkg.tgz v2.6.0/pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.

Last modified 2024-05-16: update en/zh setup/install docs (ba224e3)