Offline Install
Module:
Pigsty’s Standard Installation process requires internet access, but production database servers are often isolated from the internet.
Therefore, Pigsty offers an offline installation feature, allowing you to complete the installation and deployment in an environment without internet access.
If you have internet access, downloading the pre-made Offline Package in advance can help speed up the installation process and enhance the certainty and reliability of the installation.
Short Version
You have to download the Pigsty source tarball and the Offline Package in addition to the Standard Installation.
VERSION=v2.6.0 # pigsty version
DISTRO=el8 # available distro: el7, el8, el9, debian11, debian12, ubuntu20, ubuntu22
curl "https://get.pigsty.cc/${VERSION}/pigsty-${VERSION}.tgz" -o ~/pigsty.tgz # source tarball : ~/pigsty.tgz
curl "https://get.pigsty.cc/${VERSION}/pigsty-pkg-${VERSION}.${distro}.x86_64.tgz" -o /tmp/pkg.tgz # offline package: /tmp/pkg.tgz
Then upload them to the isolated admin node, put the offline package at /tmp/pkg.tgz, then enable it th through the bootstrap procedure.
./bootstrap # you can install the pre-packed offline-package with '-y' flag, if you have Internet access
./configure; ./install.yml # then continue with the standard configuration and installation tasks
What is offline install?
Pigsty will download the required rpm/deb packages from the upstream yum/apt repo during the installation procedure, and build a local software repo (located at /www/pigsty by default).
The local repo is served by Nginx, and serves all nodes in this environment / deployment including itself.
There are three main benefits to using a local repo:
- It can avoid repetitive download requests and traffic consumption, significantly speeding up the installation and improving its reliability.
- It will take a snapshot of available software versions, ensuring the consistency of the software versions installed across nodes in the deployment environment.
- The built local software repo can be packaged as a whole tarball and copied to an isolated environment with the same operating system for offline installation.
The principle of offline installation is: first, complete the Standard Installation process on a node with the same operating system and internet access.
Then, take a snapshot of the built local repo (/www/pigsty) and pack it to make the Offline Package, then deliver it to the isolated environment for using.
When Pigsty Installation procedure finds that the local repo already exists, It will enter offline install mode, which will install from the built local repo rather than upstream.
Pigsty will skip the process of downloading and building the local software source from the internet and complete the entire installation process using the local software source, without the need for internet access.
Criteria for Offline Install Mode
The criteria for the existence of a local repo is: the presence of a marker file located by default at `/www/pigsty/repo_complete`.
This marker file is automatically generated after the download is complete during the standard installation procedure, indicating a usable local software repo is done.
Deleting the repo_complete marker file of the local repo will mark the procedure to re-download missing packages from upstream.
Offline Package
Offline package is a tarball made by gzip and tar, placed under /tmp/pkg.tgz and extract to /www/pigsty for using.
Pigsty has pre-packed offline packages for compatible OS distros.
You can skip downloading & repo building if you are using the exact same operating system.
https://github.com/Vonng/pigsty/releases/download/v2.6.0/pigsty-pkg-v2.6.0.el7.x86_64.tgz # EL 7.9
https://github.com/Vonng/pigsty/releases/download/v2.6.0/pigsty-pkg-v2.6.0.el8.x86_64.tgz # EL 8.9
https://github.com/Vonng/pigsty/releases/download/v2.6.0/pigsty-pkg-v2.6.0.el9.x86_64.tgz # EL 9.3
https://github.com/Vonng/pigsty/releases/download/v2.6.0/pigsty-pkg-v2.6.0.debian11.x86_64.tgz # Debian 11.8
https://github.com/Vonng/pigsty/releases/download/v2.6.0/pigsty-pkg-v2.6.0.debian12.x86_64.tgz # Debian 12.4
https://github.com/Vonng/pigsty/releases/download/v2.6.0/pigsty-pkg-v2.6.0.ubuntu20.x86_64.tgz # Ubuntu 20.04.6
https://github.com/Vonng/pigsty/releases/download/v2.6.0/pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz # Ubuntu 22.04.3https://get.pigsty.cc/${VERSION}/pigsty-pkg-v2.6.0.el7.x86_64.tgz # EL 7.9
https://get.pigsty.cc/${VERSION}/pigsty-pkg-v2.6.0.el8.x86_64.tgz # EL 8.9
https://get.pigsty.cc/${VERSION}/pigsty-pkg-v2.6.0.el9.x86_64.tgz # EL 9.3
https://get.pigsty.cc/${VERSION}/pigsty-pkg-v2.6.0.debian11.x86_64.tgz # Debian 11.8
https://get.pigsty.cc/${VERSION}/pigsty-pkg-v2.6.0.debian12.x86_64.tgz # Debian 12.4
https://get.pigsty.cc/${VERSION}/pigsty-pkg-v2.6.0.ubuntu20.x86_64.tgz # Ubuntu 20.04.6
https://get.pigsty.cc/${VERSION}/pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz # Ubuntu 22.04.3Minor Version Matters!
When installing with offline packages, please ensure that the OS major version matches exactly, and try to keep the system minor version as close as possible.
Offline installation can usually succeed even if the OS minor version does not match, but there’s a chance that some packages might have missing or conflicting dependencies.
In such cases, please make your own offline package, or refer to the Frequently Asked Questions for solutions.
Bootstrap
Pigsty need ansible to run the install.yml playbook. You have to install ansible without Internet access, too.
Luckily, ansible and its dependencies is already included in Pigsty’s offline package. The process of extracting and installing ansible from the offline package is known as Bootstrap.
Ansible is a core dependency of Pigsty, and the first challenge of an offline installation is to figure out how to install Ansible without internet access or a local source.
Bootstrap Logci
-
Check preconditions
-
Check local repo exists ?
- Y -> Extract to
/www/pigstyand create repo file to enable it - N -> Download offline package from the Internet?
- Y -> Download from GitHub / CDN and extract & enable it
- N -> Add basic os upstream repo file manually ?
- Y -> add according to region / version
- N -> leave it to user’s default configuration
- Y -> Extract to
- Now we have an available repo for installing ansible
- Precedence: local
pkg.tgz> downloadedpkg.tgz> upstream > user provide
- Precedence: local
- install boot utils from the available repo
- el7,8,9:
ansible createrepo_c unzip wget yum-utils sshpass - el8 extra:
ansible python3.11-jmespath createrepo_c unzip wget dnf-utils sshpass modulemd-tools - el9 extra:
ansible python3-jmespath python3.11-jmespath createrepo_c unzip wget dnf-utils sshpass modulemd-tools - ubuntu/debian:
ansible python3-jmespath dpkg-dev unzip wget sshpass acl
- el7,8,9:
- Check
ansibleavailability.
The bootstrap script will extract /tmp/pkg.tgz into /www/pigsty, setup a local file repo, and install ansible from it.
You have to download the offline package and place it under /tmp/pkg.tgz, then run the bootstrap command:
./bootstrap # extract /tmp/pkg.tgz and install ansible
bootstrap from local packages output
If /tmp/pkg.tgz already exists, bootstrap will use it directly:
bootstrap pigsty v2.6.0 begin
[ OK ] region = china
[ OK ] kernel = Linux
[ OK ] machine = x86_64
[ OK ] release = 7.9.2009
[ OK ] sudo = vagrant ok
[ OK ] cache = /tmp/pkg.tgz exists
[ OK ] repo = extract from /tmp/pkg.tgz
[ OK ] repo file = use /etc/yum.repos.d/pigsty-local.repo
[ OK ] repo cache = created
[ OK ] install el7 utils
....(yum install ansible output)
[ OK ] ansible = ansible 2.9.27
[ OK ] boostrap pigsty complete
proceed with ./configure
If you have Internet access, you can use bootstrap variant:
./bootstrap # if /tmp/pkg.tgz exists, adding upstream repo & mirrors, then try to install ansible
./bootstrap -y # if /tmp/pkg.tgz not exists, download the pre-packed offline-package from CDN / Github
Bootstrap from Internet: Example Output
bootstrap pigsty v2.6.0 begin
[ OK ] region = china
[ OK ] kernel = Linux
[ OK ] machine = x86_64
[ OK ] release = 7.9.2009
[ OK ] sudo = vagrant ok
[ IN ] Cache /tmp/pkg.tgz not exists, download? (y/n):
=> y
[ OK ] download from Github https://get.pigsty.cc/v2.6.0/pigsty-pkg-v2.6.0.el7.x86_64.tgz to /tmp/pkg.tgz
....
[ OK ] repo = extract from /tmp/pkg.tgz
[ OK ] repo file = use /etc/yum.repos.d/pigsty-local.repo
[ OK ] repo cache = created
[ OK ] install el7 utils
...... (yum install createrepo_c sshpass unzip output)
==================================================================================================================
Package Arch Version Repository Size
==================================================================================================================
Installing:
createrepo_c x86_64 0.10.0-20.el7 pigsty-local 65 k
sshpass x86_64 1.06-2.el7 pigsty-local 21 k
unzip x86_64 6.0-24.el7_9 pigsty-local 172 k
Installing for dependencies:
createrepo_c-libs x86_64 0.10.0-20.el7 pigsty-local 89 k
Transaction Summary
==================================================================================================================
...... (yum install ansible output)
==================================================================================================================
Package Arch Version Repository Size
==================================================================================================================
Installing:
ansible noarch 2.9.27-1.el7 pigsty-local 17 M
Installing for dependencies:
PyYAML x86_64 3.10-11.el7 pigsty-local 153 k
libyaml x86_64 0.1.4-11.el7_0 pigsty-local 55 k
python-babel noarch 0.9.6-8.el7 pigsty-local 1.4 M
python-backports x86_64 1.0-8.el7 pigsty-local 5.8 k
python-backports-ssl_match_hostname noarch 3.5.0.1-1.el7 pigsty-local 13 k
python-cffi x86_64 1.6.0-5.el7 pigsty-local 218 k
python-enum34 noarch 1.0.4-1.el7 pigsty-local 52 k
python-idna noarch 2.4-1.el7 pigsty-local 94 k
python-ipaddress noarch 1.0.16-2.el7 pigsty-local 34 k
python-jinja2 noarch 2.7.2-4.el7 pigsty-local 519 k
python-markupsafe x86_64 0.11-10.el7 pigsty-local 25 k
python-paramiko noarch 2.1.1-9.el7 pigsty-local 269 k
python-ply noarch 3.4-11.el7 pigsty-local 123 k
python-pycparser noarch 2.14-1.el7 pigsty-local 104 k
python-setuptools noarch 0.9.8-7.el7 pigsty-local 397 k
python-six noarch 1.9.0-2.el7 pigsty-local 29 k
python2-cryptography x86_64 1.7.2-2.el7 pigsty-local 502 k
python2-httplib2 noarch 0.18.1-3.el7 pigsty-local 125 k
python2-jmespath noarch 0.9.4-2.el7 pigsty-local 41 k
python2-pyasn1 noarch 0.1.9-7.el7 pigsty-local 100 k
Transaction Summary
==================================================================================================================
...
Complete!
[ OK ] ansible = ansible 2.9.27
[ OK ] boostrap pigsty complete
proceed with ./configure
Make Offline Package
Pigsty has a bin/cache script to make offline packages.
Run this script on installed nodes, it will compress the /www/pigsty repo dir into /tmp/pkg.tgz tarball.
bin/cache [version=v2.6.0]
[pkg_path=/tmp/pkg.tgz]
[repo_dir=/www/pigsty]
You can copy that /tmp/pkg.tgz to isolated production nodes for using with the same bootstrap procedure.
sudo rm -rf /www/pigsty # remove obsolete files
sudo tar -xf /tmp/pkg.tgz -C /www # extract to nginx dir
Make Offline Package Example Output
Example: make a offline package on fresh installed Ubuntu 22.04 with bin/cache:
vagrant@ubuntu22:~/pigsty$ bin/cache
[ OK ] create offline package on ubuntu22
[ OK ] pkg type = deb
[ OK ] repo dir = /www/pigsty
[ OK ] copy /www/pigsty to /tmp/pigsty-build/pigsty
[ OK ] grafana plugins = found, overwrite /tmp/pigsty-build/pigsty/plugins.tgz
knightss27-weathermap-panel marcusolsson-dynamictext-panel marcusolsson-json-datasource marcusolsson-treemap-panel volkovlabs-form-panel volkovlabs-image-panel
marcusolsson-calendar-panel marcusolsson-hourly-heatmap-panel marcusolsson-static-datasource volkovlabs-echarts-panel volkovlabs-grapi-datasource volkovlabs-variable-panel
[ OK ] ubuntu 22 = no packages needs to be cleansed
[ OK ] package = making pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz
pigsty/
pigsty/liblua5.2-0_5.2.4-2_amd64.deb
.................. # Lot's of packages
pigsty/distro-info-data_0.52ubuntu0.6_all.deb
[ OK ] package = finish pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz 48f9cb2dd2cabb61b115bfe2ac9e002d
-rw-rw-r-- 1 vagrant vagrant 1.2G Feb 29 03:09 /tmp/pkg.tgz
scp ubuntu22:/tmp/pkg.tgz v2.6.0/pigsty-pkg-v2.6.0.ubuntu22.x86_64.tgz
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.