JuiceFS: S3FS & PGFS
How to build a distributed cloud-native filesystem JuiceFS using PostgreSQL and MinIO provided by Pigsty.
JuiceFS is a high-performance, cloud-native distributed filesystem.
This guide demonstrates how to build a production-grade JuiceFS cluster using PostgreSQL as the metadata engine and MinIO as the object storage engine, both provided by Pigsty.
Quick Start
Create a four-node sandbox using the full mode.
./configure -c full
./install.yml
Install JuiceFS and configure object storage:
JFSNAME=jfs
METAURL=postgres://dbuser_meta:[email protected]:5432/meta
DATAURL=(
--storage minio
--bucket https://sss.pigsty:9000/infra
--access-key minioadmin
--secret-key minioadmin
)
juicefs format "${DATAURL[@]}" ${METAURL} jfs # Format filesystem
juicefs mount ${METAURL} ~/jfs -d # Mount in background
juicefs umount ~/jfs # Unmount
For a more advanced approach: PGFS - Using database as filesystem
JFSNAME=jfs
METAURL=postgres://dbuser_meta:[email protected]:5432/meta
DATAURL=(
--storage postgres
--bucket 10.10.10.10:5432/meta
--access-key dbuser_meta
--secret-key DBUser.Meta
)
juicefs format "${DATAURL[@]}" ${METAURL} ${JFSNAME}
juicefs mount ${METAURL} ~/jfs -d # Mount in background
juicefs umount ~/jfs # Unmount
Standalone Mode
Pigsty Infra repository provides the latest JuiceFS RPM/DEB packages. Install directly using your package manager.
The following commands create a local JuiceFS filesystem using local SQLite and filesystem (/var/jfs):
juicefs format sqlite3:///tmp/jfs.db myjfs
Format Output
$ juicefs format sqlite3:///jfs.db myjfs
2025/03/19 12:07:56.956222 juicefs[62924] <INFO>: Meta address: sqlite3:///jfs.db [interface.go:504]
2025/03/19 12:07:56.958055 juicefs[62924] <INFO>: Data use file:///var/jfs/myjfs/ [format.go:484]
2025/03/19 12:07:56.966150 juicefs[62924] <INFO>: Volume is formatted as {
"Name": "myjfs",
"UUID": "1568ee2a-dc4c-4a0e-9788-be0490776dda",
"Storage": "file",
"Bucket": "/var/jfs/",
"BlockSize": 4096,
"Compression": "none",
"EncryptAlgo": "aes256gcm-rsa",
"TrashDays": 1,
"MetaVersion": 1,
"MinClientVersion": "1.1.0-A",
"DirStats": true,
"EnableACL": false
} [format.go:521]
Then mount locally using the following commands:
juicefs mount sqlite3:///tmp/jfs.db ~/jfs # Foreground mount, auto-unmounts on exit
juicefs mount sqlite3:///tmp/jfs.db ~/jfs -d # Daemon mount, requires manual unmount
juicefs umount ~/jfs # Unmount and exit process
Data Cleanup
Clear JuiceFS metadata from PostgreSQL using:
DROP TABLE IF EXISTS jfs_acl,jfs_chunk,jfs_chunk_ref,jfs_counter,jfs_delfile,jfs_delslices,jfs_detached_node,jfs_dir_quota,jfs_dir_stats,jfs_edge,jfs_flock,jfs_node,jfs_plock,jfs_session2,jfs_setting,jfs_sustained,jfs_symlink,jfs_xattr CASCADE;
Clear object storage bucket using:
mcli rm --recursive --force infra/jfs
PGFS Performance Summary
Benchmark results on second-hand physical hardware:
METAURL=postgres://dbuser_meta:DBUser.Meta@:5432/meta
OPTIONS=(
--storage postgres
--bucket :5432/meta
--access-key dbuser_meta
--secret-key DBUser.Meta
${METAURL}
jfs
)
juicefs format "${OPTIONS[@]}"
juicefs mount ${METAURL} ~/jfs -d # Mount in background
juicefs bench ~/jfs # Run benchmark
juicefs umount ~/jfs # Unmount
$ juicefs bench ~/jfs # Performance test
Write big blocks: 1024/1024 [==============================================================] 178.5/s used: 5.73782533s
Read big blocks: 1024/1024 [==============================================================] 31.7/s used: 32.314547037s
Write small blocks: 100/100 [==============================================================] 149.2/s used: 670.127171ms
Read small blocks: 100/100 [==============================================================] 543.4/s used: 184.109596ms
Stat small files: 100/100 [==============================================================] 1723.4/s used: 58.087752ms
Benchmark finished!
BlockSize: 1.0 MiB, BigFileSize: 1.0 GiB, SmallFileSize: 128 KiB, SmallFileCount: 100, NumThreads: 1
Time used: 42.2 s, CPU: 687.2%, Memory: 179.4 MiB
+------------------+------------------+---------------+
| ITEM | VALUE | COST |
+------------------+------------------+---------------+
| Write big file | 178.51 MiB/s | 5.74 s/file |
| Read big file | 31.69 MiB/s | 32.31 s/file |
| Write small file | 149.4 files/s | 6.70 ms/file |
| Read small file | 545.2 files/s | 1.83 ms/file |
| Stat file | 1749.7 files/s | 0.57 ms/file |
| FUSE operation | 17869 operations | 3.82 ms/op |
| Update meta | 1164 operations | 1.09 ms/op |
| Put object | 356 operations | 303.01 ms/op |
| Get object | 256 operations | 1072.82 ms/op |
| Delete object | 0 operations | 0.00 ms/op |
| Write into cache | 356 operations | 2.18 ms/op |
| Read from cache | 100 operations | 0.11 ms/op |
+------------------+------------------+---------------+
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.