使用juicefs 让s3 更好兼容posix协议
juicefs 是一个很不错的云原生高性能共享文件系统,以下是s3的测试
环境准备
- docker-compose
version: "3"
services:
s3:
image: minio/minio
environment:
- "MINIO_ACCESS_KEY=minio"
- "MINIO_SECRET_KEY=minio123"
command: server /data --console-address ":9001"
ports:
- "9000:9000"
- "9001:9001"
- juicefs mac 客户端
可以参考官方资料
https://www.cnblogs.com/rongfengliang/p/15734513.html
使用
测试多客户端挂载文件通知
- format
juicefs format \
--storage minio \
--bucket http://localhost:9000/appdemo \
--bucket http://localhost:9000/appdemo2 \
--bucket http://localhost:9000/appdemo4 \
--access-key minio \
--secret-key minio123 \
sqlite3://myjfs.db \
miniofs
- mount
juicefs mount sqlite3://myjfs.db miniofs
juicefs mount sqlite3://myjfs.db miniofs2
- nodejs watch 文件
基于chokidar
const chokidar = require('chokidar');
const watcher = chokidar.watch(["/Users/dalong/mylearning/juicefs-rongfl/miniofs","/Users/dalong/mylearning/juicefs-rongfl/miniofs2"], {
ignored: /(^|[\/\\\\])\../, // ignore dotfiles
persistent: true
});
// Something to use when events are received.
const log = console.log.bind(console);
// More possible events.
watcher
.on('addDir', path => log(`Directory ${path} has been added`))
.on('unlinkDir', path => log(`Directory ${path} has been removed`))
.on('error', error => log(`Watcher error: ${error}`))
.on('ready', () => log('Initial scan complete. Ready for changes'))
.on('raw', (event, path, details) => { // internal
log('Raw event info:', event, path, details);
});
- 效果
说明
以上是基于juicefs 可以很明显的提升s3 挂载的性能,而且可以更好的兼容posix(尤其是进行多端挂载进行文件处理的时候),大家如果在测试
s3fs-fuse 的时候是不能达到多端挂载实时数据变动通知的(可以参考Limitations章节)
参考资料
https://www.npmjs.com/package/chokidar
https://juicefs.com/docs/zh/community/getting-started/for_distributed
https://github.com/s3fs-fuse/s3fs-fuse