FastDFS集群部署
以下我们在Centos7操作系统上以Fastdfs5.12为例来进行一下安装
环境准备
- fastdfs5.12
- libfastcommon-zip1.0.39
- fastdfs-nginx-module.1.16
- nginx1.10.0
- ngx_cache_purge2.3
- 可用的yum源
- root角色
- 检测是否开起了防火墙，如果开启则需要添加端口策略，如果没有开启则不需要
角色分配
注意：此处角色分配并不是必须按照以下模式进行分配，还需结合自身实际服务器资源情况来进行分配，同一机器上也可以部署多个角色
| IP地址 | 角色 | 目录分配 |
|---|---|---|
| 172.45.1.122 | tracker，client，libfastcommon | /data/fastdfs/tracker |
| 172.45.1.123 | tracker，client，libfastcommon | /data/fastdfs/tracker |
| 172.45.1.124 | storage，nginx，libfastcommon，fastdfs-nginx-module | /data/fastdfs/ |
| 172.45.1.125 | storage，nginx，libfastcommon，fastdfs-nginx-module | /data/fastdfs/ |
开始安装
- 安装包上传
- tracker节点：172.45.1.122，172.45.1.123
上传
fastdfs5.12,libfastcommon-zip1.0.39至 /opt/app/packages 目录下
- storage节点：172.45.1.124，172.45.1.125
上传
fastdfs5.12,libfastcommon-zip1.0.39,fastdfs-nginx-module.1.16,nginx1.10.0,ngx_cache_purge2.3至 /opt/app/packages 目录下
- 安装nginx和fastdfs依赖环境
# 安装nginx环境依赖
yum install gcc gcc-c++ make automake autoconf libtool pcre* zlib openssl openssl-devel pcre-devel zlib-devel
# 安装fastdfs环境依赖
yum -y install make cmake gcc gcc-c++ uzip zip
- 安装libfatscommon
节点：172.45.1.122，172.45.1.123，172.45.1.124，172.45.1.125
- 进入到/opt/app/packages目录下执行以下命令
# 执行解压命令
tar -zxvf libfastcommon-1.0.39.tar.gz -C /opt/app
# 进入到解压后/opt/app/libfastcommon-1.0.39，执行编译脚本
./make.sh
# 执行编译命令
./make.sh install
- 安装fastdfs
节点：172.45.1.122，172.45.1.123，172.45.1.124，172.45.1.125
- 进入到/opt/app/packages目录下执行以下命令
# 执行解压命令
tar -zxvf fastdfs-5.11.tar.gz -C /opt/app
# 进入到解压后/opt/app/fastdfs-5.11，执行编译脚本
./make.sh
# 执行编译命令
./make.sh install
- 配置tracker并启动服务
节点：172.45.1.122，172.45.1.123
- 执行以下命令
# 复制tracker配置文件模板
cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf
# 创建tracker基础目录
mkdir /data/fastdfs/tracker
- 修改/etc/fdfs/tracker.conf配置文件
base_path=/data/fastdfs/tracker
store_group=group1
其他参数保留默认配置即可，关于其他参数含义可以参考http://bbs.chinaunix.net/thread-1941456-1-1.html 链接
- 启动tracker服务
/etc/init.d/fdfs_trackerd start
- 配置storage并启动服务
节点：172.45.1.124，172.45.1.125
- 执行以下命令
# 复制storage配置文件模板
cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf
# 创建storage基础目录
mkdir /data/fastdfs/storage
- 修改/etc/fdfs/storage.conf配置文件
base_path=/data/fastdfs/storage
store_path0=/data/fastdfs/storage
tracker_server=172.45.1.122:22122
tracker_server=172.45.1.123:22122
其他参数保留默认配置即可，关于其他参数含义可以参考http://bbs.chinaunix.net/thread-1941456-1-1.html 链接
- 启动storage服务
/etc/init.d/fdfs_storaged start
- 配置client客户端(主要用来通过命令行查看集群信息)
节点：172.45.1.122，172.45.1.123
- 执行以下命令
# 复制client配置文件模板
cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf
- 修改/etc/fdfs/client.conf配置文件
base_path=/data/fastdfs/tracker
tracker_server=172.45.1.122:22122
tracker_server=172.45.1.123:22122
通过
fdfs_monitor /etc/fdfs/client.conf命令可以查看到一下结果：
[root@localhost packages]# fdfs_monitor /etc/fdfs/client.conf
[2020-08-12 10:44:37] DEBUG - base_path=/data/fastdfs/tracker, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
server_count=2, server_index=1
tracker server is 172.45.1.123:22122
group count: 1
Group 1:
group name = group1
disk total space = 51175 MB
disk free space = 47005 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 0
current trunk file id = 0
Storage 1:
id = 172.45.1.124
ip_addr = 172.45.1.124 ACTIVE
http domain =
version = 5.11
join time = 2020-08-12 08:34:35
up time = 2020-08-12 08:34:35
total storage = 51175 MB
free storage = 47005 MB
upload priority = 10
store_path_count = 1
subdir_count_per_path = 256
storage_port = 23000
storage_http_port = 8888
current_write_path = 0
source storage id =
if_trunk_server = 0
connection.alloc_count = 256
connection.current_count = 1
connection.max_count = 1
total_upload_count = 0
success_upload_count = 0
total_append_count = 0
success_append_count = 0
total_modify_count = 0
success_modify_count = 0
total_truncate_count = 0
success_truncate_count = 0
total_set_meta_count = 0
success_set_meta_count = 0
total_delete_count = 0
success_delete_count = 0
total_download_count = 0
success_download_count = 0
total_get_meta_count = 0
success_get_meta_count = 0
total_create_link_count = 0
success_create_link_count = 0
total_delete_link_count = 0
success_delete_link_count = 0
total_upload_bytes = 0
success_upload_bytes = 0
total_append_bytes = 0
success_append_bytes = 0
total_modify_bytes = 0
success_modify_bytes = 0
stotal_download_bytes = 0
success_download_bytes = 0
total_sync_in_bytes = 0
success_sync_in_bytes = 0
total_sync_out_bytes = 0
success_sync_out_bytes = 0
total_file_open_count = 0
success_file_open_count = 0
total_file_read_count = 0
success_file_read_count = 0
total_file_write_count = 0
success_file_write_count = 0
last_heart_beat_time = 2020-08-12 10:44:35
last_source_update = 1970-01-01 08:00:00
last_sync_update = 1970-01-01 08:00:00
last_synced_timestamp = 1970-01-01 08:00:00
Storage 2:
id = 172.45.1.125
ip_addr = 172.45.1.125 ACTIVE
http domain =
version = 5.11
join time = 2020-08-12 08:34:39
up time = 2020-08-12 08:34:39
total storage = 51175 MB
free storage = 47005 MB
upload priority = 10
store_path_count = 1
subdir_count_per_path = 256
storage_port = 23000
storage_http_port = 8888
current_write_path = 0
source storage id = 172.45.1.124
if_trunk_server = 0
connection.alloc_count = 256
connection.current_count = 1
connection.max_count = 1
total_upload_count = 0
success_upload_count = 0
total_append_count = 0
success_append_count = 0
total_modify_count = 0
success_modify_count = 0
total_truncate_count = 0
success_truncate_count = 0
total_set_meta_count = 0
success_set_meta_count = 0
total_delete_count = 0
success_delete_count = 0
total_download_count = 0
success_download_count = 0
total_get_meta_count = 0
success_get_meta_count = 0
total_create_link_count = 0
success_create_link_count = 0
total_delete_link_count = 0
success_delete_link_count = 0
total_upload_bytes = 0
success_upload_bytes = 0
total_append_bytes = 0
success_append_bytes = 0
total_modify_bytes = 0
success_modify_bytes = 0
stotal_download_bytes = 0
success_download_bytes = 0
total_sync_in_bytes = 0
success_sync_in_bytes = 0
total_sync_out_bytes = 0
success_sync_out_bytes = 0
total_file_open_count = 0
success_file_open_count = 0
total_file_read_count = 0
success_file_read_count = 0
total_file_write_count = 0
success_file_write_count = 0
last_heart_beat_time = 2020-08-12 10:44:11
last_source_update = 1970-01-01 08:00:00
last_sync_update = 1970-01-01 08:00:00
last_synced_timestamp = 1970-01-01 08:00:00
- 配置nginx相关模块
节点：172.45.1.124，172.45.1.125
配置之前先说一下fastdfs-nginx-module作用：
FastDFS通过Tracker服务器,将文件放在Storage服务器存储,但是同组之间的服务器需要复制文件,有延迟的问题.假设Tracker服务器将文件上传到了172.45.1.124,文件ID已经返回客户端,这时,后台会将这个文件复制到172.45.1.125,如果复制没有完成,客户端就用这个ID在172.45.1.125取文件,肯定会出现错误。这个fastdfs-nginx-module可以重定向连接到源服务器取文件,避免客户端由于复制延迟的问题,出现错误。
-
配置fastdfs-nginx-module
- 执行以下命令
# 解压fastdfs-nginx-module安装包 tar -zxvf /opt/app/packages/fastdfs-nginx-module_v1.16.tar.gz -C /opt/app # 进入到/opt/app/fastdfs-nginx-module/src目录进行配置将 CORE_INCS="$CORE_INCS /usr/local/include/fastdfs /usr/local/include/fastcommon/" # 修改为 CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"- 拷贝配置模板并编辑/etc/fdfs/mod_fastdfs.conf
# 拷贝配置模板 cp /opt/app/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/# 编辑配置文件 base_path=/data/fastdfs/storage tracker_server=172.45.1.122:22122 tracker_server=172.45.1.123:22122 url_have_group_name=true group_name=group1 store_path_count=1 store_path0=/data/fastdfs/storage -
配置ngx_cache_purge-2.3
- 执行以下命令
# 解压fastdfs-nginx-module安装包 tar -zxvf /opt/app/packages/ngx_cache_purge-2.3.tar.gz -C /opt/app -
nginx安装与配置
- nginx安装
- 执行以下命令
# 解压nginx tar -zxvf /opt/app/packages/nginx-1.10.0.tar.gz -C /opt/app # 进入到nginx解压目录配置 ./configure --prefix=/opt/app/nginx --add-module=/opt/app/fastdfs-nginx-module/src --add-module=/opt/app/ngx_cache_purge-2.3 # 进入到/opt/app/nginx/sbin目录下编译和安装 make && make install - nginx配置
user nobody; worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; server { listen 8888; # 对外端口 server_name 172.45.1.124; # 当前nginx所在服务器IP地址，125则需要换行125即可 location ~/group1/M00 { ngx_fastdfs_module; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }!> 注意：
(1) 8888 端口值是要与/etc/fdfs/storage.conf 中的 http.server_port=8888 相对应, 因为 http.server_port 默认为 8888,如果想改成 80,则要对应修改过来。
(2) Storage 对应有多个 group 的情况下,访问路径带 group 名,如/group1/M00/00/00/xxx, 对应的 Nginx 配置为:location ~/group([0-9])/M00 { ngx_fastdfs_module; }(3) 如果通过8888下载时如发现老报 404,将 nginx.conf 第一行 user nobody 修改为 user root 后重新启动。
- nginx安装
- 复制FastDFS部分配置文件到/etc/fdfs目录下
节点：172.45.1.124，172.45.1.125
- 进入到/opt/app/fastdfs-5.11/conf目录下执行命令
cp http.conf mime.types /etc/fdfs/
- 创建数据链接
节点：172.45.1.124，172.45.1.125
ln -s /data/fastdfs/storage/data/ /data/fastdfs/storage/data/M00
- 启动nginx，进入到/opt/app/nginx/sbin目录下执行命令
./nginx
# 重启nginx服务
# ./nginx -s reload
- 到此，fastdfs的整个搭建过程就算是完了，接下来可以验证一下
验证fastdfs
命令行上传一张图片
通过nginx去访问图片
访问172.45.1.124 storage 服务器
访问172.45.1.125 storage 服务器
![WARNING|style:callout]
在安装过程中如果遇到问题可以先查看 常见问题解决
狭路相逢勇者胜！
