FastDFS安装配置手册
FastDFS安装配置手册
目录
一、 安装
(一) 下载FastDFS安装包
FastDFS官方论坛：http://www.csource.org
下载1：http://sourceforge.net/projects/fastdfs/files/
下载2：https://code.google.com/p/fastdfs/downloads/list
本手册使用CentOS 6.5 x86_64版操作系统，按照以下网络结构进行部署：
所需要下载的压缩包有：
FastDFS源代码：FastDFS_v5.01.tar.gz
nginx模块源代码：fastdfs-nginx-module_v1.15.tar.gz
nginx服务器源代码：nginx-1.4.7.tar.gz
nginx cache purge插件源代码：ngx_cache_purge-2.1.tar.gz
nginx依赖的pcre库源代码：pcre-8.34.tar.gz
nginx依赖的zlib库源代码：zlib-1.2.8.tar.gz
(二) 安装tracker
1. 安装
首先在172.16.1.202上安装FastDFS tracker，使用FastDFS_v5.01.tar.gz源代码包。
可以直接从Linux系统下载源代码包并解压安装。如果是在Windows系统下载，可以在Linux系统中挂载Windows的共享目录并且将源代码包复制进Linux系统内。挂载命令如下
|
[root@centos-db01 ~]# mount -o ro //Windows主机的IP地址/共享目录名/挂载目标目录 |
l 首先将代码包复制到系统的/usr/local/src内（这一步可选），然后使用tar命令解压
|
[root@tracker opt]# cp FastDFS_v5.01.tar.gz /usr/local/src/ [root@tracker opt]# cd /usr/local/src/ [root@tracker src]# tar zxf FastDFS_v5.01.tar.gz [root@tracker src]# cd FastDFS [root@tracker FastDFS]# ll 总用量 128 drwxrwxr-x. 3 500 500 4096 2月 6 18:07 client drwxrwxr-x. 2 500 500 4096 2月 6 18:07 common drwxrwxr-x. 2 500 500 4096 2月 6 18:07 conf -rw-rw-r--. 1 500 500 35067 7月 28 2008 COPYING-3_0.txt -rw-rw-r--. 1 500 500 29691 2月 2 13:17 HISTORY drwxrwxr-x. 2 500 500 4096 2月 6 18:07 init.d -rw-rw-r--. 1 500 500 7639 1月 5 14:08 INSTALL -rwxrwxr-x. 1 500 500 5531 12月 7 15:19 make.sh drwxrwxr-x. 2 500 500 4096 2月 6 18:07 php_client -rw-rw-r--. 1 500 500 2380 7月 28 2008 README -rwxrwxr-x. 1 500 500 1768 4月 12 2010 restart.sh -rwxrwxr-x. 1 500 500 1680 4月 10 2010 stop.sh drwxrwxr-x. 4 500 500 4096 2月 6 18:07 storage drwxrwxr-x. 2 500 500 4096 2月 6 18:07 test drwxrwxr-x. 2 500 500 4096 2月 6 18:07 tracker |
l 运行make.sh，确认make成功。期间如果有错误，可能会是缺少依赖的软件包，需安装后再次make。
|
[root@tracker FastDFS]# ./make.sh |
l 运行make.shinstall，确认install成功。
|
[root@tracker FastDFS]# ./make.sh install |
安装完成后，所有可执行文件在/usr/local/bin下，以fdfs开头：
|
[root@tracker FastDFS]# ll /usr/local/bin/fdfs* -rwxr-xr-x. 1 root root 522918 3月 25 14:57 /usr/local/bin/fdfs_appender_test -rwxr-xr-x. 1 root root 522871 3月 25 14:57 /usr/local/bin/fdfs_appender_test1 -rwxr-xr-x. 1 root root 514023 3月 25 14:57 /usr/loal/bin/fdfs_append_file -rwxr-xr-x. 1 root root 513433 3月 25 14:57 /usr/local/bin/fdfs_crc32 -rwxr-xr-x. 1 root root 513967 3月 25 14:57 /usr/local/bin/fdfs_delete_file -rwxr-xr-x. 1 root root 514377 3月 25 14:57 /usr/local/bin/fdfs_download_file -rwxr-xr-x. 1 root root 514133 3月 25 14:57 /usr/local/bin/fdfs_file_info -rwxr-xr-x. 1 root root 525064 3月 25 14:57 /usr/local/bin/fdfs_monitor -rwxr-xr-x. 1 root root 1179682 3月 25 14:57 /usr/local/bin/fdfs_storaged -rwxr-xr-x. 1 root root 529845 3月 25 14:57 /usr/local/bin/fdfs_test -rwxr-xr-x. 1 root root 527774 3月 25 14:57 /usr/local/bin/fdfs_test1 -rwxr-xr-x. 1 root root 655809 3月 25 14:57 /usr/local/bin/fdfs_trackerd -rwxr-xr-x. 1 root root 514213 3月 25 14:57 /usr/local/bin/fdfs_upload_appender -rwxr-xr-x. 1 root root 514999 3月 25 14:57 /usr/local/bin/fdfs_upload_file |
所有配置文件在/etc/fdfs下：
|
[root@tracker FastDFS]# ll /etc/fdfs/ 总用量 60 -rw-r--r--. 1 root root 1461 3月 13 15:15 client.conf -rw-r--r--. 1 root root 858 3月 13 15:15 http.conf -rw-r--r--. 1 root root 31172 3月 13 15:15 mime.types -rw-r--r--. 1 root root 3837 3月 25 10:03 mod_fastdfs.conf -rw-r--r--. 1 root root 7515 3月 24 10:36 storage.conf -rw-r--r--. 1 root root 6989 3月 13 15:15 tracker.conf |
l 至此tracker安装完成。
2. 配置
l 编辑配置文件目录下的tracker.conf，设置相关信息并保存。
|
[root@tracker FastDFS]# vim /etc/fdfs/tracker.conf |
一般只需改动以下几个参数即可：
disabled=false #启用配置文件
port=22122 #设置tracker的端口号
base_path=/fdfs/tracker #设置tracker的数据文件和日志目录（需预先创建）
http.server_port=8080 #设置http端口号
如需要进行性能调优，可以参照附录的配置文件的详细说明。
3. 运行
l 运行tracker之前，先要把防火墙中对应的fd端口打开（本例中为22122）。
|
[root@tracker FastDFS]# iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 22122 -j ACCEPT [root@tracker FastDFS]# /etc/init.d/iptables save iptables：将防火墙规则保存到 /etc/sysconfig/iptables：[确定] |
l 启动tracker，确认启动是否成功。（查看是否对应端口22122是否开始监听）
|
[root@tracker FastDFS]#/usr/local/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart [root@tracker FastDFS]#netstat -unltp | grep fdfs tcp 0 0.0.0.0:22122 0.0.0.0:* LISTEN 1766/fdfs_trackerd |
也可查看tracker的日志是否启动成功或是否有错误。
|
[root@tracker FastDFS]#cat /fdfs/tracker/logs/trackerd.log |
l 设置开机自动启动。
|
[root@tracker FastDFS]# vim /etc/rc.d/rc.local |
将运行命令行添加进文件：/usr/local/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
(三) 安装storage
1. 安装
首先在172.16.1.203上安装FastDFS storage，使用FastDFS_v5.01.tar.gz源代码包。
l 首先将代码包复制到系统的/usr/local/src内（这一步可选），然后使用tar命令解压
|
[root@storage1opt]# cp FastDFS_v5.01.tar.gz /usr/local/src/ [root@storage1opt]# cd /usr/local/src/ [root@storage1src]# tar zxf FastDFS_v5.01.tar.gz [root@storage1src]# cd FastDFS [root@storage1FastDFS]# ll 总用量 128 drwxrwxr-x. 3 500 500 4096 2月 6 18:07 client drwxrwxr-x. 2 500 500 4096 2月 6 18:07 common drwxrwxr-x. 2 500 500 4096 2月 6 18:07 conf -rw-rw-r--. 1 500 500 35067 7月 28 2008 COPYING-3_0.txt -rw-rw-r--. 1 500 500 29691 2月 2 13:17 HISTORY drwxrwxr-x. 2 500 500 4096 2月 6 18:07 init.d -rw-rw-r--. 1 500 500 7639 1月 5 14:08 INSTALL -rwxrwxr-x. 1 500 500 5531 12月 7 15:19 make.sh drwxrwxr-x. 2 500 500 4096 2月 6 18:07 php_client -rw-rw-r--. 1 500 500 2380 7月 28 2008 README -rwxrwxr-x. 1 500 500 1768 4月 12 2010 restart.sh -rwxrwxr-x. 1 500 500 1680 4月 10 2010 stop.sh drwxrwxr-x. 4 500 500 4096 2月 6 18:07 storage drwxrwxr-x. 2 500 500 4096 2月 6 18:07 test drwxrwxr-x. 2 500 500 4096 2月 6 18:07 tracker |
l 运行make.sh，确认make成功。期间如果有错误，可能会是缺少依赖的软件包，需安装后再次make。
|
[root@storage1FastDFS]# ./make.sh |
l 运行make.shinstall，确认install成功。
|
[root@storage1FastDFS]# ./make.sh install |
安装完成后，所有可执行文件在/usr/local/bin下，以fdfs开头：
|
[root@storage1FastDFS]# ll /usr/local/bin/fdfs* -rwxr-xr-x. 1 root root 522918 3月 25 14:57 /usr/local/bin/fdfs_appender_test -rwxr-xr-x. 1 root root 522871 3月 25 14:57 /usr/local/bin/fdfs_appender_test1 -rwxr-xr-x. 1 root root 514023 3月 25 14:57 /usr/local/bin/fdfs_append_file -rwxr-xr-x. 1 root root 513433 3月 25 14:57 /usr/local/bin/fdfs_crc32 -rwxr-xr-x. 1 root root 513967 3月 25 14:57 /usr/local/bin/fdfs_delete_file -rwxr-xr-x. 1 root root 514377 3月 25 14:57 /usr/local/bin/fdfs_download_file -rwxr-xr-x. 1 root root 514133 3月 25 14:57 /usr/local/bin/fdfs_file_info -rwxr-xr-x. 1 root root 525064 3月 25 14:57 /usr/local/bin/fdfs_monitor -rwxr-xr-x. 1 root root 1179682 3月 25 14:57 /usr/local/bin/fdfs_storaged -rwxr-xr-x. 1 root root 529845 3月 25 14:57 /usr/local/bin/fdfs_test -rwxr-xr-x. 1 root root 527774 3月 25 14:57 /usr/local/bin/fdfs_test1 -rwxr-xr-x. 1 root root 655809 3月 25 14:57 /usr/local/bin/fdfs_trackerd -rwxr-xr-x. 1 root root 514213 3月 25 14:57 /usr/local/bin/fdfs_upload_appender -rwxr-xr-x. 1 root root 514999 3月 25 14:57 /usr/local/bin/fdfs_upload_file |
所有配置文件在/etc/fdfs下：
|
[root@storage1FastDFS]# ll /etc/fdfs/ 总用量 60 -rw-r--r--. 1 root root 1461 3月 13 15:15 client.conf -rw-r--r--. 1 root root 858 3月 13 15:15 http.conf -rw-r--r--. 1 root root 31172 3月 13 15:15 mime.types -rw-r--r--. 1 root root 3837 3月 25 10:03 mod_fastdfs.conf -rw-r--r--. 1 root root 7515 3月 24 10:36 storage.conf -rw-r--r--. 1 root root 6989 3月 13 15:15 tracker.conf |
l 至此storage安装完成。
2. 配置
l 编辑配置文件目录下的storage.conf，设置相关信息并保存。
|
[root@storage1FastDFS]# vim /etc/fdfs/storage.conf |
一般只需改动以下几个参数即可：
disabled=false #启用配置文件
group_name=group1#组名，根据实际情况修改
port=23000#设置storage的端口号
base_path=/fdfs/storage#设置storage的日志目录（需预先创建）
store_path_count=1#存储路径个数，需要和store_path个数匹配
store_path0=/fdfs/storage#存储路径
tracker_server=172.16.1.202:22122#tracker服务器的IP地址和端口号
http.server_port=8080 #设置http端口号
如需要进行性能调优，可以参照附录的配置文件的详细说明。
3. 运行
l 运行storage之前，先要把防火墙中对应的端口打开（本例中为23000）。
|
[root@storage1 FastDFS]# iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 23000 -j ACCEPT [root@storage1FastDFS]# /etc/init.d/iptables save iptables：将防火墙规则保存到 /etc/sysconfig/iptables：[确定] |
l 启动storage，会根据配置文件的设置自动创建多级存储目录，确认启动是否成功。（查看是否对应端口23000是否开始监听）
|
[root@storage1 FastDFS]#/usr/local/bin/fdfs_storaged /etc/fdfs/storage.conf restart data path: /fdfs/storage/data, mkdir sub dir... mkdir data path: 00 ... mkdir data path: 01 ... mkdir data path: 02 ... mkdir data path: 03 ... ... data path: /fdfs/storage/data, mkdir sub dir done. [root@storage1 FastDFS]#netstat -unltp | grep fdfs tcp 0 0.0.0.0:23000 0.0.0.0:* LISTEN 1766/fdfs_storaged |
也可查看storage的日志是否启动成功或是否有错误。
|
[root@storage1 FastDFS]#cat /fdfs/storage/logs/storaged.log |
确认启动成功后，可以运行fdfs_monitor查看storage服务器是否已经登记到tracker服务器。
|
[root@storage1 FastDFS]# /usr/local/bin/fdfs_monitor /etc/fdfs/storage.conf [2014-03-26 01:51:20] DEBUG - base_path=/fdfs/storage, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
server_count=1, server_index=0
tracker server is 172.16.1.202:22122
group count: 1
Group 1: group name = group1 disk total space = 27789 MB disk free space = 23920 MB trunk free space = 0 MB storage server count = 1 active server count = 1 storage server port = 23000 storage HTTP port = 8080 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0
Storage 1: id = 172.16.1.203 ip_addr = 172.16.1.203 ACTIVE http domain = version = 5.01 join time = 2014-03-26 01:48:19 up time = 2014-03-26 01:48:19 total storage = 27789 MB free storage = 23920 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8080 current_write_path = 0 source storage id= if_trunk_server= 0 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2014-03-26 01:51:03 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 |
看到“172.16.1.203 ACTIVE”即可确认storage运行正常。
l 设置开机自动启动。
|
[root@storage1 FastDFS]# vim /etc/rc.d/rc.local |
将运行命令行添加进文件：/usr/local/bin/fdfs_storaged /etc/fdfs/storage.conf restart
l 之后依次在172.16.1.204~208上全部安装上storage并确认运行正常。注意配置文件中group名参数需要根据实际情况调整，本例中group是这样分配的：
group1：172.16.1.203，172.16.1.204
group2：172.16.1.205，172.16.1.206
group3：172.16.1.207，172.16.1.208
另外每个group中所有storage的端口号必须一致。
(四) 在storage上安装nginx
在storage上安装的nginx主要为了提供http的访问服务，同时解决group中storage服务器的同步延迟问题。(需要安装ngx_cache_purge-2.1.tar.gz)
1. 安装
首先在172.16.1.203上安装nginx，使用nginx-1.4.7.tar.gz源代码包以及FastDFS的nginx插件fastdfs-nginx-module_v1.15.tar.gz。
l 首先将代码包和插件复制到系统的/usr/local/src内（可选），然后使用tar命令解压
|
[root@storage1opt]# cp nginx-1.4.7.tar.gz /usr/local/src [root@storage1opt]# cp fastdfs-nginx-module_v1.15.tar.gz /usr/local/src [root@storage1opt]# cp pcre-8.34.tar.gz /usr/local/src [root@storage1opt]# cp zlib-1.2.8.tar.gz /usr/local/src [root@storage1opt]# cd /usr/local/src/ [root@storage1src]# tar zxf nginx-1.4.7.tar.gz [root@storage1src]# tar zxf fastdfs-nginx-module_v1.15.tar.gz [root@storage1src]# tar zxf pcre-8.34.tar.gz [root@storage1src]# tar zxf zlib-1.2.8.tar.gz [root@storage1src]# cd nginx-1.4.7 [root@storage1nginx-1.4.7]# ll 总用量 592 drwxr-xr-x. 6 1001 1001 4096 3月 26 02:34 auto -rw-r--r--. 1 1001 1001 225213 3月 18 21:17 CHANGES -rw-r--r--. 1 1001 1001 343040 3月 18 21:17 CHANGES.ru drwxr-xr-x. 2 1001 1001 4096 3月 26 02:34 conf -rwxr-xr-x. 1 1001 1001 2369 3月 18 21:17 configure drwxr-xr-x. 3 1001 1001 4096 3月 26 02:34 contrib drwxr-xr-x. 2 1001 1001 4096 3月 26 02:34 html -rw-r--r--. 1 1001 1001 1397 3月 18 21:17 LICENSE drwxr-xr-x. 2 1001 1001 4096 3月 26 02:34 man -rw-r--r--. 1 1001 1001 49 3月 18 21:17 README drwxr-xr-x. 8 1001 1001 4096 3月 26 02:34 src |
l 运行./configure进行安装前的设置，主要设置安装路径、FastDFS插件模块目录、pcre库目录、zlib库目录。
如果提示错误，可能缺少依赖的软件包，需先安装依赖包，再次运行./configure
|
[root@storage1 nginx-1.4.7]# ./configure --prefix=/usr/local/nginx --add-module=/usr/local/src/fastdfs-nginx-module/src --with-pcre=/usr/local/src/pcre-8.34/ --with-zlib=/usr/local/src/zlib-1.2.8 |
l 运行make进行编译，确保编译成功。
|
[root@storage1 nginx-1.4.7]#make |
l 运行makeinstall进行安装。
|
[root@storage1 nginx-1.4.7]#make install |
l 将FastDFS的nginx插件模块的配置文件copy到FastDFS配置文件目录。
|
[root@storage1 nginx-1.4.7]#cp /usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/ |
安装完成后，nginx所有文件在/usr/local/nginx下：
|
[root@storage1 nginx-1.4.7]# ll /usr/local/nginx/ 总用量 16 drwxr-xr-x. 2 root root 4096 3月 26 03:11 conf drwxr-xr-x. 2 root root 4096 3月 26 03:11 html drwxr-xr-x. 2 root root 4096 3月 26 03:11 logs drwxr-xr-x. 2 root root 4096 3月 26 03:11 sbin |
l 至此nginx以及FastDFS的nginx插件模块安装完成。
2. 配置
l 编辑/usr/local/nginx/conf配置文件目录下的nginx.conf，设置添加storage信息并保存。
|
[root@storage1 nginx-1.4.7]# vim /usr/local/nginx/conf/nginx.conf |
将server段中的listen端口号改为8080：
listen 8080;
在server段中添加：
location~/group[1-3]/M00{
root/fdfs/storage/data;
ngx_fastdfs_module;
}
l 编辑/etc/fdfs配置文件目录下的mod_fastdfs.conf，设置storage信息并保存。
|
[root@storage1 nginx-1.4.7]# vim /etc/fdfs/mod_fastdfs.conf |
一般只需改动以下几个参数即可：
base_path=/fdfs/storage#保存日志目录
tracker_server=172.16.1.202:22122#tracker服务器的IP地址以及端口号
storage_server_port=23000#storage服务器的端口号
group_name=group1#当前服务器的group名
url_have_group_name= true #文件url中是否有group名
store_path_count=1 #存储路径个数，需要和store_path个数匹配
store_path0=/fdfs/storage #存储路径
http.need_find_content_type=true#从文件扩展名查找文件类型（nginx时为true）
group_count= 3 #设置组的个数
在末尾增加3个组的具体信息：
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/fdfs/storage
[group2]
group_name=group2
storage_server_port=23000
store_path_count=1
store_path0=/fdfs/storage
[group3]
group_name=group3
storage_server_port=23000
store_path_count=1
store_path0=/fdfs/storage
l 建立M00至存储目录的符号连接。
|
[root@storage1 nginx-1.4.7]# ln -s /fdfs/storage/data /fdfs/storage/data/M00 [root@storage1 nginx-1.4.7]# ll /fdfs/storage/data/M00 lrwxrwxrwx. 1 root root 19 3月 26 03:44 /fdfs/storage/data/M00 -> /fdfs/storage/data/ |
l 至此，nginx以及FastDFS插件模块设置完成。
3. 运行
l 运行nginx之前，先要把防火墙中对应的端口打开（本例中为8080）。
|
[root@storage1nginx-1.4.7]# iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 8080 -j ACCEPT [root@storage1nginx-1.4.7]# /etc/init.d/iptables save iptables：将防火墙规则保存到 /etc/sysconfig/iptables：[确定] |
l 启动nginx，确认启动是否成功。（查看是否对应端口8080是否开始监听）
|
[root@storage1 nginx-1.4.7]# /usr/local/nginx/sbin/nginx ngx_http_fastdfs_set pid=40638 [root@storage1 nginx-1.4.7]# netstat -unltp | grep nginx tcp 0 0.0.0.0:8080 0.0.0.0:* LISTEN 40639/nginx |
也可查看nginx的日志是否启动成功或是否有错误。
|
[root@storage1 nginx-1.4.7]# cat /usr/local/nginx/logs/error.log ngx_http_fastdfs_process_init pid=40640 [2014-03-26 03:47:17] INFO - local_host_ip_count: 2, 127.0.0.1 172.16.1.203 [2014-03-26 03:47:17] INFO - fastdfs apache / nginx module v1.15, response_mode=proxy, base_path=/tmp, url_have_group_name=1, group_count=3, connect_timeout=2, network_timeout=30, tracker_server_count=1, if_alias_prefix=, local_host_ip_count=2, need_find_content_type=1, default_content_type=application/octet-stream, anti_steal_token=0, token_ttl=0s, anti_steal_secret_key length=0, token_check_fail content_type=, token_check_fail buff length=0, load_fdfs_parameters_from_tracker=1, storage_sync_file_max_delay=86400s, use_storage_id=0, storage server id count=0, flv_support=1, flv_extension=flv [2014-03-26 03:47:17] INFO - group 1. group_name=group1, storage_server_port=23000, path_count=1, store_path0=/fdfs/storage [2014-03-26 03:47:17] INFO - group 2. group_name=group2, storage_server_port=23000, path_count=1, store_path0=/fdfs/storage [2014-03-26 03:47:17] INFO - group 3. group_name=group3, storage_server_port=23000, path_count=1, store_path0=/fdfs/storage |
在error.log中没有错误，既启动成功。可以打开浏览器，直接访问http://172.16.1.203:8080，查看是否弹出nginx欢迎页面。
之后依次在172.16.1.204~208上全部安装上nginx并确认运行正常。
l 设置开机自动启动。
|
[root@storage1 nginx-1.4.7]# vim /etc/rc.d/rc.local |
将运行命令行添加进文件：/usr/local/nginx/sbin/nginx
(五) 在tracker上安装nginx
在tracker上安装的nginx主要为了提供http访问的反向代理、负载均衡以及缓存服务。
1. 安装
l 首先将代码包和插件复制到系统的/usr/local/src内（可选），然后使用tar命令解压
|
[root@trackeropt]# cp nginx-1.4.7.tar.gz /usr/local/src [root@tracker opt]# cp ngx_cache_purge-2.1.tar.gz /usr/local/src [root@tracker opt]# cp pcre-8.34.tar.gz /usr/local/src [root@tracker opt]# cp zlib-1.2.8.tar.gz /usr/local/src [root@tracker opt]# cd /usr/local/src/ [root@tracker src]# tar zxf nginx-1.4.7.tar.gz [root@tracker src]# tar zxf ngx_cache_purge-2.1.tar.gz [root@tracker src]# tar zxf pcre-8.34.tar.gz [root@tracker src]# tar zxf zlib-1.2.8.tar.gz [root@tracker src]# cd nginx-1.4.7 [root@tracker nginx-1.4.7]# ll 总用量 592 drwxr-xr-x. 6 1001 1001 4096 3月 26 02:34 auto -rw-r--r--. 1 1001 1001 225213 3月 18 21:17 CHANGES -rw-r--r--. 1 1001 1001 343040 3月 18 21:17 CHANGES.ru drwxr-xr-x. 2 1001 1001 4096 3月 26 02:34 conf -rwxr-xr-x. 1 1001 1001 2369 3月 18 21:17 configure drwxr-xr-x. 3 1001 1001 4096 3月 26 02:34 contrib drwxr-xr-x. 2 1001 1001 4096 3月 26 02:34 html -rw-r--r--. 1 1001 1001 1397 3月 18 21:17 LICENSE drwxr-xr-x. 2 1001 1001 4096 3月 26 02:34 man -rw-r--r--. 1 1001 1001 49 3月 18 21:17 README drwxr-xr-x. 8 1001 1001 4096 3月 26 02:34 src |
l 运行./configure进行安装前的设置，主要设置安装路径、nginx cache purge插件模块目录、pcre库目录、zlib库目录。
如果提示错误，可能缺少依赖的软件包，需先安装依赖包，再次运行./configure
|
[root@storage1 nginx-1.4.7]# ./configure --prefix=/usr/local/nginx --add-module=/usr/local/src/ngx_cache_purge-2.1--with-pcre=/usr/local/src/pcre-8.34/ --with-zlib=/usr/local/src/zlib-1.2.8 |
l 运行make进行编译，确保编译成功。
|
[root@storage1 nginx-1.4.7]#make |
l 运行makeinstall进行安装。
|
[root@storage1 nginx-1.4.7]#make install |
l 至此nginx以及nginx cachepurge插件模块安装完成。
2. 配置
l 编辑/usr/local/nginx/conf配置文件目录下的nginx.conf，设置负载均衡以及缓存。
|
[root@storage1 nginx-1.4.7]# vim /usr/local/nginx/conf/nginx.conf |
worker_processes 4; #根据CPU核心数而定
events{
worker_connections 65535; #最大链接数
use epoll;#新版本的Linux可使用epoll加快处理性能
}
http{
#设置缓存参数
server_names_hash_bucket_size 128;
client_header_buffer_size32k;
large_client_header_buffers 4 32k;
client_max_body_size300m;
sendfile on;
tcp_nopushon;
proxy_redirectoff;
proxy_set_headerHost $http_host;
proxy_set_headerX-Real-IP $remote_addr;
proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size16k;
proxy_buffers 4 64k;
proxy_busy_buffers_size128k;
proxy_temp_file_write_size128k;
#设置缓存存储路径、存储方式、分配内存大小、磁盘最大空间、缓存期限
proxy_cache_path/var/cache/nginx/proxy_cache levels=1:2 keys_zone=http-cache:500m max_size=10ginactive=30d;
proxy_temp_path/var/cache/nginx/proxy_cache/tmp;
#设置group1的服务器
upstreamfdfs_group1 {
server172.16.1.203:8080 weight=1 max_fails=2 fail_timeout=30s;
server172.16.1.204:8080 weight=1 max_fails=2 fail_timeout=30s;
}
#设置group2的服务器
upstreamfdfs_group2 {
server172.16.1.205:8080 weight=1 max_fails=2 fail_timeout=30s;
server172.16.1.206:8080 weight=1 max_fails=2 fail_timeout=30s;
}
#设置group3的服务器
upstreamfdfs_group3 {
server172.16.1.207:8080 weight=1 max_fails=2 fail_timeout=30s;
server172.16.1.208:8080 weight=1 max_fails=2 fail_timeout=30s;
}
server{
#设置服务器端口
listen 8080;
#设置group1的负载均衡参数
location/group1/M00 {
proxy_next_upstream http_502http_504 error timeout invalid_header;
proxy_cachehttp-cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://fdfs_group1;
expires30d;
}
#设置group2的负载均衡参数
location/group2/M00 {
proxy_next_upstream http_502http_504 error timeout invalid_header;
proxy_cachehttp-cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://fdfs_group2;
expires30d;
}
#设置group3的负载均衡参数
location/group3/M00 {
proxy_next_upstream http_502http_504 error timeout invalid_header;
proxy_cachehttp-cache;
proxy_cache_valid 200 304 12h;
proxy_cache_key $uri$is_args$args;
proxy_pass http://fdfs_group3;
expires30d;
}
#设置清除缓存的访问权限
location~ /purge(/.*) {
allow127.0.0.1;
allow172.16.1.0/24;
denyall;
proxy_cache_purgehttp-cache $1$is_args$args;
}
l 至此，nginx以及nginx cachepurge插件模块设置完成。
3. 运行
运行nginx之前，先要把防火墙中对应的端口打开（本例中为8080）。
|
[root@trackernginx-1.4.7]# iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 8080 -j ACCEPT [root@trackernginx-1.4.7]# /etc/init.d/iptables save iptables：将防火墙规则保存到 /etc/sysconfig/iptables：[确定] |
启动nginx，确认启动是否成功。（查看是否对应端口8080是否开始监听）
|
[root@trackernginx-1.4.7]# /usr/local/nginx/sbin/nginx ngx_http_fastdfs_set pid=40638 [root@trackernginx-1.4.7]# netstat -unltp | grep nginx tcp 0 0.0.0.0:8080 0.0.0.0:* LISTEN 40639/nginx |
l 尝试上传一个文件到FastDFS，然后访问试试。先配置client.conf文件。
|
[root@trackernginx-1.4.7]# vim /etc/fdfs/client.conf |
修改以下参数：
base_path=/fdfs/tracker #日志存放路径
tracker_server=172.16.1.202:22122 #tracker服务器IP地址和端口号
http.tracker_server_port=8080 #tracker服务器的http端口号
使用/usr/local/bin/fdfs_upload_file上传一个文件，程序会自动返回文件的URL。
|
[root@trackernginx-1.4.7]# /usr/local/bin/fdfs_upload_file /etc/fdfs/client.conf /mnt/monk.jpg group3/M00/00/00/rBABz1MzKG6Ad_hBAACKLsUDM60560.jpg |
然后使用浏览器访问：
可以看到文件被正确读取出来了。
查看nginx的access.log日志，可以看到访问返回200成功。
|
[root@trackernginx-1.4.7]# tail -n 10 -f /usr/local/nginx/logs/access.log 172.16.1.201 - - [26/Mar/2014:13:15:00 +0800] "GET /group3/M00/00/00/rBABz1MzKG6Ad_hBAACKLsUDM60560.jpg HTTP/1.1" 200 35374 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" |
查看nginx的cache目录，可以看到已经生成了缓存文件。
|
[root@tracker nginx-1.4.7]# ll /var/cache/nginx/proxy_cache/ -R /var/cache/nginx/proxy_cache/: 总用量 8 drwx------. 3 nobody nobody 4096 3月 26 12:57 6 drwxr-xr-x. 2 nobody root 4096 3月 26 13:14 tmp
/var/cache/nginx/proxy_cache/6: 总用量 4 drwx------. 2 nobody nobody 4096 3月 26 13:08 11
/var/cache/nginx/proxy_cache/6/11: 总用量 36 -rw-------. 1 nobody nobody 35686 3月 26 13:08 b1f6fb1f7266f796765b6d6965021116 |
如果要手动清除缓存，可以在文件URL之前加上purge：
l 设置开机自动启动。
|
[root@trackernginx-1.4.7]# vim /etc/rc.d/rc.local |
将运行命令行添加进文件：/usr/local/nginx/sbin/nginx
至此，tracker服务器上的http反向代理+负载均衡+缓存已经安装完成。
二、 使用
1. 上传文件
FastDFS提供了多种方式上传文件：
l 使用fdfs_upload_file上传（之前已经有演示）
l 使用C语言客户端接口上传
l 使用PHP客户端接口上传
l 使用Java客户端接口上传
l 使用Python客户端接口上传
l 使用.NET客户端接口上传
根据具体情况选择使用。上传均支持断点续传。
2. 下载文件
l 使用fdfs_download_file通过tracker服务器下载
l 使用nginx通过http方式直接从storage下载（支持断点续传，作者推荐）
3. 监视服务器资源
l 使用fdfs_monitor查看tracker和所有group的运行情况。
|
[root@tracker tmp]# /usr/local/bin/fdfs_monitor /etc/fdfs/client.conf 2014-03-26 15:01:07] DEBUG - base_path=/fdfs/tracker, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
server_count=1, server_index=0
tracker server is 172.16.1.202:22122
group count: 3
Group 1: group name = group1 disk total space = 27789 MB disk free space = 23825 MB trunk free space = 0 MB storage server count = 2 active server count = 2 storage server port = 23000 storage HTTP port = 8080 store path count = 1 subdir count per path = 256 current write server index = 1 current trunk file id = 0
Storage 1: id = 172.16.1.203 ip_addr = 172.16.1.203 ACTIVE http domain = version = 5.01 join time = 2014-03-26 01:48:19 up time = total storage = 27789 MB free storage = 23844 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8080 current_write_path = 0 source storage id= if_trunk_server= 0 total_upload_count = 1 success_upload_count = 1 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 24 success_upload_bytes = 24 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 24 success_sync_out_bytes = 24 total_file_open_count = 1 success_file_open_count = 1 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 1 success_file_write_count = 1 last_heart_beat_time = 2014-03-26 15:00:47 last_source_update = 2014-03-26 13:13:55 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 Storage 2: id = 172.16.1.204 ip_addr = 172.16.1.204 ACTIVE http domain = version = 5.01 join time = 2014-03-26 18:53:19 up time = 2014-03-26 18:53:19 total storage = 27789 MB free storage = 23825 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8080 current_write_path = 0 source storage id= 172.16.1.203 if_trunk_server= 0 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 24 success_sync_in_bytes = 24 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 1 success_file_open_count = 1 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 1 success_file_write_count = 1 last_heart_beat_time = 2014-03-26 15:00:40 last_source_update = 1970-01-01 08:00:00 last_sync_update = 2014-03-27 03:38:23 last_synced_timestamp = 2014-03-26 13:13:56 (-1s delay)
Group 2: group name = group2 disk total space = 27789 MB disk free space = 23825 MB trunk free space = 0 MB storage server count = 2 active server count = 2 storage server port = 23000 storage HTTP port = 8080 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0
Storage 1: id = 172.16.1.205 ip_addr = 172.16.1.205 ACTIVE http domain = version = 5.01 join time = 2014-03-26 19:37:36 up time = 2014-03-26 19:37:36 total storage = 27789 MB free storage = 23825 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8080 current_write_path = 0 source storage id= if_trunk_server= 0 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2014-03-26 15:01:02 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 Storage 2: id = 172.16.1.206 ip_addr = 172.16.1.206 ACTIVE http domain = version = 5.01 join time = 2014-03-26 22:38:04 up time = 2014-03-26 22:38:04 total storage = 27789 MB free storage = 23825 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8080 current_write_path = 0 source storage id= 172.16.1.205 if_trunk_server= 0 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2014-03-26 15:01:02 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00
Group 3: group name = group3 disk total space = 27789 MB disk free space = 23825 MB trunk free space = 0 MB storage server count = 2 active server count = 2 storage server port = 23000 storage HTTP port = 8080 store path count = 1 subdir count per path = 256 current write server index = 1 current trunk file id = 0
Storage 1: id = 172.16.1.207 ip_addr = 172.16.1.207 ACTIVE http domain = version = 5.01 join time = 2014-03-26 22:50:28 up time = 2014-03-26 22:50:28 total storage = 27789 MB free storage = 23825 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8080 current_write_path = 0 source storage id= 172.16.1.208 if_trunk_server= 0 total_upload_count = 1 success_upload_count = 1 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 35374 success_upload_bytes = 35374 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 1 success_file_open_count = 1 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 1 success_file_write_count = 1 last_heart_beat_time = 2014-03-26 15:01:20 last_source_update = 2014-03-27 03:20:13 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 Storage 2: id = 172.16.1.208 ip_addr = 172.16.1.208 ACTIVE http domain = version = 5.01 join time = 2014-03-26 22:49:37 up time = 2014-03-26 22:49:37 total storage = 27789 MB free storage = 23825 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8080 current_write_path = 0 source storage id= if_trunk_server= 0 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 35374 success_sync_in_bytes = 35374 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 1 success_file_open_count = 1 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 1 success_file_write_count = 1 last_heart_beat_time = 2014-03-26 15:01:32 last_source_update = 1970-01-01 08:00:00 last_sync_update = 2014-03-27 03:20:19 last_synced_timestamp = 2014-03-27 03:20:14 (-1s delay) |
4. 其他功能
l 使用fdfs_crc32获取文件CRC
l 使用fdfs_delete_file删除文件
l 使用fdfs_file_info查看文件属性信息
以上所有功能均可使用接口在代码中调用。
