fuse-dfs挂载hdfs实录
部署安装了最新稳定版hadoop2.2.0。然后在网上找来fuse-dfs编译教程。可是最后失败了。至今原因未知~~,错误描写叙述为:Transport endpoint is not connected。后将安装部署hadoop1.2.1版本号。最后測试成功,记录例如以下:
使用root完毕一下操作:
1、安装依赖包
apt-get install autoconf automake libtool make gawk g++ ant
2、卸载已有的fuse,安装fuse
apt-get purge fuse apt-get purge libfuse2 tar -zxf fuse-2.9.3.tar.gz cd fuse-2.9.3 ./configure --prefix=/usr/fuse make make install
3、环境变量设置
ln -s /usr/fuse/bin/fusermount /usr/bin/
vi /etc/profile
export FUSE_HOME=/usr/fuse export OS_ARCH=amd64(你的本机类型) export OS_BIT=64(你的操作系统位数) export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/$OS_ARCH/server:${HADOOP_HOME}/build/c++/Linux-$OS_ARCH-$OS_BIT/lib:/usr/local/lib:/usr/lib:$FUSE_HOME/lib
source /etc/profile
4、编译libhdfs,fuse-dfs与hdfs的接口
cd $HADOOP_HOME/ ant compile-c++-libhdfs -Dlibhdfs=1 -Dcompile.c++=1 ln -s c++/Linux-$OS_ARCH-$OS_BIT/lib build/libhdfs5、编译fuse-dfs
ln -s /usr/fuse/include/* /usr/include/ ln -s /usr/fuse/lib/libfuse.so /usr/lib/ ant compile-contrib -Dlibhdfs=1 -Dfusedfs=16、挂在hdfs到本地
编辑fuse_dfs_wrapper.sh,vi $HADOOP_HOME/build/contrib/fuse-dfs/fuse_dfs_wrapper.sh加上环境參数,尾行改动例如以下:
export JAVA_HOME=<你的javahome> export HADOOP_HOME=<你的hadoophome> export FUSE_HOME=/usr/fuse export PATH=$PATH:$HADOOP_HOME/contrib/fuse_dfs for f in ls $HADOOP_HOME/lib/*.jar $HADOOP_HOME/*.jar do export CLASSPATH=$CLASSPATH:$f done export OS_ARCH=amd64 export OS_BIT=64 export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/$OS_ARCH/server:${HADOOP_HOME}/build/c++/Linux-$OS_ARCH-$OS_BIT/lib:/usr/local/lib:/usr/lib:$FUSE_HOME/lib末尾改动:
fuse_dfs $@
改动fuse_dfs_wrapper.sh权限
chmod 755 ${HADOOP_HOME}/build/contrib/fuse-dfs/fuse_dfs_wrapper.sh
建立链接
ln -s ${HADOOP_HOME}/build/contrib/fuse-dfs/fuse_dfs_wrapper.sh /usr/local/bin ln -s ${HADOOP_HOME}/build/contrib/fuse-dfs/fuse_dfs /usr/local/bin/
mkdir -p /mnt/dfs fuse_dfs_wrapper.sh dfs://localhost:9005 /mnt/dfs注:localhost:9005为hadoop的core-site.xml配置文件里fs.default.name的值,前面加上“dfs”
卸载hdfs
umount /mnt/dfs
1、运行步骤5的时候。出现undefined reference to fuse_get_context等一些列的错误。
复制运行错误的命令,将-L和-l參数放到组后。运行可能会出现undefined reference to symbol 'floor@@GLIBC_2.2.5'的错误,那么最后加入-lm參数。问题解决。然后又一次运行编译
2、commons-logging#commons-logging;1.0.4: not found错误。改动ivy/libraries.properties文件的commons-logging.version=1.1.1就可以。
3、运行挂载的时候一定要将地址输正确。包含主机名和port号。否则会出现读写错误的问题。
posted on 2017-05-21 10:45 cynchanpin 阅读(2090) 评论(0) 编辑 收藏 举报