flume sink到hdfs需要hadoop相关jar包，否则启动报错

1. NoClassDefFoundError报错

Failed to start agent because dependencies were not found in classpath. Error follows.
java.lang.NoClassDefFoundError: org/apache/hadoop/io/SequenceFile$CompressionType

原因

SequenceFile$CompressionType对应缺少{HADOOP_HOME}share/hadoop/common/hadoop-common-2.4.0.jar
但是还有很多同样的问题。

解决

cp {HADOOP_HOME}share/hadoop/common/*.jar ${FLUME_HOME}/lib/

cp {HADOOP_HOME}share/hadoop/common/lib/*.jar ${FLUME_HOME}/lib/

 

2. 报错No FileSystem for scheme: hdfs

原因可能是缺少hdfs jar

解决
cp {HADOOP_HOME}share/hadoop/hdfs/hadoop-hdfs-2.4.1.jar ${FLUME_HOME}/lib/

也有可能是版本匹配问题