Hadoop启动报错：org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "hdfs"

报错信息

org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "hdfs"

	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3281)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3301)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227)
	at com.courage.hadoop.TestHdfs.conn(TestHdfs.java:22)
	
java.lang.NullPointerException
	at com.courage.hadoop.TestHdfs.close(TestHdfs.java:32)

原因

jar包或打包问题

Different JARs (hadoop-commons for LocalFileSystem, hadoop-hdfs for DistributedFileSystem) each contain a different file called org.apache.hadoop.fs.FileSystem in their META-INFO/servicesdirectory. This file lists the canonical classnames of the filesystem implementations they want to declare (This is called a Service Provider Interface implemented via java.util.ServiceLoader, see org.apache.hadoop.FileSystem line 2622).
When we use maven-assembly-plugin, it merges all our JARs into one, and all META-INFO/services/org.apache.hadoop.fs.FileSystem overwrite each-other. Only one of these files remains (the last one that was added). In this case, the FileSystem list from hadoop-commonsoverwrites the list from hadoop-hdfs, so DistributedFileSystem was no longer declared.

解决办法

1.通过配置方式：

Java代码

这个解决hdfs问题
hadoopConf.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());

这个解决本地file问题
hadoopConf.set("fs.file.impl", org.apache.hadoop.fs.LocalFileSystem.class.getName());

2.通过打包插件方式

Xml代码

<plugin>  
    <groupId>org.apache.maven.plugins</groupId>  
    <artifactId>maven-shade-plugin</artifactId>  
    <version>2.3</version>  
    <executions>  
      <execution>  
        <phase>package</phase>  
        <goals>  
          <goal>shade</goal>  
        </goals>  
        <configuration>  
          <transformers>  
            <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>  
          </transformers>  
        </configuration>  
      </execution>  
    </executions>  
  </plugin>

__EOF__

本文作者：等不到的口琴
本文链接：https://www.cnblogs.com/Courage129/p/17532810.html
关于博主：评论和私信会在第一时间回复。或者直接私信我。
版权声明：本博客所有文章除特别声明外，均采用 BY-NC-SA 许可协议。转载请注明出处！
声援博主：如果您觉得文章对您有帮助，可以点击文章右下角【推荐】一下。您的鼓励是博主的最大动力！