17、【SparkSQL】org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism异常

• 背景
  当我在使用IDEA本地连接服务器spark服务时，虽然拷贝了hive-site.xml，提供了相应的依赖，仍然报错：org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism
  依赖如下：

    <dependencies>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.12</artifactId>
            <version>3.1.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.12</artifactId>
            <version>3.1.2</version>
        </dependency>
  
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.25</version>
        </dependency>
  
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-exec</artifactId>
            <version>3.1.2</version>
        </dependency>
  
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_2.12</artifactId>
            <version>3.1.2</version>
        </dependency>
  
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-auth</artifactId>
            <version>3.2.0</version>
        </dependency>
    </dependencies>
  
  

• 分析
  在远程访问hadoop时，需要提供认证信息

  https://hadoop.apache.org/docs/stable/hadoop-auth/Examples.html
  https://blog.csdn.net/csr_hema/article/details/8147590

• 措施
  在Maven添加如下依赖：

          <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-auth</artifactId>
            <version>3.2.0</version>
        </dependency>