IDEA本地配置kerberos认证

1、项目导入之后设置maven依赖之后先将scala代码文件设为source

 

2、创建resources文件夹，将配置的资源文件放入这里

 

3、设置在本地运行代码：-Dspark.master=local，并设置其他参数

 

4、在跑模型之前先读取资源文件中的配置信息进行kerberos认证

 

def initKerberos(): Unit ={

  //kerberos权限认证

  val path = this.getClass.getClassLoader.getResource("").getPath

  val conf = new Configuration

  System.setProperty("java.security.krb5.conf", path + "krb5.conf")

  conf.addResource(new Path(path + "hdfs-site.xml"))

  conf.addResource(new Path(path + "hive-site.xml"))

  conf.set("hadoop.security.authentication", "Kerberos")

  conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem")

  conf.set("dfs.client.use.datanode.hostname", "true")

  UserGroupInformation.setConfiguration(conf)

  UserGroupInformation.loginUserFromKeytab("test001", path + "test001.keytab")

  println("login user: "+UserGroupInformation.getLoginUser())

}