kerberos相关
1.Kerberos介绍
Kerberos是一种计算机网络授权协议，用来在非安全网络中，对个人通信以安全的手段进行身份认证。这个词又指麻省理工学院为这个协议开发的一套计算机软件。软件设计上采用客户端/服务器结构，并且能够进行相互认证，即客户端和服务器端均可对对方进行身份认证。可以用于防止窃听、防止重放攻击、保护数据完整性等场合，是一种应用对称密钥体制进行密钥管理的系统。
Kerberos中的一些概念：
1）KDC：密钥分发中心，负责管理发放票据，记录授权。
2）Realm：Kerberos管理领域的标识。
3）principal：当每添加一个用户或服务的时候都需要向kdc添加一条principal，principl的形式为：主名称／实例名@领域名。
4）主名称：主名称可以是用户名或服务名，表示是用于提供各种网络服务（如hdfs，yarn，hive）的主体。
5）实例名：实例名简单理解为主机名。
2.安装及配置KDC服务
卸载老的krb
sudo apt remove --purge krb* sudo rm -rf /etc/krb5kdc sudo rm -rf /var/lib/krb5kdc
安装KDC服务和管理员服务
sudo apt-get install krb5-kdc krb5-admin-server
配置Realm域名
输入kerberos服务器的hostname
创建新的realm，耐心等，Loading random data会很慢
lintong@master:~$ sudo krb5_newrealm
This script should be run on the master KDC/admin server to initialize
a Kerberos realm. It will ask you to type in a master key password.
This password will be used to generate a key that is stored in
/etc/krb5kdc/stash. You should try to remember this password, but it
is much more important that it be a strong password than that it be
remembered. However, if you lose the password and /etc/krb5kdc/stash,
you cannot decrypt your Kerberos database.
Loading random data
Initializing database '/var/lib/krb5kdc/principal' for realm 'EXAMPLE.COM',
master key name 'K/M@HADOOP.COM'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:
Re-enter KDC database master key to verify:
此处需要输入Kerberos数据库的密码: 自己定
Now that your realm is set up you may wish to create an administrative
principal using the addprinc subcommand of the kadmin.local program.
Then, this principal can be added to /etc/krb5kdc/kadm5.acl so that
you can use the kadmin program on other computers. Kerberos admin
principals usually belong to a single user and end in /admin. For
, if jruser is a Kerberos administrator, then in addition to
the normal jruser principal, a jruser/admin principal should be
created.
Don't forget to set up DNS information so your clients can find your
KDC and admin servers. Doing so is documented in the administration
guide.
参考：
https://docs.cloudera.com/documentation/enterprise/5-15-x/topics/cm_sg_s2_jce_policy.html http://techpubs.spinlocksolutions.com/dklar/kerberos.html
此时，krb的相关配置就配好了，可以查看/etc/krb5.conf和/etc/krb5kdc下面的kadm5.acl kdc.conf stash文件
再创建一个管理员账户，需要设置密码，这里addprinc root或者addprinc admin
lintong@master:~$ sudo kadmin.local Authenticating as principal root/admin@MASTER with password. kadmin.local: addprinc admin WARNING: no policy specified for admin@HADOOP.COM; defaulting to no policy Enter password for principal "admin@HADOOP.COM": Re-enter password for principal "admin@HADOOP.COM": Principal "admin@HADOOP.COM" created.
再添加cloudera-scm/admin
kadmin.local: addprinc cloudera-scm/admin@HADOOP.COM WARNING: no policy specified for cloudera-scm/admin@HADOOP.COM; defaulting to no policy Enter password for principal "cloudera-scm/admin@HADOOP.COM": Re-enter password for principal "cloudera-scm/admin@HADOOP.COM": Principal "cloudera-scm/admin@HADOOP.COM" created
给admin赋权，添加内容 */admin@HADOOP.COM *
root@master:/etc/krb5kdc# vim kadm5.acl
然后重启kbr相关服务
systemctl restart krb*
查看kerberos主体
root@master:/etc/krb5kdc# sudo kadmin.local Authenticating as principal admin/admin@EXAMPLE.COM with password. kadmin.local: listprincs K/M@HADOOP.COM admin@HADOOP.COM cloudera-scm/admin@HADOOP.COM kadmin/admin@HADOOP.COM kadmin/changepw@HADOOP.COM kadmin/master@HADOOP.COM kiprop/master@HADOOP.COM krbtgt/HADOOP.COM@HADOOP.COM
使用kinit认证，并查看票据过期时间
kinit admin Password for admin@EXAMPLE.COM: klist Ticket cache: FILE:/tmp/krb5cc_1000 Default principal: admin@EXAMPLE.COM Valid starting Expires Service principal 2020-09-16T23:49:57 2020-09-17T09:49:57 krbtgt/EXAMPLE.COM@EXAMPLE.COM renew until 2020-09-17T23:49:54
如果要删除kerberos主体
delete_principal root/admin
安装之后查看是否正常运行
systemctl status krb*
都正常
● krb5-admin-server.service - Kerberos 5 Admin Server
Loaded: loaded (/lib/systemd/system/krb5-admin-server.service; enabled; vendor preset: enabled)
Active: active (running) since 日 2021-10-17 13:51:12 CST; 1 day 10h ago
Main PID: 822 (kadmind)
Tasks: 1
Memory: 1.4M
CPU: 12ms
CGroup: /system.slice/krb5-admin-server.service
└─822 /usr/sbin/kadmind -nofork
10月 17 13:51:12 master kadmind[822]: setsockopt(11,IPV6_V6ONLY,1) worked
10月 17 13:51:12 master kadmind[822]: listening on fd 12: tcp 0.0.0.0.464
10月 17 13:51:12 master kadmind[822]: listening on fd 11: tcp ::.464
10月 17 13:51:12 master kadmind[822]: listening on fd 13: rpc 0.0.0.0.749
10月 17 13:51:12 master kadmind[822]: setsockopt(14,IPV6_V6ONLY,1) worked
10月 17 13:51:12 master kadmind[822]: listening on fd 14: rpc ::.749
10月 17 13:51:12 master kadmind[822]: set up 6 sockets
10月 17 13:51:12 master kadmind[822]: Seeding random number generator
10月 17 13:51:22 master kadmind[822]: starting
10月 17 13:51:22 master kadmind[822]: kadmind: starting...
● krb5-kdc.service - Kerberos 5 Key Distribution Center
Loaded: loaded (/lib/systemd/system/krb5-kdc.service; enabled; vendor preset: enabled)
Active: active (running) since 日 2021-10-17 13:51:12 CST; 1 day 10h ago
Process: 806 ExecStart=/usr/sbin/krb5kdc -P /var/run/krb5-kdc.pid $DAEMON_ARGS (code=exited, status=0/SUCCESS)
Main PID: 858 (krb5kdc)
Tasks: 1
cdh 5.15中HUE集成kerberos，参考
https://docs.cloudera.com/documentation/enterprise/5-15-x/topics/cdh_sg_hue_kerberos_config.html
cdh 5.15启动kerberos认证向导，参考
https://docs.cloudera.com/documentation/enterprise/5-15-x/topics/cm_sg_intro_kerb.html
3.安装jce，java1.8_161以上可以不加
https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_sg_s2_jce_policy.html
需要去oracle的网站下载jce_policy-8.zip文件
https://www.oracle.com/de/java/technologies/javase-jce8-downloads.html
按照README.txt中的指示将US_export_policy.jar和local_policy.jar文件复制到$JAVA_HOME的jre/lib/security目录下
lintong@master:/usr/java/jdk1.8/jre/lib/security$ ls blacklist cacerts java.security local_policy.jar US_export_policy.jar blacklisted.certs java.policy javaws.policy trusted.libraries
如果不加的话，hdfs启动会报错
Socket Reader #1 for port 8022: readAndProcess from client 192.168.8.103 threw exception [javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: Encryption type AES256 CTS mode with HMAC SHA1-96 is not supported/enabled)]]
4.CDH集群启动kerberos
需要执行的步骤，参考：如何在CDH集群启用Kerberos
以及CDH禁用kerberos 的相反步骤
全部勾选
不必勾选
填写管理员账号密码
下一步
重启集群
等待集群重启
使用-norandkey来不重新生成凭证拿到keytab文件
kadmin.local: ktadd -k /var/lib/hadoop-hdfs/hdfs.keytab -norandkey hdfs/master@HADOOP.COM kadmin.local: ktadd -k /var/lib/hive/hive.keytab -norandkey hive/master@HADOOP.COM kadmin.local: ktadd -k /etc/hue/hue@HADOOP.COM -norandkey hue/master@HADOOP.COM
5.其他问题
1.kinit
kinit -kt ./hdfs.keytab hdfs/master@HADOOP.COM
2.kerberos认证覆盖问题
先显示指定KRB5CCNAME存储的路径
export KRB5CCNAME=/tmp/krb5cc_xxx kinit -kt /home/xxx.keytab xxx
3.生成keytab，重新生成之后，之前的失效
sudo kadmin.local -q "xst -k hdfs.keytab hdfs"
参考
https://wiki.shileizcc.com/confluence/display/zd/Kerberos
4.CDH keytab
可以在/run/cloudera-scm-agent/process目录下找到各个组件的keytab，比如
/run/cloudera-scm-agent/process/53582-hive-HIVESERVER2# ls cloudera-monitor.properties hive-log4j.properties sentry-site.xml cloudera-stack-monitor.properties hive-site.xml service-metrics.properties config.zip logs spark-defaults.conf core-site.xml navigator.client.properties supervisor.conf creds.localjceks navigator.lineage.client.properties yarn-conf fair-scheduler.xml proc.json hive.keytab redaction-rules.json
5.查看票据期限，默认的过期时间为10小时，10点开始，20点过期
klist -f Ticket cache: FILE:/tmp/krb5cc_5035 Default principal: xxx@XXXXX Valid starting Expires Service principal 05/06/2021 10:53:45 05/06/2021 20:53:45 krbtgt/XXXXX@XXXXX renew until 05/13/2021 10:53:45, Flags: FRIA
6.开启kerberos之后，浏览器访问HDFS，YARN的http页面会报
HTTP ERROR 401 Problem accessing /cluster/app/application
在mac下需要先使用对应的keytab进行kinit，然后再进入firefox的浏览器高级选项 about:config 中配置2个参数
注意mac需要能访问KDC认证服务器的端口，默认为88端口
network.negotiate-auth.trusted-uris: your KDC host network.auth.use-sspi:false
然后就可以访问了
参考：解决Ambari启用Kerberos后HDFS/YARN/SPARK等页面无法打开问题
本文只发表于博客园和tonglin0325的博客，作者：tonglin0325，转载请注明原文链接：https://www.cnblogs.com/tonglin0325/p/11303488.html
