Hadoop相关的Kerberos设置
Hadoop相关的Kerberos设置
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.创建Kerberos数据库
1>.Kerberos数据库概述
Kerberos数据库包含领域的所有Kerberos主体，它们的密码以及有关每个主体的其他管理信息。在大多数情况下，您将使用kdb5_util 程序来整体操作Kerberos数据库，并使用 kadmin程序来更改数据库中的条目。（一个显着的例外是用户将使用 kpasswd程序来更改其自己的密码。）kadmin程序具有其自己的命令行界面，您可以在其中键入数据库管理命令。
kdb5_util提供了一种创建，删除，加载或转储Kerberos数据库的方法。它还包含用于滚动数据库主密钥并存储密钥副本的命令，以便 kadmind和krb5kdc守护程序可以使用数据库而无需手动输入。
kadmin提供了Kerberos主体，密码策略和服务密钥表（keytabs）的维护。通常，它作为网络客户端使用Kerberos身份验证与kadmind进行通信，但是还有一个名为kadmin.local的变体，可以直接（或通过LDAP）访问本地文件系统上的Kerberos数据库。必须设置kadmin.local才能设置足够的数据库才能使用远程版本。
博主推荐阅读:
https://web.mit.edu/kerberos/krb5-latest/doc/admin/database.html
2>.Kerberos支持的数据库类型
我们知道Kerberos由三个组件组成：数据库，AS和TGS。在默认情况下必须创建一个数据库，而Kerberos官方支持三种数据，默认使用的是DB2。 Berkeley database module(简称:"db2") 默认的KDB模块是db2，它使用Berkeley DB库的版本。它基于数据库路径名创建四个文件。如果路径名以结尾，principal则四个文件为： principal: 其中包含主要条目数据 principal.ok: 主体数据库的锁定文件 principal.kadm5: 包含策略对象数据 principal.kadm5.lock: 策略数据库的锁定文件 温馨提示: 对于大型数据库，kdb5_util dump命令（可能由kprop或kadmind调用以进行增量传播）可能会导致krb5kdc在遍历数据库时停止明显的时间。 可以通过禁用帐户锁定功能来避免此延迟，以使KDC不执行数据库写操作。或者可以通过将unlockiter变量设置为来启用(true)以较慢形式的迭代。 Lightning Memory-Mapped Database module(简称"klmdb") 在版本1.17中添加了klmdb模块。它使用LMDB库，并且可以提供比db2模块更好的性能和可靠性。它基于数据库路径名创建四个文件。如果路径名以结尾principal，则四个文件为： principal.mdb: 其中包含策略对象数据和大多数主要条目数据 principal.mdb-lock: 主数据库的锁定文件 principal.lockout.mdb: 其中包含每个主体条目的帐户锁定属性（上次成功通过身份验证的时间，上次失败通过身份验证的时间以及尝试失败的次数） principal.lockout.mdb-lock: 锁定数据库的锁定文件 温馨提示: 分离锁定属性可确保KDC永远不会在管理操作（如数据库转储或装入）上阻塞。它还允许KDC运行，而无需对主数据库进行写访问。如果两个帐户锁定功能都被禁用，则将创建锁定数据库文件，但随后将不会打开该数据库文件，并且帐户锁定属性将始终为零值。 因为LMDB创建到数据库文件的内存映射，所以它需要配置的内存映射大小，这也决定了数据库的最大大小。该大小将同等地应用于两个数据库，因此将在进程地址空间中使用两倍的配置大小。这主要是对32位平台的限制。128兆字节的默认值应足以容纳数十万个主体条目。 如果达到限制，则kadmin操作将失败，并且错误消息“已达到环境mapsize限制”将出现在kadmind日志文件中。在这种情况下， 可以使用mapsize变量来增加地图大小。 LDAP module(简称:"kldap") kldap模块使用LDAP服务器存储主体和策略数据。要使用它，必须配置LDAP服务器以使用Kerberos模式。
温馨提示: 由于krb5kdc是单线程的，因此LDAP数据库访问中的延迟可能会限制KDC操作吞吐量。 如果LDAP服务器与KDC位于同一服务器主机上并通过ldapi://URL访问 ，则延迟应尽可能短。如果无法做到这一点，请考虑使用krb5kdc -w选项启动多个KDC工作进程， 以启用KDC请求的并发处理。 kldap模块不支持使用kadmin lock命令进行显式锁定 。 博主推荐阅读: https://web.mit.edu/kerberos/krb5-latest/doc/admin/dbtypes.html#berkeley-database-module-db2 https://web.mit.edu/kerberos/krb5-latest/doc/admin/dbtypes.html#lightning-memory-mapped-database-module-klmdb https://web.mit.edu/kerberos/krb5-latest/doc/admin/dbtypes.html#ldap-module-kldap https://web.mit.edu/kerberos/krb5-latest/doc/admin/conf_ldap.html#conf-ldap
3>.初始化Kerberos数据库
如下图所示，初始化Kerberos数据库时需要指定一个管理员密码(记住该密码，稍后会使用)，初始化数据库通常只需要用两个选项即可，如下所示：
-r: 指定数据库的Kerberos领域。 create [-s]: 创建一个新的数据库。如果指定了-s选项，则还将创建存储文件。如果数据库已经存在，此命令将失败。如果命令成功执行，则将打开数据库，就像第一次运行程序时已经存在一样。 博主推荐阅读: https://web.mit.edu/kerberos/krb5-latest/doc/admin/admin_commands/kdb5_util.html
[root@kdc.yinzhengjie.com ~]# ll /yinzhengjie/softwares/kerberos/data/ total 4 -rw-r--r-- 1 root root 29 Oct 3 18:49 kadm5.acl [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# kdb5_util create -r YINZHENGJIE.COM -s Loading random data Initializing database '/yinzhengjie/softwares/kerberos/data/principal' for realm 'YINZHENGJIE.COM', master key name 'K/M@YINZHENGJIE.COM' You will be prompted for the database Master Password. It is important that you NOT FORGET this password. Enter KDC database master key: Re-enter KDC database master key to verify: kdb5_util: Required parameters in kdc.conf missing while initializing the Kerberos admin interface [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# ll /yinzhengjie/softwares/kerberos/data/ total 20 -rw-r--r-- 1 root root 29 Oct 3 18:49 kadm5.acl -rw------- 1 root root 8192 Oct 4 14:42 principal -rw------- 1 root root 8192 Oct 4 14:42 principal.kadm5 -rw------- 1 root root 0 Oct 4 14:42 principal.kadm5.lock -rw------- 1 root root 0 Oct 4 14:42 principal.ok [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]#
二.创建第一个用户主体(UPN)，用于管理Kerberos数据库
1>.kadmin工具概述
kadmin和kadmin.local是Kerberos V5管理系统的命令行界面。它们提供几乎相同的功能。区别在于kadmin.local直接访问KDC数据库，而kadmin使用kadmind执行操作。 博主推荐阅读: https://web.mit.edu/kerberos/krb5-latest/doc/admin/admin_commands/kadmin_local.html
2>.为KDC创建管理员主体
[root@kdc.yinzhengjie.com ~]# cat /yinzhengjie/softwares/kerberos/data/kadm5.acl */admin@YINZHENGJIE.COM * [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# kadmin.local #该命令可以直接让我们进入到一个交互式界面，便于我们配置主体相关信息。 Authenticating as principal root/admin@YINZHENGJIE.COM with password. kadmin.local: kadmin.local: list_principals K/M@YINZHENGJIE.COM kadmin/admin@YINZHENGJIE.COM kadmin/changepw@YINZHENGJIE.COM kadmin/kdc.yinzhengjie.com@YINZHENGJIE.COM kiprop/kdc.yinzhengjie.com@YINZHENGJIE.COM krbtgt/YINZHENGJIE.COM@YINZHENGJIE.COM kadmin.local: kadmin.local: kadmin.local: addprinc root/admin No policy specified for root/admin@YINZHENGJIE.COM; defaulting to no policy Enter password for principal "root/admin@YINZHENGJIE.COM": Re-enter password for principal "root/admin@YINZHENGJIE.COM": Principal "root/admin@YINZHENGJIE.COM" created. kadmin.local: kadmin.local: list_principals K/M@YINZHENGJIE.COM kadmin/admin@YINZHENGJIE.COM kadmin/changepw@YINZHENGJIE.COM kadmin/kdc.yinzhengjie.com@YINZHENGJIE.COM kiprop/kdc.yinzhengjie.com@YINZHENGJIE.COM krbtgt/YINZHENGJIE.COM@YINZHENGJIE.COM root/admin@YINZHENGJIE.COM kadmin.local: kadmin.local: quit [root@kdc.yinzhengjie.com ~]#
三.启动Kerberos服务
1>.启动Kerberos服务概述
完成Kerberos配置后，就可以启动Kerberos守护程序了。使用kadmind和krb5kdc程序启动Kerberos服务。 kadmind概述:
kadmind启动Kerberos管理服务器。kadmind通常在主Kerberos服务器上运行，该服务器存储KDC数据库。 如果KDC数据库使用LDAP模块，则管理服务器和KDC服务器无需在同一台计算机上运行。kadmind接受来自诸如kadmin和kpasswd之类的程序的远程请求，以管理这些数据库中的信息。
krb5kdc概述:
Kerberos 的KDC服务器。
博主推荐阅读: https://web.mit.edu/kerberos/krb5-latest/doc/admin/admin_commands/kadmind.html
2>.配置环境变量
[root@kdc.yinzhengjie.com ~]# cat /etc/profile.d/kerberos.sh # Add ${KERBEROS_HOME} by yinzhengjie KERBEROS_HOME=/yinzhengjie/softwares/kerberos # 指定krb5.conf的配置文件路径 export KRB5_CONFIG=${KERBEROS_HOME}/etc/krb5.conf # 指定kdc.conf的配置文件路径 export KRB5_KDC_PROFILE=${KERBEROS_HOME}/etc/kdc.conf [root@kdc.yinzhengjie.com ~]#
3>.创建符号链接
[root@kdc.yinzhengjie.com ~]# cd /var/kerberos/ [root@kdc.yinzhengjie.com /var/kerberos]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# ll total 8 -rw------- 1 root root 22 Nov 28 2019 kadm5.acl -rw------- 1 root root 451 Nov 28 2019 kdc.conf [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# mkdir bak [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# ll total 8 drwxr-xr-x 2 root root 6 Oct 5 15:25 bak -rw------- 1 root root 22 Nov 28 2019 kadm5.acl -rw------- 1 root root 451 Nov 28 2019 kdc.conf [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# mv k* bak/ #将默认的配置文件备份起来，使用咱们自己定义的配置文件 [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# ll total 0 drwxr-xr-x 2 root root 39 Oct 5 15:25 bak [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# ln -sv /yinzhengjie/softwares/kerberos/etc/kdc.conf kdc.conf ‘kdc.conf’ -> ‘/yinzhengjie/softwares/kerberos/etc/kdc.conf’ [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# ll total 0 drwxr-xr-x 2 root root 39 Oct 5 15:25 bak lrwxrwxrwx 1 root root 44 Oct 5 15:26 kdc.conf -> /yinzhengjie/softwares/kerberos/etc/kdc.conf [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# ln -sv /yinzhengjie/softwares/kerberos/etc/kadm5.acl kadm5.acl ‘kadm5.acl’ -> ‘/yinzhengjie/softwares/kerberos/etc/kadm5.acl’ [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]# ll total 0 drwxr-xr-x 2 root root 39 Oct 5 15:25 bak lrwxrwxrwx 1 root root 45 Oct 5 15:26 kadm5.acl -> /yinzhengjie/softwares/kerberos/etc/kadm5.acl lrwxrwxrwx 1 root root 44 Oct 5 15:26 kdc.conf -> /yinzhengjie/softwares/kerberos/etc/kdc.conf [root@kdc.yinzhengjie.com /var/kerberos/krb5kdc]#
4>.启动kadmind
如果您使用源码方式安装Kerberos，想要设置开机自启动可以自定义启动脚本，并编辑"/etc/rc.local"或者"/etc/inittab"文件，使得启动操作系统时自动加载您的自定义启动脚本即可。 如果您使用yum方式安装Kerberos，只需要使用Linux自带的管理工具设置开机自启动即可。如下图所示，我是通过使用systemctl工具(对该工具不熟悉的小伙伴可以参考我之前的笔记)来管理kadmin服务的。 博主推荐阅读: https://www.cnblogs.com/yinzhengjie/p/11986414.html
[root@kdc.yinzhengjie.com ~]# systemctl status kadmin ● kadmin.service - Kerberos 5 Password-changing and Administration Loaded: loaded (/usr/lib/systemd/system/kadmin.service; disabled; vendor preset: disabled) Active: inactive (dead) Oct 05 15:24:53 kdc.yinzhengjie.com systemd[1]: Starting Kerberos 5 Password-changing an..... Oct 05 15:24:53 kdc.yinzhengjie.com _kadmind[5910]: kadmind: kadmind: Cannot open DB2 da...ng Oct 05 15:24:53 kdc.yinzhengjie.com systemd[1]: kadmin.service: control process exited, ...=1 Oct 05 15:24:53 kdc.yinzhengjie.com systemd[1]: Failed to start Kerberos 5 Password-chan...n. Oct 05 15:24:53 kdc.yinzhengjie.com systemd[1]: Unit kadmin.service entered failed state. Oct 05 15:24:53 kdc.yinzhengjie.com systemd[1]: kadmin.service failed. Oct 05 15:27:01 kdc.yinzhengjie.com systemd[1]: Starting Kerberos 5 Password-changing an..... Oct 05 15:27:01 kdc.yinzhengjie.com systemd[1]: Started Kerberos 5 Password-changing and...n. Oct 05 15:32:04 kdc.yinzhengjie.com systemd[1]: Stopping Kerberos 5 Password-changing an..... Oct 05 15:32:04 kdc.yinzhengjie.com systemd[1]: Stopped Kerberos 5 Password-changing and...n. Hint: Some lines were ellipsized, use -l to show in full. [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl start kadmin [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl status kadmin ● kadmin.service - Kerberos 5 Password-changing and Administration Loaded: loaded (/usr/lib/systemd/system/kadmin.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2020-10-05 15:32:28 CST; 16s ago Main PID: 6378 (kadmind) CGroup: /system.slice/kadmin.service └─6378 /usr/sbin/kadmind -P /var/run/kadmind.pid Oct 05 15:32:28 kdc.yinzhengjie.com systemd[1]: Starting Kerberos 5 Password-changing an..... Oct 05 15:32:28 kdc.yinzhengjie.com systemd[1]: Can't open PID file /var/run/kadmind.pid...ry Oct 05 15:32:28 kdc.yinzhengjie.com systemd[1]: Started Kerberos 5 Password-changing and...n. Hint: Some lines were ellipsized, use -l to show in full. [root@kdc.yinzhengjie.com ~]#
[root@kdc.yinzhengjie.com ~]# systemctl is-enabled kadmin disabled [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl enable kadmin Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service. [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl is-enabled kadmin enabled [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]#
5>.启动krb5kdc
如果您使用源码方式安装Kerberos，想要设置开机自启动可以自定义启动脚本，并编辑"/etc/rc.local"或者"/etc/inittab"文件，使得启动操作系统时自动加载您的自定义启动脚本即可。 如果您使用yum方式安装Kerberos，只需要使用Linux自带的管理工具设置开机自启动即可。如下图所示，我是通过使用systemctl工具(对该工具不熟悉的小伙伴可以参考我之前的笔记)来管理krb5kdc服务的。 博主推荐阅读: https://www.cnblogs.com/yinzhengjie/p/11986414.html
[root@kdc.yinzhengjie.com ~]# systemctl status krb5kdc ● krb5kdc.service - Kerberos 5 KDC Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service; disabled; vendor preset: disabled ) Active: inactive (dead) Oct 05 15:50:26 kdc.yinzhengjie.com systemd[1]: Starting Kerberos 5 KDC... Oct 05 15:50:26 kdc.yinzhengjie.com systemd[1]: Can't open PID file /var/run/krb5kdc.pid...ry Oct 05 15:50:26 kdc.yinzhengjie.com systemd[1]: Started Kerberos 5 KDC. Oct 05 15:51:03 kdc.yinzhengjie.com systemd[1]: Stopping Kerberos 5 KDC... Oct 05 15:51:03 kdc.yinzhengjie.com systemd[1]: Stopped Kerberos 5 KDC. Hint: Some lines were ellipsized, use -l to show in full. [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl start krb5kdc [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl status krb5kdc ● krb5kdc.service - Kerberos 5 KDC Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2020-10-05 15:51:20 CST; 13s ago Main PID: 7417 (krb5kdc) CGroup: /system.slice/krb5kdc.service └─7417 /usr/sbin/krb5kdc -P /var/run/krb5kdc.pid Oct 05 15:51:20 kdc.yinzhengjie.com systemd[1]: Starting Kerberos 5 KDC... Oct 05 15:51:20 kdc.yinzhengjie.com systemd[1]: Can't open PID file /var/run/krb5kdc.pid...ry Oct 05 15:51:20 kdc.yinzhengjie.com systemd[1]: Started Kerberos 5 KDC. Hint: Some lines were ellipsized, use -l to show in full. [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]#
[root@kdc.yinzhengjie.com ~]# systemctl is-enabled krb5kdc disabled [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl enable krb5kdc Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service. [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# systemctl is-enabled krb5kdc enabled [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]#
四.创建服务主体(SPN)
必须为每个Hadoop服务创建一个服务主体，如HDFS和YARN，使Hadoop守护进程能够通过Kerberos认证hdfs(用于HDFS服务)，mapred(用于MapReduce)和yarn(用于YARN)。 需要为集群中的每个服务/守护程序创建一个SPN。还必须为组件服务(如Hive，Oozie等)创建服务主体。为简单起见，本篇博客只介绍hdfs主体的创建，但可以通过相同的方法创建其他主体哟。 除了HDFS之外，还创建了HTTP主体，这是使用Kerberos所需的Web通信协议(实际上hdfs，yarn，mapred这三个服务主体都使用HTTP服务)。通过为这些服务提供HTTP服务名称，可以为我们日常工作中使用的Web界面启用Kerberos身份验证。
温馨提示:
要在Hadoop集群中实现Kerberos安全性，必须在所有集群节点上配置所有集群用户，这一点很重要。或者，可以配置Active Directory中的所有用户，并让Hadoop服务器访问该目录服务。可以限制配置用户的权限，例如使用"nologin"类型的shell。
1>.创建hdfs服务主体
[root@kdc.yinzhengjie.com ~]# hostname kdc.yinzhengjie.com [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# kadmin.local Authenticating as principal root/admin@YINZHENGJIE.COM with password. kadmin.local: kadmin.local: addprinc -randkey hdfs/kdc.yinzhengjie.com WARNING: no policy specified for hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM; defaulting to no policy Principal "hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM" created. kadmin.local: kadmin.local: addprinc -randkey HTTP/kdc.yinzhengjie.com WARNING: no policy specified for HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM; defaulting to no policy Principal "HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM" created. kadmin.local: kadmin.local:
2>.查看创建的服务主体是否创建成功
[root@kdc.yinzhengjie.com ~]# kadmin.local Authenticating as principal root/admin@YINZHENGJIE.COM with password. kadmin.local: kadmin.local: kadmin.local: listprincs HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM K/M@YINZHENGJIE.COM hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM kadmin/admin@YINZHENGJIE.COM kadmin/changepw@YINZHENGJIE.COM kadmin/kdc.yinzhengjie.com@YINZHENGJIE.COM kiprop/kdc.yinzhengjie.com@YINZHENGJIE.COM krbtgt/YINZHENGJIE.COM@YINZHENGJIE.COM root/admin@YINZHENGJIE.COM kadmin.local: kadmin.local: getprinc hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM Principal: hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM Expiration date: [never] Last password change: Mon Oct 05 19:14:25 CST 2020 Password expiration date: [never] Maximum ticket life: 2 days 00:00:00 Maximum renewable life: 7 days 00:00:00 Last modified: Mon Oct 05 19:14:25 CST 2020 (root/admin@YINZHENGJIE.COM) Last successful authentication: [never] Last failed authentication: [never] Failed password attempts: 0 Number of keys: 4 Key: vno 1, aes256-cts-hmac-sha1-96 Key: vno 1, aes128-cts-hmac-sha1-96 Key: vno 1, des3-cbc-sha1 Key: vno 1, arcfour-hmac MKey: vno 1 Attributes: Policy: [none] kadmin.local: kadmin.local:
3>.查看kadmin.local交互式界面的帮助信息
如下图所示，当你对kadmin.local的子命令不太熟悉时，可以查看相应的帮助信息。当然，如果你想要看更详细的文档可参考官方文档。这一方法无论对Kerberos使用者的新手或老手都很有用！
博主推荐阅读: https://web.mit.edu/kerberos/krb5-latest/doc/admin/admin_commands/kadmin_local.html#commands
[root@kdc.yinzhengjie.com ~]# kadmin.local Authenticating as principal root/admin@YINZHENGJIE.COM with password. kadmin.local: kadmin.local: ? Available kadmin.local requests: add_principal, addprinc, ank Add principal delete_principal, delprinc Delete principal modify_principal, modprinc Modify principal rename_principal, renprinc Rename principal change_password, cpw Change password get_principal, getprinc Get principal list_principals, listprincs, get_principals, getprincs List principals add_policy, addpol Add policy modify_policy, modpol Modify policy delete_policy, delpol Delete policy get_policy, getpol Get policy list_policies, listpols, get_policies, getpols List policies get_privs, getprivs Get privileges ktadd, xst Add entry(s) to a keytab ktremove, ktrem Remove entry(s) from a keytab lock Lock database exclusively (use with extreme caution!) unlock Release exclusive database lock purgekeys Purge previously retained old keys from a principal get_strings, getstrs Show string attributes on a principal set_string, setstr Set a string attribute on a principal del_string, delstr Delete a string attribute on a principal list_requests, lr, ? List available requests. quit, exit, q Exit program. kadmin.local: kadmin.local:
五.创建keytab文件
1>.keytab文件功能概述
每个服务主体都需要一个keytab文件来存储其密码。keytab文件包含Kerberos主体和根据其Kerberos密码派生的加密密钥对。当服务正常运行时，其用于KDC身份验证。
UPN需要使用kinit登录到安全集群，然后提供验证密码。但是SPN无法进行交互式登录尝试。keytab文件存储可用于特定SPN的加密密钥，可以将多个SPN密钥存储在同一个keytab文件中。
如下图所示，实际上，普通用户(UPN)也可以使用keytab文件代替在登录时提供的密码。
温馨提示:
必须小心保护keytab文件，因为它们掌管着大门的钥匙，尤其是管理员主体的！
[root@hadoop101.yinzhengjie.com ~]# klist klist: No credentials cache found (filename: /tmp/krb5cc_0) [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# ll total 4 -rw------- 1 root root 714 Oct 5 20:19 hdfs.keytab [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# kinit -kt hdfs.keytab hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM Valid starting Expires Service principal 10/05/2020 20:22:17 10/06/2020 20:35:38 krbtgt/YINZHENGJIE.COM@YINZHENGJIE.COM renew until 10/12/2020 20:22:17 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
2>.创建keytab文件
yarn，hdfs和mapred等用户都运行后台Hadoop守护进程，因此需要为这些服务主体创建一个keytab文件。此外，需要为http主体创建一个keytab文件，从而Kerberos可以认证Hadoop的Web UI。
使用kadmin.local命令创建keytab文件。不需要从头创建keytab文件。使用kadmin命令可以指定"xst -k"选项来提取每个服务主体的keytab文件，并将其放在该服务主体的keytab目录中。
每个服务主体的keytab文件都是以主体唯一命名的，例如"hdfs.keytab","yarn.keytab","hue.keytab","hive.keytab"和"http.keytab"等等。必须为Hadoop节点上每个Hadoop守护程序导出单独的keytab文件。
下面的案例是为HDFS服务创建keytab文件，可以执行以下操作。
[root@kdc.yinzhengjie.com ~]# ll total 0 [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# kadmin.local Authenticating as principal root/admin@YINZHENGJIE.COM with password. kadmin.local: kadmin.local: xst -k hdfs.keytab hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM Entry for principal hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:hdfs.keytab. Entry for principal hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:hdfs.keytab. Entry for principal hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:hdfs.keytab. Entry for principal hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:hdfs.keytab. Entry for principal HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:hdfs.keytab. Entry for principal HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:hdfs.keytab. Entry for principal HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:hdfs.keytab. Entry for principal HTTP/kdc.yinzhengjie.com@YINZHENGJIE.COM with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:hdfs.keytab. kadmin.local: kadmin.local: quit [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# ll total 4 -rw------- 1 root root 714 Oct 5 19:59 hdfs.keytab [root@kdc.yinzhengjie.com ~]#
3>.将服务主体的keytab文件移动到HDFS集群的每一个节点上
[root@kdc.yinzhengjie.com ~]# ll total 4 -rw------- 1 root root 714 Oct 5 19:59 hdfs.keytab [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# [root@kdc.yinzhengjie.com ~]# scp hdfs.keytab hadoop101.yinzhengjie.com:~ The authenticity of host 'hadoop101.yinzhengjie.com (172.200.6.101)' can't be established. ECDSA key fingerprint is SHA256:y6iS5ipSyWSGRmgcjivbWhd78pKfrcuQHeBPd5H9/U8. ECDSA key fingerprint is MD5:da:0f:2a:93:c0:d4:6e:7e:13:16:61:f1:93:a7:38:01. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'hadoop101.yinzhengjie.com,172.200.6.101' (ECDSA) to the list of known hosts. root@hadoop101.yinzhengjie.com's password: hdfs.keytab 100% 714 307.4KB/s 00:00 [root@kdc.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# ll total 4 -rw------- 1 root root 714 Oct 5 20:19 hdfs.keytab [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# kinit -kt hdfs.keytab hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: hdfs/kdc.yinzhengjie.com@YINZHENGJIE.COM Valid starting Expires Service principal 10/05/2020 20:19:31 10/06/2020 20:32:52 krbtgt/YINZHENGJIE.COM@YINZHENGJIE.COM renew until 10/12/2020 20:19:31 [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# ansible all -m shell -a 'mkdir /yinzhengjie/softwares/hadoop/etc/hadoop/conf' [WARNING]: Consider using the file module with state=directory rather than running 'mkdir'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. hadoop101.yinzhengjie.com | FAILED | rc=1 >> mkdir: cannot create directory ‘/yinzhengjie/softwares/hadoop/etc/hadoop/conf’: File existsnon-zero return code hadoop102.yinzhengjie.com | CHANGED | rc=0 >> hadoop104.yinzhengjie.com | CHANGED | rc=0 >> hadoop105.yinzhengjie.com | CHANGED | rc=0 >> hadoop103.yinzhengjie.com | CHANGED | rc=0 >> [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# ansible all -m shell -a 'ls -l /yinzhengjie/softwares/hadoop/etc/hadoop/conf' hadoop102.yinzhengjie.com | CHANGED | rc=0 >> total 0 hadoop101.yinzhengjie.com | CHANGED | rc=0 >> total 8 -rw-r--r-- 1 root root 115 Aug 13 18:55 host-rack.txt -rwxr-xr-x 1 root root 463 Aug 13 18:54 toplogy.py hadoop104.yinzhengjie.com | CHANGED | rc=0 >> total 0 hadoop105.yinzhengjie.com | CHANGED | rc=0 >> total 0 hadoop103.yinzhengjie.com | CHANGED | rc=0 >> total 0 [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# ll total 4 -rw------- 1 root root 714 Oct 5 20:19 hdfs.keytab [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# ansible all -m copy -a 'src=~/hdfs.keytab dest=/yinzhengjie/softwares/hadoop/etc/hadoop/conf' hadoop102.yinzhengjie.com | CHANGED => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": true, "checksum": "4895c1853599cf70ad2cde9e3606c5b160591623", "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs.keytab", "gid": 0, "group": "root", "md5sum": "3f997c0430da2208ccc1e617d3145d3e", "mode": "0644", "owner": "root", "size": 714, "src": "/root/.ansible/tmp/ansible-tmp-1601900886.29-6090-277128142165633/source", "state": "file", "uid": 0 } hadoop101.yinzhengjie.com | CHANGED => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": true, "checksum": "4895c1853599cf70ad2cde9e3606c5b160591623", "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs.keytab", "gid": 0, "group": "root", "md5sum": "3f997c0430da2208ccc1e617d3145d3e", "mode": "0644", "owner": "root", "size": 714, "src": "/root/.ansible/tmp/ansible-tmp-1601900886.31-6094-151130876303606/source", "state": "file", "uid": 0 } hadoop104.yinzhengjie.com | CHANGED => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": true, "checksum": "4895c1853599cf70ad2cde9e3606c5b160591623", "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs.keytab", "gid": 0, "group": "root", "md5sum": "3f997c0430da2208ccc1e617d3145d3e", "mode": "0644", "owner": "root", "size": 714, "src": "/root/.ansible/tmp/ansible-tmp-1601900886.3-6093-34011526978905/source", "state": "file", "uid": 0 } hadoop105.yinzhengjie.com | CHANGED => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": true, "checksum": "4895c1853599cf70ad2cde9e3606c5b160591623", "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs.keytab", "gid": 0, "group": "root", "md5sum": "3f997c0430da2208ccc1e617d3145d3e", "mode": "0644", "owner": "root", "size": 714, "src": "/root/.ansible/tmp/ansible-tmp-1601900886.31-6096-110162310342304/source", "state": "file", "uid": 0 } hadoop103.yinzhengjie.com | CHANGED => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python" }, "changed": true, "checksum": "4895c1853599cf70ad2cde9e3606c5b160591623", "dest": "/yinzhengjie/softwares/hadoop/etc/hadoop/conf/hdfs.keytab", "gid": 0, "group": "root", "md5sum": "3f997c0430da2208ccc1e617d3145d3e", "mode": "0644", "owner": "root", "size": 714, "src": "/root/.ansible/tmp/ansible-tmp-1601900886.32-6092-251321354687851/source", "state": "file", "uid": 0 } [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# ansible all -m shell -a 'ls -l /yinzhengjie/softwares/hadoop/etc/hadoop/conf' hadoop104.yinzhengjie.com | CHANGED | rc=0 >> total 4 -rw-r--r-- 1 root root 714 Oct 5 20:28 hdfs.keytab hadoop105.yinzhengjie.com | CHANGED | rc=0 >> total 4 -rw-r--r-- 1 root root 714 Oct 5 20:28 hdfs.keytab hadoop103.yinzhengjie.com | CHANGED | rc=0 >> total 4 -rw-r--r-- 1 root root 714 Oct 5 20:28 hdfs.keytab hadoop102.yinzhengjie.com | CHANGED | rc=0 >> total 4 -rw-r--r-- 1 root root 714 Oct 5 20:28 hdfs.keytab hadoop101.yinzhengjie.com | CHANGED | rc=0 >> total 12 -rw-r--r-- 1 root root 714 Oct 5 20:28 hdfs.keytab -rw-r--r-- 1 root root 115 Aug 13 18:55 host-rack.txt -rwxr-xr-x 1 root root 463 Aug 13 18:54 toplogy.py [root@hadoop101.yinzhengjie.com ~]#
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。 欢迎加入基础架构自动化运维：598432640，大数据SRE进阶之路：959042252，DevOps进阶之路：526991186
