ambari 2.7.5 安装部署和踩坑记录
ambari2.7.5 兼容 HDP的版本:
2.7.4 兼容 HDP3.1.4
2.7.5 兼容 HDP3.1.5 部分兼容HDP3.1.4 仅用于升级 partianlly compatible for upgreade only
HDP3.1.5的资源:链接：https://pan.baidu.com/s/17GRLLp3QG3g3hgI3KZcaZA 提取码：81ka
1.创建本地源
2.配置mysql驱动包
3.配置ambari
4.登录页面开始配置集群
具体操作步骤
--使用本地yum源
#将enabled=1改为enabled=0
sudo vim /etc/yum/pluginconf.d/langpacks.conf
ds3重启检查selinux服务是否关闭 getenforce 配置文件已经修改 需要重启
ntpd时间同步服务已安装
修改yum源为alibaba
关闭THP服务 需要重启(可以不关)
vim /etc/rc.d/rc.local
#添加
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
exit 0
#修改完成之后赋予执行权限
chmod +x /etc/rc.d/rc.local
查看是否被禁用
[root@localhost ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
[root@localhost ~]# cat /sys/kernel/mm/transparent_hugepage/defrag
always madvise [never]
创建ambari本地源
本地源目录:/var/www/html
ds1
解压目录:
/var/www/html/ambari-2.7.5
/var/www/html/hdp-3.1.5
/var/www/html/hdp-GPL-3.1.5
/var/www/html/hdp-utils-1.1.0.22
安装包目录:
/opt/ambari-2.7.5/ambari-2.7.5.0-centos7.tar.gz
/opt/ambari-2.7.5/HDP-3.1.5.0-centos7-rpm.tar.gz
/opt/ambari-2.7.5/HDP-GPL-3.1.5.0-centos7-gpl.tar.gz
/opt/ambari-2.7.5/HDP-UTILS-1.1.0.22-centos7.tar.gz
解压
sudo tar -zxvf /opt/ambari-2.7.5/ambari-2.7.5.0-centos7.tar.gz -C /var/www/html/ambari-2.7.5
sudo tar -zxvf /opt/ambari-2.7.5/HDP-3.1.5.0-centos7-rpm.tar.gz -C /var/www/html/hdp-3.1.5
sudo tar -zxvf /opt/ambari-2.7.5/HDP-GPL-3.1.5.0-centos7-gpl.tar.gz -C /var/www/html/hdp-GPL-3.1.5
sudo tar -zxvf /opt/ambari-2.7.5/HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp-utils-1.1.0.22
创建本地源
cd /var/www/html/
createrepo
vim /etc/yum.repos.d/ambari.repo
#VERSION_NUMBER=2.7.5.0-72
[ambari-2.7.5.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.5.0
baseurl=http://192.168.10.103/ambari-2.7.5/ambari/centos7/2.7.5.0-72/
gpgcheck=0
gpgkey=http://192.168.10.103/ambari-2.7.5/ambari/centos7/2.7.5.0-72/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
vim /etc/yum.repos.d/hdp.repo
#VERSION_NUMBER=3.1.5.0-152
[HDP-3.1.5.0]
name=HDP Version - HDP-3.1.5.0
baseurl=http://192.168.10.103/hdp-3.1.5/HDP/centos7/3.1.5.0-152/
gpgcheck=0
gpgkey=http://192.168.10.103/hdp-3.1.5/HDP/centos7/3.1.5.0-152/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://192.168.10.103/hdp-utils-1.1.0.22/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=0
gpgkey=http://192.168.10.103/hdp-utils-1.1.0.22/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
#VERSION_NUMBER=3.1.5.0-152
[HDP-GPL-3.1.5.0]
name=HDP-GPL Version - HDP-GPL-3.1.5.0
baseurl=http://192.168.10.103/hdp-GPL-3.1.5/HDP-GPL/centos7/3.1.5.0-152/
gpgcheck=0
gpgkey=http://192.168.10.103/hdp-GPL-3.1.5/HDP-GPL/centos7/3.1.5.0-152/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
vim /etc/yum.repos.d/hdp.gpl.repo
#VERSION_NUMBER=3.1.5.0-152
[HDP-GPL-3.1.5.0]
name=HDP-GPL Version - HDP-GPL-3.1.5.0
baseurl=http://192.168.10.103/hdp-GPL-3.1.5/HDP-GPL/centos7/3.1.5.0-152/
gpgcheck=0
gpgkey=http://192.168.10.103/hdp-GPL-3.1.5/HDP-GPL/centos7/3.1.5.0-152/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
#修改完之后 更新一下yum源
yum clean all
yum makecache
#查看是否有Ambari
yum repolist
#创建mysql元数据库
#安装ambari-server 启动安装程序
yum install ambari-server
#配置ambari mysql驱动包
#将mysql-connector-java.jar复制到/usr/share/java目录下并改名为mysql-connector-java.jar
103已经有了
#mkdir /usr/share/java
#cp /opt/software/mysql-libs/mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar /usr/share/java/mysql-connector-java.jar
103也有了
#将mysql-connector-java.jar复制到/var/lib/ambari-server/resources目录下并改名为mysql-jdbc-driver.jar
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar
#ambari配置文件添加mysql驱动包地址
vim /etc/ambari-server/conf/ambari.properties
server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar
#执行ambri自带的mysql建表脚本
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;
************************************************************************
#配置ambari
ambari-server setup
#启动ambari
ambari-server start
#登录页面开始配置
http://ds1:8080/
ds2:不用配置
ds3:不用配置
#修改hdfs目录权限 在网页上也能访问
sudo -u hdfs hadoop fs -chmod -R 755 /app-logs
--创建的数据库赋权
create database ranger;
GRANT ALL PRIVILEGES ON ranger.* TO 'ranger'@'%' IDENTIFIED BY 'ranger';
GRANT ALL PRIVILEGES ON ranger.* TO 'ranger'@'localhost' IDENTIFIED BY 'ranger';
flush privileges;
create database rangerkms;
GRANT ALL PRIVILEGES ON rangerkms.* TO 'rangerkms'@'%' IDENTIFIED BY 'rangerkms';
GRANT ALL PRIVILEGES ON rangerkms.* TO 'rangerkms'@'localhost' IDENTIFIED BY 'rangerkms';
flush privileges;
踩坑记录: 部署完ranger之后,启动ranger kms的时候,虽然在页面显示命令执行完毕,但是kms一直都没有正常启动成功,查看日志发现一直都是Illegal key size这个错误.
java.security.InvalidKeyException: Illegal key size
at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1039)
at javax.crypto.Cipher.implInit(Cipher.java:805)
at javax.crypto.Cipher.chooseProvider(Cipher.java:864)
at javax.crypto.Cipher.init(Cipher.java:1396)
at javax.crypto.Cipher.init(Cipher.java:1327)
at org.apache.hadoop.crypto.key.RangerMasterKey.encryptKey(RangerMasterKey.java:438)
at org.apache.hadoop.crypto.key.RangerMasterKey.encryptMasterKey(RangerMasterKey.java:391)
at org.apache.hadoop.crypto.key.RangerMasterKey.generateMasterKey(RangerMasterKey.java:199)
at org.apache.hadoop.crypto.key.RangerKeyStoreProvider.<init>(RangerKeyStoreProvider.java:135)
at org.apache.hadoop.crypto.key.RangerKeyStoreProvider$Factory.createProvider(RangerKeyStoreProvider.java:455)
at org.apache.hadoop.crypto.key.KeyProviderFactory.get(KeyProviderFactory.java:96)
at org.apache.hadoop.crypto.key.kms.server.KMSWebApp.contextInitialized(KMSWebApp.java:185)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5197)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5720)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1707)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1697)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)解决办法: 这是由于java原生的两个jar包 $JAVA_HOME/jre/lib/security的local_policy.jar和US_export_policy.jar 密码策略长度不够导致的,需要去官网下载加强包替换一下这俩jar包,然后重启就好了
JCE Unlimited Strength Jurisdiction Policy Files for JDK/JRE 8 Download (oracle.com)
绝不摆烂
