离线方式部署Ambari2.6.0.0
Hadoop生态圈-离线方式部署Ambari2.6.0.0
作者:尹正杰
版权声明:原创作品,谢绝转载!否则将追究法律责任。
我现在所在的公司用的是CDH管理Hadoop集群,前端时间去面试时发现很多企业都在用HDP,于是回来捣鼓来一下HDP的部署方式。部署时也踩过不少坑,这里把正确部署Ambari的方式记录一下。其实部署Ambari的方式和CDH的部署方式大同小异。
本片博客会详细介绍如何使用Ambari来安装HDP发行版本,想要避免去官网踩坑的小伙伴可以这篇笔记很适合你哟!HDP是Hontorworks Data Platform的简称,是Hontorworks公司的Hadoop发行版本,在企业中十分流行。同样著名的还有Cludera公司的CDH发行版本(详情请参考我的笔记:离线方式部署Cloudera Manager5.15.1)。
一.Ambari简介
和CDH一样,Ambari是一款用于部署,管理和监控Hadoop集群服务的开源系统,它实现来以下功能:
1>.安装一个Hadoop集群:
第一:提供了以向导指引的方式安装一个集群,可以在任意的主机上安装Hadoop服务。
第二:提供了对Hadoop服务的配置功能。
2>.管理一个Hadoop集群:
提供了启动,停止等集群管理功能。
3>.监控Hadoop集群:
第一:提供了一个监控Hadoop集群监控状态的仪表盘。
第二:提供了一套监控指标体系来手机监控数据。
第三:提供了一套预警框架,可以结合预定的监控指标实现通知告警。
和CDH很像,从设计上看Ambari使用的Master/Slaves架构(主/从架构,由一个Ambari-Server和多个Agent组成)。如下图所示,它是通过一个Server主进程来实现集群的管理和操作命令的发送,而具体管理动作则由安装在每台目标主机上的Agent进程进行执行。例如通过Ambari启动HDFS服务的时候,首先会由Ambari-Server向安装来HDFS服务所在主机的Agent进程发送启动指令,然后再有相关Agent进程执行其所在主机的本地命令来启动HDFS的相应服务。
二.集群服务规划设计
在正式安装之前,我们还需要完成一些前期准备工作,包括集群服务的设计,服务器环境准备和用于安装Ambari的本地yum仓库等。
yum(全称为Yellow dog Updater,Modified)是一个在Fedora和RedHat以及CentOS中的Shell前端软件包管理器。基于RPM包管理,能够从指定的服务器自动下载RPM包并且安装,可以自动处理依赖性关系,并且一次安装所有的以来的软件包,无须繁琐地一次次下载,安装。
在了解了集群服务器角色的职责划分之后,现在可以开始设计以下集群节点的服务划分了。Ambari通过集成HDP这样一个Hadoop发行版安装包来实现对应组件服务的安装。接下来,我们看看截止到HDP2.5版本,它都提供了那些核心服务。
1>.NameNode
分布式文件系统HDFS主控服务,主要用于保存文件系的元数据信息以及操作路由。
2>.SecondNameNode
专职用于帮助NameNode合并快照日志文件的服务,以缓解NameNode压力。
3>.DataNodes
分布式文件系统HDFS的数据存储服务,主要你提供文件数据块的存储和查询功能。
4>.ResourceManger
资源调度系统YARN的主控服务,负责统一的资源分配和调度工作。
5>.NodeManagers
资源调度系统YARN的容器服务,负责具体的任务执行工作,YARN只会将任务分配到安装了NodeManager服务的主机执行任务。
6>.App Timeline Server
负责存储在YARN中执行任务的历史信息,包括任务状态,任务日志等,并提供对这些信息查询的能力。
7>.Histroy Server
负责存储在YARN中执行的任务的历史信息,包括任务状态,任务日志等,并提供对这些信息查询的能力。
8>.Hive Metastore
负责存储数据仓库Hive的元数据信息,例如表名称,表字段和字段属性等。
9>.HiveServer2
Hive在线查询服务,支持通过JDBC和ODBC等标准协议查询数据。
10>.Tez
使用DAG作为计算模型的一款分布式计算引擎,HDP版本里Hive组件的计算引擎使用的正式Tez。
11>.Active HBase Master
分布式NoSQL服务,HBase的主控服务,主要负责Region的分配和DDL操作。
12>.RegionServers
分布式NoSQL服务,HBase的数据存储服务,主要提供具体数据的存储与查询功能。
13>.Zookeeper server
分布式协同管理服务,负责HBase,Ambari Infra 和Kafka等多项服务的协同工作。
14>.Kafka Broker
分布式消息系统Kafka,负责消息的接受,发送,消息主题的管理等工作。
15>.Spark History Server
分布式内存计算框架Spark的任务历史服务,负责存储记录Spark任务的历史信息,包括任务状态,执行时间,任务日志和环境变量等信息。同时也提供这些历史细腻下的查看检索功能。
16>.Spark Thrift Servers
分布式内存计算框架Spark的在线查询服务,和Hive的HiveServer2服务类似。支持通过JDBC和OBDC等标准协议查询数据。
17>.Knox Gateway
网关服务,可以代理HDP中YARN,HBase,HDFS等众多组件的Restful服务。
18>.Ranger Admin
权限管理服务,可以代理YARN,HBase,HDFS等众多组件的权限管理功能。
19>.Ranger KMS Server
Ranger的Key管理服务,用于管理Hadoop集群的认证密钥。
20>.Ranger Usersync
Rabfer的用户同步服务,负责将Hadoop的集群用户同步更新到Ranger的内部服务。
21>.Zepplin
交互式在线分析工具,可以直接通过Web界面编写SQL或代码的形式使用Spark,Hive等服务进行交互式分析。
22>.Log Search
日志采集与分析服务,负责收集所有Hadoop服务的日志文件并提供集中展示和检索功能。
23>.Ambari Infra
搜索服务,用于存储Ranger的审计日志以及Log Search采集到的系统日志等数据。
24>.Clients
各种服务的客户端程序,包括MapReduce2 Clients,YARN Clients,Hive Clients,Zookeeper Clients,Spark Clients,HBase Clients等等。
在了解上述Ambari可以安装的核心服务之后,我们可以根据服务列表自行添加服务,这和我们之前部署CDH自定义需要安装服务几乎如出一辙。想必部署过CDH的小伙伴,对这个理解起来也非常简单的,我在这里就不多费口舌啦~
三.环境准备
1>.Ambari目前只支持64位的操作系统,具体操作如下:
1.1>.RHEL(Redhat Enterprise Linux)6或者7;
1.2>.CentOS6或者CentOS7;
1.3>.OEL(Oracle Enterprise Linux)6或者7;
1.4>.SLES(SuSE Linux Enterprise Server)11;
1.5>.Debian 7;
1.6>.Ubantu12 或者14;
2>.配置主机列表(每台主机都需要配置,我本次实验共计3台服务器)
Ambari所管理的各个服务之间需要使用FQDN来进行访问,所以我们还需要为各个服务器配置他们的FQDN。Linux系统可以通过主机表配置它的 FQDN,编辑“/etc/hosts”这个配置文件即可。
[root@node101 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn [root@node101 ~]# [root@node101 ~]#
FQDN(Fully Qualified Domain Name):
完全合格域名/全称域名,是指主机名加上全路径,全路径中列出了序列中所有域成员。全域名可以从逻辑上准确地表示处主机在什么地方,也可以说全域名是主机名的一种完全表示形式。
我都实验环境配置了3台服务器,3台服务器都相同,如下图:
3>.各个主机配置免密登陆(在“node101.yinzhengjie.org.cn”上操作)
[root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa Generating public/private rsa key pair. Created directory '/root/.ssh'. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: d1:90:58:b3:ca:97:0e:91:f2:15:23:73:7d:05:d4:53 root@node101.yinzhengjie.org.cn The key's randomart image is: +--[ RSA 2048]----+ | ooBo .o+..E| | .=.B. . o | | . o + .. . | | + + o | | = S | | + | | . | | | | | +-----------------+ [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# ll ~/.ssh/id_rsa -rw-------. 1 root root 1675 Dec 5 02:35 /root/.ssh/id_rsa [root@node101 ~]# [root@node101 ~]# cat ~/.ssh/id_rsa -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAwtFCswPu3AOKXa2TzJRBKG/LC8knCtLE54zMVUaMwbWH0fdV xHpa8n+V15W3HwLFBZDgyNmnPvZkVJUvTYqf5QaLsxm5J0JAxlmBkY/ayByeULLw SVIXM/hqBulYnbUViVhGEmBbpI3ASjFVXQyiihyNhzoPZBjdiexnTh7aYJDjQ8qL IZyQu+rjeoqju839dUmm4RfmqgaPsa717CNxqUgUdviTdK2meV4hA7HtKAxMTpYK JcEAu3lvh9NRT2k8HSMvEaN/WVUF8SrNN1oUC4aFSO9tDmY3euBCccYf+RqSlzys m5dxCFDk94cEhsvIzfdzUB4usPs1inUHcQ9pqQIDAQABAoIBAAWbPA4myNLwDyVO /3fdtFpTjUZTjZoUIrJ/LMovc5GHhdcXBRLYapGb9lXz6wcrbzwPxUhCazu2n4CH oorauGtA2vhWfyuQDokmvkHQH/2uRSgRXjTd9ej69yInyPMQ6QPA1JNmznxXz/bS oKBC4tKSq8f+db7tsEJt2TM4VBR/Sg0x77LhZA70MC2+vv+9AoIKVVdZ/KNud+Dk r2c+Hmo4r7I4G2Oqaj33f7RXFsILIwl0SBvpag4GPhwnOuphXDr3DQ1v5j+4x0rx 3JE+HmlpU66XveUuy7pMurdbOCajwPQgLkDo5W7bBUMIlBS/nZvInncVdDPzWFHO ZlUh8dkCgYEA6Ykewa0oN96XLPdqbADHRqpzxL7zXJJ644Hh2HygyhR0xw3bOI57 IjkYje9e5PVUcXWuOj/oOMfKal2jW/78bnEgukX1JgkX8RHT9zsgZ0AGEpquHkjv NrB1qrdbEwnNpSLB8pwGveD9zf89yzocZE+5CUBfnsuAgZqPYkjPMwcCgYEA1Y6x o1ZxxsPGZIk+hZPuynemdZYnQWp3Z9ynVv902x1/sxcn/uZ+0DZDLQbqGA+2WdEg EQZ9BwKTCkThoPBJKEzR3Ck122ahqSKCEQRGwK0h05uaA3KSTGIbujGaoBfDWG6f 3fTSWZnTNaObmcOhxTgsMRegy/kWOIgYqdX04c8CgYA9rsf+49uBljtb1yW2zSXV VmQpjT3+L3ph6QZcBd76QRjvEovajlebnHUN2DG6S8o4BZs1K3gMt0iRe5YQUHMg QKUiTiQIHdLDC7Bu1qEiZFi2RUDEIwN78HPD2ZYmmypD+cA8clWdhXXEGSybJwgk vqHi/avD54BTux0wGocPtQKBgFY2Ud4nsqzUMg46HaRdUSQpY2jK4jQVX/js6h1D 6Y/fe8L30SylLoKttTbDONPX2/ysYO6emPCwiF236wJGnrIHOR1RAa3947tc8/cc q9dyzVoY0fccVJVDcgAbsFl63mCf4CgBgr2rTMm1r8ZGZNMyVxzGdiKB/wd5YcOG XWKfAoGAYpvjXzEYzr7Y7KKlZBR3gUZZwfngYYgf8g1L42oB1un1NUajiMCyPsm6 oxKrFj6a8SH1xy1U0S7QbPK7osjR5qU7SMsqgziouBl1UmAT5WVKoik4UvHLlKC3 hB0HxUvfVYM5lV3BaggVTC/EOAOsa+jqfbrwjBxjn/kmmv2FtLk= -----END RSA PRIVATE KEY----- [root@node101 ~]# [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node101.yinzhengjie.org.cn The authenticity of host 'node101.yinzhengjie.org.cn (172.30.1.101)' can't be established. ECDSA key fingerprint is e5:24:dc:52:ee:e6:ed:83:de:45:39:a8:c4:14:27:e7. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node101.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node101.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]# [root@node101 ~]# ssh node101.yinzhengjie.org.cn Last login: Wed Dec 5 02:42:08 2018 from 172.30.1.2 [root@node101 ~]# who root pts/0 2018-12-05 02:42 (172.30.1.2) root pts/1 2018-12-05 02:49 (node101.yinzhengjie.org.cn) [root@node101 ~]# [root@node101 ~]# logout Connection to node101.yinzhengjie.org.cn closed. [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node102.yinzhengjie.org.cn /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node102.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node102.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]# [root@node101 ~]# ssh node102.yinzhengjie.org.cn Last login: Wed Dec 5 02:51:08 2018 from 172.30.1.101 [root@nod102 ~]# [root@nod102 ~]# who root pts/0 2018-12-05 01:29 (172.30.1.2) root pts/1 2018-12-05 02:51 (172.30.1.101) [root@nod102 ~]# [root@nod102 ~]# logout Connection to node102.yinzhengjie.org.cn closed. [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node103.yinzhengjie.org.cn /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node103.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node103.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]# [root@node101 ~]# ssh node103.yinzhengjie.org.cn Last login: Wed Dec 5 02:52:15 2018 from 172.30.1.101 [root@node103 ~]# [root@node103 ~]# [root@node103 ~]# who root pts/0 2018-12-05 01:27 (172.30.1.2) root pts/1 2018-12-05 02:52 (172.30.1.101) [root@node103 ~]# [root@node103 ~]# logout Connection to node103.yinzhengjie.org.cn closed. [root@node101 ~]#
4>.编写集群管理脚本
在配置FQDN和免密钥登陆后,咱们就可以编写一些集群的管理脚本,便于我们下发文件到其他节点中。
[root@node101 ~]# vi /usr/local/bin/xcall.sh #我们创建该脚本时,直接就把它放在默认的“PATH”环境变量中。因此编辑好后只要有执行权限就可以在任意路径调用它啦! [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# cat /usr/local/bin/xcall.sh #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #判断用户是否传参 if [ $# -lt 1 ];then echo "请输入参数" exit fi #获取用户输入的命令 cmd=$@ for (( i=101;i<=103;i++ )) do #使终端变绿色 tput setaf 2 echo ============= node${i}.yinzhengjie.org.cn : $cmd ============ #使终端变回原来的颜色,即白灰色 tput setaf 7 #远程执行命令 ssh node${i}.yinzhengjie.org.cn $cmd #判断命令是否执行成功 if [ $? == 0 ];then echo "命令执行成功" fi done [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# chmod +x /usr/local/bin/xcall.sh #别忘记添加执行权限哟! [root@node101 ~]# [root@node101 ~]# xcall.sh cat /etc/redhat-release #我们在这里可以测试一些咱们的脚本是否好使~ ============= node101.yinzhengjie.org.cn : cat /etc/redhat-release ============ CentOS Linux release 7.2.1511 (Core) 命令执行成功 ============= node102.yinzhengjie.org.cn : cat /etc/redhat-release ============ CentOS Linux release 7.2.1511 (Core) 命令执行成功 ============= node103.yinzhengjie.org.cn : cat /etc/redhat-release ============ CentOS Linux release 7.2.1511 (Core) 命令执行成功 [root@node101 ~]# [root@node101 ~]#
[root@node101 ~]# xcall.sh "yum -y install rsync" ============= node101.yinzhengjie.org.cn : yum -y install rsync ============ Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.163.com Resolving Dependencies --> Running transaction check ---> Package rsync.x86_64 0:3.1.2-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: rsync x86_64 3.1.2-4.el7 base 403 k Transaction Summary ================================================================================ Install 1 Package Total download size: 403 k Installed size: 815 k Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : rsync-3.1.2-4.el7.x86_64 1/1 Verifying : rsync-3.1.2-4.el7.x86_64 1/1 Installed: rsync.x86_64 0:3.1.2-4.el7 Complete! 命令执行成功 ============= node102.yinzhengjie.org.cn : yum -y install rsync ============ Loaded plugins: fastestmirror Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast http://mirrors.njupt.edu.cn/centos/7.6.1810/os/x86_64/repodata/bc140c8149fc43a5248fccff0daeef38182e49f6fe75d9b46db1206dc25a6c1c-c7-x86_64-comps.xml.gz: [Errno 12] Timeout on http://10.10.254.10/cache/5/02/mirrors.njupt.edu.cn/033b2636ff3cf3103169fa739764fdd7/bc140c8149fc43a5248fccff0daeef38182e49f6fe75d9b46db1206dc25a6c1c-c7-x86_64-comps.xml.gz: (28, 'Connection timed out after 30682 milliseconds') Trying other mirror. http://mirrors.njupt.edu.cn/centos/7.6.1810/updates/x86_64/repodata/8e273c72093a2cfc3f3b2baccfb2b01f7f4bcdc1bb72d53e38074068a756f1c6-primary.sqlite.bz2: [Errno 12] Timeout on http://10.10.254.10/cache/4/02/mirrors.njupt.edu.cn/c35de49cce30395fb97f6c41a56fbad5/8e273c72093a2cfc3f3b2baccfb2b01f7f4bcdc1bb72d53e38074068a756f1c6-primary.sqlite.bz2: (28, 'Connection timed out after 30672 milliseconds') Trying other mirror. Determining fastest mirrors * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.neusoft.edu.cn Resolving Dependencies --> Running transaction check ---> Package rsync.x86_64 0:3.1.2-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: rsync x86_64 3.1.2-4.el7 base 403 k Transaction Summary ================================================================================ Install 1 Package Total download size: 403 k Installed size: 815 k Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : rsync-3.1.2-4.el7.x86_64 1/1 Verifying : rsync-3.1.2-4.el7.x86_64 1/1 Installed: rsync.x86_64 0:3.1.2-4.el7 Complete! 命令执行成功 ============= node103.yinzhengjie.org.cn : yum -y install rsync ============ Loaded plugins: fastestmirror Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast http://mirrors.njupt.edu.cn/centos/7.6.1810/os/x86_64/repodata/6614b3605d961a4aaec45d74ac4e5e713e517debb3ee454a1c91097955780697-primary.sqlite.bz2: [Errno 12] Timeout on http://10.10.254.10/cache/5/02/mirrors.njupt.edu.cn/fa6cd86bdbaedbbfedfbfa6d45952cd2/6614b3605d961a4aaec45d74ac4e5e713e517debb3ee454a1c91097955780697-primary.sqlite.bz2: (28, 'Connection timed out after 30867 milliseconds') Trying other mirror. Determining fastest mirrors * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.cn99.com Resolving Dependencies --> Running transaction check ---> Package rsync.x86_64 0:3.1.2-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: rsync x86_64 3.1.2-4.el7 base 403 k Transaction Summary ================================================================================ Install 1 Package Total download size: 403 k Installed size: 815 k Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : rsync-3.1.2-4.el7.x86_64 1/1 Verifying : rsync-3.1.2-4.el7.x86_64 1/1 Installed: rsync.x86_64 0:3.1.2-4.el7 Complete! 命令执行成功 [root@node101 ~]#
[root@node101 ~]# vi /usr/local/bin/xrsync.sh #编写同步脚本,注意:我们在编写脚本时,就把脚本放在Linux自带的“PATH”环境变量中。 [root@node101 ~]# [root@node101 ~]# cat /usr/local/bin/xrsync.sh #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #判断用户是否传参 if [ $# -lt 1 ];then echo "请输入参数"; exit fi #获取文件路径 file=$@ #获取子路径 filename=`basename $file` #获取父路径 dirpath=`dirname $file` #获取完整路径 cd $dirpath fullpath=`pwd -P` #同步文件到DataNode for (( i=102;i<=103;i++ )) do #使终端变绿色 tput setaf 2 echo =========== node${i}.yinzhengjie.org.cn : $file =========== #使终端变回原来的颜色,即白灰色 tput setaf 7 #远程执行命令 rsync -lr $filename `whoami`@node${i}.yinzhengjie.org.cn:$fullpath #判断命令是否执行成功 if [ $? == 0 ];then echo "命令执行成功" fi done [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# chmod +x /usr/local/bin/xrsync.sh #别忘记添加执行权限哟! [root@node101 ~]#
[root@node101 ~]# xcall.sh cat /etc/hosts | grep yinzhengjie #我们在没有同步“/etc/hosts”配置文件时,发现只有“node101.yinzhengjie.org.cn”中配置了FQDN ============= node101.yinzhengjie.org.cn : cat /etc/hosts ============ 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn ============= node102.yinzhengjie.org.cn : cat /etc/hosts ============ ============= node103.yinzhengjie.org.cn : cat /etc/hosts ============ [root@node101 ~]# [root@node101 ~]# xrsync.sh /etc/hosts #我们利用咱们自定义的脚本将“/etc/hosts”配置文件下发到"node102.yinzhengjie.org.cn"和“node103.yinzhengjie.org.cn”上去。 =========== node102.yinzhengjie.org.cn : /etc/hosts =========== 命令执行成功 =========== node103.yinzhengjie.org.cn : /etc/hosts =========== 命令执行成功 [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# xcall.sh cat /etc/hosts | grep yinzhengjie #我们验证一下,3台机器上陪孩子是否都相同~如果输出相同说明配置生效啦~ ============= node101.yinzhengjie.org.cn : cat /etc/hosts ============ 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn ============= node102.yinzhengjie.org.cn : cat /etc/hosts ============ 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn ============= node103.yinzhengjie.org.cn : cat /etc/hosts ============ 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn [root@node101 ~]#
5>.本篇博客的操作系统环境如下
由于Ambari是使用Java语言开发实现的,所以其部署的服务还需要安装Java环境,Ambari支持1.7或者1.8版本。推荐使用1.8版本,因为jdk1.7官网以及不维护了。
大家可以自行去Oracle官网下载相应的jdk,我这里给出下载地址:https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html(推荐下载tar.gz的安装包)。
[root@node101 ~]# yum -y install lrzsz Loaded plugins: fastestmirror Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/4): base/7/x86_64/primary_db | 6.0 MB 00:00:07 (2/4): base/7/x86_64/group_gz | 166 kB 00:00:11 (3/4): extras/7/x86_64/primary_db | 153 kB 00:00:22 (4/4): updates/7/x86_64/primary_db | 1.3 MB 00:00:36 Determining fastest mirrors * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.163.com Resolving Dependencies --> Running transaction check ---> Package lrzsz.x86_64 0:0.12.20-36.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================== Installing: lrzsz x86_64 0.12.20-36.el7 base 78 k Transaction Summary =================================================================================================================================================================================== Install 1 Package Total download size: 78 k Installed size: 181 k Downloading packages: lrzsz-0.12.20-36.el7.x86_64.rpm | 78 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : lrzsz-0.12.20-36.el7.x86_64 1/1 Verifying : lrzsz-0.12.20-36.el7.x86_64 1/1 Installed: lrzsz.x86_64 0:0.12.20-36.el7 Complete! [root@node101 ~]#
[root@node101 ~]# mkdir -pv /yinzhengjie/softwares mkdir: created directory ‘/yinzhengjie’ mkdir: created directory ‘/yinzhengjie/softwares’ [root@node101 ~]# [root@node101 ~]# ll -h total 183M -rw-r--r--. 1 root root 183M Dec 5 02:32 jdk-8u191-linux-x64.tar.gz [root@node101 ~]# [root@node101 ~]# tar -zxf jdk-8u191-linux-x64.tar.gz -C /yinzhengjie/softwares/ [root@node101 ~]# [root@node101 ~]# ln -s /yinzhengjie/softwares/jdk1.8.0_191/ /yinzhengjie/softwares/jdk [root@node101 ~]# [root@node101 ~]# mkdir /usr/java && cd /usr/java [root@node101 java]# [root@node101 java]# ln -s /yinzhengjie/softwares/jdk1.8.0_191/ /usr/java/jdk1.8 [root@node101 java]# ln -s /yinzhengjie/softwares/jdk1.8.0_191/ /usr/java/defalut [root@node101 java]# [root@node101 java]# tail -3 /etc/profile #ADD JAVA_PATH JAVA_HOME=/yinzhengjie/softwares/jdk PATH=$PATH:$JAVA_HOME/bin [root@node101 java]# [root@node101 java]# [root@node101 java]# source /etc/profile [root@node101 java]# [root@node101 java]# java -version java version "1.8.0_191" Java(TM) SE Runtime Environment (build 1.8.0_191-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode) [root@node101 java]#
[root@node101 java]# xcall.sh python --version ============= node101.yinzhengjie.org.cn : python --version ============ Python 2.7.5 命令执行成功 ============= node102.yinzhengjie.org.cn : python --version ============ Python 2.7.5 命令执行成功 ============= node103.yinzhengjie.org.cn : python --version ============ Python 2.7.5 命令执行成功 [root@node101 java]#
[root@node101 java]# xrsync.sh /yinzhengjie/ =========== node102.yinzhengjie.org.cn : /yinzhengjie/ =========== 命令执行成功 =========== node103.yinzhengjie.org.cn : /yinzhengjie/ =========== 命令执行成功 [root@node101 java]# [root@node101 java]# [root@node101 java]# xrsync.sh /etc/profile #同步环境变量 =========== node102.yinzhengjie.org.cn : /etc/profile =========== 命令执行成功 =========== node103.yinzhengjie.org.cn : /etc/profile =========== 命令执行成功 [root@node101 java]# [root@node101 java]# xcall.sh ln -s /yinzhengjie/softwares/jdk/bin/jps /usr/local/bin/ #做个软链接,便于xcall.sh 脚本可以直接访问到它 ============= node101.yinzhengjie.org.cn : ln -s /yinzhengjie/softwares/jdk/bin/jps /usr/local/bin/ ============ 命令执行成功 ============= node102.yinzhengjie.org.cn : ln -s /yinzhengjie/softwares/jdk/bin/jps /usr/local/bin/ ============ 命令执行成功 ============= node103.yinzhengjie.org.cn : ln -s /yinzhengjie/softwares/jdk/bin/jps /usr/local/bin/ ============ 命令执行成功 [root@node101 java]# [root@node101 java]# xcall.sh jps #说明java环境配置成功啦~ ============= node101.yinzhengjie.org.cn : jps ============ 19866 Jps 命令执行成功 ============= node102.yinzhengjie.org.cn : jps ============ 11801 Jps 命令执行成功 ============= node103.yinzhengjie.org.cn : jps ============ 19482 Jps 命令执行成功 [root@node101 java]#
5>.关闭防火墙和selinux
[root@node101 java]# xcall.sh systemctl stop firewalld ============= node101.yinzhengjie.org.cn : systemctl stop firewalld ============ 命令执行成功 ============= node102.yinzhengjie.org.cn : systemctl stop firewalld ============ 命令执行成功 ============= node103.yinzhengjie.org.cn : systemctl stop firewalld ============ 命令执行成功 [root@node101 java]#
[root@node101 java]# xcall.sh systemctl disable firewalld ============= node101.yinzhengjie.org.cn : systemctl disable firewalld ============ Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. 命令执行成功 ============= node102.yinzhengjie.org.cn : systemctl disable firewalld ============ Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. 命令执行成功 ============= node103.yinzhengjie.org.cn : systemctl disable firewalld ============ Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. 命令执行成功 [root@node101 java]#
[root@node101 java]# xcall.sh systemctl status firewalld ============= node101.yinzhengjie.org.cn : systemctl status firewalld ============ ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Dec 05 02:28:52 node101.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon... Dec 05 02:28:52 node101.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon. Dec 05 03:42:15 node101.yinzhengjie.org.cn systemd[1]: Stopping firewalld - dynamic firewall daemon... Dec 05 03:42:16 node101.yinzhengjie.org.cn systemd[1]: Stopped firewalld - dynamic firewall daemon. ============= node102.yinzhengjie.org.cn : systemctl status firewalld ============ ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Dec 05 02:27:18 nod102.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon... Dec 05 02:27:18 nod102.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon. Dec 05 03:42:16 nod102.yinzhengjie.org.cn systemd[1]: Stopping firewalld - dynamic firewall daemon... Dec 05 03:42:17 nod102.yinzhengjie.org.cn systemd[1]: Stopped firewalld - dynamic firewall daemon. ============= node103.yinzhengjie.org.cn : systemctl status firewalld ============ ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Dec 05 02:25:47 node103.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon... Dec 05 02:25:47 node103.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon. Dec 05 03:42:17 node103.yinzhengjie.org.cn systemd[1]: Stopping firewalld - dynamic firewall daemon... Dec 05 03:42:18 node103.yinzhengjie.org.cn systemd[1]: Stopped firewalld - dynamic firewall daemon. [root@node101 java]# [root@node101 java]# [root@node101 java]#
[root@node101 ~]# xcall.sh sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config ============= node101.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============ 命令执行成功 ============= node102.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============ 命令执行成功 ============= node103.yinzhengjie.org.cn : sed -i s#SELINUX=enforcing#SELINUX=disabled# /etc/selinux/config ============ 命令执行成功 [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# xcall.sh cat /etc/selinux/config | grep SELINUX= | grep -v ^# SELINUX=disabled SELINUX=disabled SELINUX=disabled [root@node101 ~]#
[root@node101 ~]# xcall.sh getenforce ============= node101.yinzhengjie.org.cn : getenforce ============ Enforcing 命令执行成功 ============= node102.yinzhengjie.org.cn : getenforce ============ Enforcing 命令执行成功 ============= node103.yinzhengjie.org.cn : getenforce ============ Enforcing 命令执行成功 [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# xcall.sh setenforce 0 ============= node101.yinzhengjie.org.cn : setenforce 0 ============ 命令执行成功 ============= node102.yinzhengjie.org.cn : setenforce 0 ============ 命令执行成功 ============= node103.yinzhengjie.org.cn : setenforce 0 ============ 命令执行成功 [root@node101 ~]# [root@node101 ~]# xcall.sh getenforce ============= node101.yinzhengjie.org.cn : getenforce ============ Permissive 命令执行成功 ============= node102.yinzhengjie.org.cn : getenforce ============ Permissive 命令执行成功 ============= node103.yinzhengjie.org.cn : getenforce ============ Permissive 命令执行成功 [root@node101 ~]#
6>.配置ntpd服务器并配置开机自启动
[root@node101 ~]# xcall.sh yum -y install ntp ============= node101.yinzhengjie.org.cn : yum -y install ntp ============ Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.163.com Resolving Dependencies --> Running transaction check ---> Package ntp.x86_64 0:4.2.6p5-28.el7.centos will be installed --> Processing Dependency: ntpdate = 4.2.6p5-28.el7.centos for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Processing Dependency: libcrypto.so.10(OPENSSL_1.0.2)(64bit) for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Processing Dependency: libopts.so.25()(64bit) for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Running transaction check ---> Package autogen-libopts.x86_64 0:5.18-5.el7 will be installed ---> Package ntpdate.x86_64 0:4.2.6p5-28.el7.centos will be installed ---> Package openssl-libs.x86_64 1:1.0.1e-42.el7.9 will be updated --> Processing Dependency: openssl-libs(x86-64) = 1:1.0.1e-42.el7.9 for package: 1:openssl-1.0.1e-42.el7.9.x86_64 ---> Package openssl-libs.x86_64 1:1.0.2k-16.el7 will be an update --> Running transaction check ---> Package openssl.x86_64 1:1.0.1e-42.el7.9 will be updated ---> Package openssl.x86_64 1:1.0.2k-16.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: ntp x86_64 4.2.6p5-28.el7.centos base 549 k Installing for dependencies: autogen-libopts x86_64 5.18-5.el7 base 66 k ntpdate x86_64 4.2.6p5-28.el7.centos base 86 k Updating for dependencies: openssl x86_64 1:1.0.2k-16.el7 base 493 k openssl-libs x86_64 1:1.0.2k-16.el7 base 1.2 M Transaction Summary ================================================================================ Install 1 Package (+2 Dependent packages) Upgrade ( 2 Dependent packages) Total download size: 2.4 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. -------------------------------------------------------------------------------- Total 650 kB/s | 2.4 MB 00:03 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : 1:openssl-libs-1.0.2k-16.el7.x86_64 1/7 Installing : ntpdate-4.2.6p5-28.el7.centos.x86_64 2/7 Installing : autogen-libopts-5.18-5.el7.x86_64 3/7 Installing : ntp-4.2.6p5-28.el7.centos.x86_64 4/7 Updating : 1:openssl-1.0.2k-16.el7.x86_64 5/7 Cleanup : 1:openssl-1.0.1e-42.el7.9.x86_64 6/7 Cleanup : 1:openssl-libs-1.0.1e-42.el7.9.x86_64 7/7 Verifying : 1:openssl-libs-1.0.2k-16.el7.x86_64 1/7 Verifying : ntpdate-4.2.6p5-28.el7.centos.x86_64 2/7 Verifying : ntp-4.2.6p5-28.el7.centos.x86_64 3/7 Verifying : autogen-libopts-5.18-5.el7.x86_64 4/7 Verifying : 1:openssl-1.0.2k-16.el7.x86_64 5/7 Verifying : 1:openssl-libs-1.0.1e-42.el7.9.x86_64 6/7 Verifying : 1:openssl-1.0.1e-42.el7.9.x86_64 7/7 Installed: ntp.x86_64 0:4.2.6p5-28.el7.centos Dependency Installed: autogen-libopts.x86_64 0:5.18-5.el7 ntpdate.x86_64 0:4.2.6p5-28.el7.centos Dependency Updated: openssl.x86_64 1:1.0.2k-16.el7 openssl-libs.x86_64 1:1.0.2k-16.el7 Complete! 命令执行成功 ============= node102.yinzhengjie.org.cn : yum -y install ntp ============ Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.neusoft.edu.cn Resolving Dependencies --> Running transaction check ---> Package ntp.x86_64 0:4.2.6p5-28.el7.centos will be installed --> Processing Dependency: ntpdate = 4.2.6p5-28.el7.centos for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Processing Dependency: libcrypto.so.10(OPENSSL_1.0.2)(64bit) for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Processing Dependency: libopts.so.25()(64bit) for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Running transaction check ---> Package autogen-libopts.x86_64 0:5.18-5.el7 will be installed ---> Package ntpdate.x86_64 0:4.2.6p5-28.el7.centos will be installed ---> Package openssl-libs.x86_64 1:1.0.1e-42.el7.9 will be updated --> Processing Dependency: openssl-libs(x86-64) = 1:1.0.1e-42.el7.9 for package: 1:openssl-1.0.1e-42.el7.9.x86_64 ---> Package openssl-libs.x86_64 1:1.0.2k-16.el7 will be an update --> Running transaction check ---> Package openssl.x86_64 1:1.0.1e-42.el7.9 will be updated ---> Package openssl.x86_64 1:1.0.2k-16.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: ntp x86_64 4.2.6p5-28.el7.centos base 549 k Installing for dependencies: autogen-libopts x86_64 5.18-5.el7 base 66 k ntpdate x86_64 4.2.6p5-28.el7.centos base 86 k Updating for dependencies: openssl x86_64 1:1.0.2k-16.el7 base 493 k openssl-libs x86_64 1:1.0.2k-16.el7 base 1.2 M Transaction Summary ================================================================================ Install 1 Package (+2 Dependent packages) Upgrade ( 2 Dependent packages) Total download size: 2.4 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. -------------------------------------------------------------------------------- Total 449 kB/s | 2.4 MB 00:05 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : 1:openssl-libs-1.0.2k-16.el7.x86_64 1/7 Installing : ntpdate-4.2.6p5-28.el7.centos.x86_64 2/7 Installing : autogen-libopts-5.18-5.el7.x86_64 3/7 Installing : ntp-4.2.6p5-28.el7.centos.x86_64 4/7 Updating : 1:openssl-1.0.2k-16.el7.x86_64 5/7 Cleanup : 1:openssl-1.0.1e-42.el7.9.x86_64 6/7 Cleanup : 1:openssl-libs-1.0.1e-42.el7.9.x86_64 7/7 Verifying : 1:openssl-libs-1.0.2k-16.el7.x86_64 1/7 Verifying : ntpdate-4.2.6p5-28.el7.centos.x86_64 2/7 Verifying : ntp-4.2.6p5-28.el7.centos.x86_64 3/7 Verifying : autogen-libopts-5.18-5.el7.x86_64 4/7 Verifying : 1:openssl-1.0.2k-16.el7.x86_64 5/7 Verifying : 1:openssl-libs-1.0.1e-42.el7.9.x86_64 6/7 Verifying : 1:openssl-1.0.1e-42.el7.9.x86_64 7/7 Installed: ntp.x86_64 0:4.2.6p5-28.el7.centos Dependency Installed: autogen-libopts.x86_64 0:5.18-5.el7 ntpdate.x86_64 0:4.2.6p5-28.el7.centos Dependency Updated: openssl.x86_64 1:1.0.2k-16.el7 openssl-libs.x86_64 1:1.0.2k-16.el7 Complete! 命令执行成功 ============= node103.yinzhengjie.org.cn : yum -y install ntp ============ Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.cn99.com Resolving Dependencies --> Running transaction check ---> Package ntp.x86_64 0:4.2.6p5-28.el7.centos will be installed --> Processing Dependency: ntpdate = 4.2.6p5-28.el7.centos for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Processing Dependency: libcrypto.so.10(OPENSSL_1.0.2)(64bit) for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Processing Dependency: libopts.so.25()(64bit) for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Running transaction check ---> Package autogen-libopts.x86_64 0:5.18-5.el7 will be installed ---> Package ntpdate.x86_64 0:4.2.6p5-28.el7.centos will be installed ---> Package openssl-libs.x86_64 1:1.0.1e-42.el7.9 will be updated --> Processing Dependency: openssl-libs(x86-64) = 1:1.0.1e-42.el7.9 for package: 1:openssl-1.0.1e-42.el7.9.x86_64 ---> Package openssl-libs.x86_64 1:1.0.2k-16.el7 will be an update --> Running transaction check ---> Package openssl.x86_64 1:1.0.1e-42.el7.9 will be updated ---> Package openssl.x86_64 1:1.0.2k-16.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: ntp x86_64 4.2.6p5-28.el7.centos base 549 k Installing for dependencies: autogen-libopts x86_64 5.18-5.el7 base 66 k ntpdate x86_64 4.2.6p5-28.el7.centos base 86 k Updating for dependencies: openssl x86_64 1:1.0.2k-16.el7 base 493 k openssl-libs x86_64 1:1.0.2k-16.el7 base 1.2 M Transaction Summary ================================================================================ Install 1 Package (+2 Dependent packages) Upgrade ( 2 Dependent packages) Total download size: 2.4 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. -------------------------------------------------------------------------------- Total 568 kB/s | 2.4 MB 00:04 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : 1:openssl-libs-1.0.2k-16.el7.x86_64 1/7 Installing : ntpdate-4.2.6p5-28.el7.centos.x86_64 2/7 Installing : autogen-libopts-5.18-5.el7.x86_64 3/7 Installing : ntp-4.2.6p5-28.el7.centos.x86_64 4/7 Updating : 1:openssl-1.0.2k-16.el7.x86_64 5/7 Cleanup : 1:openssl-1.0.1e-42.el7.9.x86_64 6/7 Cleanup : 1:openssl-libs-1.0.1e-42.el7.9.x86_64 7/7 Verifying : 1:openssl-libs-1.0.2k-16.el7.x86_64 1/7 Verifying : ntpdate-4.2.6p5-28.el7.centos.x86_64 2/7 Verifying : ntp-4.2.6p5-28.el7.centos.x86_64 3/7 Verifying : autogen-libopts-5.18-5.el7.x86_64 4/7 Verifying : 1:openssl-1.0.2k-16.el7.x86_64 5/7 Verifying : 1:openssl-libs-1.0.1e-42.el7.9.x86_64 6/7 Verifying : 1:openssl-1.0.1e-42.el7.9.x86_64 7/7 Installed: ntp.x86_64 0:4.2.6p5-28.el7.centos Dependency Installed: autogen-libopts.x86_64 0:5.18-5.el7 ntpdate.x86_64 0:4.2.6p5-28.el7.centos Dependency Updated: openssl.x86_64 1:1.0.2k-16.el7 openssl-libs.x86_64 1:1.0.2k-16.el7 Complete! 命令执行成功 [root@node101 ~]#
如果你的内网有ntpd服务器的话直接修改每台服务器的“/etc/ntp.conf”配置文件即可,将ntpd服务器的IP地址写进去。如果没有话,那就自己部署一个即可,详情请参考:https://www.cnblogs.com/yinzhengjie/p/9480665.html。我这里就是在“node101.yinzhengjie.org.cn”主机上部署了ntp服务器,并修改了以下参数:
然后在“node102.yinzhengjie.org.cn”和“node103.yinzhengjie.org.cn”这2台主机上配置了周期性的计划任务:
[root@node101 ~]# systemctl start ntpd [root@node101 ~]# [root@node101 ~]# systemctl enable ntpd Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service. [root@node101 ~]# [root@node101 ~]# systemctl status ntpd ● ntpd.service - Network Time Service Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2018-12-05 04:17:34 EST; 12s ago Main PID: 20281 (ntpd) CGroup: /system.slice/ntpd.service └─20281 /usr/sbin/ntpd -u ntp:ntp -g Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: Listen normally on 2 lo 127.0.0.1 UDP 123 Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: Listen normally on 3 eth0 172.30.1.101 UDP 123 Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: Listen normally on 4 lo ::1 UDP 123 Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: Listen normally on 5 eth0 fe80::21c:42ff:fe58:1b11 UDP 123 Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: Listening on routing socket on fd #22 for interface updates Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: 0.0.0.0 c016 06 restart Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: 0.0.0.0 c012 02 freq_set kernel 0.000 PPM Dec 05 04:17:34 node101.yinzhengjie.org.cn ntpd[20281]: 0.0.0.0 c011 01 freq_not_set Dec 05 04:17:34 node101.yinzhengjie.org.cn systemd[1]: Started Network Time Service. Dec 05 04:17:35 node101.yinzhengjie.org.cn ntpd[20281]: 0.0.0.0 c514 04 freq_mode [root@node101 ~]#
[root@node101 ~]# xcall.sh crontab -l ============= node101.yinzhengjie.org.cn : crontab -l ============ no crontab for root ============= node102.yinzhengjie.org.cn : crontab -l ============ */1 * * * * /usr/sbin/ntpdate node101.yinzhengjie.org.cn 命令执行成功 ============= node103.yinzhengjie.org.cn : crontab -l ============ */1 * * * * /usr/sbin/ntpdate node101.yinzhengjie.org.cn 命令执行成功 [root@node101 ~]#
四.搭建本地yum源仓库(实现离线安装)
安装Ambari系统本身以及通过Ambari安装HDP发行版中的Hadoop服务都需要通过yum的方式进行安装。由于本片博客场景定位的是企业级应用,而企业内部的服务器处于安全考虑通常会通过防火墙隔绝外网环境。就算服务器可以直接联网,HDP那么多达到数G的安装包大小也不适合通过互联网直接安装。所以需要在局域网搭建一套Ambari和HDP的本地yum源仓库用于进行安装。本篇博客将会安装Ambari2.6.0以及对应的HDP2.6.3.0版本。
1>.下载离线安装包(Hortonworks官网文档怎么找?)
[root@node101 ~]# yum -y install wget Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.163.com Resolving Dependencies --> Running transaction check ---> Package wget.x86_64 0:1.14-18.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================== Installing: wget x86_64 1.14-18.el7 base 547 k Transaction Summary =================================================================================================================================================================================== Install 1 Package Total download size: 547 k Installed size: 2.0 M Downloading packages: wget-1.14-18.el7.x86_64.rpm | 547 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : wget-1.14-18.el7.x86_64 1/1 Verifying : wget-1.14-18.el7.x86_64 1/1 Installed: wget.x86_64 0:1.14-18.el7 Complete! [root@node101 ~]#
[root@yinzhengjie hdp]# wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.3.0/HDP-2.6.3.0-centos7-rpm.tar.gz --2018-12-05 17:29:41-- http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.3.0/HDP-2.6.3.0-centos7-rpm.tar.gz Resolving public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)... 52.85.158.49, 52.85.158.71, 52.85.158.182, ... Connecting to public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)|52.85.158.49|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 7020369101 (6.5G) [application/x-tar] Saving to: ‘HDP-2.6.3.0-centos7-rpm.tar.gz’ 100%[=======================================================================================================================================>] 7,020,369,101 6.84MB/s in 17m 11s 2018-12-05 17:46:53 (6.49 MB/s) - ‘HDP-2.6.3.0-centos7-rpm.tar.gz’ saved [7020369101/7020369101] [root@yinzhengjie hdp]# [root@yinzhengjie hdp]# ll total 6855832 -rw-r--r-- 1 root root 7020369101 Oct 31 2017 HDP-2.6.3.0-centos7-rpm.tar.gz [root@yinzhengjie hdp]#
[root@yinzhengjie hdp]# wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.0.0/ambari-2.6.0.0-centos7.tar.gz --2018-12-05 17:47:36-- http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.0.0/ambari-2.6.0.0-centos7.tar.gz Resolving public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)... 52.85.158.117, 52.85.158.49, 52.85.158.71, ... Connecting to public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)|52.85.158.117|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 1760430816 (1.6G) [application/x-tar] Saving to: ‘ambari-2.6.0.0-centos7.tar.gz’ 100%[=======================================================================================================================================>] 1,760,430,816 6.44MB/s in 4m 26s 2018-12-05 17:52:04 (6.31 MB/s) - ‘ambari-2.6.0.0-centos7.tar.gz’ saved [1760430816/1760430816] [root@yinzhengjie hdp]# [root@yinzhengjie hdp]# ll total 9434396 -rw-r--r-- 1 root root 1760430816 Oct 31 2017 ambari-2.6.0.0-centos7.tar.gz -rw-r--r-- 1 root root 7020369101 Oct 31 2017 HDP-2.6.3.0-centos7-rpm.tar.gz [root@yinzhengjie hdp]# [root@yinzhengjie hdp]#
[root@yinzhengjie hdp]# wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz --2018-12-05 17:52:42-- http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz Resolving public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)... 52.85.158.117, 52.85.158.49, 52.85.158.71, ... Connecting to public-repo-1.hortonworks.com (public-repo-1.hortonworks.com)|52.85.158.117|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 880016614 (839M) [application/x-tar] Saving to: ‘HDP-UTILS-1.1.0.21-centos7.tar.gz’ 100%[=========================================================================================================================================>] 880,016,614 6.67MB/s in 2m 7s 2018-12-05 17:54:52 (6.60 MB/s) - ‘HDP-UTILS-1.1.0.21-centos7.tar.gz’ saved [880016614/880016614] [root@yinzhengjie hdp]# [root@yinzhengjie hdp]# ll total 9434396 -rw-r--r-- 1 root root 1760430816 Oct 31 2017 ambari-2.6.0.0-centos7.tar.gz -rw-r--r-- 1 root root 7020369101 Oct 31 2017 HDP-2.6.3.0-centos7-rpm.tar.gz -rw-r--r-- 1 root root 880016614 Oct 27 2017 HDP-UTILS-1.1.0.21-centos7.tar.gz [root@yinzhengjie hdp]#
大家可以直接在服务端跟我一样使用wget工具下载,如果网速不好端朋友,可以直接用我已经下载好端软件,我这里给出百度云链接:
2>.安装WEB服务器
这里我们可以使用Apache来当作HTTP服务器。当然你也可以用Nginx,由于CentOS的yum源默认就支持可以安装httpd服务,因此我就偷个懒就直接用httpd吧。
[root@node101 ~]# yum -y install httpd Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.163.com Resolving Dependencies --> Running transaction check ---> Package httpd.x86_64 0:2.4.6-88.el7.centos will be installed --> Processing Dependency: httpd-tools = 2.4.6-88.el7.centos for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-88.el7.centos.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.8-3.el7_4.1 will be installed ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed ---> Package httpd-tools.x86_64 0:2.4.6-88.el7.centos will be installed ---> Package mailcap.noarch 0:2.1.41-2.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================== Installing: httpd x86_64 2.4.6-88.el7.centos base 2.7 M Installing for dependencies: apr x86_64 1.4.8-3.el7_4.1 base 103 k apr-util x86_64 1.5.2-6.el7 base 92 k httpd-tools x86_64 2.4.6-88.el7.centos base 90 k mailcap noarch 2.1.41-2.el7 base 31 k Transaction Summary =================================================================================================================================================================================== Install 1 Package (+4 Dependent packages) Total download size: 3.0 M Installed size: 10 M Downloading packages: (1/5): httpd-tools-2.4.6-88.el7.centos.x86_64.rpm | 90 kB 00:00:09 (2/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:00:11 (3/5): apr-1.4.8-3.el7_4.1.x86_64.rpm | 103 kB 00:00:12 (4/5): httpd-2.4.6-88.el7.centos.x86_64.rpm | 2.7 MB 00:00:14 (5/5): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:26 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 115 kB/s | 3.0 MB 00:00:26 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : apr-1.4.8-3.el7_4.1.x86_64 1/5 Installing : apr-util-1.5.2-6.el7.x86_64 2/5 Installing : httpd-tools-2.4.6-88.el7.centos.x86_64 3/5 Installing : mailcap-2.1.41-2.el7.noarch 4/5 Installing : httpd-2.4.6-88.el7.centos.x86_64 5/5 Verifying : httpd-tools-2.4.6-88.el7.centos.x86_64 1/5 Verifying : apr-1.4.8-3.el7_4.1.x86_64 2/5 Verifying : mailcap-2.1.41-2.el7.noarch 3/5 Verifying : httpd-2.4.6-88.el7.centos.x86_64 4/5 Verifying : apr-util-1.5.2-6.el7.x86_64 5/5 Installed: httpd.x86_64 0:2.4.6-88.el7.centos Dependency Installed: apr.x86_64 0:1.4.8-3.el7_4.1 apr-util.x86_64 0:1.5.2-6.el7 httpd-tools.x86_64 0:2.4.6-88.el7.centos mailcap.noarch 0:2.1.41-2.el7 Complete! [root@node101 ~]#
[root@node101 ~]# systemctl start httpd #启动httpd服务 [root@node101 ~]# [root@node101 ~]# systemctl enable httpd #配置httpd为开机自启动服务 Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service. [root@node101 ~]# [root@node101 ~]# systemctl status httpd #查看httpd的状态 ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2018-12-05 18:57:15 CST; 12s ago Docs: man:httpd(8) man:apachectl(8) Main PID: 20561 (httpd) Status: "Total requests: 0; Current requests/sec: 0; Current traffic: 0 B/sec" CGroup: /system.slice/httpd.service ├─20561 /usr/sbin/httpd -DFOREGROUND ├─20562 /usr/sbin/httpd -DFOREGROUND ├─20563 /usr/sbin/httpd -DFOREGROUND ├─20564 /usr/sbin/httpd -DFOREGROUND ├─20565 /usr/sbin/httpd -DFOREGROUND └─20566 /usr/sbin/httpd -DFOREGROUND Dec 05 18:57:15 node101.yinzhengjie.org.cn systemd[1]: Starting The Apache HTTP Server... Dec 05 18:57:15 node101.yinzhengjie.org.cn systemd[1]: Started The Apache HTTP Server. [root@node101 ~]#
[root@node101 ~]# mkdir -pv /var/www/html/ambari/HDP-UTILS #在这里我们在Apache的默认根目录创建对应的目录,便于存放加压后的数据文件。 mkdir: created directory ‘/var/www/html/ambari’ mkdir: created directory ‘/var/www/html/ambari/HDP-UTILS’ [root@node101 ~]# [root@node101 ~]# ll /var/www/html/ total 4 drwxr-xr-x 3 root root 4096 Dec 6 13:16 ambari [root@node101 ~]# [root@node101 ~]# ll /var/www/html/ambari/ total 4 drwxr-xr-x 2 root root 4096 Dec 6 13:16 HDP-UTILS [root@node101 ~]# [root@node101 ~]# ll /var/www/html/ambari/HDP-UTILS/ total 0 [root@node101 ~]# [root@node101 ~]# ll #这是我已经下载好的软件包,我们将其解压到web服务器的指定目录中 total 9621668 -rwxrwxrwx. 1 root root 1760430816 Dec 5 23:01 ambari-2.6.0.0-centos7.tar.gz -rwxr-xr-x. 1 root root 7020369101 Dec 6 10:24 HDP-2.6.3.0-centos7-rpm.tar.gz -rwxr-xr-x. 1 root root 880016614 Dec 6 10:24 HDP-UTILS-1.1.0.21-centos7.tar.gz -rw-r--r--. 1 root root 191753373 Dec 5 15:32 jdk-8u191-linux-x64.tar.gz [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# tar -zxf ambari-2.6.0.0-centos7.tar.gz -C /var/www/html/ambari/ #我们将ambari的安装包解压到指定的目录,然后客户端可以去对应的目录访问相应的数据。加压后会生成一个ambari的目录,下面解压的文件是相同的道理。 [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# ll /var/www/html/ambari/ total 8 drwxr-xr-x. 3 root root 4096 Dec 6 12:48 ambari drwxr-xr-x. 2 root root 4096 Dec 6 12:45 HDP-UTILS [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# tar -zxf HDP-2.6.3.0-centos7-rpm.tar.gz -C /var/www/html/ambari/ #解压后会在对应的解压目录中生成一个HDP目录 [root@node101 ~]# [root@node101 ~]# ll /var/www/html/ambari/ total 12 drwxr-xr-x. 3 root root 4096 Dec 6 12:48 ambari drwxr-xr-x. 3 1001 users 4096 Oct 31 2017 HDP drwxr-xr-x. 2 root root 4096 Dec 6 12:45 HDP-UTILS [root@node101 ~]# [root@node101 ~]# ll /var/www/html/ambari/HDP-UTILS/ total 0 [root@node101 ~]# [root@node101 ~]# tar -zxf HDP-UTILS-1.1.0.21-centos7.tar.gz -C /var/www/html/ambari/HDP-UTILS/ #加压后在HDP-UTILS目录中会多处很多文件。 [root@node101 ~]# [root@node101 ~]# ll /var/www/html/ambari/HDP-UTILS/ total 104 -rw-r--r--. 1 1001 users 14548 Jun 10 2017 epel-release-6-8.noarch.rpm drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 extjs drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 fping drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 ganglia drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 hadoop-lzo -rw-r--r--. 1 1001 users 192 Jun 10 2017 hdp-util.repo drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 jna-source drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 libconfuse drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 lucid drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 lzo drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 mysql drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 mysql-connector-java drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 mysql-connector-java-source drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 nagios drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 nagios-plugins drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 openblas drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 pdsh drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 perl drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 R drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 repodata drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 RPM-GPG-KEY drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 rrdtool drwxr-xr-x. 2 1001 users 4096 Oct 27 2017 snappy [root@node101 ~]#
接下来我们也可以访问http://node101.yinzhengjie.org.cn/ambari/:
3>.制作本地源
[root@node101 ~]# yum -y install yum-utils createrepo yum-plugin-priorities Loaded plugins: fastestmirror base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 Loading mirror speeds from cached hostfile * base: mirrors.huaweicloud.com * extras: mirrors.huaweicloud.com * updates: mirrors.163.com Resolving Dependencies --> Running transaction check ---> Package createrepo.noarch 0:0.9.9-28.el7 will be installed --> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: libxml2-python for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: deltarpm for package: createrepo-0.9.9-28.el7.noarch ---> Package yum-plugin-priorities.noarch 0:1.1.31-50.el7 will be installed ---> Package yum-utils.noarch 0:1.1.31-50.el7 will be installed --> Processing Dependency: yum >= 3.4.3-143 for package: yum-utils-1.1.31-50.el7.noarch --> Processing Dependency: python-kitchen for package: yum-utils-1.1.31-50.el7.noarch --> Running transaction check ---> Package deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package libxml2-python.x86_64 0:2.9.1-6.el7_2.3 will be installed --> Processing Dependency: libxml2 = 2.9.1-6.el7_2.3 for package: libxml2-python-2.9.1-6.el7_2.3.x86_64 ---> Package python-deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package python-kitchen.noarch 0:1.1.1-5.el7 will be installed --> Processing Dependency: python-chardet for package: python-kitchen-1.1.1-5.el7.noarch ---> Package yum.noarch 0:3.4.3-132.el7.centos.0.1 will be updated ---> Package yum.noarch 0:3.4.3-161.el7.centos will be an update --> Processing Dependency: rpm >= 4.11.3-22 for package: yum-3.4.3-161.el7.centos.noarch --> Processing Dependency: python-urlgrabber >= 3.10-8 for package: yum-3.4.3-161.el7.centos.noarch --> Running transaction check ---> Package libxml2.x86_64 0:2.9.1-5.el7_1.2 will be updated ---> Package libxml2.x86_64 0:2.9.1-6.el7_2.3 will be an update ---> Package python-chardet.noarch 0:2.2.1-1.el7_1 will be installed ---> Package python-urlgrabber.noarch 0:3.10-7.el7 will be updated ---> Package python-urlgrabber.noarch 0:3.10-9.el7 will be an update ---> Package rpm.x86_64 0:4.11.3-17.el7 will be updated --> Processing Dependency: rpm = 4.11.3-17.el7 for package: rpm-libs-4.11.3-17.el7.x86_64 --> Processing Dependency: rpm = 4.11.3-17.el7 for package: rpm-python-4.11.3-17.el7.x86_64 ---> Package rpm.x86_64 0:4.11.3-35.el7 will be an update --> Running transaction check ---> Package rpm-libs.x86_64 0:4.11.3-17.el7 will be updated --> Processing Dependency: rpm-libs(x86-64) = 4.11.3-17.el7 for package: rpm-build-libs-4.11.3-17.el7.x86_64 ---> Package rpm-libs.x86_64 0:4.11.3-35.el7 will be an update ---> Package rpm-python.x86_64 0:4.11.3-17.el7 will be updated ---> Package rpm-python.x86_64 0:4.11.3-35.el7 will be an update --> Running transaction check ---> Package rpm-build-libs.x86_64 0:4.11.3-17.el7 will be updated ---> Package rpm-build-libs.x86_64 0:4.11.3-35.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================== Installing: createrepo noarch 0.9.9-28.el7 base 94 k yum-plugin-priorities noarch 1.1.31-50.el7 base 29 k yum-utils noarch 1.1.31-50.el7 base 121 k Installing for dependencies: deltarpm x86_64 3.6-3.el7 base 82 k libxml2-python x86_64 2.9.1-6.el7_2.3 base 247 k python-chardet noarch 2.2.1-1.el7_1 base 227 k python-deltarpm x86_64 3.6-3.el7 base 31 k python-kitchen noarch 1.1.1-5.el7 base 267 k Updating for dependencies: libxml2 x86_64 2.9.1-6.el7_2.3 base 668 k python-urlgrabber noarch 3.10-9.el7 base 108 k rpm x86_64 4.11.3-35.el7 base 1.2 M rpm-build-libs x86_64 4.11.3-35.el7 base 106 k rpm-libs x86_64 4.11.3-35.el7 base 277 k rpm-python x86_64 4.11.3-35.el7 base 83 k yum noarch 3.4.3-161.el7.centos base 1.2 M Transaction Summary =================================================================================================================================================================================== Install 3 Packages (+5 Dependent packages) Upgrade ( 7 Dependent packages) Total download size: 4.7 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. (1/15): libxml2-2.9.1-6.el7_2.3.x86_64.rpm | 668 kB 00:00:00 (2/15): python-deltarpm-3.6-3.el7.x86_64.rpm | 31 kB 00:00:00 (3/15): deltarpm-3.6-3.el7.x86_64.rpm | 82 kB 00:00:01 (4/15): python-kitchen-1.1.1-5.el7.noarch.rpm | 267 kB 00:00:00 (5/15): python-chardet-2.2.1-1.el7_1.noarch.rpm | 227 kB 00:00:01 (6/15): python-urlgrabber-3.10-9.el7.noarch.rpm | 108 kB 00:00:00 (7/15): rpm-build-libs-4.11.3-35.el7.x86_64.rpm | 106 kB 00:00:01 (8/15): rpm-4.11.3-35.el7.x86_64.rpm | 1.2 MB 00:00:01 (9/15): yum-3.4.3-161.el7.centos.noarch.rpm | 1.2 MB 00:00:01 (10/15): yum-plugin-priorities-1.1.31-50.el7.noarch.rpm | 29 kB 00:00:00 (11/15): yum-utils-1.1.31-50.el7.noarch.rpm | 121 kB 00:00:00 (12/15): rpm-libs-4.11.3-35.el7.x86_64.rpm | 277 kB 00:00:04 (13/15): libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm | 247 kB 00:00:05 (14/15): rpm-python-4.11.3-35.el7.x86_64.rpm | 83 kB 00:00:05 createrepo-0.9.9-28.el7.noarch FAILED http://centos.ustc.edu.cn/centos/7.6.1810/os/x86_64/Packages/createrepo-0.9.9-28.el7.noarch.rpm: [Errno 12] Timeout on http://centos.ustc.edu.cn/centos/7.6.1810/os/x86_64/Packages/createrepo-0.9.9-28.el7.noarch.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds') Trying other mirror. (15/15): createrepo-0.9.9-28.el7.noarch.rpm | 94 kB 00:00:00 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 157 kB/s | 4.7 MB 00:00:30 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : rpm-4.11.3-35.el7.x86_64 1/22 Updating : rpm-libs-4.11.3-35.el7.x86_64 2/22 Installing : deltarpm-3.6-3.el7.x86_64 3/22 Installing : python-deltarpm-3.6-3.el7.x86_64 4/22 Updating : rpm-build-libs-4.11.3-35.el7.x86_64 5/22 Updating : rpm-python-4.11.3-35.el7.x86_64 6/22 Installing : python-chardet-2.2.1-1.el7_1.noarch 7/22 Installing : python-kitchen-1.1.1-5.el7.noarch 8/22 Updating : libxml2-2.9.1-6.el7_2.3.x86_64 9/22 Installing : libxml2-python-2.9.1-6.el7_2.3.x86_64 10/22 Updating : python-urlgrabber-3.10-9.el7.noarch 11/22 Updating : yum-3.4.3-161.el7.centos.noarch 12/22 Installing : yum-plugin-priorities-1.1.31-50.el7.noarch 13/22 Installing : createrepo-0.9.9-28.el7.noarch 14/22 Installing : yum-utils-1.1.31-50.el7.noarch 15/22 Cleanup : yum-3.4.3-132.el7.centos.0.1.noarch 16/22 Cleanup : rpm-python-4.11.3-17.el7.x86_64 17/22 Cleanup : python-urlgrabber-3.10-7.el7.noarch 18/22 Cleanup : rpm-build-libs-4.11.3-17.el7.x86_64 19/22 Cleanup : rpm-libs-4.11.3-17.el7.x86_64 20/22 Cleanup : rpm-4.11.3-17.el7.x86_64 21/22 Cleanup : libxml2-2.9.1-5.el7_1.2.x86_64 22/22 Verifying : rpm-python-4.11.3-35.el7.x86_64 1/22 Verifying : rpm-libs-4.11.3-35.el7.x86_64 2/22 Verifying : python-urlgrabber-3.10-9.el7.noarch 3/22 Verifying : libxml2-2.9.1-6.el7_2.3.x86_64 4/22 Verifying : yum-plugin-priorities-1.1.31-50.el7.noarch 5/22 Verifying : rpm-4.11.3-35.el7.x86_64 6/22 Verifying : rpm-build-libs-4.11.3-35.el7.x86_64 7/22 Verifying : libxml2-python-2.9.1-6.el7_2.3.x86_64 8/22 Verifying : deltarpm-3.6-3.el7.x86_64 9/22 Verifying : createrepo-0.9.9-28.el7.noarch 10/22 Verifying : python-deltarpm-3.6-3.el7.x86_64 11/22 Verifying : yum-utils-1.1.31-50.el7.noarch 12/22 Verifying : python-chardet-2.2.1-1.el7_1.noarch 13/22 Verifying : python-kitchen-1.1.1-5.el7.noarch 14/22 Verifying : yum-3.4.3-161.el7.centos.noarch 15/22 Verifying : rpm-4.11.3-17.el7.x86_64 16/22 Verifying : libxml2-2.9.1-5.el7_1.2.x86_64 17/22 Verifying : rpm-libs-4.11.3-17.el7.x86_64 18/22 Verifying : rpm-build-libs-4.11.3-17.el7.x86_64 19/22 Verifying : yum-3.4.3-132.el7.centos.0.1.noarch 20/22 Verifying : rpm-python-4.11.3-17.el7.x86_64 21/22 Verifying : python-urlgrabber-3.10-7.el7.noarch 22/22 Installed: createrepo.noarch 0:0.9.9-28.el7 yum-plugin-priorities.noarch 0:1.1.31-50.el7 yum-utils.noarch 0:1.1.31-50.el7 Dependency Installed: deltarpm.x86_64 0:3.6-3.el7 libxml2-python.x86_64 0:2.9.1-6.el7_2.3 python-chardet.noarch 0:2.2.1-1.el7_1 python-deltarpm.x86_64 0:3.6-3.el7 python-kitchen.noarch 0:1.1.1-5.el7 Dependency Updated: libxml2.x86_64 0:2.9.1-6.el7_2.3 python-urlgrabber.noarch 0:3.10-9.el7 rpm.x86_64 0:4.11.3-35.el7 rpm-build-libs.x86_64 0:4.11.3-35.el7 rpm-libs.x86_64 0:4.11.3-35.el7 rpm-python.x86_64 0:4.11.3-35.el7 yum.noarch 0:3.4.3-161.el7.centos Complete! [root@node101 ~]#
[root@node101 ~]# cd /var/www/html/ambari/ [root@node101 ambari]# [root@node101 ambari]# createrepo ./ #创建本地源,会在当前目录下生产一个repodata的目录。 Spawning worker 0 with 156 pkgs Spawning worker 1 with 156 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@node101 ambari]#
执行上述操作后,会生产一个目录:
为了能让yum命令能够找到我们的安装文件,还需要在yum的配置目录中“/etc/yum.repos.d/”创建两个仓库的配置文件,名字可以自定义,如下:
[root@node101 ~]# cat /etc/yum.repos.d/ambari.repo #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com [ambari-2.6.0.0] name=ambari Version - ambari-2.6.0.0 baseurl=http://node101.yinzhengjie.org.cn/ambari/ambari/centos7/2.6.0.0-267 gpgcheck=1 gpgkey=http://node101.yinzhengjie.org.cn/ambari/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [root@node101 ~]#
[root@node101 ~]# cat /etc/yum.repos.d/hdp.repo #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com [HDP-2.6.3.0] name=HDP Version - HDP-2.6.3.0 baseurl=http://node101.yinzhengjie.org.cn/ambari/HDP/centos7/2.6.3.0-235/ gpgcheck=1 gpgkey=http://node101.yinzhengjie.org.cn/ambari/HDP/centos7/2.6.3.0-235/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [HDP-UTILS-1.1.0.21] name=HDP-UTILS Version - HDP-UTILS-1.1.0.21 baseurl=http://node101.yinzhengjie.org.cn/ambari/HDP-UTILS/ gpgcheck=1 gpgkey=http://node101.yinzhengjie.org.cn/ambari/HDP-UTILS/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [root@node101 ~]#
[root@node101 ~]# yum clean all Loaded plugins: fastestmirror, priorities Cleaning repos: HDP-2.6.3.0 HDP-UTILS-1.1.0.21 ambari-2.6.0.0 base extras updates Cleaning up list of fastest mirrors [root@node101 ~]#
[root@node101 ~]# yum makecache Loaded plugins: fastestmirror, priorities HDP-2.6.3.0 | 2.9 kB 00:00:00 HDP-UTILS-1.1.0.21 | 2.9 kB 00:00:00 ambari-2.6.0.0 | 2.9 kB 00:00:00 base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/21): HDP-2.6.3.0/filelists_db | 418 kB 00:00:00 (2/21): HDP-2.6.3.0/primary_db | 100 kB 00:00:00 (3/21): HDP-2.6.3.0/other_db | 14 kB 00:00:00 (4/21): HDP-UTILS-1.1.0.21/filelists_db | 95 kB 00:00:00 (5/21): HDP-UTILS-1.1.0.21/other_db | 20 kB 00:00:00 (6/21): HDP-UTILS-1.1.0.21/primary_db | 38 kB 00:00:00 (7/21): ambari-2.6.0.0/filelists_db | 167 kB 00:00:00 (8/21): ambari-2.6.0.0/other_db | 1.3 kB 00:00:00 (9/21): ambari-2.6.0.0/primary_db | 8.6 kB 00:00:00 (10/21): base/7/x86_64/group_gz | 166 kB 00:00:02 (11/21): extras/7/x86_64/prestodelta | 33 kB 00:00:01 (12/21): base/7/x86_64/other_db | 2.6 MB 00:00:04 (13/21): extras/7/x86_64/primary_db | 153 kB 00:00:01 (14/21): extras/7/x86_64/other_db | 106 kB 00:00:00 (15/21): extras/7/x86_64/filelists_db | 330 kB 00:00:04 (16/21): updates/7/x86_64/prestodelta | 173 kB 00:00:00 (17/21): updates/7/x86_64/primary_db | 1.3 MB 00:00:02 (18/21): updates/7/x86_64/other_db | 182 kB 00:00:02 (19/21): updates/7/x86_64/filelists_db | 1.3 MB 00:00:07 (20/21): base/7/x86_64/filelists_db | 7.1 MB 00:00:20 (21/21): base/7/x86_64/primary_db | 6.0 MB 00:00:26 Determining fastest mirrors * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Metadata Cache Created [root@node101 ~]#
[root@node101 ~]# hostname node101.yinzhengjie.org.cn [root@node101 ~]# [root@node101 ~]# yum repolist #检查一下yum的列表清单 Loaded plugins: fastestmirror, priorities Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn 17 packages excluded due to repository priority protections repo id repo name status HDP-2.6.3.0 HDP Version - HDP-2.6.3.0 236 #这个yum源是咱们自己自定义的 HDP-UTILS-1.1.0.21 HDP-UTILS Version - HDP-UTILS-1.1.0.21 64 #这远yum源也是咱们自定义的 ambari-2.6.0.0 ambari Version - ambari-2.6.0.0 12 #这个yum元依旧是咱们自定的,如果你发现有这3个yum源是存在的,那么说明你的yum仓库部署就没有问题哟! base/7/x86_64 CentOS-7 - Base 10,003+16 extras/7/x86_64 CentOS-7 - Extras 311+1 updates/7/x86_64 CentOS-7 - Updates 594 repolist: 11,220 [root@node101 ~]#
4>.同步yum源的配置文件(将node101.yinzhengjie.org.cn的2个yum源配置文件拷贝到node102.yinzhengjie.org.cn和node103.yinzhengjie.org.cn上去。)
[root@node101 ~]# xrsync.sh /etc/yum.repos.d/ambari.repo =========== node102.yinzhengjie.org.cn : /etc/yum.repos.d/ambari.repo =========== 命令执行成功 =========== node103.yinzhengjie.org.cn : /etc/yum.repos.d/ambari.repo =========== 命令执行成功 [root@node101 ~]#
[root@node101 ~]# xrsync.sh /etc/yum.repos.d/hdp.repo =========== node102.yinzhengjie.org.cn : /etc/yum.repos.d/hdp.repo =========== 命令执行成功 =========== node103.yinzhengjie.org.cn : /etc/yum.repos.d/hdp.repo =========== 命令执行成功 [root@node101 ~]#
[root@node101 ~]# ssh node102.yinzhengjie.org.cn #登陆到node102yinzhengjie.org.cn节点 Last login: Thu Dec 6 12:57:56 2018 from 172.30.1.2 [root@nod102 ~]# [root@nod102 ~]# yum clean all #情况yum的缓存 Loaded plugins: fastestmirror Cleaning repos: HDP-2.6.3.0 HDP-UTILS-1.1.0.21 ambari-2.6.0.0 base extras updates Cleaning up everything Cleaning up list of fastest mirrors [root@nod102 ~]# [root@nod102 ~]# yum makecache #重新制造缓存 Loaded plugins: fastestmirror HDP-2.6.3.0 | 2.9 kB 00:00:00 HDP-UTILS-1.1.0.21 | 2.9 kB 00:00:00 ambari-2.6.0.0 | 2.9 kB 00:00:00 base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/21): HDP-2.6.3.0/primary_db | 100 kB 00:00:00 (2/21): HDP-2.6.3.0/filelists_db | 418 kB 00:00:00 (3/21): HDP-2.6.3.0/other_db | 14 kB 00:00:00 (4/21): HDP-UTILS-1.1.0.21/filelists_db | 95 kB 00:00:00 (5/21): HDP-UTILS-1.1.0.21/other_db | 20 kB 00:00:00 (6/21): HDP-UTILS-1.1.0.21/primary_db | 38 kB 00:00:00 (7/21): ambari-2.6.0.0/filelists_db | 167 kB 00:00:00 (8/21): ambari-2.6.0.0/other_db | 1.3 kB 00:00:00 (9/21): ambari-2.6.0.0/primary_db | 8.6 kB 00:00:00 (10/21): base/7/x86_64/group_gz | 166 kB 00:00:01 (11/21): extras/7/x86_64/prestodelta | 33 kB 00:00:00 (12/21): extras/7/x86_64/primary_db | 153 kB 00:00:01 (13/21): extras/7/x86_64/other_db | 106 kB 00:00:01 (14/21): extras/7/x86_64/filelists_db | 330 kB 00:00:07 (15/21): base/7/x86_64/other_db | 2.6 MB 00:00:08 (16/21): updates/7/x86_64/prestodelta | 173 kB 00:00:01 (17/21): updates/7/x86_64/primary_db | 1.3 MB 00:00:01 (18/21): updates/7/x86_64/other_db | 182 kB 00:00:01 (19/21): updates/7/x86_64/filelists_db | 1.3 MB 00:00:10 (20/21): base/7/x86_64/primary_db | 6.0 MB 00:00:18 (21/21): base/7/x86_64/filelists_db | 7.1 MB 00:01:12 Determining fastest mirrors * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Metadata Cache Created You have new mail in /var/spool/mail/root [root@nod102 ~]# [root@nod102 ~]# [root@nod102 ~]# yum repolist #查看yum的列表 Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn repo id repo name status HDP-2.6.3.0 HDP Version - HDP-2.6.3.0 236 HDP-UTILS-1.1.0.21 HDP-UTILS Version - HDP-UTILS-1.1.0.21 64 ambari-2.6.0.0 ambari Version - ambari-2.6.0.0 12 base/7/x86_64 CentOS-7 - Base 10,019 extras/7/x86_64 CentOS-7 - Extras 312 updates/7/x86_64 CentOS-7 - Updates 594 repolist: 11,237 [root@nod102 ~]#
[root@node101 ~]# ssh node103.yinzhengjie.org.cn Last login: Thu Dec 6 12:58:02 2018 from 172.30.1.2 [root@node103 ~]# You have new mail in /var/spool/mail/root [root@node103 ~]# [root@node103 ~]# yum clean all Loaded plugins: fastestmirror Cleaning repos: HDP-2.6.3.0 HDP-UTILS-1.1.0.21 ambari-2.6.0.0 base extras updates Cleaning up everything Cleaning up list of fastest mirrors [root@node103 ~]# [root@node103 ~]# yum makecache Loaded plugins: fastestmirror HDP-2.6.3.0 | 2.9 kB 00:00:00 HDP-UTILS-1.1.0.21 | 2.9 kB 00:00:00 ambari-2.6.0.0 | 2.9 kB 00:00:00 base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/21): HDP-2.6.3.0/primary_db | 100 kB 00:00:00 (2/21): HDP-2.6.3.0/filelists_db | 418 kB 00:00:00 (3/21): HDP-2.6.3.0/other_db | 14 kB 00:00:00 (4/21): HDP-UTILS-1.1.0.21/filelists_db | 95 kB 00:00:00 (5/21): HDP-UTILS-1.1.0.21/other_db | 20 kB 00:00:00 (6/21): HDP-UTILS-1.1.0.21/primary_db | 38 kB 00:00:00 (7/21): ambari-2.6.0.0/primary_db | 8.6 kB 00:00:00 (8/21): ambari-2.6.0.0/filelists_db | 167 kB 00:00:00 (9/21): ambari-2.6.0.0/other_db | 1.3 kB 00:00:00 (10/21): base/7/x86_64/group_gz | 166 kB 00:00:01 (11/21): extras/7/x86_64/prestodelta | 33 kB 00:00:00 (12/21): extras/7/x86_64/primary_db | 153 kB 00:00:01 (13/21): extras/7/x86_64/filelists_db | 330 kB 00:00:03 (14/21): extras/7/x86_64/other_db | 106 kB 00:00:01 (15/21): updates/7/x86_64/prestodelta | 173 kB 00:00:01 (16/21): base/7/x86_64/other_db | 2.6 MB 00:00:07 (17/21): updates/7/x86_64/other_db | 182 kB 00:00:00 (18/21): updates/7/x86_64/primary_db | 1.3 MB 00:00:06 (19/21): base/7/x86_64/primary_db | 6.0 MB 00:00:16 (20/21): updates/7/x86_64/filelists_db | 1.3 MB 00:00:46 (21/21): base/7/x86_64/filelists_db | 7.1 MB 00:01:13 Determining fastest mirrors * base: mirrors.aliyun.com * extras: mirrors.huaweicloud.com * updates: mirrors.huaweicloud.com Metadata Cache Created You have new mail in /var/spool/mail/root [root@node103 ~]# [root@node103 ~]# [root@node103 ~]# yum repolist Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.huaweicloud.com * updates: mirrors.huaweicloud.com repo id repo name status HDP-2.6.3.0 HDP Version - HDP-2.6.3.0 236 HDP-UTILS-1.1.0.21 HDP-UTILS Version - HDP-UTILS-1.1.0.21 64 ambari-2.6.0.0 ambari Version - ambari-2.6.0.0 12 base/7/x86_64 CentOS-7 - Base 10,019 extras/7/x86_64 CentOS-7 - Extras 312 updates/7/x86_64 CentOS-7 - Updates 594 repolist: 11,237 [root@node103 ~]#
五.安装ambari-server
1>.安装postgresql数据库
和CDH一样,Ambari默认是使用postgresql数据库来保存元数据它的元数据信息,这需要咱们运维的小伙伴对postgresql数据库要具有一定的了解,掌握常见的故障拍错,包括如何配置数据库的主从同步等等。有的人没有懒得去学习也可以使用oracle或者mysql来存放元数据信息,当然,即使你使用来别的数据库,推荐你依然要配置数据库的主从同步,目的是防治单点故障嘛,你懂的。因此,我们可以总结一下为什么不实用postgrsql数据库的原因:
第一:需要学习数据库的成本提升(主从复制);
第二:postgresql数据库的性能我们无法保证,因为我们对他的了解情况不深入;
第三:postgresql数据库的可靠性我们无法保证!
综上所述,我们在生产环境中不推荐使用内置的数据库选项,而是应该使用自己安装的数据库(单本篇博客依旧会介绍如何使用postgresql数据库来安装Ambari服务。),比如关于如何使用mysql存放Ambari元数据的配置案例请参考:https://www.cnblogs.com/yinzhengjie/p/10074613.html。
[root@node101 ~]# yum -y install postgresql-server Loaded plugins: fastestmirror, priorities Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn 17 packages excluded due to repository priority protections Resolving Dependencies --> Running transaction check ---> Package postgresql-server.x86_64 0:9.2.24-1.el7_5 will be installed --> Processing Dependency: postgresql-libs(x86-64) = 9.2.24-1.el7_5 for package: postgresql-server-9.2.24-1.el7_5.x86_64 --> Processing Dependency: postgresql(x86-64) = 9.2.24-1.el7_5 for package: postgresql-server-9.2.24-1.el7_5.x86_64 --> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-9.2.24-1.el7_5.x86_64 --> Running transaction check ---> Package postgresql.x86_64 0:9.2.24-1.el7_5 will be installed ---> Package postgresql-libs.x86_64 0:9.2.24-1.el7_5 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================== Package Arch Version Repository Size ================================================================================================================== Installing: postgresql-server x86_64 9.2.24-1.el7_5 base 3.8 M Installing for dependencies: postgresql x86_64 9.2.24-1.el7_5 base 3.0 M postgresql-libs x86_64 9.2.24-1.el7_5 base 234 k Transaction Summary ================================================================================================================== Install 1 Package (+2 Dependent packages) Total download size: 7.1 M Installed size: 33 M Downloading packages: (1/3): postgresql-libs-9.2.24-1.el7_5.x86_64.rpm | 234 kB 00:00:01 (2/3): postgresql-server-9.2.24-1.el7_5.x86_64.rpm | 3.8 MB 00:00:04 (3/3): postgresql-9.2.24-1.el7_5.x86_64.rpm | 3.0 MB 00:00:32 ------------------------------------------------------------------------------------------------------------------ Total 223 kB/s | 7.1 MB 00:00:32 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : postgresql-libs-9.2.24-1.el7_5.x86_64 1/3 Installing : postgresql-9.2.24-1.el7_5.x86_64 2/3 Installing : postgresql-server-9.2.24-1.el7_5.x86_64 3/3 Verifying : postgresql-server-9.2.24-1.el7_5.x86_64 1/3 Verifying : postgresql-libs-9.2.24-1.el7_5.x86_64 2/3 Verifying : postgresql-9.2.24-1.el7_5.x86_64 3/3 Installed: postgresql-server.x86_64 0:9.2.24-1.el7_5 Dependency Installed: postgresql.x86_64 0:9.2.24-1.el7_5 postgresql-libs.x86_64 0:9.2.24-1.el7_5 Complete! [root@node101 ~]#
2>.安装Ambari-Server
注意,在安装Ambari-Server请确保两点:
第一,是否配置好yum源;
第二:是否安装来postgresql数据库(如果没有安装psotgresql数据库的话安装Ambari-Server是会报错的);
[root@node101 ~]# yum -y install ambari-server Loaded plugins: fastestmirror, priorities Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn 17 packages excluded due to repository priority protections Resolving Dependencies --> Running transaction check ---> Package ambari-server.x86_64 0:2.6.0.0-267 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================== Package Arch Version Repository Size ================================================================================================================== Installing: ambari-server x86_64 2.6.0.0-267 ambari-2.6.0.0 712 M Transaction Summary ================================================================================================================== Install 1 Package Total download size: 712 M Installed size: 776 M Downloading packages: warning: /var/cache/yum/x86_64/7/ambari-2.6.0.0/packages/ambari-server-2.6.0.0-267.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY Public key for ambari-server-2.6.0.0-267.x86_64.rpm is not installed ambari-server-2.6.0.0-267.x86_64.rpm | 712 MB 00:00:07 Retrieving key from http://node101.yinzhengjie.org.cn/ambari/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins Importing GPG key 0x07513CAD: Userid : "Jenkins (HDP Builds) <jenkin@hortonworks.com>" Fingerprint: df52 ed4f 7a3a 5882 c099 4c66 b973 3a7a 0751 3cad From : http://node101.yinzhengjie.org.cn/ambari/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : ambari-server-2.6.0.0-267.x86_64 1/1 Verifying : ambari-server-2.6.0.0-267.x86_64 1/1 Installed: ambari-server.x86_64 0:2.6.0.0-267 Complete! [root@node101 ~]#
3>.Ambari-Server目录结构介绍
Ambari-Server安装完毕后我们主要关注4个目录,他们分别是以下几个:
[root@node101 ~]# ll /etc/ambari-server/conf/ total 24 -rwxr-xr-x 1 root root 8098 Oct 31 2017 ambari.properties -rwxr-xr-x 1 root root 286 Oct 31 2017 krb5JAASLogin.conf -rwxr-xr-x 1 root root 4929 Dec 6 14:56 log4j.properties -rwxr-xr-x 1 root root 2630 Oct 31 2017 metrics.properties [root@node101 ~]#
[root@node101 ~]# ll /usr/lib/ambari-server/ total 71588 -rw-r--r-- 1 root root 62983 Oct 31 2017 activation-1.1.jar -rw-r--r-- 1 root root 3800845 Oct 31 2017 ambari-metrics-common-2.6.0.0.267.jar -rw-r--r-- 1 root root 5810699 Oct 31 2017 ambari-server-2.6.0.0.267.jar -rw-r--r-- 1 root root 27175 Oct 31 2017 ambari-views-2.6.0.0.267.jar -rw-r--r-- 1 root root 1034049 Oct 31 2017 ant-1.6.5.jar -rw-r--r-- 1 root root 12143 Oct 31 2017 ant-launcher-1.7.1.jar -rw-r--r-- 1 root root 445288 Oct 31 2017 antlr-2.7.7.jar -rw-r--r-- 1 root root 4467 Oct 31 2017 aopalliance-1.0.jar -rw-r--r-- 1 root root 44990 Oct 31 2017 apacheds-i18n-2.0.0-M19.jar -rw-r--r-- 1 root root 689835 Oct 31 2017 apacheds-kerberos-codec-2.0.0-M19.jar -rw-r--r-- 1 root root 448794 Oct 31 2017 apache-log4j-extras-1.2.17.jar -rw-r--r-- 1 root root 15418 Oct 31 2017 api-asn1-api-1.0.0-M26.jar -rw-r--r-- 1 root root 42197 Oct 31 2017 api-asn1-ber-1.0.0-M26.jar -rw-r--r-- 1 root root 44647 Oct 31 2017 api-i18n-1.0.0-M26.jar -rw-r--r-- 1 root root 910102 Oct 31 2017 api-ldap-model-1.0.0-M26.jar -rw-r--r-- 1 root root 82413 Oct 31 2017 api-util-1.0.0-M26.jar -rw-r--r-- 1 root root 43581 Oct 31 2017 asm-3.3.1.jar -rw-r--r-- 1 root root 303139 Oct 31 2017 avro-1.7.4.jar -rw-r--r-- 1 root root 497865 Oct 31 2017 c3p0-0.9.5.2.jar -rw-r--r-- 1 root root 287192 Oct 31 2017 cglib-2.2.2.jar -rw-r--r-- 1 root root 25346 Oct 31 2017 cloning-1.9.2.jar -rw-r--r-- 1 root root 21007 Oct 31 2017 commonj.sdo-2.1.1.jar -rw-r--r-- 1 root root 188671 Oct 31 2017 commons-beanutils-1.7.0.jar -rw-r--r-- 1 root root 206035 Oct 31 2017 commons-beanutils-core-1.8.0.jar -rw-r--r-- 1 root root 52988 Oct 31 2017 commons-cli-1.3.1.jar -rw-r--r-- 1 root root 263865 Oct 31 2017 commons-codec-1.8.jar -rw-r--r-- 1 root root 575389 Oct 31 2017 commons-collections-3.2.1.jar -rw-r--r-- 1 root root 256241 Oct 31 2017 commons-compress-1.5.jar -rw-r--r-- 1 root root 298829 Oct 31 2017 commons-configuration-1.6.jar -rw-r--r-- 1 root root 36888 Oct 31 2017 commons-csv-1.1.jar -rw-r--r-- 1 root root 143602 Oct 31 2017 commons-digester-1.8.jar -rw-r--r-- 1 root root 305001 Oct 31 2017 commons-httpclient-3.1.jar -rw-r--r-- 1 root root 163151 Oct 31 2017 commons-io-2.1.jar -rw-r--r-- 1 root root 279193 Oct 31 2017 commons-lang-2.5.jar -rw-r--r-- 1 root root 60686 Oct 31 2017 commons-logging-1.1.1.jar -rw-r--r-- 1 root root 1599627 Oct 31 2017 commons-math3-3.1.1.jar -rw-r--r-- 1 root root 180792 Oct 31 2017 commons-net-1.4.1.jar -rw-r--r-- 1 root root 69500 Oct 31 2017 curator-client-2.7.1.jar -rw-r--r-- 1 root root 186273 Oct 31 2017 curator-framework-2.7.1.jar -rw-r--r-- 1 root root 270342 Oct 31 2017 curator-recipes-2.7.1.jar -rw-r--r-- 1 root root 1674737 Oct 31 2017 ecj-3.5.1.jar -rw-r--r-- 1 root root 9071591 Oct 31 2017 eclipselink-2.6.2.jar -rw-r--r-- 1 root root 8908872 Oct 31 2017 ehcache-2.10.0.jar -rw-r--r-- 1 root root 189285 Oct 31 2017 gson-2.2.2.jar -rw-r--r-- 1 root root 2225441 Oct 31 2017 guava-16.0.jar -rw-r--r-- 1 root root 710492 Oct 31 2017 guice-3.0.jar -rw-r--r-- 1 root root 36998 Oct 31 2017 guice-assistedinject-3.0.jar -rw-r--r-- 1 root root 34471 Oct 31 2017 guice-multibindings-3.0.jar -rw-r--r-- 1 root root 27633 Oct 31 2017 guice-persist-3.0.jar -rw-r--r-- 1 root root 65012 Oct 31 2017 guice-servlet-3.0.jar -rw-r--r-- 1 root root 1791969 Oct 31 2017 h2-1.4.193.jar -rw-r--r-- 1 root root 17035 Oct 31 2017 hadoop-annotations-2.6.0.jar -rw-r--r-- 1 root root 70685 Oct 31 2017 hadoop-auth-2.7.2.jar -rw-r--r-- 1 root root 3443040 Oct 31 2017 hadoop-common-2.7.2.jar -rw-r--r-- 1 root root 1475955 Oct 31 2017 htrace-core-3.1.0-incubating.jar -rw-r--r-- 1 root root 433368 Oct 31 2017 httpclient-4.2.5.jar -rw-r--r-- 1 root root 227275 Oct 31 2017 httpcore-4.2.4.jar -rw-r--r-- 1 root root 55784 Oct 31 2017 jackson-annotations-2.8.0.jar -rw-r--r-- 1 root root 232102 Oct 31 2017 jackson-core-asl-1.9.9.jar -rw-r--r-- 1 root root 18324 Oct 31 2017 jackson-jaxrs-1.9.9.jar -rw-r--r-- 1 root root 780664 Oct 31 2017 jackson-mapper-asl-1.9.13.jar -rw-r--r-- 1 root root 27074 Oct 31 2017 jackson-xc-1.9.9.jar -rw-r--r-- 1 root root 2497 Oct 31 2017 javax.inject-1.jar -rw-r--r-- 1 root root 85147 Oct 31 2017 javax.json-1.0.4.jar -rw-r--r-- 1 root root 18490 Oct 31 2017 java-xmlbuilder-0.4.jar -rw-r--r-- 1 root root 162116 Oct 31 2017 javax.persistence-2.1.1.jar -rw-r--r-- 1 root root 200387 Oct 31 2017 javax.servlet-3.0.0.v201112011016.jar -rw-r--r-- 1 root root 95806 Oct 31 2017 javax.servlet-api-3.1.0.jar -rw-r--r-- 1 root root 105134 Oct 31 2017 jaxb-api-2.2.2.jar -rw-r--r-- 1 root root 890168 Oct 31 2017 jaxb-impl-2.2.3-1.jar -rw-r--r-- 1 root root 2254 Oct 31 2017 jcip-annotations-1.0.jar -rw-r--r-- 1 root root 134021 Oct 31 2017 jersey-client-1.19.jar -rw-r--r-- 1 root root 436689 Oct 31 2017 jersey-core-1.19.jar -rw-r--r-- 1 root root 16151 Oct 31 2017 jersey-guice-1.19.jar -rw-r--r-- 1 root root 165345 Oct 31 2017 jersey-json-1.19.jar -rw-r--r-- 1 root root 53275 Oct 31 2017 jersey-multipart-1.19.jar -rw-r--r-- 1 root root 702882 Oct 31 2017 jersey-server-1.19.jar -rw-r--r-- 1 root root 128719 Oct 31 2017 jersey-servlet-1.19.jar -rw-r--r-- 1 root root 539735 Oct 31 2017 jets3t-0.9.0.jar -rw-r--r-- 1 root root 539912 Oct 31 2017 jetty-6.1.26.jar -rw-r--r-- 1 root root 89706 Oct 31 2017 jetty-client-8.1.19.v20160209.jar -rw-r--r-- 1 root root 21163 Oct 31 2017 jetty-continuation-8.1.19.v20160209.jar -rw-r--r-- 1 root root 96382 Oct 31 2017 jetty-http-8.1.19.v20160209.jar -rw-r--r-- 1 root root 106036 Oct 31 2017 jetty-io-8.1.19.v20160209.jar -rw-r--r-- 1 root root 89945 Oct 31 2017 jetty-security-8.1.19.v20160209.jar -rw-r--r-- 1 root root 360317 Oct 31 2017 jetty-server-8.1.19.v20160209.jar -rw-r--r-- 1 root root 102235 Oct 31 2017 jetty-servlet-8.1.19.v20160209.jar -rw-r--r-- 1 root root 96061 Oct 31 2017 jetty-servlets-8.1.19.v20160209.jar -rw-r--r-- 1 root root 288218 Oct 31 2017 jetty-util-8.1.19.v20160209.jar -rw-r--r-- 1 root root 110106 Oct 31 2017 jetty-webapp-8.1.19.v20160209.jar -rw-r--r-- 1 root root 39081 Oct 31 2017 jetty-xml-8.1.19.v20160209.jar -rw-r--r-- 1 root root 208781 Oct 31 2017 jline-2.11.jar -rw-r--r-- 1 root root 1137286 Oct 31 2017 jna-4.2.2.jar -rw-r--r-- 1 root root 185746 Oct 31 2017 jsch-0.1.42.jar -rw-r--r-- 1 root root 51484 Oct 31 2017 json-smart-1.1.1.jar -rw-r--r-- 1 root root 1026814 Oct 31 2017 jsp-2.1-glassfish-2.1.v20100127.jar -rw-r--r-- 1 root root 132298 Oct 31 2017 jsp-api-2.1-glassfish-2.1.v20100127.jar -rw-r--r-- 1 root root 33015 Oct 31 2017 jsr305-1.3.9.jar -rw-r--r-- 1 root root 46367 Oct 31 2017 jsr311-api-1.1.1.jar -rw-r--r-- 1 root root 4952 Oct 31 2017 jul-to-slf4j-1.7.2.jar -rw-r--r-- 1 root root 57383 Oct 31 2017 kerberos-client-2.0.0-M19.jar drwxr-xr-x 6 root root 4096 Dec 6 14:56 lib -rw-r--r-- 1 root root 19925 Oct 31 2017 libpam4j-1.8.jar -rw-r--r-- 1 root root 489884 Oct 31 2017 log4j-1.2.17.jar -rw-r--r-- 1 root root 301475 Oct 31 2017 mailapi-1.5.2.jar -rw-r--r-- 1 root root 606472 Oct 31 2017 mchange-commons-java-0.2.11.jar -rw-r--r-- 1 root root 111908 Oct 31 2017 metrics-core-3.1.0.jar -rw-r--r-- 1 root root 35907 Oct 31 2017 metrics-jvm-3.1.0.jar -rw-r--r-- 1 root root 62135 Oct 31 2017 mimepull-1.9.3.jar -rw-r--r-- 1 root root 632771 Oct 31 2017 mina-core-2.0.9.jar -rw-r--r-- 1 root root 1208356 Oct 31 2017 netty-3.7.0.Final.jar -rw-r--r-- 1 root root 178648 Oct 31 2017 nimbus-jose-jwt-3.9.jar -rw-r--r-- 1 root root 41755 Oct 31 2017 objenesis-2.1.jar -rw-r--r-- 1 root root 60299 Oct 31 2017 objenesis-tck-1.2.jar -rw-r--r-- 1 root root 65261 Oct 31 2017 oro-2.0.8.jar -rw-r--r-- 1 root root 29555 Oct 31 2017 paranamer-2.3.jar -rw-r--r-- 1 root root 588974 Oct 31 2017 postgresql-9.3-1101-jdbc4.jar -rw-r--r-- 1 root root 533455 Oct 31 2017 protobuf-java-2.5.0.jar -rw-r--r-- 1 root root 660316 Oct 31 2017 quartz-2.2.1.jar -rw-r--r-- 1 root root 33973 Oct 31 2017 quartz-jobs-2.2.1.jar -rw-r--r-- 1 root root 26083 Oct 31 2017 slf4j-api-1.7.2.jar -rw-r--r-- 1 root root 8819 Oct 31 2017 slf4j-log4j12-1.7.2.jar -rw-r--r-- 1 root root 53789 Oct 31 2017 smtp-1.5.2.jar -rw-r--r-- 1 root root 995968 Oct 31 2017 snappy-java-1.0.4.1.jar -rw-r--r-- 1 root root 469539 Oct 31 2017 snmp4j-1.10.1.jar -rw-r--r-- 1 root root 321435 Oct 31 2017 spring-aop-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 53082 Oct 31 2017 spring-asm-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 556747 Oct 31 2017 spring-beans-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 670388 Oct 31 2017 spring-context-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 383621 Oct 31 2017 spring-core-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 169749 Oct 31 2017 spring-expression-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 386083 Oct 31 2017 spring-jdbc-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 376095 Oct 31 2017 spring-ldap-core-2.0.4.RELEASE.jar -rw-r--r-- 1 root root 203707 Oct 31 2017 spring-security-config-3.1.2.RELEASE.jar -rw-r--r-- 1 root root 340048 Oct 31 2017 spring-security-core-3.1.2.RELEASE.jar -rw-r--r-- 1 root root 20626 Oct 31 2017 spring-security-kerberos-core-1.0.1.RELEASE.jar -rw-r--r-- 1 root root 7060 Oct 31 2017 spring-security-kerberos-web-1.0.1.RELEASE.jar -rw-r--r-- 1 root root 103988 Oct 31 2017 spring-security-ldap-4.0.4.RELEASE.jar -rw-r--r-- 1 root root 251053 Oct 31 2017 spring-security-web-3.1.2.RELEASE.jar -rw-r--r-- 1 root root 232066 Oct 31 2017 spring-tx-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 398383 Oct 31 2017 spring-web-3.0.7.RELEASE.jar -rw-r--r-- 1 root root 23346 Oct 31 2017 stax-api-1.0-2.jar -rw-r--r-- 1 root root 63777 Oct 31 2017 validation-api-1.1.0.Final.jar -rw-r--r-- 1 root root 449505 Oct 31 2017 velocity-1.7.jar drwxr-xr-x 9 root root 4096 Dec 6 14:56 web -rw-r--r-- 1 root root 15010 Oct 31 2017 xmlenc-0.52.jar -rw-r--r-- 1 root root 94816 Oct 31 2017 xz-1.2.jar -rw-r--r-- 1 root root 792964 Oct 31 2017 zookeeper-3.4.6.jar [root@node101 ~]#
[root@node101 ~]# ll /var/log/ambari-server/ total 308 -rw-r--r-- 1 root root 0 Dec 6 15:17 ambari-alerts.log -rw-r--r-- 1 root root 292 Dec 6 15:43 ambari-audit.log -rw-r--r-- 1 root root 0 Dec 6 15:17 ambari-config-changes.log -rw-r--r-- 1 root root 624 Dec 6 15:35 ambari-eclipselink.log -rw-r--r-- 1 root root 2268 Dec 6 15:35 ambari-server-check-database.log -rw-r--r-- 1 root root 7007 Dec 6 15:35 ambari-server-command.log -rw-r--r-- 1 root root 247628 Dec 6 15:50 ambari-server.log -rw-r--r-- 1 root root 156 Dec 6 15:35 ambari-server.out drwxr-xr-x 2 root root 4096 Dec 6 15:35 capshed-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 files-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 hive20-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 hive-next-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 huetoambarimigration-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 pig-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 slider-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 storm-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 tez-view drwxr-xr-x 2 root root 4096 Dec 6 15:35 wfmanager-view [root@node101 ~]#
4>.配置Ambari-Server
在安装Ambari-Server之后,如果我们直接跳过当前的步骤,直接去执行第5步骤,我们会发现启动服务报错,这是因为在启动服务之前必须要先完成初始配置,执行ambari-server setup命令开始配置。
[root@node101 ~]# ambari-server setup Using python /usr/bin/python #看到这里的数据,同学们估计已经猜到了,没错Ambari-Server会通过Python脚本启动一个交互式的shell程序来引导用户完成配置,这和CDH几乎是如出一辙啊!!! Setup ambari-server Checking SELinux... #咱们再看看这个输出,说明程序配置首先会检查是否已经禁用了SELinux,这项配置在之前步骤已经设置过了,所以下面价差的状态为“disabled” SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? n #紧接着,出现了一个交互的场景,问咱们是否要指定Ambari的用户,这里用户使用默认的Amabri用户,输入字母“n”,或者直接按回车即可。 Adjusting ambari-server permissions and ownership... Checking firewall status... #看到没有,上面检查了selinux的状态,现在又开始检查防火墙的状态啦!下面有紧接着检查JDK环境 Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7 [3] Custom JDK ============================================================================== Enter choice (1): 3 #看到没?让你从上面选择对应的数字,是JDK8还是JDK7,还是咱们自定义安装的jdk,我这里是自定义自定义安装JDK环境的,因此我输入了数字”3”,下面会要求我们指定安装JDK的路径。 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /yinzhengjie/softwares/jdk #既然上面我们选择的是自定义JDK环境的,那么我们得告诉它我们的JAVA_HOME路径在哪。实事求是,按照你安装的路径输入进去即可。 Validating JDK on Ambari Server...done. Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? n #最后是数据库设置,这里为了方便,我就使用了默认的postgresql数据库。大家看到小括号的提示没?他表示默认就是“n”,因此我们是可以直接回车的!我之前也给大家分析了为什么不要使用内置的数据库的原因,但本片博客会展示内置的postgresql数据库存放元数据信息。 Configuring database... Default properties detected. Using built-in database. Configuring ambari database... Checking PostgreSQL... Running initdb: This may take up to a minute. Initializing database ... OK About to start PostgreSQL Configuring local database... Configuring PostgreSQL... Restarting PostgreSQL Creating schema and user... done. Creating tables... done. Extracting system views... ..........ambari-admin-2.6.0.0.267.jar . Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. [root@node101 ~]#
5>.启动Ambari-Server
我们初始化了Ambari-Server的存放元数据信息的数据库以及配置jdk等环境后(也就是第4步骤),我们就可以启动Ambari Serveri数据库啦!
[root@node101 ~]# ambari-server start #输入相应的启动命令,你就会发现启动数据是成功的 Using python /usr/bin/python Starting ambari-server Ambari Server running with administrator privileges. Organizing resource files at /var/lib/ambari-server/resources... Ambari database consistency check started... Server PID at: /var/run/ambari-server/ambari-server.pid Server out at: /var/log/ambari-server/ambari-server.out Server log at: /var/log/ambari-server/ambari-server.log #这个文件是存放日志的,上面2个文件存放的是PID以及 Waiting for server start...................... Server started listening on 8080 #这里自然就是监听服务的端口啦~我们可以通过webUI的方式去访问它。 DB configs consistency check: no errors and warnings were found. Ambari Server 'start' completed successfully. #这里提示我们服务启动成功啦~ [root@node101 ~]#
好啦,既然服务已经启动成功了,那我们也别闲着了,说干就干吧,访问一下WebUI吧:
如上图所示,登陆系统之后,会看到Ambari空空如也的欢迎界面,接下来我们就需要介绍如何通过Ambari新建Hadoop集群。详情请参考我的博客:手把手教你通过Ambari新建Hadoop集群图解案例。