基于Ambari Server部署HDP集群实战案例

          基于Ambari Server部署HDP集群实战案例

                              作者:尹正杰

版权声明:原创作品,谢绝转载!否则将追究法律责任。

 

 

 

一.部署Ambari Server端

  博主推荐阅读:
    https://www.cnblogs.com/yinzhengjie2020/p/12219108.html

 

二.时钟同步

  我们在生产环境中必须同步各个服务器时间是一致的,最好是有内网的时间服务器。

  配置时间同步的好处:
    可以定位集群故障发生的具体时间;
    解决HBase,Kudu等存储依赖于时间同步的问题(若时间不同步会报错);

  CentOS 6.x默认使用ntpd软件来同步时间的,但在CentOS 7.x版本之后就被更替为chrony来实现时间同步,因此咱们安装时间同步就是用CentOS 7.x推荐的Chrony来进行时间同步。

1>.安装chrony

[root@hdp101.yinzhengjie.org.cn ~]# yum -y install chrony
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirror.bit.edu.cn
 * extras: mirror.bit.edu.cn
 * updates: mirrors.huaweicloud.com
ambari-repo                                                                                                                                                                                                                                            | 2.9 kB  00:00:00     
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
mysql-connectors-community                                                                                                                                                                                                                             | 2.5 kB  00:00:00     
mysql-tools-community                                                                                                                                                                                                                                  | 2.5 kB  00:00:00     
mysql80-community                                                                                                                                                                                                                                      | 2.5 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
(2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package chrony.x86_64 0:3.4-1.el7 will be installed
--> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
--> Running transaction check
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                            Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
Installing for dependencies:
 libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 306 k
Installed size: 788 k
Downloading packages:
(1/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
(2/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         957 kB/s | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
  Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 

Installed:
  chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   

Dependency Installed:
  libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             

Complete!
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# yum -y install chrony

2>.禁用本地时间服务器从公网同步时间

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf 
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#\1@g' /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 

3>.新增时间同步服务器为本机地址

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

4>.设置允许来chronyd服务器端同步时间的客户端地址

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# grep ^#allow /etc/chrony.conf 
#allow 192.168.0.0/16
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@#(allow) 192.168.0.0/16@\1 172.200.0.0/21@' /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 172.200.0.0/21          #允许客户端IP地址同步
allow 127/8                #允许本地同步时间
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

5>.设置chronyd服务器端允许返回本地时间

[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 172.200.0.0/21
allow 127/8
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# grep ^#local /etc/chrony.conf
#local stratum 10
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# sed -r -i 's@#(local)@\1@' /etc/chrony.conf 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 172.200.0.0/21
allow 127/8
local stratum 10           #即使自己未能通过网络时间服务器同步到时间,也允许将本地时间作为标准时间授时给其它客户端,而我们上面设置了禁用默认公网的时间服务器进行同步(换句话说,这里表示不去同步任何服务器的时间)。
logdir /var/log/chrony
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

6>.启动chronyd并设置为开机自启动

[root@hdp101.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# systemctl enable chronyd
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep chronyd
chronyd.service                               enabled 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

7>.查看chronyd正在使用哪个时间服务器进行时间同步

[root@hdp101.yinzhengjie.org.cn ~]# chronyc sources
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* hdp101.yinzhengjie.org.cn    10   6   377    39  -1609ns[-4236ns] +/-   16us
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

8>.hdp102.yinzhengjie.org.cn客户端同步时间

[root@hdp102.yinzhengjie.org.cn ~]# yum -y install chrony
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
(2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package chrony.x86_64 0:3.4-1.el7 will be installed
--> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
--> Running transaction check
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                            Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
Installing for dependencies:
 libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 306 k
Installed size: 788 k
Downloading packages:
(1/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
(2/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         1.5 MB/s | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
  Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 

Installed:
  chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   

Dependency Installed:
  libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             

Complete!
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# yum -y install chrony
[root@hdp102.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#\1@g' /etc/chrony.conf
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# vim /etc/chrony.conf
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:chronyd(8)
           man:chrony.conf(5)
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl enable chronyd
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl list-units -t service | grep chronyd
  chronyd.service                    loaded active running NTP client/server
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-01-31 23:38:04 CST; 20s ago
     Docs: man:chronyd(8)
           man:chrony.conf(5)
 Main PID: 5424 (chronyd)
   CGroup: /system.slice/chronyd.service
           └─5424 /usr/sbin/chronyd

Jan 31 23:38:04 hdp102.yinzhengjie.org.cn systemd[1]: Starting NTP client/server...
Jan 31 23:38:04 hdp102.yinzhengjie.org.cn chronyd[5424]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
Jan 31 23:38:04 hdp102.yinzhengjie.org.cn systemd[1]: Started NTP client/server.
Jan 31 23:38:09 hdp102.yinzhengjie.org.cn chronyd[5424]: Selected source 172.200.1.101
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# 

9>.hdp103.yinzhengjie.org.cn客户端同步时间

[root@hdp103.yinzhengjie.org.cn ~]# yum -y install chrony
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
(2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package chrony.x86_64 0:3.4-1.el7 will be installed
--> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
--> Running transaction check
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                            Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
Installing for dependencies:
 libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 306 k
Installed size: 788 k
Downloading packages:
(1/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
(2/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         1.5 MB/s | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
  Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 

Installed:
  chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   

Dependency Installed:
  libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             

Complete!
[root@hdp103.yinzhengjie.org.cn ~]# 
 
[root@hdp103.yinzhengjie.org.cn ~]# yum -y install chrony
[root@hdp103.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#\1@g' /etc/chrony.conf
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server hdp101.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:chronyd(8)
           man:chrony.conf(5)
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-01-31 23:42:09 CST; 1s ago
     Docs: man:chronyd(8)
           man:chrony.conf(5)
  Process: 5457 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS)
  Process: 5453 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 5455 (chronyd)
   CGroup: /system.slice/chronyd.service
           └─5455 /usr/sbin/chronyd

Jan 31 23:42:09 hdp103.yinzhengjie.org.cn systemd[1]: Starting NTP client/server...
Jan 31 23:42:09 hdp103.yinzhengjie.org.cn chronyd[5455]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
Jan 31 23:42:09 hdp103.yinzhengjie.org.cn systemd[1]: Started NTP client/server.
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# 
[root@hdp103.yinzhengjie.org.cn ~]# systemctl start chronyd
[root@hdp103.yinzhengjie.org.cn ~]# systemctl enable chronyd
Created symlink from /etc/systemd/system/multi-user.target.wants/chronyd.service to /usr/lib/systemd/system/chronyd.service.
[root@hdp103.yinzhengjie.org.cn ~]# 

10>.chronyd服务端修改时间

[root@hdp101.yinzhengjie.org.cn ~]# date 
Fri Jan 31 23:44:30 CST 2020
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# date -s "2020-01-31 15:45:00"
Fri Jan 31 15:45:00 CST 2020
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# hwclock -w
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# date 
Fri Jan 31 15:45:02 CST 2020
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# chronyc sources
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^? hdp101.yinzhengjie.org.cn     0   7   377     -     +0ns[   +0ns] +/-    0ns
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# chronyc sourcestats -v
210 Number of sources = 1
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
hdp101.yinzhengjie.org.cn   0   0     0     +0.000   2000.000     +0ns  4000ms
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

11>.chronyd客户端同步时间

[root@hdp102.yinzhengjie.org.cn ~]# date 
Fri Jan 31 23:56:05 CST 2020
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# chronyc sourcestats -v
210 Number of sources = 1
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
hdp101.yinzhengjie.org.cn   6   4   347     -0.085      3.307  -4050ns    64us
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# systemctl restart chronyd
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# chronyc sourcestats -v
210 Number of sources = 1
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
hdp101.yinzhengjie.org.cn   3   3     4    -20.657   1349.596   -382ns  6474ns
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# 
[root@hdp102.yinzhengjie.org.cn ~]# date 
Fri Jan 31 15:57:07 CST 2020
[root@hdp102.yinzhengjie.org.cn ~]# 

 

三.部署Ambari集群

1>.浏览器访问"http://hdp101.yinzhengjie.org.cn:8080/",使用默认的用户名(admin)和密码(admin)登录成功后启动安装向导(如下图所示,点击"LAUNCH INSTALL WIZARD") 

2>.自定义集群名称,并点击下一步

3>.选择HDP版本,并选择本地软件源仓库

4>.删除不相关操作系统的仓库信息,只保留部署Ambari的操作系统的仓库信息

5>.注册主机

6>.确认匹配的主机名称

7>.等待自动注册

8>.等待集群自

9>.集群自检完成

  如果此步骤和上图一样直接跳过下面的阅读,如果你安装其它节点存在问题,如下图所示,别慌,查看日志解决问题才是正道。

  查看报错信息,如下图所示。

  根据报错信息将hdp101.yinzhengjie.org.cn与hdp[102-103].yinzhengjie.org.cn不同的文件同步后,之前两个失败的节点最终状态变为"Success"状态。

10>.查看集群自建结果(此步骤可省略,直接进行下一步)

11>.自定义需要安装的服务(服务勾选少了也没多大关系,后面可以手动添加服务)

12>.勾选Ambari Metircs(会自动安装监控已经勾选服务的状态)服务并点击"NEXT",如果弹出如下图所示的提示,点击"PROCEED ANYWAY"

13>.根据服务器实际资源分配Master服务角色安装到具体的主机

14>.除了NFSGateway角色(如有需要后续可以自行安装)不勾选,其它的slave服务都勾选

15>.为Grafana Admin和Activity Explorer's Admin服务设置密码

16>.自定义各个服务对应数据的存储路径(由于我们使用的是root用户安装,因此尽管指定的目录不存在,Ambari会通过root用户自动创建的哟~)

17>.确认各个服务在操作系统中以哪个用户身份运行(保持默认即可,直接下一步)

 18>.自定义安装服务的配置并点击下一步

19>.在Review步骤无需做任何配置,而是对之前几个步骤的配置有一个大概的信息概览,如果确认配置五五后点击"DEPLOY"就正式部署啦

20>.点击"DEPLOY"会开始初始化任务 

21>.初始化完就开始部署了  

22>.等待服务自动安装,启动和测试

23>.服务安装成功

24>.安装完成后点击上图的“COMPLETE”会自动进入到"Dashboard"的监控页面

25>.手动测试HDFS服务是否安装成功

[root@hdp101.yinzhengjie.org.cn ~]# hdfs dfs -ls /
Found 6 items
drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /atsv2
drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /hdp
drwxr-xr-x   - mapred hdfs            0 2020-01-31 23:37 /mapred
drwxrwxrwx   - hdfs   hdfs            0 2020-01-31 23:39 /tmp
drwxr-xr-x   - hdfs   hdfs            0 2020-01-31 23:37 /user
drwxrwxrwx   - mapred hadoop          0 2020-01-31 23:40 /yinzhengjie
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# hdfs dfs -df -h
Filesystem                              Size   Used  Available  Use%
hdfs://hdp101.yinzhengjie.org.cn:8020  4.0 T  1.2 G      4.0 T    0%
[root@hdp101.yinzhengjie.org.cn ~]# 
[root@hdp101.yinzhengjie.org.cn ~]# 

 

posted @ 2020-01-22 00:34  JasonYin2020  阅读(1029)  评论(0编辑  收藏  举报