CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager
CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.搭建CM私有仓库
详情请参考我的笔记： https://www.cnblogs.com/yinzhengjie/p/10412455.html。
1>.安装web服务器并启动
[root@node105 ~]# yum -y install httpd Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/2): extras/7/x86_64/primary_db | 180 kB 00:00:04 (2/2): updates/7/x86_64/primary_db | 2.4 MB 00:01:01 Resolving Dependencies --> Running transaction check ---> Package httpd.x86_64 0:2.4.6-88.el7.centos will be installed --> Processing Dependency: httpd-tools = 2.4.6-88.el7.centos for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-88.el7.centos.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-88.el7.centos.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.8-3.el7_4.1 will be installed ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed ---> Package httpd-tools.x86_64 0:2.4.6-88.el7.centos will be installed ---> Package mailcap.noarch 0:2.1.41-2.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================================== Installing: httpd x86_64 2.4.6-88.el7.centos base 2.7 M Installing for dependencies: apr x86_64 1.4.8-3.el7_4.1 base 103 k apr-util x86_64 1.5.2-6.el7 base 92 k httpd-tools x86_64 2.4.6-88.el7.centos base 90 k mailcap noarch 2.1.41-2.el7 base 31 k Transaction Summary ======================================================================================================================================================================== Install 1 Package (+4 Dependent packages) Total download size: 3.0 M Installed size: 10 M Downloading packages: (1/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:00:00 (2/5): apr-1.4.8-3.el7_4.1.x86_64.rpm | 103 kB 00:00:00 (3/5): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:00 (4/5): httpd-tools-2.4.6-88.el7.centos.x86_64.rpm | 90 kB 00:00:00 (5/5): httpd-2.4.6-88.el7.centos.x86_64.rpm | 2.7 MB 00:00:21 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 143 kB/s | 3.0 MB 00:00:21 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : apr-1.4.8-3.el7_4.1.x86_64 1/5 Installing : apr-util-1.5.2-6.el7.x86_64 2/5 Installing : httpd-tools-2.4.6-88.el7.centos.x86_64 3/5 Installing : mailcap-2.1.41-2.el7.noarch 4/5 Installing : httpd-2.4.6-88.el7.centos.x86_64 5/5 Verifying : httpd-tools-2.4.6-88.el7.centos.x86_64 1/5 Verifying : apr-1.4.8-3.el7_4.1.x86_64 2/5 Verifying : mailcap-2.1.41-2.el7.noarch 3/5 Verifying : httpd-2.4.6-88.el7.centos.x86_64 4/5 Verifying : apr-util-1.5.2-6.el7.x86_64 5/5 Installed: httpd.x86_64 0:2.4.6-88.el7.centos Dependency Installed: apr.x86_64 0:1.4.8-3.el7_4.1 apr-util.x86_64 0:1.5.2-6.el7 httpd-tools.x86_64 0:2.4.6-88.el7.centos mailcap.noarch 0:2.1.41-2.el7 Complete! [root@node105 ~]#
[root@node105 ~]# cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.`date +%F` [root@node105 ~]# [root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #' AddType application/x-gzip .gz .tgz [root@node105 ~]# [root@node105 ~]# sed -i s'#.tgz#.tgz .parcel#' /etc/httpd/conf/httpd.conf [root@node105 ~]# [root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #' AddType application/x-gzip .gz .tgz .parcel #其实我就是在.tgz后面加了一个“.parcel”的后缀，这是CM官方强调要做的一个操作！ [root@node105 ~]# [root@node105 ~]#
[root@node105 ~]# systemctl enable httpd Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service. [root@node105 ~]# [root@node105 ~]# systemctl start httpd [root@node105 ~]# [root@node105 ~]# [root@node105 ~]# systemctl status httpd ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Active: active (running) since Fri 2019-02-22 11:02:22 CST; 3s ago Docs: man:httpd(8) man:apachectl(8) Main PID: 4674 (httpd) Status: "Processing requests..." CGroup: /system.slice/httpd.service ├─4674 /usr/sbin/httpd -DFOREGROUND ├─4675 /usr/sbin/httpd -DFOREGROUND ├─4676 /usr/sbin/httpd -DFOREGROUND ├─4677 /usr/sbin/httpd -DFOREGROUND ├─4678 /usr/sbin/httpd -DFOREGROUND └─4679 /usr/sbin/httpd -DFOREGROUND Feb 22 11:02:22 node105.yinzhengjie.org.cn systemd[1]: Starting The Apache HTTP Server... Feb 22 11:02:22 node105.yinzhengjie.org.cn systemd[1]: Started The Apache HTTP Server. [root@node105 ~]# [root@node105 ~]#
2>.下载CM对应到版本到对应到自建到web路径中（https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.16.1/RPMS/x86_64/）
[root@node105 ~]# mkdir /var/www/html/cm-5.16.1 #创建CM安装包存放目录 [root@node105 ~]# [root@node105 ~]# ll /var/www/html/cm-5.16.1/ #去官方网站把你需要的rpm包都下载到你到安装目录即可！ total 953108 -rwxr-xr-x 1 root root 9864584 Feb 21 18:22 cloudera-manager-agent-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 789872988 Feb 21 12:21 cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 8704 Feb 21 12:21 cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 10612 Feb 21 12:21 cloudera-manager-server-db-2-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 176209195 Feb 21 18:22 jdk-8u201-linux-x64.rpm #这个软件是我从Oracle官方上download下来的，CM默认的JDK是6或者7，而Oracle官方已经不支持使用JDK7啦！
-rw-r--r-- 1 root root 1690 Feb 21 2018 RPM-GPG-KEY-cloudera [root@node105 ~]# [root@node105 ~]#
3>.制作本地镜像
[root@node105 ~]# yum -y install yum-utils createrepo yum-plugin-priorities Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package createrepo.noarch 0:0.9.9-28.el7 will be installed --> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: libxml2-python for package: createrepo-0.9.9-28.el7.noarch --> Processing Dependency: deltarpm for package: createrepo-0.9.9-28.el7.noarch ---> Package yum-plugin-priorities.noarch 0:1.1.31-50.el7 will be installed ---> Package yum-utils.noarch 0:1.1.31-50.el7 will be installed --> Processing Dependency: python-kitchen for package: yum-utils-1.1.31-50.el7.noarch --> Running transaction check ---> Package deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package libxml2-python.x86_64 0:2.9.1-6.el7_2.3 will be installed ---> Package python-deltarpm.x86_64 0:3.6-3.el7 will be installed ---> Package python-kitchen.noarch 0:1.1.1-5.el7 will be installed --> Processing Dependency: python-chardet for package: python-kitchen-1.1.1-5.el7.noarch --> Running transaction check ---> Package python-chardet.noarch 0:2.2.1-1.el7_1 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================================== Installing: createrepo noarch 0.9.9-28.el7 base 94 k yum-plugin-priorities noarch 1.1.31-50.el7 base 29 k yum-utils noarch 1.1.31-50.el7 base 121 k Installing for dependencies: deltarpm x86_64 3.6-3.el7 base 82 k libxml2-python x86_64 2.9.1-6.el7_2.3 base 247 k python-chardet noarch 2.2.1-1.el7_1 base 227 k python-deltarpm x86_64 3.6-3.el7 base 31 k python-kitchen noarch 1.1.1-5.el7 base 267 k Transaction Summary ======================================================================================================================================================================== Install 3 Packages (+5 Dependent packages) Total download size: 1.1 M Installed size: 4.8 M Downloading packages: (1/8): libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm | 247 kB 00:00:00 (2/8): createrepo-0.9.9-28.el7.noarch.rpm | 94 kB 00:00:00 (3/8): python-deltarpm-3.6-3.el7.x86_64.rpm | 31 kB 00:00:00 (4/8): deltarpm-3.6-3.el7.x86_64.rpm | 82 kB 00:00:00 (5/8): yum-utils-1.1.31-50.el7.noarch.rpm | 121 kB 00:00:00 (6/8): python-chardet-2.2.1-1.el7_1.noarch.rpm | 227 kB 00:00:00 (7/8): yum-plugin-priorities-1.1.31-50.el7.noarch.rpm | 29 kB 00:00:00 (8/8): python-kitchen-1.1.1-5.el7.noarch.rpm | 267 kB 00:00:03 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 332 kB/s | 1.1 MB 00:00:03 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : deltarpm-3.6-3.el7.x86_64 1/8 Installing : libxml2-python-2.9.1-6.el7_2.3.x86_64 2/8 Installing : python-deltarpm-3.6-3.el7.x86_64 3/8 Installing : python-chardet-2.2.1-1.el7_1.noarch 4/8 Installing : python-kitchen-1.1.1-5.el7.noarch 5/8 Installing : yum-utils-1.1.31-50.el7.noarch 6/8 Installing : createrepo-0.9.9-28.el7.noarch 7/8 Installing : yum-plugin-priorities-1.1.31-50.el7.noarch 8/8 Verifying : yum-utils-1.1.31-50.el7.noarch 1/8 Verifying : python-deltarpm-3.6-3.el7.x86_64 2/8 Verifying : yum-plugin-priorities-1.1.31-50.el7.noarch 3/8 Verifying : libxml2-python-2.9.1-6.el7_2.3.x86_64 4/8 Verifying : deltarpm-3.6-3.el7.x86_64 5/8 Verifying : createrepo-0.9.9-28.el7.noarch 6/8 Verifying : python-chardet-2.2.1-1.el7_1.noarch 7/8 Verifying : python-kitchen-1.1.1-5.el7.noarch 8/8 Installed: createrepo.noarch 0:0.9.9-28.el7 yum-plugin-priorities.noarch 0:1.1.31-50.el7 yum-utils.noarch 0:1.1.31-50.el7 Dependency Installed: deltarpm.x86_64 0:3.6-3.el7 libxml2-python.x86_64 0:2.9.1-6.el7_2.3 python-chardet.noarch 0:2.2.1-1.el7_1 python-deltarpm.x86_64 0:3.6-3.el7 python-kitchen.noarch 0:1.1.1-5.el7 Complete! [root@node105 ~]#
[root@node105 ~]# cd /var/www/html/cm-5.16.1/ [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# ll total 953108 -rwxr-xr-x 1 root root 9864584 Feb 21 18:22 cloudera-manager-agent-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 789872988 Feb 21 12:21 cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 8704 Feb 21 12:21 cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 10612 Feb 21 12:21 cloudera-manager-server-db-2-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 176209195 Feb 21 18:22 jdk-8u201-linux-x64.rpm -rw-r--r-- 1 root root 1690 Feb 21 2018 RPM-GPG-KEY-cloudera [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# createrepo . #创建yum源的元数据文件！ Spawning worker 0 with 3 pkgs Spawning worker 1 with 2 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]# ll total 953112 -rwxr-xr-x 1 root root 9864584 Feb 21 18:22 cloudera-manager-agent-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 789872988 Feb 21 12:21 cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 8704 Feb 21 12:21 cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 10612 Feb 21 12:21 cloudera-manager-server-db-2-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm -rwxr-xr-x 1 root root 176209195 Feb 21 18:22 jdk-8u201-linux-x64.rpm drwxr-xr-x 2 root root 4096 Feb 22 11:12 repodata #这个目录就是我们执行上面的命令后生成的目录,该文件不可被删除！否则你配置的yum的使用者依旧无法使用yum命令进行安装 -rw-r--r-- 1 root root 1690 Feb 21 2018 RPM-GPG-KEY-cloudera [root@node105 cm-5.16.1]# [root@node105 cm-5.16.1]#
4>.下载CDH的版本存放到指定路径（https://archive.cloudera.com/cdh5/parcels/5.15.1/）
[root@node105 ~]# mkdir /var/www/html/cdh-5.15.1 [root@node105 ~]# [root@node105 ~]# cd /var/www/html/cdh-5.15.1/ [root@node105 cdh-5.15.1]# [root@node105 cdh-5.15.1]# ll total 2070488 -rw-r--r-- 1 root root 2120090032 Feb 22 11:19 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Feb 22 11:19 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha -rw-r--r-- 1 root root 73767 Feb 22 11:19 manifest.json [root@node105 cdh-5.15.1]#
5>.各个主机配置CM自建的yum仓库
[root@node101 ~]# cat /etc/yum.repos.d/cloudera-manager.repo [cloudera-manager] # Packages for Cloudera Manager, Version 5, on RedHat or CentOS 7 x86_64 name=Cloudera Manager baseurl=http://node105.yinzhengjie.org.cn/cm-5.16.1/ gpgkey =http://node105.yinzhengjie.org.cn/cm-5.16.1//RPM-GPG-KEY-cloudera enabled = 1 gpgcheck = 0 [root@node101 ~]#
二.安装和配置元数据库
详情请参考我的笔记：安装和配置元数据库。
三.Cloudera Manager 部署
安装服务前请确保你磁盘的可用空间大于50G！
1>.安装Cloudera Manager Server端（由于我们使用自建的镜像地址安装，而且也配置好了yum源，我们这里直接安装即可！）
[root@node101 ~]# yum -y install cloudera-manager-daemons cloudera-manager-server Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.163.com * extras: mirror01.idc.hinet.net * updates: centos.cs.nctu.edu.tw Resolving Dependencies --> Running transaction check ---> Package cloudera-manager-daemons.x86_64 0:5.16.1-1.cm5161.p0.1.el7 will be installed ---> Package cloudera-manager-server.x86_64 0:5.16.1-1.cm5161.p0.1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================================================== Package Arch Version Repository Size ======================================================================================================================================================================== Installing: cloudera-manager-daemons x86_64 5.16.1-1.cm5161.p0.1.el7 cloudera-manager 753 M cloudera-manager-server x86_64 5.16.1-1.cm5161.p0.1.el7 cloudera-manager 8.5 k Transaction Summary ======================================================================================================================================================================== Install 2 Packages Total download size: 753 M Installed size: 935 M Downloading packages: (1/2): cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm | 8.5 kB 00:00:00 (2/2): cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64.rpm | 753 MB 00:00:14 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 51 MB/s | 753 MB 00:00:14 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64 1/2 Installing : cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64 2/2 Verifying : cloudera-manager-server-5.16.1-1.cm5161.p0.1.el7.x86_64 1/2 Verifying : cloudera-manager-daemons-5.16.1-1.cm5161.p0.1.el7.x86_64 2/2 Installed: cloudera-manager-daemons.x86_64 0:5.16.1-1.cm5161.p0.1.el7 cloudera-manager-server.x86_64 0:5.16.1-1.cm5161.p0.1.el7 Complete! [root@node101 ~]#
2>.配置Cloudera Manager Server的默认堆内存大小
[root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server #Cloudera Manager默认分配的JVM堆内存是2G，某些情况下会出现OOM的情况，我们可以适当将其调大。 export CMF_JAVA_OPTS="-Xmx2G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" [root@node101 ~]# [root@node101 ~]# sed -i s'#Xmx2G#Xmx4G#' /etc/default/cloudera-scm-server #我们将Cloudera Manager的堆内存调大到4G。 [root@node101 ~]# [root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server export CMF_JAVA_OPTS="-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp" [root@node101 ~]# [root@node101 ~]#
3>.初始化数据库配置
Cloudera官网已经提供了一个sh脚本用户配置Cloudera Manager连接元数据库，官方文档参考：
我们的安装规划下，元数据库和Cloudera Manager运行于同一个节点，且已经预先创建了scm库和用户，因此，命令很简单，具体操作如下：
[root@node101 ~]# /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm yinzhengjie JAVA_HOME=/usr/java/jdk1.8 Verifying that we can write to /etc/cloudera-scm-server Creating SCM configuration file in /etc/cloudera-scm-server Executing: /usr/java/jdk1.8/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly! #注意，如果你看到了这一行提示，说明配置完成，否则需要根据提示解决相应的错误！ [root@node101 ~]#
4>.启动Cloudera Manager Server服务
[root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: inactive (dead) Docs: man:systemd-sysv-generator(8) Feb 21 12:29:08 node101.yinzhengjie.org.cn systemd[1]: [/run/systemd/generator.late/cloudera-scm-server.service:13] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument Feb 21 12:29:08 node101.yinzhengjie.org.cn systemd[1]: [/run/systemd/generator.late/cloudera-scm-server.service:13] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument [root@node101 ~]# [root@node101 ~]# systemctl start cloudera-scm-server #启动Cloudera Manager，所需要的时间可能在5～10秒左右！ [root@node101 ~]# [root@node101 ~]# systemctl status cloudera-scm-server ● cloudera-scm-server.service - LSB: Cloudera SCM Server Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled) Active: active (exited) since Thu 2019-02-21 12:53:21 CST; 2min 38s ago Docs: man:systemd-sysv-generator(8) Process: 7253 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS) Feb 21 12:53:16 node101.yinzhengjie.org.cn systemd[1]: Starting LSB: Cloudera SCM Server... Feb 21 12:53:16 node101.yinzhengjie.org.cn su[7277]: PAM unable to dlopen(/usr/lib64/security/pam_fprintd.so): /usr/lib64/security/pam_fprintd.so: cannot open shared object file: No such file or directory Feb 21 12:53:16 node101.yinzhengjie.org.cn su[7277]: PAM adding faulty module: /usr/lib64/security/pam_fprintd.so Feb 21 12:53:16 node101.yinzhengjie.org.cn su[7277]: (to cloudera-scm) root on none Feb 21 12:53:21 node101.yinzhengjie.org.cn cloudera-scm-server[7253]: Starting cloudera-scm-server: [ OK ] Feb 21 12:53:21 node101.yinzhengjie.org.cn systemd[1]: Started LSB: Cloudera SCM Server. [root@node101 ~]#
5>.查看Cloudera Manager Server的日志并访问WebUI
[root@node101 ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log #查看Cloudera Manager的服务的日志 2019-02-21 12:54:38,541 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Initializing SearchTemplateManager:2019-02-21T04:54:38.540Z 2019-02-21 12:54:38,673 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating entities:2019-02-21T04:54:38.673Z 2019-02-21 12:54:38,721 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num entities:208 2019-02-21 12:54:38,721 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating documents:2019-02-21T04:54:38.721Z 2019-02-21 12:54:38,795 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num docs:221 2019-02-21 12:54:38,823 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Constructing repo:2019-02-21T04:54:38.823Z 2019-02-21 12:54:39,541 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4 2019-02-21 12:54:39,631 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180 #注意，如果出现这一行提示，说明你的服务启动成功啦！默认端口为7180，下面有WebServerImpl：Started Jetty server提示！ 2019-02-21 12:54:39,631 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server. 2019-02-21 12:54:39,827 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2019-02-21T04:54:39.827Z
打开浏览器，访问：http://node101.yinzhengjie.org.cn:7180/cmf/login，如下图所示：
四.安装CDH5.15.1
1>.登陆Cloudera Manager Server 的WebUI
2>.接受协议，点击继续
3>.选择企业版（该版本的CDH是免费的）
4>.选择CDH
5>.为 CDH 群集安装指定主机。
6>.点击更多选项
7>.选择自建的CDH Parcels包镜像地址
8>.配置Cloudera Manager Agent的地址
9>.不要选中安装oracle Java，因为我们已经安装过JDK8，选中的话会装会JDK7，直接点击继续即可！
10>.不启用单用户模式安装，直接点击继续即可
11>.提供SSH登陆凭据
12>.等待Cloudera Manager Agent 安装完成
13>.查看安装过程的日志详情
14>.安装 Cloudera Manager Agent成功（根据集群性而定，我使用的是MAC，该过程大概5～10分钟左右！）
15>.下载CDH（从我们自建的镜像地址下载）
16>.CDH安装完成，点击继续
17>.等待主机检查
18>.点击完成
19>.完成上述步骤就已经完成了CDH的安装，还有一种方法安装CDH，就是下载CDH并存放到指定目录（了解即可！）
第一步：下载CDH 5.15.1的parcels包（https://archive.cloudera.com/cdh5/parcels/5.15.1/）
第二步：将下载的rpm包存放到CM默认到目录中（/opt/cloudera/parcel-repo）
[root@node101 ~]# ll /opt/cloudera/parcel-repo/ total 2070488 -rw-r--r-- 1 root root 2120090032 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 -rw-r--r-- 1 root root 73767 Feb 21 17:12 manifest.json [root@node101 ~]# [root@node101 ~]# mv /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha #我们这里需要做一个重命名的操作！ [root@node101 ~]# [root@node101 ~]# ll /opt/cloudera/parcel-repo/ total 2070488 -rw-r--r-- 1 root root 2120090032 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Feb 21 17:12 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha -rw-r--r-- 1 root root 73767 Feb 21 17:12 manifest.json [root@node101 ~]#
五.安装Hadoop初始化的服务
1>.选择自定义服务
2>.选择需要安装的服务，点击继续（为了减少虚拟机的开销和安装时间，我这里只演示安装zookeeper集群和hdfs集群即可！其他服务的安装方法类似）
3>.自定义角色分配
4>.审核更改（这里咱们可以修改一些默认的配置，比如hdfs的块大小，数据存储的路径等等，实验环境我们默认即可，生产环境需要根据实际情况稍作修改即可！）
5>.安装初始化服务（安装过程时间可能需要3～5分钟）
6>.服务安装完成
7>.查看Cloudera Manager的监控界面
8>.安装服务完成后，我们可以查看到剩余的可用空间
本文来自博客园，作者：尹正杰，转载请注明原文链接：https://www.cnblogs.com/yinzhengjie/p/10413793.html，个人微信: "JasonYin2020"（添加时请备注来源及意图备注，有偿付费）
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。
