[补充资料] 手动搭建 Cloudera 集群
本课主题
- 集群搭建
- 设置 Web 服务器
- 启动 ClouderManager
- 登入 Cloudera Manager
引言
这部份是一个补充资料，记录如何安装 Cloudera 服务器
集群搭建
- 查看你主机的 IP 地址并且设置静态 IP，这里显示的是 192.168.80.145
ifconfig -a
用图型界面来设置静态 IP
- 修改 network 文件来修改 HOSTNAME
vim /etc/sysconfig/network hostname elephant service network restart
- 修改 /etc/hosts 文件
192.168.80.145 elephant 192.168.80.146 monkey
设置 Web 服务器
- 首先是关掉防火墙
service iptables stop chkconfig iptables off vim /etc/selinux/config setenforce 0
- 查看 httpd 服务和启动状况，如果发现没有安装 httpd 服务，请先安装 sudo yum -y install httpd
rpm -qa | grep httpd[enmoedu@elephant softwares]$ rpm -qa | grep httpd httpd-2.2.15-59.el6.centos.x86_64 httpd-tools-2.2.15-59.el6.centos.x86_64
service httpd status[root@elephant softwares]# service httpd status httpd is stopped [root@elephant softwares]# service httpd start Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 192.168.80.145 for ServerName [ OK ] [root@elephant softwares]# service httpd status httpd (pid 3002) is running...
- 查看 WEB 服务器的状况: http://elephant
- 安装好之后把 cloudera-cdh5 和 cloudera-manager 移动到 /var/www/html/ 文件夹里。
http://elephant/cloudera-manager5.9 http://elephant/cloudera-cdh5.9
此时你可以创建安装包啦，进入 cloudera-cdh5 和 cloudera-manager 创建 repo
createrepo .
createrepo[root@elephant html]# cd cloudera-cdh5.9/ [root@elephant cloudera-cdh5.9]# ls repodata RPMS [root@elephant cloudera-cdh5.9]# createrepo . Spawning worker 0 with 117 pkgs Workers Finished Gathering worker results Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete [root@elephant cloudera-cdh5.9]# cd .. [root@elephant html]# ls cloudera-cdh5.9 cloudera-manager5.9 [root@elephant html]# cd cloudera-manager5.9/ [root@elephant cloudera-manager5.9]# ls repodata RPMS [root@elephant cloudera-manager5.9]# createrepo . Spawning worker 0 with 6 pkgs Workers Finished Gathering worker results Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete
- 完成以上的步骤，服务器已经搭建好了，剩下来是客户端的配置。创建好 repo 之后可以到 /etc/yum.repos.d/ 文件夹创建 repo文件，新建 cloudera-cdh5 的 yum 源。
Cloudera-cdh5.repo[Cloudera-cdh5] name=Cloudera-cdh5 baseurl=http://elephant/Cloudera-cdh5 gpgcheck=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
cloudera-manager.repo[cloudera-manager] name=cloudera-manager baseurl=http://elephant/cloudera-manager gpgcheck=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
completed-codemv Cloudera-cdh5/ cloudera-manager/ /var/www/html/ service httpd status cd /var/www/html/ cd Cloudera-cdh5/ createrepo . cd cloudera-manager/ createrepo . cd /etc/yum.repos.d/ cat << EOF > /etc/yum.repos.d/Cloudera-cdh5.repo [Cloudera-cdh5] name=Cloudera-cdh5 baseurl=http://elephant/Cloudera-cdh5 gpgcheck=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 EOF cat << EOF > /etc/yum.repos.d/cloudera-manager.repo [cloudera-manager] name=cloudera-manager baseurl=http://elephant/cloudera-manager gpgcheck=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 EOF
清理一下 yum
yum clean all[root@elephant yum.repos.d]# yum clean all Loaded plugins: fastestmirror, refresh-packagekit Cleaning repos: base cloudera-cdh5.9 cloudera-manager extras updates Cleaning up Everything Cleaning up list of fastest mirrors
yum install spark-core[root@elephant yum.repos.d]# yum install spark-core Loaded plugins: fastestmirror, refresh-packagekit Setting up Install Process Determining fastest mirrors * base: centos.01link.hk * extras: centos.01link.hk * updates: centos.01link.hk base | 3.7 kB 00:00 base/primary_db | 4.7 MB 00:02 cloudera-cdh5.9 | 2.9 kB 00:00 cloudera-cdh5.9/primary_db | 64 kB 00:00 cloudera-manager | 2.9 kB 00:00 cloudera-manager/primary_db | 8.8 kB 00:00 extras | 3.4 kB 00:00 extras/primary_db | 37 kB 00:00 updates | 3.4 kB 00:00 updates/primary_db | 803 kB 00:00 Resolving Dependencies --> Running transaction check ---> Package spark-core.noarch 0:1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6 will be installed base/filelists_db | 6.4 MB 00:02 cloudera-cdh5.9/filelists_db | 413 kB 00:00 cloudera-manager/filelists_db | 94 kB 00:00 extras/filelists_db | 38 kB 00:00 updates/filelists_db | 644 kB 00:00 --> Processing Dependency: bigtop-utils >= 0.7 for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: parquet for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: hive for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: hadoop-client for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: flume-ng for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: avro-libs for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: /lib/lsb/init-functions for package: spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch --> Running transaction check ---> Package avro-libs.noarch 0:1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6 will be installed ---> Package bigtop-utils.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 will be installed ---> Package flume-ng.noarch 0:1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: kite >= 0.10.0+cdh5.1.0 for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: hive-jdbc >= 1.1.0+cdh5.4.0 for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: hadoop-yarn for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: hadoop-mapreduce for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: hadoop-hdfs for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch --> Processing Dependency: hadoop for package: flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch ---> Package hadoop-client.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: hadoop-0.20-mapreduce = 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 for package: hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 --> Processing Dependency: hadoop-0.20-mapreduce = 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 for package: hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 --> Processing Dependency: zookeeper for package: hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 ---> Package hive.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: sentry for package: hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch ---> Package parquet.noarch 0:1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: parquet-format >= 2.1.0 for package: parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch ---> Package redhat-lsb-core.x86_64 0:4.0-7.el6.centos will be installed --> Processing Dependency: perl-Test-Simple for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: perl-Test-Harness for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: perl-ExtUtils-MakeMaker for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: perl-CGI for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /usr/bin/time for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /usr/bin/pax for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /usr/bin/man for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /usr/bin/bc for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /usr/bin/batch for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /usr/bin/at for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /bin/mailx for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Processing Dependency: /bin/ed for package: redhat-lsb-core-4.0-7.el6.centos.x86_64 --> Running transaction check ---> Package at.x86_64 0:3.1.10-49.el6 will be installed ---> Package bc.x86_64 0:1.06.95-1.el6 will be installed ---> Package ed.x86_64 0:1.1-3.3.el6 will be installed ---> Package hadoop.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: nc for package: hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 ---> Package hadoop-0.20-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed ---> Package hadoop-hdfs.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: bigtop-jsvc for package: hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 ---> Package hadoop-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed ---> Package hadoop-yarn.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 will be installed ---> Package hive-jdbc.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 will be installed ---> Package kite.noarch 0:1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: solr >= 4.4.0+cdh5.1.0 for package: kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch ---> Package mailx.x86_64 0:12.4-8.el6_6 will be installed ---> Package man.x86_64 0:1.6f-39.el6 will be installed ---> Package parquet-format.noarch 0:2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6 will be installed ---> Package pax.x86_64 0:3.4-10.1.el6 will be installed ---> Package perl-CGI.x86_64 0:3.51-144.el6 will be installed --> Processing Dependency: perl = 4:5.10.1-144.el6 for package: perl-CGI-3.51-144.el6.x86_64 ---> Package perl-ExtUtils-MakeMaker.x86_64 0:6.55-144.el6 will be installed --> Processing Dependency: perl-devel for package: perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64 ---> Package perl-Test-Harness.x86_64 0:3.17-144.el6 will be installed ---> Package perl-Test-Simple.x86_64 0:0.92-144.el6 will be installed ---> Package sentry.noarch 0:1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6 will be installed ---> Package time.x86_64 0:1.7-38.el6 will be installed ---> Package zookeeper.x86_64 0:3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6 will be installed --> Running transaction check ---> Package bigtop-jsvc.x86_64 0:0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6 will be installed ---> Package nc.x86_64 0:1.84-24.el6 will be installed ---> Package perl.x86_64 4:5.10.1-141.el6_7.1 will be updated --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: perl-IO-Compress-Base-2.021-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Compress-Raw-Zlib-2.021-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: perl-IO-Compress-Zlib-2.021-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: perl-Compress-Zlib-2.021-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 3:perl-version-0.77-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 4:perl-libs-5.10.1-141.el6_7.1.x86_64 --> Processing Dependency: perl = 4:5.10.1-141.el6_7.1 for package: 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64 ---> Package perl.x86_64 4:5.10.1-144.el6 will be an update ---> Package perl-devel.x86_64 4:5.10.1-144.el6 will be installed --> Processing Dependency: perl(ExtUtils::ParseXS) for package: 4:perl-devel-5.10.1-144.el6.x86_64 --> Processing Dependency: gdbm-devel for package: 4:perl-devel-5.10.1-144.el6.x86_64 --> Processing Dependency: db4-devel for package: 4:perl-devel-5.10.1-144.el6.x86_64 ---> Package solr.noarch 0:4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6 will be installed --> Processing Dependency: bigtop-tomcat for package: solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch --> Running transaction check ---> Package bigtop-tomcat.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 will be installed ---> Package db4-devel.x86_64 0:4.7.25-22.el6 will be installed --> Processing Dependency: db4-cxx = 4.7.25-22.el6 for package: db4-devel-4.7.25-22.el6.x86_64 --> Processing Dependency: db4 = 4.7.25-22.el6 for package: db4-devel-4.7.25-22.el6.x86_64 --> Processing Dependency: libdb_cxx-4.7.so()(64bit) for package: db4-devel-4.7.25-22.el6.x86_64 ---> Package gdbm-devel.x86_64 0:1.8.0-39.el6 will be installed ---> Package perl-Compress-Raw-Zlib.x86_64 1:2.021-141.el6_7.1 will be updated ---> Package perl-Compress-Raw-Zlib.x86_64 1:2.021-144.el6 will be an update ---> Package perl-Compress-Zlib.x86_64 0:2.021-141.el6_7.1 will be updated ---> Package perl-Compress-Zlib.x86_64 0:2.021-144.el6 will be an update ---> Package perl-ExtUtils-ParseXS.x86_64 1:2.2003.0-144.el6 will be installed ---> Package perl-IO-Compress-Base.x86_64 0:2.021-141.el6_7.1 will be updated ---> Package perl-IO-Compress-Base.x86_64 0:2.021-144.el6 will be an update ---> Package perl-IO-Compress-Zlib.x86_64 0:2.021-141.el6_7.1 will be updated ---> Package perl-IO-Compress-Zlib.x86_64 0:2.021-144.el6 will be an update ---> Package perl-Module-Pluggable.x86_64 1:3.90-141.el6_7.1 will be updated ---> Package perl-Module-Pluggable.x86_64 1:3.90-144.el6 will be an update ---> Package perl-Pod-Escapes.x86_64 1:1.04-141.el6_7.1 will be updated ---> Package perl-Pod-Escapes.x86_64 1:1.04-144.el6 will be an update ---> Package perl-Pod-Simple.x86_64 1:3.13-141.el6_7.1 will be updated ---> Package perl-Pod-Simple.x86_64 1:3.13-144.el6 will be an update ---> Package perl-libs.x86_64 4:5.10.1-141.el6_7.1 will be updated ---> Package perl-libs.x86_64 4:5.10.1-144.el6 will be an update ---> Package perl-version.x86_64 3:0.77-141.el6_7.1 will be updated ---> Package perl-version.x86_64 3:0.77-144.el6 will be an update --> Running transaction check ---> Package db4.x86_64 0:4.7.25-20.el6_7 will be updated --> Processing Dependency: db4 = 4.7.25-20.el6_7 for package: db4-utils-4.7.25-20.el6_7.x86_64 ---> Package db4.x86_64 0:4.7.25-22.el6 will be an update ---> Package db4-cxx.x86_64 0:4.7.25-22.el6 will be installed --> Running transaction check ---> Package db4-utils.x86_64 0:4.7.25-20.el6_7 will be updated ---> Package db4-utils.x86_64 0:4.7.25-22.el6 will be an update --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================== Installing: spark-core noarch 1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 112 M Installing for dependencies: at x86_64 3.1.10-49.el6 base 61 k avro-libs noarch 1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 45 M bc x86_64 1.06.95-1.el6 base 110 k bigtop-jsvc x86_64 0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 27 k bigtop-tomcat noarch 0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 7.5 M bigtop-utils noarch 0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 11 k db4-cxx x86_64 4.7.25-22.el6 base 589 k db4-devel x86_64 4.7.25-22.el6 base 6.6 M ed x86_64 1.1-3.3.el6 base 72 k flume-ng noarch 1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 52 M gdbm-devel x86_64 1.8.0-39.el6 base 26 k hadoop x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 27 M hadoop-0.20-mapreduce x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 36 M hadoop-client x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 34 k hadoop-hdfs x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 24 M hadoop-mapreduce x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 32 M hadoop-yarn x86_64 2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 15 M hive noarch 1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 42 M hive-jdbc noarch 1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 46 M kite noarch 1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 114 M mailx x86_64 12.4-8.el6_6 base 235 k man x86_64 1.6f-39.el6 base 265 k nc x86_64 1.84-24.el6 base 57 k parquet noarch 1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 26 M parquet-format noarch 2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 467 k pax x86_64 3.4-10.1.el6 base 69 k perl-CGI x86_64 3.51-144.el6 base 210 k perl-ExtUtils-MakeMaker x86_64 6.55-144.el6 base 294 k perl-ExtUtils-ParseXS x86_64 1:2.2003.0-144.el6 base 46 k perl-Test-Harness x86_64 3.17-144.el6 base 232 k perl-Test-Simple x86_64 0.92-144.el6 base 113 k perl-devel x86_64 4:5.10.1-144.el6 base 424 k redhat-lsb-core x86_64 4.0-7.el6.centos base 25 k sentry noarch 1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 56 M solr noarch 4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 76 M time x86_64 1.7-38.el6 base 26 k zookeeper x86_64 3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6 cloudera-cdh5.9 4.4 M Updating for dependencies: db4 x86_64 4.7.25-22.el6 base 564 k db4-utils x86_64 4.7.25-22.el6 base 131 k perl x86_64 4:5.10.1-144.el6 base 10 M perl-Compress-Raw-Zlib x86_64 1:2.021-144.el6 base 70 k perl-Compress-Zlib x86_64 2.021-144.el6 base 46 k perl-IO-Compress-Base x86_64 2.021-144.el6 base 70 k perl-IO-Compress-Zlib x86_64 2.021-144.el6 base 136 k perl-Module-Pluggable x86_64 1:3.90-144.el6 base 41 k perl-Pod-Escapes x86_64 1:1.04-144.el6 base 33 k perl-Pod-Simple x86_64 1:3.13-144.el6 base 213 k perl-libs x86_64 4:5.10.1-144.el6 base 579 k perl-version x86_64 3:0.77-144.el6 base 52 k Transaction Summary ============================================================================================================================================================== Install 38 Package(s) Upgrade 12 Package(s) Total download size: 736 M Is this ok [y/N]:y Downloading Packages: (1/50): at-3.1.10-49.el6.x86_64.rpm | 61 kB 00:00 (2/50): avro-libs-1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6.noarch.rpm | 45 MB 00:00 (3/50): bc-1.06.95-1.el6.x86_64.rpm | 110 kB 00:00 (4/50): bigtop-jsvc-0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 27 kB 00:00 (5/50): bigtop-tomcat-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch.rpm | 7.5 MB 00:00 (6/50): bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch.rpm | 11 kB 00:00 (7/50): db4-4.7.25-22.el6.x86_64.rpm | 564 kB 00:00 (8/50): db4-cxx-4.7.25-22.el6.x86_64.rpm | 589 kB 00:00 (9/50): db4-devel-4.7.25-22.el6.x86_64.rpm | 6.6 MB 00:02 (10/50): db4-utils-4.7.25-22.el6.x86_64.rpm | 131 kB 00:00 (11/50): ed-1.1-3.3.el6.x86_64.rpm | 72 kB 00:00 (12/50): flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch.rpm | 52 MB 00:00 (13/50): gdbm-devel-1.8.0-39.el6.x86_64.rpm | 26 kB 00:00 (14/50): hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 27 MB 00:00 (15/50): hadoop-0.20-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 36 MB 00:00 (16/50): hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 34 kB 00:00 (17/50): hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 24 MB 00:00 (18/50): hadoop-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 32 MB 00:00 (19/50): hadoop-yarn-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 15 MB 00:00 (20/50): hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch.rpm | 42 MB 00:00 (21/50): hive-jdbc-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch.rpm | 46 MB 00:00 (22/50): kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch.rpm | 114 MB 00:01 (23/50): mailx-12.4-8.el6_6.x86_64.rpm | 235 kB 00:00 (24/50): man-1.6f-39.el6.x86_64.rpm | 265 kB 00:00 (25/50): nc-1.84-24.el6.x86_64.rpm | 57 kB 00:00 (26/50): parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch.rpm | 26 MB 00:00 (27/50): parquet-format-2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6.noarch.rpm | 467 kB 00:00 (28/50): pax-3.4-10.1.el6.x86_64.rpm | 69 kB 00:00 (29/50): perl-5.10.1-144.el6.x86_64.rpm | 10 MB 00:05 (30/50): perl-CGI-3.51-144.el6.x86_64.rpm | 210 kB 00:00 (31/50): perl-Compress-Raw-Zlib-2.021-144.el6.x86_64.rpm | 70 kB 00:00 (32/50): perl-Compress-Zlib-2.021-144.el6.x86_64.rpm | 46 kB 00:00 (33/50): perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64.rpm | 294 kB 00:00 (34/50): perl-ExtUtils-ParseXS-2.2003.0-144.el6.x86_64.rpm | 46 kB 00:00 (35/50): perl-IO-Compress-Base-2.021-144.el6.x86_64.rpm | 70 kB 00:00 (36/50): perl-IO-Compress-Zlib-2.021-144.el6.x86_64.rpm | 136 kB 00:00 (37/50): perl-Module-Pluggable-3.90-144.el6.x86_64.rpm | 41 kB 00:00 (38/50): perl-Pod-Escapes-1.04-144.el6.x86_64.rpm | 33 kB 00:00 (39/50): perl-Pod-Simple-3.13-144.el6.x86_64.rpm | 213 kB 00:00 (40/50): perl-Test-Harness-3.17-144.el6.x86_64.rpm | 232 kB 00:00 (41/50): perl-Test-Simple-0.92-144.el6.x86_64.rpm | 113 kB 00:00 (42/50): perl-devel-5.10.1-144.el6.x86_64.rpm | 424 kB 00:00 (43/50): perl-libs-5.10.1-144.el6.x86_64.rpm | 579 kB 00:00 (44/50): perl-version-0.77-144.el6.x86_64.rpm | 52 kB 00:00 (45/50): redhat-lsb-core-4.0-7.el6.centos.x86_64.rpm | 25 kB 00:00 (46/50): sentry-1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6.noarch.rpm | 56 MB 00:00 (47/50): solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch.rpm | 76 MB 00:00 (48/50): spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch.rpm | 112 MB 00:01 (49/50): time-1.7-38.el6.x86_64.rpm | 26 kB 00:00 (50/50): zookeeper-3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6.x86_64.rpm | 4.4 MB 00:00 -------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 35 MB/s | 736 MB 00:21 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : avro-libs-1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6.noarch 1/62 Installing : bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 2/62 Installing : zookeeper-3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6.x86_64 3/62 Updating : db4-4.7.25-22.el6.x86_64 4/62 Updating : 1:perl-Pod-Escapes-1.04-144.el6.x86_64 5/62 Updating : 4:perl-libs-5.10.1-144.el6.x86_64 6/62 Updating : 1:perl-Pod-Simple-3.13-144.el6.x86_64 7/62 Updating : 1:perl-Module-Pluggable-3.90-144.el6.x86_64 8/62 Updating : 3:perl-version-0.77-144.el6.x86_64 9/62 Updating : 4:perl-5.10.1-144.el6.x86_64 10/62 Updating : 1:perl-Compress-Raw-Zlib-2.021-144.el6.x86_64 11/62 Updating : perl-IO-Compress-Base-2.021-144.el6.x86_64 12/62 Updating : perl-IO-Compress-Zlib-2.021-144.el6.x86_64 13/62 Installing : perl-CGI-3.51-144.el6.x86_64 14/62 Installing : bigtop-jsvc-0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6.x86_64 15/62 Installing : bigtop-tomcat-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 16/62 Installing : nc-1.84-24.el6.x86_64 17/62 Installing : ed-1.1-3.3.el6.x86_64 18/62 Installing : mailx-12.4-8.el6_6.x86_64 19/62 Installing : man-1.6f-39.el6.x86_64 20/62 Installing : db4-cxx-4.7.25-22.el6.x86_64 21/62 Installing : db4-devel-4.7.25-22.el6.x86_64 22/62 Installing : gdbm-devel-1.8.0-39.el6.x86_64 23/62 Installing : perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64 24/62 Installing : perl-Test-Harness-3.17-144.el6.x86_64 25/62 Installing : 1:perl-ExtUtils-ParseXS-2.2003.0-144.el6.x86_64 26/62 Installing : 4:perl-devel-5.10.1-144.el6.x86_64 27/62 Installing : perl-Test-Simple-0.92-144.el6.x86_64 28/62 Installing : time-1.7-38.el6.x86_64 29/62 Installing : bc-1.06.95-1.el6.x86_64 30/62 Installing : pax-3.4-10.1.el6.x86_64 31/62 Installing : at-3.1.10-49.el6.x86_64 32/62 Installing : redhat-lsb-core-4.0-7.el6.centos.x86_64 33/62 Installing : parquet-format-2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6.noarch 34/62 Installing : parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch 35/62 Installing : hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 36/62 Installing : hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 37/62 Installing : hadoop-yarn-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 38/62 Installing : hadoop-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 39/62 Installing : hadoop-0.20-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 40/62 Installing : hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 41/62 Installing : hive-jdbc-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 42/62 Installing : hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 43/62 Installing : sentry-1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6.noarch 44/62 Installing : solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch 45/62 The following warning applies to any collections configured to use Non-SolrCloud mode. Any such collection configuration will need to be upgraded, see Upgrading Cloudera Search for details. Installing : kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch 46/62 Installing : flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch 47/62 Installing : spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch 48/62 Updating : perl-Compress-Zlib-2.021-144.el6.x86_64 49/62 Updating : db4-utils-4.7.25-22.el6.x86_64 50/62 Cleanup : perl-Compress-Zlib-2.021-141.el6_7.1.x86_64 51/62 Cleanup : perl-IO-Compress-Zlib-2.021-141.el6_7.1.x86_64 52/62 Cleanup : perl-IO-Compress-Base-2.021-141.el6_7.1.x86_64 53/62 Cleanup : 1:perl-Compress-Raw-Zlib-2.021-141.el6_7.1.x86_64 54/62 Cleanup : 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64 55/62 Cleanup : 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64 56/62 Cleanup : 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64 57/62 Cleanup : 3:perl-version-0.77-141.el6_7.1.x86_64 58/62 Cleanup : 4:perl-libs-5.10.1-141.el6_7.1.x86_64 59/62 Cleanup : 4:perl-5.10.1-141.el6_7.1.x86_64 60/62 Cleanup : db4-utils-4.7.25-20.el6_7.x86_64 61/62 Cleanup : db4-4.7.25-20.el6_7.x86_64 62/62 Verifying : solr-4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6.noarch 1/62 Verifying : spark-core-1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6.noarch 2/62 Verifying : zookeeper-3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6.x86_64 3/62 Verifying : perl-Compress-Zlib-2.021-144.el6.x86_64 4/62 Verifying : 4:perl-5.10.1-144.el6.x86_64 5/62 Verifying : at-3.1.10-49.el6.x86_64 6/62 Verifying : perl-ExtUtils-MakeMaker-6.55-144.el6.x86_64 7/62 Verifying : redhat-lsb-core-4.0-7.el6.centos.x86_64 8/62 Verifying : pax-3.4-10.1.el6.x86_64 9/62 Verifying : perl-Test-Harness-3.17-144.el6.x86_64 10/62 Verifying : db4-utils-4.7.25-22.el6.x86_64 11/62 Verifying : 1:perl-ExtUtils-ParseXS-2.2003.0-144.el6.x86_64 12/62 Verifying : flume-ng-1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6.noarch 13/62 Verifying : hadoop-0.20-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 14/62 Verifying : 4:perl-libs-5.10.1-144.el6.x86_64 15/62 Verifying : perl-CGI-3.51-144.el6.x86_64 16/62 Verifying : bc-1.06.95-1.el6.x86_64 17/62 Verifying : bigtop-jsvc-0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6.x86_64 18/62 Verifying : parquet-1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6.noarch 19/62 Verifying : hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 20/62 Verifying : hive-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 21/62 Verifying : hadoop-mapreduce-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 22/62 Verifying : bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 23/62 Verifying : 4:perl-devel-5.10.1-144.el6.x86_64 24/62 Verifying : 1:perl-Pod-Simple-3.13-144.el6.x86_64 25/62 Verifying : 1:perl-Pod-Escapes-1.04-144.el6.x86_64 26/62 Verifying : bigtop-tomcat-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6.noarch 27/62 Verifying : parquet-format-2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6.noarch 28/62 Verifying : time-1.7-38.el6.x86_64 29/62 Verifying : avro-libs-1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6.noarch 30/62 Verifying : 1:perl-Compress-Raw-Zlib-2.021-144.el6.x86_64 31/62 Verifying : sentry-1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6.noarch 32/62 Verifying : gdbm-devel-1.8.0-39.el6.x86_64 33/62 Verifying : db4-cxx-4.7.25-22.el6.x86_64 34/62 Verifying : man-1.6f-39.el6.x86_64 35/62 Verifying : db4-4.7.25-22.el6.x86_64 36/62 Verifying : db4-devel-4.7.25-22.el6.x86_64 37/62 Verifying : 1:perl-Module-Pluggable-3.90-144.el6.x86_64 38/62 Verifying : mailx-12.4-8.el6_6.x86_64 39/62 Verifying : 3:perl-version-0.77-144.el6.x86_64 40/62 Verifying : ed-1.1-3.3.el6.x86_64 41/62 Verifying : hive-jdbc-1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6.noarch 42/62 Verifying : hadoop-hdfs-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 43/62 Verifying : perl-IO-Compress-Zlib-2.021-144.el6.x86_64 44/62 Verifying : perl-Test-Simple-0.92-144.el6.x86_64 45/62 Verifying : kite-1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6.noarch 46/62 Verifying : hadoop-client-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 47/62 Verifying : hadoop-yarn-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64 48/62 Verifying : perl-IO-Compress-Base-2.021-144.el6.x86_64 49/62 Verifying : nc-1.84-24.el6.x86_64 50/62 Verifying : db4-utils-4.7.25-20.el6_7.x86_64 51/62 Verifying : perl-IO-Compress-Zlib-2.021-141.el6_7.1.x86_64 52/62 Verifying : 4:perl-5.10.1-141.el6_7.1.x86_64 53/62 Verifying : 1:perl-Compress-Raw-Zlib-2.021-141.el6_7.1.x86_64 54/62 Verifying : perl-IO-Compress-Base-2.021-141.el6_7.1.x86_64 55/62 Verifying : 1:perl-Pod-Simple-3.13-141.el6_7.1.x86_64 56/62 Verifying : 1:perl-Pod-Escapes-1.04-141.el6_7.1.x86_64 57/62 Verifying : db4-4.7.25-20.el6_7.x86_64 58/62 Verifying : 3:perl-version-0.77-141.el6_7.1.x86_64 59/62 Verifying : 4:perl-libs-5.10.1-141.el6_7.1.x86_64 60/62 Verifying : 1:perl-Module-Pluggable-3.90-141.el6_7.1.x86_64 61/62 Verifying : perl-Compress-Zlib-2.021-141.el6_7.1.x86_64 62/62 Installed: spark-core.noarch 0:1.6.0+cdh5.9.0+229-1.cdh5.9.0.p0.30.el6 Dependency Installed: at.x86_64 0:3.1.10-49.el6 avro-libs.noarch 0:1.7.6+cdh5.9.0+115-1.cdh5.9.0.p0.30.el6 bc.x86_64 0:1.06.95-1.el6 bigtop-jsvc.x86_64 0:0.6.0+cdh5.9.0+855-1.cdh5.9.0.p0.30.el6 bigtop-tomcat.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 bigtop-utils.noarch 0:0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el6 db4-cxx.x86_64 0:4.7.25-22.el6 db4-devel.x86_64 0:4.7.25-22.el6 ed.x86_64 0:1.1-3.3.el6 flume-ng.noarch 0:1.6.0+cdh5.9.0+74-1.cdh5.9.0.p0.30.el6 gdbm-devel.x86_64 0:1.8.0-39.el6 hadoop.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hadoop-0.20-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hadoop-client.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hadoop-hdfs.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hadoop-mapreduce.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hadoop-yarn.x86_64 0:2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6 hive.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 hive-jdbc.noarch 0:1.1.0+cdh5.9.0+752-1.cdh5.9.0.p0.30.el6 kite.noarch 0:1.0.0+cdh5.9.0+138-1.cdh5.9.0.p0.30.el6 mailx.x86_64 0:12.4-8.el6_6 man.x86_64 0:1.6f-39.el6 nc.x86_64 0:1.84-24.el6 parquet.noarch 0:1.5.0+cdh5.9.0+176-1.cdh5.9.0.p0.30.el6 parquet-format.noarch 0:2.1.0+cdh5.9.0+14-1.cdh5.9.0.p0.30.el6 pax.x86_64 0:3.4-10.1.el6 perl-CGI.x86_64 0:3.51-144.el6 perl-ExtUtils-MakeMaker.x86_64 0:6.55-144.el6 perl-ExtUtils-ParseXS.x86_64 1:2.2003.0-144.el6 perl-Test-Harness.x86_64 0:3.17-144.el6 perl-Test-Simple.x86_64 0:0.92-144.el6 perl-devel.x86_64 4:5.10.1-144.el6 redhat-lsb-core.x86_64 0:4.0-7.el6.centos sentry.noarch 0:1.5.1+cdh5.9.0+261-1.cdh5.9.0.p0.30.el6 solr.noarch 0:4.10.3+cdh5.9.0+458-1.cdh5.9.0.p0.30.el6 time.x86_64 0:1.7-38.el6 zookeeper.x86_64 0:3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6 Dependency Updated: db4.x86_64 0:4.7.25-22.el6 db4-utils.x86_64 0:4.7.25-22.el6 perl.x86_64 4:5.10.1-144.el6 perl-Compress-Raw-Zlib.x86_64 1:2.021-144.el6 perl-Compress-Zlib.x86_64 0:2.021-144.el6 perl-IO-Compress-Base.x86_64 0:2.021-144.el6 perl-IO-Compress-Zlib.x86_64 0:2.021-144.el6 perl-Module-Pluggable.x86_64 1:3.90-144.el6 perl-Pod-Escapes.x86_64 1:1.04-144.el6 perl-Pod-Simple.x86_64 1:3.13-144.el6 perl-libs.x86_64 4:5.10.1-144.el6 perl-version.x86_64 3:0.77-144.el6 Complete! [root@elephant yum.repos.d]#
- 安裝 cloudera-manager
yum install cloudera-manager-server[root@elephant yum.repos.d]# yum install cloudera-manager-server Loaded plugins: fastestmirror, refresh-packagekit Setting up Install Process Loading mirror speeds from cached hostfile * base: centos.01link.hk * extras: centos.01link.hk * updates: centos.01link.hk Resolving Dependencies --> Running transaction check ---> Package cloudera-manager-server.x86_64 0:5.9.0-1.cm590.p0.249.el6 will be installed --> Processing Dependency: cloudera-manager-daemons = 5.9.0 for package: cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64 --> Running transaction check ---> Package cloudera-manager-daemons.x86_64 0:5.9.0-1.cm590.p0.249.el6 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================== Installing: cloudera-manager-server x86_64 5.9.0-1.cm590.p0.249.el6 cloudera-manager 8.3 k Installing for dependencies: cloudera-manager-daemons x86_64 5.9.0-1.cm590.p0.249.el6 cloudera-manager 528 M Transaction Summary ============================================================================================================================================================== Install 2 Package(s) Total download size: 528 M Installed size: 679 M Is this ok [y/N]: y Downloading Packages: (1/2): cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64.rpm | 528 MB 00:15 (2/2): cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64.rpm | 8.3 kB 00:00 -------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 34 MB/s | 528 MB 00:15 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64 1/2 Installing : cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64 2/2 Verifying : cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64 1/2 Verifying : cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64 2/2 Installed: cloudera-manager-server.x86_64 0:5.9.0-1.cm590.p0.249.el6 Dependency Installed: cloudera-manager-daemons.x86_64 0:5.9.0-1.cm590.p0.249.el6 Complete! [root@elephant yum.repos.d]#
- 在 monkey 主机下安装 cloudera-agent
cloudera-manager-agentyum install cloudera-manager-agent
- 在 elephant 主机下安装 cloudera-server 和 cloudera-manager-server-db
cloudera-manager-serveryum install cloudera-manager-server yum install cloudera-manager-server-db-2
- 完成后可以查看安装了什么程序
rpm -qa | grep cloudera[root@elephant enmoedu]# rpm -qa | grep cloudera cloudera-manager-server-5.9.0-1.cm590.p0.249.el6.x86_64 cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64 cloudera-manager-server-db-2-5.9.0-1.cm590.p0.249.el6.x86_64 cloudera-manager-agent-5.9.0-1.cm590.p0.249.el6.x86_64 [root@monkey ~]# rpm -qa | grep cloudera cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el6.x86_64 cloudera-manager-agent-5.9.0-1.cm590.p0.249.el6.x86_64
chkconfig | grep cloudera[root@elephant yum.repos.d]# chkconfig | grep cloudera cloudera-scm-agent 0:off 1:off 2:off 3:on 4:on 5:on 6:off cloudera-scm-server 0:off 1:off 2:off 3:on 4:on 5:on 6:off cloudera-scm-server-db 0:off 1:off 2:off 3:on 4:on 5:on 6:off
-
安装 Java 并设置环境变量
yum install oracle-j2sdk1.7[root@elephant x86_64]# cd /var/www/html/cloudera-manager5.9/RPMS/x86_64 [root@elephant x86_64]# yum install oracle-j2sdk1.7 Failed to set locale, defaulting to C Loaded plugins: fastestmirror, refresh-packagekit Setting up Install Process Examining oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm: oracle-j2sdk1.7-1.7.0+update67-1.x86_64 Marking oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm to be installed Loading mirror speeds from cached hostfile * base: ftp.cuhk.edu.hk * extras: ftp.cuhk.edu.hk * updates: ftp.cuhk.edu.hk Resolving Dependencies --> Running transaction check ---> Package oracle-j2sdk1.7.x86_64 0:1.7.0+update67-1 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================== Installing: oracle-j2sdk1.7 x86_64 1.7.0+update67-1 /oracle-j2sdk1.7-1.7.0+update67-1.x86_64 279 M Transaction Summary ============================================================================================================================================================== Install 1 Package(s) Total size: 279 M Installed size: 279 M Is this ok [y/N]: y Downloading Packages: Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : oracle-j2sdk1.7-1.7.0+update67-1.x86_64 1/1 Verifying : oracle-j2sdk1.7-1.7.0+update67-1.x86_64 1/1 Installed: oracle-j2sdk1.7.x86_64 0:1.7.0+update67-1 Complete! [root@elephant x86_64]# java -version java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
安装 java 并设置环境变量echo -e 'export JAVA_HOME=/usr/java/default' >> /etc/profile echo -e 'export PATH=$JAVA_HOME/bin:$PATH'>> /etc/profile alternatives --install /usr/bin/java java /usr/java/jdk1.7.0_67-cloudera/bin/java 200 alternatives --set java /usr/java/jdk1.7.0_67-cloudera/bin/java alternatives --install /usr/bin/javac javac /usr/java/jdk1.7.0_67-cloudera/bin/javac 200 alternatives --set javac /usr/java/jdk1.7.0_67-cloudera/bin/javac source /etc/profile
启动 ClouderManager
启动 Cloudera DB 并登录 Clouder Manager 去安装所需要的应用软件
[root@elephant cloudera-manager5.9]# service cloudera-scm-server-db status pg_ctl: no server running [root@elephant cloudera-manager5.9]# service cloudera-scm-server-db start The files belonging to this database system will be owned by user "cloudera-scm". This user must also own the server process. The database cluster will be initialized with locale en_US.UTF8. The default text search configuration will be set to "english". fixing permissions on existing directory /var/lib/cloudera-scm-server-db/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 32MB creating configuration files ... ok creating template1 database in /var/lib/cloudera-scm-server-db/data/base/1 ... ok initializing pg_authid ... ok setting password ... ok initializing dependencies ... ok creating system views ... ok loading system objects' descriptions ... ok creating conversions ... ok creating dictionaries ... ok setting privileges on built-in objects ... ok creating information schema ... ok vacuuming database template1 ... ok copying template1 to template0 ... ok copying template1 to postgres ... ok Success. You can now start the database server using: postgres -D /var/lib/cloudera-scm-server-db/data or pg_ctl -D /var/lib/cloudera-scm-server-db/data -l logfile start Adding configs Creating SCM configuration file: /etc/cloudera-scm-server/db.properties waiting for server to start.... done server started CREATE ROLE CREATE DATABASE Created db properties file /etc/cloudera-scm-server/db.properties Creating DB amon for role ACTIVITYMONITOR CREATE ROLE CREATE DATABASE Created DB for role ACTIVITYMONITOR Creating DB rman for role REPORTSMANAGER CREATE ROLE CREATE DATABASE Created DB for role REPORTSMANAGER Creating DB nav for role NAVIGATOR CREATE ROLE CREATE DATABASE Created DB for role NAVIGATOR Creating DB navms for role NAVIGATORMETASERVER CREATE ROLE CREATE DATABASE Created DB for role NAVIGATORMETASERVER Enabled remote connections waiting for server to shut down.... done server stopped DB initialization done. waiting for server to start.... done server started [root@elephant cloudera-manager5.9]#
[root@elephant cloudera-manager5.9]# service cloudera-scm-server start Starting cloudera-scm-server: [OK]
查看Cloudera DB 的状态
[root@elephant yum.repos.d]# service cloudera-scm-server-db status pg_ctl: server is running (PID: 2062) /usr/bin/postgres "-D" "/var/lib/cloudera-scm-server-db/data"
登入 Cloudera Manager
- 登入 ClouderaManager 然后输入用户名和密码：admin/ admin
http://elephant:7180
- 选择版本
- 选择安装的主机名称
- 选择 yum 的安装源
http://elephant/cloudera-cdh5.9 http://elephant/cloudera-manager5.9
- 安装中
- 检查系统
- 检查 DB
- 安装并启动所有服务
- 然后登入 Cloudera Manager
问题
- cloudera-scm-server 启动不了，请检查 clouder 的日志 /var/log/cloudera-scm-server/cloudera-scm-server.out
/var/log/cloudera-scm-server/cloudera-scm-server.out[root@elephant cloudera-manager5.9]# service cloudera-scm-server start Starting cloudera-scm-server: [FAILED] [root@elephant log]# cd /var/log/cloudera-scm-server/ [root@elephant cloudera-scm-server]# ls cloudera-scm-server.out db.log [root@elephant cloudera-scm-server]# cat cloudera-scm-server.out +======================================================================+ | Error: JAVA_HOME is not set and Java could not be found | +----------------------------------------------------------------------+ | Please download the latest Oracle JDK from the Oracle Java web site | | > http://www.oracle.com/technetwork/java/javase/index.html < | | | | Cloudera Manager requires Java 1.6 or later. | | NOTE: This script will find Oracle Java whether you install using | | the binary or the RPM based installer. | +======================================================================+
- check the log file: tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log[root@elephant enmoedu]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log 2017-05-04 23:34:08,832 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties 2017-05-04 23:34:08,833 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties 2017-05-04 23:34:08,834 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties 2017-05-04 23:34:08,840 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.mgmt.properties 2017-05-04 23:34:08,840 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.properties 2017-05-04 23:34:08,840 INFO main:com.cloudera.server.cmf.components.EmbeddedDbManager: Reading embedded database password from /var/lib/cloudera-scm-server-db/data/generated_password.txt 2017-05-04 23:34:08,841 INFO main:com.cloudera.enterprise.CommonMain: Reading database properties from /etc/cloudera-scm-server/db.properties 2017-05-04 23:34:08,895 INFO main:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Staleness check will use thread pool of size 1 2017-05-04 23:34:08,901 INFO main:com.cloudera.cmf.cdhclient.util.CDHUrlClassLoader: Detected that this program is running in a JAVA 1.7.0_67 JVM. CDH5 jars will be loaded from:lib/cdh5 2017-05-04 23:34:08,902 INFO main:com.cloudera.cmf.cdhclient.CdhExecutorFactory: Adding options to bad options list:
- 可能会遇到 Host Service is not running 信息，此时你需要检查你有没有启动 Cloudera Manager Service
- 启动 Host Monitor 和 Service Monitor
- 启动 Cluster
- Request to the Service Monitor failed. This may cause slow page responses. View the status of the Service Monitor.
/var/log/cloudera-scm-firehose/[root@elephant cloudera-scm-server]# cd /var/log/cloudera-scm-firehose/ [root@elephant cloudera-scm-firehose]# ls mgmt-cmf-mgmt-HOSTMONITOR-elephant.log.out mgmt-cmf-mgmt-SERVICEMONITOR-elephant.log.out stacks
https://community.cloudera.com/t5/Cloudera-Manager-Installation/Request-to-the-Host-Monitor-failed-This-may-cause-slow-page/td-p/24126 - stopping the server cloudera-scm-server
service cloudera-scm-server stop[root@elephant enmoedu]# service cloudera-scm-server stop Stopping cloudera-scm-server: [ OK ] [root@elephant enmoedu]# service cloudera-scm-server-db stop waiting for server to shut down.... done server stopped
-
遇上这个错误请检查是不是两个主机都启动了 httpd 服务
service httpd status[root@monkey ~]# service httpd status httpd is stopped [root@monkey ~]# service httpd start Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.16.201.156 for ServerName [ OK ] [root@monkey ~]# service httpd status httpd (pid 24668) is running...
# cloudera master server alias elephant='ssh -i "~/.ssh/key.pem" centos@xxx.xxx.xxx' alias monkey='ssh -i "~/.ssh/key.pem" centos@xxx.xxx.xxx' alias tiger='ssh -i "~/.ssh/key.pem" centos@xxx.xxx.xxx' scp -i ~/.ssh/key.pem training_materials.zip centos@xxx.xxx.xxx:/home/centos #AWS centos os 6 sudo yum update -y sudo yum install vim -y sudo yum install wget -y sudo yum install gcc -y sudo yum install unzip -y echo "done" cat << EOF > ~/.vimrc set number syntax enable set ts=4 set expandtab set cursorline set shiftwidth=4 EOF cat << EOF >> ~/.bashrc export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera/ export CLASSPATH=/usr/java/jdk1.7.0_67-cloudera/jre/ export PYTHON_HOME=/usr/src/Python-3.4.5 alias python3=python3.4 alias pip3=pip3.4 PATH=/usr/local/bin:$JAVA_HOME/bin:$PYTHON_HOME/bin:$PATH:$HOME/bin EOF cat << EOF > /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 xxx.xxx.xxx elephant xxx.xxx.xxx monkey xxx.xxx.xxx tiger EOF # update hostname cat << EOF > /etc/sysconfig/network NETWORKING=yes HOSTNAME=elephant NETWORKING_IPV6=no NOZEROCONF=yes HOSTS=elephant EOF hostname elephant # update hostname cat << EOF > /etc/sysconfig/network NETWORKING=yes HOSTNAME=monkey NETWORKING_IPV6=no NOZEROCONF=yes HOSTS=monkey EOF hostname monkey # update hostname cat << EOF > /etc/sysconfig/network NETWORKING=yes HOSTNAME=tiger NETWORKING_IPV6=no NOZEROCONF=yes HOSTS=tiger EOF hostname tiger useradd enmoedu passwd enmoedu cat << EOF > /etc/sysconfig/selinux # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=disabled # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targeted EOF service iptables stop chkconfig iptables off sudo setenforce 0 sestatus # check the selinux #update hosts file vim /etc/ssh/sshd_config # update a few confguration to yes RSAAuthentication yes PubkeyAuthentication yes PermitRootLogin yes PasswordAuthentication yes # restart the sshd service sudo systemctl restart sshd sudo service sshd restart sudo systemctl stop iptables.service ssh-keygen -f $HOME/.ssh/id_rsa -t rsa -N '' # generate ssh key su enmoedu # switch to enmoedu to generate the ssh key ssh-keygen -f $HOME/.ssh/id_rsa -t rsa -N '' exit # switch back to root cat /home/enmoedu/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys cat /home/enmoedu/.ssh/id_rsa.pub >> /home/enmoedu/.ssh/authorized_keys cat /root/.ssh/id_rsa.pub >> /home/enmoedu/.ssh/authorized_keys chown enmoedu:enmoedu /home/enmoedu/.ssh/authorized_keys cat << EOF >> /etc/sudoers.d/90-cloud-init-users root ALL=(ALL:ALL) ALL %admin ALL=(ALL) ALL %sudo ALL=(ALL:ALL) ALL enmoedu ALL=(ALL)NOPASSWD:ALL EOF cat << EOF >> /etc/sudoers root ALL=(ALL:ALL) ALL %admin ALL=(ALL) ALL %sudo ALL=(ALL:ALL) ALL enmoedu ALL=(ALL)NOPASSWD:ALL EOF # edit the vm swswappiness cat << EOF > /etc/sysctl.conf # sysctl settings are defined through files in # /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/. # # Vendors settings live in /usr/lib/sysctl.d/. # To override a whole file, create a new file with the same in # /etc/sysctl.d/ and put new settings there. To override # only specific settings, add a file with a lexically later # name in /etc/sysctl.d/ and put new settings there. # # For more information, see sysctl.conf(5) and sysctl.d(5). vm.swappiness=10 EOF cat << EOF > /etc/rc.local #!/bin/bash # THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES # # It is highly advisable to create own systemd services or udev rules # to run scripts during boot instead of using this file. # # In contrast to previous versions due to parallel execution during boot # this script will NOT be run after all other services. # # Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure # that this script will be executed during boot. touch /var/lock/subsys/local "echo never > /sys/kernel/mm/transparent_hugepage/defrag" "echo never > /sys/kernel/mm/transparent_hugepage/enabled" EOF cd ~/ wget http://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin chmod +x cloudera-manager-installer.bin sudo ./cloudera-manager-installer.bin # create a enmoedu folder in hadoop sudo -u hdfs hadoop fs -mkdir /user/enmoedu sudo -u hdfs hadoop fs -chmod +w /user/enmoedu sudo -u hdfs hadoop fs -chown enmoedu:enmoedu /user/enmoedu # download the python source code cd /usr/src wget https://www.python.org/ftp/python/3.4.5/Python-3.4.5.tgz tar xzf Python-3.4.5.tgz cd Python-3.4.5 ./configure make altinstall rm Python-3.4.5.tgz python3 -V pip3 install numpy pip3 install matplotlib pip3 install kazoo reboot -h now # upload the cloudera training material to Server sudo -s sudo mkdir /home/enmoedu/src sudo chown enmoedu:enmoedu /home/enmoedu/src sudo mv training_materials.zip /home/enmoedu/src su enmoedu unzip training_materials.zip mv training_materials/ ../ sudo chown enmoedu:enmoedu training_materials.zip # Set install Mysql (switch to monkey server) rpm -Uvh http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm yum install mysql-community-server /usr/bin/systemctl enable mysqld reboot -h now # download MySQL JDBC connector cd ~ wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.43.tar.gz tar zxvf mysql-connector-java-5.1.43.tar.gz mkdir /usr/share/java/ cd /usr/share/java/ cp ~/mysql-connector-java-5.1.43/mysql-connector-java-5.1.43-bin.jar ./ mv mysql-connector-java-5.1.43-bin.jar mysql-connector-java.jar # Set Up Hive mysql -u root CREATE USER 'hive'@'%' IDENTIFIED BY 'hive'; GRANT ALL ON *.* TO 'hive'@'%'; FLUSH PRIVILEGES; EXIT; mysql -h monkey -u hive -p # Connect to Beeline beeline -u jdbc:hive2://monkey:10000 create database testdb; show databases; use testdb; show tables; create table testdb.test_tb1(id INT, name STRING); drop table testdb.test_tb1; insert into testdb.test_tb1 values (1, "Alex");
参考资料
[1] Cloudera Manager 5 和 CDH5 本地（离线）安装指南
