打赏

Ubuntu14.04下完美安装cloudermanage多种方式（图文详解）（博主推荐）

 

 

　　

说在前面的话

　　我的机器是总共4台，分别为ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

 

　　ClouderaManager官网

https://www.cloudera.com/documentation/manager/5-0-x/Cloudera-Manager-Installation-Guide/cm5ig_install_path_B.html

 

 

 

 

　　在ubuntucmbigdata1上执行了

sudo apt-get install cloudera-manager-daemons cloudera-manager-server 


sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent

　　即，ubuntucmbigdata1是既做server，又做agent。

 

 

　　在ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4上执行了

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent

　　即，ubuntucmbigdata2、3、和4是只做agent。

 

 

　　在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list，复制以下内容到cloudera.list里

# Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

 

　　并且在在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

　　向apt添加cloudera源公钥，在所有节点执行如下命令：

curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

然后，再

sudo apt-get update

　

 

 

 

 

方式1、将CDH5的软件源添加到Apt中，通过在线安装。

参考

Ubuntu14.04用apt在线/离线安装CDH5.1.2[Apache Hadoop 2.3.0]

 https://www.zybuluo.com/ncepuwanghui/note/474966

 配置Cloudera软件仓库

Cloudera官方文档提供了两种管理仓库策略： 
• Standard Cloudera repositories 
• Internally hosted repositories

本文采用Standard Cloudera repositories，步骤如下： 
① 在ubuntucmbigdata1节点下载cloudera.list

wget https://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/cloudera.list?_ga=1.37493987.2113753785.1471598714

 

② 在ubuntucmbigdata1节点新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list，复制cloudera.list的内容

# Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

③ 将cloudera-manager.list分发到所有slave节点：

scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata2:/etc/apt/sources.list.d/cloudera-manager.list
scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata3:/etc/apt/sources.list.d/cloudera-manager.list
scp /etc/apt/sources.list.d/cloudera-manager.list bigdata@ubuntucmbigdata4:/etc/apt/sources.list.d/cloudera-manager.list

 

  1. ④ 向apt添加cloudera源公钥，在所有节点执行如下命令：
curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

 

  1. ⑤ 在所有节点，更新源
sudo apt-get update

 

 

若更新出错，提示如下：

  1. W: GPG error: http://archive.cloudera.com trusty-cm5 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 327574EE02A818DD

可执行如下命令，添加cloudera源的公钥，重新更新源：

    1. curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

 　　

 

　　

 

安装Cloudera Manager Server

    • 安装Cloudera Manager Server软件包 
      在maubuntucmbigdata1ster节点上执行：

      1. sudo apt-get install cloudera-manager-daemons cloudera-manager-server

    • 创建Cloudera Manager Server数据库scm

    • mysql -u root -p
create database scm DEFAULT CHARACTER SET utf8;
grant all on scm.* TO 'scm'@'%' IDENTIFIED BY 'scm_password';
grant all on *.* TO 'scm'@'ubuntucmbigdata1' IDENTIFIED BY 'scm_password';
grant all on scm.* TO 'scm'@'localhost' IDENTIFIED BY 'scm_password';
flush privileges;

       

    • 同时，关于这里的mysql配置，还要加上，我这篇博客里的mysql内容

      Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04）（一）

      • mysql> create database hive DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hive'@'ubuntucmbigdata1' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';  
Ignoring query to other database
mysql> create database oozie DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on *.* TO 'oozie'@'ubuntucmbigdata1' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> create database amon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'localhost' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'amon'@'ubuntucmbigdata1' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> create database smon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'localhost' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'smon'@'ubuntucmbigdata1' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> create database rman DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'localhost' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on *.* TO 'rman'@'ubuntucmbigdata1' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> create database hmon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'localhost' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hmon'@'ubuntucmbigdata1' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> create database nav DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'localhost' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on *.* TO 'nav'@'ubuntucmbigdata1' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'%' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> flush privileges;
        mysql> exit;
Bye
root@ubuntucmbigdata1:~#

         

    • 配置启动Cloudera Manager Server （只在ubuntucmbigdata1上执行）

      1. sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password

       

    • 也许大家会出现这个错误。

    • root@ubuntucmbigdata1:~# sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password
+======================================================================+
|      Error: JAVA_HOME is not set and Java could not be found         |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site  |
|  > http://www.oracle.com/technetwork/java/javase/index.html <        |
|                                                                      |
| Cloudera Manager requires Java 1.6 or later.                         |
| NOTE: This script will find Oracle Java whether you install using    |
|       the binary or the RPM based installer.                         |
+======================================================================+
root@ubuntucmbigdata1:~#

      则，需要安装jdk。

    • 成功的话，则是如下
    • bigdata@ubuntucmbigdata1:/usr/java$ sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password
JAVA_HOME=/usr/java/jdk1.7.0_80
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.7.0_80/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!
bigdata@ubuntucmbigdata1:/usr/java$

       

    • 安装JDK1.7 （所有节点）
      CDH要求至少是Oracle JDK7，Ubuntu自带的是OpenJDK，Ubuntu没有自带的Oracle JDK,必须添加外部源才能安装JDK。 
      ① 采用Cloudera软件源 
      在上一步配置Cloudera软件源后，可直接执行下面命令安装：

      1. sudo apt-get install oracle-j2sdk1.7

      ② 采用PPP源 
      添加PPA repository到系统,配置PPP源安装：

      1. sudo add-apt-repository ppa:webupd8team/java
      2. sudo apt-get update
      3. sudo apt-get upgrade
      4. sudo apt-get install oracle-java7-installer
      5. sudo apt-get install oracle-java7-set-default
      6. java -version

      注：若提示如下信息：

      1. sha256sum mismatch jdk-7u80-linux-x64.tar.gz
      2. Oracle JDK 7 is NOT installed.

      可能是jdk软件包没有下载完全，可执行如下命令删除后重新安装：

      1. sudo apt-get purge oracle-java7-installer

      ③ 手动安装 
      从Oracle官网下载JDK7u80：jdk-7u80-linux-x64.tar.gz，解压到/usr/java/jdk.1.7.0_80

      1. mkdir /usr/java
      2. sudo tar -zxvf jdk-7u80-linux-x64.tar.gz -C /usr/java

      配置JDK环境变量： 

    • vim /etc/profile

      • source /etc/profile

        #jdk
export JAVA_HOME=/usr/java/jdk1.7.0_80
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin

         

 

　　如果上述安装错了，查看和卸载命令。

Ubuntu系统里如何正确卸载oracle-j2sdk1.6（图文详解）

　　则为

dpkg --list | grep -i jdk
sudo apt-get remove oracle-j2sdk1.6
sudo apt-get purge oracle-java7-set-default
sudo apt-get purge  oracle-java7-installer

 

　　　　安装orcal-java7-installer出错，则见

　　orcal-java7-installer中断错误 CSDN

 

 

 

 

  • 　sudo service cloudera-scm-server start（只在ubuntucmbigdata1上执行）
  • 成功，则出现
  • bigdata@ubuntucmbigdata1:/usr/java$ sudo service cloudera-scm-server start
Starting cloudera-scm-server:  * cloudera-scm-server started

bigdata@ubuntucmbigdata1:/usr/java$

     

  •  

    若启动报错：

    1. /etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")
    2. /etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")

    修改配置文件/etc/default/cloudera-scm-server，找到export CM_JAVA_OPTS部分，将-Xmx2G修改为-Xmx4G

  • 查看日志，验证是否成功启动

    1. tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
 
 
 
 
 

安装Cloudera Manager Agents

    • 安装Cloudera Manager Agent软件包 
      在所有节点（包括ubuntucmbigdata1）上执行：

      1. sudo apt-get install cloudera-manager-agent cloudera-manager-daemons

    • 配置Cloudera Manager Agent 
      在所有节点上，修改配置文件sudo vi /etc/cloudera-scm-agent/config.ini，配置Cloudera Manager Server的主机和端口：

      1. server_host=ubuntucmbigdata1
      2. server_port=7182 #默认端口即可

    • 启动Cloudera Manager Agent（包含ubuntucmbigdata1）

      1. sudo service cloudera-scm-agent start

 

bigdata@ubuntucmbigdata1:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 

bigdata@ubuntucmbigdata2:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 



bigdata@ubuntucmbigdata3:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$ 





bigdata@ubuntucmbigdata4:~$ sudo service cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata1:~$

 

 

 

 　　

 

 

 

 

 

　　

方式2、放在/opt/softwares/cm-5.6.0

参考

Cloudera Manager安装之Cloudera Manager 5.3.X安装（三）（tar方式、rpm方式和yum方式）

　　

 

　　先不用安装什么jdk，以及什么其他的。只如下执行就是

 

 

　　执行cloudera-manager-installer.bin

复制代码
复制代码
bigdata@ubuntucmbigdata1:~$ sudo mkdir -p /opt/softwares/cm-5.6.0
[sudo] password for bigdata: 
bigdata@ubuntucmbigdata1:~$ cd /opt/softwares/cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ pwd
/opt/softwares/cm-5.6.0
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 22:19 ./
drwxr-xr-x 3 root root 4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata /opt/softwares/*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 22:19 ./
drwxr-xr-x 3 root    root    4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 root    root    517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata cloudera-manager-installer.bin 
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chmod +x ./cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$
复制代码
复制代码

 




 

复制代码
复制代码
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ pwd
/etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 12
drwxr-xr-x 2 root root 4096  6月 25 20:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl "http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/cloudera.list" -o /etc/apt/sources.list.d/cloudera_precise.list
[sudo] password for bigdata: 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   263  100   263    0     0    422      0 --:--:-- --:--:-- --:--:--   422
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096  6月 25 22:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  263  6月 25 22:26 cloudera_precise.list
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
OK
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo apt-get update
复制代码
复制代码

 

 

 

 

 

bigdata@ubuntucmbigdata1:~$ cd /etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096 Jun 26 11:10 ./
drwxr-xr-x 6 root root 4096 Jun 26 11:10 ../
-rw-r--r-- 1 root root  175 Oct 20  2015 cloudera-cm5.list
-rw-r--r-- 1 root root  263 Jun 26 11:10 cloudera_precise.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$

 

 

 

 

 

bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo ./cloudera-manager-installer.bin

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 　　省略。

 

 

　　也许中间，会报错，这个一一解决就是了

 

 

bigdata@ubuntucmbigdata1:~$ cat /var/log/cloudera-manager-installer/3.install-oracle-j2sdk1.7.log
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  libntdb1 python-ntdb
Use 'apt-get autoremove' to remove them.
The following NEW packages will be installed:
  oracle-j2sdk1.7
0 upgraded, 1 newly installed, 0 to remove and 6 not upgraded.
2 not fully installed or removed.
Need to get 142 MB of archives.
After this operation, 292 MB of additional disk space will be used.
Get:1 http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/ trusty-cm5/contrib oracle-j2sdk1.7 amd64 1.7.0+update67-1 [142 MB]
Fetched 142 MB in 15min 9s (157 kB/s)
Selecting previously unselected package oracle-j2sdk1.7.
(Reading database ... 172281 files and directories currently installed.)
Preparing to unpack .../oracle-j2sdk1.7_1.7.0+update67-1_amd64.deb ...
Unpacking oracle-j2sdk1.7 (1.7.0+update67-1) ...
Setting up oracle-java7-installer (7u80+7u60arm-0~webupd8~1) ...
Downloading Oracle Java 7...
--2017-06-26 00:20:41--  http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
Resolving download.oracle.com (download.oracle.com)... 23.2.16.218, 23.2.16.121
Connecting to download.oracle.com (download.oracle.com)|23.2.16.218|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz [following]
--2017-06-26 00:20:41--  https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
Resolving edelivery.oracle.com (edelivery.oracle.com)... 96.16.169.83, 2402:4f00:4001:197::2d3e, 2402:4f00:4001:180::2d3e
Connecting to edelivery.oracle.com (edelivery.oracle.com)|96.16.169.83|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1498407762_def59b5e0df725a8c8cee2b6d12ac74e [following]
--2017-06-26 00:20:42--  http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1498407762_def59b5e0df725a8c8cee2b6d12ac74e
Connecting to download.oracle.com (download.oracle.com)|23.2.16.218|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-06-26 00:20:43 ERROR 404: Not Found.

download failed
Oracle JDK 7 is NOT installed.
dpkg: error processing package oracle-java7-installer (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of oracle-java7-set-default:
 oracle-java7-set-default depends on oracle-java7-installer; however:
  Package oracle-java7-installer is not configured yet.

dpkg: error processing package oracle-java7-set-default (--configure):
 dependency problems - leaving unconfigured
Setting up oracle-j2sdk1.7 (1.7.0+update67-1) ...
No apport report written because the error message indicates its a followup error from a previous failure.
Errors were encountered while processing:
 oracle-java7-installer
 oracle-java7-set-default
E: Sub-process /usr/bin/dpkg returned an

 

 

 

　　解决这个问题，还是得自己手动下载来安装，才得以解决。

 　　它的下载链接

http://download.oracle.com/otn/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz

 

 

 

#java
export JAVA_HOME=/usr/java/jdk1.7.0_80
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin

 

 

 

 

 

bigdata@ubuntucmbigdata1:/usr/java$ sudo vim ~/.bashrc

 

 

 

 

 

 

总结

　　对于这步安装jdk，需要注意，见

https://www.zybuluo.com/ncepuwanghui/note/474966

Ubuntu Server 14.04 安装部署 CDH5.7.2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

　　这里，我也觉得很奇怪，在ubuntu系统里，是localhost，可我明明注销掉了。这跟centos下安装有点不同。

 

 

 　　但是呢，竟然可以用ubuntucmbigdata1:7810或者192.168.86.31来访问。

 

 

 

　　当安装完成，Cloudera Manager 管理控制台会提供一个完成的url包括默认端口7180。（这个安装过程可能持续很长时间，网速较慢，可能会持续一上午，甚至一天）
这时候把这个地址输入浏览器地址栏，就能看到页面了。默认用户名和密码都为admin.

 

 

 

 

 

 　　成功！

 

 

 　　若是通过bin方式安装的话，则

 

bigdata@ubuntucmbigdata1:/etc/init.d$ pwd
/etc/init.d
bigdata@ubuntucmbigdata1:/etc/init.d$ ls
acpid    apparmor      bluetooth            cloudera-scm-server-db  cups          dns-clean          halt        killprocs  mysql       ondemand    procps      rc.local  reboot      rsync    sendsigs  speech-dispatcher  thermald  umountnfs.sh         urandom
anacron  apport        brltty               console-setup           cups-browsed  friendly-recovery  irqbalance  kmod       networking  postgresql  pulseaudio  rcS       resolvconf  rsyslog  single    ssh                udev      umountroot           x11-common
apache2  avahi-daemon  cloudera-scm-server  cron                    dbus          grub-common        kerneloops  lightdm    ntp         pppd-dns    rc          README    rpcbind     saned    skeleton  sudo               umountfs  unattended-upgrades
bigdata@ubuntucmbigdata1:/etc/init.d$ sudo /etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * cloudera-scm-server started

bigdata@ubuntucmbigdata1:/etc/init.d$ sudo /etc/init.d/cloudera-scm-server-db start
 * Cloudera manager database already started
bigdata@ubuntucmbigdata1:/etc/init.d$

 

 

 

 

 

bigdata@ubuntucmbigdata3:~$ cd /etc/init.d/
bigdata@ubuntucmbigdata3:/etc/init.d$ ls
acpid    apparmor      bluetooth           console-setup  cups-browsed  friendly-recovery  irqbalance  kmod        ntp       procps      rc.local  reboot      rsync    sendsigs  speech-dispatcher  thermald  umountnfs.sh         urandom
anacron  apport        brltty              cron           dbus          grub-common        kerneloops  lightdm     ondemand  pulseaudio  rcS       resolvconf  rsyslog  single    ssh                udev      umountroot           x11-common
apache2  avahi-daemon  cloudera-scm-agent  cups           dns-clean     halt               killprocs   networking  pppd-dns  rc          README    rpcbind     saned    skeleton  sudo               umountfs  unattended-upgrades
bigdata@ubuntucmbigdata3:/etc/init.d$ sudo ./cloudera-scm-agent start
cloudera-scm-agent is already running
bigdata@ubuntucmbigdata3:/etc/init.d$

 

 

 

 

 

 

 

 

 

 

 

 方式3，通过tar,gz方法

其实很简单，网上写的资料各种杂活。就是如下！（博主，我是很喜欢这个人写的这篇博客）！！！！

在线和离线安装Cloudera CDH 5.6.0

 

　　的确，我也跟这位博主一样，也需要在实验室里部署。以及也是4台机器。（至于原因，大家都懂得，中国高校研究生喜欢4、3、2、1来进行加速比做实验论文）

 

     在实验室安装过社区版的Hadoop，当时用的是普通PC机，安装的过程遇到了好多问题。当解决了大部分的安装问题之后由于实验室的需求，需要安装一个商业发行版的hadoop ，我们就采用的Cloudera 公司发行的版本。首先要说明的是CDH对内存要求比较高，当我们采用原来的4G内存的PC机，只要启动基本内存就满了，并且会拖慢整个集群的运算速度。所以说安装CDH对硬件的要求比较高。安装CDH 有三种安装方式：在线安装、半离线安装、离线安装。在线安装和离线安装我都安装过，虽然在线安装貌似简单但是实际用起来还是没有离线安装方便，我还是建议大家采用离线安装的方式。不过下面都会介绍在线安装的方式和离线安装的方式。希望对别人有所帮助。

 

 　　关于，前期的介绍、硬件基础、安装Ubuntu14.04系统、设置hosts、关闭防火墙、设置静态IP、安装Oracle JDK 8（所有节点）、所有节点免密码SSH登陆、建立数据库（仅主机）、设置集群时间同步(所有节点）。我都上面已经做好了。

 

 

 

相关软件和下载

Cloudera Manager : http://archive.cloudera.com/cm5/cm/5/   
Cloudera CDH：http://archive.cloudera.com/cdh5 
MySql的JDBC驱动jar包 

 

 

　　因为，我这里是ubuntu14.04，所以

 

 

 

　　同理，我这里，也不去一枚地追求新，采用的是5.6.0

 

 

 

 　　这个网上多的是，大家随便去下载就是了

 

 

 

 

 

 

 


根据系统下载所相应的版本，下载的文件如下

cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha
manifest.json 
mysql-connector-java-5.1.32-bin.jar

 



 


 


 


 　　


　　解压 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 到 /opt 目录下


 　　先弄清楚，为什么这么做，是因为，在线安装的话，默认是安装在/opt下。所以，我这里带大家离线安装Cloudera安装搭建部署大数据集群，更是如此，要放在/opt下了。


 


 


 


 


 




bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 8
drwxr-xr-x  2 root root 4096  2月 18  2016 ./
drwxr-xr-x 23 root root 4096  6月 24 15:09 ../
bigdata@ubuntucmbigdata1:/opt$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt$ ll
total 505696
drwxr-xr-x  2 root root      4096  6月 25 18:37 ./
drwxr-xr-x 23 root root      4096  6月 24 15:09 ../
-rw-r--r--  1 root root 517818680  6月 25 18:22 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
bigdata@ubuntucmbigdata1:/opt$ sudo tar -zxvf cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 




 


 


 






复制代码


bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 505700
drwxr-xr-x  3 1106  592      4096  2月 12  2016 ./
drwxr-xr-x 23 root root      4096  6月 24 15:09 ../
-rw-r--r--  1 root root 517818680  6月 25 18:22 cloudera-manager-trusty-cm5.6.0_amd64.tar.gz
drwxr-xr-x  8 1106  592      4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ sudo rm cloudera-manager-trusty-cm5.6.0_amd64.tar.gz 
bigdata@ubuntucmbigdata1:/opt$ sudo chown -R bigdata:bigdata cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ ll
total 12
drwxr-xr-x  3    1106     592 4096  6月 25 18:46 ./
drwxr-xr-x 23 root    root    4096  6月 24 15:09 ../
drwxr-xr-x  8 bigdata bigdata 4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ 


复制代码




 


 


 


 


 


　　添加mysql connector包到/opt/cm-5.6.0/share/cmf/lib下


 


 






复制代码


bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/share/cmf/lib$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/share/cmf/lib$ ls
activation-1.1.1.jar                       commons-fileupload-1.2.2.jar       gson-2.2.jar                                   jaxb-impl-2.1.13.jar                     mx4j-3.0.1.jar                            spring-security-crypto-3.1.0.RELEASE.jar
agent-5.6.0.jar                            commons-httpclient-3.1.jar         guava-14.0.jar                                 jaxp-api-1.4.jar                         mx4j-tools-3.0.1.jar                      spring-security-ldap-3.1.0.RELEASE.jar
alarms-5.6.0.jar                           commons-io-2.4.jar                 hawtjni-runtime-1.8.jar                        jboss-logging-3.1.0.GA.jar               mysql-connector-java-5.1.32-bin.jar       spring-security-saml2-core-1.0.1.RELEASE.jar
alertpublisher-5.6.0.jar                   commons-lang-2.6.jar               headlamp-5.6.0.jar                             jcommander-1.30.jar                      nav-common-5.6.0.jar                      spring-security-web-3.1.0.RELEASE.jar


复制代码




 


 


 


 


 　　下面，参考这篇博客，非常方便


Cloudera Manager 5和CDH5离线安装


 


 


 


创建用户(所有节点)


　　创建cloudera-scm用户 
 　　即，在我这里就是，


 




bigdata@ubuntucmbigdata1:~$ sudo useradd --system --home=/opt/cm-5.3.3/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm




　  我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。


 


 


 


 


 


 


 　　初始化CM5的数据库数据库(j仅ubuntucmbigdata1节点)






复制代码


bigdata@ubuntucmbigdata1:~$ sudo /opt/cm-5.6.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -prootroot --scm-host localhost scm scm scm
+======================================================================+
|      Error: JAVA_HOME is not set and Java could not be found         |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site  |
|  > http://www.oracle.com/technetwork/java/javase/index.html <        |
|                                                                      |
| Cloudera Manager requires Java 1.6 or later.                         |
| NOTE: This script will find Oracle Java whether you install using    |
|       the binary or the RPM based installer.                         |
+======================================================================+
bigdata@ubuntucmbigdata1:~$ 


复制代码




 


 


 


 


 


Agent 配置


 　　修改/opt/cm-5.6.0/etc/cloudera-scm-agent/config.ini中的server_host为主节点的主机名。 






bigdata@ubuntucmbigdata1:~$ sudo vim /opt/cm-5.6.0/etc/cloudera-scm-agent/config.ini




 


 




 


 


 


 




[General]
# Hostname of the CM server.
server_host=ubuntucmbigdata1

# Port that the CM server is listening on.
server_port=7182




 


 


 


 


 


 


Parcels离线包设置，用以安装（仅主节点）


　　将下载好的


   CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel 
　  CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha 
　　manifest.json


      这三个文件移至/opt/cloudera/parcel-repo/中,没有这个目录的话，手动创建。


 


 


 


 


 






复制代码


bigdata@ubuntucmbigdata1:/opt$ pwd
/opt
bigdata@ubuntucmbigdata1:/opt$ ll
total 12
drwxr-xr-x  3    1106     592 4096  6月 25 18:46 ./
drwxr-xr-x 23 root    root    4096  6月 24 15:09 ../
drwxr-xr-x  8 bigdata bigdata 4096  2月 12  2016 cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt$ sudo mkdir -p /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt$ cd /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ pwd
/opt/cloudera/parcel-repo
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 19:43 ./
drwxr-xr-x 3 root root 4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 


复制代码




 


 


 


 


 


 




复制代码


bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ pwd
/opt/cloudera/parcel-repo
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 19:43 ./
drwxr-xr-x 3 root root 4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo chown -R bigdata:bigdata /opt/cloudera/parcel-repo/
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:43 ./
drwxr-xr-x 3 root    root    4096  6月 25 19:43 ../
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 1497448
drwxr-xr-x 2 bigdata bigdata       4096  6月 25 19:48 ./
drwxr-xr-x 3 root    root          4096  6月 25 19:43 ../
-rw-r--r-- 1 root    root    1533315179  6月 25 18:31 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
-rw-r--r-- 1 root    root            41  6月 25 18:22 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha1
-rw-r--r-- 1 root    root         49913  6月 25 18:32 manifest.json
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo chown -R bigdata:bigdata /opt/cloudera/parcel-repo/*
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ ll
total 1497448
drwxr-xr-x 2 bigdata bigdata       4096  6月 25 19:48 ./
drwxr-xr-x 3 root    root          4096  6月 25 19:43 ../
-rw-r--r-- 1 bigdata bigdata 1533315179  6月 25 18:31 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel
-rw-r--r-- 1 bigdata bigdata         41  6月 25 18:22 CDH-5.6.0-1.cdh5.6.0.p0.45-trusty.parcel.sha1
-rw-r--r-- 1 bigdata bigdata      49913  6月 25 18:32 manifest.json
bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 


复制代码




 


 


 


 


 


相关启动脚本 
通过/opt/cm-5.3.3/etc/init.d/cloudera-scm-server start启动服务端。 
通过/opt/cm-5.3.3/etc/init.d/cloudera-scm-agent start启动Agent服务。 
我们启动的其实是个service脚本，需要停止服务将以上的start参数改为stop就可以了，重启是restart。 


 




复制代码


bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * Couldn't start cloudera-scm-server

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-agent start 
Starting cloudera-scm-agent:  * cloudera-scm-agent started

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * Couldn't start cloudera-scm-server

bigdata@ubuntucmbigdata1:/opt/cloudera/parcel-repo$ 


复制代码




 


 　　查看日志


注：查看安装目录下log文件夹下cloudera-scm-agent/cloudera-scm-agent.out 看Agent 是否启动成功 。 




 




复制代码


bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ pwd
/opt/cm-5.6.0/log
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ l
cloudera-scm-agent/  cloudera-scm-server/  cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ ll
total 20
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ./
drwxr-xr-x 8 bigdata bigdata 4096  2月 12  2016 ../
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:53 cloudera-scm-agent/
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 cloudera-scm-server/
drwxr-xr-x 2 bigdata bigdata 4096  2月 12  2016 cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ cd cloudera-scm-server/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-server$ ll
total 12
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 ./
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ../
-rw-r--r-- 1 root    root     730  6月 25 19:53 cloudera-scm-server.out
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-server$ cat cloudera-scm-server.out 
+======================================================================+
|      Error: JAVA_HOME is not set and Java could not be found         |
+----------------------------------------------------------------------+
| Please download the latest Oracle JDK from the Oracle Java web site  |
|  > http://www.oracle.com/technetwork/java/javase/index.html <        |
|                                                                      |
| Cloudera Manager requires Java 1.6 or later.                         |
| NOTE: This script will find Oracle Java whether you install using    |
|       the binary or the RPM based installer.                         |
+======================================================================+


复制代码




 　　查看日志，可以看出，JAVA_HOME，没有设置。（是这种对ubuntu系统不适用，但是对centos系统是可以的。无解）


 


 


 


 


 


然后，发现，得用


 　　原来，在


在线和离线安装Cloudera CDH 5.6.0


　　里




 


 






bigdata@ubuntucmbigdata1:~$ sudo add-apt-repository ppa:webupd8team/java








复制代码


bigdata@ubuntucmbigdata1:~$ sudo add-apt-repository ppa:webupd8team/java
 Oracle Java (JDK) Installer (automatically downloads and installs Oracle JDK7 / JDK8 / JDK9). There are no actual Java files in this PPA.

Important -> Why Oracle Java 7 And 6 Installers No Longer Work: http://www.webupd8.org/2017/06/why-oracle-java-7-and-6-installers-no.html

More info (and Ubuntu installation instructions):
- for Oracle Java 7: http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html
- for Oracle Java 8: http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html

Debian installation instructions:
- Oracle Java 7: http://www.webupd8.org/2012/06/how-to-install-oracle-java-7-in-debian.html
- Oracle Java 8: http://www.webupd8.org/2014/03/how-to-install-oracle-java-8-in-debian.html

Oracle Java 9 (for both Ubuntu and Debian): http://www.webupd8.org/2015/02/install-oracle-java-9-in-ubuntu-linux.html

For JDK9, the PPA uses standard builds from: http://jdk.java.net/9/ (and not the Jigsaw builds!).

Important!!! For now, you should continue to use Java 8 because Oracle Java 9 is available as an early access release! You should only use Oracle Java 9 if you explicitly need it, because it may contain bugs and it might not include the latest security patches! Also, some Java options were removed in JDK9, so you may encounter issues with various Java apps. More information and installation instructions (Ubuntu / Linux Mint / Debian): http://www.webupd8.org/2015/02/install-oracle-java-9-in-ubuntu-linux.html
 More info: https://launchpad.net/~webupd8team/+archive/ubuntu/java
Press [ENTER] to continue or ctrl-c to cancel adding it （回车）

gpg: keyring `/tmp/tmpel2bk0b8/secring.gpg' created
gpg: keyring `/tmp/tmpel2bk0b8/pubring.gpg' created
gpg: requesting key EEA14886 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpel2bk0b8/trustdb.gpg: trustdb created
gpg: key EEA14886: public key "Launchpad VLC" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
OK
bigdata@ubuntucmbigdata1:~$ 


复制代码




 　　 我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。


 


 


 


 











bigdata@ubuntucmbigdata1:~$ sudo apt-get update




 　　 我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。


 


 


 


 




 


 


 


 


 




bigdata@ubuntucmbigdata1:~$ sudo apt-get upgrade
...
Do you want to continue? [Y/n] y
...
*** sudoers (Y/I/N/O/D/Z) [default=N] ? y





 


 


 


 


 


 


 




bigdata@ubuntucmbigdata1:~$ sudo apt-get install oracle-java7-installer
[sudo] password for bigdata: 
Do you want to continue? [Y/n] y




 






 


 


 


 


 


 


 


 


 




bigdata@ubuntucmbigdata1:~$ sudo apt-get install oracle-java7-set-default




 


 


 


 


 


　　最后，我们需要如下，就可以了






bigdata@ubuntucmbigdata1:/etc/cloudera-scm-server$ dpkg --list | grep -i jdk
iF  oracle-java7-installer                                7u80+7u60arm-0~webupd8~1                            all          Oracle Java(TM) Development Kit (JDK) 7
iU  oracle-java7-set-default                              7u80+7u60arm-0~webupd8~1                            all          Set Oracle JDK 7 as default Java
bigdata@ubuntucmbigdata1:/etc/cloudera-scm-server$ 




 


 　　如果有多余的，如1.6版本的话，则


Ubuntu系统里如何正确卸载oracle-j2sdk1.6（图文详解）


 


 


 


 


 


 




 


 


 


 


 






bigdata@ubuntucmbigdata1:~$ sudo /opt/cm-5.6.0/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:  * cloudera-scm-server started

bigdata@ubuntucmbigdata1:~$ 




 　　成功！




 


 


 


 


 


 


 


 　　查看cloudera-scm-agent的运行日志






复制代码


bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ ll
total 20
drwxr-xr-x 5 bigdata bigdata 4096  2月 12  2016 ./
drwxr-xr-x 8 bigdata bigdata 4096  2月 12  2016 ../
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:53 cloudera-scm-agent/
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 19:52 cloudera-scm-server/
drwxr-xr-x 2 bigdata bigdata 4096  2月 12  2016 cmf/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log$ cd cloudera-scm-agent/
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ ll
total 40
drwxr-xr-x 2 bigdata bigdata  4096  6月 25 19:53 ./
drwxr-xr-x 5 bigdata bigdata  4096  2月 12  2016 ../
-rw-r--r-- 1 root    root    14757  6月 25 19:57 cloudera-scm-agent.log
-rw-r--r-- 1 root    root      935  6月 25 19:53 cloudera-scm-agent.out
-rw-r--r-- 1 root    root      202  6月 25 19:53 cmf_listener.log
-rw-r--r-- 1 root    root      539  6月 25 19:53 supervisord.log
-rw-r--r-- 1 root    root      449  6月 25 19:53 supervisord.out
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ cat cloudera-scm-agent.out 
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     SCM Agent Version: 5.6.0
[25/Jun/2017 19:53:01 +0000] 9651 MainThread util         INFO     Generating Agent UUID for the first time: '/opt/cm-5.6.0/lib/cloudera-scm-agent/uuid'
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     Adding env vars that start with CMF_AGENT_
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     explicitly passing env var to supervisee: MGMT_HOME: /opt/cm-5.6.0/share/cmf
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     explicitly passing env var to supervisee: ROOT: /opt/cm-5.6.0/lib/cmf
[25/Jun/2017 19:53:01 +0000] 9651 MainThread agent        INFO     Logging to /opt/cm-5.6.0/log/cloudera-scm-agent/cloudera-scm-agent.log
Error: could not find config file /opt/cm-5.6.0/run/cloudera-scm-agent/supervisor/supervisord.conf
For help, use /opt/cm-5.6.0/lib/cmf/agent/src/cmf/agent.py -h
bigdata@ubuntucmbigdata1:/opt/cm-5.6.0/log/cloudera-scm-agent$ 


复制代码




 


 


 


 



　　当cloudera-scm-server、cloudera-scm-agent start启动成功之后，可通过http://主节点IP:7180来访问CM，用户名密码都为admin。剩下的就是一直点下一步和根据具体的需求来配置集群了。


 


 


 


 


 　　然后，到这一步，我的


cloudera manager的7180 web界面访问不了的解决办法（图文详解）


 


　　转换成，在线安装


　　如下


　　放在


/opt/softwares/cm-5.6.0






复制代码


bigdata@ubuntucmbigdata1:~$ sudo mkdir -p /opt/softwares/cm-5.6.0
[sudo] password for bigdata: 
bigdata@ubuntucmbigdata1:~$ cd /opt/softwares/cm-5.6.0/
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ pwd
/opt/softwares/cm-5.6.0
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 root root 4096  6月 25 22:19 ./
drwxr-xr-x 3 root root 4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata /opt/softwares/*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 8
drwxr-xr-x 2 bigdata bigdata 4096  6月 25 22:19 ./
drwxr-xr-x 3 root    root    4096  6月 25 22:19 ../
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 root    root    517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chown -R bigdata:bigdata cloudera-manager-installer.bin 
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rw-r--r-- 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ sudo chmod +x ./cloudera-manager-installer.bin
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ ll
total 516
drwxr-xr-x 2 bigdata bigdata   4096  6月 25 22:21 ./
drwxr-xr-x 3 root    root      4096  6月 25 22:19 ../
-rwxr-xr-x 1 bigdata bigdata 517878  6月 25 22:17 cloudera-manager-installer.bin*
bigdata@ubuntucmbigdata1:/opt/softwares/cm-5.6.0$ 


复制代码




 







 




复制代码


bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ pwd
/etc/apt/sources.list.d
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 12
drwxr-xr-x 2 root root 4096  6月 25 20:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl "http://archive.cloudera.com/cm5/ubuntu/precise/amd64/cm/cloudera.list" -o /etc/apt/sources.list.d/cloudera_precise.list
[sudo] password for bigdata: 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   263  100   263    0     0    422      0 --:--:-- --:--:-- --:--:--   422
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ ll
total 16
drwxr-xr-x 2 root root 4096  6月 25 22:26 ./
drwxr-xr-x 6 root root 4096  6月 25 20:26 ../
-rw-r--r-- 1 root root  263  6月 25 22:26 cloudera_precise.list
-rw-r--r-- 1 root root  136  6月 25 20:26 webupd8team-java-trusty.list
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo curl -s http://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -
OK
bigdata@ubuntucmbigdata1:/etc/apt/sources.list.d$ sudo apt-get update




 


 


 　　然后，到这一步，我的


cloudera manager的7180 web界面访问不了的解决办法（图文详解）


 


 


 


 


 


 


 参考


Cloudera Manager5及CDH5安装指导（终极在线安装）


 


 


离线安装CM5和CDH5.13完全教程


 https://blog.csdn.net/u013019431/article/details/78540683


 


 


 


 


 


 


 


 


 


 


欢迎大家，加入我的微信公众号：大数据躺过的坑     免费给分享


 


 


 




同时，大家可以关注我的个人博客：


   http://www.cnblogs.com/zlslch/   和  http://www.cnblogs.com/lchzls/ 


 


　　人生苦短，我愿分享。本公众号将秉持活到老学到老学习无休止的交流分享开源精神，汇聚于互联网和个人学习工作的精华干货知识，一切来于互联网，反馈回互联网。
　　目前研究领域：大数据、机器学习、深度学习、人工智能、数据挖掘、数据分析。 语言涉及：Java、Scala、Python、Shell、Linux等 。同时还涉及平常所使用的手机、电脑和互联网上的使用技巧、问题和实用软件。 只要你一直关注和呆在群里，每天必须有收获


 


       以及对应本平台的QQ群：161156071（大数据躺过的坑）


 


 


 


 


 


 


 


 







 


 




 







            

            
posted @ 
2017-06-25 22:30 
大数据和AI躺过的坑 
阅读(3854) 
评论(0) 
编辑 
收藏 
举报