Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04）（一）

其实，基本思路跟如下差不多，我就不多详细说了，贴出主要图。

　　博主，我是直接借鉴下面这位博主，来进行安装的！（灰常感谢他们！）

Ubuntu14.04用apt在线/离线安装CDH5.1.2[Apache Hadoop 2.3.0

https://www.zybuluo.com/ncepuwanghui/note/474966

ClouderManager官网

https://www.cloudera.com/documentation/manager/5-1-x/Cloudera-Manager-Installation-Guide/cm5ig_mysql.html#id_trn_pps_xm_unique_1

说在前面的话

　　我的机器是总共4台，分别为ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

　　在ubuntucmbigdata1上执行了

sudo apt-get install cloudera-manager-daemons cloudera-manager-server 


sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent

　　即，ubuntucmbigdata1是既做server，又做agent。

　　在ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4上执行了

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent

　　即，ubuntucmbigdata2、3、和4是只做agent。

　　在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list，复制以下内容到cloudera.list里

# Packages for Cloudera Manager, Version 5, on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

　　并且在在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

　　向apt添加cloudera源公钥，在所有节点执行如下命令：

curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

然后，再

sudo apt-get update

　　具体在后面我会详细教大家怎么安装。

　　的确，我也跟这位博主一样，也需要在实验室里部署。以及也是4台机器。（至于原因，大家都懂得，中国高校研究生喜欢4、3、2、1来进行加速比做实验论文）

在实验室安装过社区版的Hadoop，当时用的是普通PC机，安装的过程遇到了好多问题。当解决了大部分的安装问题之后由于实验室的需求，需要安装一个商业发行版的hadoop ，我们就采用的Cloudera 公司发行的版本。首先要说明的是CDH对内存要求比较高，当我们采用原来的4G内存的PC机，只要启动基本内存就满了，并且会拖慢整个集群的运算速度。所以说安装CDH对硬件的要求比较高。安装CDH 有三种安装方式：在线安装、半离线安装、离线安装。在线安装和离线安装我都安装过，虽然在线安装貌似简单但是实际用起来还是没有离线安装方便，我还是建议大家采用离线安装的方式。不过下面都会介绍在线安装的方式和离线安装的方式。希望对别人有所帮助

　　其它的，ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

　参考第一步：

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

Ubuntu14.04安装之后的一些配置

参考

ubuntu 下 visudo的使用

　　在文件末尾（按快捷键，Shift + g）添加bigdata ALL=(ALL) NOPASSWD:ALL

ctrl+o 保存

ctrl+x 退出。

　　最后，得到如下

　　我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

　　防火墙

　　参考

hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz的集群搭建（单节点）（Ubuntu系统）

　　由此，表明Ubuntu14.04是默认没开启防火墙的。

　我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

Ubuntu14.04安装之后的一些配置

　　打开sshd_config文件，在 /etc/ssh 目录下。

# Authentication:
LoginGraceTime 120
#PermitRootLogin without-password
PermitRootLogin yes
StrictModes yes

　　注释掉 #PermitRootLogin without-password 行。

　　添加 PermitRootLogin yes 行

　　修改后，如下：

　我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

修改hosts文件

　　vi /etc/hosts

在里面添加主机名和IP之间的对应关系

#127.0.1.1      ubuntucmbigdata1

192.168.86.31  ubuntucmbigdata1
192.168.86.32  ubuntucmbigdata2
192.168.86.33  ubuntucmbigdata3
192.168.86.34  ubuntucmbigdata4

　我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

　　主机名

bigdata@ubuntucmbigdata1:~$ sudo vim /etc/hostname 
bigdata@ubuntucmbigdata2:~$ sudo vim /etc/hostname 
bigdata@ubuntucmbigdata3:~$ sudo vim /etc/hostname 
bigdata@ubuntucmbigdata4:~$ sudo vim /etc/hostname

　　这里肯定是对的。

配置ssh免密码登录（对于这一步，其实可以不做，因为，在CM时会自动安装，大家可以跳过）

　　见

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

bigdata@ubuntucmbigdata1:~$ mkdir .ssh
bigdata@ubuntucmbigdata1:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/bigdata/.ssh/id_rsa):（回车） 
Enter passphrase (empty for no passphrase):（回车） 
Enter same passphrase again: （回车）
Your identification has been saved in /home/bigdata/.ssh/id_rsa.
Your public key has been saved in /home/bigdata/.ssh/id_rsa.pub.
The key fingerprint is:
58:c2:3f:25:cb:82:5b:d5:b4:b2:ea:54:ac:e2:ea:c8 bigdata@ubuntucmbigdata1
The key's randomart image is:
+--[ RSA 2048]----+
|          .      |
|     .   o .     |
|      o = +      |
|     . O *       |
|    . + S        |
|     o = .       |
|    o +          |
|.. . +           |
|.Eo.. .          |
+-----------------+
bigdata@ubuntucmbigdata1:~$ cd .ssh
bigdata@ubuntucmbigdata1:~/.ssh$ cat id_rsa.pub >> authorized_keys
bigdata@ubuntucmbigdata1:~/.ssh$ cd ..
bigdata@ubuntucmbigdata1:~$ chmod 700 .ssh
bigdata@ubuntucmbigdata1:~$ chmod 600 .ssh/*
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
The authenticity of host 'ubuntucmbigdata1 (127.0.1.1)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Sun Jun 25 14:45:01 2017 from 192.168.86.1
bigdata@ubuntucmbigdata1:~$ exit;
logout
Connection to ubuntucmbigdata1 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Sun Jun 25 15:52:15 2017 from localhost
bigdata@ubuntucmbigdata1:~$ exit;

　我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

　　将所有节点（ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4）中的共钥id_ras.pub拷贝到ubuntucmbigdata1中的authorized_keys文件中。

bigdata@ubuntucmbigdata2:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password: 
bigdata@ubuntucmbigdata2:~$

bigdata@ubuntucmbigdata3:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password: 
bigdata@ubuntucmbigdata3:~$

bigdata@ubuntucmbigdata4:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password: 
bigdata@ubuntucmbigdata4:~$

　　得到

bigdata@ubuntucmbigdata1:~$ cd .ssh
bigdata@ubuntucmbigdata1:~/.ssh$ ls
authorized_keys  id_rsa  id_rsa.pub  known_hosts
bigdata@ubuntucmbigdata1:~/.ssh$ cat authorized_keys 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrEPrX5HHQCnwWoavXUoM8N+fINUckzZ7EcDjJqs1EYkrzltdmMrKQv9VeGGj2BPaGoxzmgit+DPFGuPmPgftBOPWQnLVoILqtQd+H5m0wrK67W7Y98cUpZ+0thCbnmUZRuAES1mt2xS/bL2ibw8AvRrd6HUHOhH17Tcd/2gok32ximn48kPzs6li5d2ay8rPqgqD3QQw3NlAhVmK7RVmpsvlHCTnhrQIoNCz8A+adtK5r1jSgH7uKmPAJTqMUP2Zgv02XQTf/kjW3eacNAWR96Khdh0HsktaK6HuC+FFC+8bXqAj4Hih25EX8RTMFMl3nhGT5FPMLyt7C0Z65QhKV bigdata@ubuntucmbigdata1
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDi7mZhUMzB1h7XjWPIZo3w/Tps0Oxr3mgOeKVfoD4pme/lzgTz61FpEmgggc2WB+mNLKSG/WiazTzuSXKUdBXnM5LNeCGV3ZcWBxFrwp2+wAYH6L5wOGqRq+oQzKUHIsvqEc7TBpN3JmaqZOw4zVCwRBBzQ4cg1B2SjSsjDFLBpjlfM+N8UDG+g9OaZDlEu1z5oUyz0KsKbBtvkkSott+sB+EX4zYSDz/cwUPmSf76RJLk89AI8aX4wGS+P4Eahs5/J2wfLOnAWBG2NQoYVPKyN4sXj7OWw+aErriUJXOqLayYV5xSM2yJjT9WuzbNOiR/D7HGZBlC4wXWG+11GdO5 bigdata@ubuntucmbigdata2
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZtE7Z4cXPjv/t0bC4Ozi3HuovI6XiNe+tJ7cQ6WKp/m44a8w/FqgB4HVZLLQusuXwQM/mEzHKiL1EdPsi5om/s7A7xyDakl2OSUOqki871+/TSjFVTUOOz/H/wwHvl2QzBkb8/dMZLphU1/E4odrFhhlhGGpjK5WUdY8cYH6yyli6W3oDPZ55UG+8EnM89T1AyhC1E81xmyMzGMCj4603oZFwckOeff5n2dqeXg0CRhXO9HhsF9kSZQnE2GmR5pLZlORMkitsCKIhaf5UOBzrGENeJylHGCV1F+KlYjQ0ND4ufTUq14aAnUwBOCqCMhB1tBOBGI4O3S0+j27xumwN bigdata@ubuntucmbigdata3
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXPtqzCU+AmRi+l5lysCEG8gbFKPItYjczvSZIe6+arOfgE/3Ozu2cwr9MABTjgd80hk7ZG4YNRxBPxKvXKtcZxGAe6poe1G5UOrlKOMLeqv1mnUiwtRVsiWp0qnPFcmMt17mpCyN7XnCOLTsdTcl+BVl0c2S2i1xoC6vRKvqV1C/nneJUmuD3KRR1zmR3+a9Wj4yhRRahO6ro/LhgqdwrEaR+NOuT+/1d058fXANToaqyUMa72pz5M2kdxZprumzmHAJ5KlOYImlGrahdLk1wPFckA2m1BMKKycxZHFm1X1O9HeSFrD9N+pdtZY1CXdc6N9hmWKzjBDqw71rZgdUd bigdata@ubuntucmbigdata4
bigdata@ubuntucmbigdata1:~/.ssh$

　　说明成功！

　　然后将中ubuntucmbigdata1的authorized_keys文件分发到所有节点上面。

bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata2:~/.ssh/
The authenticity of host 'ubuntucmbigdata2 (192.168.86.32)' can't be established.
ECDSA key fingerprint is 17:a2:fb:1e:47:dc:79:d5:2d:5a:aa:79:9b:10:c2:eb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata2,192.168.86.32' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata2's password: 
authorized_keys                                                                                                                                                                                                                             100% 1624     1.6KB/s   00:00    
bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata3:~/.ssh/
The authenticity of host 'ubuntucmbigdata3 (192.168.86.33)' can't be established.
ECDSA key fingerprint is 6a:32:65:61:98:78:3f:ad:08:2c:f3:a1:bc:c6:c4:91.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata3,192.168.86.33' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata3's password: 
authorized_keys                                                                                                                                                                                                                             100% 1624     1.6KB/s   00:00    
bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata4:~/.ssh/
The authenticity of host 'ubuntucmbigdata4 (192.168.86.34)' can't be established.
ECDSA key fingerprint is f4:d5:92:17:dc:b1:8f:95:61:11:95:cd:d3:fa:2e:57.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata4,192.168.86.34' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata4's password: 
authorized_keys                                                                                                                                                                                                                             100% 1624     1.6KB/s   00:00    
bigdata@ubuntucmbigdata1:~/.ssh$

　　大家通过ssh 相互访问，如果都能无密码访问，代表ssh配置成功。这里很简单，我就不多赘述了。

bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Sun Jun 25 15:52:23 2017 from localhost
bigdata@ubuntucmbigdata1:~$ exit;Connection to ubuntucmbigdata1 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata2
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Sun Jun 25 15:56:00 2017 from localhost
bigdata@ubuntucmbigdata2:~$ exit;
logout
Connection to ubuntucmbigdata2 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata3
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Sun Jun 25 15:56:14 2017 from localhost
bigdata@ubuntucmbigdata3:~$ exit;
logout
Connection to ubuntucmbigdata3 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata4
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Sun Jun 25 15:56:27 2017 from localhost
bigdata@ubuntucmbigdata4:~$ exit;
logout
Connection to ubuntucmbigdata4 closed.
bigdata@ubuntucmbigdata1:~$

　　这样，就说明是成功！

JDK的安装（建议1.8）（对于这一步，其实可以不做，因为，在CM时会自动安装，大家可以跳过）（但是要先卸载linux系统自带的openjdk，因为自动也是安装的是orcal公司的）

　　见

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

　　这里，因为是ubuntu14.04版本，作为专业人员来说，

Ubuntu14.04下Neo4j图数据库官网安装部署步骤（图文详解）（博主推荐）

bigdata@ubuntucmbigdata1:~$ sudo apt-get remove openjdk*

bigdata@ubuntucmbigdata1:/usr/local$ pwd
/usr/local
bigdata@ubuntucmbigdata1:/usr/local$ ll
total 40
drwxr-xr-x 10 root root 4096  2月 18  2016 ./
drwxr-xr-x 10 root root 4096  2月 18  2016 ../
drwxr-xr-x  2 root root 4096  2月 18  2016 bin/
drwxr-xr-x  2 root root 4096  2月 18  2016 etc/
drwxr-xr-x  2 root root 4096  2月 18  2016 games/
drwxr-xr-x  2 root root 4096  2月 18  2016 include/
drwxr-xr-x  4 root root 4096  2月 18  2016 lib/
lrwxrwxrwx  1 root root    9  6月 24 15:00 man -> share/man/
drwxr-xr-x  2 root root 4096  2月 18  2016 sbin/
drwxr-xr-x  7 root root 4096  2月 18  2016 share/
drwxr-xr-x  2 root root 4096  2月 18  2016 src/
bigdata@ubuntucmbigdata1:/usr/local$ sudo mkdir jdk
bigdata@ubuntucmbigdata1:/usr/local$ ll
total 44
drwxr-xr-x 11 root root 4096  6月 25 16:08 ./
drwxr-xr-x 10 root root 4096  2月 18  2016 ../
drwxr-xr-x  2 root root 4096  2月 18  2016 bin/
drwxr-xr-x  2 root root 4096  2月 18  2016 etc/
drwxr-xr-x  2 root root 4096  2月 18  2016 games/
drwxr-xr-x  2 root root 4096  2月 18  2016 include/
drwxr-xr-x  2 root root 4096  6月 25 16:08 jdk/
drwxr-xr-x  4 root root 4096  2月 18  2016 lib/
lrwxrwxrwx  1 root root    9  6月 24 15:00 man -> share/man/
drwxr-xr-x  2 root root 4096  2月 18  2016 sbin/
drwxr-xr-x  7 root root 4096  2月 18  2016 share/
drwxr-xr-x  2 root root 4096  2月 18  2016 src/
bigdata@ubuntucmbigdata1:/usr/local$ sudo chown -R bigdata:bigdata jdk/
bigdata@ubuntucmbigdata1:/usr/local$ cd jdk/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ pwd
/usr/local/jdk
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 177000
drwxr-xr-x  2 bigdata bigdata      4096  6月 25 16:09 ./
drwxr-xr-x 11 root    root         4096  6月 25 16:08 ../
-rw-r--r--  1 root    root    181238643  9月  3  2016 jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo tar -zxvf jdk-8u60-linux-x64.tar.gz

bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 177004
drwxr-xr-x  3 bigdata bigdata      4096  6月 25 16:09 ./
drwxr-xr-x 11 root    root         4096  6月 25 16:08 ../
drwxr-xr-x  8 uucp        143      4096  8月  5  2015 jdk1.8.0_60/
-rw-r--r--  1 root    root    181238643  9月  3  2016 jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo rm jdk-8u60-linux-x64.tar.gz 
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo chown -R bigdata:bigdata jdk1.8.0_60/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 12
drwxr-xr-x  3 bigdata bigdata 4096  6月 25 16:10 ./
drwxr-xr-x 11 root    root    4096  6月 25 16:08 ../
drwxr-xr-x  8 bigdata bigdata 4096  8月  5  2015 jdk1.8.0_60/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo vim /etc/profile
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo source /etc/profile
sudo: source: command not found
bigdata@ubuntucmbigdata1:/usr/local/jdk$ source /etc/profile
bigdata@ubuntucmbigdata1:/usr/local/jdk$ java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
bigdata@ubuntucmbigdata1:/usr/local/jdk$

　我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

以及，我们还可以设置文件打开数量和用于最大进程数

　　文件打开数量

　　参考

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

bigdata@ubuntucmbigdata1:~$ sudo vi /etc/security/limits.conf

* soft nofile 65535
* hard nofile 65535
* soft nproc  32000
* hard nproc  32000

　我这里，仅拿ubuntucmbigdata1来说，其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

　　ubuntu系统的root用户开启

　　参考

Ubuntu14.04安装之后的一些配置

bigdata@ubuntucmbigdata1:~$ sudo -s
root@ubuntucmbigdata1:~# pwd
/home/bigdata
root@ubuntucmbigdata1:~#

MySQL的安装（以下步骤仅仅在ubuntucmbigdata1这台机器上做）

　　注意：Cloudera-Manages-Server 端需要安装 MySQL，Cloudera-Manages-Agent 端可以不装。

　　即，在我这里就是，Cloudera-Manages-Server 是cmbigdata1。同时，我也是喜欢把oozie、hive等这些，都是安装在cmbigdata1这台机器上。

　　Cloudera-Manages-Agent，在我这里就是，cmbigdata2、cmbigdata3和cmbigdata4。

　　参考

60分钟内从零起步驾驭Hive实战学习笔记（Ubuntu里安装mysql）

Ubuntu 14.04 MySQL 安装配置 For Cloudera Manages 5

root@ubuntucmbigdata1:~# apt-get install mysql-server

。。。
。。。
Setting up libdbd-mysql-perl (4.025-1ubuntu0.1) ...
Setting up libterm-readkey-perl (2.31-1) ...
Setting up mysql-client-5.5 (5.5.55-0ubuntu0.14.04.1) ...
Setting up mysql-server-5.5 (5.5.55-0ubuntu0.14.04.1) ...
170625 16:28:33 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
170625 16:28:33 [Note] Ignoring --secure-file-priv value as server is running with --bootstrap.
170625 16:28:33 [Note] /usr/sbin/mysqld (mysqld 5.5.55-0ubuntu0.14.04.1) starting as process 7930 ...
mysql start/running, process 8062
Setting up libhtml-template-perl (2.95-1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up mysql-server (5.5.55-0ubuntu0.14.04.1) ...
Processing triggers for libc-bin (2.19-0ubuntu6.7) ...
root@ubuntucmbigdata1:~#

安装成功!

　　启动mysql服务

安装MySql成功后，mysql服务会自动启动

root@ubuntucmbigdata1:~# service mysql start
 
或者

root@ubuntucmbigdata1:~# sudo /etc/init.d/mysql start

　　安装mysql JDBC驱动

root@ubuntucmbigdata1:~# apt-get install libmysql-java

设置MySQL的root用户设置密码，这一步，就不需要做了，在CentOS版本里，需要做。在Ubuntu版本，在安装时，就已经设置了。

　　跳过，就是

创建Cloudera-Manages-Server 需要用到的数据库表 for MySql。

　　Creating Databases for Activity Monitor, Reports Manager, Hive Metastore Server, Sentry Server, Cloudera Navigator AuditServer, and Cloudera Navigator Metadata Server

root@ubuntucmbigdata1:~# mysql -root -p
Enter password: （密码为rootroot）
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 42
Server version: 5.5.55-0ubuntu0.14.04.1 (Ubuntu)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

　　执行创建数据库脚本语句。

mysql> create database hive DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hive'@'ubuntucmbigdata1' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';  
Ignoring query to other database
mysql> create database oozie DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on *.* TO 'oozie'@'ubuntucmbigdata1' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> create database amon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'localhost' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'amon'@'ubuntucmbigdata1' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> create database smon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'localhost' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'smon'@'ubuntucmbigdata1' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> create database rman DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'localhost' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on *.* TO 'rman'@'ubuntucmbigdata1' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> create database hmon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'localhost' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hmon'@'ubuntucmbigdata1' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> create database nav DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'localhost' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on *.* TO 'nav'@'ubuntucmbigdata1' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'%' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql>

mysql> flush privileges;
Query OK, 0 rows affected (0.08 sec)

mysql> select user,host,password from mysql.user;
+------------------+------------------+-------------------------------------------+
| user             | host             | password                                  |
+------------------+------------------+-------------------------------------------+
| root             | localhost        | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root             | ubuntucmbigdata1 | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root             | 127.0.0.1        | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root             | ::1              | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| debian-sys-maint | localhost        | *0C134AA3949C6DEE1804B1D1D641A7C2824C599A |
| scm              | %                | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| scm              | ubuntucmbigdata1 | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| scm              | localhost        | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| hive             | localhost        | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| hive             | ubuntucmbigdata1 | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| hive             | %                | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| oozie            | localhost        | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| oozie            | ubuntucmbigdata1 | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| oozie            | %                | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| amon             | localhost        | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| amon             | ubuntucmbigdata1 | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| amon             | %                | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| smon             | localhost        | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| smon             | ubuntucmbigdata1 | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| smon             | %                | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| rman             | localhost        | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| rman             | ubuntucmbigdata1 | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| rman             | %                | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| hmon             | localhost        | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| hmon             | ubuntucmbigdata1 | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| hmon             | %                | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| nav              | localhost        | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
| nav              | ubuntucmbigdata1 | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
| nav              | %                | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
+------------------+------------------+-------------------------------------------+
29 rows in set (0.00 sec)

mysql> exit;

欢迎大家，加入我的微信公众号：大数据躺过的坑免费给分享

同时，大家可以关注我的个人博客：

http://www.cnblogs.com/zlslch/ 和 http://www.cnblogs.com/lchzls/

　　人生苦短，我愿分享。本公众号将秉持活到老学到老学习无休止的交流分享开源精神，汇聚于互联网和个人学习工作的精华干货知识，一切来于互联网，反馈回互联网。
　　目前研究领域：大数据、机器学习、深度学习、人工智能、数据挖掘、数据分析。语言涉及：Java、Scala、Python、Shell、Linux等。同时还涉及平常所使用的手机、电脑和互联网上的使用技巧、问题和实用软件。只要你一直关注和呆在群里，每天必须有收获

以及对应本平台的QQ群：161156071（大数据躺过的坑）

posted @ 2017-06-25 23:12 大数据和AI躺过的坑阅读(3055) 评论(0) 编辑收藏举报

刷新页面返回顶部

大数据和人工智能躺过的坑

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04）（一）

Ubuntu14.04用apt在线/离线安装CDH5.1.2[Apache Hadoop 2.3.0

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

Ubuntu14.04安装之后的一些配置

ubuntu 下 visudo的使用

hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz的集群搭建（单节点）（Ubuntu系统）

相关服务的安装

Ubuntu14.04安装之后的一些配置

修改hosts文件

配置ssh免密码登录（对于这一步，其实可以不做，因为，在CM时会自动安装，大家可以跳过）

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

JDK的安装（建议1.8）（对于这一步，其实可以不做，因为，在CM时会自动安装，大家可以跳过）（但是要先卸载linux系统自带的openjdk，因为自动也是安装的是orcal公司的）

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

Ubuntu14.04下Neo4j图数据库官网安装部署步骤（图文详解）（博主推荐）

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5）（一）

Ubuntu14.04安装之后的一些配置

60分钟内从零起步驾驭Hive实战学习笔记（Ubuntu里安装mysql）

启动mysql服务

安装mysql JDBC驱动

创建Cloudera-Manages-Server 需要用到的数据库表 for MySql。

公告

相关服务的安装

修改hosts文件

配置ssh免密码登录 （对于这一步，其实可以不做，因为，在CM时会自动安装，大家可以跳过）

JDK的安装（建议1.8）（对于这一步，其实可以不做，因为，在CM时会自动安装，大家可以跳过）（但是要先卸载linux系统自带的openjdk，因为自动也是安装的是orcal公司的）

启动mysql服务

安装mysql JDBC驱动

创建Cloudera-Manages-Server 需要用到的数据库表 for MySql。

公告

配置ssh免密码登录（对于这一步，其实可以不做，因为，在CM时会自动安装，大家可以跳过）

　　启动mysql服务

　　安装mysql JDBC驱动