CentOS 6.5 下 CDH 5.2.1 集群安装(一)
集群节点数量3 个
192.168.1.170 cdh-master
192.168.1.171 cdh-slave-1
192.168.1.171 cdh-slave-2
一、安装CentOS6.5 (64位)并搭建基本环境,包括:
(1)添加sudo 权限
(2)修改主机名、网关、静态IP地址、DNS
(3)关闭SELINUX、防火墙
可参考文章
(4)修改系统时区、配置ntp服务(首先可选宿主机器配置ntp server,参考文章)
二、克隆为三台机器,分别修改静态IP 地址、MAC、主机名、hosts文件:
主机名、网关修改 /etc/sysconfig/network文件
MAC 修改 /etc/udev/rules.d/70-persistent-net.rules
IP地址是修改 /etc/sysconfig/network-scripts/ifcfg-eth0
host 文件修改 /etc/hosts 需要添加集群内所有的主机名和ip的对应关系,而且主机名的名称要和真实的主机名(/etc/sysconfig/network)完全一致
三、下载clouder-manger的安装包及parcel文件:
cm 文件下载地址
cloudera-manager-daemons-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
cloudera-manager-server-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
cloudera-manager-agent-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
jdk-6u31-linux-amd64.rpm
oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
parcel 下载地址
CDH-5.2.1-1.cdh5.2.1.p0.12-el6.parcel
四、在master 节点安装daemons、server、agent(先装daemons)
yum --nogpgcheck localinstall cloudera-manager-daemons-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
yum --nogpgcheck localinstall cloudera-manager-server-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
yum --nogpgcheck localinstall cloudera-manager-agent-5.2.1-1.cm521.p0.109.el6.x86_64.rpm(注:agent安装需要联网)
五、在slave-1、slave-2节点安装daemons、agent(先装daemons)
yum --nogpgcheck localinstall cloudera-manager-daemons-5.2.1-1.cm521.p0.109.el6.x86_64.rpm
yum --nogpgcheck localinstall cloudera-manager-agent-5.2.1-1.cm521.p0.109.el6.x86_64.rpm(注:agent安装需要联网)
六、在master、slave-1、slave-2 节点安装JDK、oraclejdk
rpm -ivh jdk-6u31-linux-amd64.rpm
七、在master节点安装mysql 数据库,并配置cdh需要的数据库选项
yum install mysql-server mysql mysql-deve (注:需要联网)
chkconfig mysqld on
service mysqld start
mysql –u root
use mysql
update user set password=password('1234') where user='root'
update user set password=password('1234') where host='localhost'
update user set password=password('1234') where host='cdh-mater'
service mysqld restart
mysql -u root -p1234
create database cloudera
八、在master节点配置cloudera manager 数据库并启动cm的server及agent程序
1.拷贝mysql-connector-java-5.1.7-bin.jar 到 /usr/share/java 下并重命名mysql-connector-java.jar
2.运行 /usr/share/cmf/schema/scm_prepare_database.sh -h cdh-master mysql cloudera root 1234
3.启动cm server :service cloudera-scm-server start
4.添加cm server服务 :chkconfig cloudera-scm-server on
5.启动cm agent :chkconfig cloudera-scm-agent on
6.添加cm agent服务 :service cloudera-scm-server start
九、修改所有节点的agent 配置文件
/etc/cloudera-scm-agent/config.ini 将配置文件中的host 改成 cdh-master
十、在slave节点配置cloudera manager agent程序
1.启动cm agent :chkconfig cloudera-scm-agent on
2.添加cm agent服务 :service cloudera-scm-server start
十一、测试agent和server是否通信成功
service cloudera-scm-server status
service cloudera-scm-agent status
netstat –anp | grep 7182
# server 端开启的是7182端口,用于和agent进行通讯
启动失败时可以查看日志
server 日志 /var/log/cloudera-scm-server
agent 日志 /var/log/cloudera-scm-agent
十二、将下载的parcel文件及官方网站上获取对应版本的 sha 文件和 manifest.json文件上传至/opt/cloudera/parcel-repo目录下。
十三、启动浏览器,开始集群web安装配置