Cloudera

 官方文档： http://www.cloudera.com/content/cloudera/en/documentation/core/latest/

 博客教程 http://www.wangyongkui.com/hadoop-cdh5/

http://www.aboutyun.com/thread-9190-1-1.html

http://www.linuxidc.com/Linux/2014-03/98842.htm

CDH（Cloudera Distribution Hadoop）

官方教程 http://www.cloudera.com/content/cloudera/en/downloads/cdh/

在线安装

从网站下的one-click-install目录选择对应的Linux系统版本号，自动添加安装源和进行相关配置，然后根据机器的功能进行下面的安装

Resource Manager host :

　　sudo apt-get update; sudo apt-get install hadoop-yarn-resourcemanager
NameNode host running:

　　sudo apt-get install hadoop-hdfs-namenode
Secondary NameNode host (if used) running:

　　sudo apt-get install hadoop-hdfs-secondarynamenode
All cluster hosts except the Resource Manager running:

　　sudo apt-get install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce
One host in the cluster running:

　　sudo apt-get install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver
All client hosts running:

　　sudo apt-get install hadoop-client

详细安装说明参见 http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_cdh5_install.html

 

离线安装

 http://archive.cloudera.com/cdh5/cdh/5/ 独立功能包下载，不推荐，需要时间识别文件版本和安装到对应目录

 http://archive.cloudera.com/cdh5/parcels/ 完整功能包 在Cloudera Manager中使用来部署

 下载好相应系统版本的.parcel文件后，拷贝到本地的parcel源目录下（由Cloudera Manager Server指定，默认是/opt/cloudera/parcel-repo）

 然后打开同目录下的mainfest.json（也要拷贝到其中）文件，从里面搜索到与你下载的包同名的部分，然后根据其hash值创建一个后缀名为.sha的文本文件，名字与下载的parcel同名，然后将sha文件也拷贝到本地parcel源的目录下。例如：

# cat > CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel.sha
df5cc61b2d257aaf625341f709a4f8e09754038a

总之，要确保/opt/cloudera/parcel-repo下面有三个文件：
CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel
mainfest.json
CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel.sha

cloudera-manager安装

离线安装文件目录http://archive.cloudera.com/cm5/cm/5/

下载tar.gz 解压

sudo tar -zxvf cloudera-manager-trusty-cm5.2.1_amd64.tar.gz -C /opt/cloudera

添加用户

sudo useradd --system --home=/opt/cloudera/cm-5.2.1/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

配置服务器ip和端口（单机测试无须此步骤）

修改/etc/cloudera-scm-agent/config.ini 下面的 server_port 和 server_host

配置mysql数据库

从http://www.mysql.com/products/connector/下载JDBC，复制到CM目录下./share/cmf/lib

sudo cp /home/manhua/下载/mysql-connector-java-5.1.34-bin.jar ./share/cmf/lib/

生成数据库

 

sudo apt-get install mysql-server

 

sudo ./share/cmf/schema/scm_prepare_database.sh mysql scm  -h localhost -u root -p root

 

install ssh

create no password sudo user

https://help.ubuntu.com/community/RootSudo

sudo visudo ,add the following to the END of the file

<username> ALL=NOPASSWD: ALL

由于我用的是linux mint，所以在安装时它会拒绝，需要修改lsb

sudo gedit /etc/lsb-release

DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=17.1
DISTRIB_CODENAME=rebecca
DISTRIB_DESCRIPTION="Linux Mint 17.1 Rebecca"

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Linux Mint 17.1 Rebecca"

  

启动

sudo ./etc/init.d/cloudera-scm-server start

其中对于agents，scp复制CM的整个目录到相同位置后，运行

sudo ./etc/init.d/cloudera-scm-agent start

启动 Cloudera Manager 管理控制台

http://localhost:7180

用户名密码均为admin

然后用上面下载的cdh的parcel文件来安装