《OD大数据实战》Oozie环境搭建

一、Oozie环境搭建

1. 下载oozie-4.0.0-cdh5.3.6.tar.gz

下载地址：http://archive.cloudera.com/cdh5/cdh/5/

2. 解压

tar -zxvf oozie-4.0.0-cdh5.3.6.tar.gz -C /opt/modules/cdh/

3. 修改配置

1）cd /opt/modules/cdh/oozie-4.0.0-cdh5.3.6

mkdir libext

tar -zxvf oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz -C /tmp/

cd /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/libext

cp -R /tmp/oozie-4.0.0-cdh5.3.6/hadooplibs/hadooplib-2.5.0-cdh5.3.6.oozie-4.0.0-cdh5.3.6/* ./

即在libext中添加hadooplib的jar包依赖

2）下载ext-2.2.zip

下载地址 http://download.csdn.net/detail/farawayplace613/708463

上传到linux中，

cp ext-2.2.zip  /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/libext

3）复制mysql jdbc连接驱动包到libext

mysql-connector-java-5.1.27-bin.jar

cp mysql-connector-java-5.1.27-bin.jar /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/libext

4）修改hadoop的配置

cd /opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop

vi core-site.xml

 <!-- OOZIE -->
  <property>
        <name>hadoop.proxyuser.beifeng.hosts</name>
        <value>beifeng-hadoop-02</value>
  </property>
   <property>
        <name>hadoop.proxyuser.beifeng.groups</name>
        <value>*</value>
   </property>

5）创建数据库

在mysql里创建一个数据库，名称为oozie

create database oozie default character set utf8;

6）修改oozie-site.xml

 <property>
        <name>oozie.service.JPAService.jdbc.url</name>
        <value>jdbc:mysql://beifeng-hadoop-02:3306/oozie</value>
        <description>
            JDBC URL.
        </description>
    </property>

    <property>
        <name>oozie.service.JPAService.jdbc.username</name>
        <value>root</value>
        <description>
            DB user name.
        </description>
    </property>

    <property>
        <name>oozie.service.JPAService.jdbc.password</name>
        <value>beifeng</value>
        <description>
            DB user password.

            IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value,
                       if empty Configuration assumes it is NULL.
        </description>
    </property>

7）指定所连接的hadoop集群的配置文件目录

    <property>
        <name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
        <value>*=/opt/modules/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop</value>
        <description>
            Comma separated AUTHORITY=HADOOP_CONF_DIR, where AUTHORITY is the HOST:PORT of
            the Hadoop service (JobTracker, HDFS). The wildcard '*' configuration is
            used when there is no exact match for an authority. The HADOOP_CONF_DIR contains
            the relevant Hadoop *-site.xml files. If the path is relative is looked within
            the Oozie configuration directory; though the path can be absolute (i.e. to point
            to Hadoop client conf/ directories in the local filesystem.
        </description>
    </property>

8）打war包

bin/oozie-setup.sh prepare-war

9）将oozie运行过程中需要的依赖包共享到hdfs上

bin/oozie-setup.sh sharelib create -fs hdfs://beifeng-hadoop-02:9000 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz

10）初始化mysql oozie库

bin/oozie-setup.sh db create-run oozie.sql

11）启动检测运行

bin/oozied.sh start 

ps -ef | grep oozie

netstat -tlnup | grep 11000

12）访问web ui

http://beifeng-hadoop-02:11000/oozie/