|NO.Z.00020|——————————|Deployment|——|Hadoop&PB级离线数仓.v02|——|Atlas.v02|Hive血缘关系导入|

一、Hive血缘关系导入

### --- 配置HIVE_HOME环境变量；

~~~     将atlas服务下atlas-application.properties配置文件拷贝到hive服务目录下

~~~     # 创建软连接：
~~~     将 $ATLAS_HOME/conf/atlasapplication.properties 拷贝到 $HIVE_HOME/conf 目录下
[root@hadoop02 ~]# ln -s $ATLAS_HOME/conf/atlas-application.properties \
$HIVE_HOME/conf/atlas-application.properties

~~~     # 查看创建的软连接
[root@hadoop02 ~]# ll $HIVE_HOME/conf/atlas-application.properties
/opt/yanqi/servers/hive-2.3.7/conf/atlas-application.properties -> /opt/yanqi/servers/atlas-1.2.0/conf/atlas-application.properties

### --- 拷贝jar包
~~~     注：若是在执行的时候发现少很多包
~~~     可以将$ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/目录下的jar包拷贝到$ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/目录下

~~~     # 查看ATLAS_HOME下的三个jackson包
~~~     $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/ 目录下的3个jar，
~~~     拷贝到 $ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/ 目录下

[root@hadoop02 ~]# ls $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/
jackson-jaxrs-base-2.9.9.jar           
jackson-jaxrs-json-provider-2.9.9.jar 
jackson-module-jaxb-annotations-2.9.9.jar

~~~     # 创建软连接

ln -s $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-base-2.9.9.jar \
$ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-base-2.9.9.jar

ln -s $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.9.jar \
$ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-json-provider-2.9.9.jar

ln -s $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.9.jar \
$ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/jackson-module-jaxb-annotations-2.9.9.jar

~~~     # 查看创建的软连接

[root@hadoop02 ~]# cd /opt/yanqi/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl
 
[root@hadoop02 atlas-hive-plugin-impl]# ll jackson-jaxrs-base-2.9.9.jar \
jackson-jaxrs-json-provider-2.9.9.jar \
jackson-module-jaxb-annotations-2.9.9.jar
~~~ 输出参数
jackson-jaxrs-base-2.9.9.jar -> /opt/yanqi/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-base-2.9.9.jar
jackson-jaxrs-json-provider-2.9.9.jar -> /opt/yanqi/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.9.jar
jackson-module-jaxb-annotations-2.9.9.jar -> /opt/yanqi/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.9.jar

二、修改hive配置

### --- 修改Hive的配置：hive-site.xml增加 hook

[root@hadoop02 ~]# vim $HIVE_HOME/conf/hive-site.xml
<!-- Atlas-hive配置 -->
<property>
        <name>hive.exec.post.hooks</name>
        <value>org.apache.atlas.hive.hook.HiveHook</value>
</property>

### --- $HIVE_HOME/conf/hive-env.sh中添加HIVE_AUX_JARS_PATH变量

~~~     # $HIVE_HOME/conf/hive-env.sh中添加HIVE_AUX_JARS_PATH变量
[root@hadoop02 ~]# cd $HIVE_HOME/conf/
[root@hadoop02 conf]# cp hive-env.sh.template hive-env.sh
[root@hadoop01 conf]# vim hive-env.sh
~~~ 最后一行加入如下参数
export HIVE_AUX_JARS_PATH=/opt/yanqi/servers/atlas-1.2.0/hook/hive

三、向Atlas导入hive数据

### --- 启动hive

~~~     # 在Hadoop01启动hdfs和yarn服务
[root@hadoop01 ~]# start-dfs.sh
[root@hadoop01 ~]# start-yarn.sh
~~~     # 在Hadoop01和Hadoop05启动hive.server
[root@hadoop01 ~]# nohup hive --service metastore &
[root@hadoop01 ~]# lsof -i:9083
~~~     # 在Hadoop02进入hive.client端
[root@hadoop02 ~]# hive

### --- 批量导入hive数据

~~~     # hive批量导入数据
~~~     # 备注：Hive能正常启动；在执行的过程中需要用户名/口令：admin/admin
[root@hadoop02 ~]# cd /opt/yanqi/servers/atlas-1.2.0/bin/
[root@hadoop02 bin]# ./import-hive.sh
~~~输入参数
Enter username for atlas :- admin
Enter password for atlas :- 
~~~输入成功参数   
Hive Meta Data imported successfully!!!     # 成功导出可以看见最后的提示信息：

### --- 在浏览器中可以看见：Search 中的选项有变化

### --- Hive hook 可捕获以下操作：

create database
create table/view, create table as select
load, import, export
DMLs (insert)
alter database
alter table
alter view

Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart

——W.S.Landor