软件环境:
linux系统: CentOS6.7
Hadoop版本: 2.6.5
zookeeper版本: 3.4.8
##主机配置: ######一共m1, m2, m3这五部机, 每部主机的用户名都为centos ``` 192.168.179.201: m1 192.168.179.202: m2 192.168.179.203: m3
m1: Zookeeper, Namenode, DataNode, ResourceManager, NodeManager, Master, Worker
m2: Zookeeper, Namenode, DataNode, ResourceManager, NodeManager, Worker
m3: Zookeeper, DataNode, NodeManager, Worker
#####资料:
官方资料:
https://cwiki.apache.org/confluence/display/Hive/Parquet
#####注意:
Hive扩展Parquet有以下两种方案:(hive自0.13版本以后已自带paruqet功能, 不必再扩展parquet功能)
---
<br>
####1.方案一:
####(1) 下载Parquet扩展包
通过yum下载parquet资源包 (此方案大多数情况下无法找到parquet, 原因是没有配置YUM源, 如何配置YUM源请自行查找资料)
sudo yum -y install parquet
######(2)将下载好的扩展包拷贝到指定位置
将下载到`/usr/lib/parquet`目录下的`jar`(除`javadoc.jar`和`sources.jar`外)拷贝到$HIVE_HOME/lib目录下
cp /usr/lib/parquet/* $HIVE_HOME/lib
---
<br>
####2.方案二:
####(1)打开```$HIVE_HOME/lib```目录
cd /home/centos/soft/hive/lib
####(2)下载Parquet资源包
执行以下指令, 下载parquet资源包(注意:`parquet1.2.5`版本对应`Hive 0.12`版本,每个版本的Hive对应的Parquet版本不同, 具体Hive所对应的版本请自行下载源码查看)
for f in parquet-avro parquet-cascading parquet-column parquet-common parquet-encoding parquet-generator parquet-hadoop parquet-hive parquet-pig parquet-pig-bundle parquet-scrooge parquet-test-hadoop2 parquet-thrift
do
curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/${f}/1.2.5/${f}-1.2.5.jar
curl -O https://oss.sonatype.org/service/local/repositories/releases/content/com/twitter/parquet-format/1.2.5/parquet-format-1.2.5.jar
done
---
<br>
<br>
<br>