Hadoop 单机搭建 hive单机搭建

使用hive版本：apache-hive-2.3.6-bin.tar.gz 下载地址：http://www.apache.org/dyn/closer.cgi/hive

 在安装前必须要先安装好hadoop环境，可参考：Hadoop 单机搭建 hadoop单机搭建

1、下载解压安装文件

 tar -zxvf apache-hive-2.3.6-bin.tar.gz -C /opt/modules/

2、配置环境变量

修改文件
[hadoop@centos04 modules]$ sudo vi /etc/profile

输入以下内容：

export HIVE_HOME=/opt/modules/apache-hive-2.3.6-bin
export PATH=$PATH:$HIVE_HOME/bin

刷新使之生效
[hadoop@centos04 modules]$ source /etc/profile

显示版本号
[hadoop@centos04 modules]$ hive -version
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/modules/jdk1.8.0_181/bin:/opt/modules/hadoop-2.8.2/bin:/opt/modules/hadoop-2.8.2/sbin:/home/hadoop/.local/bin:/home/hadoop/bin:/opt/modules/jdk1.8.0_181/bin:/opt/modules/hadoop-2.8.2/bin:/opt/modules/hadoop-2.8.2/sbin:/opt/modules/apache-hive-2.3.6-bin/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/modules/hadoop-2.8.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/hive-common-2.3.6.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> 

 

3、关联hodoop

Hive依赖hadoop，需要在Hive中指定hadoop的安装目录

复制Hive安装目录的conf/hive-env.sh.template 文件为hive-env.sh ,然后添加以下内容，指定hadoop安装目录

复制
cp hive-env.sh.template hive-env.sh

修改
vi hive-env.sh

添加
export HADOOP_HOME=/opt/modules/hadoop-2.8.2/

 

4、创建数据仓库

hadoop fs -mkdir -p /tmp
hadoop fs -mkdir -p /user/hive/warehouse
hadoop fs -chmod g+w /tmp
hadoop fs -chmod g+w /user/hive/warehouse


/tmp hive任务在hdfs中的缓存目录
/user/hive/warehouse hive数据仓库，存储hive创建的数据库

hive默认向这2个目录写入数据
如果希望任意用户对这2个目录拥有可写权限，g+w 改为 a+w

 

5、初始化元数据信息

从Hive2.1开始，需要运行schematool命令对Hive数据库的元数据进行初始化。默认Hive使用内嵌的Derby数据库来存储元数据信息

[hadoop@centos04 bin]$ pwd
/opt/modules/apache-hive-2.3.6-bin/bin
[hadoop@centos04 bin]$ schematool -dbType derby -initSchema

[hadoop@centos04 bin]$ schematool -dbType derby -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/modules/hadoop-2.8.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :    org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:       APP
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.derby.sql
Initialization script completed
schemaTool completed

注意：Hive创建一个名为metastore_db的Derby数据库

metastore_db数据库位置默认在初始化命令的执行目录

[hadoop@centos04 bin]$ ll
总用量 44
-rwxr-xr-x. 1 hadoop hadoop  881 8月  13 2019 beeline
-rw-rw-r--. 1 hadoop hadoop  709 3月  17 15:09 derby.log
drwxrwxr-x. 3 hadoop hadoop 4096 3月  17 13:51 ext
-rwxr-xr-x. 1 hadoop hadoop 9838 8月  13 2019 hive
-rwxr-xr-x. 1 hadoop hadoop 1900 8月  13 2019 hive-config.sh
-rwxr-xr-x. 1 hadoop hadoop  885 8月  13 2019 hiveserver2
-rwxr-xr-x. 1 hadoop hadoop  880 8月  13 2019 hplsql
drwxrwxr-x. 5 hadoop hadoop  133 3月  17 15:09 metastore_db
-rwxr-xr-x. 1 hadoop hadoop  832 8月  13 2019 metatool
-rwxr-xr-x. 1 hadoop hadoop  884 8月  13 2019 schematool
[hadoop@centos04 bin]$ pwd
/opt/modules/apache-hive-2.3.6-bin/bin
[hadoop@centos04 bin]$ ll metastore_db/
总用量 28
-rw-rw-r--. 1 hadoop hadoop    4 3月  17 15:09 dbex.lck
-rw-rw-r--. 1 hadoop hadoop   38 3月  17 15:09 db.lck
drwxrwxr-x. 2 hadoop hadoop   97 3月  17 15:09 log
-rw-rw-r--. 1 hadoop hadoop  608 3月  17 15:09 README_DO_NOT_TOUCH_FILES.txt
drwxrwxr-x. 2 hadoop hadoop 8192 3月  17 15:09 seg0
-rw-rw-r--. 1 hadoop hadoop  930 3月  17 15:09 service.properties
drwxrwxr-x. 2 hadoop hadoop    6 3月  17 15:09 tmp
[hadoop@centos04 bin]$ 

 

6、启动Hive Cli

[hadoop@centos04 metastore_db]$ pwd
/opt/modules/apache-hive-2.3.6-bin/bin
[hadoop@centos04 metastore_db]$ hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/modules/jdk1.8.0_181/bin:/opt/modules/hadoop-2.8.2/bin:/opt/modules/hadoop-2.8.2/sbin:/opt/modules/apache-hive-2.3.6-bin/bin:/home/hadoop/.local/bin:/home/hadoop/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/modules/hadoop-2.8.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/hive-common-2.3.6.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> 

注意：上述命令必须在 metastore_db 所在的目录中执行（初始化命令的执行目录）

hive> show databases;
OK
default
Time taken: 11.034 seconds, Fetched: 1 row(s)
hive> 

 

7、验证