Hive环境部署实战篇
Hive环境部署实战篇
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.Hive安装地址
官网地址: http://hive.apache.org/ 文档查看地址: https://cwiki.apache.org/confluence/display/HIVE 下载地址: http://archive.apache.org/dist/hive/ 关于Hive版本适用于哪个Hadoop版本，可参考官方说明: http://hive.apache.org/downloads.html
二.部署Hive环境(使用默认derby的数据库)
1>.下载Hive支持hadoop 2.x的最新版本（可以使用wget工具进行下载，也可以在windows下载，而后直接上传到服务器端即可）
下载地址: http://archive.apache.org/dist/hive/hive-2.3.7/
2>.解压hive压缩包到指定目录并创建符号链接
[root@hadoop105.yinzhengjie.com ~]# tar -zxf apache-hive-2.3.7-bin.tar.gz -C /yinzhengjie/softwares/ [root@hadoop105.yinzhengjie.com ~]# cd /yinzhengjie/softwares/ [root@hadoop105.yinzhengjie.com /yinzhengjie/softwares]# [root@hadoop105.yinzhengjie.com /yinzhengjie/softwares]# ln -sv apache-hive-2.3.7-bin hive ‘hive’ -> ‘apache-hive-2.3.7-bin’ [root@hadoop105.yinzhengjie.com /yinzhengjie/softwares]#
3>.编辑hive-env.sh文件
[root@hadoop105.yinzhengjie.com ~]# cp /yinzhengjie/softwares/hive/conf/hive-env.sh.template /yinzhengjie/softwares/hive/conf/hive-env.sh [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# grep "HIVE_CONF_DIR=" /yinzhengjie/softwares/hive/conf/hive-env.sh # export HIVE_CONF_DIR= [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# sed -ri "s@# (export HIVE_CONF_DIR=)@\1/yinzhengjie/softwares/hive/conf@" /yinzhengjie/softwares/hive/conf/hive-env.sh [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# grep "HIVE_CONF_DIR=" /yinzhengjie/softwares/hive/conf/hive-env.sh export HIVE_CONF_DIR=/yinzhengjie/softwares/hive/conf [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# grep "HADOOP_HOME=" /yinzhengjie/softwares/hive/conf/hive-env.sh # HADOOP_HOME=${bin}/../../hadoop [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# sed -ri 's@# (HADOOP_HOME=)\$\{bin\}/../../hadoop@export \1/yinzhengjie/softwares/hadoop@' /yinzhengjie/softwares/hive/conf/hive-env.sh [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# grep "HADOOP_HOME=" /yinzhengjie/softwares/hive/conf/hive-env.sh export HADOOP_HOME=/yinzhengjie/softwares/hadoop [root@hadoop105.yinzhengjie.com ~]#
4>.配置Hive的环境变量
[root@hadoop105.yinzhengjie.com ~]# vim /etc/profile.d/hive.sh [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# cat /etc/profile.d/hive.sh #Add ${HIVE_HOME} by yinzhengjie HIVE_HOME=/yinzhengjie/softwares/hive PATH=$PATH:${HIVE_HOME}/bin [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# source /etc/profile.d/hive.sh
5>.启动Hadoop集群
关于Hadoop集群的部署我这里就不罗嗦了，能看到这篇文章说明你对Hadoop的环境部署肯定是了解的，若不太了解的小伙伴可以自行参考我之前分享的笔记。
接下来我们要做的事情就是启动Hadoop集群。
[root@hadoop101.yinzhengjie.com ~]# manage-hdfs.sh start hadoop101.yinzhengjie.com | CHANGED | rc=0 >> starting namenode, logging to /yinzhengjie/softwares/hadoop-2.10.0-fully-mode/logs/hadoop-root-namenode-hadoop101.yinzhengjie.com.out hadoop105.yinzhengjie.com | CHANGED | rc=0 >> starting secondarynamenode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-secondarynamenode-hadoop105.yinzhengjie.com.out hadoop103.yinzhengjie.com | CHANGED | rc=0 >> starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop103.yinzhengjie.com.out hadoop102.yinzhengjie.com | CHANGED | rc=0 >> starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop102.yinzhengjie.com.out hadoop104.yinzhengjie.com | CHANGED | rc=0 >> starting datanode, logging to /yinzhengjie/softwares/hadoop/logs/hadoop-root-datanode-hadoop104.yinzhengjie.com.out Starting HDFS: [ OK ] [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# manage-hdfs.sh status hadoop105.yinzhengjie.com | CHANGED | rc=0 >> 7203 SecondaryNameNode 7320 Jps hadoop102.yinzhengjie.com | CHANGED | rc=0 >> 6548 DataNode 6758 Jps hadoop101.yinzhengjie.com | CHANGED | rc=0 >> 10809 NameNode 11097 Jps hadoop103.yinzhengjie.com | CHANGED | rc=0 >> 6546 DataNode 6748 Jps hadoop104.yinzhengjie.com | CHANGED | rc=0 >> 6514 DataNode 6719 Jps [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# manage-yarn.sh start hadoop101.yinzhengjie.com | CHANGED | rc=0 >> starting resourcemanager, logging to /yinzhengjie/softwares/hadoop/logs/yarn-root-resourcemanager-hadoop101.yinzhengjie.com.out Error: Cannot find configuration directory: /yinzhengjie/softwares/hadoop/conf/yarn hadoop102.yinzhengjie.com | CHANGED | rc=0 >> starting nodemanager, logging to /yinzhengjie/softwares/hadoop/logs/yarn-root-nodemanager-hadoop102.yinzhengjie.com.out hadoop104.yinzhengjie.com | CHANGED | rc=0 >> starting nodemanager, logging to /yinzhengjie/softwares/hadoop/logs/yarn-root-nodemanager-hadoop104.yinzhengjie.com.out hadoop103.yinzhengjie.com | CHANGED | rc=0 >> starting nodemanager, logging to /yinzhengjie/softwares/hadoop/logs/yarn-root-nodemanager-hadoop103.yinzhengjie.com.out Starting YARN: [ OK ] [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# manage-yarn.sh status hadoop105.yinzhengjie.com | CHANGED | rc=0 >> 7203 SecondaryNameNode 7423 Jps hadoop101.yinzhengjie.com | CHANGED | rc=0 >> 11456 Jps 10809 NameNode hadoop103.yinzhengjie.com | CHANGED | rc=0 >> 6881 NodeManager 6546 DataNode 7005 Jps hadoop102.yinzhengjie.com | CHANGED | rc=0 >> 6548 DataNode 7000 Jps 6879 NodeManager hadoop104.yinzhengjie.com | CHANGED | rc=0 >> 6514 DataNode 6852 NodeManager 7001 Jps [root@hadoop101.yinzhengjie.com ~]#
6>.安装数据库并授权
如下图所示，执行以下三条命令就可以完成授权啦:(一定要进行权限测试，还有就是要注意Hive支持的MySQL最低版本问题哟~) CREATE DATABASE hive_metastore CHARACTER SET utf8 COLLATE utf8_bin; CREATE USER hive@hadoop105.yinzhengjie.com IDENTIFIED BY 'yinzhengjie'; GRANT ALL PRIVILEGES ON hive_metastore.* TO hive@hadoop105.yinzhengjie.com; 博主推荐阅读:(下面的链接是关于部署MySQL的方法以及Hive支持的MySQL版本) https://www.cnblogs.com/yinzhengjie/p/11733897.html https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+Administration
7>.拷贝MySQL的驱动包到hive的依赖库目录
[root@hadoop105.yinzhengjie.com ~]# ll /yinzhengjie/softwares/hive/lib/ | wc -l 256 [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# ll # 驱动包直接在Oracle官网就可以下载哟，如果你安装的MySQL版本较高，选择驱动时也应该选择对应的版本！(当然这也依赖于Hive是否支持更高版本的数据库哟~) total 852 -rw-r--r-- 1 root root 872303 Mar 1 2020 mysql-connector-java-5.1.27-bin.jar [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# mv mysql-connector-java-5.1.27-bin.jar /yinzhengjie/softwares/hive/lib/ [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# ll /yinzhengjie/softwares/hive/lib/ | wc -l 257 [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]#
8>.配置Hive的MetaStore到MySQL
[root@hadoop105.yinzhengjie.com ~]# vim /yinzhengjie/softwares/hive/conf/hive-site.xml [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# cat /yinzhengjie/softwares/hive/conf/hive-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <!-- 配置数据库链接相关的参数 --> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://hadoop101.yinzhengjie.com:3306/hive_metastore?createDatabaseIfNotExist=true</value> <description> JDBC connect string for a JDBC metastore. To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL. For example, jdbc:postgresql://myhost/db?ssl=true for postgres database. </description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> <description>Username to use against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>yinzhengjie</value> <description>password to use against metastore database</description> </property> <!-- 指定数仓在HDFS的存储路径 --> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> <description>location of default database for the warehouse</description> </property> <property> <name>hive.metastore.schema.verification</name> <value>false</value> <description> Enforce metastore schema version consistency. True: Verify that version information stored in is compatible with one from Hive jars. Also disable automatic schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures proper metastore schema migration. (Default) False: Warn if the version information stored in metastore doesn't match with one from in Hive jars. </description> </property> </configuration> [root@hadoop105.yinzhengjie.com ~]#
9>.初始化元数据库
[root@hadoop105.yinzhengjie.com ~]# schematool -initSchema -dbType mysql SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4 SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://hadoop101.yinzhengjie.com:3306/hive_metastore?c Metastore Connection Driver : com.mysql.jdbc.Driver Metastore connection User: hive Starting metastore schema initialization to 2.3.0 Initialization script hive-schema-2.3.0.mysql.sql Initialization script completed schemaTool completed [root@hadoop105.yinzhengjie.com ~]#
10>.查看数据仓库的默认数据库
11>.创建一张表，观察元数据库及HDFS集群中的变化
Hive终端执行的操作如下所示:
hive> CREATE TABLE teachers (id int, name string, address string);
MySQL终端执行的操作如下所示: MySQL [hive_metastore]> select * from DBS; +-------+-----------------------+-----------------------------------------------------------+---------+------------+------------+ | DB_ID | DESC | DB_LOCATION_URI | NAME | OWNER_NAME | OWNER_TYPE | +-------+-----------------------+-----------------------------------------------------------+---------+------------+------------+ | 1 | Default Hive database | hdfs://hadoop101.yinzhengjie.com:9000/user/hive/warehouse | default | public | ROLE | +-------+-----------------------+-----------------------------------------------------------+---------+------------+------------+ 1 row in set (0.00 sec) MySQL [hive_metastore]> select * from TBLS; Empty set (0.00 sec) MySQL [hive_metastore]> MySQL [hive_metastore]> select * from TBLS; +--------+-------------+-------+------------------+-------+-----------+-------+----------+---------------+--------------------+--------------------+--------------------+ | TBL_ID | CREATE_TIME | DB_ID | LAST_ACCESS_TIME | OWNER | RETENTION | SD_ID | TBL_NAME | TBL_TYPE | VIEW_EXPANDED_TEXT | VIEW_ORIGINAL_TEXT | IS_REWRITE_ENABLED | +--------+-------------+-------+------------------+-------+-----------+-------+----------+---------------+--------------------+--------------------+--------------------+ | 1 | 1605529291 | 1 | 0 | root | 0 | 1 | teachers | MANAGED_TABLE | NULL | NULL | | +--------+-------------+-------+------------------+-------+-----------+-------+----------+---------------+--------------------+--------------------+--------------------+ 1 row in set (0.00 sec) MySQL [hive_metastore]>
三.hive非交互式命令概述
1>. 查看hive命令的帮助信息("-h选项")
[root@hadoop105.yinzhengjie.com ~]# hive -H which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softw ares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] usage: hive -d,--define <key=value> Variable substitution to apply to Hive commands. e.g. -d A=B or --define A=B --database <databasename> Specify the database to use -e <quoted-query-string> SQL from command line -f <filename> SQL from files -H,--help Print help information --hiveconf <property=value> Use value for given property --hivevar <key=value> Variable substitution to apply to Hive commands. e.g. --hivevar A=B -i <filename> Initialization SQL file -S,--silent Silent mode in interactive shell -v,--verbose Verbose mode (echo executed SQL to the console) [root@hadoop105.yinzhengjie.com ~]#
2>.非交互式(在命令行中直接)运行SQL语句("-e"选项)
[root@hadoop105.yinzhengjie.com ~]# hive -e "SHOW DATABASES;SHOW TABLES;" which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softw ares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true OK default Time taken: 4.009 seconds, Fetched: 1 row(s) OK students teachers Time taken: 0.068 seconds, Fetched: 2 row(s) [root@hadoop105.yinzhengjie.com ~]#
3>.执行HQL文件("-f"选项)
[root@hadoop105.yinzhengjie.com ~]# vim test.hql [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# cat test.hql SHOW DATABASES; SHOW TABLES; SELECT * FROM students; [root@hadoop105.yinzhengjie.com ~]#
[root@hadoop105.yinzhengjie.com ~]# hive -f test.hql which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softw ares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true OK default Time taken: 4.585 seconds, Fetched: 1 row(s) OK students teachers Time taken: 0.079 seconds, Fetched: 2 row(s) OK 1 Jason BeiJing 2 YinZhengjie ShanXi 3 Dilraba XinJiang Time taken: 1.449 seconds, Fetched: 3 row(s) [root@hadoop105.yinzhengjie.com ~]#
4>.在命令行中定义变量名称("-d"选项)
[root@hadoop105.yinzhengjie.com ~]# hive -e "set" | grep "my_name=" which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softw ares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# hive -d my_name="jason Yin" -e "set" | grep "my_name=" which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softw ares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true hivevar:my_name=jason Yin system:sun.java.command=org.apache.hadoop.util.RunJar /yinzhengjie/softwares/hive/lib/hive-cli-2.3.7.jar org.apache.hadoop.hive.cli.CliDriver -d my_name=jason Yin -e set [root@hadoop105.yinzhengjie.com ~]#
5>.初始化交互式字符界面("-i"选项)
[root@hadoop105.yinzhengjie.com ~]# cat test.hql SHOW DATABASES; SHOW TABLES; SELECT * FROM students; [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# hive -i test.hql which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softw ares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: true default students teachers 1 Jason BeiJing 2 YinZhengjie ShanXi 3 Dilraba XinJiang Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. hive>
6>.在交互式字符界面中开启静默模式("-S"选项)
[root@hadoop105.yinzhengjie.com ~]# hive -S which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softw ares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] hive> show databases; default hive> show tables; students teachers hive>
7>.在交互式字符界面中开启详细模式("-v"选项)
[root@hadoop105.yinzhengjie.com ~]# hive -v which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares /hadoop/bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf 4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4 j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7 -bin/lib/hive-common-2.3.7.jar!/hive-log4j2.properties Async: trueHive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.hive> SHOW DATABASES; SHOW DATABASES OK default Time taken: 4.716 seconds, Fetched: 1 row(s) hive> SHOW TABLES; SHOW TABLES OK students teachers Time taken: 0.043 seconds, Fetched: 2 row(s) hive> SELECT * FROM students; SELECT * FROM students OK 1 Jason BeiJing 2 YinZhengjie ShanXi 3 Dilraba XinJiang Time taken: 1.439 seconds, Fetched: 3 row(s) hive>
四.hive交互式命令概述
1>.查看HDFS分布式文件系统的命令
[root@hadoop105.yinzhengjie.com ~]# hive which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/ bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl- 2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j1 2-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib /hive-common-2.3.7.jar!/hive-log4j2.properties Async: trueHive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.hive> dfs -ls /; Found 6 items drwxr-xr-x - root admingroup 0 2020-11-09 07:21 /LOGS drwxrwxrwx - root admingroup 0 2020-10-22 19:38 /input drwxr-xr-x - root admingroup 0 2020-11-09 06:23 /output drwx-wx-wx - root admingroup 0 2020-11-16 05:41 /tmp drwxrwxrwx - root admingroup 0 2020-11-16 20:21 /user drwxrwxrwx - root admingroup 0 2020-11-09 03:41 /yinzhengjie hive> dfs -ls /yinzhengjie; Found 3 items drwxrwxrwx - root admingroup 0 2020-11-01 13:48 /yinzhengjie/history_logs drwxrwxrwx - root admingroup 0 2020-11-09 04:03 /yinzhengjie/logs drwxrwxrwx - root admingroup 0 2020-11-01 13:18 /yinzhengjie/workspace hive> queue -status yinzhengjie;
2>.查看Linux系统相关的命令
[root@hadoop105.yinzhengjie.com ~]# hive which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/yinzhengjie/softwares/hadoop/ bin:/yinzhengjie/softwares/hadoop/sbin:/yinzhengjie/softwares/hive/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/yinzhengjie/softwares/jdk1.8.0_201/bin:/root/bin)SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl- 2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j1 2-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib /hive-common-2.3.7.jar!/hive-log4j2.properties Async: trueHive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.hive> !ls -l /yinzhengjie/softwares/; total 0 drwxr-xr-x 11 root root 221 Nov 16 16:06 apache-hive-2.3.7-bin drwxr-xr-x 10 root root 161 Aug 12 21:07 hadoop lrwxrwxrwx 1 root root 21 Nov 16 04:53 hive -> apache-hive-2.3.7-bin drwxr-xr-x 7 10 143 245 Dec 16 2018 jdk1.8.0_201 hive> !yarn queue -status yinzhengjie; SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl- 2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/yinzhengjie/softwares/hadoop/share/hadoop/common/lib/slf4j-log4j1 2-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2020-11-19T19:51:38,493 INFO [main] org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceMa nager at hadoop101.yinzhengjie.com/172.200.6.101:8032Queue Information : Queue Name : root.yinzhengjie State : RUNNING Capacity : .0% Current Capacity : .0% Maximum Capacity : -100.0% Default Node Label expression : <DEFAULT_PARTITION> Accessible Node Labels : hive>
3>.查看hive历史的命令
[root@hadoop105.yinzhengjie.com ~]# ll .hivehistory -rw-r--r-- 1 root root 1451 Nov 19 19:53 .hivehistory [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]# tail .hivehistory yarn queue -status yinzhengjie ; dfs -ls / ; dfs -ls /yinzhengjie; queue -status yinzhengjie; !ls -l /yinzhengjie/softwares/ ; !ls -l /yinzhengjie/softwares/; !yarn queue -status yinzhengjie; [root@hadoop105.yinzhengjie.com ~]# [root@hadoop105.yinzhengjie.com ~]#
五.博主推荐阅读
Hive常用的属性配置及调优参数概述: https://www.cnblogs.com/yinzhengjie2020/p/13939518.html Hive的JDBC环境部署: https://www.cnblogs.com/yinzhengjie2020/p/13961702.html Hive常用的DDL数据操作: https://www.cnblogs.com/yinzhengjie2020/p/13983292.html Hive常用的DML数据操作: https://www.cnblogs.com/yinzhengjie2020/p/13991817.html
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。 欢迎加入基础架构自动化运维：598432640，大数据SRE进阶之路：959042252，DevOps进阶之路：526991186
