点击查看代码
<-- 跳过了一些错误信息，省略部分重复内容 -->
[root@master ~]# hive
which: no hbase in (/root/perl5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/jdk1.8.0_401/bin:/root/hadoop-3.4.0/bin:/root/hadoop-3.4.0/sbin:/root/hadoop-3.4.0/libexec:/root/apache-hive-3.1.3-bin/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = dfda412b-ed00-4c9c-bbd8-709e210db544
Logging initialized using configuration in jar:file:/root/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 2b24f675-9a60-41b6-9405-ce0ace11367e
hive> set hive.exec.max.dynamic.partitions.pernode=500;
hive> set mapreduce.reduce.memory.mb=3072;
hive> set mapreduce.map.memory.mb=1536;
hive> set mapred.reduce.max.attempts = 10;
hive> set mapred.map.max.attempts = 10;
hive> create database hive_test;
OK
Time taken: 2.405 seconds
hive> use hive_test;
OK
Time taken: 0.151 seconds
hive> CREATE TABLE testa (
> id INT,
> name string,
> area string
> ) PARTITIONED BY (create_time string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
OK
Time taken: 0.789 seconds
hive> exit;
[root@master ~]# cd /tmp
[root@master tmp]# mkdir data
[root@master tmp]# cd data
[root@master data]# vi test.txt
1,fish1,SZ
2,fish2,SH
3,fish3,HZ
4,fish4,QD
5,fish5,SR
[root@master data]# hadoop fs -put /tmp/data/test.txt /tmp/
[root@master fd]# hive
which: no hbase in (/root/perl5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/jdk1.8.0_401/bin:/root/hadoop-3.4.0/bin:/root/hadoop-3.4.0/sbin:/root/hadoop-3.4.0/libexec:/root/apache-hive-3.1.3-bin/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 6131c400-df24-4105-b81e-1dfecdeabd2e
Logging initialized using configuration in jar:file:/root/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 1bb6dc10-7502-489a-90bf-d3b094ecd5a1
hive > use hive_test;
OK
Time taken: 0.165 seconds
hive> load data inpath '/tmp/test.txt' into table testa;
Query ID = root_20240501214150_3911db36-b76a-452d-ab6b-2f6642d2f220
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
Starting Job = job_1714567164972_0002, Tracking URL = http://master:8088/proxy/application_1714567164972_0002/
Kill Command = /root/hadoop-3.4.0/bin/mapred job -kill job_1714567164972_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2024-05-01 21:42:17,033 Stage-1 map = 0%, reduce = 0%
2024-05-01 21:42:50,223 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 7.62 sec
2024-05-01 21:43:04,572 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 12.6 sec
MapReduce Total cumulative CPU time: 12 seconds 600 msec
Ended Job = job_1714567164972_0002
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to directory hdfs://master:9000/user/hive_remote/warehouse/hive_test.db/testa/.hive-staging_hive_2024-05-01_21-41-51_247_7083536465862239585-1/-ext-10000
Loading data to table hive_test.testa partition (create_time=null)
Time taken to load dynamic partitions: 0.486 seconds
Time taken for adding to write entity : 0.004 seconds
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.StatsTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 12.6 sec HDFS Read: 16278 HDFS Write: 490 SUCCESS
Total MapReduce CPU Time Spent: 12 seconds 600 msec
hive> select * from testa;
OK
1 fish1 SZ __HIVE_DEFAULT_PARTITION__
2 fish2 SH __HIVE_DEFAULT_PARTITION__
3 fish3 HZ __HIVE_DEFAULT_PARTITION__
4 fish4 QD __HIVE_DEFAULT_PARTITION__
5 fish5 SR __HIVE_DEFAULT_PARTITION__
NULL NULL NULL __HIVE_DEFAULT_PARTITION__
NULL NULL NULL __HIVE_DEFAULT_PARTITION__
Time taken: 3.4 seconds, Fetched: 14 row(s)
hive>
