hive实现update和delete功能

如果一个表要实现update和delete功能,该表就必须支持ACID,而支持ACID,就必须满足以下条件:
1、表的存储格式必须是ORC(STORED AS ORC);
2、表必须进行分桶(CLUSTERED BY (col_name, col_name, ...)  INTO num_buckets BUCKETS);
3、Table property中参数transactional必须设定为True(tblproperties('transactional'='true'));
4、以下配置项必须被设定:
Hive->配置->类别->高级
Client端:hive-site.xml 的 Hive 客户端高级配置代码段(安全阀)
hive.support.concurrency – true
hive.enforce.bucketing – true
hive.exec.dynamic.partition.mode – nonstrict  
hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager  
服务端:hive-site.xml 的 Hive 服务高级配置代码段(安全阀)
hive.compactor.initiator.on – true
hive.compactor.worker.threads – 1
hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager

hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;

创建表
create table t1(id int, name string)
clustered by (id) into 8 buckets
stored as orc TBLPROPERTIES ('transactional'='true');

create table test_trancaction
(user_id Int,name String)
clustered by (user_id) into 3 buckets
stored as orc TBLPROPERTIES ('transactional'='true');

hive> create table test_insert_test(id int,name string) row format delimited fields TERMINATED BY ','; ---临时表
hive> LOAD DATA LOCAL INPATH '/data/test.txt' OVERWRITE INTO TABLE test_insert_test;
hive> select * from test_insert_test;
OK
1,jerrick
2,tom
3,jerry
4,lily
5,hanmei
6,limlei
7,lucky

hive>insert into test_trancaction select * from test_insert_test;
FAILED: SemanticException [Error 10265]: This command is not allowed on an ACID table merge_data.transactions with a non-ACID transaction manager. Failed

hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
hive>update test_trancaction set name='ccx' where user_id=2;

posted @ 2020-03-05 21:54  舟山渔夫  阅读(5508)  评论(0编辑  收藏  举报