Hive任务执行报错:FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

报错内容如下 

com.cloudera.hiveserver2.support.exceptions.GeneralException: [Cloudera][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 2, SQL state: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask, Query: select count(*) from (select msisdn, row_number() over (partition by msisdn) rn from (select msisdn from (select msisdn_a msisdn from db_der.zb_m_cbesim010 ... 
 

这个错误是 map 内存不足导致的,在执行一些hivesql 的时候建议还是设置一些内存参数

set mapreduce.map.memory.mb=4096; --设置map内存
set yarn.scheduler.minimum-allocation-mb=4096; --设置yarn容器最小内存
set mapreduce.map.java.opts=-Xmx3600m; --如果有gc异常可以设置该参数

 

posted @ 2022-10-12 10:26  晓枫的春天  阅读(983)  评论(0编辑  收藏  举报