Hive优化~参数优化
转载:https://blog.csdn.net/weixin_46163590/article/details/106191042?utm_medium=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-3.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-3.control
转载:https://www.cnblogs.com/sandbank/p/6408762.html
转载:https://meihuakaile.github.io/2018/10/19/hive-set%E8%AE%BE%E7%BD%AE/
map阶段慢,原因是split size 设置太大,task任务比较少,导致从HDFS读数据比较慢,把split size 设置小点,通过增大task数量,优化任务
set tez.grouping.max-size=1024000;
set tez.grouping.min-size=1024000;
tez引擎运行,查看map,reduce 运行情况 可查看map ,reduce的运行时长
![](https://img2020.cnblogs.com/blog/820962/202105/820962-20210513113946708-297011686.png)