Spark排错与优化
摘要:一. 运维1. Master挂掉,standby重启也失效Master默认使用512M内存,当集群中运行的任务特别多时,就会挂掉,原因是master会读取每个task的event log日志去生成spark ui,内存不足自然会OOM,可以在master的运行日志中看到,通过HA启...
阅读全文
posted @ 2015-10-15 17:08
posted @ 2015-10-15 17:08
posted @ 2015-10-13 10:58
posted @ 2015-10-09 21:57
posted @ 2015-10-08 17:55
posted @ 2015-10-03 18:05
posted @ 2015-10-02 07:19
posted @ 2015-10-01 06:40