异常:System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-mem
报错信息:
System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-mem
spark连接hadoop集群
解决方法:
这主要是运行内存的问题,所以我们在运行的时候更改JVM的资源分配:
或者:
val sparkConf = new SparkConf() .set("spark.testing.memory","2147480000")