yarn集群客户端gateway常用限制

spark默认集群模式，省略上传依赖包过程：spark-default.sh

spark.yarn.jars hdfs:///${PATH}/sparkJar/jars/*.jar
spark.submit.deployMode cluster

hive限定本地JVM内存大小：hive-default.xml

  <property>
    <name>hive.mapred.local.mem</name>
    <!--<value>10230 M K B</value>-->
    <!--<value>default Byte</value>-->
    <value>10230000000</value>
    <description>mapper/reducer memory in local mode</description>
  </property>

 

posted @ 2018-03-07 14:41  澄轶  阅读(468)  评论(0)  编辑  收藏  举报