【异常】Spark提交到yarn报错:java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]

1 详细异常

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]
        at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
        at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
        at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:220)
        at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:468)
        at org.apache.spark.deploy.yarn.ApplicationMaster.org$apache$spark$deploy$yarn$ApplicationMaster$runImpl(ApplicationMaster.scala:305)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply$mcV$sp(ApplicationMaster.scala:245)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:245)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:245)
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:773)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
        at org.apache.spark.deploy.yarn.ApplicationMaster.doAsUser(ApplicationMaster.scala:772)
        at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:244)
        at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:797)
        at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)

  

2 错误原因,配置文件错误

提交到yarn 但是配置文件  应该改成yarn 
spark.master=local
修改为 spark.master=yarn 解决问题
 
posted @   Questions张  阅读(2403)  评论(0编辑  收藏  举报
编辑推荐:
· 理解Rust引用及其生命周期标识(上)
· 浏览器原生「磁吸」效果!Anchor Positioning 锚点定位神器解析
· 没有源码,如何修改代码逻辑?
· 一个奇形怪状的面试题:Bean中的CHM要不要加volatile?
· [.NET]调用本地 Deepseek 模型
阅读排行:
· Blazor Hybrid适配到HarmonyOS系统
· Obsidian + DeepSeek:免费 AI 助力你的知识管理,让你的笔记飞起来!
· 解决跨域问题的这6种方案,真香!
· 分享4款.NET开源、免费、实用的商城系统
· 一套基于 Material Design 规范实现的 Blazor 和 Razor 通用组件库
历史上的今天:
2019-06-01 【异常】~/.bash_profile:source:44: no such file or directory: /usr/local/Cellar/nvm/0.34.0/nvm.sh
2019-06-01 【异常】org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=36, exceptions:
2019-06-01 【异常】org.apache.hadoop.hdfs.server.common.InconsistentFSStateException
2019-06-01 【异常】org.apache.phoenix.exception.PhoenixIOException: SYSTEM:CATALOG
点击右上角即可分享
微信分享提示