大叔经验分享(13)spark运行报错WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
本地运行spark报错
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 WARN Utils: Service 'sparkDriver' could not bind on port 0. Attempting port 1.
18/12/18 12:56:55 ERROR SparkContext: Error initializing SparkContext.
java.net.BindException: Cannot assign requested address: bind: Service 'sparkDriver' failed after 16 retries (starting from 0)! Consider explicitly setting the appropriate port for the service 'sparkDriver' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:127)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:501)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1218)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:965)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:210)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:353)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:408)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:455)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:745)
18/12/18 12:56:55 INFO SparkContext: Successfully stopped SparkContext
看起来一直在尝试绑定端口0失败,默认起始端口应该是4040,设置spark.driver.port=4040还是报相同的错,后来发现增加环境变量即可
SPARK_LOCAL_IP=127.0.0.1
---------------------------------------------------------------- 结束啦,我是大魔王先生的分割线 :) ----------------------------------------------------------------
- 由于大魔王先生能力有限,文中可能存在错误,欢迎指正、补充!
- 感谢您的阅读,如果文章对您有用,那么请为大魔王先生轻轻点个赞,ありがとう