org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Webapps failed to start. Ignoring for now:java.lang.NullPointerException报错 || 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster报错

解决“HA机制hadoop集群中job提交之后提示Faling the application，但是不抱任何错，程序甚至没有执行，但是在log中有报错org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Webapps failed to start. Ignoring for now:java.lang.NullPointerException”的问题

转载
最近在学习Hadoop，由于我搭建的环境比较特殊（在5台不同的供应商的云服务器上搭建）以及我学习的教程版本偏旧，学习的过程中遇到异常多坑，将自己遇到的难以解决的坑以及解决方案记录下来，希望对大家有帮助。

1.HA机制下mapreduce程序无法执行
无论是将本地运行正常的程序或是示例程序PI，在集群上跑，都无法执行，还未开始执行就退出了。
在yarn的管理界面上查看mapreduce的日志发现

2018-06-04 16:00:58,564 ERROR [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Webapps failed to start. Ignoring for now:
java.lang.NullPointerException
at org.apache.hadoop.util.StringUtils.join(StringUtils.java:930)
at org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer.initFilter(AmFilterInitializer.java:75)
at org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:466)
at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:412)
at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:115)
at org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:336)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:315)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:401)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:397)
at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:143)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1269)

我根据异常跟踪源码，问题大概就是MRClientService的WebApp创建过程出错，导致WebApp对象为null，后边调用了WebApp的getHttpPort()方法，导致空指针，而具体为什么会创建出错，百度上也找不到答案，后来经过不懈努力后才找到答案：
HA机制下yarn-site.xml需要加入以下配置:

<property> 
    <name>yarn.resourcemanager.webapp.address.rm1</name>  
    <value>xxx1:8088</value> 
</property>  

<property> 
    <name>yarn.resourcemanager.webapp.address.rm2</name>  
    <value>xxx2:8088</value> 
</property>

其中xxx1和xxx2需要替换成你的ResourceManager的主机名，比较坑的的就是即使你不配置，一样可以访问到管理界面，但是运行就会报错，我学的教程可能是旧版本的Hadoop，并没有提到这一点。
增加了这一个配置以后，mapreduce程序就可以正常运行了。
————————————————
版权声明：本文为CSDN博主「_zhqy」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/qq_38337823/article/details/80580422

  • log报错如下：
2022-03-24 19:29:44,075 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1648119700908_0006_000002
2022-03-24 19:29:44,325 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: 
/************************************************************
[system properties]
os.name: Linux
os.version: 3.10.0-1160.el7.x86_64
java.home: /home/hadoop/software/java/jdk1.8.0_201/jre
java.runtime.version: 1.8.0_201-b09
java.vendor: Oracle Corporation
java.version: 1.8.0_201
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
java.class.path: /home/hadoop/software/hadoop/hadoop-3.2.2/tmp/nm-local-dir/usercache/hadoop/appcache/application_1648119700908_0006/container_1648119700908_0006_02_000001:/home/hadoop/software/hadoop/hadoop-3.2.2/etc/hadoop:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-security-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/asm-5.0.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-server-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/nimbus-jose-jwt-7.9.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/dnsjava-2.1.7.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-cli-1.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/httpcore-4.4.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jsch-0.1.55.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jackson-core-2.9.10.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-http-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/re2j-1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-util-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jackson-databind-2.9.10.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/failureaccess-1.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/error_prone_annotations-2.2.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/httpclient-4.5.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-text-1.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/json-smart-2.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-compress-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/avro-1.7.7.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/guava-27.0-jre.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jsp-api-2.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/hadoop-auth-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jersey-server-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/netty-3.10.6.Final.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jersey-core-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-net-3.6.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jersey-json-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-io-2.5.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/paranamer-2.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/javax.activation-api-1.2.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/hadoop-annotations-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jsr305-3.0.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-codec-1.11.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-servlet-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-lang3-3.7.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/checker-qual-2.5.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-xml-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-io-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jackson-annotations-2.9.10.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/gson-2.2.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jetty-webapp-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jettison-1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/log4j-1.2.17.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/hadoop-common-3.2.2-tests.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/hadoop-nfs-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/hadoop-kms-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/common/hadoop-common-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-security-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-server-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-7.9.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jackson-core-2.9.10.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-http-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-util-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jackson-databind-2.9.10.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/error_prone_annotations-2.2.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-text-1.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-compress-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/hadoop-auth-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/javax.activation-api-1.2.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/hadoop-annotations-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/netty-all-4.1.48.Final.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-servlet-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-lang3-3.7.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-xml-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-io-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jackson-annotations-2.9.10.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jetty-webapp-9.4.20.v20190813.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-client-3.2.2-tests.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.2-tests.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.2-tests.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-client-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-3.2.2-tests.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.2.2-tests.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/javax.inject-1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.9.10.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/guice-4.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.9.10.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.9.10.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/fst-2.50.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-submarine-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-services-core-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-common-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-router-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-services-api-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-client-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-api-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-registry-3.2.2.jar:/home/hadoop/software/hadoop/hadoop-3.2.2/share/hadoop/yarn/hadoop-yarn-common-3.2.2.jar:/share/hadoop/mapreduce/*:/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/home/hadoop/software/hadoop/hadoop-3.2.2/tmp/nm-local-dir/usercache/hadoop/appcache/application_1648119700908_0006/container_1648119700908_0006_02_000001/job.jar
java.io.tmpdir: /home/hadoop/software/hadoop/hadoop-3.2.2/tmp/nm-local-dir/usercache/hadoop/appcache/application_1648119700908_0006/container_1648119700908_0006_02_000001/tmp
user.dir: /home/hadoop/software/hadoop/hadoop-3.2.2/tmp/nm-local-dir/usercache/hadoop/appcache/application_1648119700908_0006/container_1648119700908_0006_02_000001
user.name: hadoop
************************************************************/
2022-03-24 19:29:44,480 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration
2022-03-24 19:29:44,759 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens: [Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 6 cluster_timestamp: 1648119700908 } attemptId: 2 } keyId: 660552221)]
2022-03-24 19:29:44,851 INFO [main] org.apache.hadoop.conf.Configuration: resource-types.xml not found
2022-03-24 19:29:44,851 INFO [main] org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'resource-types.xml'.
2022-03-24 19:29:44,899 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.
2022-03-24 19:29:44,901 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2022-03-24 19:29:45,022 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2
2022-03-24 19:29:45,022 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false
2022-03-24 19:29:46,501 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2022-03-24 19:29:46,938 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2022-03-24 19:29:46,939 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2022-03-24 19:29:46,940 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2022-03-24 19:29:46,941 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2022-03-24 19:29:46,942 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2022-03-24 19:29:46,954 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2022-03-24 19:29:46,955 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2022-03-24 19:29:46,956 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2022-03-24 19:29:46,999 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ns1:8020]
2022-03-24 19:29:47,032 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ns1:8020]
2022-03-24 19:29:47,054 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ns1:8020]
2022-03-24 19:29:47,104 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled
2022-03-24 19:29:47,107 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Attempting to recover.
2022-03-24 19:29:47,131 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ns1:8020]
2022-03-24 19:29:47,136 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Previous history file is at hdfs://ns1:8020/tmp/hadoop-yarn/staging/hadoop/.staging/job_1648119700908_0006/job_1648119700908_0006_1.jhist
2022-03-24 19:29:47,915 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Read completed tasks from history 0
2022-03-24 19:29:47,926 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Starting to clean up previous job's temporary files
2022-03-24 19:29:47,937 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Finished cleaning up previous job temporary files
2022-03-24 19:29:48,041 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2022-03-24 19:29:48,229 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2022-03-24 19:29:48,612 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2022-03-24 19:29:48,612 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2022-03-24 19:29:48,641 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1648119700908_0006 to jobTokenSecretManager
2022-03-24 19:29:48,715 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1648119700908_0006 because: not enabled; too much input;
2022-03-24 19:29:48,807 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1648119700908_0006 = 278004645. Number of splits = 2
2022-03-24 19:29:48,808 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1648119700908_0006 = 1
2022-03-24 19:29:48,808 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1648119700908_0006Job Transitioned from NEW to INITED
2022-03-24 19:29:48,810 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1648119700908_0006.
2022-03-24 19:29:48,857 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 100, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2022-03-24 19:29:48,875 INFO [Socket Reader #1 for port 0] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 0
2022-03-24 19:29:49,222 INFO [Listener at 0.0.0.0/44982] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2022-03-24 19:29:49,226 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2022-03-24 19:29:49,228 INFO [IPC Server listener on 0] org.apache.hadoop.ipc.Server: IPC Server listener on 0: starting
2022-03-24 19:29:49,228 INFO [Listener at 0.0.0.0/44982] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at hadoop03/192.168.17.3:44982
2022-03-24 19:29:49,309 INFO [Listener at 0.0.0.0/44982] org.eclipse.jetty.util.log: Logging initialized @7227ms to org.eclipse.jetty.util.log.Slf4jLog
2022-03-24 19:29:49,645 INFO [Listener at 0.0.0.0/44982] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
2022-03-24 19:29:49,649 INFO [Listener at 0.0.0.0/44982] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2022-03-24 19:29:49,654 INFO [Listener at 0.0.0.0/44982] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2022-03-24 19:29:49,739 ERROR [Listener at 0.0.0.0/44982] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Webapps failed to start. Ignoring for now:
java.lang.NullPointerException
	at org.apache.hadoop.util.StringUtils.join(StringUtils.java:967)
	at org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer.initFilter(AmFilterInitializer.java:74)
	at org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:626)
	at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:582)
	at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:117)
	at org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:448)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:349)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:442)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:438)
	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:147)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1290)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
2022-03-24 19:29:49,746 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator: JOB_CREATE job_1648119700908_0006
2022-03-24 19:29:49,755 INFO [Listener at 0.0.0.0/44982] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 3000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false.
2022-03-24 19:29:49,760 INFO [Socket Reader #1 for port 0] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 0
2022-03-24 19:29:49,779 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2022-03-24 19:29:49,781 INFO [IPC Server listener on 0] org.apache.hadoop.ipc.Server: IPC Server listener on 0: starting
2022-03-24 19:29:49,848 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2022-03-24 19:29:49,848 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2022-03-24 19:29:49,848 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2022-03-24 19:29:49,852 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: 0% of the mappers will be scheduled using OPPORTUNISTIC containers
2022-03-24 19:29:49,973 ERROR [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Exception while registering
java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
2022-03-24 19:29:49,974 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.service.AbstractService: Service RMCommunicator failed in state STARTED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
	... 14 more
2022-03-24 19:29:49,975 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2022-03-24 19:29:49,975 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter failed in state STARTED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
	... 14 more
2022-03-24 19:29:49,975 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
	... 14 more
2022-03-24 19:29:49,976 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 3
2022-03-24 19:29:49,976 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event AM_STARTED
2022-03-24 19:29:50,093 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1648119700908_0006, File: hdfs://ns1:8020/tmp/hadoop-yarn/staging/hadoop/.staging/job_1648119700908_0006/job_1648119700908_0006_2.jhist
2022-03-24 19:29:50,716 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event AM_STARTED
2022-03-24 19:29:50,723 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event JOB_SUBMITTED
2022-03-24 19:29:50,792 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2022-03-24 19:29:50,792 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Skipping cleaning up the staging dir. assuming AM will be retried.
2022-03-24 19:29:50,793 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.ipc.Server: Stopping server on 43966
2022-03-24 19:29:50,800 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2022-03-24 19:29:50,801 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
2022-03-24 19:29:50,801 INFO [Ping Checker] org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: TaskAttemptFinishingMonitor thread interrupted
2022-03-24 19:29:50,801 INFO [IPC Server listener on 0] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 0
2022-03-24 19:29:50,805 ERROR [Listener at 0.0.0.0/43966] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
	... 14 more
2022-03-24 19:29:50,808 INFO [Listener at 0.0.0.0/43966] org.apache.hadoop.util.ExitUtil: Exiting with status 1: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
  • 提交job报错如下：

解决“hadoop集群提交job时报找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster错误”的问题

在命令行下输入hadoop classpath，复制返回的内容，然后在yarn-site.xml的下加上

   <property>
       <name>yarn.application.classpath</name>
       <value>输入刚才返回的Hadoop classpath路径</value>
   </property>

————————————————
版权声明：本文为博客园博主 {hunter}ZY 的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://www.cnblogs.com/zhaoyids/p/15026181.html

  • 提交job报错截图如下：

希望对后来者有所帮助
以上
/抱拳

posted @   醉生梦死_0423  阅读(942)  评论(2)  编辑  收藏  举报
相关博文：
·  hive 2.3.9 解压启动hive报错 & 正常启动之后无法创建数据库
·  Storm提交topology小记
·  Hadoop集群搭建
·  9、hadoop高可用HA集群部署及三种方式验证
·  Kylin启动时报错：ERROR [main] imps.CuratorFrameworkImpl:566 : Background exception was not retry-able or retry gave up
阅读排行：
· 地球OL攻略 —— 某应届生求职总结
· 周边上新：园子的第一款马克杯温暖上架
· Open-Sora 2.0 重磅开源！
· 提示词工程——AI应用必不可少的技术
· .NET周刊【3月第1期 2025-03-02】
历史上的今天：
2020-03-24 hdoj_2012年复试题（贪吃蛇）——java实现
点击右上角即可分享
微信分享提示
AI IDE Trae