Hadoop集群遇到错误处理

作者：@kuaiquxie
作者的github：https://github.com/bitebita
本文为作者原创，如需转载，请注明出处：https://www.cnblogs.com/dzwj/p/16734766.html

---

java.net.ConnectException: Call From hadoop102/192.168.10.102 to hadoop102:9000 failed on connection exception: java.net.ConnectException: 拒绝连接;

2022-09-23 12:06:48,600 ERROR [master/hadoop102:16000:becomeActiveMaster] master.HMaster: ***** ABORTING master hadoop102,16000,1663906002774: Unhandled exception. Starting shutdown. *****

2022-09-23 12:06:51,492 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: HMaster Aborted

解决
2022-09-23 12:06:51,492 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: HMaster Aborted:

https://blog.csdn.net/weixin_43719616/article/details/119941375
将
cd /opt/module/hadoop-3.2.2/etc/hadoop/
vim core-site.xml
与
cd /opt/module/hbase-1.7.1/conf
vim hbase-site.xml
内的 hdfs://hadoop100:8020 修改一致
logs 不在报错

配置了 etc/hosts 没有分发 将127.0.0.1 和 ::1 注释掉
配置了 hbase-site.xml 没有分发

Call From hadoop102/192.168.10.102 to hadoop102:8020 failed on connection exception:
https://blog.csdn.net/qq_43668173/article/details/115021479

 

netstat -ntulp | grep 1099

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version
解决：
jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true
把这句话加在连接 字符串

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
解决：

 

[pid=11578,containerID=container_1569779921172_0010_01_000005] is running 261372416B beyond the ‘VIRTUAL’ memory limit.
Current usage: 106.1 MB of 1 GB physical memory used; 2.3 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1569779921172_0010_01_000005 :
原文链接：https://blog.csdn.net/cao1315020626/article/details/101712223

错误：
Caused by: java.lang.RuntimeException: Can't parse input data: 'buy_nbr'
解决：
网上的方法没用之后，自己摸索得解：

bin/sqoop export \
--connect "jdbc:mysql://hadoop102:3306/sqoop?useUnicode=true&characterEncoding=utf-8" \
--username root \
--password 123456 \
--table table1 \
--num-mappers 1 \
--export-dir /user/hive/warehouse/hivetest.db/table1 \
--input-fields-terminated-by ","

sqoop在导出数据的时候，
win 下的 mysql 表名 与 hive 下的表名一致
/user/hive/warehouse/hivetest.db/table1