hadoop常见问题

错误描述

 # 启动hadoop集群后DataNode不显示
# 执行命令 hdfs dfs -ls -R / 不起作用

解决方案

 # 先重启并格式化
hdfs namenode -format
 
# 如果不起作用
# 删除data目录并新建
cd /opt/software/hadoop-2.9.2/dfs
rm -rf data
mkdir data

错误2

 # 启动hive，向表中插入1条数据时报错
# 浏览器查看http://192.168.128.103:8088/cluster
 
# 报错详情
Application application_1705164916779_0001 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1705164916779_0001_000001 exited with exitCode: 1
Failing this attempt.Diagnostics: [2024-01-14 00:57:25.540]Exception from container-launch.
Container id: container_1705164916779_0001_01_000001
Exit code: 1
[2024-01-14 00:57:25.550]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.spark.deploy.yarn.ApplicationMaster
[2024-01-14 00:57:25.551]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.spark.deploy.yarn.ApplicationMaster
For more detailed output, check the application tracking page: http://slave3:8088/cluster/app/application_1705164916779_0001 Then click on links to logs of each attempt.
. Failing the application.

解决方案

 # 错误原因，当前环境配置了hive on spark，hdfs中jar包被删除了
 
# hdfs中新建目录，将spark/jars中的包复制到该目录
hdfs dfs -mkdir /spark/hive-jars
hadoop fs -put /usr/local/software/spark-2.2.0/jars/* /spark/hive-jars

posted @ 2024-01-13 21:35 DogLeftover 阅读(123) 评论(0) 编辑收藏举报

刷新页面返回顶部

登录后才能查看或发表评论，立即登录或者逛逛博客园首页

相关博文：

· hadoop常见问题

· hadoop集群常见错误

· hadoop 运行测试

· 超详细【入门精讲】数据仓库原理&实战一步一步搭建数据仓库内附相应实验代码和镜像数据和脚本

· 启动Hadoop集群 namenode 未启动检查日志和xml配置原因 java 版本太高

阅读排行：
· DeepSeek 开源周回顾「GitHub 热点速览」
· 物流快递公司核心技术能力-地址解析分单基础技术分享
· .NET 10首个预览版发布：重大改进与新特性概览！
· AI与.NET技术实操系列（二）：开始使用ML.NET
· 单线程的Redis速度为什么快？

hadoop常见问题

我的标签

合集 (4)

随笔分类 (1350)

阅读排行榜

推荐排行榜

	# 启动hadoop集群后DataNode不显示
	# 执行命令 hdfs dfs -ls -R / 不起作用

	# 先重启并格式化
	hdfs namenode -format

	# 如果不起作用
	# 删除data目录并新建
	cd /opt/software/hadoop-2.9.2/dfs
	rm -rf data
	mkdir data

	# 启动hive，向表中插入1条数据时报错
	# 浏览器查看http://192.168.128.103:8088/cluster

	# 报错详情
	Application application_1705164916779_0001 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1705164916779_0001_000001 exited with exitCode: 1
	Failing this attempt.Diagnostics: [2024-01-14 00:57:25.540]Exception from container-launch.
	Container id: container_1705164916779_0001_01_000001
	Exit code: 1
	[2024-01-14 00:57:25.550]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
	Last 4096 bytes of prelaunch.err :
	Last 4096 bytes of stderr :
	错误: 找不到或无法加载主类 org.apache.spark.deploy.yarn.ApplicationMaster
	[2024-01-14 00:57:25.551]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
	Last 4096 bytes of prelaunch.err :
	Last 4096 bytes of stderr :
	错误: 找不到或无法加载主类 org.apache.spark.deploy.yarn.ApplicationMaster
	For more detailed output, check the application tracking page: http://slave3:8088/cluster/app/application_1705164916779_0001 Then click on links to logs of each attempt.
	. Failing the application.

	# 错误原因，当前环境配置了hive on spark，hdfs中jar包被删除了

	# hdfs中新建目录，将spark/jars中的包复制到该目录
	hdfs dfs -mkdir /spark/hive-jars
	hadoop fs -put /usr/local/software/spark-2.2.0/jars/* /spark/hive-jars