随笔分类 - Hadoop
1
摘要:错误:hive> show tables;FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeExceptio...
阅读全文
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-
摘要:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable ...
阅读全文
摘要:进行初始化OK
阅读全文
摘要:错误提示:org.apache.hadoop.security.AccessControlException: Permission denied: user=TIAN, access=ALL, inode="/data/output":grip:superg...
阅读全文
摘要:在master节点的slaves文件中增加新节点的主机名 在新节点手动启动 hadoop-daemon.sh start datanode yarn-daemon.sh start nodemanager 在主节点 hdfs dfsadmin -refresh...
阅读全文
摘要:这种方式的话,顾名思义,就是说,将散落在集群中各个机器上的日志,最后都给聚合起来,让我们可以统一查看 如果打开了日志聚合的选项,即yarn.log-aggregation-enable,container的日志会拷贝到hdfs上去,并从机器中删除 对于这种情...
阅读全文
HADOOP Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission deni
摘要:错误:Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=PC-TIAN, access=WRITE, in...
阅读全文
摘要:完整错误: 解决方法,添加一下内容:vi $HADOOP_HOME/etc/hadoop/yarn-site.xml yarn.nodemanager.aux-services mapreduce_shuffle ...
阅读全文
摘要:错误:java.lang.NoClassDefFoundError: org/apache/tez/dag/api/TezConfiguration at org.apache.hadoop.hive.ql.exec.tez.TezSessio...
阅读全文
摘要:vi core-site.xml
阅读全文
摘要:1、vi /etc/profile 添加一下内容 export HDFS_NAMENODE_USER=root export HDFS_DATANODE_USER=root export HDFS_SECONDARYNAMENODE_USER=root exp...
阅读全文
HDP Caused by: org.apache.hadoop.ipc.RemoteException: Invalid resource request, requested resource t
摘要:完整错误:Caused by: org.apache.hadoop.ipc.RemoteException: Invalid resource request, requested resource type=[memory-mb] , maximum all...
阅读全文
HDP WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.securi
摘要:KERBEROS安装完以后hdfs命令不能正常使用,修改:重启ambari-server ipactl
阅读全文
master: Warning: Permanently added the ECDSA host key for IP address '192.168.5.10' to the list of k
摘要:找到known_hosts删除掉,重新ssh xxx生成免密即可
阅读全文
HIVE Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Directory hdfs://m
摘要:错误:Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Directory hdfs://mycluster/data/hive/warehouse/ods...
阅读全文
摘要:chmod -R 755 hsperfdata_hadoop
阅读全文
摘要:添加以下文件: yarn.nodemanager.aux-services mapreduce_shuffle yarn.nodemanager.aux-services.mapr...
阅读全文
FLUME Exception in thread “SinkRunner-PollingRunner-DefaultSinkProcessor“ java.lang.NoSuchMethodErro
摘要:更换与hadoop的版本一直,我是hadoop3.1.3
阅读全文
摘要:解决:编译时候缺少了相关jar包,需要重新编译:添加内容-Phive-thriftserver ./dev/make-distribution.sh \ --name "hadoop3-without-hive" \ --tgz "-Pyarn -Phive...
阅读全文
摘要:解决:vi .bash_profile export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_OPTS="-Djava.library.path=$HADOOP_H...
阅读全文
1