展开
拓展 关闭
订阅号推广码
GitHub
视频
公告栏 关闭

hadoop常见问题

  • 报错1:进入sqoop容器,执行脚本时报错
[root@15b0369d3f2a one_make]# ssh full_import_tables.sh 
ssh: Could not resolve hostname full_import_tables.sh: Name or service not known
[root@15b0369d3f2a one_make]# sh full_import_tables.sh 
mkdir: cannot create directory '/opt/sqoop/one_make/log': File exists
Warning: /opt/sqoop/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /opt/sqoop/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
24/01/18 15:26:09 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
24/01/18 15:26:09 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
24/01/18 15:26:09 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
24/01/18 15:26:09 INFO manager.SqlManager: Using default fetchSize of 1000
24/01/18 15:26:09 INFO tool.CodeGenTool: Beginning code generation
24/01/18 15:26:10 INFO manager.OracleManager: Time zone has been set to GMT
24/01/18 15:26:10 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM CISS4.CISS_BASE_AREAS t WHERE 1=0
24/01/18 15:26:10 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/hadoop-2.7.0
Note: /tmp/sqoop-root/compile/3d43c70bd6c722c79bc5f80ae3d7ebfa/CISS4_CISS_BASE_AREAS.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
24/01/18 15:26:11 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/3d43c70bd6c722c79bc5f80ae3d7ebfa/CISS4.CISS_BASE_AREAS.jar
24/01/18 15:26:11 INFO manager.OracleManager: Time zone has been set to GMT
24/01/18 15:26:11 INFO manager.OracleManager: Time zone has been set to GMT
24/01/18 15:26:11 INFO mapreduce.ImportJobBase: Beginning import of CISS4.CISS_BASE_AREAS
24/01/18 15:26:11 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
24/01/18 15:26:11 INFO manager.OracleManager: Time zone has been set to GMT
24/01/18 15:26:12 INFO manager.OracleManager: Time zone has been set to GMT
24/01/18 15:26:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM CISS4.CISS_BASE_AREAS t WHERE 1=0
24/01/18 15:26:12 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM CISS4.CISS_BASE_AREAS t WHERE 1=0
24/01/18 15:26:12 INFO mapreduce.DataDrivenImportJob: Writing Avro schema file: /tmp/sqoop-root/compile/3d43c70bd6c722c79bc5f80ae3d7ebfa/CISS4_CISS_BASE_AREAS.avsc
24/01/18 15:26:12 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
24/01/18 15:26:12 INFO client.RMProxy: Connecting to ResourceManager at hadoop.bigdata.cn/172.33.0.121:8032
24/01/18 15:26:13 INFO ipc.Client: Retrying connect to server: hadoop.bigdata.cn/172.33.0.121:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
24/01/18 15:26:14 INFO ipc.Client: Retrying connect to server: hadoop.bigdata.cn/172.33.0.121:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
24/01/18 15:26:15 INFO ipc.Client: Retrying connect to server: hadoop.bigdata.cn/172.33.0.121:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
  • 解决方案,进入hadoop容器,查看yarn是否启动
[root@f35c2d30005b /]# jps
545 SecondaryNameNode
357 DataNode
2725 Jps
202 NameNode
[root@f35c2d30005b /]# start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /opt/hadoop-2.7.0/logs/yarn-root-resourcemanager-f35c2d30005b.out
hadoop.bigdata.cn: starting nodemanager, logging to /opt/hadoop-2.7.0/logs/yarn-root-nodemanager-f35c2d30005b.out
[root@f35c2d30005b /]# jps
545 SecondaryNameNode
2913 NodeManager
357 DataNode
2790 ResourceManager
202 NameNode
3210 Jps
posted @ 2024-01-18 23:36  DogLeftover  阅读(27)  评论(0编辑  收藏  举报