sqoop使用的问题

 

找不到表

17/05/02 18:15:47 ERROR tool.ImportTool: Imported Failed: There is no column found in the target table tysecdb.secsight_config_count. Please ensure that your table name is correct

sqoop导入到出时 对应RDBMS数据库的表名必须大写 不支持小写。

 

 

无法建立连接

Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: Io 异常: The Network Adapter could not establish the connection at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.RuntimeException: java.sql.SQLException: Io 异常: The Network Adapter could not establish the connection at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220) at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165) ... 9 more Caused by: java.sql.SQLException: Io 异常: The Network Adapter could not establish the connection at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146) at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255) at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387) at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:439) at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:165) at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35) at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302) at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213) ... 10 more

昨天发了从DB2中导入数据的错误日志，后来定位为JDBC驱动包的问题，但是替换之后还是没用，然后又定位为是由于JDBC线程未关闭，然后换成了从Oracle中导入数据，又来了另外一个问题：连接有时能通，有时不能通，对于-m 4是基本通不了，原因在于Mapreduce是自动分发匹配的，所以有些结点可能是不可见的，这样对我们连接又会出现网络不通的问题，初步解决方案：自行修改源代码，找到任务分发代码，直接指定一个专门用来做数据导入的结点。正在实践ing。。。。

【HADOOP集群中的每一个节点 都必须能连通数据库】