1、问题描述
(1)问题示例:
[Hadoop@master TestDir]$ sqoop list-databases --connect jdbc:mysql://master:3306/ --username Hive --password ******
2021-11-05 16:18:52,289 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
2021-11-05 16:18:52,316 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2021-11-05 16:18:52,379 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
at org.apache.sqoop.manager.MySQLManager.initOptionDefaults(MySQLManager.java:73)
at org.apache.sqoop.manager.SqlManager.<init>(SqlManager.java:89)
at com.cloudera.sqoop.manager.SqlManager.<init>(SqlManager.java:33)
at org.apache.sqoop.manager.GenericJdbcManager.<init>(GenericJdbcManager.java:51)
at com.cloudera.sqoop.manager.GenericJdbcManager.<init>(GenericJdbcManager.java:30)
at org.apache.sqoop.manager.CatalogQueryManager.<init>(CatalogQueryManager.java:46)
at com.cloudera.sqoop.manager.CatalogQueryManager.<init>(CatalogQueryManager.java:31)
at org.apache.sqoop.manager.InformationSchemaManager.<init>(InformationSchemaManager.java:38)
at com.cloudera.sqoop.manager.InformationSchemaManager.<init>(InformationSchemaManager.java:31)
at org.apache.sqoop.manager.MySQLManager.<init>(MySQLManager.java:65)
at org.apache.sqoop.manager.DefaultManagerFactory.accept(DefaultManagerFactory.java:67)
at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:184)
at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:272)
at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:44)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 20 more
(2)问题综述:
命令:sqoop list-databases --connect jdbc:mysql://master:3306/ --username Hive --password ******
上述命令,是用于查看MySQL已有的数据库。不能成功连接,不能显示出MySQL中已有数据库。
2、解决方案
参考:https://www.cndba.cn/dave/article/3303
(1)根据参考,下载驱动包:commons-lang-2.6.jar
下载地址:https://mirrors.tuna.tsinghua.edu.cn/apache//commons/lang/binaries/commons-lang-2.6-bin.zip
下载地址可能存在失效,只需要找到commons-lang-2.6.jar
(2)上述下载压缩包:commons-lang-2.6-bin.zip,须解压,然后将其中的“commons-lang-2.6.jar”拷贝至Sqoop中lib中。
(3)再次测试:
[Hadoop@master TestDir]$ sqoop list-databases --connect jdbc:mysql://master:3306/ --username Hive --password ******
2021-11-05 17:07:59,566 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
2021-11-05 17:07:59,584 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2021-11-05 17:07:59,645 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
mysql
information_schema
performance_schema
sys
Test
source
hive
上述测试,成功查看MySQL中数据库,说明问题已解决。
3、问题剖析
原因是:笔者所安装的Sqoop版本为sqoop-1.4.7,缺少驱动包:commons-lang-2.6.jar,已存在commons-lang3-3.4.jar不合适。
因此需要另外下载驱动包:commons-lang-2.6.jar。