Centos7安装Sqoop

1、将sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz上传至/opt/software

2、解压

[bigdata@bigdata01 software]$ tar -zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz -C /opt/module/

3、修改名称

[bigdata@bigdata01 module]$ mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha/ sqoop-1.4.6

4、修改文件

[bigdata@bigdata01 module]$ cd /opt/module/sqoop-1.4.6/conf/

4.1、修改文件名称

[bigdata@bigdata01 conf]$ mv sqoop-env-template.sh sqoop-env.sh

4.2修改文件内容

[bigdata@bigdata01 conf]$ vim sqoop-env.sh 
#增加以下内容
export HADOOP_COMMON_HOME=/opt/module/hadoop-3.1.3
export HADOOP_MAPRED_HOME=/opt/module/hadoop-3.1.3
export HIVE_HOME=/opt/module/hive-3.1.2
export ZOOKEEPER_HOME=/opt/module/zookeeper-3.5.7
export ZOOCFGDIR=/opt/module/zookeeper-3.5.7/conf

5、拷贝jdbc驱动

将mysql-connector-java-5.1.27-bin.jar上传至/opt/module/sqoop-1.4.6/lib/目录

6、配置环境变量

[bigdata@bigdata01 lib]$ sudo vim /etc/profile.d/my_env.sh 
#增加如下内容
#SQOOP_HOME
export SQOOP_HOME=/opt/module/sqoop-1.4.6
export PATH=$PATH:$SQOOP_HOME/bin
#使配置生效
[bigdata@bigdata01 lib]$ source /etc/profile.d/my_env.sh 

7、测试是否安装成功

[bigdata@bigdata01 sqoop-1.4.6]$ sqoop list-databases --connect jdbc:mysql://bigdata01:3306 --username root --password root
Warning: /opt/module/sqoop-1.4.6/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/module/sqoop-1.4.6/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/module/sqoop-1.4.6/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
No such sqoop tool: --list-databases. See 'sqoop help'.
[bigdata@bigdata01 sqoop-1.4.6]$ sqoop list-databases --connect jdbc:mysql://bigdata01:3306 --username root --password root
Warning: /opt/module/sqoop-1.4.6/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/module/sqoop-1.4.6/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/module/sqoop-1.4.6/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
2021-09-11 15:49:18,691 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
2021-09-11 15:49:18,732 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2021-09-11 15:49:18,889 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
gmall
mysql
performance_schema
sys

 8、示例，将mysql表数据导入到hdfs

sqoop import --connect jdbc:mysql://bigdata01:3306/mall --username root --password root --table user_info --columns id,user_name,create_time --where "id BETWEEN 100 AND 200" --target-dir /testsqoop --delete-target-dir --num-mappers 2 --split-by id  --fields-terminated-by "\t"

9、示例2:使用query必须在where中加入$CONDITIONS，并在$前加上\进行转义

sqoop import --connect jdbc:mysql://bigdata01:3306/mall --username root --password root --query "SELECT id,name,create_time FROM user_info WHERE 1=1 and \$CONDITIONS "  --target-dir /testhadoop --delete-target-dir --split-by id --fields-terminated-by "\t" --num-mappers 2