sqoop安装配置
之所以选择Sqoop1是因为Sqoop2目前问题太多。无法正常使用,综合比较后选择Sqoop1。
Sqoop1安装配置比较简单
一、安装部署
(1)、下载地址:http://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.5.2.tar.gz
解压到/opt/cdh5/sqoop
(2)、拷贝mysql的jdbc驱动包mysql-connector-java-5.1.31-bin.jar到sqoop/lib目录下。
(3)、配置环境变量
#sqoop
export SQOOP_HOME=/opt/cdh5/sqoop
export PATH=$PATH:$SQOOP_HOME/bin
(4)、复制sqoop/conf/sqoop-env-template.sh为sqoop-env.sh
添加相关的配置
#Setpath to where bin/hadoop is available
exportHADOOP_COMMON_HOME=/opt/cdh5/hadoop
#Setpath to where hadoop-*-core.jar isavailable
exportHADOOP_MAPRED_HOME=/opt/cdh5/hadoop
#setthe path to where bin/hbase isavailable
exportHBASE_HOME=/opt/cdh5/hbase
#Setthe path to where bin/hive is available
exportHIVE_HOME= /opt/cdh5/hive
#Setthe path for where zookeper config diris
exportZOOCFGDIR= /opt/cdh5/zookeeper
(5)、测试Sqoop
发现有警告
修改$SQOOP_HOME/bin/configure-sqoop
注释掉HCatalog,Accumulo检查(除非你准备使用HCatalog,Accumulo等HADOOP上的组件)
## Moved to be a runtime check in sqoop.
#if [ ! -d "${HCAT_HOME}" ]; then
# echo "Warning: $HCAT_HOME does not exist! HCatalog jobs willfail."
# echo 'Please set $HCAT_HOME to the root of your HCatalog installation.'
#fi
#if [ ! -d "${ACCUMULO_HOME}" ];then
# echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports willfail."
# echo 'Please set $ACCUMULO_HOME to the root of your Accumuloinstallation.'
#fi
再次执行sqoop version
你也可以查看某一个命令的使用说明:
$ sqoopimport --help
$ sqoophelp import
sqoop import 的一个示例如下:
$ sqoop import --connect jdbc:mysql://192.168.56.121:3306/metastore --username hiveuser --password redhat --table TBLS
你还可以使用 --options-file 来传入一个文件,使用这种方式可以重用一些配置参数:
$ sqoop --options-file /users/homer/work/import.txt --table TEST
/users/homer/work/import.txt 文件内容如下:
import
--connect
jdbc:mysql://192.168.56.121:3306/metastore
--username
hiveuser
--password
redhat