Hive的安装与配置
2016-05-02 13:28 猎手家园 阅读(869) 评论(0) 编辑 收藏 举报1、因为我使用MySQL做为Hive的元数据库,所以先安装MySQL。
参考:http://www.cnblogs.com/hunttown/p/5452205.html
登录命令:mysql -h主机地址 -u用户名 -p用户密码
mysql –u root #初始登录没有密码
修改密码
格式:mysqladmin -u用户名 -p旧密码 password 新密码
mysql>mysqladmin -uroot –password 123456
注:因为开始时root没有密码,所以-p旧密码一项就可以省略了。
创建用于hive的用户hadoopuser
创建用户命令:CREATE USER username@"host" IDENTIFIED BY 'password';
mysql> CREATE USER hadoopuser@"192.168.254.151" IDENTIFIED BY '123456';
授权命令:GRANT privileges ON databasename.tablename TO 'username'@'host'
mysql> GRANT ALL PRIVILEGES ON *.* TO hadoopuser@"192.168.254.151" WITH GRANT OPTION;
创建用户和授权可以一起做:
mysql> GRANT ALL PRIVILEGES ON *.* TO hadoopuser@"192.168.254.151" IDENTIFIED BY '123456' WITH GRANT OPTION;
创建数据库hive,用于hive数据存储
mysql> create database hive
2、解压hive到 /home/hadoopuser/
3、切换到root用户,加入环境变量:
export HIVE_HOME=/home/hadoopuser/hive export PATH=$PATH:$HIVE_HOME/bin
4、使用root用户,给hive/bin增加权限
chmod 777 /hive/bin/*
5、配置文件
切换到 /hive/conf
cp hive-default.xml.template hive-site.xml cp hive-log4j.properties.template hive-log4j.properties #Hive-2.1.0没有此配置项
(1)配置hive-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://192.168.254.156:3306/hive?createDatabaseIfNotExist=true</value> <description>JDBC connect string for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hadoop</value> <description>username to use against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> <description>password to use against metastore database</description> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> <description>location of default database for the warehouse</description> </property> </configuration>
如果使用derby元数据库,则JDBC要配置成:
<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:derby:/opt/hive/metastore_db;create=true</value> <description>JDBC connect string for a JDBC metastore</description> </property>
注1:仓库目录如果没有要创建
hdfs dfs –mkdir /user/hive
hdfs dfs –mkdir -p /user/hive/warehouse
注2:mysql的驱动jar包要上传到hive/lib下
(2)配置hive-log4j.properties
#log4j.appender.EventCounter=org.apache.hadoop.metrics.jvm.EventCounter log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter
6、在HDFS中创建/tmp和/user/hive/warehouse并设置权限
hadoop fs -mkdir /tmp hadoop fs -mkdir -p /user/hive/warehouse hadoop fs -chmod g+w /tmp hadoop fs -chmod g+w /user/hive/warehouse
注:hadoop 命令换成了hdfs命令,上面的命令如同下面的命令
hdfs dfs -mkdir /tmp hdfs dfs -mkdir -p /user/hive/warehouse hdfs dfs -chmod g+w /tmp hdfs dfs -chmod g+w /user/hive/warehouse
7、手动上传mysql的jdbc库到hive/lib目录。
http://mirror.bit.edu.cn/mysql/Downloads/Connector-J/
mysql-connector-java-5.1.22-bin.jar
8、初始化 如果使用derby元数据库,那么需要进行初始化:
[hadoopuser@Hadoop-NN-01 ~]# schematool -initSchema -dbType derby #执行成功信息 Starting metastore schema initialization to 2.0.0 Initialization script hive-schema-2.0.0.derby.sql Initialization script completed schemaTool completed
如果运行时出现以下错误,说明上面的步骤没有执行,请执行:
Exception in thread "main" java.lang.RuntimeException: Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql)
如果使用schematool初始化数据库时出现以下错误:
Initialization script hive-schema-2.1.0.derby.sql Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000) org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !! *** schemaTool failed ***
说明数据库文件夹中已经存在一些文件,解决方法就是清空数据库文件夹(也就是前面配置的/opt/hive/metastore_db文件夹)
9、启动hive
hive --service metastore & #启动metastore服务 hive --service hiveserver2 & #启动hiveserver服务 hive shell #启动hive客户端
Hive 使用
1、创建数据库
CREATE DATABASE myhive;
2、创建表
CREATE TABLE doc_hive (id int, username string, sex int, age int, email string, createtime string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';
3、导入数据
LOAD DATA LOCAL INPATH '/home/hadoopuser/doc/t-1.txt' OVERWRITE INTO TABLE doc_hive ;
验证:select * from myhive.doc_hive;
Hive的具体使用,在接下来的博客中会有体现。
![](https://images.cnblogs.com/cnblogs_com/hunttown/823638/o_yunqiyu.jpg)