Apache Hive 3.1.3 单机部署

环境需求

安装JDK1.8

根据需要选择openjdk、Oracle JDK

安装MySQL8.0.25

见博客
https://www.cnblogs.com/happy-king/p/16408631.html

安装apache-hive3.1.3

目录规划

• hive的数据目录：/app/data/apache-hive-3.1.3
• supervisor日志存储目录：/app/logs/hive-server2/、/app/logs/hive-metastore

下载安装包

wget https://mirrors.aliyun.com/apache/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz
tar -xvf apache-hive-3.1.3-bin.tar.gz -C /app/3rd/

修改配置

修改/app/3rd/apache-hive-3.1.3-bin/conf/hive-site.xml

主要修改hive元数据使用的数据库

  <!-- 元数据存放到mysql -->
  <property>
      <name>javax.jdo.option.ConnectionURL</name>
      <!-- jdbc url,hive的元素据存储到mysql的hive数据库中，如果mysql没有hive数据库则自动创建 -->
      <value>jdbc:mysql://172.30.96.41:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value>
  </property>
  <!-- mysql驱动名称 -->
  <property>
      <name>javax.jdo.option.ConnectionDriverName</name>
      <!-- <value>com.mysql.jdbc.Driver</value> -->
      <value>com.mysql.cj.jdbc.Driver</value>
  </property>
  <!-- mysql用户名 -->
  <property>
      <name>javax.jdo.option.ConnectionUserName</name>
      <value>hive</value>
  </property>
  <!-- mysql密码 -->
  <property>
      <name>javax.jdo.option.ConnectionPassword</name>
      <value>password</value>
  </property>

其他的见服务器文件内容

默认hive shell会输出很多日志（info级别），添加log4j.properties，文件内容如下（根据实际需要进行更改）

log4j.rootLogger=WARN, CA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

初始化数据库

schematool -dbType mysql --initSchema

服务管理：metastore和hiveserver2

使用supervisor进行服务管理
cd /etc/supervisord.d/

metastore

hive-metastore.ini

[program:hive-metastore]
user=root
environment=HADOOP_HOME=/app/3rd/hadoop-3.3.1
command=/app/3rd/apache-hive-3.1.3-bin/bin/hive --service metastore
directory=/app/3rd/apache-hive-3.1.3-bin
autostart=True
autorestart=True
redirect_stderr=True
stopsignal=INT
stopasgroup=True
stdout_logfile_maxbytes=100MB
stdout_logfile_backups=1
stderr_logfile_maxbytes=100MB
stderr_logfile_backups=1
stdout_logfile=/app/logs/hive-metastore/supervisor_stdout.log
stderr_logfile=/app/logs/hive-metastore/supervisor_stderr.log

systemctl update
supervisorctl status hive-metastore

hiveserver2

hive-server2.ini

[program:hive-server2]
user=root
environment=HADOOP_HOME=/app/3rd/hadoop-3.3.1
command=/app/3rd/apache-hive-3.1.3-bin/bin/hive --service hiveserver2
directory=/app/3rd/apache-hive-3.1.3-bin
autostart=True
autorestart=True
redirect_stderr=True
stopsignal=INT
stopasgroup=True
stdout_logfile_maxbytes=100MB
stdout_logfile_backups=1
stderr_logfile_maxbytes=100MB
stderr_logfile_backups=1
stdout_logfile=/app/logs/hive-server2/supervisor_stdout.log
stderr_logfile=/app/logs/hive-server2/supervisor_stderr.log

systemctl update
supervisorctl status hive-server2