spark整合hive

spark整合hive

启动hdfs

start-dfs.sh

 

启动spark

sbin/start-all.sh

 

添加hive-site.xml,core-site.xml,hdfs-site.xml

把hive-site.xml,core-site.xml,hdfs-site.xml拷贝到/opt/module/spark-2.1.1-bin-hadoop2.7/conf

 

启动spark-sql

bin/spark-sql --master spark://master:7077 --driver-class-path /opt/software/mysql-connector-java-5.1.43-bin.jar

spark-sql连hive

spark-sql>show databases;

spark-sql>use default;

spark-sql>show tables;