随笔分类 - Spark
摘要:在spark-shell不能退格,写错了不能删除
阅读全文
摘要:错误详情:egalArgumentException: System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-memor...
阅读全文
摘要:Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resour...
阅读全文
摘要:错误提示:Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "DBCP" plugin to create a ConnectionPool gave a...
阅读全文
SPARK The root scratch dir: /tmp/hive-grip on HDFS should be writable. Current permissions are: rwxr
摘要:错误提示:ERROR scheduler.JobScheduler: Error running job streaming job 1532317741000 ms.0 java.lang.RuntimeException: java.lang.Runtim...
阅读全文
摘要:错误提示:Attempt to invoke the "DBCP" plugin to create a ConnectionPool gave an error 将$HIVE_HOME/conf/hive-site.xml拷贝至$SPARK_HOME/c...
阅读全文
摘要:在diea里面跑spark程序发生jar包冲突,kafka的LZ4版本屏蔽掉
阅读全文
摘要:去hive-site.XML里面查看jdbc:mysql://master:3306/sparksql?createDatabaseIfNotExist=true这个链接的sparksql表里面设置: alter table COLUMNS_V2 modify...
阅读全文
摘要:添加:option("driver","com.mysql.jdbc.Driver")
阅读全文
摘要:找到/hive/bin进行编辑vi hive
阅读全文
摘要:第一章 初探大数据 1、什么是大数据? 大数据特征:4V 数据量(Volume) PB、EB、ZB 给予高度分析的新价值(Value) 巨额数据里面提取需要的高价值数据 ...
阅读全文
摘要:报错内容:ERROR deploy.ClientEndpoint: Exception from cluster was: java.io.FileNotFoundException: /usr/log_data/java/spark-project-0.0....
阅读全文
SPARK 编译2.3.0 Cannot download zinc-0.3.15.tgz with cURL or wget; please install manually and try aga
摘要:yum -y install wget;解决
阅读全文