10 2020 档案

spark bulkload 报错异常:Caused by: java.io.IOException: Added a key not lexically larger than previous

摘要：恢复内容开始 Caused by: java.io.IOException: Added a key not lexically larger than previous. Current cell = 2c90807b73996dff0173b758788600a5,104/CF:dimensio 阅读全文

posted @ 2020-10-29 17:53 夜半钟声到客船阅读(1891) 评论(0) 推荐(0) 编辑

idea或者java远程提交spark任务到yarn,Exception1

摘要：spark通過idea远程提交job到yarn: Caused by: java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to fiel 阅读全文

posted @ 2020-10-29 17:41 夜半钟声到客船阅读(1445) 评论(0) 推荐(0) 编辑

hdp、cdh版本大数据maven仓库

摘要：Hdp版本: <repositories> <repository> <id>nexus</id> <name>local private nexus</name> <url> https://repo.hortonworks.com/content/repositories/releases/</ 阅读全文

posted @ 2020-10-23 14:35 夜半钟声到客船阅读(691) 评论(0) 推荐(1) 编辑

maven 导出所有依赖jar到指定路径

摘要：mvn dependency:copy-dependencies -DoutputDirectory=lib 阅读全文

posted @ 2020-10-20 19:54 夜半钟声到客船阅读(298) 评论(0) 推荐(0) 编辑

spark使用idea向yarn提交报错:Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/api/client/config/ClientConfig

摘要：解决方法: 找到1.19版本放到spark的jars目录下阅读全文

posted @ 2020-10-20 11:12 夜半钟声到客船阅读(1201) 评论(0) 推荐(0) 编辑