摘要: redis 阅读全文
posted @ 2019-02-21 15:26 JackSun924 阅读(1149) 评论(0) 推荐(0) 编辑
摘要: 何谓五横,基本还是根据数据的流向自底向上划分五层,跟传统的数据仓库其实很类似,数据类的系统,概念上还是相通的,分别为数据采集层、数据处理层、数据分析层、数据访问层及应用层。同时,大数据平台架构跟传统数据仓库有一个不同,就是同一层次,为了满足不同的场景,会采用更多的技术组件,体现百花齐放的特点,这是一 阅读全文
posted @ 2019-02-15 11:08 JackSun924 阅读(47190) 评论(0) 推荐(0) 编辑
摘要: combineByKey-->>aggregateByKey-->>foldByKey-->>reduceByKey-->>groupByKey-->>countByKey 0> combineByKey(createCombiner, mergeValue, mergeCombiners, num 阅读全文
posted @ 2019-01-28 18:11 JackSun924 阅读(966) 评论(0) 推荐(0) 编辑
摘要: Flume Interceptors Flume has the capability to modify/drop events in-flight. This is done with the help of interceptors. Interceptors are classes that 阅读全文
posted @ 2019-01-15 17:41 JackSun924 阅读(582) 评论(0) 推荐(0) 编辑
摘要: Sqoop User Guide (v1.4.6) 1. Introduction Sqoop is a tool designed to transfer data between Hadoop and relational databases or mainframes. You can use 阅读全文
posted @ 2019-01-14 16:07 JackSun924 阅读(1289) 评论(0) 推荐(0) 编辑
摘要: Import参数; Table 1. Common arguments ArgumentDescription --connect <jdbc-uri> Specify JDBC connect string --connection-manager <class-name> Specify con 阅读全文
posted @ 2019-01-14 15:56 JackSun924 阅读(586) 评论(0) 推荐(0) 编辑
摘要: 名词解释 ▪ Operations are eager when they are executed as soon as the statement is reached in the code; 勤快运行:接收到代码立刻执行; ▪ Operations are lazy when the exe 阅读全文
posted @ 2018-12-28 08:20 JackSun924 阅读(217) 评论(0) 推荐(0) 编辑
摘要: 名词解释 CDH #(Cloudera’s Distribution including Apache Hadoop) ecosystem projects #生态系统项目 Subscription #订阅 Volume #容积 Velocity #速度 Variety #多样的 ETL #Extr 阅读全文
posted @ 2018-12-21 15:27 JackSun924 阅读(313) 评论(0) 推荐(0) 编辑
摘要: MySQL安装 这个安装很简单,是在线安装,只需要按顺序执行一下几个命令就ok了。(1)sudo apt-get install mysql-server(2)sudo apt-get install mysql-client(3)sudo apt-get install libmysqlclien 阅读全文
posted @ 2018-12-18 18:07 JackSun924 阅读(253) 评论(0) 推荐(0) 编辑
摘要: Privileges Required for Hive Operations Codes Y: Privilege required. Y + G: Privilege "WITH GRANT OPTION" required. Action Select Insert Update Delete 阅读全文
posted @ 2018-12-18 16:36 JackSun924 阅读(374) 评论(0) 推荐(0) 编辑