摘要:
工作中使用MapReduce任务导出一批含有路径的文件,共计行数300W+,需要检测文件是否在对应的服务器中存在,而文件所在的服务器并非hadoop集群的服务器,因此打算采用bash脚本进行。具体的方法如下(可直接看方法2,方法1效率较低): 2. 采用的方法 a. 方法1 原本打算使用如下脚本,进 阅读全文
摘要:
1. Translating Java Classes to Scala Classes Example 1: Example 2: Example 3: Example 4: Example 5: Translating Java imports to Scala imports: Example 阅读全文
摘要:
1. Unified Type System Scala has a unified type system, enclosed by the type Any at the top of the hierarchy and the type Nothing at the bottom of the 阅读全文
摘要:
A trait provides code reusability in Scala by encapsulating method and state and then offing possibility of mixing them into classes thus allowing cod 阅读全文
摘要:
1. 基本原理: 通过hadoop fs -ls *命令获取相关文件或目录的修改时间,然后与设定的过期时间进行比较,之后执行删除操作即可 2. 相关代码: 参考:Hadoop运维-删除目录下过期的数据 阅读全文
摘要:
Scala's object-oriented collections support mutable and immutable type hierarchies. Also support functional higher-order operations such as map, filte 阅读全文
摘要:
The basic functional cornerstones of Scala: immutable data types, passing of functions as parameters and pattern matching. 1. Basic Pattern Matching I 阅读全文
摘要:
1. Functional programming treats computation as the evaluation of mathematical and avoids state and mutable data. Scala encourages an expression-orien 阅读全文
摘要:
1. The three principles of OOP are encapsulation(封装性), inheritance(继承性) and polymorphism(多态性). example: A subtype is guaranteed to have all the member 阅读全文
摘要:
1. Variables (1) Three ways to define variables: 1) val refers to define an immutable variable; 2) var refers to define a mutable variable. 3) lazy va 阅读全文