sparksql(review part...to be continue)
- relational database
every relation has a schema defining fields in columns
a set of relations
relation: a table with rows and columns
schema: name of relation+column name and type
Students(sid:String, name:String)
instance: actual data
结构僵化-结构well,indices占disk space-便于high performance,transaction慢-数据一致性,sparse data效果差
- transactions used to modify data
- sparse data
- SQL
多种join ,处理unmatched keys。inner,left outer,right outer,fullouter
Spark joins都支持。
- data quality
missing , entity resolution, unit mismatch...
https://www.youtube.com/watch?v=Lzhb8xm0fuk&index=7&list=PLy8rR4qeciOfumPMgluuLYVN9TopVGhJw
From satrys,
Kristen wang