sparksql(review part...to be continue)

• relational database

every relation has a schema defining fields in columns

a set of relations

relation: a table with rows and columns

schema: name of relation+column name and type

Students(sid:String, name:String)

instance: actual data

结构僵化-结构well，indices占disk space-便于high performance，transaction慢-数据一致性，sparse data效果差

• transactions used to modify data
• sparse data
• SQL

多种join ,处理unmatched keys。inner，left outer，right outer，fullouter

Spark joins都支持。

• data quality

missing ， entity resolution， unit mismatch...
https://www.youtube.com/watch?v=Lzhb8xm0fuk&index=7&list=PLy8rR4qeciOfumPMgluuLYVN9TopVGhJw