阿里云ODPS <====>蚂蚁大数据
1.命令行客户端工具的安装参考文档:http://repo.aliyun.com/odpscmd/?spm=a2c4g.11186623.2.17.5c185c23zHshCq
2.创建和查看表:https://help.aliyun.com/document_detail/27808.html?spm=a2c4g.11186623.6.567.6a677f32HOWgC5
3.阿里云网页查询工具 dataworks数据工厂 <===============>蚂蚁 [数据智能研发平台(数据工厂)]
CTRL + G
创建表、分区表、上传下载数据、在表下建立分区、向分区表内上传下载数据等操作:
创建分区表:<数据需要在分到不同的文件后上传到分区表中才有作用,上传到分区时必须选择分区>
odps@ sdrtest>create table t_people_p (id int, name string) partitioned by (gender string); \\创建分区,分区以“性别”为分区标致
odps@ sdrtest>alter table t_people_p add if not exists partition (gender='male'); \\创建分区表,分区表的标致为 性别为 “男性”
odps@ sdrtest>tunnel upload men.txt t_people_p/gender='male'; \\将事先整理好的男性的数据上传到建立好的男性的分区表内
通过odpscmd查看到的写入到分区表的数据:
odps@ sdrtest>read t_people_p; +------------+------------+------------+ | id | name | gender | +------------+------------+------------+ | 1 | Michael Jordan | male | | 3 | Bruce Willis | male | | 5 | Jhon Knight | male | | 7 | Chiang Kai-shek | male | | 9 | David Beckham | male | +------------+------------+------------+
通过上述方式继续上传女性数据到女性分区表:
odps@ sdrtest>alter table t_people_p add if not exists partition (gender='female'); odps@ sdrtest>tunnel upload women.txt t_people_p/gender='female';
通过odpscmd查看到的写入到分区表的数据:
odps@ sdrtest>read t_people_p; +------------+------------+------------+ | id | name | gender | +------------+------------+------------+ | 2 | Angela Dorthea Merkel | female | | 4 | Kim Kardashian | female | | 6 | Maria Sharapova | female | | 8 | Jennifer Aniston | female | | 10 | Dragon Lady | female | | 1 | Michael Jordan | male | | 3 | Bruce Willis | male | | 5 | Jhon Knight | male | | 7 | Chiang Kai-shek | male | | 9 | David Beckham | male | +------------+------------+------------+
odps@ sdrtest> tunnel download t_people_p/gender='male' down_male; odps@ sdrtest> tunnel download t_people_p/gender='female' down_female; 查看下载的男性分区里面的数据: [root@sdrtest .odpscmd]# cat down_male 1, Michael Jordan 3, Bruce Willis 5, Jhon Knight 7, Chiang Kai-shek 9, David Beckham
查看下载下来的女性分区内的数据: [root@sdrtest .odpscmd]# cat down_female 2, Angela Dorthea Merkel 4, Kim Kardashian 6, Maria Sharapova 8, Jennifer Aniston 10, Dragon Lady