RDD变换

 

对Key/Value型RDD进行变换

 

 

groupBy按Key汇聚

 

fruit,apple
vegetable,cucumber
fruit,cherry
vegetable,bean
fruit,banana
vegetable,pepper

 

sc.textFile("D:\\LearnSpark\\win\\comineByKeyTest.txt").map(line=>line.split(",")).map(a=>(a(0),a(1))).groupByKey().collect();

 

https://stackoverflow.com/questions/28833926/spark-value-reducebykey-is-not-a-member
scala - What is "WARN ParallelCollectionRDD: Spark does not support nested RDDs (see SPARK-5063)"? - Stack Overflow https://stackoverflow.com/questions/41340698/what-is-warn-parallelcollectionrdd-spark-does-not-support-nested-rdds-see-spa