Spark_4_2：Spark函数之collect、toArray和collectAsMap

collect、toArray

将RDD转换为Scala的数组。

collectAsMap

与collect、toArray相似。collectAsMap将key-value型的RDD转换为Scala的map。

注意：map中如果有相同的key，其value只保存最后一个值。

# 创建一个2分区的RDD
scala> var z = sc.parallelize(List( ("cat",2), ("cat", 5), ("mouse", 4),("cat", 12), ("dog", 12), ("mouse", 2)), 2)
z: org.apache.spark.rdd.RDD[(String, Int)] = ParallelCollectionRDD[129] at parallelize at <console>:21

# 输出所有分区的数据
scala> z.collect
res44: Array[(String, Int)] = Array((cat,2), (cat,5), (mouse,4), (cat,12), (dog,12), (mouse,2))

# 转化为字典
scala> z.collectAsMap
res45: scala.collection.Map[String,Int] = Map(dog -> 12, cat -> 12, mouse -> 2)
scala>