Spark中RDD的基本转化操作

对一个数据为{1,2,3,4}的RDD进行基本的RDD转化操作:

下面用scala语言进行实现:

1:val example = sc.parallelize(List(1, 2, 3,  3))

 val result = example.map(x => x+1)

 println(result.collect().mkString(","))

2:val example = sc.parallelize(List(1, 2, 3, 3))

 val result = example.flatMap(x =>x.to(3))

 println(result.collect().mkString(","))

3:val example = sc.parallelize(List(1, 2, 3, 3))

 val result = example.filter(x => x !=1)

 println(result.collect().mkString(","))

4:val example = sc.parallelize(List(1, 2, 3, 3))

 val result = example.distinct()

 println(result.collect().mkString(","))

5:val example = sc.parallelize(List(1, 2, 3, 3))

 val result = example.sample()

 println(result.collect().mkString(","))

posted @ 2017-07-04 08:39  reedom1991  阅读(270)  评论(0编辑  收藏  举报