Spark wordcount 编译错误 -- reduceByKey is not a member of RDD
Attempting to run http://spark.apache.org/docs/latest/quick-start.html#a-standalone-app-in-scala from source.
This line val wordCounts = textFile.flatMap(line => line.split(" ")).map(word => (word, 1)).reduceByKey(_+_)
reports compile
value reduceByKey is not a member of org.apache.spark.rdd.RDD[(String, Int)]
Resolution:
import the implicit conversions from SparkContext
:
import org.apache.spark.SparkContext._
They use the 'pimp up my library' pattern to add methods to RDD's of specific types. If curious, seeSparkContext:1296