Spark如何把NA设置为-1
masterDF = masterDF.na.fill(-1)
其它:
也可以在fill中使用Map,其中key是列名,value可以是Int, Long, Float, Double, String, Boolean.
masterDF.na.fill(masterDF.columns.map(_ -> false).toMap)
来自:
https://stackoverflow.com/questions/51073493/spark-dataframe-na-fill-boolean-column-type
* For example, the following replaces null values in column "A" with string "unknown", and * null values in column "B" with numeric value 1.0. * {{{ * df.na.fill(Map( * "A" -> "unknown", * "B" -> 1.0 * )) * }}}