Spark 读取json字符串转为DataFrame

来源:https://stackoverflow.com/questions/39619782/how-to-read-in-memory-json-string-into-spark-dataframe


先将字符串转为RDD,再由RDD转为DF。

From Spark SQL guide:

val otherPeopleRDD = spark.sparkContext.makeRDD(
"""{"name":"Yin","address":{"city":"Columbus","state":"Ohio"}}""" :: Nil)
val otherPeople = spark.read.json(otherPeopleRDD)
otherPeople.show()

This creates a DataFrame from an intermediate RDD (created by passing a String).

 

posted @ 2020-11-26 17:47  梦醒江南·Infinite  阅读(1884)  评论(0编辑  收藏  举报