Spark2 Dataset统计指标:mean均值,variance方差,stddev标准差,corr(Pearson相关系数),skewness偏度,kurtosis峰度

val df4=spark.sql("SELECT mean(age),variance(age),stddev(age),corr(age,yearsmarried),skewness(age),kurtosis(age) FROM Affairs")

df4.show
+--------+------------------+------------------+-----------------------+-----------------+--------------------+
|avg(age)|     var_samp(age)|  stddev_samp(age)|corr(age, yearsmarried)|    skewness(age)|       kurtosis(age)|
+--------+------------------+------------------+-----------------------+-----------------+--------------------+
|    34.0|173.33333333333334|13.165611772087667|     0.7456766124552038|0.965388004190285|-0.43417159763313595|
+--------+------------------+------------------+-----------------------+-----------------+--------------------+

 

posted @ 2016-11-25 17:55  智能先行者  阅读(9241)  评论(0编辑  收藏  举报