spark2.1:flatMap的用法
代码示例:
val sample_data_combine_result=List( (0,(List(FitModel(4022,1447.92,-8.38983306721434,2.0)),1)), (1,(List(FitModel(1083,389.88,-11.057139857142857,3.0)),1)), (2,(List(FitModel(4022,1447.92,-8.38983306721434,2.0)),1)) ) scala> val fitResult = sample_data_combine_result.flatMap(s => { | var fitResultItems: ListBuffer[FitModelWithSceneType] = null | if(s._1==0){ | fitResultItems=ListBuffer( | FitModelWithSceneType(0,1,3.0,-3.9110667661835516,0.6989700043360189), | FitModelWithSceneType(0,51,3.0,-3.9110667661835516,1.1760912590556813) | ) | } | if(s._1==1){ | fitResultItems=ListBuffer( | FitModelWithSceneType(1,1,3.0,-3.9110667661835516,0.6989700043360189), | FitModelWithSceneType(1,51,3.0,-3.9110667661835516,1.1760912590556813), | FitModelWithSceneType(1,101,3.0,-3.9110667661835516,1.3979400086720377) | ) | } | if(s._1==2){ | fitResultItems=ListBuffer( | FitModelWithSceneType(2,1,3.0,-3.9110667661835516,0.6989700043360189), | FitModelWithSceneType(2,51,3.0,-3.9110667661835516,1.1760912590556813), | FitModelWithSceneType(2,101,3.0,-3.9110667661835516,1.3979400086720377), | FitModelWithSceneType(2,101,3.0,-3.9110667661835516,1.3979400086720377), | FitModelWithSceneType(2,101,3.0,-3.9110667661835516,1.3979400086720377) | ) | } | | fitResultItems | }) fitResult: List[FitModelWithSceneType] = List( FitModelWithSceneType(0,1,3.0,-3.9110667661835516,0.6989700043360189), FitModelWithSceneType(0,51,3.0,-3.9110667661835516,1.1760912590556813), FitModelWithSceneType(1,1,3.0,-3.9110667661835516,0.6989700043360189), FitModelWithSceneType(1,51,3.0,-3.9110667661835516,1.1760912590556813), FitModelWithSceneType(1,101,3.0,-3.9110667661835516,1.3979400086720377), FitModelWithSceneType(2,1,3.0,-3.9110667661835516,0.6989700043360189), FitModelWithSceneType(2,51,3.0,-3.9110667661835516,1.1760912590556813), FitModelWithSceneType(2,101,3.0,-3.9110667661835516,1.3979400086720377), FitModelWithSceneType(2,101,3.0,-3.9110667661835516,1.3979400086720377), FitModelWithSceneType(2,101,3.0,-3.9110667661835516,1.3979400086720377) )
基础才是编程人员应该深入研究的问题,比如:
1)List/Set/Map内部组成原理|区别
2)mysql索引存储结构&如何调优/b-tree特点、计算复杂度及影响复杂度的因素。。。
3)JVM运行组成与原理及调优
4)Java类加载器运行原理
5)Java中GC过程原理|使用的回收算法原理
6)Redis中hash一致性实现及与hash其他区别
7)Java多线程、线程池开发、管理Lock与Synchroined区别
8)Spring IOC/AOP 原理;加载过程的。。。
【+加关注】。