06 RDD编程

 

 

 

  • 总共有多少学生?map(), distinct(), count()
  •  

     

  • 开设了多少门课程?
  •  

     

  • 每个学生选修了多少门课?map(), countByKey()
  •  

     

  • 每门课程有多少个学生选?map(), countByValue()
  •  

     

  • Tom选修了几门课?每门课多少分?filter(), map() RDD
  •  

     

  • Tom选修了几门课?每门课多少分?map(),lookup()  list

 

 

 

  • Tom的成绩按分数大小排序。filter(), map(), sortBy()
  •  

     

  • Tom的平均分。map(),lookup(),mean()
  •  

     

  • 求每门课的选修人数及平均分。combineByKey()
  •  course_list = stu_rdd_cource_count_reduce.collect()

      sk = ''
      rs = 0
      zf = 0
      for j in range(len(course_list)):
        sk = str(course_list[j]).split(',')[0].replace("('",'').replace("'",'')
        rs = int(str(course_list[j]).split(',')[1].replace(')',''))
        zf = int(str(stu_rdd_cource_sum_reduce.collect()[j]).split(',')[1].replace(')',''))
        print(sk, rs,round(zf/rs,2))

    结果可视化。 pyecharts.charts,Bar()

     

     

    import pyecharts.options as opts
    from pyecharts.charts import Bar

    x = ['ComputerNetwork', 'Software', 'DataBase', 'Algorithm', 'OperatingSystem', 'Python', 'DataStructure', 'CLanguage']
    y = [
    [142, 132, 126, 144, 134, 136, 131, 128],
    [51.9, 50.91, 50.54, 48.83, 54.94, 57.82, 47.57, 50.61]
    ]

    bar = (
    Bar()
    .add_xaxis(x)
    .add_yaxis(series_name='总人数', y_axis=y[0])
    .add_yaxis(series_name='平均分', y_axis=y[1])
    .set_series_opts(label_opts=opts.LabelOpts(is_show=False))
    .set_global_opts(title_opts=opts.TitleOpts(title='课程', pos_left='right'))
    .set_global_opts(toolbox_opts=opts.ToolboxOpts(is_show=True),
    yaxis_opts=opts.AxisOpts(name="总人数"),
    xaxis_opts=opts.AxisOpts(name="课程名"),axislabel_opts=opts.LabelOpts(rotate=15))
    )
    bar.render()
    bar.render('./bar.html')
posted @ 2021-04-16 16:44  15521419744  阅读(44)  评论(0编辑  收藏  举报