求pv
pv定义
衡量网站用户访问的网页的数量,PV与来访者的数量成正比,但是PV并不直接决定页面的正式来访者数量,如同一个来访者通过不断的刷新页面,也可以制造出非常高的PV。
实现代码
public class TestPV {
public static void main(String[] args) {
SparkConf conf = new SparkConf()
.setMaster("local").setAppName("transformation");
JavaSparkContext jsc = new JavaSparkContext(conf);
JavaRDD<String> lineRDD = jsc.textFile("data/pvuvdata.txt");
lineRDD.flatMap(new FlatMapFunction<String, Tuple2<String, String>>() {
@Override
public Iterator<Tuple2<String, String>> call(String s) throws Exception {
String[] str = s.split(" ");
Tuple2<String, String> tuple2= new Tuple2<String, String>(str[5], str[0]);
List<Tuple2<String, String>> list = new ArrayList<>();
list.add(tuple2);
return list.iterator();
}
}).mapToPair(new PairFunction<Tuple2<String, String>, String, Integer>() {
@Override
public Tuple2<String, Integer> call(Tuple2<String, String> tuple2) throws Exception {
Tuple2<String, Integer> tuple = new Tuple2<String, Integer>(tuple2._1, 1);
return tuple;
}
}).reduceByKey(new Function2<Integer, Integer, Integer>() {
@Override
public Integer call(Integer integer, Integer integer2) throws Exception {
return integer+integer2;
}
}).foreach(new VoidFunction<Tuple2<String, Integer>>() {
@Override
public void call(Tuple2<String, Integer> stringIntegerTuple2) throws Exception {
System.out.println(stringIntegerTuple2);
}
});
jsc.stop();
}
}
本文来自博客园,作者:jsqup,转载请注明原文链接:https://www.cnblogs.com/jsqup/p/16388612.html