mapreducer调优

一、mapreduce当中的计数器可以共我们调优

 

自定义计数器的两种方式?

1、在mapper里面
　　Counter counter = context.getCounter("MAP_INPUT_RECORD_COUNTER", "MAP_RECORD_NUM");
　　counter.increment(1L);
2、在reduce里面,自定义枚举类型:
　　

//定义枚举类
public static enum Counter{
     //定义接收了多少个key
    REDUCE_INPUT_KEY_NUM,
    //定义接收了多少个value
    REDUCE_INPUT_VALUE_NUM
}

//计数接收的key
org.apache.hadoop.mapreduce.Counter counter = context.getCounter(Counter.REDUCE_INPUT_KEY_NUM);
counter.increment(1L);



//计数接受的value
org.apache.hadoop.mapreduce.Counter counter1 = context.getCounter(Counter.REDUCE_INPUT_VALUE_NUM);

counter1.increment(1L);

二、