spark报错：Class org.apache.mapreduce.io.compress.GzipCodec not found

现象：在进行spark本地IDEA运行的时候，报错

报错：

Exception in thread "main" java.lang.RuntimeException: Error in configuring object
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
     at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
     at org.apache.spark.rdd.HadoopRDD.getInputFormat(HadoopRDD.scala:185)
     at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:198)
     at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)
     at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)
     at scala.Option.getOrElse(Option.scala:121)
     at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)
     at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:46)
     at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)
     at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)
     at scala.Option.getOrElse(Option.scala:121)
     at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)
     at org.apache.spark.SparkContext.runJob(SparkContext.scala:2094)
     at org.apache.spark.rdd.RDD.count(RDD.scala:1166)
     at spark._core.describe.sparkRunJob$.main(sparkRunJob.scala:20)
     at spark._core.describe.sparkRunJob.main(sparkRunJob.scala)
Caused by: java.lang.reflect.InvocationTargetException
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
     ... 17 more
Caused by: java.lang.IllegalArgumentException: Compression codec org.apache.mapreduce.io.compress.GzipCodec not found.
     at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
     at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:180)
     at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
     ... 22 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.mapreduce.io.compress.GzipCodec not found
     at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2171)
     at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
     ... 24 more

原因: 程序在运行的时候回去读取配置文件，在IDEA中也就是配置在resource中的core.xml文件，xml文件中如果配置以下属性：

<property>
  <name>io.compression.codecs</name>
  <value>org.apache.mapreduce.io.compress.GzipCodec,org.apache.mapreduce.io.compress.DefaultCodec,org.apache.mapreduce.io.compress.SnappyCodec</value>
</property>

,程序如果发现没有这几个类对应的jar包的话，就会包上面的错，如果已经导入了上面几个类对应的包的话，在配置上面这个属性的时候，不要有换行符，不要有空格等，否则会引起另外的错误

解决方法：

第一种：本地测试可以将这个配置给注释掉

第二种，如果是生产上的话，则需要将报错的类对应的jar包放到对应的额lib目录下