problems_spark

目录

• 1 执行sparksql报错
• 2 wordcount程序报错
• 3
• 4
• 5
• 6
• 7

1 执行sparksql报错

报错内容如下：
org.apache.spark.sql.AnalysisException: Unable to generate an encoder for inner class cn.develop.spark.sql.Intro$Person without access to the scope that this class was defined in.Try moving this class out of its parent class.

解决方法：将case class Person(name:String,age:Int)这个样例类移动到父类的外面，就是移到最外面，不要让其包含在任何的类中。

2 wordcount程序报错

报错内容如下：
Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrder
解决步骤：

1. 根据网上的方法，在maven中添加如下依赖：

<dependency>
          <groupId>com.fasterxml.jackson.core</groupId>
          <artifactId>jackson-core</artifactId>
          <version>2.2.4</version>
      </dependency>
      <dependency>
          <groupId>com.fasterxml.jackson.core</groupId>
          <artifactId>jackson-databind</artifactId>
          <version>2.2.4</version>
      </dependency>

2. 再次执行，又报以下错误：
   Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;
   根据网上方法，maven中删除之前添加的依赖，改为添加以下依赖：

<dependency>
            <groupId>com.fasterxml.jackson.module</groupId>
            <artifactId>jackson-module-scala_2.11</artifactId>
            <version>2.7.4</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.module</groupId>
            <artifactId>jackson-module-jaxb-annotations</artifactId>
            <version>2.7.4</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.7.4</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <version>2.7.4</version>
        </dependency>

3

4

5

6

7