Error while processing transaction.java.lang.IllegalStateException: begin() called when transaction is OPEN!

 

Spark Streaming从flume 中使用Poll拉取数据时，报如下错误：

 

Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
　　at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
　　at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
　　at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:114)
　　at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:113)
　　at scala.Option.foreach(Option.scala:236)
　　at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)
　　at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)
　　at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)
　　at java.util.concurrent.FutureTask.run(FutureTask.java:262)
　　at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
　　at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
　　at java.lang.Thread.run(Thread.java:745)
2020-04-08 05:20:15,120 (Spark Sink Processor Thread - 10) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Spark was unable to successfully process the events. Transaction is being rolled back.
2020-04-08 05:20:15,120 (New I/O worker #1) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Received an error batch - no events were received from channel!

 

 

报错原因：　　

　　flume的lib目录下的scala-library.xx jar 版本有问题。

解决办法：

  1、将flume的lib目录下的 scala-library-2.10.5.jar 删除或者重命名。

  2、建议上传 scala-library-2.11.x.jar 包。

  如下图：

  3、附上 scala-library-2.11.8.jar ，需要请自提。

　　[百度网盘：]
　　链接：https://pan.baidu.com/s/1GsNwQyt89pobWHWkZO52jw

　　提取码：41ik