easy-batch 学习试用
easy-rules 是一个简单，但是封装比较完备的java etl 框架，提供了比较完备的数据处理能力
是一个可以使用的数据处理框架,以下是一个简单的学习
项目说明
项目是官方的一个demo，主要是学习下easy-rules 的基本使用,主要是转换一个csv文件为xml
项目准备
- 项目结构
├── pom.xml
├── src
│ ├── main
│ │ ├── java
│ │ │ └── com
│ │ │ └── dalong
│ │ │ ├── Application.java
│ │ │ └── Tweet.java
│ │ └── resources
│ │ └── tweets.csv
│ └── test
│ └── java
- pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.dalongapp</groupId>
<artifactId>easy-batch-app</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>7</source>
<target>7</target>
</configuration>
</plugin>
</plugins>
</build>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.jeasy</groupId>
<artifactId>easy-batch-core</artifactId>
<version>6.0.0</version>
</dependency>
<dependency>
<groupId>org.jeasy</groupId>
<artifactId>easy-batch-flatfile</artifactId>
<version>6.0.0</version>
</dependency>
<dependency>
<groupId>org.jeasy</groupId>
<artifactId>easy-batch-xml</artifactId>
<version>6.0.0</version>
</dependency>
</dependencies>
</project>
- 代码说明
easy-batch的使用还是比较清晰简单的，我们只需要关注业务逻辑代码，复杂的都交给easy-batch处理
Application.java
package com.dalong;
import org.jeasy.batch.core.filter.HeaderRecordFilter;
import org.jeasy.batch.core.job.Job;
import org.jeasy.batch.core.job.JobBuilder;
import org.jeasy.batch.core.job.JobExecutor;
import org.jeasy.batch.core.job.JobReport;
import org.jeasy.batch.core.writer.FileRecordWriter;
import org.jeasy.batch.flatfile.DelimitedRecordMapper;
import org.jeasy.batch.flatfile.FlatFileRecordReader;
import org.jeasy.batch.xml.XmlRecordMarshaller;
import javax.xml.bind.JAXBException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
public class Application {
public static void main(String[] args) throws JAXBException {
Path inputFile = Paths.get(Application.class.getClassLoader().getResource("tweets.csv").getFile());
Path outputFile = Paths.get("tweets.xml");
Job job = new JobBuilder()
.reader(new FlatFileRecordReader(inputFile))
.filter(new HeaderRecordFilter())
.mapper(new DelimitedRecordMapper(Tweet.class, "id", "user", "message"))
.marshaller(new XmlRecordMarshaller(Tweet.class))
.writer(new FileRecordWriter(outputFile))
.batchSize(10)
.build();
JobExecutor jobExecutor = new JobExecutor();
JobReport report = jobExecutor.execute(job);
System.out.println(report.toString());
jobExecutor.shutdown();
}
}
Tweet.java
package com.dalong;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlList;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement(name = "tweets")
public class Tweet {
private int id;
public int getId() {
return id;
}
@Override
public String toString() {
return "Tweet{" +
"id=" + id +
", user='" + user + '\'' +
", message='" + message + '\'' +
'}';
}
public void setId(int id) {
this.id = id;
}
public String getUser() {
return user;
}
public void setUser(String user) {
this.user = user;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
private String user;
private String message;
}
csv 文件
tweets.csv
id,user,message
1,foo,hello
2,bar,@foo hi!
运行效果
- 运行
启动main - 效果
easy-batch参考运行
参考资料
https://github.com/j-easy/easy-batch
https://github.com/j-easy/easy-batch/wiki
