JAVA大文件分片上传示例，断点续传思路
分片上传
就是前端把file对象切片一点一点的上传，后端把文件一点一点的保存，要么前端发送完毕发送合并请求要么后端判断通过chunk和chunks的关系是否上传完毕再进行合并，为什么不边上传不边合并？，如果网络中断，合并断开引起的问题需要斟酌一下，还要确实不要把一个接口方法写的太多了，一个方法就做一个功能，后续的修改和优化也方便。
依赖
<dependencies>
<!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.47</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.3</version>
</dependency>
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.8.20</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>transmittable-thread-local</artifactId>
<version>2.14.2</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.8.20</version>
</dependency>
</dependencies>
yaml配置文件
spring:
servlet:
multipart:
# 最大请求内存
max-request-size: 8000MB
# springboot临时文件存放的文职
location: E:\\es\\
# 最大的文件内存
max-file-size: 8000MB
# file-size-threshold: 100MB
public class Result<T> {
private int code;
private String message;
private T data;
public Result setCode(Integer code) {
this.code = code;
return this;
}
public int getCode() {
return code;
}
public String getMessage() {
return message;
}
public Result setMessage(String message) {
this.message = message;
return this;
}
public T getData() {
return data;
}
public Result setData(T data) {
this.data = data;
return this;
}
@Override
public String toString() {
return JSON.toJSONString(this);
}
public static <T> Result<T> fail(Integer code,T data) {
Result<T> ret = new Result<T>();
ret.setCode(code);
ret.setData(data);
return ret;
}
public static <T> Result<T> failMessage(Integer code,String msg) {
Result<T> ret = new Result<T>();
ret.setCode(code);
ret.setMessage(msg);
return ret;
}
public static <T> Result<T> successMessage(Integer code,String msg) {
Result<T> ret = new Result<T>();
ret.setCode(code);
ret.setMessage(msg);
return ret;
}
public static <T> Result<T> success(Integer code,T data) {
Result<T> ret = new Result<T>();
ret.setCode(code);
ret.setData(data);
return ret;
}
}
controller
相关重要参数说明，chunk是第几片，chunks是分了多少片
/**
* @author luYuHan
* @date 2023/8/5 22:26
*/
@CrossOrigin
@Controller
@RequestMapping("/file/upload")
public class MyFileUploadController {
@PostMapping("/part")
@ResponseBody
public Result bigFile(HttpServletRequest request, String guid, Integer chunk, MultipartFile file, Integer chunks) {
try {
String projectUrl = System.getProperty("user.dir").replaceAll("\\\\", "/");
boolean isMultipart = ServletFileUpload.isMultipartContent(request);
if (isMultipart){
if (chunk==null) chunk=0;
// 临时目录用来存放所有分片文件
String tempFileDir = projectUrl + "/upload/"+guid;
File parentFileDir = new File(tempFileDir);
if (!parentFileDir.exists()) {
parentFileDir.mkdirs();
}
// 存放临时文件
FileUtils.copyInputStreamToFile(file.getInputStream(), new File(parentFileDir,guid+"_chunk"+chunk+".part"));
}
} catch (IOException e) {
Result.failMessage(400,"上传失败");
}
return Result.success(200,"上传成功");
}
@RequestMapping("merge")
@ResponseBody
public Result mergeFile(String guid, String fileName) {
try {
// 得到 destTempFile 就是最终的文件
String projectUrl = System.getProperty("user.dir").replaceAll("\\\\", "/");
String suffixName = fileName.substring(fileName.lastIndexOf("."));
String finaFileName = projectUrl + "/upload/" + UUID.randomUUID() + suffixName;
String temFileDir = projectUrl + "/upload/" + guid;
File finalFile = new File(finaFileName);
File allfile = new File(temFileDir);
for (File file : Objects.requireNonNull(allfile.listFiles())) {
FileOutputStream fileOutputStream = new FileOutputStream(finalFile,true);
FileUtils.copyFile(file,fileOutputStream);
fileOutputStream.close();
}
} catch (IOException e) {
return Result.failMessage(400,"合并失败");
}
return Result.successMessage(200,"合并成功");
}
}
前端前置准备
简单去看一下这个文档 - Web Uploader (baidu.com)
前端相关文件，这个webuploader的文件夹就是在上面链接官网中去下载的，其他的除了html都会有
前端web
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link href="webuploader.css" rel="stylesheet" type="text/css" />
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="webuploader.min.js"></script>
</head>
<body>
<div id="uploader">
<div class="btns">
<div id="picker">选择文件</div>
<button id="startBtn" class="btn btn-default">开始上传</button>
</div>
</div>
</body>
<script type="text/javascript">
var GUID = WebUploader.Base.guid();//一个GUID
var uploader = WebUploader.create({
// swf文件路径
swf: 'dist/Uploader.swf',
// 文件接收服务端。
server: 'http://localhost:8080/file/upload/part',
formData:{
guid : GUID
},
pick: '#picker',
chunked : true, // 分片处理
chunkSize : 1 * 1024 * 1024, // 每片1M,
chunkRetry : false,// 如果失败，则不重试
threads : 1,// 上传并发数。允许同时最大上传进程数。
resize: false
});
$("#startBtn").click(function () {
uploader.upload();
});
//当文件上传成功时触发。
uploader.on( "uploadSuccess", function( file ) {
$.post('http://localhost:8080/file/upload/merge', { guid: GUID, fileName: file.name}, function (data) {
if(data.code == 200){
alert('上传成功!');
}
});
});
</script>
</html>
前端请求网络，请注意chunk和chunks的关系
上传成功
断点续传
关于断点上传，其实可以思考一下不难，就是重新上传要保证上传的文件有唯一的文件夹路径并且要把对应的文件夹下保证有所有的分片的文件然后合并到对应文件中即可。可以设置文件上传的临时文件夹location，然后通过切片的片数进行判断我们从多少片开始缺失，当片数到达需求点就可以把临时文件放在搬运的待合并的文件夹下，然后在发送合并请求即可。
参考文章：http://blog.ncmem.com/wordpress/2023/10/12/java%e5%a4%a7%e6%96%87%e4%bb%b6%e5%88%86%e7%89%87%e4%b8%8a%e4%bc%a0%e7%a4%ba%e4%be%8b%ef%bc%8c%e6%96%ad%e7%82%b9%e7%bb%ad%e4%bc%a0%e6%80%9d%e8%b7%af/
欢迎入群一起讨论
