spring集成tus的简单总结
spring集成tus的简单总结
背景
项目需要一个可靠、稳定的页面大文件上传实现，故选择了tus。
一、tus
- tus协议是一个基于http的断点续传的开放协议。
- 它提供了多种功能，有很好的官方和非官方实现示例，并且支持多种语言。
- 官方：https://tus.io/
核心协议
下面列出的为核心协议，其余可选的协议可点击此处
| 标头 | 简介 |
|---|---|
| Upload-Offset | 当前传输的偏移量 |
| Upload-Length | 文件总大小 |
| Tus-Version | 支持的协议版本列表 |
| Tus-Resumable | 当前使用的协议版本 |
| Tus-Extension | 服务器支持的扩展列表 |
| Tus-Max-Size | 允许的文件大小 |
| X-HTTP-Method-Override | http 方法覆盖 |
-
创建上传资源
- 首次上传时创建上传资源
请求头
POST /files HTTP/1.1 Host: tus.example.org Content-Length: 0 Upload-Length: 100 Tus-Resumable: 1.0.0 Upload-Metadata: filename d29ybGRfZG9taW5hdGlvbl9wbGFuLnBkZg==,is_confidential响应
HTTP/1.1 201 Created Location: https://tus.example.org/files/24e533e02ec3bc40c387f1a0e460e216 Tus-Resumable: 1.0.0 -
偏移量查询
请求头
- 检查由 HEAD 方法请求中断的上传的偏移量
HEAD /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1 Host: tus.example.org Tus-Resumable: 1.0.0响应
- 如果没有中断上传，则返回 404 响应
HTTP/1.1 200 OK Upload-Offset: 70 Tus-Resumable: 1.0.0 -
继传
请求头
- 使用 PATCH 方法请求恢复上传
PATCH /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1 Host: tus.example.org Content-Type: application/offset+octet-stream Content-Length: 30 Upload-Offset: 70 Tus-Resumable: 1.0.0响应
HTTP/1.1 204 No Content Tus-Resumable: 1.0.0 Upload-Offset: 100 -
检查服务器配置信息
请求头
OPTIONS /files HTTP/1.1 Host: tus.example.org响应
HTTP/1.1 204 No Content Tus-Resumable: 1.0.0 Tus-Version: 1.0.0,0.2.2,0.2.1 Tus-Max-Size: 1073741824 Tus-Extension: creation,expiration
二、库的选择以及使用
- https://github.com/tomdesair/tus-java-server
- 注意下面的代码需要适配你自己的项目，无法直接拿来使用
添加依赖项
<dependency>
<groupId>me.desair.tus</groupId>
<artifactId>tus-java-server</artifactId>
<version>1.0.0-2.1</version>
</dependency>
配置
| 配置项 | 简介 |
|---|---|
| withUploadURI | 用作上传端点的 uri |
| withMaxUploadSize | 上传的最大字节数（默认Long.MAX_VALUE） |
| withStoragePath | 存放上传信息的路径 |
| withChunkedTransferDecoding | 是否启用分块上传（默认为false） |
| withThreadLocalCache | 是否启用ThreadLocal缓存（默认为false） |
| withUploadExpirationPeriod | 过期时间（毫秒） |
@Configuration
@Slf4j
public class TusConfig {
@Value("${store.path}")
private String tusDataPath;
@Value("${upload.expiration.period}")
private Long expirationPeriod;
@PreDestroy
public void exit() throws IOException {
// cleanup any expired uploads and stale locks
tusFileUploadService().cleanup();
}
@Bean
public TusFileUploadService tusFileUploadService() {
return new TusFileUploadService()
.withStoragePath(tusDataPath + "/tus")
.withUploadURI("/__URL__")
.withThreadLocalCache(true)
//停止某个extension
.disableTusExtension("creation")
//添加自定义的extension实现
.addTusExtension(new MyCreationExtension())
.addTusExtension(new AddPathExtension(tusDataPath))
.withUploadExpirationPeriod(expirationPeriod * 1000 * 60 * 60 * 24);
}
}
控制器
@RestController
@RequestMapping(value = "/__URL__")
@Slf4j
@Api(tags = "文件上传管理")
@CrossOrigin(exposedHeaders = {"Location", "Upload-Offset", "Upload-Length"})
public class FileUploadController {
@Value("${store.path}")
private String storePath;
@Resource
private TusFileUploadService tusFileUploadService;
@ApiOperation("文件上传")
@RequestMapping(value = {"/**"}, method = {RequestMethod.POST, RequestMethod.PATCH, RequestMethod.HEAD,
RequestMethod.DELETE, RequestMethod.OPTIONS, RequestMethod.GET})
public ResponseEntity<String> processUpload(final HttpServletRequest servletRequest, final HttpServletResponse servletResponse) {
log.info("processUpload------start");
try {
tusFileUploadService.process(servletRequest, servletResponse);
} catch (Exception e) {
return ResponseEntity.badRequest();
}
//access response header Location,Upload-Offset,Upload-length
servletResponse.addHeader(HttpHeaders.ACCESS_CONTROL_EXPOSE_HEADERS, "Location,Upload-Offset,Upload-Length");
String uploadUrl = servletRequest.getRequestURI();
log.info("uploadUrl:{}", uploadUrl);
UploadInfo info;
try {
info = this.tusFileUploadService.getUploadInfo(uploadUrl);
} catch (IOException | TusException e) {
log.error("get upload info", e);
return ResponseEntity.ok();
}
Path tusDataPath = Paths.get(storePath).resolve("tus");
//上传完成后，将上传完成的data文件重命名为原文件
if (info != null && !info.isUploadInProgress()) {
Path parPath = tusDataPath.resolve("uploads").resolve(info.getId().getOriginalObject().toString());
Path filePath = parPath.resolve(info.getFileName());
try (InputStream uploadedBytes = tusFileUploadService.getUploadedBytes(uploadUrl, null)) {
FileUtils.copyToFile(uploadedBytes, filePath.toFile());
} catch (Exception e) {
try {
this.tusFileUploadService.deleteUpload(uploadUrl);
} catch (IOException | TusException ee) {
log.error("delete upload", ee);
return ResponseEntity.badRequest().body("删除错误的文件失败");
}
return ResponseEntity.badRequest().body("重命名文件失败");
}
}
return ResponseEntity.ok();
}
}
定时任务
tusFileUploadService.cleanup()通过该方法可以轻松清理停止上传的数据，以及超过config中设置的过期时间的数据。
创建定时任务以定期删除过期数据以保护服务器空间。
@Service
@Slf4j
public class FileOptSchedule {
@Value("${store.path}")
private String storePath;
@Resource
private TusFileUploadService tusFileUploadService;
@Scheduled(fixedDelayString = "PT24H")
public void cleanup() {
Path path = new File(storePath).toPath();
Path locksDir = path.resolve("locks");
if (Files.exists(locksDir)) {
try {
tusFileUploadService.cleanup();
} catch (IOException e) {
log.error("清理文件上传目录:", e);
}
}
}
}
自定义extension
-
将文件地址存储在uploadInfo中
@Slf4j public class AddPathExtension extends AbstractTusExtension { private final Path tusDataPath; public AddPathExtension(String path) { this.tusDataPath = Paths.get(path); } @Override public String getName() { return "addPath"; } @Override public Collection<HttpMethod> getMinimalSupportedHttpMethods() { return Collections.singletonList(HttpMethod.PATCH); } @Override protected void initValidators(List<RequestValidator> requestValidators) { } @Override protected void initRequestHandlers(List<RequestHandler> requestHandlers) { requestHandlers.add(new AbstractExtensionRequestHandler() { @Override protected void appendExtensions(StringBuilder extensionBuilder) { addExtension(extensionBuilder, "addPath"); } }); requestHandlers.add(new AbstractRequestHandler() { @Override public boolean supports(HttpMethod method) { return HttpMethod.PATCH.equals(method); } @Override public void process(HttpMethod method, TusServletRequest servletRequest, TusServletResponse servletResponse, UploadStorageService uploadStorageService, String ownerKey) throws IOException, TusException { log.info("AddPathRequestHandlers======start"); String uploadUrl = servletRequest.getRequestURI(); UploadInfo info = uploadStorageService.getUploadInfo(uploadUrl, ownerKey); if (info == null) { throw new UploadNotFoundException("Upload " + uploadUrl + " is not found"); } else if (!info.isUploadInProgress()) { String encodedMetadata = info.getEncodedMetadata(); Path parPath = tusDataPath.resolve("tus").resolve("uploads").resolve(info.getId().getOriginalObject().toString()); Path filePath = parPath.resolve(info.getFileName()); encodedMetadata = encodedMetadata + ",filePath " + new String(Base64.encodeBase64(filePath.toString().getBytes(StandardCharsets.UTF_8)), Charsets.UTF_8); info.setEncodedMetadata(encodedMetadata); uploadStorageService.update(info); } } }); } } -
通过网关转发的url和后端获取的url不一致，导致无法正常上传
解决方法是重写CreationExtension，扩展一下CreationExtension的initRequestHandlers的CreationPostRequestHandler即可
@Override public void process(HttpMethod method, TusServletRequest servletRequest, TusServletResponse servletResponse, UploadStorageService uploadStorageService, String ownerKey) throws IOException { UploadInfo info = buildUploadInfo(servletRequest); info = uploadStorageService.create(info, ownerKey); //We've already validated that the current request URL matches our upload URL so we can safely use it. String uploadURI = servletRequest.getRequestURI(); //It's important to return relative UPLOAD URLs in the Location header in order to support HTTPS proxies //that sit in front of the web app String url = "这里是你自己的服务地址" + uploadURI + (StringUtils.endsWith(uploadURI, "/") ? "" : "/") + info.getId(); servletResponse.setHeader(HttpHeader.LOCATION, url); servletResponse.setStatus(HttpServletResponse.SC_CREATED); log.info("Created upload with ID {} at {} for ip address {} with location {}", info.getId(), info.getCreationTimestamp(), info.getCreatorIpAddresses(), url); }
其他的一些方法
{
//获取上传的文件详情
UploadInfo uploadInfo = tusFileUploadService.getUploadInfo(uploadUrl);
//删除指定文件
tusFileUploadService.deleteUpload(uploadUrl);
}
