【Server】对象存储OSS - Minio
官方文档：
https://docs.min.io/docs/minio-quickstart-guide.html
看中文文档CV命令发现下不下来安装包，应该是地址问题
单击搭建非常简单，只有三个步骤：
wget https://dl.min.io/server/minio/release/linux-amd64/minio chmod +x minio ./minio server /data
若需要后台执行，则这样：
nohup ./minio server /data > /data/minio.log 2>&1 &
这里我使用的是Centos6最终版
[root@centos6-3 ~]# cat /etc/redhat-release CentOS release 6.10 (Final)
发现启动后警告
WARNING: Detected Linux kernel version older than 4.0.0 release, there are some known potential performance problems with this kernel version. MinIO recommends a minimum of 4.x.x linux kernel version for best performance 警告：检测到Linux内核版本早于4.0.0版本，此内核版本存在一些已知的潜在性能问题。MinIO建议至少4个。x、 x linux内核版本以获得最佳性能
Minio还提供了一个访问页面
http://localhost:9000/
账号密码：
minioadmin / minioadmin
下面SDK参考视频：
https://www.bilibili.com/video/BV1wh411B7QC
Java客户端 SDK
需要的依赖：
<!-- https://mvnrepository.com/artifact/io.minio/minio --> <dependency> <groupId>io.minio</groupId> <artifactId>minio</artifactId> <version>8.3.7</version> </dependency> <!-- https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp --> <dependency> <groupId>com.squareup.okhttp3</groupId> <artifactId>okhttp</artifactId> <version>4.8.1</version> </dependency> <!-- https://mvnrepository.com/artifact/me.tongfei/progressbar --> <dependency> <groupId>me.tongfei</groupId> <artifactId>progressbar</artifactId> <version>0.5.3</version> </dependency>
文件上传则需要：
<dependency> <groupId>commons-fileupload</groupId> <artifactId>commons-fileupload</artifactId> <version>1.4</version> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.6</version> </dependency>
上传案例：
这里和官方文档上的已经不太一样了
老版本是使用字符串入参，新版本改用这种建造者模式注入参数
package cn.cloud9.mybatisplusgenerator; import io.minio.BucketExistsArgs; import io.minio.MakeBucketArgs; import io.minio.MinioClient; import io.minio.UploadObjectArgs; import org.junit.jupiter.api.Test; import org.springframework.boot.test.context.SpringBootTest; @SpringBootTest class ApplicationTests { @Test void minioClientTest() throws Exception { // 1、创建Minio客户端 MinioClient minioClient = MinioClient .builder() .endpoint("http://centos6-2:9000") // Minio服务器地址 .credentials("minioadmin", "minioadmin") // 账号 .build(); // 2、创建一个桶 // 桶名称规范 https://blog.csdn.net/y_bccl27/article/details/118411780 String bucketName = "my-bucket"; // 判断是否存在这个桶 final boolean exists = minioClient.bucketExists( BucketExistsArgs .builder() .bucket(bucketName) .build() ); // 不存在则创建这个桶 // 请求时发现报错：The difference between the request time and the server's time is too large // 原因是时间不同步 if (!exists) { minioClient.makeBucket( MakeBucketArgs .builder() .bucket(bucketName) .build() ); } // 文件上传 String fileName = "C:\\Users\\Administrator\\Desktop\\fox.png"; minioClient.uploadObject( UploadObjectArgs .builder() .bucket(bucketName) .object(fileName) // 存储对象名称 .filename(fileName) // 文件名称，直接写文件绝对路径位置 .build() ); } }
文件下载案例：
还是使用之前的文件上传下载，存取都要使用这个object名字
@Test void minioClientDownloadTest() throws Exception { // 第一步拿到客户端 // 1、创建Minio客户端 MinioClient minioClient = MinioClient .builder() .endpoint("http://centos6-2:9000") .credentials("minioadmin", "minioadmin") .build(); String bucketName = "my-bucket"; String fileName = "C:\\Users\\Administrator\\Desktop\\fox.png"; // 2、下载文件 minioClient.downloadObject( DownloadObjectArgs .builder() .bucket(bucketName) .object(fileName) .filename("C:\\Users\\Administrator\\Desktop\\fox-2.png") .build() ); }
SpringBoot整合Minio的SDK
追加MVC配置：
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency>
封装客户端方便调用：
1、封装一个属性配置类：
package cn.cloud9.mybatisplusgenerator.config; import lombok.Data; import lombok.NoArgsConstructor; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.stereotype.Component; @Data @NoArgsConstructor @Component @ConfigurationProperties(prefix = "minio-client") public class MinioClientProperties { private String endpoint; private String accessKey; private String secretKey; private String bucketName; }
2、封装客户端Bean
package cn.cloud9.mybatisplusgenerator.config; import io.minio.MinioClient; import io.minio.MinioProperties; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class MinioClientConfiguration { @Autowired private MinioClientProperties mcp; @Bean public MinioClient minioClient() { return MinioClient .builder() .endpoint(mcp.getEndpoint()) .credentials(mcp.getAccessKey(), mcp.getSecretKey()) .build(); } }
配置文件编写客户端连接信息：
minio-client: endpoint: http://centos6-2:9000 accessKey: minioadmin secretKey: minioadmin bucket-name: my-bucket
编写Minio的Controller
package cn.cloud9.mybatisplusgenerator.controller; import cn.cloud9.mybatisplusgenerator.config.MinioClientProperties; import com.alibaba.fastjson.JSON; import io.minio.*; import io.minio.messages.Item; import org.apache.commons.io.IOUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.*; import org.springframework.web.multipart.MultipartFile; import javax.servlet.http.HttpServletResponse; import java.io.InputStream; import java.net.URLEncoder; import java.nio.charset.StandardCharsets; import java.util.ArrayList; import java.util.Iterator; import java.util.List; @RestController @RequestMapping("/minio") public class MinioController { @Autowired private MinioClient minioClient; @Autowired MinioClientProperties mcp; }
文件下载接口：
/** * /list/download/{fileName} * @param resp * @param fileName * @throws Exception */ @GetMapping("/download/{fileName}") public void download(HttpServletResponse resp, @PathVariable("fileName") String fileName) throws Exception { InputStream inputStream = null; try { // 取出对象信息 final StatObjectResponse statObject = minioClient.statObject( StatObjectArgs .builder() .bucket(mcp.getBucketName()) .object(fileName) .build() ); // 设置下载类型的响应信息 resp.setContentType(statObject.contentType()); resp.setHeader("Content-Disposition", "attachment;filename=" + URLEncoder.encode(fileName, "UTF-8")); // 下载逻辑 // 从客户端获取输入流（获取对象响应） final GetObjectResponse object = minioClient.getObject( GetObjectArgs .builder() .bucket(mcp.getBucketName()) .object(fileName) .build() ); inputStream = object; // 输出文件 IOUtils.copy(inputStream, resp.getOutputStream()); } catch (Exception exception) { exception.printStackTrace(); } finally { assert inputStream != null; inputStream.close(); } }
文件上传接口：
/** * /minio/upload * @param files * @return */ @PostMapping("/upload") public String upload(@RequestParam(name = "file", required = false)MultipartFile[] files) { if (null == files || files.length == 0) return "没有上传的文件！"; final ArrayList<String> objects = new ArrayList<>(files.length); for (MultipartFile mFile : files) { // 先保存名称 final String filename = mFile.getOriginalFilename(); objects.add(filename); // 然后上传 try { InputStream inputStream = mFile.getInputStream(); // 上传可以获取一个输入流 final ObjectWriteResponse objectWriteResponse = minioClient.putObject(PutObjectArgs .builder() .bucket(mcp.getBucketName()) .object(filename) .stream(inputStream, mFile.getSize(), -1) .build() ); final String etag = objectWriteResponse.etag(); System.out.println(etag); inputStream.close(); } catch (Exception exception) { exception.printStackTrace(); return "上传失败！"; } } return "上传成功 " + objects; }
注意上传这里，MVC默认大小限制会报错，需要自己增加配置
package cn.cloud9.mybatisplusgenerator.config; import org.springframework.boot.web.servlet.MultipartConfigFactory; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.util.unit.DataSize; import javax.servlet.MultipartConfigElement; @Configuration public class MultipartConfiguration { /** * 版权声明：本文为CSDN博主「养生编程大队长」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。 * 原文链接：https://blog.csdn.net/qq_27886773/article/details/102695145 * @return */ @Bean public MultipartConfigElement multipartConfigElement() { MultipartConfigFactory factory = new MultipartConfigFactory(); factory.setMaxRequestSize(DataSize.ofGigabytes(1)); factory.setMaxFileSize(DataSize.ofGigabytes(1)); return factory.createMultipartConfig(); } }
Postman测试：
展示文件列表：
/** * /minio/list * @return * @throws Exception */ @GetMapping("/list") public List<String> listBucketObject() throws Exception { final Iterable<Result<Item>> results = minioClient.listObjects( ListObjectsArgs .builder() .bucket(mcp.getBucketName()) .build() ); final Iterator<Result<Item>> iterator = results.iterator(); final ArrayList<String> objects = new ArrayList<>(); while (iterator.hasNext()) { Item item = iterator.next().get(); objects.add(item.objectName() + " " + item.size()); } return objects; }
测试：
删除文件：
@DeleteMapping("/delete/{fileName}")
public String delete(@PathVariable("fileName") String filename) {
try {
minioClient.removeObject(
RemoveObjectArgs
.builder()
.bucket(mcp.getBucketName())
.object(filename)
.build()
);
} catch (Exception exception) {
exception.printStackTrace();
return "删除失败！";
}
return "删除成功";
}
测试：
Docker构建Minio分布式集群
参考文章：
https://blog.csdn.net/networken/article/details/111500745
尝试在Centos6上面安装Docker无果，官方停止6版本的维护，撤掉了所有的相关包资源
只能改用7版本部署
[root@centos7-1 ~]# cat /etc/redhat-release CentOS Linux release 7.9.2009 (Core)
三台机器，每台机器配置1个Minio并挂载两个磁盘空间
192.168.124.31 centos7-1 192.168.124.32 centos7-2 192.168.124.33 centos7-3
配置IP映射和主机名的更改
# 1、配置机器IP映射 每台机器都要执行 cat >> /etc/hosts << EOF 192.168.124.31 centos7-1 192.168.124.32 centos7-2 192.168.124.33 centos7-3 EOF # 2、对应机器IP执行 echo centos7-1 > /etc/hostname echo centos7-2 > /etc/hostname echo centos7-3 > /etc/hostname
一条命令自动安装Docker：
curl -sSL https://get.daocloud.io/docker | sh
直接执行容器运行命令：
# 1号机器 docker run -d --name minio \ --restart=always --net=host \ -e MINIO_ACCESS_KEY=admin \ -e MINIO_SECRET_KEY=admin123 \ -v minio-data1:/data1 \ -v minio-data2:/data2 \ minio/minio server \ --address 192.168.124.31:9000 \ http://centos7-{1...3}/data{1...2} # 2号机器 docker run -d --name minio \ --restart=always --net=host \ -e MINIO_ACCESS_KEY=admin \ -e MINIO_SECRET_KEY=admin123 \ -v minio-data1:/data1 \ -v minio-data2:/data2 \ minio/minio server \ --address 192.168.124.32:9000 \ http://centos7-{1...3}/data{1...2} # 3号机器 docker run -d --name minio \ --restart=always --net=host \ -e MINIO_ACCESS_KEY=admin \ -e MINIO_SECRET_KEY=admin123 \ -v minio-data1:/data1 \ -v minio-data2:/data2 \ minio/minio server \ --address 192.168.124.33:9000 \ http://centos7-{1...3}/data{1...2}
可以访问1号机的Minio的面板，创建一个桶，
访问2号机和3号机的面板看看是否有这个桶
检查是否成功执行，查看minio容器运行日志
[root@centos7-3 ~]# docker logs -f minio WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated. Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD WARNING: Detected Linux kernel version older than 4.0.0 release, there are some known potential performance problems with this kernel version. MinIO recommends a minimum of 4.x.x linux kernel version for best performance Waiting for all MinIO sub-systems to be initialized.. lock acquired Automatically configured API requests per node based on available memory on the system: 19 All MinIO sub-systems initialized successfully Waiting for all MinIO IAM sub-system to be initialized.. lock acquired Status: 6 Online, 0 Offline. API: http://192.168.124.33:9000 Console: http://192.168.124.33:45149 http://172.17.0.1:45149 http://127.0.0.1:45149 Documentation: https://docs.min.io WARNING: Console endpoint is listening on a dynamic port (45149), please use --console-address ":PORT" to choose a static port. Exiting on signal: TERMINATED WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated. Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD WARNING: Detected Linux kernel version older than 4.0.0 release, there are some known potential performance problems with this kernel version. MinIO recommends a minimum of 4.x.x linux kernel version for best performance Waiting for all MinIO sub-systems to be initialized.. lock acquired Automatically configured API requests per node based on available memory on the system: 19 All MinIO sub-systems initialized successfully Waiting for all MinIO IAM sub-system to be initialized.. lock acquired Status: 6 Online, 0 Offline. API: http://192.168.124.33:9000 Console: http://192.168.124.33:35367 http://172.17.0.1:35367 http://127.0.0.1:35367 Documentation: https://docs.min.io WARNING: Console endpoint is listening on a dynamic port (35367), please use --console-address ":PORT" to choose a static port.
翻译后：
警告：不推荐使用MINIO_ACCESS_密钥和MINIO_SECRET_密钥。 请使用MINIO_ROOT_用户和MINIO_ROOT_密码 警告：检测到Linux内核版本早于4.0.0版本，此内核版本存在一些已知的潜在性能问题。MinIO建议至少4个。x、 x linux内核版本以获得最佳性能 正在等待初始化所有MinIO子系统。。获得锁 根据系统上的可用内存自动配置每个节点的API请求：19 所有MinIO子系统均已成功初始化 正在等待初始化所有MinIO IAM子系统。。获得锁 状态：6在线，0离线。 应用程序编程接口：http://192.168.124.33:9000 慰问：http://192.168.124.33:45149 http://172.17.0.1:45149 http://127.0.0.1:45149 文档：https://docs.min.io 警告：控制台端点正在侦听动态端口（45149），请使用--Console address“：port”选择静态端口。 信号退出：终止 警告：不推荐使用MINIO_ACCESS_密钥和MINIO_SECRET_密钥。 请使用MINIO_ROOT_用户和MINIO_ROOT_密码 警告：检测到Linux内核版本早于4.0.0版本，此内核版本存在一些已知的潜在性能问题。MinIO建议至少4个。x、 x linux内核版本以获得最佳性能 正在等待初始化所有MinIO子系统。。获得锁 根据系统上的可用内存自动配置每个节点的API请求：19 所有MinIO子系统均已成功初始化 正在等待初始化所有MinIO IAM子系统。。获得锁 状态：6在线，0离线。 应用程序编程接口：http://192.168.124.33:9000 慰问：http://192.168.124.33:35367 http://172.17.0.1:35367 http://127.0.0.1:35367 文档：https://docs.min.io 警告：控制台端点正在侦听动态端口（35367），请使用--Console address“：port”选择静态端口。
客户端查看minio状态，3个节点及每个节点上的Drivers全部在线：
这里注意，客户端还需要自己下载：
下载参考地址：
https://zhuanlan.zhihu.com/p/39572049
下载安装：
wget https://dl.minio.io/client/mc/release/linux-amd64/mc chmod +x mc
客户端添加一个主机账号配置：
[root@centos7-1 ~]# ./mc config host add minio http://centos7-1:9000 admin admin123 mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials. mc: Successfully created `/root/.mc/share`. mc: Initialized share uploads `/root/.mc/share/uploads.json` file. mc: Initialized share downloads `/root/.mc/share/downloads.json` file. Added `minio` successfully. [root@centos7-1 ~]#
查看集群节点状态：
[root@centos7-1 ~]# ./mc admin info minio ● centos7-1:9000 Uptime: 11 minutes Version: 2022-03-17T06:34:49Z Network: 3/3 OK Drives: 2/2 OK Pool: 1st ● centos7-2:9000 Uptime: 11 minutes Version: 2022-03-17T06:34:49Z Network: 3/3 OK Drives: 2/2 OK Pool: 1st ● centos7-3:9000 Uptime: 11 minutes Version: 2022-03-17T06:34:49Z Network: 3/3 OK Drives: 2/2 OK Pool: 1st 0 B Used, 1 Bucket, 0 Objects 6 drives online, 0 drives offline [root@centos7-1 ~]#
断开其中一个节点（centos7-3）：
docker stop minio
docker rm minio
