高并发架构的搭建（二）

一、Ceph Swift API接口开发

Swift是由Rackspace开发的用来为云计算提供可扩展存储的项目。专注于对象存储， 并提供一套REST风格的Api来访问， 与Ceph强一致性不同， 它是最终一致性。两者都是优秀的开源项目， 并无明显优劣之分，在使用场景上有所不同， 如果是专注于对象存储， 那么可以选择swift即可满足需要， 如果还有块存储要求， 那么选择Ceph更为合适。这里选择Ceph， 因为通过网关可以适配兼容swift api， 同时在数据访问上具有较强的扩展性。

1.1准备工作

创建Swift用户, 用于接口请求认证，在140上执行下面命令

sudo radosgw-admin user create --subuser="cephtester:subtester" -- uid="cephtester" --display-name="cephtester" --key-type=swift --secret="ljx" - 
-access=full

uid 为主用户， subuser为子用户信息， secret指定密钥， 不指定则随机生成， access拥有权限设定,代码中需使用返回信息中的user和secret_key。

 

{
    "user_id": "cephtester",
    "display_name": "cephtester",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [
        {
            "id": "cephtester:subtester",
            "permissions": "full-control"
        }
    ],
    "keys": [],
    "swift_keys": [
        {
            "user": "cephtester:subtester",
            "secret_key": "ljx"
        }
    ],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

 

创建管理员账号：

radosgw-admin user create --uid=mgruser --display-name=mgruser --system

{
    "user_id": "mgruser",
    "display_name": "mgruser",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [],
    "keys": [
        {
            "user": "mgruser",
            "access_key": "KI0XONO70L11Q9XGRE3C",
            "secret_key": "RHkmPPMvkvy10mi4kmJA3vStJe2bmcoArYYsDwn6"
        }
    ],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "system": "true",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

根据生成的access_key与secret_key， 执行：这两个值是上面生成的

 

ceph dashboard set-rgw-api-access-key KI0XONO70L11Q9XGRE3C 
ceph dashboard set-rgw-api-secret-key RHkmPPMvkvy10mi4kmJA3vStJe2bmcoArYYsDwn6

 

打开管理界面，http://192.168.32.140:8443/#/rgw/user 可以查看到我们刚才创建的两个用户：

 

 

1.2、文件服务器搭建

搭建一个单独的Maven工程，专门用于实现文件上传和文件下载，工程坐标如下：

 

 POM.XML

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>spring-cloud-service</artifactId>
        <groupId>com.ghy</groupId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.ghy</groupId>
    <artifactId>spring-cloud-file-service</artifactId>

    <description>
        文件上传服务
    </description>
    <dependencies>
        <!-- Rados Java Api依赖 -->
        <dependency>
            <groupId>com.ceph</groupId>
            <artifactId>rados</artifactId>
            <version>0.6.0</version>
        </dependency>
        <!-- Cephfs 文件系统依赖 -->
        <dependency>
            <groupId>com.ceph</groupId>
            <artifactId>libcephfs</artifactId>
            <version>0.80.5</version>
        </dependency>
        <!--swift-->
        <dependency>
            <groupId>org.javaswift</groupId>
            <artifactId>joss</artifactId>
            <version>0.10.2</version>
        </dependency>
    </dependencies>
</project>

bootstrap.yml

server:
  port: 8082
spring:
  application:
    name: spring-cloud-file-service
  cloud:
    nacos:
      config:
        file-extension: yaml
        server-addr: 192.168.32.197:8848
      discovery:
        #Nacos的注册地址
        server-addr: 192.168.32.197:8848

ceph:
  username: cephtester:subtester  #Ceph配置 主用户名：子用户名
  password: ljx #秘钥
  authUrl: http://192.168.32.195:8443/auth/1.0 #接口访问路径
  defaultContainerName: user_datainfo #默认容器名字(可以自己随便写无所谓)
#图片路径
cephurl: http://localhost:8082/file/download/

#日志配置
logging:
  pattern:
    console: "%msg%n"

接下来创建一个ContainerConfig类来完成配置文件中ceph的初始化操作

@Data
@Configuration
@ConfigurationProperties(prefix = "ceph")
public class ContainerConfig {

    private String username;
    private String password;
    private String authUrl;
    private String defaultContainerName;


    /*****
     * 1、创建账号信息
     */
    @Bean
    public Account account(){
        AccountConfig config = new AccountConfig();
        config.setUsername(username);
        config.setPassword(password);
        config.setAuthUrl(authUrl);
        config.setAuthenticationMethod(AuthenticationMethod.BASIC);
        return new AccountFactory(config).createAccount();
    }


    /*****
     * 2、创建容器对象
     */
    @Bean
    public Container container(){
        Container container = account().getContainer(defaultContainerName);
        if(!container.exists()){
            return  container.create();
        }
        return container;
    }

}

创建文件上传下载工具类FileHandler ，代码如下：

@Component
public class FileHandler {

    @Autowired
    private Container container;

    /****
     * 文件上传
     */
    public void upload(String filename,byte[] buffer){
        //获取容器
        StoredObject object = container.getObject(filename);
        //文件上传
        object.uploadObject(buffer);
    }


    /***
     * 文件下载
     */
    public byte[] download(String filename){
        //获取容器
        StoredObject object = container.getObject(filename);
        //下载
        byte[] bytes = object.downloadObject();
        return bytes;
    }
}

控制器创建：FileController类

@RestController
@RequestMapping(value = "/file")
public class FileController {

    @Autowired
    private FileHandler fileHandler;

    /****
     * 文件上传
     * @param file
     * @return
     */
    @PostMapping(value = "/upload")
    public RespResult upload(MultipartFile file) throws IOException {
        //调用
        fileHandler.upload(file.getOriginalFilename(),file.getBytes());
        return RespResult.ok();
    }

    /****
     * 文件下载
     * @return
     */
    @GetMapping(value = "/download/{filename}")
    public void download(@PathVariable String filename, HttpServletResponse response) throws IOException {
        //调用
        byte[] buffer = fileHandler.download(filename);
        ServletOutputStream os = response.getOutputStream();
        os.write(buffer);
        os.flush();
        os.close();
    }
}

创建启动类：SpringCloudFileServiceApplication

@SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
public class SpringCloudFileServiceApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringCloudFileServiceApplication.class,args);
    }
}

文件上传测试http://localhost:8082/file/upload

 

 

文件下载测试：http://localhost:8082/file/download/1624446315(1).jpg

 

 git源码：https://gitee.com/TongHuaShuShuoWoDeJieJu/spring-cloud-alibaba1.git