在SpringBoot当中集合ElasticSearch进行搜索

前言

在学习easyLive项目当中接触到了elasticSearch中间件来进行搜索.本随笔记录一下项目中用到的操作. 这里就省略掉一些es的基础介绍和为什么要使用es了.

1. 准备阶段

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

在这里使用的是SpringData集合的elasticSearch,初始化方法和直接引入highLevelRestClient有所区别(新版本已经废弃掉这个方法了,现在还没学).
创建Config类,这里我们可以参考SpringData文档 中的操作来对es初始化,创建一个Config类
EsConfiguration

@Configuration
public class EsConfiguration extends AbstractElasticsearchConfiguration implements DisposableBean {
 
 
    private RestHighLevelClient client;
 
    @Resource
    private AppConfig appConfig;
 
    @Override
    public void destroy() throws Exception {
        if (client != null) {
            client.close();
        }
    }
 
    @Override
    public RestHighLevelClient elasticsearchClient() {
        final ClientConfiguration clientConfiguration = ClientConfiguration.builder()
                .connectedTo(appConfig.getEsHostPort())
                .build();
        client = RestClients.create(clientConfiguration).rest();
        return client;
    }
}

AppConfig类

//省去了getter和setter方法来节省空间,之后的随笔也是这样
@Configuration
public class AppConfig {
 
    @Value("${project.folder:}")
    private String projectFolder;
 
    @Value("${admin.account:}")
    private String adminAccount;
 
    @Value("${admin.password:}")
    private String adminPassword;
 
    @Value("${showFFmpegLog:true}")
    private boolean showFFmpegLog;
 
//这里还没琢磨过有没有AutoConfig类能直接在application.yml里直接配置,之后再看吧,其实这里写yml配置文件里更好,这样子docker部署的时候可以通过外置挂载卷来修改es地址
    @Value("${es.host.port:127.0.0.1:9200}")
    private String esHostPort;
 
    @Value("${ex.index.video.name:easylive_video}")
    private String esIndexVideoName;
}

在每次启动的时候初始化es而不是在es里好了再找,这样子在换运行环境的时候可以方便很多
所以我们创建一个esComponent

@Component("esSearchComponent")
public class EsSearchComponent {
    @Resource
    private AppConfig appConfig;
 
    @Resource
    private RestHighLevelClient restHighLevelClient;
 
    private Boolean isExsitIndex() throws IOException {
        GetIndexRequest getIndexRequest = new GetIndexRequest(appConfig.getEsIndexVideoName());
        return restHighLevelClient.indices().exists(getIndexRequest, RequestOptions.DEFAULT);
    }
 
    public void createIndex() {
        try {
            if (isExsitIndex()) {
                return;
            }
 
            CreateIndexRequest request = new CreateIndexRequest(appConfig.getEsIndexVideoName());
            request.settings("{\"analysis\": {\n" +
                    "      \"analyzer\": {\n" +
                    "       \"comma\": {\n" +
                    "         \"type\": \"pattern\",\n" +
                    "         \"pattern\": \",\"\n" +
                    "       }\n" +
                    "      }\n" +
                    "     }}", XContentType.JSON);
            request.mapping("{\"properties\": {\n" +
                    "      \"videoName\":{\n" +
                    "        \"type\": \"text\",\n" +
                    "        \"analyzer\": \"ik_max_word\"\n" +
                    "      },\n" +
                    "      \"tags\":{\n" +
                    "        \"type\": \"text\",\n" +
                    "        \"analyzer\": \"comma\"\n" +
                    "      },\n" +
                    "      \"playCount\":{\n" +
                    "        \"type\":\"integer\",\n" +
                    "        \"index\":false\n" +
                    "      },\n" +
                    "      \"danmuCount\":{\n" +
                    "        \"type\":\"integer\",\n" +
                    "        \"index\":false\n" +
                    "      },\n" +
                    "      \"collectCount\":{\n" +
                    "        \"type\":\"integer\",\n" +
                    "        \"index\":false\n" +
                    "      },\n" +
                    "      \"createTime\":{\n" +
                    "        \"type\":\"date\",\n" +
                    "        \"format\": \"yyyy-MM-dd HH:mm:ss\",\n" +
                    "        \"index\": false\n" +
                    "      },\n" +
                    "      \"videoId\":{\n" +
                    "        \"type\":\"text\",\n" +
                    "        \"index\": false\n" +
                    "      }\n" +
                    "   }" +
                    "}", XContentType.JSON);
 
            CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);
            Boolean acknowledged = createIndexResponse.isAcknowledged();
 
            if (!acknowledged) {
                throw new BusinessException("初始化ES失败");
            }
        } catch (IOException e) {
            log.error("初始化ES失败", e);
            throw new BusinessException("初始化ES失败");
        }
    }
}

因为Es是Docs存储的,所以这里的配置得用JS格式来发送请求到esClient,这里其实就是创建视频的搜索信息,约束和分词器.
在创建号Component之后启动SpringBoot程序,用一个实现了ApplicationRunner接口的类实现run回调函数

public void run(ApplicationArguments args) throws Exception {
        esSearchComponent.createIndex();
    }

一切正常没有报错的话就进去下一步

2.使用阶段

在此之前,我们需要创建一个和es里docs存储mapping一致的DTO类,因为当有多余属性的话,es就会自动添加一些属性进去,虽然目前没什么问题,但这是预期之外的为了防止问题就先解决掉.

public class VideoInfoEsDto {
 
    /**
     * 视频ID
     */
    private String videoId;
 
    /**
     * 视频封面
     */
    private String videoCover;
 
    /**
     * 视频名称
     */
    private String videoName;
 
    /**
     * 用户ID
     */
    private String userId;
 
    /**
     * 创建时间
     */
    @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss", timezone = "GMT+8")
    @JSONField(format = "yyyy-MM-dd HH:mm:ss")
    private Date createTime;
 
    /**
     * 标签
     */
    private String tags;
 
    /**
     * 播放数量
     */
    private Integer playCount;
 
    /**
     * 弹幕数量
     */
    private Integer danmuCount;
 
    /**
     * 收藏数量
     */
    private Integer collectCount;

这里需要注意一点的就是createTime上的注解JsonFormat是Jackson的格式化,我们用的是fastJson所以得使用fastJson的格式化注解@JsonField 不然没有序列化进去就不是我们预期的效果了

1.保存对象Dto到es当中

在我们的后台审核视频业务当中,我们在通过视频审核之后就要将VideoInfo保存到es当中,在auditVideo方法的最后才使用saveDocs方法

这是考虑到事务的问题,因为es不支持Transactional的rollback,也就是说当es里写入成功之后如果后面抛出异常的话那es里的信息其实是不会回滚的

public void saveDoc(VideoInfo videoInfo) {
        VideoInfoEsDto videoInfoEsDto = CopyTools.copy(videoInfo, VideoInfoEsDto.class);
        try {
            if (docExist(videoInfo.getVideoId())) {
                updateDoc(videoInfoEsDto);
            } else {
                insertDoc(videoInfoEsDto);
            }
 
        } catch (IOException e) {
            log.error("保存到Es失败", e);
            throw new BusinessException("保存到Es失败");
        }
    }

private void updateDoc(VideoInfoEsDto videoInfoEsDto) throws IOException {
        try {
            videoInfoEsDto.setCreateTime(null);
 
            HashMap<String, Object> dataMap = new HashMap<>();
            Field[] fields = videoInfoEsDto.getClass().getDeclaredFields();
            for (Field field : fields) {
 
                String methodName = "get" + StringTools.upperCaseFirstLetter(field.getName());
                Method method = videoInfoEsDto.getClass().getMethod(methodName);
                Object invoke = method.invoke(videoInfoEsDto);
 
                if (Objects.nonNull(invoke) && invoke instanceof String && !StringTools.isEmpty(invoke.toString()) || Objects.nonNull(invoke) && !(invoke instanceof String)) {
                    dataMap.put(field.getName(), invoke);
                }
                if (dataMap.isEmpty()) {
                    return;
                }
                UpdateRequest updateRequest = new UpdateRequest(appConfig.getEsIndexVideoName(), videoInfoEsDto.getVideoId());
                updateRequest.doc(dataMap);
                restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
            }
        } catch (Exception e) {
            log.error("保存视频失败", e);
            throw new BusinessException("保存视频失败");
        }
    }

private void insertDoc(VideoInfoEsDto videoInfoEsDto) throws IOException {
        videoInfoEsDto.setCollectCount(0);
        videoInfoEsDto.setDanmuCount(0);
        videoInfoEsDto.setPlayCount(0);
 
        IndexRequest indexRequest = new IndexRequest(appConfig.getEsIndexVideoName());
        indexRequest.id(videoInfoEsDto.getVideoId()).source(JsonUtils.convertObj2Json(videoInfoEsDto), XContentType.JSON);
 
        restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
    }

 private Boolean docExist(String id) throws IOException {
        GetRequest getIndexRequest = new GetRequest(appConfig.getEsIndexVideoName(), id);
        GetResponse getResponse = restHighLevelClient.get(getIndexRequest, RequestOptions.DEFAULT);
        return getResponse.isExists();
    }

2. 更新数量到ES

public void updateDocCount(String videoId, String fieldName, Integer count) {
        try {
            UpdateRequest updateRequest = new UpdateRequest(appConfig.getEsIndexVideoName(), videoId);
//这里类似于SQL里的行锁,来防止高并发的情况下导致数据出现错误
            Script script = new Script(ScriptType.INLINE, "painless", "ctx._source." + fieldName + " += params.count", Collections.singletonMap("count", count));
            updateRequest.script(script);
            restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);
        } catch (Exception e) {
            log.error("保存视频失败", e);
            throw new BusinessException("保存视频失败");
        }
    }

3. 从es里删除视频

public void delDoc(String videoId) {
        try {
            DeleteRequest deleteRequest = new DeleteRequest(appConfig.getEsIndexVideoName(), videoId);
            restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
        } catch (Exception e) {
            log.error("删除视频失败", e);
            throw new BusinessException("删除视频失败");
        }
    }

4. 搜索

public PaginationResultVO<VideoInfo> search(Boolean highLight, String keyWord, Integer orderType, Integer pageNo, Integer pageSize) {
        try {
            SearchOrderTypeEnum searchOrderTypeEnum = SearchOrderTypeEnum.getByType(orderType);
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            searchSourceBuilder.query(QueryBuilders.multiMatchQuery(keyWord, "videoName", "tags"));
//        高亮
            if (highLight) {
                HighlightBuilder highlightBuilder = new HighlightBuilder();
                highlightBuilder.field("videoName");
//            <span class = 'highlight'> { 关键字 } </span>
                highlightBuilder.preTags("<span class = 'highlight'>");
                highlightBuilder.postTags("</span>");
                searchSourceBuilder.highlighter(highlightBuilder);
            }
//        排序
            searchSourceBuilder.sort("_score", SortOrder.ASC);
            if (Objects.nonNull(orderType)) {
                searchSourceBuilder.sort(searchOrderTypeEnum.getField(), SortOrder.DESC);
            }
 
            pageNo = pageNo == null ? Constants.ONE : pageNo;
            pageSize = pageSize == null ? PageSize.SIZE20.getSize() : pageSize;
            searchSourceBuilder.size(pageSize);
            searchSourceBuilder.from((pageNo - 1) * pageSize);
 
            SearchRequest searchRequest = new SearchRequest(appConfig.getEsIndexVideoName());
            searchRequest.source(searchSourceBuilder);
            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
 
            SearchHits searchHits = searchResponse.getHits();
            Integer totalCount = (int) searchHits.getTotalHits().value;
 
            List<VideoInfo> videoInfoList = new ArrayList<>();
 
            List<String> userIdList = new ArrayList<>();
 
            for (SearchHit searchHit : searchHits.getHits()) {
                VideoInfo videoInfo = JsonUtils.convertJson2Obj(searchHit.getSourceAsString(), VideoInfo.class);
                if (Objects.nonNull(searchHit.getHighlightFields().get("videoName"))) {
                    videoInfo.setVideoName(searchHit.getHighlightFields().get("videoName").fragments()[0].string());
                }
                videoInfoList.add(videoInfo);
                userIdList.add(videoInfo.getUserId());
            }
 
            UserInfoQuery userInfoQuery = new UserInfoQuery();
            userInfoQuery.setUserIdList(userIdList);
            List<UserInfo> userInfoList = userInfoMapper.selectList(userInfoQuery);
 
            Map<String, UserInfo> userInfoMap = userInfoList.stream()
                    .collect(Collectors.toMap(item -> item.getUserId(), Function.identity(), (data1, data2) -> data2));
 
            videoInfoList.forEach(item -> {
                item.setNickName(userInfoMap.get(item.getUserId()).getNickName());
            });
 
            SimplePage simplePage = new SimplePage(pageNo, totalCount, pageSize);
            PaginationResultVO<VideoInfo> resultVO = new PaginationResultVO<>(totalCount, simplePage.getPageSize(), simplePage.getPageNo(), simplePage.getPageTotal(), videoInfoList);
            return resultVO;
        } catch (IOException e) {
            log.error("视频搜索失败", e);
            throw new BusinessException("视频搜索失败");
        }
    }

3. 总结

在这段学习的时间里查看文档了解了一些elasticSearch的API方法,以及最重要的一句话:"注释不是让你写每行代码逐字逐句的去解释,而是让你一眼就明白这代码的业务功能".这个问题我再之前的filePost文件上传的时候犯了一次,因为我逐字逐句的解释每行代码的功能从而导致后续找Bug的时候阅读到那段代码及其吃力,因为我现在已经有了阅读代码的能力不需要注释这么多荣誉的注解,有这个时间不如提高一下代码的可阅读性.