在Fitnesse用例中获取ELK日志
概述
在执行一条隔了几个项目的全流程的接口自动化用例时，往往会遇到一个问题。
如果用例执行失败，该从哪个服务的日志开始定位问题。
以往的解决办法：回忆一段时间，理清业务流程后，才开始逐个对服务的日志进行排查。
这个办法没有问题，但是很耗时。
新方法
现在很多公司都是用ELK来查看日志。它也提供了sdk，来设计自己的检索功能。
我这里就利用了ELK的sdk新建了一个服务，提供了一些接口，对access、debug、extra以及error日志进行检索。
在Fitnesse中利用这些接口，创建了一些关键字，来获取日志。
好处
- 在自动化用例编写和调试时，就将检索日志的关键字保留下来
- 针对access日志，将需要关注的接口，保留下来
- ......
核心实现
依赖
<properties>
<elasticsearch>7.14.0</elasticsearch>
</properties>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch}</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>${elasticsearch}</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>${elasticsearch}</version>
</dependency>
获取client
//创建HttpHost
HttpHost host = new HttpHost("elasticsearch_Ip",9200);
// 创建RestClientBuilder
RestClientBuilder builder = RestClient.builder(host);
// 创建RestHighLevelClient
RestHighLevelClient client = new RestHighLevelClient(builder);
获取日志
/*
* @name: queryAccessLog
* @description: TODO
* @param fromTime 用例执行开始时间
* @param size 日志记录条数
* @param env 测试环境
* @param filterOptions 过滤条件
* #格式说明
* # a=b;c;d|e=f|g=h
* # 表示检索三个字段a,e,g 并且是“与”的关系
* # 其中b,c,d是字段a的匹配值，是“或“的关系
* @return: java.util.List<java.lang.String>
* @date: 2021/9/18 4:41 下午
* @auther: hch
*
*/
public List<String> queryAccessLog(String fromTime, String size, String env, String filterOptions) {
logger.debug("fromTime:{},size:{},env:{},filterOptions:{}",fromTime,size,env,filterOptions);
List<String> contents=new ArrayList<>();
logger.info("tomcat_code_access.log-begin");
String index = "tomcat_code_access.log-" + GetTime.getCurrentTimeWithFormat("yyyy.MM.dd");
if(!exists(index)){
logger.debug("index does not exist");
contents.add("index does not exist");
//close();
return contents;
}else{
logger.debug("index exists");
}
SearchResponse searchResponse = getLog(fromTime,size,env,filterOptions, index);
if(null !=searchResponse){
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
//access_req_params
//access_res_params
//project_name
//@timestamp
String access_req_params = (String) sourceAsMap.get("access_req_params");
String access_res_params = (String) sourceAsMap.get("access_res_params");
String project_name = (String) sourceAsMap.get("project_name");
String access_url=(String) sourceAsMap.get("access_url");
String time = (String) sourceAsMap.get("@timestamp");
//String hostName = String.valueOf(JSONPath.read(jsonObjectString,"$.host.name"));
StringBuffer sb = new StringBuffer();
sb.append(DateLocalUtcUtil.utcToLocal(time)).append(" : ")
.append(project_name).append(" : ")
.append(access_url).append(" : ")
.append(access_res_params).append(" : ")
.append(access_req_params);
contents.add(sb.toString());
}
}
logger.info("tomcat_code_access.log-end");
return contents;
}
private boolean exists(String index) {
GetIndexRequest request = new GetIndexRequest(index);
try {
return client.indices().exists(request, RequestOptions.DEFAULT);
} catch (IOException e) {
logger.info(e.getMessage());
e.printStackTrace();
// this.close();
return false;
}
}
private SearchResponse getLog(String fromTime,String size,String env,String filterOptions, String index) {
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.filter(QueryBuilders.wildcardQuery("host.name", "*" + env + "*"));
if (!StringUtils.equals("null", fromTime)) {
boolQueryBuilder.filter(QueryBuilders.rangeQuery("@timestamp")
.from(DateLocalUtcUtil.localToUtc(fromTime))
);
} else {
boolQueryBuilder.filter(QueryBuilders.rangeQuery("@timestamp")
.from(DateLocalUtcUtil.localToUtc(GetTime.getAnyMinuteOfCurrentHourWithFormat(DateFormatConstant.DATA_TIME_FROMAT, "-5")))
);
}
Map<String,String> paramsMap=filterOptionsToMap(filterOptions);
logger.debug("filterOptions:{}", JSONObject.toJSONString(paramsMap));
for (Map.Entry<String, String> entry : paramsMap.entrySet()) {
String key = entry.getKey();
String value = entry.getValue();
if (StringUtils.contains(value, ";")) {
String[] values = StringUtils.split(value, ";");
BoolQueryBuilder bqb = QueryBuilders.boolQuery();
for (String s_value : values) {
bqb.should(QueryBuilders.matchQuery(key, s_value));
}
boolQueryBuilder.filter(bqb);
} else {
boolQueryBuilder.filter(QueryBuilders.matchQuery(key, value));
}
}
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(boolQueryBuilder);
sourceBuilder.sort(new FieldSortBuilder("@timestamp").order(SortOrder.DESC));
if (!StringUtils.equals("null", size)) {
sourceBuilder.from(0);
sourceBuilder.size(Integer.valueOf(size));
} else {
sourceBuilder.from(0);
sourceBuilder.size(100);
}
sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.source(sourceBuilder);
try {
return client.search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
logger.info(e.getMessage());
e.printStackTrace();
return null;
}finally {
// close();
}
}
API参考
资源参考
https://blog.csdn.net/ass23313/article/details/94627725
https://www.cnblogs.com/reycg-blog/p/9946821.html
https://blog.csdn.net/qq_34624315/article/details/81041479
https://www.jianshu.com/p/b0197c6ff895
Fitnesse关键字
当前的设计思路是，通过http请求上面的服务，获取到数据后，打印到fitnesse的执行日志中。
public class EsFixture {
private final static Logger logger = LoggerFactory.getLogger(EsFixture.class);
private RestTemplate rest;
private String DEBUG_URL="http://ip:port/elk/queryDebugLog";
private String ACCESS_URL="http://ip:port/elk/queryAccessLog";
private String EXTRA_URL="http://ip:port/elk/queryExtraLog";
private String ERROR_URL="http://ip:port/elk/queryErrorLog";
public EsFixture(){
rest = RestContext.getInstance().restTemplate;
}
public int queryDebugLog(Map<String, String> paramsMap){
List<String> list= this.postFormForObject(DEBUG_URL,null,paramsMap);
logger.info("debug log - begin");
for(String s:list){
logger.info(s);
}
logger.info("debug log - end");
return list.size();
}
public int queryAccessLog(Map<String, String> paramsMap){
List<String> list= this.postFormForObject(ACCESS_URL,null,paramsMap);
logger.info("access log - begin");
for(String s:list){
logger.info(s);
}
logger.info("access log - end");
return list.size();
}
public int queryErrorLog(Map<String, String> paramsMap){
List<String> list= this.postFormForObject(ERROR_URL,null,paramsMap);
logger.info("error log - begin");
for(String s:list){
logger.info(s);
}
logger.info("error log - end");
return list.size();
}
public int queryExtraLog(Map<String, String> paramsMap){
List<String> list= this.postFormForObject(EXTRA_URL,null,paramsMap);
logger.info("extra log - begin");
for(String s:list){
logger.info(s);
}
logger.info("extra log - end");
return list.size();
}
private List<String> postFormForObject(String url, Map<String, String> headermap, Map<String, String> paramsmap) {
MultiValueMap<String, Object> postParameters = new LinkedMultiValueMap<>();
if (null != paramsmap) {
for (Map.Entry<String, String> entry : paramsmap.entrySet()) {
if(!"null".equals(entry.getValue().toLowerCase().trim())){
postParameters.add(entry.getKey(), entry.getValue());
}
}
}
HttpHeaders headers = new HttpHeaders();
MediaType type = MediaType.parseMediaType("application/x-www-form-urlencoded; charset=UTF-8");
headers.setContentType(type);
//headers.add("Accept", MediaType.APPLICATION_JSON.toString());
if (null != headermap) {
for (Map.Entry<String, String> entry : headermap.entrySet()) {
headers.add(entry.getKey(), entry.getValue());
}
}
HttpEntity<MultiValueMap<String, Object>> formEntity = new HttpEntity<>(postParameters, headers);
Map<String,Object> map = this.rest.postForObject(url, formEntity, Map.class);
List<String> list= (List<String>)map.getOrDefault("data",new ArrayList<String>());
return list;
}
}
用例执行
fitnesse执行日志
本文来自博客园，作者：月色深潭，交流群：733423266，转载请注明原文链接：https://www.cnblogs.com/moonpool/p/15309421.html
