CloseableHttpClient 连接超时导致XxlJob调度阻塞,影响调度任务的执行
CloseableHttpClient 连接超时导致XxlJob调度阻塞,影响调度任务的执行
问题原因
1.分析日志发现,xxlJob后台界面没有执行时间和执行结果,在某一个时间点之后,某一个任务因为阻塞全部执行失败,影响业务系统未正常进行。比如:定时投保,购买保险等。
2.临时解决:先重启服务,XxlJob恢复调度,可以正常执行任务。
3.优化解决:排查logger日志,发现请求的日志有,返回的日志没有,分析代码发现,CloseableHttpClient未设置超时时间,加上该代码,重新上线。
4.业务数据的拉取,提供给业务方来做线下处理等操作。
5.加上python监控,根据SQL查询业务执行结果,每隔2个小时查询一次,如果没有执行结果,则报警提示。达到监控的目的。
DEMO代码示例:
package com.example.core.mydemo.http.httpclient; import com.alibaba.fastjson.JSON; import com.example.core.mydemo.http.CancelOrderReqVO; import com.example.core.mydemo.http.CancelRenyunOrderReqVO; import com.example.core.mydemo.json2.GsonUtils; import org.apache.http.HttpEntity; import org.apache.http.StatusLine; import org.apache.http.client.config.RequestConfig; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpPost; import org.apache.http.entity.StringEntity; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils; import org.springframework.http.HttpStatus; import java.io.IOException; import java.nio.charset.StandardCharsets; public class CloseableHttpTest { public static void main(String[] args) throws Exception { CancelOrderReqVO data = new CancelOrderReqVO(); data.setOrderNum("111123333"); data.setServerType("1"); //仅仅测试 String url = "http://www.baidu.com"; String jsonParam = GsonUtils.toJson(data); System.out.println("调用外部接口入参jsonParam:{}" + jsonParam); String resJson = doPost(url,jsonParam); System.out.println("调用外部接口返回:{}" + resJson); } static String doPost(String url, String params) throws Exception { CloseableHttpClient httpclient = HttpClients.createDefault(); HttpPost httpPost = new HttpPost(url);// 创建httpPost System.out.println("请求url:"+url+",param:" + JSON.toJSONString(params)); httpPost.setHeader("Accept", "application/json;charset=utf-8"); httpPost.setHeader("Content-Type", "application/json;charset=utf-8"); String charSet = "UTF-8"; StringEntity entity = new StringEntity(params, charSet); httpPost.setEntity(entity); //设置超时时间【关键】 // 设置连接超时时间(毫秒) int connectTimeout = 10000; // 设置读取超时时间(毫秒) int socketTimeout = 10000; // 设置从连接池中获取连接的超时时间(毫秒) int connectionRequestTimeout = 10000; RequestConfig requestConfig = RequestConfig.custom().setSocketTimeout(socketTimeout).setConnectTimeout(connectTimeout).setConnectionRequestTimeout(connectionRequestTimeout).build(); httpPost.setConfig(requestConfig); CloseableHttpResponse response = null; try { response = httpclient.execute(httpPost); StatusLine status = response.getStatusLine(); int state = status.getStatusCode(); if (state == HttpStatus.OK.value()) { HttpEntity responseEntity = response.getEntity(); String jsonString = EntityUtils.toString(responseEntity); // return new String(jsonString.getBytes(),"UTF-8"); return jsonString; } else{ //logger.error("请求返回:"+state+"("+url+")"); } } finally { if (response != null) { try { response.close(); } catch (IOException e) { e.printStackTrace(); } } try { httpclient.close(); } catch (IOException e) { e.printStackTrace(); } } return null; } }