使用httpclient时候，出现“Too many open files”问题

最近接触的sendHttpPostRequest的问题比较多，近期碰到了关于 "java.net.SocketException: Too many open files" 的问题

异常信息如下：

ERROR | 2016-01-12 03:19:36.642 | ERROR | IREAD | [Exception]exception stack info | com.huawei.bme.commons.util.debug.DebugLogImpl.doLog(DebugLogImpl.java:480)
java.net.SocketException: Too many open files
    at java.net.Socket.createImpl(Socket.java:397)
    at java.net.Socket.getImpl(Socket.java:460)
    at java.net.Socket.bind(Socket.java:577)
    at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.commons.httpclient.protocol.ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:139)
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:125)
    at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
    at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
    at com.huawei.iread.duibaintf.duiba.sns.SNSServer.sendPostRequestWithCode(SNSServer.java:296)
    at com.huawei.iread.duibaintf.duiba.sns.SNSEngine.sendPostRequestWithCode(SNSEngine.java:92)
    at com.huawei.iread.duibaintf.duiba.duibamall.GetDuibaExchangeResultNotifyImpl.getDuibaExchangeResultNotify(GetDuibaExchangeResultNotifyImpl.java:91)

其实也就是因为 sendPostRequestWithCode 和 getDuibaExchangeResultNotify 方法调用了 HttpClient.executeMethod 引起的。

---

 

查找linux服务器中open files文件数，发现好多是下面的情况，意思是socket资源泄漏，进程使用的sock未关闭：

COMMAND     PID       USER   FD      TYPE             DEVICE   SIZE/OFF       NODE NAME
java      27185      duiba  100u     sock                0,7      0t0   92280355 can't identify protocol
java      27185      duiba  101u     sock                0,7      0t0   92282035 can't identify protocol
java      27185      duiba  102u     sock                0,7      0t0   92280901 can't identify protocol
java      27185      duiba  103u     sock                0,7      0t0   92282036 can't identify protocol
java      27185      duiba  104u     sock                0,7      0t0   92280389 can't identify protocol
java      27185      duiba  105u     sock                0,7      0t0   92280390 can't identify protocol
java      27185      duiba  106u     sock                0,7      0t0   92280391 can't identify protocol
java      27185      duiba  107u     sock                0,7      0t0   92280392 can't identify protocol

---

Java源代码中是这样写的：

try{
    httpMethod = new PostMethod(url)
    // 设置header信息，传输XML格式的
    httpMethod.setRequestHeader(CONTENT_TYPE_NAME, CONTENT_TYPE_VALUE_XML_UTF_8);
    httpMethod.setRequestEntity(entity);

    HttpClient httpClient = new HttpClient(new MultiThreadedHttpConnectionManager());//使用连接池技术创建
    httpClient.executeMethod(httpMethod);
}catch{...}
finally{

    if (null != httpMethod)
    {
        httpMethod.releaseConnection();//释放资源
    }

}

从上面看，最后， httpMethod.releaseConnection();//释放资源。实际上，HttpClient建立Socket时 ，post.releaseConnection()并没有真正关闭连接，而是将该连接提交给 MultiThreadedHttpConnectionManager，等待复用。而http的连接是等待timeout才会自动断开的，所以，当用完系统的句柄后，自然会报Too many open files。

所以，应该添加下面的代码进行释放socket：

try{
    httpMethod = new PostMethod(url)
    // 设置header信息，传输XML格式的
    httpMethod.setRequestHeader(CONTENT_TYPE_NAME, CONTENT_TYPE_VALUE_XML_UTF_8);
    httpMethod.setRequestEntity(entity);
    //请求头信息中添加关闭连接
    httpMethod.addRequestHeader("Connection", "close");

    HttpClient httpClient = new HttpClient(new MultiThreadedHttpConnectionManager());//使用连接池技术创建
    httpClient.executeMethod(httpMethod);
    //连发三次机制                           
    httpClient.getParams().setBooleanParameter("http.protocol.expect-continue", false);
}catch{...}
finally{
    if (null != httpMethod)
    {
        httpMethod.releaseConnection();//释放资源
    }

}  

---

还有一种说法：

linux的文件打开数超过了默认的1024个文件，具体查看linux下面的文件打开最大值的命令ulimit -a，出现信息如下：

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 69631
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 69631
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimite

其中open files                      (-n) 1024 默认为1024，
有说解决方法是加大文件打开的最大值，ulimit -n 8192
但是由于Httpclient引起的。当并发访问量过大的时候，这个方法只是治标不治本的。所有还是解决Httpclient远程请求所造成Socket没有释放，使用上面的方法较好。