22文件描述符耗尽（一）mac【本地】
在21Why httpclient is recommended to go with a connection pool in server-to-server request?中阐述了文件描述符耗尽导致异常，本文予以证明
mac环境
ulimit -n
10240
sysctl -a | grep kern.max
kern.maxfiles: 12288 系统级（所有进程）最大12288个文件
kern.maxfilesperproc: 10240 用户级（单个进程）最大10240个文件
lsof | wc -l 3760 程序未启动就占用3760个文件
================================
1 纯文件
public class FileClient {
public static void main(String [] f) {
/**
* 如果缺少，连类加载器都不能加载了，让其提前加载
* FileNotFoundException: /Users/joyce/work/netty/netty-test/target/classes/com/jds/test/bio/p12/Client.class (Too many open files in system)
*/
Client client = new Client();
run();
Client.zuse();
}
/**
* print 8489 不稳定
* lsof -p 12322 |wc -l 8567 不稳定 执行该句时，应先打开terminal占用一个文件描述符
*/
public static void run() {
try {
for(int i=0; i<20000; ++i) {
File file = new File("/Users/joyce/Downloads/xx/xxxx"+i);
file.createNewFile();
FileOutputStream outputStream = new FileOutputStream(file);
byte[] bytes = new byte[2];
outputStream.write(bytes);
System.out.println(i+1);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
/*
** 使命令行有足够时间操作
*/
public static void zuse() {
while (true) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
1）too many open files in system 甚至会影响进程内类加载
2）触达系统级限制，会影响terminal的打开
3）8000左右时（这个值每次不固定），报too many open files in system 异常，为什么没达到用户级10240的限制就不行了，应该是因为所有进程加起来达到了系统级限制12288
4）本例中，打开文件数量和lsof命令都不稳定
2 纯socket
public class Server {
public static final int PORT = 12123;
private static AtomicInteger atomicInteger = new AtomicInteger(0);
public static void main(String [] f) throws IOException {
new Server().server();
}
/**
* single socket + socket and file
* print 4059 稳定
* lsof -p 12534 |wc -l 4138 稳定
* @throws IOException
*/
public void server() throws IOException {
ServerSocket ss = new ServerSocket(PORT);
while(true) {
Socket s = ss.accept();
// 连接成功后计数
System.out.println(atomicInteger.incrementAndGet());
}
}
}
public class TcpClient {
private static AtomicInteger atomicInteger = new AtomicInteger(0);
public static final int PORT = 12123;
public static final int BUFFER_SIZE = 1024;
public static void main(String []f) {
Client client = new Client();
run();
Client.zuse();
}
/**
* print 4059 稳定
* lsof -p 12534 |wc -l 4558 不稳定
*/
public static void run() {
for(int i=0; i<5000; ++i) {
try {
new TcpClient().client();
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
public void client() throws Exception {
final String s2 = "localhost";
Socket s = new Socket();
s.connect(new InetSocketAddress(s2, PORT));
// 连接成功后计数
System.out.println(atomicInteger.incrementAndGet());
}
}
client输出：
4055
4056
4057
4058
4059
Socket closed
Socket closed
Socket closed
Socket closed
Socket closed
Socket closed
。。。
Socket closed
Socket closed
Socket closed
Socket closed
Too many open files in system
Too many open files in system
Too many open files in system
Too many open files in system
。。。。
Too many open files in system
Too many open files in system
Too many open files in system
Too many open files in system
server输出：
4055
4056
4057
4058
4059
注意，服务端没有奔溃
1）socket连接server端与client端都打印到4059 ，报xxx，加起来两个进程共打开8118个文件，与1中纯file的案例相同，虽单进程未触达用户级限制，但已触达系统级限制
2）本例中，server端socket数量稳定，lsof命令稳定
3）本例中，client端socket数量稳定，lsof命令不稳定
3 socket + file
public static void main(String [] f) throws InterruptedException {
Client client = new Client();
run();
Client.zuse();
}
/**
* print tcp 4059 稳定
* print file 1 不稳定
* lsof -p 13152|wc -l 4635 不稳定
* @throws InterruptedException
*/
public static void run() throws InterruptedException {
/**
* 如果缺少，连类加载器都不能加载了，让其提前加载
* FileNotFoundException: /Users/joyce/work/netty/netty-test/target/classes/com/jds/test/bio/p12/Client.class (Too many open files in system)
*/
FileClient fileClient = new FileClient();
TcpClient.run();
System.out.println("tcp done");
Thread.sleep(5000);
System.out.println("start file");
FileClient.run();
}
client输出：
4056
4057
4058
4059
Socket closed
Socket closed
Socket closed
Socket closed
Socket closed
。。。。
Socket closed
Socket closed
Socket closed
Socket closed
Too many open files in system
Too many open files in system
Too many open files in system
Too many open files in system
Too many open files in system
tcp done
start file
1
java.io.IOException: Too many open files in system
Server端输出：
4055
4056
4057
4058
4059
注意，服务端没有奔溃
1）socket连接server端与client端都打印到4059 ，报xxx，加起来两个进程共打开8118个文件，与1中纯file的案例相同，虽单进程未触达用户级限制，但已触达系统级限制
2）接下去的同进程文件读取，成功读取了一个本地文件后报错；不同进程想通过其它进程再执行一次1-纯文件读取时无法make
3）本例中，执行lsof -p pid | wc -l时挂了一次-- process information unavailable
4）本例中，server端socket数量稳定，lsof命令稳定
5）本例中，client端socket数量稳定，后续打开文件数量不稳定，lsof命令不稳定
结论：
0 证明了mac系统默认所有进程（系统级）12288，单进程（用户级）10240
1 mac环境下，立足系统级（虽未触达用户级，但触达系统级），通过纯文件、纯socket、socket+文件3种方式验证了文件描述符耗尽导致too many open files in system
2 mac环境下，立足系统级，证明了系统级fd耗尽除了socket与纯file，还会影响本进程类文件读取、其它进程如terminal打开、jps命令执行、idea的运行、java make、lsof命令，甚至导致系统不可用
3 mac环境下，立足系统级，通过socket+文件方式验证了socket导致的too many open files 同样会继续影响到后续程序纯文件读取，证明是socket和纯file是同一个文件的概念
4 windows下似乎文件描述符限制较大，直奔2w个socket，出现了No buffer space available (maximum connections reached?): connect
