nginx open files limits 导致大量错误信息

nginx  error.log 中出现大量如下错误信息：

[root@localhost nginx]# grep -aP '^20.* \[crit\]' error.log

2017/03/14 12:06:31 [crit] 3549#0: accept4() failed (24: Too many open files)

[root@localhost nginx]# grep -aP '^20.* \[alert\]' error.log

2017/03/14 16:04:27 [alert] 3551#0: *84168270 socket() failed (24: Too many open files) while connecting to upstream, client: 1.1.1.1, server:... 

由于系统limits  open files 限制导致以上错误，所以：

root@open-mk-push-1:~# cat /etc/security/limits.conf 
*   soft    nofile  655360
*    hard    nofile  655360
root    soft    nofile  655360
root    hard    nofile  655360
*   soft    core unlimited
*    hard    core unlimited
root    soft    core unlimited

root@open-mk-push-1:~# ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 512240
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 655360
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 512240
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

但以上只是系统的open files 限制设置为655350；而nginx的open files 是否继承系统open files 设置还需要重新启动nginx 进程。可以通过以下命令查看：

root@open-mk-push-1:~# cat /proc/`ps -ef | grep nginx|grep -v grep|head -1|awk '{print $2}'`/limits
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             512240               512240               processes 
Max open files            1024                 4096                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       512240               512240               signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us           

可以看到nginx进程的Max open files的Soft Limit和Hard Limit分别只有1024和4096，这是一个初始的值可以说是很低了，故之前出现的Too many open files就可以理解了。

总结：ulimit -a && cat /proc/`ps -ef | grep nginx|grep -v grep| head -1 | awk '{print $2}'`/limits 必须保持一致，否则会导致大量连接失败，此时需要重启nginx 进程 。

由于我们nginx是通过supervisor纳管的，所以得重启supervisor这个服务才能使nginx的max open files与系统值保持一致。