openstack Too many open files错误

oopenstack突然web页面无法打开，页面报500错误，查看httpd日志报如下错误

[Fri Feb 01 13:47:36.120854 2019] [:error] [pid 28948] [remote 192.168.14.83:680] IOError: [Errno 24] Too many open files: '/usr/share/openstack-dashboard/openstack_dashboard/themes/default/templates/500.html'
[Fri Feb 01 13:52:21.837032 2019] [mpm_prefork:notice] [pid 5454] AH00170: caught SIGWINCH, shutting down gracefully
[Fri Feb 01 13:54:50.565154 2019] [suexec:notice] [pid 5490] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Feb 01 13:54:50.579447 2019] [auth_digest:notice] [pid 5490] AH01757: generating secret for digest authentication ...
[Fri Feb 01 13:54:50.579837 2019] [lbmethod_heartbeat:notice] [pid 5490] AH02282: No slotmem from mod_heartmonitor
[Fri Feb 01 13:54:50.590593 2019] [mpm_prefork:notice] [pid 5490] AH00163: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations
[Fri Feb 01 13:54:50.590627 2019] [core:notice] [pid 5490] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Fri Feb 01 13:56:15.154790 2019] [:error] [pid 5495] WARNING:root:Use of this 'djano.wsgi' file has been deprecated since the Rocky release in favor of 'wsgi.py' in the 'openstack_dashboard' module. This file is a legacy naming from before Django 1.4 and an importable 'wsgi.py' is now the default. This file will be removed in the T release cycle.

 

突然想起在安装的时候忘记修改操作系统打开的文件数；登录到控制节点执行

[root@ipr-controller httpd]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 62931
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 62931
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@ipr-controller httpd]# 
[root@ipr-controller httpd]# 
[root@ipr-controller httpd]# ulimit -n
1024

查看当前打开文件数，明显超过了1024

[root@ipr-controller httpd]# lsof | wc -l  
98026

 

修改默认值

vim /etc/security/limits.conf  在最后加入如下信息：

*               soft    nofile            1024000
*               hard    nofile            1024000

 

 *表示所有用户

 

修改后重启服务器，配置生效