(原)docker容器内无core dump文件生成
寻找core dump文件可以使用:coredumpctl info
root@VM-8-11-ubuntu:~# coredumpctl info
PID: 1387579 (xxxx)
UID: 0 (root)
GID: 0 (root)
Signal: 11 (SEGV)
Timestamp: Thu 2021-12-02 16:49:26 CST (35s ago)
Command Line: ./xxxx --i=rtmp://xxxxxxx/lhp/stream --use_rtc=true --use_test_env=false
Executable: /xxx/build/ctyunAVTranscoder
Control Group: /system.slice/containerd.service
Unit: containerd.service
Slice: system.slice
Boot ID: 857c5d10dfe642258f3641b16dacbe29
Machine ID: 6ff74844f0c747a08406ce78ec4766c1
Hostname: VM-8-11-ubuntu
Storage: /var/lib/systemd/coredump/core.xxx.0.857c5d10dfe642258f3641b16dacbe29.1387579.1638434966000000000000.lz4
Message: Process 1387579 (xxx) of user 0 dumped core.
Stack trace of thread 15363:
#0 0x00007f2f85d517e4 n/a (libc.so.6 + 0x18e7e4)
#1 0x00007f2f86cc1599 n/a (/xxx/src/3rdparty/ZegoExpressEngine/lib/libZegoExpressEngine.so + 0xcea599)
#2 0x2e656c6200000000 n/a (n/a + 0x0)
#3 0x00007f2f86cb7eb0 n/a (/xxx/src/3rdparty/ZegoExpressEngine/lib/libZegoExpressEngine.so + 0xce0eb0)
例如我再docker容器下跑一个后台的server应用,然后他core dump了,于是,我在主机上执行coredumpctl info 就可以找到这个core dump的信息了。
如果找不到core dump文件的话,就需要查看下容器下和宿主机的ulimit -a信息了,可能是ulimit 限制了core文件的生成,这个时候,如果要长期生效的话,需要修改/etc/security/limits.conf文件,然后将改文件的内容如下信息:
root(你的用户) soft core 100000
root(你的用户) hard core 100000
要生效,只需要断开连接,重新ssh登录一下主机,就可以了。
root@VM-8-11-ubuntu:~# ulimit -a
core file size (blocks, -c) 100000
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 15363
max locked memory (kbytes, -l) 65536
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 15363
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
root@VM-8-11-ubuntu:~#
在有coredump info信息输出的时候,例如上面的输出信息,然后我们可以使用coredumpctl dump命令去获取core文件。
完整参考例子:(宿主机上执行)
root@VM-8-11-ubuntu:~# coredumpctl info
PID: 1597476 (ctyunAVTranscod)
UID: 0 (root)
GID: 0 (root)
Signal: 11 (SEGV)
Timestamp: Fri 2021-12-03 10:25:32 CST (47s ago)
Command Line: ./ctyunAVTranscoder --i=/data1/video/son_20211101124400.mp4 --o=taekwondo.mp4 --vcodec=libx264
Executable: /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Control Group: /system.slice/containerd.service
Unit: containerd.service
Slice: system.slice
Boot ID: 857c5d10dfe642258f3641b16dacbe29
Machine ID: 6ff74844f0c747a08406ce78ec4766c1
Hostname: VM-8-11-ubuntu
Storage: /var/lib/systemd/coredump/core.ctyunAVTranscod.0.857c5d10dfe642258f3641b16dacbe29.1597476.1638498332000000000000.lz4
Message: Process 1597476 (ctyunAVTranscod) of user 0 dumped core.
Stack trace of thread 15645:
#0 0x00007fe4274e2256 n/a (/usr/lib/x86_64-linux-gnu/libavfilter.so.7.57.100 + 0x12c256
root@VM-8-11-ubuntu:~#
root@VM-8-11-ubuntu:~# coredumpctl list
TIME PID UID GID SIG COREFILE EXE
Thu 2021-12-02 15:54:58 CST 1376459 0 0 11 present /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Thu 2021-12-02 16:49:27 CST 1387579 0 0 11 present /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:03:29 CST 1592934 0 0 11 present /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:14:25 CST 1595147 0 0 11 present /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:21:24 CST 1596570 0 0 11 present /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:23:02 CST 1596957 0 0 11 present /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:25:32 CST 1597476 0 0 11 present /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
root@VM-8-11-ubuntu:~#
root@VM-8-11-ubuntu:~# coredumpctl dump 1597476 --output /tmp/core.1597476
PID: 1597476 (ctyunAVTranscod)
UID: 0 (root)
GID: 0 (root)
Signal: 11 (SEGV)
Timestamp: Fri 2021-12-03 10:25:32 CST (8min ago)
Command Line: ./ctyunAVTranscoder --i=/data1/video/son_20211101124400.mp4 --o=taekwondo.mp4 --vcodec=libx264
Executable: /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Control Group: /system.slice/containerd.service
Unit: containerd.service
Slice: system.slice
Boot ID: 857c5d10dfe642258f3641b16dacbe29
Machine ID: 6ff74844f0c747a08406ce78ec4766c1
Hostname: VM-8-11-ubuntu
Storage: /var/lib/systemd/coredump/core.ctyunAVTranscod.0.857c5d10dfe642258f3641b16dacbe29.1597476.1638498332000000000000.lz4
Message: Process 1597476 (ctyunAVTranscod) of user 0 dumped core.
Stack trace of thread 15645:
#0 0x00007fe4274e2256 n/a (/usr/lib/x86_64-linux-gnu/libavfilter.so.7.57.100 + 0x12c256)
root@VM-8-11-ubuntu:~#
参考:https://unix.stackexchange.com/questions/65110/no-more-coredumps-after-migrating-to-systemd
Per default, systemd writes coredumps to the journal. coredumpctl list
lists the missing coredumps.
The files are stored in /var/lib/systemd/coredump
. Use coredumpctl dump
to get access to the core files.
For instance, if the PID was 10666, you can use
coredumpctl dump 10666 --output /tmp/core.10666
Not related to systemd, but if it is still not working, make sure that the coredump limit is turned off:
# ulimit -c unlimited
Also verify that you can write to the working directory.