(原)docker容器内无core dump文件生成

寻找core dump文件可以使用:coredumpctl info

root@VM-8-11-ubuntu:~# coredumpctl info
          PID: 1387579 (xxxx)
          UID: 0 (root)
          GID: 0 (root)
      Signal: 11 (SEGV)
    Timestamp: Thu 2021-12-02 16:49:26 CST (35s ago)
Command Line: ./xxxx --i=rtmp://xxxxxxx/lhp/stream --use_rtc=true --use_test_env=false
  Executable: /xxx/build/ctyunAVTranscoder
Control Group: /system.slice/containerd.service
        Unit: containerd.service
        Slice: system.slice
      Boot ID: 857c5d10dfe642258f3641b16dacbe29
  Machine ID: 6ff74844f0c747a08406ce78ec4766c1
    Hostname: VM-8-11-ubuntu
      Storage: /var/lib/systemd/coredump/core.xxx.0.857c5d10dfe642258f3641b16dacbe29.1387579.1638434966000000000000.lz4
      Message: Process 1387579 (xxx) of user 0 dumped core.

              Stack trace of thread 15363:
               #0 0x00007f2f85d517e4 n/a (libc.so.6 + 0x18e7e4)
               #1 0x00007f2f86cc1599 n/a (/xxx/src/3rdparty/ZegoExpressEngine/lib/libZegoExpressEngine.so + 0xcea599)
               #2 0x2e656c6200000000 n/a (n/a + 0x0)
               #3 0x00007f2f86cb7eb0 n/a (/xxx/src/3rdparty/ZegoExpressEngine/lib/libZegoExpressEngine.so + 0xce0eb0)

例如我再docker容器下跑一个后台的server应用,然后他core dump了,于是,我在主机上执行coredumpctl info 就可以找到这个core dump的信息了。

 

如果找不到core dump文件的话,就需要查看下容器下和宿主机的ulimit -a信息了,可能是ulimit 限制了core文件的生成,这个时候,如果要长期生效的话,需要修改/etc/security/limits.conf文件,然后将改文件的内容如下信息:


root(你的用户)           soft   core           100000
root(你的用户)           hard   core           100000

要生效,只需要断开连接,重新ssh登录一下主机,就可以了。


root@VM-8-11-ubuntu:~# ulimit -a
core file size         (blocks, -c) 100000
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 15363
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) unlimited
open files                     (-n) 1024
pipe size           (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority             (-r) 0
stack size             (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes             (-u) 15363
virtual memory         (kbytes, -v) unlimited
file locks                     (-x) unlimited
root@VM-8-11-ubuntu:~#

在有coredump info信息输出的时候,例如上面的输出信息,然后我们可以使用coredumpctl dump命令去获取core文件。

完整参考例子:(宿主机上执行)

root@VM-8-11-ubuntu:~# coredumpctl info
          PID: 1597476 (ctyunAVTranscod)
          UID: 0 (root)
          GID: 0 (root)
      Signal: 11 (SEGV)
    Timestamp: Fri 2021-12-03 10:25:32 CST (47s ago)
Command Line: ./ctyunAVTranscoder --i=/data1/video/son_20211101124400.mp4 --o=taekwondo.mp4 --vcodec=libx264
  Executable: /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Control Group: /system.slice/containerd.service
        Unit: containerd.service
        Slice: system.slice
      Boot ID: 857c5d10dfe642258f3641b16dacbe29
  Machine ID: 6ff74844f0c747a08406ce78ec4766c1
    Hostname: VM-8-11-ubuntu
      Storage: /var/lib/systemd/coredump/core.ctyunAVTranscod.0.857c5d10dfe642258f3641b16dacbe29.1597476.1638498332000000000000.lz4
      Message: Process 1597476 (ctyunAVTranscod) of user 0 dumped core.

              Stack trace of thread 15645:
               #0 0x00007fe4274e2256 n/a (/usr/lib/x86_64-linux-gnu/libavfilter.so.7.57.100 + 0x12c256
root@VM-8-11-ubuntu:~#
root@VM-8-11-ubuntu:~# coredumpctl list
TIME                           PID   UID   GID SIG COREFILE EXE
Thu 2021-12-02 15:54:58 CST  1376459     0     0  11 present   /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Thu 2021-12-02 16:49:27 CST  1387579     0     0  11 present   /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:03:29 CST  1592934     0     0  11 present   /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:14:25 CST  1595147     0     0  11 present   /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:21:24 CST  1596570     0     0  11 present   /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:23:02 CST  1596957     0     0  11 present   /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Fri 2021-12-03 10:25:32 CST  1597476     0     0  11 present   /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
root@VM-8-11-ubuntu:~#
root@VM-8-11-ubuntu:~# coredumpctl dump 1597476 --output /tmp/core.1597476
          PID: 1597476 (ctyunAVTranscod)
          UID: 0 (root)
          GID: 0 (root)
      Signal: 11 (SEGV)
    Timestamp: Fri 2021-12-03 10:25:32 CST (8min ago)
Command Line: ./ctyunAVTranscoder --i=/data1/video/son_20211101124400.mp4 --o=taekwondo.mp4 --vcodec=libx264
  Executable: /lhp/app/ctyun_avtranscoder/build/ctyunAVTranscoder
Control Group: /system.slice/containerd.service
        Unit: containerd.service
        Slice: system.slice
      Boot ID: 857c5d10dfe642258f3641b16dacbe29
  Machine ID: 6ff74844f0c747a08406ce78ec4766c1
    Hostname: VM-8-11-ubuntu
      Storage: /var/lib/systemd/coredump/core.ctyunAVTranscod.0.857c5d10dfe642258f3641b16dacbe29.1597476.1638498332000000000000.lz4
      Message: Process 1597476 (ctyunAVTranscod) of user 0 dumped core.

              Stack trace of thread 15645:
               #0 0x00007fe4274e2256 n/a (/usr/lib/x86_64-linux-gnu/libavfilter.so.7.57.100 + 0x12c256)
root@VM-8-11-ubuntu:~#

 

参考:https://unix.stackexchange.com/questions/65110/no-more-coredumps-after-migrating-to-systemd

Per default, systemd writes coredumps to the journal. coredumpctl list lists the missing coredumps.

The files are stored in /var/lib/systemd/coredump. Use coredumpctl dump to get access to the core files.

For instance, if the PID was 10666, you can use


coredumpctl dump 10666 --output /tmp/core.10666

Not related to systemd, but if it is still not working, make sure that the coredump limit is turned off:


# ulimit -c unlimited

Also verify that you can write to the working directory.

 

 

posted @ 2021-12-03 10:39  lihaiping  阅读(1406)  评论(0编辑  收藏  举报