ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm)

简介:

   使用YOLO11 在Docker 里面训练,出现一堆报错, 下面是关键报错信息:

ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm)

RuntimeError: DataLoader worker (pid 838) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory.

 

代码原文:

# ImageWoof 数据集 分析狗
from ultralytics import YOLO

# Load a model
model = YOLO("yolo11n-cls.pt")  # load a pretrained model (recommended for training)

# Train the model
model.train(data="imagewoof160", epochs=100, imgsz=224, workers=4)

 

 

问题分析:

  翻译得知明显是共享内存不足(shm)问题,原因是YOLO 使用PyTorch 训练,PyTorch 使用共享内存在进程之间共享数据,当workers 比较多的时候容器运行时的默认共享内存段大小是不够的,比如workers 改为2就不报错了。

  GitHub 官方原文: https://github.com/pytorch/pytorch#docker-image

 

 

解决方法:

  1、docker run 时使用--ipc=host或--shm-size命令行选项增加共享内存大小

docker run -itd --name=yolo --shm-size=2g

# 查看共享内存使用率
docker exec yolo df -h /dev/shm

  

  2、将 workers 改的小点。

 

 

posted @ 2024-10-31 09:59  龙虚度  阅读(23)  评论(0编辑  收藏  举报