ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm)
简介:
使用YOLO11 在Docker 里面训练,出现一堆报错, 下面是关键报错信息:
ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm)
RuntimeError: DataLoader worker (pid 838) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory.
代码原文:
# ImageWoof 数据集 分析狗
from ultralytics import YOLO
# Load a model
model = YOLO("yolo11n-cls.pt") # load a pretrained model (recommended for training)
# Train the model
model.train(data="imagewoof160", epochs=100, imgsz=224, workers=4)
问题分析:
翻译得知明显是共享内存不足(shm)问题,原因是YOLO 使用PyTorch 训练,PyTorch 使用共享内存在进程之间共享数据,当workers 比较多的时候容器运行时的默认共享内存段大小是不够的,比如workers 改为2就不报错了。
GitHub 官方原文: https://github.com/pytorch/pytorch#docker-image
解决方法:
1、docker run 时使用--ipc=host或--shm-size命令行选项增加共享内存大小
docker run -itd --name=yolo --shm-size=2g
# 查看共享内存使用率
docker exec yolo df -h /dev/shm
2、将 workers 改的小点。