报错--RuntimeError: Too many open files. Communication with the workers is no longer possible.

本文主要是介绍报错--RuntimeError: Too many open files. Communication with the workers is no longer possible.，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

问题描述

pytorch 的 dataloader 在读取数据时,设置了较大的 batchsize 和 num_workers. 然后训练一段时间报错:

RuntimeError: Too many open files. Communication with the workers is no longer possible. Please increase the limit using ulimit -n in the shell or change the sharing strategy by calling torch.multiprocessing.se t_sharing_strategy('file_system') at the beginning of your code

解决方法

在代码开头引入 torch 之后加上:

import torch.multiprocessing
torch.multiprocessing.set_sharing_strategy('file_system')

参考

• https://github.com/pytorch/pytorch/issues/11201

这篇关于报错--RuntimeError: Too many open files. Communication with the workers is no longer possible.的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---