本文主要是介绍runner.logger.info(‘loss become infinite or NaN,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
在使用mmdetection训练的时候出现了如下错误:
2022-05-07 21:35:13,488 - mmdet - INFO - Epoch [1][50/5111] lr: 1.978e-04, eta: 18:35:19, time: 0.546, data_time: 0.217, memory: 1781, loss_cls: 17.7358, loss_bbox: 4.1217, loss: 21.8
575
2022-05-07 21:35:24,322 - mmdet - INFO - Epoch [1][100/5111] lr: 3.976e-04, eta: 12:58:42, time: 0.217, data_time: 0.003, memory: 1781, loss_cls: 13.2629, loss_bbox: 4.4149, loss: 17.6
778
2022-05-07 21:35:35,155 - mmdet - INFO - Epoch [1][150/5111] lr: 5.974e-04, eta: 11:06:25, time: 0.217, data_time: 0.003, memory: 1781, loss_cls: 9.6519, loss_bbox: 4.5696, loss: 14.22
16
2022-05-07 21:35:45,901 - mmdet - INFO - Epoch [1][200/5111] lr: 7.972e-04, eta: 10:09:16, time: 0.215, data_time: 0.003, memory: 1781, loss_cls: 13.5727, loss_bbox: 5.6242, loss: 19.1
969runner.run(data_loaders, cfg.workflow)File "D:\ProgramData\Anaconda3\envs\mytorch\lib\site-packages\mmcv\runner\epoch_based_runner.py", line 127, in runepoch_runner(data_loaders[i], **kwargs)File "D:\ProgramData\Anaconda3\envs\mytorch\lib\site-packages\mmcv\runner\epoch_based_runner.py", line 51, in trainself.call_hook('after_train_iter')File "D:\ProgramData\Anaconda3\envs\mytorch\lib\site-packages\mmcv\runner\base_runner.py", line 309, in call_hookgetattr(hook, fn_name)(self)File "D:\ProgramData\Anaconda3\envs\mytorch\lib\site-packages\mmdet-2.24.1-py3.7.egg\mmdet\core\hook\checkloss_hook.py", line 24, in after_train_iterrunner.logger.info('loss become infinite or NaN!')
AssertionError: None
出现这个问题是因为学习率太大了,需要降低学习率;
把学习率改为2e-4就可以解决问题了。
这篇关于runner.logger.info(‘loss become infinite or NaN的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!