2023 安装 facebookresearch slowfast

0 前言

facebookresearch SlowFast ：https://github.com/facebookresearch/SlowFast

b站视频：https://www.bilibili.com/video/BV1Uh4y1q7R2/

1 准备

平台：Autodl：https://www.autodl.com/
环境：
PyTorch 1.8.1
Python 3.8(ubuntu18.04)
Cuda 11.1

需要先将这两个权重下载到AI平台的：/root/slowfastFile

model_final_280758.pkl：https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl

SLOWFAST_32x2_R101_50_50.pkl：https://dl.fbaipublicfiles.com/pyslowfast/model_zoo/ava/SLOWFAST_32x2_R101_50_50.pkl

2 安装与运行

2.1 基础环境

pip install 'git+https://gitee.com/YFwinston/fvcore'

pip install simplejson

conda install av -c conda-forge -y

conda install x264 ffmpeg -c conda-forge -y

pip install -U iopath

pip install psutil

pip install opencv-python

// pip install torchvision

pip install tensorboard

pip install moviepy

pip install pytorchvideo

pip install 'git+https://gitee.com/YFwinston/fairscale'

2.2 detectron2_repo 安装

---

// pip install -U torch torchvision cython
pip install -U  cython

pip install -U 'git+https://gitee.com/YFwinston/fvcore.git' 'git+https://gitee.com/YFwinston/cocoapi.git#subdirectory=PythonAPI'

git clone https://gitee.com/YFwinston/detectron2 detectron2_repo

pip install typing-extensions==4.3.0

pip install -e detectron2_repo

---

pip install pillow
pip install pyyamlpip install scipy
pip install pandas
pip install scikit-learn

2.3 slowfast 安装

git clone https://gitee.com/YFwinston/slowfast

export PYTHONPATH=/root/slowfast:$PYTHONPATH

cd slowfast
python setup.py build develop
cd /root

cd /root/slowfast/demo/AVA
cp /root/autodl-nas/slowfast/ava.json ./
cp /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50s.yaml ./
cd /root

cd /rootgit clone https://gitee.com/YFwinston/pytorchvideo.git
cd pytorchvideo
pip install -e .
cd /root

2.4 相关文件

2.4.1 ava.json

然后在/root/slowfast/demo/AVA下面的ava.json写入：

{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, "paint": 38, "play board game": 39, "play musical instrument": 40, "play with pets": 41, "point to (an object)": 42, "press": 43, "pull (an object)": 44, "push (an object)": 45, "put down": 46, "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60, "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, "give/serve (an object) to (a person)": 64, "grab (a person)": 65, "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}

2.4.2 SLOWFAST_32x2_R101_50_50s.yaml

然后在/root/slowfast/demo/AVA下面的SLOWFAST_32x2_R101_50_50s.yaml 写入：

TRAIN:ENABLE: FalseDATASET: avaBATCH_SIZE: 1EVAL_PERIOD: 1CHECKPOINT_PERIOD: 1AUTO_RESUME: TrueCHECKPOINT_FILE_PATH: /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50.pkl  #path to pretrain modelCHECKPOINT_TYPE: pytorch
DATA:NUM_FRAMES: 16SAMPLING_RATE: 2TRAIN_JITTER_SCALES: [256, 320]TRAIN_CROP_SIZE: 224TEST_CROP_SIZE: 256INPUT_CHANNEL_NUM: [3, 3]
DETECTION:ENABLE: TrueALIGNED: False
AVA:BGR: FalseDETECTION_SCORE_THRESH: 0.8TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:ALPHA: 4BETA_INV: 8FUSION_CONV_CHANNEL_RATIO: 2FUSION_KERNEL_SZ: 5
RESNET:ZERO_INIT_FINAL_BN: TrueWIDTH_PER_GROUP: 64NUM_GROUPS: 1DEPTH: 101TRANS_FUNC: bottleneck_transformSTRIDE_1X1: FalseNUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]INSTANTIATION: dot_productPOOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:USE_PRECISE_STATS: FalseNUM_BATCHES_PRECISE: 200
SOLVER:MOMENTUM: 0.9WEIGHT_DECAY: 1e-7OPTIMIZING_METHOD: sgd
MODEL:NUM_CLASSES: 80ARCH: slowfastMODEL_NAME: SlowFastLOSS_FUNC: bceDROPOUT_RATE: 0.5HEAD_ACT: sigmoid
TEST:ENABLE: FalseDATASET: avaBATCH_SIZE: 1
DATA_LOADER:NUM_WORKERS: 1PIN_MEMORY: TrueNUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .
#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:ENABLE: TrueLABEL_FILE_PATH: "demo/AVA/ava.json" # Add local label file path here.INPUT_VIDEO: "/root/autodl-nas/slowfast/1.mp4"OUTPUT_FILE: "/root/autodl-nas/slowfast/1_1.mp4"DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"DETECTRON2_WEIGHTS: "/root/autodl-nas/slowfast/model_final_280758.pkl"

2.5 demo测试

在/home/slowfast/demo/中传入一个视频：1.mp4
执行：

cd /root/slowfast/
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50s.yaml

2.6 相关报错

2.6.1 报错1

This maybe due to another process holding this lock file. If you are sure no other Matplotlib process is running, remove this file and try again.

2.6.1 解决1

pip install --upgrade matplotlib

2.6.2 报错2

ModuleNotFoundError: No module named ‘torch._six’

2.6.2 解决2

 vim /root/slowfast/slowfast/datasets/multigrid_helper.py

if TORCH_MAJOR >= 1 and TORCH_MINOR >= 8:_int_classes = int
else:int_classes = int_int_classes = int#from torch._six import int_classes as _int_classes

---

2.6.3 报错3

报错：

    ERROR: Command errored out with exit status 1:command: /root/miniconda3/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/root/detectron2_repo/setup.py'"'"'; __file__='"'"'/root/detectron2_repo/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-yoefyzdpcwd: /root/detectron2_repo/Complete output (13 lines):Traceback (most recent call last):File "<string>", line 1, in <module>File "/root/detectron2_repo/setup.py", line 10, in <module>import torchFile "/root/miniconda3/lib/python3.8/site-packages/torch/__init__.py", line 1429, in <module>from torch import optim as optimFile "/root/miniconda3/lib/python3.8/site-packages/torch/optim/__init__.py", line 8, in <module>from .adadelta import AdadeltaFile "/root/miniconda3/lib/python3.8/site-packages/torch/optim/adadelta.py", line 4, in <module>from .optimizer import (Optimizer, _use_grad_for_differentiable, _default_to_fused_or_foreach,File "/root/miniconda3/lib/python3.8/site-packages/torch/optim/optimizer.py", line 23, in <module>from typing_extensions import ParamSpec, Self, TypeAliasImportError: cannot import name 'Self' from 'typing_extensions' (/root/miniconda3/lib/python3.8/site-packages/typing_extensions.py)----------------------------------------
WARNING: Discarding file:///root/detectron2_repo. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

2.6.3 解决3

pip install typing-extensions==4.3.0

2.6.4 报错4

Traceback (most recent call last):File "tools/run_net.py", line 6, in <module>from slowfast.utils.misc import launch_jobFile "/root/slowfast/slowfast/utils/misc.py", line 19, in <module>import slowfast.utils.logging as loggingFile "/root/slowfast/slowfast/utils/logging.py", line 15, in <module>import slowfast.utils.distributed as duFile "/root/slowfast/slowfast/utils/distributed.py", line 12, in <module>from pytorchvideo.layers.distributed import (  # noqa
ImportError: cannot import name 'cat_all_gather' from 'pytorchvideo.layers.distributed' (/root/miniconda3/lib/python3.8/site-packages/pytorchvideo/layers/distributed.py)

2.6.4 解决4

cd /rootgit clone https://gitee.com/YFwinston/pytorchvideo.git
cd pytorchvideo
pip install -e .
cd /root

3 yolo数据集转ava数据集

https://github.com/Whiffe/SCB-dataset/tree/main/yolo2ava

python yolo2ava.py --yolo_path /root/5k_HRW_yolo_Dataset --ava_path /root/autodl-tmp/SCB-ava-Dataset4

执行后的目录结构
SCB-ava-Dataset4
├── annotations
│ ├── ava_action_list_v2.2_for_activitynet_2019.pbtxt
│ ├── ava_included_timestamps_v2.2.txt
│ ├── ava_test_excluded_timestamps_v2.2.csv
│ ├── ava_train_excluded_timestamps_v2.2.csv
│ ├── ava_train_v2.2.csv
│ ├── ava_val_excluded_timestamps_v2.2.csv
│ ├── ava_val_v2.2.csv
│ └── person_box_67091280_iou90
│ ├── ava_detection_train_boxes_and_labels_include_negative_v2.2.csv
│ └── ava_detection_val_boxes_and_labels.csv
├── frame_lists
│ ├── train.csv
│ └── val.csv
└── frames
├── 0001001
│ ├── 0001001_000001.png
│ ├── 0001001_000002.png
│ ├── …
│
├── 0001002
│ ├── 0001002_000001.png
│ ├── 0001002_000002.png
│ ├── …

可以用下面的命令查看当前路径的文件占了多少空间

du -sh

4 训练

3.1 训练的相关配置文件

3.1.1 SLOWFAST_32x2_R101_50_50s.yaml

configs/AVA/c2/SLOWFAST_32x2_R101_50_50s.yaml

TRAIN:ENABLE: TrueDATASET: avaBATCH_SIZE: 4EVAL_PERIOD: 1CHECKPOINT_PERIOD: 1AUTO_RESUME: TrueCHECKPOINT_FILE_PATH: /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50.pkl CHECKPOINT_TYPE: pytorch
DATA:NUM_FRAMES: 32SAMPLING_RATE: 2TRAIN_JITTER_SCALES: [256, 320]TRAIN_CROP_SIZE: 224TEST_CROP_SIZE: 256INPUT_CHANNEL_NUM: [3, 3]PATH_TO_DATA_DIR: '/root/autodl-tmp/SCB-ava-Dataset4'
DETECTION:ENABLE: TrueALIGNED: FalseAVA:FRAME_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frames'FRAME_LIST_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frame_lists'ANNOTATION_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/annotations'DETECTION_SCORE_THRESH: 0.8TRAIN_PREDICT_BOX_LISTS: ["ava_train_v2.2.csv","person_box_67091280_iou90/ava_detection_train_boxes_and_labels_include_negative_v2.2.csv",]TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]FULL_TEST_ON_VAL: TrueSLOWFAST:ALPHA: 4BETA_INV: 8FUSION_CONV_CHANNEL_RATIO: 2FUSION_KERNEL_SZ: 5
RESNET:ZERO_INIT_FINAL_BN: TrueWIDTH_PER_GROUP: 64NUM_GROUPS: 1DEPTH: 101TRANS_FUNC: bottleneck_transformSTRIDE_1X1: FalseNUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]INSTANTIATION: dot_productPOOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:USE_PRECISE_STATS: FalseNUM_BATCHES_PRECISE: 200
SOLVER:MOMENTUM: 0.9WEIGHT_DECAY: 1e-7OPTIMIZING_METHOD: sgd
MODEL:NUM_CLASSES: 80ARCH: slowfastMODEL_NAME: SlowFastLOSS_FUNC: bceDROPOUT_RATE: 0.5HEAD_ACT: sigmoid
TEST:ENABLE: TrueDATASET: avaBATCH_SIZE: 8
DATA_LOADER:NUM_WORKERS: 2PIN_MEMORY: True
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: '/root/autodl-tmp/'

3.1.2 ava_action_list_v2.2_for_activitynet_2019.pbtxt

SCB-ava-Dataset4/annotations/ava_action_list_v2.2_for_activitynet_2019.pbtxt

item {name: "hand-raising"id: 1
}
item {name: "reading"id: 2
}
item {name: "writing"id: 3
}

3.1.3 ava_included_timestamps_v2.2.txt

SCB-ava-Dataset4/annotations/ava_included_timestamps_v2.2.txt

0002

3.1.4 ava_train/test_excluded_timestamps_v2.2.csv

空
SCB-ava-Dataset4/annotations/ava_test_excluded_timestamps_v2.2.csv
SCB-ava-Dataset4/annotations/ava_train_excluded_timestamps_v2.2.csv

3.2 训练

python tools/run_net.py \--cfg configs/AVA/c2/SLOWFAST_32x2_R101_50_50s.yaml 

3.3 相关报错

3.3.1 报错1

报错：
Traceback (most recent call last):
File “tools/run_net.py”, line 52, in
main()
File “tools/run_net.py”, line 27, in main
launch_job(cfg=cfg, init_method=args.init_method, func=train)
File “/root/slowfast/slowfast/utils/misc.py”, line 430, in launch_job
func(cfg=cfg)
File “/root/slowfast/tools/train_net.py”, line 681, in train
train_epoch(
File “/root/slowfast/tools/train_net.py”, line 276, in train_epoch
del inputs
UnboundLocalError: local variable ‘inputs’ referenced before assignment

3.3.1 解决1

解决：
https://github.com/facebookresearch/SlowFast/issues/547

更改batch_size的大小，报错是因为batch_size大于视频数量

3.3.2 报错2

报错：
Traceback (most recent call last):
File “tools/run_net.py”, line 52, in
main()
File “tools/run_net.py”, line 27, in main
launch_job(cfg=cfg, init_method=args.init_method, func=train)
File “/root/slowfast/slowfast/utils/misc.py”, line 430, in launch_job
func(cfg=cfg)
File “/root/slowfast/tools/train_net.py”, line 681, in train
train_epoch(
File “/root/slowfast/tools/train_net.py”, line 110, in train_epoch
optim.set_lr(optimizer, lr)
File “/root/slowfast/slowfast/models/optimizer.py”, line 267, in set_lr
param_group[“lr”] = new_lr * param_group[“layer_decay”]
KeyError: ‘layer_decay’

3.3.2 解决2

解决：
https://github.com/facebookresearch/SlowFast/issues/654

slowfast/slowfast/models/optimizer.py

def set_lr(optimizer, new_lr):"""Sets the optimizer lr to the specified value.Args:optimizer (optim): the optimizer using to optimize the current network.new_lr (float): the new learning rate to set."""for param_group in optimizer.param_groups:try:param_group["lr"] = new_lr * param_group["layer_decay"]except:param_group["lr"] = new_lr * 1.0

3.3.3 报错3

错误：

/root/slowfast/ava_evaluation/metrics.py:41: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.or labels.dtype != np.bool
Traceback (most recent call last):File "tools/run_net.py", line 52, in <module>main()File "tools/run_net.py", line 27, in mainlaunch_job(cfg=cfg, init_method=args.init_method, func=train)File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_jobfunc(cfg=cfg)File "/root/slowfast/tools/train_net.py", line 748, in traineval_epoch(File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_contextreturn func(*args, **kwargs)File "/root/slowfast/tools/train_net.py", line 420, in eval_epochval_meter.log_epoch_stats(cur_epoch)File "/root/slowfast/slowfast/utils/meters.py", line 235, in log_epoch_statsself.finalize_metrics(log=False)File "/root/slowfast/slowfast/utils/meters.py", line 208, in finalize_metricsself.full_map = evaluate_ava(File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 162, in evaluate_avaresults = run_evaluation(categories, groundtruth, detections, excluded_keys)File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 238, in run_evaluationmetrics = pascal_evaluator.evaluate()File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 307, in evaluate) = self._evaluation.evaluate()File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 799, in evaluateprecision, recall = metrics.compute_precision_recall(File "/root/slowfast/ava_evaluation/metrics.py", line 41, in compute_precision_recallor labels.dtype != np.boolFile "/root/miniconda3/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

3.3.3 解决3

成功解决AttributeError: module ‘numpy‘ has no attribute ‘bool‘.
pip uninstall numpy

pip install numpy==1.23.2

报错：

Traceback (most recent call last):File "tools/run_net.py", line 52, in <module>main()File "tools/run_net.py", line 27, in mainlaunch_job(cfg=cfg, init_method=args.init_method, func=train)File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_jobfunc(cfg=cfg)File "/root/slowfast/tools/train_net.py", line 748, in traineval_epoch(File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_contextreturn func(*args, **kwargs)File "/root/slowfast/tools/train_net.py", line 420, in eval_epochval_meter.log_epoch_stats(cur_epoch)File "/root/slowfast/slowfast/utils/meters.py", line 235, in log_epoch_statsself.finalize_metrics(log=False)File "/root/slowfast/slowfast/utils/meters.py", line 208, in finalize_metricsself.full_map = evaluate_ava(File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 162, in evaluate_avaresults = run_evaluation(categories, groundtruth, detections, excluded_keys)File "/root/slowfast/slowfast/utils/ava_eval_helper.py", line 221, in run_evaluationpascal_evaluator.add_single_detected_image_info(File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 275, in add_single_detected_image_infoself._evaluation.add_single_detected_image_info(File "/root/slowfast/ava_evaluation/object_detection_evaluation.py", line 710, in add_single_detected_image_info) = self.per_image_eval.compute_object_detection_metrics(File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 114, in compute_object_detection_metricsscores, tp_fp_labels = self._compute_tp_fp(File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 210, in _compute_tp_fpscores, tp_fp_labels = self._compute_tp_fp_for_single_class(File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 310, in _compute_tp_fp_for_single_class) = self._get_overlaps_and_scores_box_mode(File "/root/slowfast/ava_evaluation/per_image_evaluation.py", line 253, in _get_overlaps_and_scores_box_modegt_non_group_of_boxlist = np_box_list.BoxList(File "/root/slowfast/ava_evaluation/np_box_list.py", line 58, in __init__raise ValueError(
ValueError: Invalid box data. data must be a numpy array of N*[y_min, x_min, y_max, x_max]

4 测试

4.1 配置文件

slowfast/configs/AVA/c2/SLOWFAST_32x2_R101_50_50s2.yaml

TRAIN:ENABLE: FalseDATASET: avaBATCH_SIZE: 4EVAL_PERIOD: 1CHECKPOINT_PERIOD: 1AUTO_RESUME: TrueCHECKPOINT_FILE_PATH: /root/autodl-nas/slowfast/SLOWFAST_32x2_R101_50_50.pkl CHECKPOINT_TYPE: pytorchDATA:NUM_FRAMES: 32SAMPLING_RATE: 2TRAIN_JITTER_SCALES: [256, 320]TRAIN_CROP_SIZE: 224TEST_CROP_SIZE: 256INPUT_CHANNEL_NUM: [3, 3]PATH_TO_DATA_DIR: '/root/autodl-tmp/SCB-ava-Dataset4'
DETECTION:ENABLE: TrueALIGNED: FalseAVA:FRAME_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frames'FRAME_LIST_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/frame_lists'ANNOTATION_DIR: '/root/autodl-tmp/SCB-ava-Dataset4/annotations'DETECTION_SCORE_THRESH: 0.8TRAIN_PREDICT_BOX_LISTS: ["ava_train_v2.2.csv","person_box_67091280_iou90/ava_detection_train_boxes_and_labels_include_negative_v2.2.csv",]TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]FULL_TEST_ON_VAL: TrueSLOWFAST:ALPHA: 4BETA_INV: 8FUSION_CONV_CHANNEL_RATIO: 2FUSION_KERNEL_SZ: 5
RESNET:ZERO_INIT_FINAL_BN: TrueWIDTH_PER_GROUP: 64NUM_GROUPS: 1DEPTH: 101TRANS_FUNC: bottleneck_transformSTRIDE_1X1: FalseNUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]INSTANTIATION: dot_productPOOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:USE_PRECISE_STATS: FalseNUM_BATCHES_PRECISE: 200
SOLVER:MOMENTUM: 0.9WEIGHT_DECAY: 1e-7OPTIMIZING_METHOD: sgd
MODEL:NUM_CLASSES: 80ARCH: slowfastMODEL_NAME: SlowFastLOSS_FUNC: bceDROPOUT_RATE: 0.5HEAD_ACT: sigmoid
TEST:ENABLE: TrueDATASET: avaBATCH_SIZE: 1CHECKPOINT_FILE_PATH: /root/autodl-tmp/slowfast_train/checkpoints/checkpoint_epoch_00034.pyth
DATA_LOADER:NUM_WORKERS: 2PIN_MEMORY: True
NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: '/root/autodl-tmp/'

4.2 测试

python tools/run_net.py \--cfg configs/AVA/c2/SLOWFAST_32x2_R101_50_50s2.yaml  \

错误：

Traceback (most recent call last):File "tools/run_net.py", line 52, in <module>main()File "tools/run_net.py", line 37, in mainlaunch_job(cfg=cfg, init_method=args.init_method, func=test)File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_jobfunc(cfg=cfg)File "/root/slowfast/tools/test_net.py", line 252, in testtest_meter = perform_test(test_loader, model, test_meter, cfg, writer)File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_contextreturn func(*args, **kwargs)File "/root/slowfast/tools/test_net.py", line 129, in perform_testtest_meter.iter_toc()File "/root/slowfast/slowfast/utils/meters.py", line 159, in iter_tocself.iter_timer.pause()File "/root/miniconda3/lib/python3.8/site-packages/fvcore/common/timer.py", line 30, in pauseraise ValueError("Trying to pause a Timer that is already paused!")
ValueError: Trying to pause a Timer that is already paused!

解决：
https://github.com/facebookresearch/SlowFast/issues/599#ref-issue-1441689338
https://github.com/facebookresearch/SlowFast/issues/623

修改test_net.py中的内容，加两组try except

if cfg.NUM_GPUS > 1:preds, labels, video_idx = du.all_gather([preds, labels, video_idx])if cfg.NUM_GPUS:preds = preds.cpu()labels = labels.cpu()video_idx = video_idx.cpu()try:test_meter.iter_toc()except:passif not cfg.VIS_MASK.ENABLE:# Update and log stats.test_meter.update_stats(preds.detach(), labels.detach(), video_idx.detach())try:test_meter.log_iter_stats(cur_iter)except:test_meter.log_iter_stats(None,cur_iter)test_meter.iter_tic()

报错：

Traceback (most recent call last):File "tools/run_net.py", line 52, in <module>main()File "tools/run_net.py", line 37, in mainlaunch_job(cfg=cfg, init_method=args.init_method, func=test)File "/root/slowfast/slowfast/utils/misc.py", line 430, in launch_jobfunc(cfg=cfg)File "/root/slowfast/tools/test_net.py", line 257, in testtest_meter = perform_test(test_loader, model, test_meter, cfg, writer)File "/root/miniconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_contextreturn func(*args, **kwargs)File "/root/slowfast/tools/test_net.py", line 166, in perform_testtest_meter.finalize_metrics()File "/root/slowfast/slowfast/utils/meters.py", line 200, in finalize_metricsall_ori_boxes = torch.cat(self.all_ori_boxes, dim=0)
RuntimeError: Sizes of tensors must match except in dimension 0. Got 5 and 80 in dimension 1 (The offending index is 1)

4.2 demo测试

4.2.1 配置文件

slowfast/demo/AVA/SLOWFAST_32x2_R101_50_50s2.yaml

TRAIN:ENABLE: FalseDATASET: avaBATCH_SIZE: 1EVAL_PERIOD: 1CHECKPOINT_PERIOD: 1AUTO_RESUME: TrueCHECKPOINT_FILE_PATH: /root/slowfast_train/checkpoints/checkpoint_epoch_00034.pyth  #path to pretrain modelCHECKPOINT_TYPE: pytorch
DATA:NUM_FRAMES: 16SAMPLING_RATE: 2TRAIN_JITTER_SCALES: [256, 320]TRAIN_CROP_SIZE: 224TEST_CROP_SIZE: 256INPUT_CHANNEL_NUM: [3, 3]
DETECTION:ENABLE: TrueALIGNED: False
AVA:BGR: FalseDETECTION_SCORE_THRESH: 0.8TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:ALPHA: 4BETA_INV: 8FUSION_CONV_CHANNEL_RATIO: 2FUSION_KERNEL_SZ: 5
RESNET:ZERO_INIT_FINAL_BN: TrueWIDTH_PER_GROUP: 64NUM_GROUPS: 1DEPTH: 101TRANS_FUNC: bottleneck_transformSTRIDE_1X1: FalseNUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]INSTANTIATION: dot_productPOOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:USE_PRECISE_STATS: FalseNUM_BATCHES_PRECISE: 200
SOLVER:MOMENTUM: 0.9WEIGHT_DECAY: 1e-7OPTIMIZING_METHOD: sgd
MODEL:NUM_CLASSES: 80ARCH: slowfastMODEL_NAME: SlowFastLOSS_FUNC: bceDROPOUT_RATE: 0.5HEAD_ACT: sigmoid
TEST:ENABLE: FalseDATASET: avaBATCH_SIZE: 1
DATA_LOADER:NUM_WORKERS: 1PIN_MEMORY: TrueNUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: .
#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:ENABLE: TrueLABEL_FILE_PATH: "demo/AVA/student_behaviors.json" # Add local label file path here.INPUT_VIDEO: "/root/autodl-tmp/0_0_0.mp4"OUTPUT_FILE: "/root/autodl-tmp/0_1_1.mp4"DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"DETECTRON2_WEIGHTS: "/root/autodl-nas/slowfast/model_final_280758.pkl"

4.2.2 student_behaviors.json

slowfast/demo/AVA/student_behaviors.json

{"hand-raising": 0, "reading": 1, "writing": 2}

4.2.2 demo测试

cd /root/slowfast/
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50s2.yaml

5 在非云平台安装（本地机器上）

相关参考

[深度学习][原创]slowfast的demo跑起来的正确方法

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-开山篇

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-流程篇

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-配置文件篇

[深度学习][原创]使用labelImg+yolov5完成所有slowfast时空动作检测项目-训练测试篇

[深度学习][原创]mmaction2时空动作检测过滤成自己想要的类别

三分钟快速安装 facebookresearch SlowFast

PyTorchVideo SlowFast