SlowFast-入门1-动作识别-部署与测试

SlowFast-部署与测试

SlowFast系列：
SlowFast-入门1-动作识别-部署与测试
Slowfast-入门2-学论文【SlowFast Networks for Video Recognition】

简介

GitHub：
https://github.com/facebookresearch/SlowFast
官方给的运行示例图是这样的。
特点是：识别的动作为原子动作；每个人同一时刻可能被识别出多个动作。

安装部署

部署平台：极链AI云

部署机器：Tesla V100
部署环境：Pytorch 1.6.0, Python 3.7, CUDA 10.2
这里注意！！！！！！！！
选择Pytorch框架版本可以为1.6.0的机器，这样才能选择到3.7的Python。
由于Detectron2需要Pytorch>=1.7，所以之后会需要再升Pytorch的版本。
但一定要选择Python为3.7！！！！！！！！
要先保证Python为3.7！！！！！！！！

安装过程：

pip install 'git+https://github.com/facebookresearch/fvcore'
pip install simplejson
conda install av -c conda-forge
y
pip install -U iopath
pip install psutil
pip install moviepy
pip install tensorboard
pip install pytorchvideo
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
git clone https://github.com/facebookresearch/detectron2.git
pip install -e detectron2
git clone https://github.com/facebookresearch/SlowFast.git
cd SlowFast
python setup.py build develop

安装结束后，成功安装好了fvcore、detectron2、slowfast。
有提示以下错误。
error: Could not find suitable distribution for Requirement.parse(‘PIL’)
但是我看安装教程中，没有特意提到这个PIL库，而且用pip list查看的时候，已经有Pillow库了。
Pillow、PIL更多了解
然后后续的推断测试中也没影响，爷累了，所以先不管他。

安装完后，再用pip list查看：

fvcore                             0.1.5
detectron2                         0.5                 /root/detectron2
sklearn                            0.0
slowfast                           1.0                 /root/SlowFast
torch                              1.7.1+cu110
torchaudio                         0.7.2
torchvision                        0.8.2+cu110

测试

基本安装部署了，接下来要怎么用呢。
Getting Started with PySlowFast
This document provides a brief intro of launching jobs in PySlowFast for training and testing. Before launching any job, make sure you have properly installed the PySlowFast following the instruction in README.md and you have prepared the dataset following DATASET.md with the correct format.

准备pkl文件并上传

看了一下DATASET.md里的介绍，有Kinetics（132GB）、AVA（好像是500GB+）、Charades、Something-Something V2。
下载、处理，都很艰难。
博主试过按照AVA的准备方法准备。下载、裁剪、提取帧、下载标注文件（这个还好）。原始视频19GB多的，整套流程下来电脑跑了两天多，昼夜不停的跑。
【以后有机会再写对这个部分AVA数据集的处理训练方法】

所以，先用人家官方给的，玩玩。在MODEL_ZOO.md,下载了如下的模型。
（这个表的解释，之后在解读论文的时候，可能再解释解释）
下载下来的文件名为：【SLOWFAST_32x2_R101_50_50.pkl】
下载后上传到云端。我是上传到/root/SlowFast路径。

上传ava.json文件

制作ava.json文件内容如下，记录了80个动作类别。
上传至/root/SlowFast/demo/AVA路径下。

{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, "paint": 38, "play board game": 39, "play musical instrument": 40, "play with pets": 41, "point to (an object)": 42, "press": 43, "pull (an object)": 44, "push (an object)": 45, "put down": 46, "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60, "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, "give/serve (an object) to (a person)": 64, "grab (a person)": 65, "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}

准备yaml文件

在/root/SlowFast/demo/AVA路径下，对【SLOWFAST_32x2_R101_50_50.yaml】文件进行修改：

1. TRAIN下的CHECKPOINT_FILE_PATH，修改成上一步中下载并上传到云端的pkl文件的位置。
2. TENSORBOARD、MODEL_VIS、TOPK这三行注释掉。
3. 在DEMO下，注释掉WEBCAM，增加INPUT_VIDEO、OUTPUT_FILE，赋值的路径、名称可自定义。
4. 在DEMO下，LABEL_FILE_PATH写ava.json的位置。
   

准备素材

准备mp4的测试素材文件，上传到对应的文件夹。
我在/root/SlowFast路径下，创建了Video_Input、Video_Output文件夹。
在Video_Input文件夹下放了个命名为1.mp4的素材。

运行

在/root/SlowFast路径下，执行：

python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yaml

运行结果

这次运行了三个视频片段。
第一个视频片段来源于《初来乍到》美剧。
第二、三个视频是成龙的打戏，下载自b站。虽然有一个视频有点糊，但是能识别出martial art就很感人。

后记——不重要的痛苦的过程

一开始我在这样的环境里配置：
Pytorch 1.7.0, Python 3.8.8, CUDA 11.0
与上面步骤是一样的（这个不需要升pytorch的那句），
最后安装slowfast的时候，也会出现PIL的问题
但是重点不是这个
运行推断的时候会出现：

Traceback (most recent call last):File "tools/run_net.py", line 6, in <module>from slowfast.utils.misc import launch_jobFile "/root/SlowFast/slowfast/utils/misc.py", line 21, in <module>from slowfast.models.batchnorm_helper import SubBatchNorm3dFile "/root/SlowFast/slowfast/models/__init__.py", line 6, in <module>from .video_model_builder import ResNet, SlowFast  # noqaFile "/root/SlowFast/slowfast/models/video_model_builder.py", line 18, in <module>from . import head_helper, resnet_helper, stem_helperFile "/root/SlowFast/slowfast/models/head_helper.py", line 8, in <module>from detectron2.layers import ROIAlign
ImportError: cannot import name 'ROIAlign' from 'detectron2.layers' (unknown location)

感谢这位博主的这篇博客：
https://blog.csdn.net/WhiffeYF/article/details/113527759
让我还是回去从Python3.7开始配，即使安装detectron2的时候，要先将Pytorch升级为1.7.0，但是就不会出现这个难搞的错误了。

还有升级为Pytorch1.7的时候，用的是这句：

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

因为当时，单纯升级pytorch为1.7，不同时将cuda10.2升级为11的时候，会出现Could not run ‘torchvision::nms’这个问题。
幸好这个我之前遇到过解决过了。
在这篇博客：
https://blog.csdn.net/weixin_41793473/article/details/118574493?spm=1001.2014.3001.5501

前面提到的PIL的那个问题：

Installed /root/SlowFast
Processing dependencies for slowfast==1.0
Searching for sklearn
Reading https://pypi.org/simple/sklearn/
Downloading https://files.pythonhosted.org/packages/1e/7a/dbb3be0ce9bd5c8b7e3d87328e79063f8b263b2b1bfa4774cb1147bfcd3f/sklearn-0.0.tar.gz#sha256=e23001573aa194b834122d2b9562459bf5ae494a2d59ca6b8aa22c85a44c0e31
Best match: sklearn 0.0
Processing sklearn-0.0.tar.gz
Writing /tmp/easy_install-bhbw6vsc/sklearn-0.0/setup.cfg
Running sklearn-0.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-bhbw6vsc/sklearn-0.0/egg-dist-tmp-2gs9d87i
file wheel-platform-tag-is-broken-on-empty-wheels-see-issue-141.py (for module wheel-platform-tag-is-broken-on-empty-wheels-see-issue-141) not found
file wheel-platform-tag-is-broken-on-empty-wheels-see-issue-141.py (for module wheel-platform-tag-is-broken-on-empty-wheels-see-issue-141) not found
file wheel-platform-tag-is-broken-on-empty-wheels-see-issue-141.py (for module wheel-platform-tag-is-broken-on-empty-wheels-see-issue-141) not found
warning: install_lib: 'build/lib' does not exist -- no Python modules to installcreating /opt/conda/lib/python3.8/site-packages/sklearn-0.0-py3.8.egg
Extracting sklearn-0.0-py3.8.egg to /opt/conda/lib/python3.8/site-packages
Adding sklearn 0.0 to easy-install.pth fileInstalled /opt/conda/lib/python3.8/site-packages/sklearn-0.0-py3.8.egg
Searching for PIL
Reading https://pypi.org/simple/PIL/
No local packages or working download links found for PIL
error: Could not find suitable distribution for Requirement.parse('PIL')