PaddleDetection训练目标检测模型

本文主要是介绍PaddleDetection训练目标检测模型，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

流程：
标注-训练-预测-标注-训练

一，安装标注软件

标注文件保存为voc格式
1，labelimg的安装(python>3.0)

pip install labelImg
labelimg

打包成单独的exe文件

先进入labelimg包的目录

C:\Users\Administrator\AppData\Local\Programs\Python\Python310\Lib\site-packages\labelImg

不知道的可以用

# 查询pip的目录
where pip

pip install pyinstaller
pyinstaller --hidden-import=pyqt5 --hidden-import=lxml -F -n "labelImg" -c labelImg.py -p ./libs -p ./

打包之后在同目录的dist下面

2，labelme的安装
标注文件保存为json格式

conda create --name=labelme python=3
conda activate labelme
pip install labelme
labelme

附带
voc转json代码：

# --- utf-8 ---# --- function: 将Labeling标注的格式转化为Labelme标注格式，并读取imageData ---import os
import glob
import shutil
import xml.etree.ElementTree as ET
import json
from base64 import b64encode
from json import dumps
def get(root, name):return root.findall(name)
# 检查读取xml文件是否出错def get_and_check(root, name, length):vars = root.findall(name)if len(vars) == 0:raise NotImplementedError('Can not fing %s in %s.' % (name, root.tag))if length > 0 and len(vars) != length:raise NotImplementedError('The size of %s is supposed to be %d, but is %d.' % (name, length, len(vars)))if length == 1:vars = vars[0]return varsdef convert(xml_file, json_file, save_dir, name, data):# 定义通过Labelme标注后生成的json文件json_dict = {"version": "3.16.2","flags": {},"shapes": [],"imagePath": "","imageData": None,"imageHeight": 0,"imageWidth": 0}# img_name = xml_file.split('.')[0]img_path = name + '.jpg'json_dict["imagePath"] = img_pathtree = ET.parse(xml_file)  # 读取xml文件root = tree.getroot()size = get_and_check(root, 'size', 1)  # 读取xml中<>size<>字段中的内容# 读取二进制图片，获得原始字节码with open(data, 'rb') as jpg_file:byte_content = jpg_file.read()# 把原始字节码编码成base64字节码base64_bytes = b64encode(byte_content)# 把base64字节码解码成utf-8格式的字符串base64_string = base64_bytes.decode('utf-8')# 用字典的形式保存数据json_dict["imageData"] = base64_string# 获取图片的长宽信息width = int(get_and_check(size, 'width', 1).text)height = int(get_and_check(size, 'height', 1).text)json_dict["imageHeight"] = heightjson_dict["imageWidth"] = width# 当标注中有多个目标时全部读取出来for obj in get(root, 'object'):# 定义图片的标注信息img_mark_inf = {"label": "", "points": [], "group_id": None, "shape_type": "rectangle", "flags": {}}category = get_and_check(obj, 'name', 1).text  # 读取当前目标的类别img_mark_inf["label"] = categorybndbox = get_and_check(obj, 'bndbox', 1)  # 获取标注宽信息xmin = float(get_and_check(bndbox, 'xmin', 1).text)ymin = float(get_and_check(bndbox, 'ymin', 1).text)xmax = float(get_and_check(bndbox, 'xmax', 1).text)ymax = float(get_and_check(bndbox, 'ymax', 1).text)img_mark_inf["points"].append([xmin, ymin])img_mark_inf["points"].append([xmax, ymax])# print(img_mark_inf["points"])json_dict["shapes"].append(img_mark_inf)# print("{}".format(json_dict))save = save_dir +'/'+ json_file  # json文件的路径地址json_fp = open(save, 'w')  #json_str = json.dumps(json_dict, indent=4)  # 缩进，不需要的可以将indent=4去掉json_fp.write(json_str)  # 保存json_fp.close()# print("{}, {}".format(width, height))def do_transformation(xml_dir, save_path):cnt = 0for fname in os.listdir(xml_dir):name = fname.split(".")[0]  # 获取图片名字path = os.path.join(xml_dir, fname)  # 文件路径save_json_name = name + '.json'data = img +'/'+ name + '.jpg'  # xml文件对应的图片路径convert(path, save_json_name, save_path, name, data)cnt += 1if __name__ == '__main__':img = r"D:\zsh\biaozhu\basketball_count\F_field\labelimg\voc\JPEGImages"    # xml对应图片文件夹xml_path = r"D:\zsh\biaozhu\basketball_count\F_field\labelimg\voc\Annotations"    # xml文件夹save_json_path = r"D:\zsh\biaozhu\basketball_count\F_field\labelimg\voc\json"    # 存放json文件夹if not os.path.exists(save_json_path):os.makedirs(save_json_path)do_transformation(xml_path, save_json_path)# xml = "2007_000039.xml"# xjson = "2007_000039.json"# convert(xml, xjson)

二，数据标注和清洗

1，用labelimg标注voc格式的标注数据
2，生成数据集，分为训练集和验证集
生成脚本：

import glob
import random
import multiprocessingdef process_file(file_name):output = glob.glob('dataset/' + dir + '/images/' + file_name + '.???')[0]output.replace('\\', '/').split('/')[-1]return './images/' + output + ' ./annotations/' + file_name + '.xml\n'dir = '6.19_gray_court_voc'
num_processes = multiprocessing.cpu_count() * 1.5  # 指定使用的进程数为 CPU 数量的两倍path = 'dataset/' + dir
tmp = []
for i in glob.glob(path + '/annotations/*.xml'):name = i.replace('\\', '/').split('/')[-1][:-4]tmp.append(name)
random.shuffle(tmp)train = tmp[:int(len(tmp) * 0.8)]
val = tmp[int(len(tmp) * 0.8):]
print('train:', len(train), 'val:', len(val))# Create a pool of worker processes with specified number of processes
pool = multiprocessing.Pool(processes=num_processes)with open('dataset/' + dir + '/train.txt', 'w', encoding='utf-8') as f:# Process train data using multiple processesresults = pool.map(process_file, train)f.writelines(results)with open('dataset/' + dir + '/valid.txt', 'w', encoding='utf-8') as f:# Process validation data using multiple processesresults = pool.map(process_file, val)f.writelines(results)# Close the pool of worker processes
pool.close()
pool.join()

运行后目录下生成train.txt，val.txt
创建label_list.txt，写入标注数据的类别
目录：

person
---images---xx.jpg
---annotation---xx.xml
---train.txt
---val.txt
---label_list.txt

github主页：一些常用的清洗数据的脚本

https://github.com/zsh123abc/Data_analysis_related_py

三，安装PaddleDetection环境

1，安装docker

# 下载地址
https://desktop.docker.com/win/main/amd64/Docker%20Desktop%20Installer.exe

安装到除c盘外的其他盘
docker的默认安装路径：C:\Program Files\Docker
用管理员打开cmd

# 通过软连接把实际储存移到E盘中
mklink C:\Program Files\Docker  E:\Docker

双击运行docker.exe

2，拉paddle镜像

docker pull paddlepaddle/paddle:2.5.2-gpu-cuda12.0-cudnn8.9-trt8.6

3，下载PPaddleDetection 2.6 源码

git clone --branch 2.6  https://github.com/PaddlePaddle/PaddleDetection.git

3，gpu启动docker容器

docker run -it --privileged=true --name paddle_test --gpus all -d  -p 8040:8040 -v E:\PaddleDetection:/PaddleDetection paddlepaddle/paddle:2.5.2-gpu-cuda12.0-cudnn8.9-trt8.6 /bin/bash

四，修改配置文件，本文选择的是 PP-PicoDet算法

把标注数据集的文件夹放到

/PaddleDetection/dataset/voc

修改配置文件：

cd /PaddleDetection
# cp一份
cp configs/datasets/voc.yml configs/datasets/test_voc.yml
vi configs/datasets/test_voc.yml

test_voc.yml

metric: VOC
map_type: 11point
num_classes: 20 #类别数量TrainDataset:name: VOCDataSetdataset_dir: dataset/voc #数据集目录anno_path: trainval.txtlabel_list: label_list.txtdata_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']EvalDataset:name: VOCDataSetdataset_dir: dataset/voc #数据集目录anno_path: test.txtlabel_list: label_list.txtdata_fields: ['image', 'gt_bbox', 'gt_class', 'difficult']TestDataset:name: ImageFolderanno_path: dataset/voc/label_list.txt #类别名文件

修改配置文件：

/PaddleDetection/configs/picodet/legacy_model/picodet_s_320_voc.yml

picodet_s_320_voc.yml

_BASE_: ['../../datasets/voc.yml','../../runtime.yml','_base_/picodet_esnet.yml','_base_/optimizer_300e.yml','_base_/picodet_320_reader.yml',
]pretrain_weights: https://paddledet.bj.bcebos.com/models/pretrained/ESNet_x0_75_pretrained.pdparams
weights: output/picodet_s_320_coco/model_final
find_unused_parameters: True
use_ema: true
cycle_epoch: 40
snapshot_epoch: 10ESNet:scale: 0.75feature_maps: [4, 11, 14]act: hard_swishchannel_ratio: [0.875, 0.5, 0.5, 0.5, 0.625, 0.5, 0.625, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5]CSPPAN:out_channels: 96PicoHead:conv_feat:name: PicoFeatfeat_in: 96feat_out: 96num_convs: 2num_fpn_stride: 4norm_type: bnshare_cls_reg: Truefeat_in_chan: 96EvalReader:collate_batch: false

五，训练模型

python -m tools/train.py -c configs/picodet/legacy_model/picodet_s_320_voc.yml

单机多卡

python -m paddle.distributed.launch \
--selected_gpus='0,1,2' \
--log_dir=./test_voc/ \
tools/train.py \
-c configs/picodet/legacy_model/picodet_s_320_voc.yml \
--use_vdl=true \
--vdl_log_dir=vdl_dir/scratch_log \
--eval>test_voc.log 2>&1 &

多机多卡

python -m paddle.distributed.launch \--cluster_node_ips=192.168.100.1,192.168.100.2 \--node_ip=192.168.100.1 \--started_port=6170 \--selected_gpus=0 \--log_dir=./ping-pang \tools/train.py -c configs/picodet/legacy_model/picodet_s_320_voc.yml --use_vdl=true --vdl_log_dir=vdl_dir/scratch_log  --eval>scratch.log 2>&1&

可视化训练，visualdl安装

pip install --upgrade visualdl

在训练命令中加入

--use_vdl=true \
--vdl_log_dir=vdl_dir/scratch_log \

启用，指定log文件夹

visualdl --logdir ./scratch_log --port 8080

浏览器输入

http://127.0.0.1:8080

六，训练完成之后导出模型

python tools/export_model.py \
-c configs/picodet/legacy_model/picodet_s_320_voc.yml \
-o weights=output/test_voc/best_model.pdparams \
--output_dir=inference_model

优化模型，转换格式，方便移动端部署

paddle_lite_opt --valid_targets=arm \
--model_file=inference_model/test_voc/model.pdmodel \
--param_file=inference_model/test_voc/model.pdiparams \
--optimize_out=inference_model/test_voc/test_voc

七，模型预测

图片预测

python deploy/python/infer.py \
--model_dir=inference_model/test_voc \
--output=output/test_img_output \
--image_file=output/img_test.jpg \
--threshold=0.5 \
--device=GPU

多张图片预测

python deploy/python/infer.py \
--model_dir=inference_model/test_voc \
--output=output/test_images_output \
--image_dir=output/images_test \
--threshold=0.5 
--device=GPU

视频预测

python deploy/python/infer.py \
--model_dir=inference_model/test_voc \
--video_file=dataset/test_video \
--output=output/test_video_output \
--threshold=0.5 \
--device=GPU

附带 批量运行视频脚本

find dataset/court_video_test/628_video/ -type f -name "*.mp4" -exec sh -c 'python deploy/python/infer.py --model_dir=inference_model/test_voc --video_file="{}" --output=output/test_video_output --threshold=0.5 --device=GPU' \;

八，半自动标注：

生成预标注 --save_results 保存推理结果，需要修改推理源码，保存推理结果至json文件

python deploy/python/infer.py \
--model_dir=inference_model/test_voc \
--output=output/test_json_output \
--image_dir=dataset/test_images \
--threshold=0.5 \
--device=GPU \
--save_results

九，移动端部署

参考paddle github官网项目

git clone https://github.com/PaddlePaddle/Paddle-Lite-Demo/tree/develop/object_detection/android/app/cxx/picodet_detection_demo

1，下载 Android Stuido
官网地址：https://developer.android.com/studio

2，下载JDK，SDK，NDK，CMake
参考网址：https://developer.android.com/studio/projects/install-ndk?hl=zh-cn

这篇关于PaddleDetection训练目标检测模型的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---