装CTC

2024-06-24 12:38
文章标签 ctc

本文主要是介绍装CTC,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

https://github.com/SeanNaren/warp-ctc

下载下来,传到服务器上

解压

更名

cd warp-ctc
mkdir build; cd build
cmake ..
make

上面这些过程缺啥装啥,失败了,把缺的东西装上,重复下面这个过程,直到成功

cd build

make clean

cmake .. make

 

下一步开始安装

cd pytorch_binding
python setup.py install

报下面这个错:

generating build/warpctc_pytorch/_warp_ctc/__warp_ctc.c
(already up-to-date)
not modified: 'build/warpctc_pytorch/_warp_ctc/__warp_ctc.c'
running install
running bdist_egg
running egg_info
writing warpctc_pytorch.egg-info/PKG-INFO
writing dependency_links to warpctc_pytorch.egg-info/dependency_links.txt
writing top-level names to warpctc_pytorch.egg-info/top_level.txt
reading manifest file 'warpctc_pytorch.egg-info/SOURCES.txt'
writing manifest file 'warpctc_pytorch.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying warpctc_pytorch/_warp_ctc/__init__.py -> build/lib.linux-x86_64-3.6/warpctc_pytorch/_warp_ctc
running build_ext
building 'warpctc_pytorch._warp_ctc.__warp_ctc' extension
gcc -pthread -B /home1/fzp/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-pib/include -I/home1/fzp/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home1/fzp/anaconda3/li-I/home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/include -I/home1/fzp/anaconda3/include/python3.6m -c build/warpctc_pyto__warp_ctc.o -std=c++11 -fPIC -std=c99 -DWARPCTC_ENABLE_GPU
cc1: warning: command line option ‘-std=c++11’ is valid for C++/ObjC++ but not for C [enabled by default]
gcc -pthread -B /home1/fzp/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-pib/include -I/home1/fzp/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/TH -I/home1/fzp/anaconda3/li-I/home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/include -I/home1/fzp/anaconda3/include/python3.6m -c /home1/fzp/attenti4-3.6/home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/pytorch_binding/src/binding.o -std=c++11 -fPIC -std=c99 -DWARPCTC_EN
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
cc1plus: warning: command line option ‘-std=c99’ is valid for C/ObjC but not for C++ [enabled by default]
/home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/pytorch_binding/src/binding.cpp: In function ‘int gpu_ctc(THCudaTensor*, 
/home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/pytorch_binding/src/binding.cpp:92:49: error: cannot convert ‘THCudaTensotTensor*, int)’
     int probs_size = THFloatTensor_size(probs, 2);
                                                 ^
/home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/pytorch_binding/src/binding.cpp:105:61: error: invalid conversion from ‘s
     void* gpu_workspace = THCudaMalloc(state, gpu_size_bytes);
                                                             ^
/home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/pytorch_binding/src/binding.cpp:105:61: error: too few arguments to funct
In file included from /home1/fzp/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC/THC.h:4:0,
                 from /home1/fzp/attention-OCR/warp-ctc-pytorch_bindings/pytorch_binding/src/binding.cpp:9:
/home1/fzp/anaconda3/lib/python3.6/site-packages/torch/utils/ffi/../../lib/include/THC/THCGeneral.h:209:21: note: declared h
 THC_API cudaError_t THCudaMalloc(THCState *state, void **ptr, size_t size);
                     ^
error: command 'gcc' failed with exit status 1
解决方案:

改了之后,执行成功

开始测试是否安装成功:

  1. cd /home/xxx/warp-ctc/pytorch_binding/tests

  2. python test_gpu.py

报错1:

Traceback (most recent call last):
  File "test_gpu.py", line 1, in <module>
    import torch
ImportError: No module named torch

  尽管pytorch到我写下这篇文档时已经更新到0.5.0+版本了,但是SeanNaren/warp-ctc项目目前只兼容pytorch 0.4.0版本。所以安装过程中,请不要使用git clone法安装 pytorch,而是选用conda进行安装。我试过0.5+的版本报错,所以最好还是0.4版本

pytorch 0.4.0版本安装

报错2:

Traceback (most recent call last):
  File "test_gpu.py", line 3, in <module>
    import pytest
ImportError: No module named pytest

conda install pytest

装了下列两个东西

   py:     1.4.34-py27_0 
   pytest: 3.2.1-py27_0  

(crnn) [XXXXX@JXQ-240-26-65 tests]$ python test_cpu.py 
===================================================================================================================== test session starts ======================================================================================================================
platform linux2 -- Python 2.7.13, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /export/gpudata/fujingling/projects/warp-ctc/pytorch_binding, inifile: setup.cfg
collected 4 items                                                                                                                                                                                                                                               

test_cpu.py ....

=================================================================================================================== 4 passed in 0.09 seconds ===================================================================================================================

看了其他人的博客:

在build下建一个test.py,加入以下代码,运行并没有报错呢

import torch
from warpctc_pytorch import CTCLoss
ctc_loss = CTCLoss()
# expected shape of seqLength x batchSize x alphabet_size
probs = torch.FloatTensor([[[0.1, 0.6, 0.1, 0.1, 0.1], [0.1, 0.1, 0.6, 0.1, 0.1]]]).transpose(0, 1).contiguous()
labels = torch.IntTensor([1, 2])
label_sizes = torch.IntTensor([2])
probs_sizes = torch.IntTensor([2])
probs.requires_grad_(True)  # tells autograd to compute gradients for probs
cost = ctc_loss(probs, labels, probs_sizes, label_sizes)
cost.backward()

 

于是,我就想着可以跑我的CRNN了,

cd  crnn.pytorch-master

 cp -r /xxx/xxx/XXX/projects/warp-ctc/pytorch_binding/warpctc_pytorch .

然而,万万没想到又报错了,应该是没装成功

 python train.py 
Traceback (most recent call last):
  File "train.py", line 12, in <module>
    from warpctc_pytorch import CTCLoss
  File "/export/gpudata/fujingling/projects/crnn.pytorch-master/warpctc_pytorch/__init__.py", line 6, in <module>
    from ._warp_ctc import *
  File "/export/gpudata/fujingling/projects/crnn.pytorch-master/warpctc_pytorch/_warp_ctc/__init__.py", line 3, in <module>
    from .__warp_ctc import lib as _lib, ffi as _ffi
ImportError: No module named __warp_ctc

 

解决方案:

>>> import torch
print torch.__version__
>>> print torch.__version__
0.1.12

估计是这里的问题

conda uninstall pytorch
Fetching package metadata .........................
Solving package specifications: .

Package plan for package removal in environment /export/gpudata/fujingling/conda/envs/crnn:

The following packages will be REMOVED:

    pytorch:     0.1.12-py27cuda7.5cudnn5.1_1 http://conda.jdfin.local/conda/free
    torchvision: 0.1.8-py27_0                 http://conda.jdfin.local/conda/free

Proceed ([y]/n)? y

又报错,cudnn的版本太低

 conda install pytorch==0.4
Fetching package metadata .........................
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - pytorch ==0.4 -> cudnn 7.*
  - pytorch-gpu -> cudnn ==5.1
Use "conda info <package>" to see the dependencies for each package.
 

卸载,继续装

conda uninstall cudnn
Fetching package metadata .........................
Solving package specifications: .

Package plan for package removal in environment /export/gpudata/fujingling/conda/envs/crnn:

The following packages will be REMOVED:

    cudnn:       5.1-0         http://conda.jdfin.local/conda/free
    pytorch-gpu: 0.1.12-py27_0 http://conda.jdfin.local/conda/free

Proceed ([y]/n)? y

 

重新安装cudnn之前,先查看以下我的cuda版本,看看硬件能不能支持cudnn7.*

cat /usr/local/cuda/version.txt
CUDA Version 8.0.61

下面是CUDA对应能支持的cudnn列表

7.0.5和7.1.3版本的cudnn比较合适我的机器

(crnn) [xxx@xxxxxx crnn.pytorch-master]$ conda install cudnn==7.1.3
Fetching package metadata .........................
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - cudnn ==7.1.3 -> cudatoolkit 8.0*
  - libtorch-gpu -> cudatoolkit ==7.5
Use "conda info <package>" to see the dependencies for each package.

报错,版本不对应

再卸载再安装:

conda uninstall cudatoolkit
Fetching package metadata .........................
Solving package specifications: .

Package plan for package removal in environment /export/gpudata/fujingling/conda/envs/crnn:

The following packages will be REMOVED:

    cudatoolkit:  7.5-2           http://conda.jdfin.local/conda/free
    libtorch-gpu: 0.1.12-0        http://conda.jdfin.local/conda/free
    nccl:         1.3.4-cuda7.5_1 http://conda.jdfin.local/conda/free

一个一个的装回去

 conda install cudatoolkit==8.0
Fetching package metadata .........................
Solving package specifications: .

Package plan for installation in environment /export/gpudata/fujingling/conda/envs/crnn:

The following NEW packages will be INSTALLED:

    cudatoolkit: 8.0-3 http://conda.jdfin.local/conda/free

Proceed ([y]/n)? y

 

装cudnn

 conda install cudnn==7.1.3(省略日志)

装pytorch

conda install pytorch==0.4
Fetching package metadata .........................
Solving package specifications: .

Package plan for installation in environment /export/gpudata/fujingling/conda/envs/crnn:

The following NEW packages will be INSTALLED:

    intel-openmp: 2019.0-118           http://repos.jd.com/conda/main     
    libgcc-ng:    8.2.0-hdf63c60_1     http://repos.jd.com/conda/main     
    libgfortran:  3.0.0-1              http://conda.jdfin.local/conda/free
    libstdcxx-ng: 8.2.0-hdf63c60_1     http://repos.jd.com/conda/main     
    nccl:         1.3.5-cuda9.0_0      http://repos.jd.com/conda/main     
    ninja:        1.7.2-0              http://conda.jdfin.local/conda/free
    openblas:     0.2.14-4             http://conda.jdfin.local/conda/free
    pytorch:      0.4.0-py27hdf912b8_0 http://repos.jd.com/conda/main     

The following packages will be UPDATED:

    cudatoolkit:  8.0-3                http://conda.jdfin.local/conda/free --> 9.0-h13b8566_0  http://repos.jd.com/conda/main     
    mkl:          2017.0.3-0           http://conda.jdfin.local/conda/free --> 2019.0-118      http://repos.jd.com/conda/main     

The following packages will be DOWNGRADED:

    cudnn:        7.1.3-cuda8.0_0      http://repos.jd.com/conda/main      --> 7.1.2-cuda9.0_0 http://repos.jd.com/conda/main     
    numpy:        1.13.1-py27_0        http://conda.jdfin.local/conda/free --> 1.10.2-py27_0   http://conda.jdfin.local/conda/free

Proceed ([y]/n)? y

最后经过一番折腾,我发现,把warp-ctc/pytorch_binding/warpctc_pytorch 这个文件夹移动到crnn工程的上一级目录下就好了

这篇关于装CTC的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1090179

相关文章

tensorflow LSTM+CTC实现端到端的不定长数字串识别

转载地址: https://www.jianshu.com/p/45828b18f133 上一篇文章tensorflow 实现端到端的OCR:二代身份证号识别实现了定长18位数字串的识别,并最终达到了98%的准确率。但是实际应用场景中,常常需要面对无法确定字串长度的情况,这时候除了需要对识别字符模型参数进行训练外,还需要对字符划分模型进行训练,本文实现了上文提到的方法2,使用LST

lstm+ctc 实现ocr识别

转载地址: https://zhuanlan.zhihu.com/p/21344595 OCR是一个古老的研究领域,简单说就是把图片上的文字转化为文本的过程。在最近几年随着大数据的发展,广大爬虫工程师在对抗验证码时也得用上OCR。所以,这篇文章主要说的OCR其实就是图片验证码的识别。OCR并不是我的研究方向,我研究这个问题是因为OCR是一个可以同时用CNN,RNN两种算法都可以很好解决的

2-D CTC Loss

2D-CTC for Scene Text Recognition,1-D CTC Loss参考CTC Loss和Focal CTC Loss Motivation 普通的CTC仅支持1-d,但是文字识别不像语音识别,很多时候文字不是水平的,如果强行“压”到1d,对识别影响很大,如下图所示 Review 1-D CTC 首先对alphabeta进行扩充,加入blank符号,然后定义

CTC Loss和Focal CTC Loss

最近一直在做手写体识别的工作,其中有个很重要的loss那就是ctc loss,之前在文档识别与分析课程中学习过,但是时间久远,早已忘得一干二净,现在重新整理记录下 本文大量引用了- CTC Algorithm Explained Part 1:Training the Network(CTC算法详解之训练篇),只是用自己的语言理解了一下,原论文:Connectionist Temporal

CTC,RNN-Transducer, LAS

1、CTC ( Connectionist Temporal Classification) 网络结构 参考地址: CTC白话参考地址 其中CTC-loss 部分使用到了动态规划的思想。 大目标是: m i n ∑ B ( π ) = z ∏ t = 1 T y π t t m i n ∑ B ( π ) = z ∑ t = 1 T l o g ( y π t t ) min \su

基于CTPN(tensorflow)+CRNN(pytorch)+CTC的不定长文本检测和识别

转发来源:https://swift.ctolib.com/ooooverflow-chinese-ocr.html chinese-ocr 基于CTPN(tensorflow)+CRNN(pytorch)+CTC的不定长文本检测和识别 环境部署 sh setup.sh 使用环境: python 3.6 + tensorflow 1.10 +pytorch 0.4.1 注:CPU环境

【机器学习】基于CTC模型的语音转换可编辑文本研究

1.引言 1.1语音识别技术的研究背景 1.1.1.语音识别技术的需求 语音识别技术的研究和发展,对于提升人类与机器的交互方式具有深远的影响。首先,它极大地提高了工作效率和便利性。通过语音指令控制设备,用户可以更快捷地完成任务,无需手动输入或操作。例如,在办公环境中,语音识别可以快速完成文档编辑、邮件发送等任务;在家庭中,智能家居设备响应语音指令,实现灯光、温度等的调节。这种交互方式不仅节省

【CTC】CTC1D原理/代码/资料+2D CTC LOSS

1 1D CTC 1.1 简介 就不写了 1.2 核心思想 和大多数有监督学习一样,CTC 使用最大似然标准进行训练。 给定输入 x,输出 l 的条件概率为: 其中,B-1(l)表示了长度为 T 且示经过 B 结果为 l 字符串的集合。 CTC 假设每一步输出的概率是(相对于输入)条件独立的,因此有: p ( π ∣ x ) = ∏ y π t t , ∀ π ∈ L ′ T p

图像验证码识别,字母数字汉子均可cnn+lstm+ctc

图形验证码如下:   训练两轮时的准确率:上边显示的是未识别的     config_demo.yaml System:GpuMemoryFraction: 0.7TrainSetPath: 'train/'TestSetPath: 'test/'ValSetPath: 'dev/'LabelRegex: '([\u4E00-\u9FA5]{4,8}).jpg'MaxTextLenth:

文字识别 Optical Character Recognition,OCR CTC STN

文字识别 Optical Character Recognition,OCR 自然场景文本检测识别技术综述 将图片上的文字内容,智能识别成为可编辑的文本。 场景文字识别(Scene Text Recognition,STR) OCR(Optical Character Recognition, 光学字符识别)传统上指对输入扫描文档图像进行分析处理,识别出图像中文字信息。场景文字识别(