目标识别和文字检测算法 Faster R-CNN、CTPN

Faster R-CNN 目标检测算法

Towards Real-Time Object Detection with Region Proposal Networks

R-CNN：Regions with CNN features

1. Input image
2. Extract region proposals(~2k)
3. Compute CNN features
4. Classify regions

IoU Intersection over Union

测量在特定数据集中检测相应物体准确度的一个标准

预测范围： bounding boxex

ground-truth bounding boxes（人为在训练集图像中标出要检测物体的大概范围）

$IoU=\frac{AreaofOverlap}{AreaofUnion}$

NMS (Non-Maximum Suppression)

Fast R-CNN

Selection search

Anchor sliding window Feature extraction

RPN Loss

Cls label 二分类，是否有物体，使用IoU gt bounding box anchor box

Loc label
$$\begin{gathered} t_x^{\ast }=(x^{\ast }-x_a)/w_a,t_y^{\ast }=(y^{\ast }-y_a)/h_a, \\ {t}_w^{\ast }=log(w^{\ast }/w_a),t_h^{\ast }=log(t_w^{\ast }) \end{gathered}$$

$$\begin{gathered} t_x=(x-x_a)/w_a,t_y=(y-ya)/h_a, \\ {t}_w=log(w/w_a),t_h=log(h/h_a) \end{gathered}$$

Cls loss

Cross Entropy交叉熵

Loc Loss
$$\begin{gathered} z_i=0.5(x_i-y_i{)}^2/beta,if|x_i-y_i|<beta \\ {z}_i=|x_i-y_i|-0.5\ast beta,otherwise \end{gathered}$$

RoI Head Region of Interest

Mask R-CNN

$L=L_{cls}+L_{box}+L_{mask}$

To this we apply a per-pixel sigmoid,and define $L_{mask}$ as the average binary cross-entropy loss. For an RoI associated with gorund-truth k, $L_{mask}$ is only defined o the k-th mask(other mask outputs do not contribute to the loss).

RoI Align不对齐，保留浮点，在小区域之内继续划分

CTPN 文字检测算法

Detecting Text in Natural Image with Connectionist Text Proposal Network

• Detecting text in fine-scale proposals
• Recurrent connectionist text proposals
• Side-refinement

$$\begin{gathered} v_c=(c_y-c_y^a)/h^a \\ {v}_c^{\ast }=(c_y^{\ast }-c_y^a)/h^a \\ {v}_h=log(h/h_a) \\ {v}_h^{\ast }=log(h^{\ast }/h^a) \end{gathered}$$

Text line construction

$$o^{\ast }=(x_{side}^{\ast }-c_x^a)/w^a$$

Code

bounding box

CRNN 文字识别算法

An End-yo-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition

• CRNN

• Code

• CTC

• lexicon-based

• lexicon-free

feature sequence —— receptive field感受野

CRNN——CTC
$$\begin{gathered} \pi =--hh-e-l-ll-oo-- \\ B(\pi )=hello \\ p(l|y)=\sum _{\pi :B(\pi )=1}p(\pi |y),p{(}^{\prime }hell{o}^{\prime }|y)=\sum _{\pi :B(\pi ){=}^{\prime }hell{o}^{\prime }}p(\pi |y) \end{gathered}$$

CTC Theory

$$\begin{gathered} p(l|x)=\sum _{\pi \in B^{-1}(1)}p(\pi |x). \\ h(x)=argma{x}_{1\in L\le T}p(l|x). \\ {O}^{ML}(S,N_w)=-\sum _{(x,z)\in S}ln(p(z|x))=-\sum _{(x,z)\in S}ln(\sum _{\pi \in B^{-1}(z)}p(\pi |x)) \end{gathered}$$

为了让所有的path都能在图中唯一、合法的表示，结点转换有如下约束：

1. 转换只能往右下方向，其他方向不允许
2. 相同的字符之间起码要有一个空字符
3. 非空字符不能被跳过
4. 起点必须从前两个字符开始
5. 终点必须落在结尾两个字符

forward-backward

定义在时刻t经过节点s的全部前缀子路径的概率总和为前向概率${\alpha }_t(s)$
$${\alpha }_3(4)=p{(}_{a}p)+p(aap)+p(a_p)+p(app)$$

• 情况1：第s个符号为空符号blank
  $${\alpha }_t(s)=({\alpha }_{t-1}(s)+{\alpha }_{t-1}(s-1))\cdot y_{seq(s)}^t$$

• 情况2：第s个符号等于第s-2个符号
  $${\alpha }_t(s)=({\alpha }_{t-1}(s)+{\alpha }_{t-1}(s-1))\cdot y_{seq(s)}^t$$

• 情况3：既不属于情况1，也不属于情况2
  $${\alpha }_t(s)=({\alpha }_{t-1}(s)+{\alpha }_{t-1}(s-1)+{\alpha }_{t-1}(s-2))\cdot y_{seq(s)}^t$$
  

不属于情况2
$${\alpha }_t(s)=({\alpha }_{t-1}(s)+{\alpha }_{t-1}(s-1)+{\alpha }_{t-1}(s-2))\cdot y_{seq(s)}^t$$