【注意力】ESE：Effective Squeeze-and-Excitation Block

原文：https://arxiv.org/pdf/1911.06667.pdf

 ESE(Effective Squeeze and Extraction) layer是模型中的一个block，基于SE（Squeeze and Extraction）而来。与SE的区别在于，ESE block只有一个fc层，《CenterMask : Real-Time Anchor-Free Instance Segmentation》的作者注意到SE模块有一个缺点：由于维度的减少导致的通道信息损失。为了避免这种大模型的计算负担，se的2个fc层需要减少通道维度。特别的，当第一个fc层使用r减少输入特征通道，将通道数从c变为c/r的时候，第二个fc层又需要扩张减少的通道数到原始的通道c.在这个过程中，通道维度的减少导致了通道信息的损失。因而，effective SE(eSE)仅仅使用一个通道数为c的fc层代替了两个fc层，避免了通道信息DE丢失；

代码：

def get_act_fn(act=None, trt=False):assert act is None or isinstance(act, (str, dict)), 'name of activation should be str, dict or None'if not act:return identityif isinstance(act, dict):name = act['name']act.pop('name')kwargs = actelse:name = actkwargs = dict()if trt and name in TRT_ACT_SPEC:fn = TRT_ACT_SPEC[name]elif name in ACT_SPEC:fn = ACT_SPEC[name]else:fn = getattr(F, name)return lambda x: fn(x, **kwargs)class EffectiveSELayer(nn.Layer):""" Effective Squeeze-ExcitationFrom `CenterMask : Real-Time Anchor-Free Instance Segmentation` - https://arxiv.org/abs/1911.06667"""def __init__(self, channels, act='hardsigmoid'):super(EffectiveSELayer, self).__init__()self.fc = nn.Conv2D(channels, channels, kernel_size=1, padding=0)self.act = get_act_fn(act) if act is None or isinstance(act, (str, dict)) else actdef forward(self, x):x_se = x.mean((2, 3), keepdim=True)x_se = self.fc(x_se)return x * self.act(x_se)

代码摘自pp-yoloe（https://github.com/PaddlePaddle/PaddleDetection）