Squeeze-and-Excitation Networks

URL

https://arxiv.org/pdf/1709.01507.pdf

TL;DR

• SENet 给每个通道赋予权重，Attention to Channel

Algorithm

数学表达

$z_c = F_{sq}(u_c) = \frac{1}{H \times W}\sum_{i=1}^H\sum_{j=1}^W u_c(i, j),\ \ \ \ \ z \in \mathbb R^C$

$s = F_{ex}(z, W) = \sigma(g(z, W)) = \sigma(W_2\delta(W_1z)), \ \ \ W_1 \in \mathbb R^{\frac{C}{r}\times C},\ \ \ W_2 \in \mathbb R^{C \times \frac{C}{r}}$

$\tilde X_c = F_{scale}(u_c, s_c) = s_cu_c,\ \ \ \ X \in \mathbb R^C$

SENet实验结果

• ImageNet

• other
  

Thoughts

• SENet 与 SKNet 属于 Attention to channel，ULSAM 属于 Attention to HW，两个合起来是否可以替代 Non-local——在 THW上的 Attention