Attention Mechanism

This paper introduces attentional activation units (ATAC), a unification of activation functions and attention mechanisms. ATAC units include a local channel attention module for simultaneous non-linear activation and element-wise feature refinement, locally aggregating point-wise cross-channel feature contexts in convolutional networks.

This study presents an open dataset for single-frame infrared small target detection and introduces an asymmetric contextual modulation module. This module utilizes a top-down global contextual feedback and a point-wise channel attention-based bottom-up modulation pathway to enhance the detection of infrared small targets.

This paper introduces attentional feature fusion, a uniform scheme for merging features from different network layers or branches. The method utilizes a multi-scale channel attention module for fusing inconsistent semantics and scales, and iterative attentional feature fusion to alleviate integration bottlenecks.