• 摘要: 针对医学图像分割中细粒度边界特征易丢失与空间长距离依赖难兼顾的问题,提出一种融合频谱区域自注意力与空间交错级联双向Mamba模型的双分支分割框架FreqMambaSeg。该框架采用频域与空间域并行建模机制,分别组织频谱响应关系与跨区域空间结构信息,从而实现细粒度病灶表征与长距离结构建模的互补。具体而言,频域Transformer分支引入频谱区域自注意力 (spectral region self-attention, SRSA)和频谱多尺度前馈网络 (spectral multi-scale feed-forward, SMS-FF)。前者将查询、键、值特征映射至傅里叶频域,并按频率半径重排频谱特征,以显式建模连续区间内不同频率成分间的依赖关系;后者通过原始与下采样尺度的可学习频域调制,自适应补充不同尺度下病灶主体、边界与背景的上下文响应。同时,空间分支构建了空间交错级联双向Mamba模型 (spatially interleaved cascaded bidirectional Mamba, SI-CBSM),以空间交错跳跃扫描为策略,将二维特征按固定步长和多空间偏移划分为交错子序列,并结合双向状态空间模型 (state space model, SSM)、深度可分离卷积与门控机制,增强息肉边缘、皮肤病变弥散边界等不规则结构的跨区域特征响应。最后,两分支特征经轻量级通道拼接与卷积重组进行融合,并由解码器输出最终结果。实验结果表明,FreqMambaSeg仅含4.39 M可训练参数,在五个基准医学数据集上的平均DSC达到89.57%,并在五个数据集的DSC和mIoU指标上均获得最高数值,体现了模型在分割精度、模型规模和实际推理效率之间达到较好平衡。

       

      Abstract:
      Objective Accurate lesion segmentation is essential for computer-aided diagnosis, quantitative disease assessment, treatment planning, and image-guided intervention. However, medical lesions often present weak contrast, irregular shapes, blurred boundaries, heterogeneous textures, and modality-dependent artifacts, which make it difficult to simultaneously preserve fine boundaries and model long-range structural context. CNN-based methods are effective for local texture extraction but are limited by local receptive fields, whereas Transformer-based methods improve global context modeling at the cost of high token-level computation. Mamba-based state space models provide an efficient alternative for long-sequence modeling, yet most existing visual Mamba segmentation methods mainly rely on spatial-domain scanning and make limited use of frequency-domain information associated with lesion boundaries, textures, and background variations. To address these issues, this study proposes FreqMambaSeg, a frequency-spatial collaborative dual-branch network for two-dimensional medical lesion segmentation.
      Methods FreqMambaSeg adopts a symmetric U-shaped encoder-decoder architecture and constructs parallel frequency-domain and spatial-domain branches at each feature level. The frequency-domain branch is implemented by a frequency-domain Transformer (FDT) block composed of spectral region self-attention (SRSA) and a spectral multi-scale feed-forward (SMS-FF) network. In SRSA, input features are projected into query, key, and value representations and transformed into the Fourier domain. After fftshift, spectral tokens are rearranged according to their radial distance from the frequency center, so that frequency components with related spectral properties are organized into contiguous token intervals. Multi-head self-attention is then performed within each spectral region to model dependencies among frequency components. The attended tokens are restored to their original spectral positions and transformed back to the spatial domain by inverse Fourier transform. SMS-FF further introduces learnable frequency-domain modulation at the original and downsampled scales, enabling adaptive spectral representation for lesion bodies, boundaries, and background regions.
      The spatial-domain branch is implemented using the spatially interleaved cascaded bidirectional Mamba model (SI-CBSM). Its SI-Scan strategy partitions two-dimensional features into interleaved subsequences with fixed strides and multiple spatial offsets, changing the adjacency relationship of tokens in one-dimensional sequences and enabling state propagation to connect spatial positions that are distant in the original feature map. Each SI-CBSM unit combines bidirectional state space modeling, depthwise convolution, channel-attention gating, nonlinear activation, and residual cascading. The two branches are integrated through lightweight channel concatenation and 1×1 convolutional reorganization, allowing frequency-domain discrimination and spatial-domain structural modeling to complement each other within the encoder-decoder framework. This design avoids heavy fusion modules and assigns the main representational roles to the two complementary branches, thereby maintaining a compact parameter scale.
      Results and Discussions Experiments were conducted on five public benchmark datasets covering three lesion segmentation scenarios: ISIC2017 and ISIC2018 for dermoscopic skin lesions, CVC-ClinicDB and CVC-ColonDB for endoscopic polyps, and BUSI for breast ultrasound lesions. All comparison methods were evaluated using consistent data partitions, preprocessing procedures, and training settings. Dice similarity coefficient (DSC), mean intersection over union (mIoU), specificity, sensitivity, and accuracy were used as evaluation metrics.
      On ISIC 2017, FreqMambaSeg achieved 89.35% DSC and 80.59% mIoU. On ISIC 2018, it achieved 90.66% DSC and 82.35% mIoU, showing effectiveness for skin lesions with variable shapes and low-contrast boundaries. For polyp segmentation, the model obtained 93.55% DSC and 87.79% mIoU on CVC-ClinicDB, and 91.43% DSC and 84.10% mIoU on CVC-ColonDB, indicating that frequency-domain discrimination and cross-region spatial modeling provide complementary cues under complex intestinal backgrounds. On the BUSI breast ultrasound dataset, which contains speckle noise, acoustic shadowing, and ambiguous lesion margins, FreqMambaSeg achieved 82.85% DSC and 73.26% mIoU.
      Ablation experiments further validated the contributions of the proposed components. Replacing the baseline spatial modeling unit with SI-CBSM improved segmentation performance, while introducing SRSA and SMS-FF further strengthened frequency-domain representation. Additional analyses of the SI-Scan stride, scanning direction, state propagation direction, SMS-FF downsampling ratio, and SI-CBSM cascade depth verified the rationality of the main architectural configurations. Across the five datasets, FreqMambaSeg achieved an average DSC of 89.57% and an average mIoU of 81.62%. The model contains 4.39 M trainable parameters, requires 7.76 G FLOPs for an input size of 1 × 3 × 256 × 256, and reaches 66.73 f/s on an NVIDIA RTX 4090 GPU with a batch size of 1. Qualitative comparisons and feature-response analyses show that the FDT and SI-CBSM branches provide complementary response patterns and that feature fusion produces more concentrated lesion-related responses. Frequency-band visualizations further indicate that low-frequency responses are mainly associated with lesion bodies and global structures, whereas mid- and high-frequency responses are more sensitive to contour transitions, local textures, and possible artifacts, supporting the necessity of adaptive spectral modeling.
      Conclusions FreqMambaSeg provides an efficient frequency-spatial collaborative framework for two-dimensional medical lesion segmentation. By combining spectral region attention, multi-scale frequency-domain modulation, spatially interleaved scanning, and bidirectional state space modeling, the proposed architecture jointly captures lesion body structure, boundary details, and long-range spatial context. Experiments on dermoscopy, endoscopy, and ultrasound datasets demonstrate competitive segmentation accuracy with a small model size and a favorable accuracy-efficiency trade-off. The results suggest that organizing frequency-domain cues and spatial structural cues in separate but collaborative branches is a practical strategy for complex lesion segmentation. Future work will investigate uncertainty-aware boundary supervision and artifact suppression, and extend the framework to three-dimensional medical volumes, real-time surgical videos, and structure-prior-guided segmentation.