| Περίληψη: |
Highlights: What are the main findings? A spectral-guided fusion network, SpecGateNet, is proposed for cloud and cloud shadow segmentation in optical remote sensing imagery using only RGB inputs. SpecGateNet integrates CNN-Swin cross-feature interaction, frequency-domain dynamic filtering, and spectral-guided decoder fusion to improve global context modeling and boundary-detail preservation. What are the implications of the main findings? The proposed spectral-guided fusion strategy improves the segmentation of thin cloud boundaries, cloud shadow contours, and homogeneous cloud regions by adaptively balancing semantic and spatial detail features. Among the compared general-purpose baselines under a unified experimental protocol, SpecGateNet attains leading mIoU on CloudSEN-12, SPARCS-Val, and 38-Cloud, demonstrating strong adaptability across different source imagery, datasets, and task granularities. Accurate segmentation of clouds and cloud shadows in optical remote sensing imagery is an indispensable preprocessing step for downstream land-surface observation tasks. However, existing methods strike different trade-offs between local detail preservation and global context modeling, yet they still exhibit two major limitations: insufficient interaction during the dual-branch encoding stage, and decoder-side multi-scale fusion that applies uniform operations to all spatial locations, making it unable to distinguish boundary regions, such as thin cloud boundaries and cloud shadow contours, from homogeneous regions, such as thick cloud interiors. To address these issues, we propose SpecGateNet, a fusion network guided by spectral information from a frequency-domain perspective. The core idea is that the Fourier amplitude spectrum of deep features summarizes how much energy is carried by different spatial frequencies, where high-frequency components typically correspond to rapidly varying regions such as boundaries and textures, whereas low-frequency components are more closely associated with smooth regions. Building on this insight, we use an amplitude-derived, phase-free spectral energy cue constructed from the amplitude spectrum, and integrate it with a learnable gating generation network to produce spatially adaptive fusion modulation signals. SpecGateNet is built upon a CNN-Swin Transformer dual-branch encoder and consists of three key components: a Cross-Feature Fusion module (CFF) that enables stage-wise bidirectional interaction, a frequency-domain dynamic filtering bottleneck that enhances global context through input-adaptive frequency-domain filtering, and a Spectral-Guided Fusion decoder (SGF) that generates spatial gates from the amplitude spectrum to adaptively balance high-level semantics and low-level details. Together, the bottleneck filter and SGF constitute a spectral utilization framework that operates at both the encoder and decoder sides. Using only RGB inputs without relying on infrared auxiliary bands, SpecGateNet achieves the highest mIoU among the compared general-purpose baselines under a unified experimental protocol on three public datasets, namely CloudSEN-12, SPARCS-Val, and 38-Cloud, with mIoU scores of 77.80%, 76.75%, and 93.30%, respectively, outperforming the second-best method by 1.33, 5.26, and 1.44 percentage points. Ablation studies confirm that the spectral modules account for 70.1% of the total performance gain, with SGF contributing the largest single-step improvement among individual modules (+1.22%). [ABSTRACT FROM AUTHOR] |