DMDC: a cross-attention network for dynamic mask-based dual-camera snapshot hyperspectral Photography

文献类型： 外文期刊

第一作者： Cai, Zeyu

作者： Cai, Zeyu;Da, Feipeng;Zhang, Ziyu;Jin, Chengqian;Cai, Zeyu;Da, Feipeng

作者机构：

关键词： Hyperspectral-imaging; RGB; CASSI; Dynamic; Cross-attention

期刊名称：VISUAL COMPUTER （ 影响因子：2.9； 五年影响因子：2.9 ）

ISSN： 0178-2789

年卷期： 2025 年 41 卷 7 期

页码：

收录情况： SCI

摘要： Spectral images can enrich the material information of the reconstructed scene and have an essential role in computer visualization. Coded aperture snapshot spectral imaging (CASSI) for dynamic scenes still faces two problems in reconstruction: 1) The single input limits the network's performance. 2) The spatial light modulator's (SLM) performance has yet to fully develop due to the limitation of fixed mask coding. This paper proposes a cross-attention-based dual-stream network for a dual-camera CASSI system. We argue that RGB images and CASSI measurements are projections of spectral 3D cubes in different 2D spaces and that fusing the spatial features of RGB and the spectral features of CASSI improves the quality of the reconstruction. Upon that, we embed a dynamic mask module in front of the cross-attention-based dual-stream network to further improve the reconstruction quality of the system. Specifically, the dynamic mask module utilizes RGB images to pre-learn the spatial feature distribution of the scene. Then the dynamic mask module guides the SLM in encoding the CASSI. Finally, the RGB and CASSI images are reconstructed using a cross-attention-based dual-stream network to obtain high-quality reconstruction results. Comprehensive experiments on various datasets demonstrate the superior performance of our method. At similar speeds, our method provides a 4.0 dB improvement over existing SOTA methods on clean and noisy datasets. In the snapshot video imaging task, the single snapshot imaging time of DMDC-1stg is less than 50 ms, which verifies the feasibility of our method. (The code has been released at https://github.com/caizeyu1992/DMDC.)

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序