Research on Multi-source Image Transfer Fusion and Target Recognition Algorithm Based on Generative Adversarial Network

CAO Zhaorui; Li Xinye; PAN Qian; YE Mingliang

doi:10.3788/gzxb20265508.0810003

Chinese

您当前的位置：

首页 >

文章列表页 >

Research on Multi-source Image Transfer Fusion and Target Recognition Algorithm Based on Generative Adversarial Network

更新时间：2026-07-15

- Research on Multi-source Image Transfer Fusion and Target Recognition Algorithm Based on Generative Adversarial Network
- Acta Photonica Sinica Pages: 1-15(2026)
- 作者机构：
  
  1.沈阳理工大学装备工程学院，辽宁沈阳110159
  2.沈阳理工大学机械工程学院，辽宁沈阳110159
- 作者简介：
- 基金信息：
- DOI：10.3788/gzxb20265508.0810003
  CLC： TP391.41
- CSTR：32255.14.gzxb20265508.0810003
- Received：09 April 2026，
  
  Revised：2026-06-12，
  
  Accepted：22 June 2026，
- 稿件说明：
移动端阅览
曹昭睿，李昕烨，潘骞，等. 基于生成对抗网络的多源图像迁移融合与目标识别算法研究［J］.光子学报，2026，55（8）：

CAO Zhaorui， Li Xinye， PAN Qian， et al. Research on Multi-source Image Transfer Fusion and Target Recognition Algorithm Based on Generative Adversarial Network［J］. Acta Photonica Sinica， 2026， 55（8）：0810003
曹昭睿，李昕烨，潘骞，等. 基于生成对抗网络的多源图像迁移融合与目标识别算法研究［J］.光子学报，2026，55（8）： DOI： 10.3788/gzxb20265508.0810003. CSTR： 32255.14.gzxb20265508.0810003.

CAO Zhaorui， Li Xinye， PAN Qian， et al. Research on Multi-source Image Transfer Fusion and Target Recognition Algorithm Based on Generative Adversarial Network［J］. Acta Photonica Sinica， 2026， 55（8）：0810003 DOI： 10.3788/gzxb20265508.0810003. CSTR： 32255.14.gzxb20265508.0810003.

摘要

针对现有红外迁移算法存在的泛化性弱、细节表现率不足和结构冗余等问题，面向无人武器低成本可见光系统探测光谱域拓展和低照度下可见光成像系统拓展光谱探测域的需求，提出一种基于可见光图像对红外图像迁移方法。通过构建一种强泛化红外迁移生成器，完成可见光特征向红外特征的转化与迁移；设计轻量特征敏感对抗鉴别器并利用二元交叉熵对数损失函数进行训练矫正，提升特征提取能力与捕捉图像细节的能力；通过梯度残差语义感知融合模块对伪红外图像与可见光图像进行融合，实现复合频段下多尺度目标的精确识别。结果表明，所设计的网络能够有效地生成目标红外物理特征，减小了伪红外图像与真实红外图像的差异，降低了低光环境下目标识别成本。在全天候复杂环境条件下，针对多尺度热源目标识别任务具有良好的计算精度与鲁棒性。

Abstract

To address the limitations of existing infrared image migration algorithms—namely weak generalization， insufficient detail representation， and structural redundancy—this paper proposes a novel visible-to-infrared image migration method. The research is motivated by the need to extend the spectral detection domain of low-cost visible light systems for unmanned weapons and to enhance low-light visible imaging systems. The primary objective is to achieve robust infrared feature transfer from visible images， thereby enabling accurate multi-scale target recognition under complex all-weather conditions while reducing dependency on physical infrared sensors.A Multi-source Image Migration Recognition Network （MIMRN） is designed， integrating several innovative components. First， a strongly generalized infrared migration generator is constructed， replacing the conventional U-shaped architecture with a multi-level residual connection structure. This modification reduces structural redundancy， accelerates training convergence， and facilitates reaching the Nash equilibrium in adversarial learning. Second， a lightweight feature-sensitive adversarial discriminator is developed， where the number of convolutional layers in the PatchGAN architecture is adjusted to enhance feature extraction and detail capture. The discriminator is trained using a binary cross-entropy log loss function instead of the traditional L1 norm loss， improving sensitivity to both high-frequency edge information and low-frequency gradient variations. Third， a deformable convolution and a bottleneck attention mechanism are introduced to enable more flexible sampling and to extract more discriminative features， thereby enhancing robustness against various image transformations. Fourth， a gradient residual semantic-aware fusion module is employed to merge the pseudo-infrared images with the original visible images， facilitating precise recognition of multi-scale targets across composite spectral bands. The entire network is optimized to balance the generator–discriminator game during training， improving structural similarity and detail fidelity in the generated pseudo-infrared images.Experimental results demonstrate that the proposed MIMRN effectively generates target-specific infrared physical characteristics from visible images， significantly reducing the perceptual and structural differences between pseudo-infrared and real infrared images. The method notably lowers the cost of target recognition in low-light environments. Under all-weather and complex environmental conditions， the network achieves high computational accuracy and robustness for multi-scale heat-source target recognition tasks. Quantitative evaluations show that training recognition models with pseudo-infrared images yields improved target identification accuracy compared to baseline methods. Furthermore， fusing visible images with pseudo-infrared images provides additional performance enhancement， enabling visible-light cameras to partially compensate for the absence of physical infrared sensors. The approach successfully strengthens the all-weather reconnaissance and detection capabilities of low-cost weapon platforms.This paper presents a multi-source image migration recognition network that integrates multiple collaborative optimization modules to achieve visible-to-pseudo-infrared image generation and fused-image target recognition. By incorporating deformable convolution for adaptive offset sampling and a lightweight two-dimensional bottleneck attention mechanism for discriminative feature weighting， the method extracts more robust features while suppressing irrelevant information interference， thereby enhancing robustness to image variations. The lightweight two-stage convolution design coupled with the binary cross-entropy log loss function effectively balances the adversarial training process between the generator and discriminator， improving the capture of both high-frequency edge details and low-frequency gradient structures. Experimental validation confirms that the proposed network reduces target recognition costs under low-light conditions while significantly enhancing the all-weather reconnaissance and detection capabilities of low-cost weapon platforms. This work provides effective technical support for improving the combat effectiveness of unmanned weapon systems in practical deployment scenarios.

关键词

Keywords

references

LIU Zuoli ， CUI Shoujun . Evolution of unmanned weapons and its implications for asymmetric security ［J］. Journal of International Security Studies ， 2023 ， 41 （ 2 ）： 23-48， 157 - 158 .

刘祚黎，崔守军 . 无人武器变革及其对非对称安全关系的影响［J］. 国际安全研究， 2023 ， 41 （ 2 ）： 23-48， 157 - 158 .

COCKBURN A . Kill chain： the rise of the high tech assassins ［M］. New York ： Henry Holt ， 2015 .

ZHANG F ， FAN H ， WANG K ， et al . Research on intelligent target recognition integrated with knowledge ［J］. IEEE Access ， 2021 ， 9 ： 137107 - 137115 .

KHAN S ， MAZHAR T ， SHAHZAD T ， et al . A systematic literature review on the applications， models， limitations， and future directions of generative adversarial networks ［J］. IET Circuits ， Devices Systems， 2025 ， 2025 （ 1 ）： 5384331 . DOI： 10.1049/cdt2/5384331 http://dx.doi.org/10.1049/cdt2/5384331 .

TU Hangyao ， WANG Wanliang ， CHEN Jiacheng ， et al . A survey of image translation based on conditional generative adversarial networks ［J］. Journal of Computer-Aided Design Computer Graphics ， 2024 ， 36 （ 1 ）： 14 - 32 .

屠杭垚，王万良，陈嘉诚，等 . 基于条件生成对抗网络的图像翻译综述［J］. 计算机辅助设计与图形学学报， 2024 ， 36 （ 1 ）： 14 - 32 .

ZHOU X ， ZHENG Y ， YANG J . Bridging the metrics gap in image style transfer： a comprehensive survey of models and criteria ［J］. Neurocomputing ， 2025 ， 624 ： 129430 .

WANG I ， OH T . Design and experimental research of on-device style transfer models for mobile environments ［J］. Scientific Reports ， 2025 ， 15 ： 13724 .

CHEN J ， CHEN S ， WEE L ， et al . Deep learning based unpaired image-to-image translation applications for medical physics： a systematic review ［J］. Physics in Medicine Biology ， 2023 ， 68 （ 5 ）： 05TR01 .

ZHOU L ， ZHANG T . AttCST： attention improves style transfer via contrastive learning ［J］. The Visual Computer ， 2023 ， 39 （ 12 ）： 6371 - 6383 .

CHONG M J ， FORSYTH D . JoJoGAN： one shot face stylization ［C］// Computer Vision-ECCV 2022 . Cham ： Springer Nature Switzerland ， 2022 ： 128 - 152 .

MEN Y ， YAO Y ， CUI M ， et al . DCT-Net： domain-calibrated translation for portrait stylization ［J］. ACM Transactions on Graphics ， 2022 ， 41 （ 4 ）： 1 - 9 .

LI N ， WANG H ， ZHAO H ， et al . Cross-modal visible-to-infrared image translation in remote sensing guided by thermal features ［J］. IEEE Transactions on Geoscience and Remote Sensing ， 2025 ， 63 ： 1 - 16 .

ZHANG Xuhui . Infrared-visible image fusion and target detection method in complex sea surface scenes ［D］. Guangzhou ： Guangdong University of Technology ， 2025 .

张旭辉 . 海面复杂场景红外—可见光图像融合及目标检测方法［D］. 广州：广东工业大学， 2025 .

HAN G ， SHANG C ， TAO Z ， et al . Intelligent identification of micro-NPR bolt shear deformation based on modular convolutional neural network ［J］. Sensors ， 2025 ， 26 （ 1 ）： 184 .

SU Yi ， HU Boran ， WEN Yan ， et al . Visible-infrared image translation algorithm based on conditional generative adversarial networks ［J］. Infrared and Laser Engineering ， 2025 ， 54 （ 12 ）： 244 - 257 .

苏毅，胡浡然，温焱，等 . 基于条件生成对抗网络的可见光-红外图像转换算法［J］. 红外与激光工程， 2025 ， 54 （ 12 ）： 244 - 257 .

JinggangLYU ， GAO Shuo ， LI Yuzhi ， et al . Facial expression recognition with channel attention guided global-local semantic collaboration ［J］. Computer Science ， 2026 ， 53 （ 1 ）： 195 - 205 .

吕景刚，高硕，李玉芝，等 . 通道注意力指导全局-局部语义协同的表情识别［J］. 计算机科学， 2026 ， 53 （ 1 ）： 195 - 205 .

LIU Xingke ， YANG Qian ， LUO Jianqiao ， et al . Branch segmentation of pepper image based on spatial and channel attention ［J］. Computer Applications and Software ， 2025 ， 42 （ 3 ）： 202 - 208，243 .

刘兴科，杨前，罗建桥，等 . 基于空间和通道注意力的花椒图像枝条分割［J］. 计算机应用与软件， 2025 ， 42 （ 3 ）： 202 - 208，243 .

ARORA S ， GE R ， LIANG Y ， et al . Generalization and equilibrium in generative adversarial nets （GANs）［C］// Proceedings of the 34th International Conference on Machine Learning . Sydney ： PMLR ， 2017 ： 224 - 232 .

CHANG Y ， MA D ， DING Y ， et al . Lightweight design for infrared dim and small target detection in complex environments ［J］. Remote Sensing ， 2024 ， 16 （ 20 ）： 3761 . DOI： 10.3390/rs16203761 http://dx.doi.org/10.3390/rs16203761 .

HUANG Qidong . Research on adversarial robustness enhancement methods for vision foundation models ［D］. Hefei ： University of Science and Technology of China ， 2025 .

黄启栋 . 面向视觉基础模型的对抗鲁棒性增强方法研究［D］. 合肥：中国科学技术大学， 2025 .

TANG Lei ， PAN Bailin ， LONG Bo ， et al . Image domain translation algorithm based on generative adversarial networks ［J］. Infrared Technology ， 2025 ， 47 （ 12 ）： 1567 - 1575 .

汤磊，潘柏林，龙波，等 . 基于生成对抗网络的图像域转换算法［J］. 红外技术， 2025 ， 47 （ 12 ）： 1567 - 1575 .

PASZKE A ， GROSS S ， MASSA F ， et al . PyTorch： an imperative style， high-performance deep learning library ［C］// Advances in Neural Information Processing Systems 32 . Red Hook ： Curran Associates ， 2019 ： 8024 - 8035 .

CHEN Xiefa ， LI Min ， ZHAO Jingyu ， et al . Low light target detection algorithm based on feature fusion enhancement ［J］. Computer Engineering and Applications ， 2025 ， 61 （ 22 ）： 235 - 244 .

陈谢发，李敏，赵晶钰，等 . 基于特征融合增强的低光照目标检测算法［J］. 计算机工程与应用， 2025 ， 61 （ 22 ）： 235 - 244 .

WANG Ting . Research and application of infrared and visible image fusion ［D］. Xi'an ： Xi'an University of Technology ， 2019 .

汪廷 . 红外图像与可见光图像融合研究与应用［D］. 西安：西安理工大学， 2019 . DOI: 10.23919/chicc.2019.8865405 http://dx.doi.org/10.23919/chicc.2019.8865405

FENG Yufang ， YIN Hong ， LU Houqing ， et al . Infrared and visible image fusion based on improved convolutional neural network ［J］. Computer Engineering ， 2020 ， 46 （ 8 ）： 243 - 249，257 .

冯玉芳，殷宏，卢厚清，等 . 基于改进卷积神经网络的红外与可见光图像融合［J］. 计算机工程， 2020 ， 46 （ 08 ）： 243 - 249+257 .

JEE S ， KANG M G . Sensitivity improvement of extremely low light scenes with RGB-NIR multispectral filter array sensor ［J］. Sensors ， 2019 ， 19 （ 5 ）： 1256 .

JUNG C K ， ZHOU K L ， FENG J W . FusionNet： multispectral fusion of RGB and NIR images using two stage convolutional neural networks ［J］. IEEE Access ， 2020 ， 8 ： 23912 - 23919 .

HUI T ， TANG X ， LOY C C . LiteFlowNet： a lightweight convolutional neural network for optical flow estimation ［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway ： IEEE ， 2018 ： 8981 - 8989 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on Infrared Visible Image Fusion and Target Recognition Algorithm Based on Region of Interest Mask Convolution Neural Network

Feature-image Space Generative Adversarial Networks for Image Fusion

HFDFusion： Hierarchical Feature Decomposition Method for Multimodal Image Fusion

A Method for Infrared and Visible Image Fusion Based on CNN-Mamba Feature Extraction Combined with Information Selection

Design of a Wide-field Multispectral Imaging Optical System for Martian Dust Storm Detection

Related Author

Yongping HAO

Zhaorui CAO

Fan BAI

Haoyang SUN

Xing WANG

Jie QIN

ZHANG Can

FAN Fan

Related Institution

College of Equipment Engineering， Shenyang Ligong University

College of Mechanical Engineering， Shenyang Ligong University

Electronic Information School， Wuhan University

School of Robotics， Wuhan University

Xi'an Institute of Optics and Precision Mechanics of CAS

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰