Deep Learning-Based Multi-Frame Fringe 3D Reconstruction for Dynamic Scenes

SHEN Siyuan; LU Rongsheng; LI Shuge; SHI Wensong

doi:10.3788/gzxb20265507.0712001

Chinese

您当前的位置：

首页 >

文章列表页 >

Deep Learning-Based Multi-Frame Fringe 3D Reconstruction for Dynamic Scenes

更新时间：2026-06-10

- Deep Learning-Based Multi-Frame Fringe 3D Reconstruction for Dynamic Scenes
- Acta Photonica Sinica Pages: 1-12(2026)
- 作者机构：
  
  1.合肥工业大学仪器科学与光电工程学院，合肥230009
  2.无锡维度机器视觉产业技术研究院有限公司，无锡214101
- 作者简介：
- 基金信息：
  
  "Light of the Taihu Lake Lake" scientific and technological breakthrough （industry foresight and key technology research and development）(G20252001)
- DOI：10.3788/gzxb20265507.0712001
  CLC： TH741
- CSTR：32255.14.gzxb20265507.0712001
- Received：21 January 2026，
  
  Revised：2026-04-12，
  
  Accepted：17 April 2026，
- 稿件说明：
移动端阅览
沈思源，卢荣胜，李书阁，等. 基于深度学习的动态场景多帧条纹三维重建技术［J］.光子学报，2026，55（7）：0712001

SHEN Siyuan， LU Rongsheng， LI Shuge， et al. Deep Learning-Based Multi-Frame Fringe 3D Reconstruction for Dynamic Scenes［J］. Acta Photonica Sinica， 2026， 55（7）：0712001
沈思源，卢荣胜，李书阁，等. 基于深度学习的动态场景多帧条纹三维重建技术［J］.光子学报，2026，55（7）：0712001 DOI： 10.3788/gzxb20265507.0712001. CSTR： 32255.14.gzxb20265507.0712001.

SHEN Siyuan， LU Rongsheng， LI Shuge， et al. Deep Learning-Based Multi-Frame Fringe 3D Reconstruction for Dynamic Scenes［J］. Acta Photonica Sinica， 2026， 55（7）：0712001 DOI： 10.3788/gzxb20265507.0712001. CSTR： 32255.14.gzxb20265507.0712001.

摘要

条纹轮廓术通过相移条纹图案实现高分辨率的三维重建。然而，在动态场景中，物体的运动会引发纹理错位并引入额外的相移，从而导致重建误差。为此，本文提出一种基于深度学习的方法，用于减少由刚体运动引起的动态误差。具体而言，首先利用深度网络从连续的相移复合条纹图像中快速估计各帧的粗点云；随后，通过点云配准将多帧点云统一至同一坐标系，并反投影至图像平面以实现条纹对齐；最后，再次利用网络提取高精度相位，完成高保真度的三维重建。实验结果表明，所提方法能够有效抑制动态场景中由物体运动引入的误差，实现高精度的三维重建。

Abstract

Optical three-dimensional （3D） measurement techniques， particularly fringe projection profilometry （FPP）， have been widely adopted in industrial inspection， intelligent recognition， and cultural heritage preservation due to their high accuracy， low cost， and fast acquisition speed. However， conventional FPP methods typically rely on multiple phase-shifted fringe patterns captured under static conditio-ns. In dynamic scenes， object motion introduces inter-frame misalignment and additional phase shifts， leading to significant reconstruction errors. To address this challenge， we propose a novel deep learning-based multi-frame fringe 3D reconstruction method specifically designed for dynamic scenes with rigid motion.Our approach consists of three main stages. First， a deep neural network is employed to rapidly estimate coarse 3D point clouds from single-frame composite fringe images. This network is trained to predict the numerator and denominator components required for phase calculation， enabling direct phase retrieval from a single image without the need for traditional multi-step phase shifting. Second， to leverage temporal information while compensating for motion-induced misalignment， we introduce a point cloud registration step. Using the Iterative Closest Point （ICP） algorithm， we align the point clouds of adjacent frames into a common coordinate system. The aligned point clouds are then back-projected onto the image plane to generate motion-compensated fringe patterns. Finally， a multi-frame deep learning enhancement module processes the aligned fringe images to extract high-fidelity phase information. This module integrates spatial and temporal features through a dual-component network architecture， combining a single-frame phase extraction backbone with a multi-frame enhancement encoder and adaptive feature fusion mechanisms.The performance of the method was tested in both static and dynamic scenes in this article. Results demonstrate that the proposed method significantly outperforms traditional phase-shifting methods and single-frame deep learning approaches. In static scenes， the multi-frame model demonstrated superior performance by reducing the reconstruction error compared to its single-frame counterpart. In dynamic scenes， it effectively eliminated motion artifacts such as ripple distortions and achieved a substantial reduction in phase error compared to conventional methods. Moreover， the method showed strong robustness in reconstructing fine surface details， even under complex motion conditions.In conclusion， this study presents a robust and accurate solution for 3D reconstruction in dynamic environments by integrating deep learning with image alignment techniques. The proposed method successfully combines the advantages of single-frame inference speed and multi-frame temporal consistency， offering a promising direction for real-time high-precision 3D measurement in motion scenarios. Future work will focus on extending the method to non-rigid objects and improving computational efficiency for real-time applications.

关键词

Keywords

references

Curless ， Brian . From range scans to 3D models . ACM SIGGRAPH Computer Graphics . 1999 ； 33 （ 4 ）： 38 - 41 .

Srinivasan V ， Liu H ， Halioua M . Automated phase-measuring profilometry of 3-D diffuse objects . Applied Optics . 1984 ； 23 （ 18 ）： 3105 - 08 . 10.1364/AO.23.003105 http://dx.doi.org/10.1364/AO.23.003105

Takeda M ， Mutoh K . Fourier transform profilometry for the automatic measurement of 3-D object shapes . Applied Optics . 1983 ； 22 （ 24 ）： 3977 .

Duda R.O ， Hart P.E. ， Pattern classification and scene analysis ， Pattern classification and scene analysis 1973 .

Zuo C ， Feng S ， Huang L ， et al . Phase shifting algorithms for fringe projection profilometry： A review . Optics and Lasers in Engineering . 2018 ； 109 ： 23 - 59 . 10.1016/j.optlaseng.2018.04.019 http://dx.doi.org/10.1016/j.optlaseng.2018.04.019

Han M ， Chen W . Two-dimensional complex wavelet with directional selectivity used in fringe projection profilometry . Optics Letters . 2021 ； 46 （ 15 ）. 10.1364/ol.420460 http://dx.doi.org/10.1364/ol.420460

Zuo C ， Tao T ， Feng S ， et al . Micro Fourier Transform Profilometry （μFTP）： 3D shape measurement at 10，000 frames per second . Optics and Lasers in Engineering . 2018 ； 102 ： 70 - 91 . 10.1016/j.optlaseng.2017.10.013 http://dx.doi.org/10.1016/j.optlaseng.2017.10.013

Qian J ， Tao T ， Feng S ， et al . Motion-artifact-free dynamic 3D shape measurement with hybrid Fourier-transform phase-shifting profilometry . Opt Express . 2019 ； 27 （ 3 ）： 2713 - 31 . 10.1364/OE.27.002713 http://dx.doi.org/10.1364/OE.27.002713

Lu L. ， Jia Z. ， Luan Y. ， et al . Reconstruction of isolated moving objects with high 3D frame rate based on phase shifting profilometry . Optics Communications . 2019 ； 438 ： 61 - 66 . 10.1016/j.optcom.2018.12.092 http://dx.doi.org/10.1016/j.optcom.2018.12.092

Liu X ， Tao T ， Wan Y ， et al . Real-time motion-induced-error compensation in 3D surface-shape measurement . Optics Express . 2019 ； 27 （ 18 ）. 10.1364/oe.27.025265 http://dx.doi.org/10.1364/oe.27.025265

Duan C ， Tong J. ， Lu L ， et al . Improving the Performance of 3D Shape Measurement of Moving Objects by Fringe Projection and Data Fusion . IEEE Access . 2021 ； 9 ： 34682 - 91 . 10.1109/access.2021.3061415 http://dx.doi.org/10.1109/access.2021.3061415

Guo W ， Wu Z ， Li Y ， et al . Real-time 3D shape measurement with dual-frequency composite grating and motion-induced error reduction . Opt Express . 2020 ； 28 （ 18 ）： 26882 - 97 . 10.1364/OE.403474 http://dx.doi.org/10.1364/OE.403474

Wang Y ， Suresh V ， Li B . Motion-induced error reduction for binary defocusing profilometry via additional temporal sampling . Optics Express . 2019 ； 27 （ 17 ）. 10.1364/oe.27.023948 http://dx.doi.org/10.1364/oe.27.023948

Liu Z ， Zibley P.C ， Zhang S . Motion-induced error compensation for phase shifting profilometry . Optics Express . 2018 ； 26 （ 10 ）. 10.1364/oe.26.012632 http://dx.doi.org/10.1364/oe.26.012632

Wang Y ， Liu Z ， Jiang C. ， et al . Motion induced phase error reduction using a Hilbert transform . Optics Express . 2018 ； 26 （ 26 ）. 10.1364/oe.26.034224 http://dx.doi.org/10.1364/oe.26.034224

Wang H ， Wang Y. ， Chen Z ， et al . Real-time motion-induced error reduction for phase-shifting profilometry with projection points tracking method . Measurement . 2025 ； 239 . 10.1016/j.measurement.2024.115450 http://dx.doi.org/10.1016/j.measurement.2024.115450

Li Y ， Qian J ， Feng S ， et al . Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement . Opto-Electronic Advances . 2022 ； 5 （ 5 ）： 210021 - 21 . 10.29026/oea.2022.210021 http://dx.doi.org/10.29026/oea.2022.210021

Sui C ， He K ， Lyu C ， et al . Accurate 3D Reconstruction of Dynamic Objects by Spatial-Temporal Multiplexing and Motion-Induced Error Elimination . IEEE Transactions on Image Processing . 2022 ； 31 ： 2106 - 21 . 10.1109/tip.2022.3150297 http://dx.doi.org/10.1109/tip.2022.3150297

Guo X ， Li Y ， Qian J ， et al . Unifying temporal phase unwrapping framework using deep learning . Opt Express . 2023 ； 31 （ 10 ）： 16659 - 75 . 10.1364/OE.488597 http://dx.doi.org/10.1364/OE.488597

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Speckle Image Deformation Measurement Method Based on Convolutional Neural Network UNet++

Adaptive Sand and Dust Image Restoration Based on Atmospheric Optical Constraints and Multi-scale Distribution

Detection of Salt Spray Aging in Wind Turbine Blade Coatings Based on Terahertz Spectroscopy and Deep Learning

Research on Structured Light Wrapped Phase Unwrapping Algorithm Based on Deep Learning

Remote Sensing Image Object Detection Based on Adaptive Feature Weighting

Related Author

CHEN Qiang

LIANG Junzhe

LIANG Jin

TANG Yonghua

LIU Hongpu

LIN Sen

BAI Tianyu

DUAN Xiaoteng

Related Institution

State Key Laboratory of Precision Manufacturing Technology， School of Mechanical Engineering， Xi'an Jiaotong University

School of Information Science and Engineering， Shenyang University of Technology

Liaoning Provincial Key Laboratory of Machine Vision

School of Automation and Electrical Engineering， Shenyang Ligong University

School of Intelligent Manufacturing， Putian University

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰