Pedestrian crossing intent inference method based on multi-feature fusion

Yin Shouguo; Du Quancheng; Li Lingxi; Wang Xiao; Sun Changyin

doi:10.11959/j.issn.2096-6652.202601

Chinese

您当前的位置：

首页 >

文章列表页 >

Pedestrian crossing intent inference method based on multi-feature fusion

Papers and Reports | 更新时间：2026-04-27

- Pedestrian crossing intent inference method based on multi-feature fusion
- Chinese Journal of Intelligent Science and Technology Vol. 8, Issue 1, Pages: 47-60(2026)
- 作者机构：
  
  1.安徽大学人工智能学院，安徽合肥 230601
  2.北京科技大学计算机与通信工程学院，北京 100083
  3.美国普渡大学电子与计算机工程系，印第安纳州西拉法叶 IN 46204
  4.光电信息获取与防护技术全国重点实验室，安徽合肥 230031
  5.安徽大学，安徽合肥 230601
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62522601)
- DOI：10.11959/j.issn.2096-6652.202601
  CLC： TP391.4
- Received：27 September 2025，
  
  Revised：2025-12-08，
  
  Accepted：15 December 2025，
  
  Published：15 March 2026
- 稿件说明：
移动端阅览
尹守国,杜泉成,李灵犀等.基于多特征融合的行人过街意图推理方法[J].智能科学与技术学报,2026,08(01):47-60.

Yin Shouguo,Du Quancheng,Li Lingxi,et al.Pedestrian crossing intent inference method based on multi-feature fusion[J].Chinese Journal of Intelligent Science and Technology,2026,08(01):47-60.
尹守国,杜泉成,李灵犀等.基于多特征融合的行人过街意图推理方法[J].智能科学与技术学报,2026,08(01):47-60. DOI： 10.11959/j.issn.2096-6652.202601.

Yin Shouguo,Du Quancheng,Li Lingxi,et al.Pedestrian crossing intent inference method based on multi-feature fusion[J].Chinese Journal of Intelligent Science and Technology,2026,08(01):47-60. DOI： 10.11959/j.issn.2096-6652.202601.

摘要

准确理解和预测行人过街意图对自动驾驶汽车的行驶安全至关重要。现有方法多依赖于行人轨迹或整体身体姿态等视觉运动特征，而对行人与车辆之间的交互信号关注不足，因此难以捕捉行人通过手势、头部朝向等细微信号所传递的通行意图。为了解决这些限制，提出了准确推理行人过街意图（accurate reasoning for pedestrian crossing intent，ARPCI）框架，这是一个多特征融合框架。具体而言，设计了一个行人特征模块，该模块首先关注行人的骨架特征以捕捉行人的运动趋势，在此基础上利用MobileNet提取头部姿态特征，结合YOLOv8n识别手部动作，从而获得行人与车辆间的交互信号。此外，还引入了场景编码模块和自车特征模块，这能够有效融合环境上下文与车辆动态信息，增强模型对复杂交通场景的适应能力，提高对行人过街意图的预测准确率。在广泛使用的JAAD数据集上进行的实验表明，该方法准确率达到了88%，优于多个同类模型SOTA（state of the art），消融实验也进一步验证了各输入特征的有效性。

Abstract

Accurately understanding and predicting pedestrian crossing intent is crucial for ensuring the safety of autonomous vehicles. Existing approaches are often limited to visual motion cues such as pedestrian trajectories or body poses

while overlooking interactive signals like gestures and head orientations

making it difficult to capture key cues of pedestrian-vehicle interaction. To address these limitations

ARPCI (accurate reasoning for pedestrian crossing intent) was proposed

a multi-feature fusion framework designed for pedestrian intent inference. Specifically

a pedestrian feature module was developed that first focused on skeleton-based features to capture motion trends

and further leveraged MobileNet to extract head pose features. Combined with YOLOv8n for gesture recognition

pedestrian-vehicle interaction signals were captured more comprehensively by the model. In addition

a scene encoding module and a self-vehicle feature module were introduced to integrate contextual and ego-dynamic information

thereby enhancing adaptability to complex traffic environments and improving prediction accuracy. Extensive experiments on the widely used JAAD dataset show that the approach achieves an accuracy of 88%

surpassing several state-of-the-art counterparts. Moreover

the ablation studies provide further evidence of the effectiveness of the proposed input features.

关键词

Keywords

references

Hu J W, Flannagan C, Ganesan S, et al. Understanding the new trends in pedestrian injury distribution and mechanism through data linkage and modeling[J]. Accident; Analysis and Prevention, 2023, 188: 107095.

Wang X, Tang K, Dai X, et al. S4TP: social-suitable and safety-sensitive trajectory planning for autonomous vehicles[EB]. 2024.

Liu J M, Lin H, Wang X D, et al. Reliable trajectory prediction in scene fusion based on spatio-temporal structure causal model[J]. Information Fusion, 2024, 107: 102309.

Wang X D, Liu J M, Lin H, et al. A multi-modal spatial-temporal model for accurate motion forecasting with visual fusion[J]. Information Fusion, 2024, 102: 102046.

Landry F G, Akhloufi M A. Predicting pedestrian crossing intention in autonomous vehicles: a review[J]. Neurocomputing, 2025, 618: 129105.

Du Q C, Xu L L, Wu Q, et al. SafeCrossNet: multi-modal fusion with social-aware for pedestrian crossing intention prediction[J]. Information Fusion, 2026, 126: 103609.

Li Z R, Gong C, Lin Y L, et al. Continual driver behaviour learning for connected vehicles and intelligent transportation systems: framework, survey and challenges[J]. Green Energy and Intelligent Transportation, 2023, 2(4): 100103.

Du Q C, Wu Q, Li L X, et al. Review and perspectives on pedestrian trajectory prediction for safe transportation[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, PP(99): 1-27.

Wang X, Liu J G, Mei T, et al. CoSeg: cognitively inspired unsupervised generic event segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(9): 12507-12517.

Wang X, Wu Z Z, Jiang B, et al. HARDVS: revisiting human activity recognition with dynamic vision sensors[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Piscataway: AAAI Press, 2024, 38(6): 5615-5623.

Azarmi M, Rezaei M, Wang H. PIP-net: pedestrian intention prediction in the wild[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, 26(7): 9824-9837.

Wang X, Tang K, Dai X Y, et al. Safety-balanced driving-style aware trajectory planning in intersection scenarios with uncertain environment[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(4): 2888-2898.

Du Q C, Wang X, Yin S G, et al. Social force embedded mixed graph convolutional network for multi-class trajectory prediction[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(9): 5571-5580.

Helbing D, Molnár P. Social force model for pedestrian dynamics[J]. Physical Review E, Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics, 1995, 51(5): 4282-4286.

Fang Z J, Vázquez D, López A M. On-board detection of pedestrian intentions[J]. Sensors, 2017, 17(10): 2193.

Zhang W, Zhu F H, Chen Y Y, et al. Differential time-variant traffic flow prediction based on deep learning[C]//Proceedings of the 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). Piscataway: IEEE Press, 2020: 1-6.

Wang J G, Wang X, Shen T Y, et al. A long-tail regularization method for traffic sign recognition based on parallel vision[J]. IEEE Journal of Radio Frequency Identification, 2022, 6: 957-961.

Li F, Fan S W, Chen P Z, et al. Pedestrian motion state estimation from 2D pose[C]//Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV). Piscataway: IEEE Press, 2020: 1682-1687.

Varytimidis D, Alonso-Fernandez F, Duran B, et al. Action and intention recognition of pedestrians in urban traffic[C]//Proceedings of the 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). Piscataway: IEEE Press, 2018: 676-682.

Perdana M I, Anggraeni W, Sidharta H A, et al. Early warning pedestrian crossing intention from its head gesture using head pose estimation[C]//Proceedings of the 2021 International Seminar on Intelligent Technology and Its Applications (ISITIA). Piscataway: IEEE Press, 2021: 402-407.

杜泉成, 王晓, 李灵犀, 等. 行人轨迹预测方法关键问题研究: 现状及展望[J]. 智能科学与技术学报, 2023, 5(2): 143-162.

Du Q C, Wang X, Li L X, et al. Key problems and progress of pedestrian trajectory prediction methods: the state of the art and prospects[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(2): 143-162.

李琳辉, 周彬, 任威威, 等. 行人轨迹预测方法综述[J]. 智能科学与技术学报, 2021, 3(4): 399-411.

Li L H, Zhou B, Ren W W, et al. Review of pedestrian trajectory prediction methods[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 399-411.

Guo J R, Ding Y T, Tian A S. Multimodal feature fusion for pedestrian crossing intention prediction based on hybrid attention mechanism[C]//Proceedings of the 2024 6th International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI). Piscataway: IEEE Press, 2024: 70-74.

Dey D, Terken J. Pedestrian interaction with vehicles: roles of explicit and implicit communication[C]//Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. New York: ACM Press, 2017: 109-113.

Gupta A, Johnson J, Li F F, et al. Social GAN: socially acceptable trajectories with generative adversarial networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 2255-2264.

Lin Y W, Hu C, Zhao B X, et al. Anchor-based multi-modal transformer network for pedestrian trajectory and intention prediction[C]//Proceedings of the 2023 7th CAA International Conference on Vehicular Control and Intelligence (CVCI). Piscataway: IEEE Press, 2023: 1-6.

Wang Y, Wan W X, Zhang H Q, et al. Pedestrian trajectory intention prediction in autonomous driving scenarios based on spatio-temporal attention mechanism[C]//Proceedings of the 2024 4th International Conference on Electronic Information Engineering and Computer Communication (EIECC). Piscataway: IEEE Press, 2024: 1519-1522.

Mohamed A, Qian K, Elhoseiny M, et al. Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2020: 14412-14420.

Tang W X, Liu K, Shakeel M S, et al. DDAD: detachable crowd density estimation assisted pedestrian detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(2): 1867-1878.

胡远志, 蒋涛, 刘西, 等. 基于双流自适应图卷积神经网络的行人过街意图识别[J]. 汽车安全与节能学报, 2022, 13(2): 325-332.

Hu Y Z, Jiang T, Liu X, et al. Pedestrian-crossing intention-recognition based on dual-stream adaptive graph-convolutional neural-network[J]. Journal of Automotive Safety and Energy, 2022, 13(2): 325-332.

Chaabane M, Trabelsi A, Blanchard N, et al. Looking ahead: anticipating pedestrians crossing with future frames prediction[C]//Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2020: 2286-2295.

Xie C, Lin C Y, Zheng X Y, et al. GTransPDM: a graph-embedded transformer with positional decoupling for pedestrian crossing intention prediction[J]. IEEE Signal Processing Letters, 2025, 32: 2109-2113.

Cadena P R G, Qian Y Q, Wang C X, et al. Pedestrian graph: a fast pedestrian crossing prediction model based on graph convolutional networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 21050-21061.

Liu B B, Adeli E, Cao Z J, et al. Spatiotemporal relationship reasoning for pedestrian intent prediction[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3485-3492.

Piccoli F, Balakrishnan R, Perez M J, et al. FuSSI-Net: fusion of spatio-temporal skeletons for intention prediction network[EB]. 2020.

Yang T Y, Chen Y T, Lin Y, et al. FSA-net: learning fine-grained structure aggregation for head pose estimation from a single image[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2019: 1087-1096.

Zhang J Y, Xu Y N, Chen X. Multimodal-based pedestrian crossing intention recognition method[C]//Proceedings of the 2023 China Automation Congress (CAC). Piscataway: IEEE Press, 2023: 3508-3513.

Chen H, Wang Y, Guo J, et al. Vanillanet: the power of minimalism in deep learning[J]. Advances in Neural Information Processing Systems, 2023, 36: 7050-7064.

Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV) 2018. Cham: Springer, 2018: 3-19.

Wang X, Huang J, Tian Y L, et al. Parallel driving with big models and foundation intelligence in cyber-physical-social spaces[J]. Research, 2024, 7: 0349.

Kotseruba I, Rasouli A, Tsotsos J K. Joint attention in autonomous driving (JAAD)[J]. 2016.

Bhattacharyya A, Fritz M, Schiele B. Long-term on-board prediction of people in traffic scenes under uncertainty[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4194-4202.

Rasouli A, Kotseruba I, Tsotsos J K. Pedestrian action anticipation using contextual feature fusion in stacked rnns[EB]. 2020.

Gesnouin J, Pechberti S, Stanciulcscu B, et al. TrouSPI-Net: spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction[C]//Proceedings of the 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021). New York: ACM Press, 2021: 1-7.

Kotseruba I, Rasouli A, Tsotsos J K. Benchmark for evaluating pedestrian action prediction[C]//Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2021: 1257-1267.

Yang D F, Zhang H L, Yurtsever E, et al. Predicting pedestrian crossing intention with feature fusion and spatio-temporal attention[J]. IEEE Transactions on Intelligent Vehicles, 2022, 7(2): 221-230.

Rasouli A, Yau T, Rohani M, et al. Multi-modal hybrid architecture for pedestrian action prediction[C]//Proceedings of the 2022 IEEE Intelligent Vehicles Symposium (IV). Piscataway: IEEE Press, 2022: 91-97.

Alofi A, Greer R, Gopalkrishnan A, et al. Pedestrian safety by intent prediction: a lightweight LSTM-attention architecture and experimental evaluations with real-world datasets[C]//Proceedings of the 2024 IEEE Intelligent Vehicles Symposium (IV). Piscataway: IEEE Press, 2024: 77-84.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Cerberus: a cross-site social bot detection system based on deep learning

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰