基于多特征融合的行人过街意图推理方法

尹守国; 杜泉成; 李灵犀; 王晓; 孙长银

doi:10.11959/j.issn.2096-6652.202601

Chinese

您当前的位置：

首页 >

文章列表页 >

基于多特征融合的行人过街意图推理方法

学术论文 | 更新时间：2026-04-27

- 基于多特征融合的行人过街意图推理方法
- Pedestrian crossing intent inference method based on multi-feature fusion
- 智能科学与技术学报 2026年8卷第1期页码：47-60
- 作者机构：
  
  1.安徽大学人工智能学院，安徽合肥 230601
  2.北京科技大学计算机与通信工程学院，北京 100083
  3.美国普渡大学电子与计算机工程系，印第安纳州西拉法叶 IN 46204
  4.光电信息获取与防护技术全国重点实验室，安徽合肥 230031
  5.安徽大学，安徽合肥 230601
- 作者简介：
  
  [ "尹守国（2002- ），男，安徽大学人工智能学院硕士生，主要研究方向为意图推理、轨迹预测、智能交通系统。" ]
  [ "杜泉成（1994- ），男，北京科技大学计算机与通信工程学院博士生，主要研究方向为轨迹预测和车辆规划决策。" ]
  [ "李灵犀（1977- ），男，博士，美国普渡大学电子与计算机工程系教授，主要研究方向为复杂系统的建模、分析、控制与优化、智能交通系统、离散事件动态系统。" ]
  [ "王晓（1988- ），女，安徽大学人工智能学院教授，主要研究方向为社会计算、群体行为建模、无人自主系统及其平行测试。" ]
  [ "孙长银（1975- ），男，安徽大学校长，主要研究方向为智能控制与优化、强化学习、神经网络和数据驱动控制。" ]
- 基金信息：
  
  国家自然科学基金项目(62522601)
- DOI：10.11959/j.issn.2096-6652.202601
  中图分类号： TP391.4
- 收稿：2025-09-27，
  
  修回：2025-12-08，
  
  录用：2025-12-15，
  
  纸质出版：2026-03-15
- 稿件说明：
移动端阅览
尹守国,杜泉成,李灵犀等.基于多特征融合的行人过街意图推理方法[J].智能科学与技术学报,2026,08(01):47-60.

Yin Shouguo,Du Quancheng,Li Lingxi,et al.Pedestrian crossing intent inference method based on multi-feature fusion[J].Chinese Journal of Intelligent Science and Technology,2026,08(01):47-60.
尹守国,杜泉成,李灵犀等.基于多特征融合的行人过街意图推理方法[J].智能科学与技术学报,2026,08(01):47-60. DOI： 10.11959/j.issn.2096-6652.202601.

Yin Shouguo,Du Quancheng,Li Lingxi,et al.Pedestrian crossing intent inference method based on multi-feature fusion[J].Chinese Journal of Intelligent Science and Technology,2026,08(01):47-60. DOI： 10.11959/j.issn.2096-6652.202601.

摘要

准确理解和预测行人过街意图对自动驾驶汽车的行驶安全至关重要。现有方法多依赖于行人轨迹或整体身体姿态等视觉运动特征，而对行人与车辆之间的交互信号关注不足，因此难以捕捉行人通过手势、头部朝向等细微信号所传递的通行意图。为了解决这些限制，提出了准确推理行人过街意图（accurate reasoning for pedestrian crossing intent，ARPCI）框架，这是一个多特征融合框架。具体而言，设计了一个行人特征模块，该模块首先关注行人的骨架特征以捕捉行人的运动趋势，在此基础上利用MobileNet提取头部姿态特征，结合YOLOv8n识别手部动作，从而获得行人与车辆间的交互信号。此外，还引入了场景编码模块和自车特征模块，这能够有效融合环境上下文与车辆动态信息，增强模型对复杂交通场景的适应能力，提高对行人过街意图的预测准确率。在广泛使用的JAAD数据集上进行的实验表明，该方法准确率达到了88%，优于多个同类模型SOTA（state of the art），消融实验也进一步验证了各输入特征的有效性。

Abstract

Accurately understanding and predicting pedestrian crossing intent is crucial for ensuring the safety of autonomous vehicles. Existing approaches are often limited to visual motion cues such as pedestrian trajectories or body poses

while overlooking interactive signals like gestures and head orientations

making it difficult to capture key cues of pedestrian-vehicle interaction. To address these limitations

ARPCI (accurate reasoning for pedestrian crossing intent) was proposed

a multi-feature fusion framework designed for pedestrian intent inference. Specifically

a pedestrian feature module was developed that first focused on skeleton-based features to capture motion trends

and further leveraged MobileNet to extract head pose features. Combined with YOLOv8n for gesture recognition

pedestrian-vehicle interaction signals were captured more comprehensively by the model. In addition

a scene encoding module and a self-vehicle feature module were introduced to integrate contextual and ego-dynamic information

thereby enhancing adaptability to complex traffic environments and improving prediction accuracy. Extensive experiments on the widely used JAAD dataset show that the approach achieves an accuracy of 88%

surpassing several state-of-the-art counterparts. Moreover

the ablation studies provide further evidence of the effectiveness of the proposed input features.

关键词

Keywords

references

Hu J W, Flannagan C, Ganesan S, et al. Understanding the new trends in pedestrian injury distribution and mechanism through data linkage and modeling[J]. Accident; Analysis and Prevention, 2023, 188: 107095.

Wang X, Tang K, Dai X, et al. S4TP: social-suitable and safety-sensitive trajectory planning for autonomous vehicles[EB]. 2024.

Liu J M, Lin H, Wang X D, et al. Reliable trajectory prediction in scene fusion based on spatio-temporal structure causal model[J]. Information Fusion, 2024, 107: 102309.

Wang X D, Liu J M, Lin H, et al. A multi-modal spatial-temporal model for accurate motion forecasting with visual fusion[J]. Information Fusion, 2024, 102: 102046.

Landry F G, Akhloufi M A. Predicting pedestrian crossing intention in autonomous vehicles: a review[J]. Neurocomputing, 2025, 618: 129105.

Du Q C, Xu L L, Wu Q, et al. SafeCrossNet: multi-modal fusion with social-aware for pedestrian crossing intention prediction[J]. Information Fusion, 2026, 126: 103609.

Li Z R, Gong C, Lin Y L, et al. Continual driver behaviour learning for connected vehicles and intelligent transportation systems: framework, survey and challenges[J]. Green Energy and Intelligent Transportation, 2023, 2(4): 100103.

Du Q C, Wu Q, Li L X, et al. Review and perspectives on pedestrian trajectory prediction for safe transportation[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, PP(99): 1-27.

Wang X, Liu J G, Mei T, et al. CoSeg: cognitively inspired unsupervised generic event segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(9): 12507-12517.

Wang X, Wu Z Z, Jiang B, et al. HARDVS: revisiting human activity recognition with dynamic vision sensors[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Piscataway: AAAI Press, 2024, 38(6): 5615-5623.

Azarmi M, Rezaei M, Wang H. PIP-net: pedestrian intention prediction in the wild[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, 26(7): 9824-9837.

Wang X, Tang K, Dai X Y, et al. Safety-balanced driving-style aware trajectory planning in intersection scenarios with uncertain environment[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(4): 2888-2898.

Du Q C, Wang X, Yin S G, et al. Social force embedded mixed graph convolutional network for multi-class trajectory prediction[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(9): 5571-5580.

Helbing D, Molnár P. Social force model for pedestrian dynamics[J]. Physical Review E, Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics, 1995, 51(5): 4282-4286.

Fang Z J, Vázquez D, López A M. On-board detection of pedestrian intentions[J]. Sensors, 2017, 17(10): 2193.

Zhang W, Zhu F H, Chen Y Y, et al. Differential time-variant traffic flow prediction based on deep learning[C]//Proceedings of the 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). Piscataway: IEEE Press, 2020: 1-6.

Wang J G, Wang X, Shen T Y, et al. A long-tail regularization method for traffic sign recognition based on parallel vision[J]. IEEE Journal of Radio Frequency Identification, 2022, 6: 957-961.

Li F, Fan S W, Chen P Z, et al. Pedestrian motion state estimation from 2D pose[C]//Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV). Piscataway: IEEE Press, 2020: 1682-1687.

Varytimidis D, Alonso-Fernandez F, Duran B, et al. Action and intention recognition of pedestrians in urban traffic[C]//Proceedings of the 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). Piscataway: IEEE Press, 2018: 676-682.

Perdana M I, Anggraeni W, Sidharta H A, et al. Early warning pedestrian crossing intention from its head gesture using head pose estimation[C]//Proceedings of the 2021 International Seminar on Intelligent Technology and Its Applications (ISITIA). Piscataway: IEEE Press, 2021: 402-407.

杜泉成, 王晓, 李灵犀, 等. 行人轨迹预测方法关键问题研究: 现状及展望[J]. 智能科学与技术学报, 2023, 5(2): 143-162.

Du Q C, Wang X, Li L X, et al. Key problems and progress of pedestrian trajectory prediction methods: the state of the art and prospects[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(2): 143-162.

李琳辉, 周彬, 任威威, 等. 行人轨迹预测方法综述[J]. 智能科学与技术学报, 2021, 3(4): 399-411.

Li L H, Zhou B, Ren W W, et al. Review of pedestrian trajectory prediction methods[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 399-411.

Guo J R, Ding Y T, Tian A S. Multimodal feature fusion for pedestrian crossing intention prediction based on hybrid attention mechanism[C]//Proceedings of the 2024 6th International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI). Piscataway: IEEE Press, 2024: 70-74.

Dey D, Terken J. Pedestrian interaction with vehicles: roles of explicit and implicit communication[C]//Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. New York: ACM Press, 2017: 109-113.

Gupta A, Johnson J, Li F F, et al. Social GAN: socially acceptable trajectories with generative adversarial networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 2255-2264.

Lin Y W, Hu C, Zhao B X, et al. Anchor-based multi-modal transformer network for pedestrian trajectory and intention prediction[C]//Proceedings of the 2023 7th CAA International Conference on Vehicular Control and Intelligence (CVCI). Piscataway: IEEE Press, 2023: 1-6.

Wang Y, Wan W X, Zhang H Q, et al. Pedestrian trajectory intention prediction in autonomous driving scenarios based on spatio-temporal attention mechanism[C]//Proceedings of the 2024 4th International Conference on Electronic Information Engineering and Computer Communication (EIECC). Piscataway: IEEE Press, 2024: 1519-1522.

Mohamed A, Qian K, Elhoseiny M, et al. Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2020: 14412-14420.

Tang W X, Liu K, Shakeel M S, et al. DDAD: detachable crowd density estimation assisted pedestrian detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(2): 1867-1878.

胡远志, 蒋涛, 刘西, 等. 基于双流自适应图卷积神经网络的行人过街意图识别[J]. 汽车安全与节能学报, 2022, 13(2): 325-332.

Hu Y Z, Jiang T, Liu X, et al. Pedestrian-crossing intention-recognition based on dual-stream adaptive graph-convolutional neural-network[J]. Journal of Automotive Safety and Energy, 2022, 13(2): 325-332.

Chaabane M, Trabelsi A, Blanchard N, et al. Looking ahead: anticipating pedestrians crossing with future frames prediction[C]//Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2020: 2286-2295.

Xie C, Lin C Y, Zheng X Y, et al. GTransPDM: a graph-embedded transformer with positional decoupling for pedestrian crossing intention prediction[J]. IEEE Signal Processing Letters, 2025, 32: 2109-2113.

Cadena P R G, Qian Y Q, Wang C X, et al. Pedestrian graph: a fast pedestrian crossing prediction model based on graph convolutional networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 21050-21061.

Liu B B, Adeli E, Cao Z J, et al. Spatiotemporal relationship reasoning for pedestrian intent prediction[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3485-3492.

Piccoli F, Balakrishnan R, Perez M J, et al. FuSSI-Net: fusion of spatio-temporal skeletons for intention prediction network[EB]. 2020.

Yang T Y, Chen Y T, Lin Y, et al. FSA-net: learning fine-grained structure aggregation for head pose estimation from a single image[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2019: 1087-1096.

Zhang J Y, Xu Y N, Chen X. Multimodal-based pedestrian crossing intention recognition method[C]//Proceedings of the 2023 China Automation Congress (CAC). Piscataway: IEEE Press, 2023: 3508-3513.

Chen H, Wang Y, Guo J, et al. Vanillanet: the power of minimalism in deep learning[J]. Advances in Neural Information Processing Systems, 2023, 36: 7050-7064.

Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV) 2018. Cham: Springer, 2018: 3-19.

Wang X, Huang J, Tian Y L, et al. Parallel driving with big models and foundation intelligence in cyber-physical-social spaces[J]. Research, 2024, 7: 0349.

Kotseruba I, Rasouli A, Tsotsos J K. Joint attention in autonomous driving (JAAD)[J]. 2016.

Bhattacharyya A, Fritz M, Schiele B. Long-term on-board prediction of people in traffic scenes under uncertainty[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4194-4202.

Rasouli A, Kotseruba I, Tsotsos J K. Pedestrian action anticipation using contextual feature fusion in stacked rnns[EB]. 2020.

Gesnouin J, Pechberti S, Stanciulcscu B, et al. TrouSPI-Net: spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction[C]//Proceedings of the 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021). New York: ACM Press, 2021: 1-7.

Kotseruba I, Rasouli A, Tsotsos J K. Benchmark for evaluating pedestrian action prediction[C]//Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE Press, 2021: 1257-1267.

Yang D F, Zhang H L, Yurtsever E, et al. Predicting pedestrian crossing intention with feature fusion and spatio-temporal attention[J]. IEEE Transactions on Intelligent Vehicles, 2022, 7(2): 221-230.

Rasouli A, Yau T, Rohani M, et al. Multi-modal hybrid architecture for pedestrian action prediction[C]//Proceedings of the 2022 IEEE Intelligent Vehicles Symposium (IV). Piscataway: IEEE Press, 2022: 91-97.

Alofi A, Greer R, Gopalkrishnan A, et al. Pedestrian safety by intent prediction: a lightweight LSTM-attention architecture and experimental evaluations with real-world datasets[C]//Proceedings of the 2024 IEEE Intelligent Vehicles Symposium (IV). Piscataway: IEEE Press, 2024: 77-84.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

Cerberus：基于深度学习的跨站社交网络机器人检测系统