基于CAM-MADDPG的水下物联网目标节点协同追踪算法

林川; 董芮含; 韩光洁; 田凯; 马家奥

doi:10.11959/j.issn.2096-3750.WLW26053

Chinese

您当前的位置：

首页 >

文章列表页 >

基于CAM-MADDPG的水下物联网目标节点协同追踪算法

更新时间：2026-09-22

- 基于CAM-MADDPG的水下物联网目标节点协同追踪算法
- A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG
- 物联网学报 2026年
- 作者机构：
  
  1.东北大学软件学院，辽宁沈阳 110819
  2.河海大学信息工程与科学学院，江苏常州 213200
- 作者简介：
  
  林川（1988‒ ），男，东北大学软件学院副教授，主要研究方向为多智能体人工智能。
  董芮含（2002‒ ），女，东北大学软件学院在读，主要研究方向为AUV水下无人机。
  韩光洁（1972‒ ），男，河海大学二级教授，信息学院院长，主要研究方向为多智能体人工智能、水下物联网。
  田凯（2005-），男，东北大学软件学院在读，主要研究方向是多智能体强化学习、机器学习。
  马家奥(2006-)，男，东北大学软件学院在读，主要研究方向为强化学习，人工智能。
- 基金信息：
  
  国家自然科学基金资助项目(62572107;62572169)
- DOI：10.11959/j.issn.2096-3750.WLW26053
  中图分类号：
- 收稿：2026-05-06，
  
  修回：2026-09-06，
  
  录用：2026-09-18，
- 稿件说明：
移动端阅览
林川, 董芮含, 韩光洁, 等. 基于CAM-MADDPG的水下物联网目标节点协同追踪算法[J/OL]. 物联网学报, 2026.

LIN Chuan, DONG Ruihan, HAN Guangjie, et al. A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG[J/OL]. Chinese Journal on Internet of Things, 2026.
林川, 董芮含, 韩光洁, 等. 基于CAM-MADDPG的水下物联网目标节点协同追踪算法[J/OL]. 物联网学报, 2026. DOI： 10.11959/j.issn.2096-3750.WLW26053.

LIN Chuan, DONG Ruihan, HAN Guangjie, et al. A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG[J/OL]. Chinese Journal on Internet of Things, 2026. DOI： 10.11959/j.issn.2096-3750.WLW26053.

摘要

水下物联网（Internet of Underwater Things

IoUT）中目标节点的协同追踪是实现水下感知网络持续覆盖与数据采集的关键环节。针对多AUV协同追踪目标节点任务中高维观测冗余、关键特征动态识别困难及多智能体协同决策稳定性不足等问题，本文在多智能体深度确定性策略梯度（MADDPG）框架基础上，提出基于因果注意力机制的CAM-MADDPG算法。该方法利用PC算法构建状态、动作与奖励之间的有向无环因果图，依据特征节点的因果连通关系生成注意力权重掩码，实现对关键观测特征的自适应聚焦，并引入KL散度约束与反事实干预机制保障权重稳定更新。在此基础上，设计多层次综合奖励函数，构建面向水下物联网场景的多AUV协同节点追踪决策框架，并在理想化协同信息可用条件下验证所提方法的决策性能。仿真结果表明，CAM-MADDPG在追踪精度、速度一致性及回报收敛稳定性等指标上均优于MADDPG、MAAC、MASAC、MATD3等基线方法，验证了所提方法的有效性与鲁棒性。

Abstract

Collaborative tracking of target nodes within the Internet of Underwater Things (IoUT) is a crucial component for achieving continuous coverage and data acquisition in underwater sensing networks. Addressing challenges in multi-AUV collaborative tracking of target nodes

such as high-dimensional observation redundancy

difficulties in identifying critical dynamic features

and insufficient stability in multi-agent collaborative decision-making

this paper proposes the CAM-MADDPG algorithm based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework. The proposed method leverages the PC algorithm to construct a directed acyclic causal graph among states

actions

and rewards. Based on the causal connectivity of feature nodes

it generates an attention weight mask to achieve adaptive focusing on critical observational features. Furthermore

it introduces a KL divergence constraint and a counterfactual intervention mechanism to ensure stable weight updates. On this basis

we design a multi-level composite reward function and construct a multi-AUV cooperative node tracking decision-making framework tailored for underwater Internet of Things (UIoT) scenarios. The decision-making performance of the proposed approach is evaluated under an idealized condition where collaborative information among agents is assumed to be fully available. Simulation results demonstrate that CAM-MADDPG outperforms baseline methods such as MADDPG

MAAC

MASAC

and MATD3 in metrics including tracking accuracy

velocity consistency

and return convergence stability

thereby validating the effectiveness and robustness of the proposed method.

关键词

Keywords

references

Qiu T , Zhao Z , Zhang T , et al . Underwater internet of things in smart ocean: system architecture and open issues [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 : 4297 - 4307 .

Luo J , Yang Y , Wang Z , et al . Localization algorithm for underwater sensor network: a review [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 17 ): 13126 - 13144 .

张新文 , 刘同木 , 周保成 , 等 . 基于物联网的浮标数据采集与控制系统设计 [J ] . 海洋科学进展 , 2024 , 42 ( 1 ): 196 - 206 .

ZHANG X W , LIU T M , ZHOU B C , et al . Design of buoy data acquisition and control system based on Internet of Things [J ] . Advances in Marine Science , 2024 , 42 ( 1 ): 196 - 206 .

Yang Y , Xiao Y , Li T . A survey of autonomous underwater vehicle formation: performance, formation control, and communication capability [J ] . IEEE Communications Surveys & Tutorials , 2021 , 23 ( 2 ): 815 - 841 .

Nkenyereye L , Nkenyereye L , Ndibanje B . Internet of underwater things: a survey on simulation tools and 5G-based underwater networks [J ] . Electronics , 2024 .

Jahanbakht M , Xiang W , Hanzo L , et al . Internet of underwater things and big marine data analytics—a comprehensive survey [J ] . IEEE Communications Surveys & Tutorials , 2021 , 23 ( 2 ): 904 - 956 .

Wen M , Kuba J G , Lin R , et al . Multi-agent reinforcement learning is a sequence modeling problem [J/OL ] . arXiv preprint , 2022 , arXiv: 2205 . 14953 . https://arxiv.org/abs/2205.14953 https://arxiv.org/abs/2205.14953 .

罗彪 , 胡天萌 , 周育豪 , 等 . 多智能体强化学习控制与决策研究综述 [J ] . 自动化学报 , 2025 , 51 ( 3 ): 510 - 539 .

Luo B , Hu T M , Zhou Y H , et al . Survey on multi-agent reinforcement learning for control and decision-making [J ] . Acta Automatica Sinica , 2025 , 51 ( 3 ): 510 - 539 .

Wang Y , Zhang Y , Jia L . Multi-UAV maneuvering targets tracking based on mobility-focused multi-agent deep deterministic policy gradient [C ] // 2024 3rd International Conference on Automation, Robotics and Computer Engineering (ICARCE) . China : IEEE , 2024 : 498 - 502 .

Li S , Wu Y , Cui X , et al . Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient [C ] // Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence . Honolulu : AAAI Press , 2019 .

Zuo W , Liao S , Cui Y , et al . Multi-agent deep reinforcement learning-based distributed task assignment in multi-UAV cooperative edge computing [J ] . IEEE Transactions on Cloud Computing , 2026 .

Wibisono A , Song H K , Lee B M . A survey of multi-agent reinforcement learning for cooperative control in multi-AUV systems [J ] . IEEE Access , 2025 , 13 : 161505 - 161528 .

Lu K , Li R D , Li M C , et al . MADDPG-based joint optimization of task partitioning and computation resource allocation in mobile edge computing [J ] . Neural Computing and Applications , 2023 , 35 ( 22 ): 16559 - 16576 .

Wu R , Zhong J , Wallace B A , et al . Human-robotic prosthesis as collaborating agents for symmetrical walking [C ] // Proceedings of the 36th International Conference on Neural Information Processing Systems . New Orleans : Curran Associates Inc. , 2022 .

于长东 , 刘新阳 , 陈聪 , 等 . 基于多智能体深度强化学习的无人艇集群博弈对抗研究 [J ] . 水下无人系统学报 , 2024 , 32 ( 1 ): 79 - 86 .

YU C D , LIU X Y , CHEN C , et al . Research on game confrontation of unmanned surface vehicles swarm based on multi-agent deep reinforcement learning [J ] . Journal of Unmanned Undersea Systems , 2024 , 32 ( 1 ): 79 - 86 .

Iqbal S , Sha F . Actor-attention-critic for multi-agent reinforcement learning [C ] // Proceedings of the 36th International Conference on Machine Learning (ICML) . 2019 , 97 : 2961 - 2970 .

Su C , Yan Y , Wang T , et al . A graph attention mechanism based multi-agent reinforcement learning method for efficient traffic light control [C ] // 2021 International Wireless Communications and Mobile Computing (IWCMC) . Harbin : IEEE , 2021 : 1332 - 1337 .

Ni K , Chen J , Wang J , et al . Multi-agent reinforcement learning with multi-head attention [C ] // 2023 IEEE 6th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) . Chongqing : IEEE , 2023 : 1311 - 1315 .

Grimbly S J , Shock J , Pretorius A . Causal multi-agent reinforcement learning: review and open problems [J/OL ] . arXiv preprint , 2021 , arXiv: 2111 . 06721 . https://arxiv.org/abs/2111.06721 https://arxiv.org/abs/2111.06721 .

Zhu S , Han G , Lin C , et al . Underwater target tracking based on hierarchical software-defined multi-AUV reinforcement learning: a multi-AUV advantage-attention actor-critic approach [J ] . IEEE Transactions on Mobile Computing , 2024 , 23 ( 12 ): 13639 - 13653 .

Kalisch M , Bühlmann P . Estimating high-dimensional directed acyclic graphs with the PC-algorithm [J ] . Journal of Machine Learning Research , 2007 , 8 : 613 - 636 .

李聪 , 张震 , 刘鹏昌 . 基于KL散度和迁移强化学习的AMR路径规划算法 [J ] . 电光与控制 , 2026 , 33 ( 01 ): 78 - 83+90 .

Li C , Zhang Z , Liu P C . AMR path planning algorithm based on KL divergence and transfer reinforcement learning [J ] . Electronics Optics & Control , 2026 , 33 ( 1 ): 78 - 83, 90 .

Foerster J N , Farquhar G , Afouras T , et al . Counterfactual multi-agent policy gradients [C ] // Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence . New Orleans : AAAI Press , 2018 .

Ororbia M E , Warn G P . Design synthesis through a Markov decision process and reinforcement learning framework [J ] . Journal of Computing and Information Science in Engineering , 2022 , 22 ( 2 ): 021002 .

Lowe R , Wu Y , Tamar A , et al . Multi-agent actor-critic for mixed cooperative-competitive environments [J/OL ] . arXiv preprint , 2017 , arXiv: 1706 . 02275 . https://arxiv.org/abs/1706.02275 https://arxiv.org/abs/1706.02275 .

Benson H P . Multi-objective optimization: pareto optimal solutions, properties [M ] //Floudas C A, Pardalos P M. Encyclopedia of optimization . Boston : Springer , 2001 .

Jiang Y , Renner B C . Low-cost underwater swarm acoustic localization: a review [J ] . IEEE Access , 2024 , 12 : 25779 - 25796 .

Lyu X , Sun Y , Wang L , et al . End-to-end AUV local motion planning method based on deep reinforcement learning [J ] . Journal of Marine Science and Engineering , 2023 , 11 ( 9 ): 1796 .

Fujimoto S , Hoof H van , Meger D . Addressing function approximation error in actor-critic methods [C ] // Proceedings of the 35th International Conference on Machine Learning . Stockholm : PMLR , 2018 , 80 : 1587 - 1596 .

方城亮 , 杨飞生 , 潘泉 . 基于MASAC强化学习算法的多无人机协同路径规划 [J ] . 中国科学（信息科学） , 2024 ( 8 ).

Fang C L , Yang F S , Pan Q . Multi-UAV cooperative path planning based on MASAC reinforcement learning algorithm [J ] . Scientia Sinica (Informationis) , 2024 ( 8 ).

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向应急响应群智感知的异构群体协作任务分配方法

基于注意力机制的轻量级SAR船舶检测器

融合多尺度和上下文的无人机救援小目标检测算法

一种基于深度可分离卷积和注意力机制的入侵检测方法

基于多智能体深度强化学习的多域协同抗干扰方法研究