A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG

LIN Chuan; DONG Ruihan; HAN Guangjie; TIAN Kai; MA Jao

doi:10.11959/j.issn.2096-3750.WLW26053

Chinese

您当前的位置：

首页 >

文章列表页 >

A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG

更新时间：2026-09-22

- A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG
- Chinese Journal on Internet of Things (2026)
- 作者机构：
  
  1.东北大学软件学院，辽宁沈阳 110819
  2.河海大学信息工程与科学学院，江苏常州 213200
- 作者简介：
- 基金信息：
  
  The National Natural Science Foundation of China(62572107;62572169)
- DOI：10.11959/j.issn.2096-3750.WLW26053
  CLC：
- Received：06 May 2026，
  
  Revised：2026-09-06，
  
  Accepted：18 September 2026，
- 稿件说明：
移动端阅览
LIN Chuan, DONG Ruihan, HAN Guangjie, et al. A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG[J/OL]. Chinese Journal on Internet of Things, 2026.
DOI：

LIN Chuan, DONG Ruihan, HAN Guangjie, et al. A Cooperative Tracking Algorithm for Underwater IoT Target Nodes Based on CAM-MADDPG[J/OL]. Chinese Journal on Internet of Things, 2026. DOI： 10.11959/j.issn.2096-3750.WLW26053.

摘要

水下物联网（Internet of Underwater Things

IoUT）中目标节点的协同追踪是实现水下感知网络持续覆盖与数据采集的关键环节。针对多AUV协同追踪目标节点任务中高维观测冗余、关键特征动态识别困难及多智能体协同决策稳定性不足等问题，本文在多智能体深度确定性策略梯度（MADDPG）框架基础上，提出基于因果注意力机制的CAM-MADDPG算法。该方法利用PC算法构建状态、动作与奖励之间的有向无环因果图，依据特征节点的因果连通关系生成注意力权重掩码，实现对关键观测特征的自适应聚焦，并引入KL散度约束与反事实干预机制保障权重稳定更新。在此基础上，设计多层次综合奖励函数，构建面向水下物联网场景的多AUV协同节点追踪决策框架，并在理想化协同信息可用条件下验证所提方法的决策性能。仿真结果表明，CAM-MADDPG在追踪精度、速度一致性及回报收敛稳定性等指标上均优于MADDPG、MAAC、MASAC、MATD3等基线方法，验证了所提方法的有效性与鲁棒性。

Abstract

Collaborative tracking of target nodes within the Internet of Underwater Things (IoUT) is a crucial component for achieving continuous coverage and data acquisition in underwater sensing networks. Addressing challenges in multi-AUV collaborative tracking of target nodes

such as high-dimensional observation redundancy

difficulties in identifying critical dynamic features

and insufficient stability in multi-agent collaborative decision-making

this paper proposes the CAM-MADDPG algorithm based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework. The proposed method leverages the PC algorithm to construct a directed acyclic causal graph among states

actions

and rewards. Based on the causal connectivity of feature nodes

it generates an attention weight mask to achieve adaptive focusing on critical observational features. Furthermore

it introduces a KL divergence constraint and a counterfactual intervention mechanism to ensure stable weight updates. On this basis

we design a multi-level composite reward function and construct a multi-AUV cooperative node tracking decision-making framework tailored for underwater Internet of Things (UIoT) scenarios. The decision-making performance of the proposed approach is evaluated under an idealized condition where collaborative information among agents is assumed to be fully available. Simulation results demonstrate that CAM-MADDPG outperforms baseline methods such as MADDPG

MAAC

MASAC

and MATD3 in metrics including tracking accuracy

velocity consistency

and return convergence stability

thereby validating the effectiveness and robustness of the proposed method.

关键词

Keywords

references

Qiu T , Zhao Z , Zhang T , et al . Underwater internet of things in smart ocean: system architecture and open issues [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 : 4297 - 4307 .

Luo J , Yang Y , Wang Z , et al . Localization algorithm for underwater sensor network: a review [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 17 ): 13126 - 13144 .

张新文 , 刘同木 , 周保成 , 等 . 基于物联网的浮标数据采集与控制系统设计 [J ] . 海洋科学进展 , 2024 , 42 ( 1 ): 196 - 206 .

ZHANG X W , LIU T M , ZHOU B C , et al . Design of buoy data acquisition and control system based on Internet of Things [J ] . Advances in Marine Science , 2024 , 42 ( 1 ): 196 - 206 .

Yang Y , Xiao Y , Li T . A survey of autonomous underwater vehicle formation: performance, formation control, and communication capability [J ] . IEEE Communications Surveys & Tutorials , 2021 , 23 ( 2 ): 815 - 841 .

Nkenyereye L , Nkenyereye L , Ndibanje B . Internet of underwater things: a survey on simulation tools and 5G-based underwater networks [J ] . Electronics , 2024 .

Jahanbakht M , Xiang W , Hanzo L , et al . Internet of underwater things and big marine data analytics—a comprehensive survey [J ] . IEEE Communications Surveys & Tutorials , 2021 , 23 ( 2 ): 904 - 956 .

Wen M , Kuba J G , Lin R , et al . Multi-agent reinforcement learning is a sequence modeling problem [J/OL ] . arXiv preprint , 2022 , arXiv: 2205 . 14953 . https://arxiv.org/abs/2205.14953 https://arxiv.org/abs/2205.14953 .

罗彪 , 胡天萌 , 周育豪 , 等 . 多智能体强化学习控制与决策研究综述 [J ] . 自动化学报 , 2025 , 51 ( 3 ): 510 - 539 .

Luo B , Hu T M , Zhou Y H , et al . Survey on multi-agent reinforcement learning for control and decision-making [J ] . Acta Automatica Sinica , 2025 , 51 ( 3 ): 510 - 539 .

Wang Y , Zhang Y , Jia L . Multi-UAV maneuvering targets tracking based on mobility-focused multi-agent deep deterministic policy gradient [C ] // 2024 3rd International Conference on Automation, Robotics and Computer Engineering (ICARCE) . China : IEEE , 2024 : 498 - 502 .

Li S , Wu Y , Cui X , et al . Robust multi-agent reinforcement learning via minimax deep deterministic policy gradient [C ] // Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence . Honolulu : AAAI Press , 2019 .

Zuo W , Liao S , Cui Y , et al . Multi-agent deep reinforcement learning-based distributed task assignment in multi-UAV cooperative edge computing [J ] . IEEE Transactions on Cloud Computing , 2026 .

Wibisono A , Song H K , Lee B M . A survey of multi-agent reinforcement learning for cooperative control in multi-AUV systems [J ] . IEEE Access , 2025 , 13 : 161505 - 161528 .

Lu K , Li R D , Li M C , et al . MADDPG-based joint optimization of task partitioning and computation resource allocation in mobile edge computing [J ] . Neural Computing and Applications , 2023 , 35 ( 22 ): 16559 - 16576 .

Wu R , Zhong J , Wallace B A , et al . Human-robotic prosthesis as collaborating agents for symmetrical walking [C ] // Proceedings of the 36th International Conference on Neural Information Processing Systems . New Orleans : Curran Associates Inc. , 2022 .

于长东 , 刘新阳 , 陈聪 , 等 . 基于多智能体深度强化学习的无人艇集群博弈对抗研究 [J ] . 水下无人系统学报 , 2024 , 32 ( 1 ): 79 - 86 .

YU C D , LIU X Y , CHEN C , et al . Research on game confrontation of unmanned surface vehicles swarm based on multi-agent deep reinforcement learning [J ] . Journal of Unmanned Undersea Systems , 2024 , 32 ( 1 ): 79 - 86 .

Iqbal S , Sha F . Actor-attention-critic for multi-agent reinforcement learning [C ] // Proceedings of the 36th International Conference on Machine Learning (ICML) . 2019 , 97 : 2961 - 2970 .

Su C , Yan Y , Wang T , et al . A graph attention mechanism based multi-agent reinforcement learning method for efficient traffic light control [C ] // 2021 International Wireless Communications and Mobile Computing (IWCMC) . Harbin : IEEE , 2021 : 1332 - 1337 .

Ni K , Chen J , Wang J , et al . Multi-agent reinforcement learning with multi-head attention [C ] // 2023 IEEE 6th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) . Chongqing : IEEE , 2023 : 1311 - 1315 .

Grimbly S J , Shock J , Pretorius A . Causal multi-agent reinforcement learning: review and open problems [J/OL ] . arXiv preprint , 2021 , arXiv: 2111 . 06721 . https://arxiv.org/abs/2111.06721 https://arxiv.org/abs/2111.06721 .

Zhu S , Han G , Lin C , et al . Underwater target tracking based on hierarchical software-defined multi-AUV reinforcement learning: a multi-AUV advantage-attention actor-critic approach [J ] . IEEE Transactions on Mobile Computing , 2024 , 23 ( 12 ): 13639 - 13653 .

Kalisch M , Bühlmann P . Estimating high-dimensional directed acyclic graphs with the PC-algorithm [J ] . Journal of Machine Learning Research , 2007 , 8 : 613 - 636 .

李聪 , 张震 , 刘鹏昌 . 基于KL散度和迁移强化学习的AMR路径规划算法 [J ] . 电光与控制 , 2026 , 33 ( 01 ): 78 - 83+90 .

Li C , Zhang Z , Liu P C . AMR path planning algorithm based on KL divergence and transfer reinforcement learning [J ] . Electronics Optics & Control , 2026 , 33 ( 1 ): 78 - 83, 90 .

Foerster J N , Farquhar G , Afouras T , et al . Counterfactual multi-agent policy gradients [C ] // Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence . New Orleans : AAAI Press , 2018 .

Ororbia M E , Warn G P . Design synthesis through a Markov decision process and reinforcement learning framework [J ] . Journal of Computing and Information Science in Engineering , 2022 , 22 ( 2 ): 021002 .

Lowe R , Wu Y , Tamar A , et al . Multi-agent actor-critic for mixed cooperative-competitive environments [J/OL ] . arXiv preprint , 2017 , arXiv: 1706 . 02275 . https://arxiv.org/abs/1706.02275 https://arxiv.org/abs/1706.02275 .

Benson H P . Multi-objective optimization: pareto optimal solutions, properties [M ] //Floudas C A, Pardalos P M. Encyclopedia of optimization . Boston : Springer , 2001 .

Jiang Y , Renner B C . Low-cost underwater swarm acoustic localization: a review [J ] . IEEE Access , 2024 , 12 : 25779 - 25796 .

Lyu X , Sun Y , Wang L , et al . End-to-end AUV local motion planning method based on deep reinforcement learning [J ] . Journal of Marine Science and Engineering , 2023 , 11 ( 9 ): 1796 .

Fujimoto S , Hoof H van , Meger D . Addressing function approximation error in actor-critic methods [C ] // Proceedings of the 35th International Conference on Machine Learning . Stockholm : PMLR , 2018 , 80 : 1587 - 1596 .

方城亮 , 杨飞生 , 潘泉 . 基于MASAC强化学习算法的多无人机协同路径规划 [J ] . 中国科学（信息科学） , 2024 ( 8 ).

Fang C L , Yang F S , Pan Q . Multi-UAV cooperative path planning based on MASAC reinforcement learning algorithm [J ] . Scientia Sinica (Informationis) , 2024 ( 8 ).

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Task allocation method for collaborative crowdsensing of heterogeneous entities in emergency response

Lightweight attention-based SAR ship detector

Fusion of multi-scale and context for small target detection algorithm of unmanned aerial vehicle rescue

An intrusion detection method based on depthwise separable convolution and attention mechanism

Multi-domain collaborative anti-jamming based on multi-agent deep reinforcement learning

Related Author

ZHAO Yong

JI Yatai

ZHANG Qi

XU Kai

ZHU Zhengqiu

LU Wenhao

ZHU Ying

LU Ping

Related Institution

College of Systems Engineering, National University of Defense Technology

State Key Laboratory of Mobile Network and Mobile Multimedia Technology

Zhongxing Telecom Equipment Corporation

China Academy of Information and Communications Technology

College of Electronics and Information Engineering, Shenzhen University

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰