基于强化学习的参数化电路优化算法

唐长成; 叶佐昌

Chinese

您当前的位置：

首页 >

文章列表页 >

基于强化学习的参数化电路优化算法

更新时间：2026-09-04

- 基于强化学习的参数化电路优化算法
- Parametric Circuit Optimization with Reinforcement Learning
- 微电子学与计算机 2019年36卷第1期页码：46-50
- 作者机构：
  
  清华大学微电子所,北京,100083
- 作者简介：
- 基金信息：
- DOI：
  中图分类号：
- 纸质出版：2019
- 稿件说明：
移动端阅览
唐长成, 叶佐昌. 基于强化学习的参数化电路优化算法[J]. 微电子学与计算机, 2019, 36(1): 46-50.

TANG Chang-cheng, YE Zuo-chang. Parametric Circuit Optimization with Reinforcement Learning[J]. 2019, 36(1): 46-50.
唐长成, 叶佐昌. 基于强化学习的参数化电路优化算法[J]. 微电子学与计算机, 2019, 36(1): 46-50. DOI：

TANG Chang-cheng, YE Zuo-chang. Parametric Circuit Optimization with Reinforcement Learning[J]. 2019, 36(1): 46-50. DOI：

摘要

本文主要致力于解决参数化形式的优化问题

即(

)

其中是需要优化的变量

w则是对应不同优化问题的参数

在现实中经常会遇到需要解决一系列不同参数下的优化问题.在对某种特定结构的问题下

通过对不同的参数训练一个模型来解决所有参数下的优化问题.和传统的方法不一样

并不是通过对不同的参数多次独立抽样来训练我们的模型

而是利用强化学习的方法加速训练过程.强化学习算法中分别用策略网络来得到优化结果和利用价值网络来评价策略好坏

通过迭代地训练两个网络来优化策略.在后面一些数学例子和电路优化的例子中显示强化学习的方法取得了比较好的效果.

Abstract

In this paper we are focusing on solving parametric optimization problems

i.e. min(

)

where is the variable to be optimized and w is a vector that parameterize the optimization problem. Such kind of problems are very commonly seen in reality. We propose an efficient method to train a model that connects the solution to the parameters and thus solve all the problems with the same structure and different parameters at the same time. During training process

instead of solving a series of optimization problems with randomly sampled w independently

we adopt reinforcement learning to accelerate the training process. Two networks are trained alternately. The first network is a value network

and it is trained to fit the target loss function. The second network is a policy network

whose output is connected to the input of the value network and it is trained to minimize the output of the value network. Experiments demonstrate the effectiveness of the proposed method.

关键词

Keywords

references

Bakker B. Reinforcement learning with long short-term memory[C]// International Conference on Neural Information Processing Systems: Natural and Synthetic. Cambridge, Massachusetts, MIT Press, 2001: 1475-1482.

Silver D, Lever G, Heess N, et al. Deterministic policy gradient algorithms[C]// International Conference on International Conference on Machine Learning. Beijing, China, 2014: 387-395.

Balduzzi D, Ghifary M. Compatible value gradients for reinforcement learning of continuous deep policies[J]. Computer Science, 2015, 8(6):A187. DOI: A187

Heess N, Wayne G, Silver D, et al. Learning continuous control policies by stochastic value gradients[J]. International Conference on Neural Information Processing Systems, 2015:2944-2952. DOI: 2944

Peters J, Schaal S. Reinforcement learning of motor skills with policy gradients[J]. Neural Networks, 2008, 21(4):682-697. DOI: 682

Das S, Suganthan P N. Differential evolution: a survey of the state-of-the-art[J]. IEEE Transactions on Evolutionary Computation, 2011, 15(1):4-31. DOI: 4

Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous control with deep reinforcement learning[J]. Computer Science, 2015, 8(6):A187. DOI: A187

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

AlN/GaN混合缓冲层双异质结p-GaN HEMT直流特性仿真

基于改进RT-DETR的航拍小目标检测算法

大模型发展综述：从BERT到Deepseek的技术演进

超低功耗实时时钟电路的设计

V型沟槽SiC中子探测器的蒙特卡洛分析