Parametric Circuit Optimization with Reinforcement Learning

TANG Chang-cheng; YE Zuo-chang

Chinese

您当前的位置：

首页 >

文章列表页 >

Parametric Circuit Optimization with Reinforcement Learning

更新时间：2026-09-04

- Parametric Circuit Optimization with Reinforcement Learning
- Vol. 36, Issue 1, Pages: 46-50(2019)
- 作者机构：
  
  清华大学微电子所,北京,100083
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2019
- 稿件说明：
移动端阅览
唐长成, 叶佐昌. 基于强化学习的参数化电路优化算法[J]. 微电子学与计算机, 2019, 36(1): 46-50.

TANG Chang-cheng, YE Zuo-chang. Parametric Circuit Optimization with Reinforcement Learning[J]. 2019, 36(1): 46-50.
唐长成, 叶佐昌. 基于强化学习的参数化电路优化算法[J]. 微电子学与计算机, 2019, 36(1): 46-50. DOI：

TANG Chang-cheng, YE Zuo-chang. Parametric Circuit Optimization with Reinforcement Learning[J]. 2019, 36(1): 46-50. DOI：

摘要

本文主要致力于解决参数化形式的优化问题

即(

)

其中是需要优化的变量

w则是对应不同优化问题的参数

在现实中经常会遇到需要解决一系列不同参数下的优化问题.在对某种特定结构的问题下

通过对不同的参数训练一个模型来解决所有参数下的优化问题.和传统的方法不一样

并不是通过对不同的参数多次独立抽样来训练我们的模型

而是利用强化学习的方法加速训练过程.强化学习算法中分别用策略网络来得到优化结果和利用价值网络来评价策略好坏

通过迭代地训练两个网络来优化策略.在后面一些数学例子和电路优化的例子中显示强化学习的方法取得了比较好的效果.

Abstract

In this paper we are focusing on solving parametric optimization problems

i.e. min(

)

where is the variable to be optimized and w is a vector that parameterize the optimization problem. Such kind of problems are very commonly seen in reality. We propose an efficient method to train a model that connects the solution to the parameters and thus solve all the problems with the same structure and different parameters at the same time. During training process

instead of solving a series of optimization problems with randomly sampled w independently

we adopt reinforcement learning to accelerate the training process. Two networks are trained alternately. The first network is a value network

and it is trained to fit the target loss function. The second network is a policy network

whose output is connected to the input of the value network and it is trained to minimize the output of the value network. Experiments demonstrate the effectiveness of the proposed method.

关键词

Keywords

references

Bakker B. Reinforcement learning with long short-term memory[C]// International Conference on Neural Information Processing Systems: Natural and Synthetic. Cambridge, Massachusetts, MIT Press, 2001: 1475-1482.

Silver D, Lever G, Heess N, et al. Deterministic policy gradient algorithms[C]// International Conference on International Conference on Machine Learning. Beijing, China, 2014: 387-395.

Balduzzi D, Ghifary M. Compatible value gradients for reinforcement learning of continuous deep policies[J]. Computer Science, 2015, 8(6):A187. DOI: A187

Heess N, Wayne G, Silver D, et al. Learning continuous control policies by stochastic value gradients[J]. International Conference on Neural Information Processing Systems, 2015:2944-2952. DOI: 2944

Peters J, Schaal S. Reinforcement learning of motor skills with policy gradients[J]. Neural Networks, 2008, 21(4):682-697. DOI: 682

Das S, Suganthan P N. Differential evolution: a survey of the state-of-the-art[J]. IEEE Transactions on Evolutionary Computation, 2011, 15(1):4-31. DOI: 4

Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous control with deep reinforcement learning[J]. Computer Science, 2015, 8(6):A187. DOI: A187

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Implementation and verification of a dual-chain power gating strategy for low power design

A feature-oriented context understanding and slice analysis network for tuberculosis detection

Monte carlo analysis of V-layout trench SiC neutron detector

Design of ultra-low power real-time clock circuits

Research on the design of blockchain-based AI-generated content traceability framework

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰