TD3-based energy management strategy for hybrid energy storage system of electric vehicle

Jiacheng LIU; Xiangwen ZHANG

doi:10.11959/j.issn.2096-6652.202230

您当前的位置：

首页 >

文章列表页 >

TD3-based energy management strategy for hybrid energy storage system of electric vehicle

Papers and Reports | 更新时间：2024-06-05

- TD3-based energy management strategy for hybrid energy storage system of electric vehicle
- Chinese Journal of Intelligent Science and Technology Vol. 4, Issue 2, Pages: 277-287(2022)
- 作者机构：
  
  1. 桂林电子科技大学电子工程与自动化学院，广西桂林 541004
  2. 广西自动检测技术与仪器重点实验室，广西桂林 541004
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202230
  CLC： U469.72
- Published Online：2022-06，
  
  Published：15 June 2022
- 稿件说明：
移动端阅览
Jiacheng LIU, Xiangwen ZHANG. TD3-based energy management strategy for hybrid energy storage system of electric vehicle[J]. Chinese journal of intelligent science and technology, 2022, 4(2): 277-287.
DOI：

Jiacheng LIU, Xiangwen ZHANG. TD3-based energy management strategy for hybrid energy storage system of electric vehicle[J]. Chinese journal of intelligent science and technology, 2022, 4(2): 277-287. DOI： 10.11959/j.issn.2096-6652.202230.

摘要

将蓄电池与超级电容组成复合电源系统并结合有效的能量管理策略，能显著提高能量利用率，延长储能系统的使用寿命。为了实现复合电源系统能耗损失的最小化，设计了一种基于双延迟深度确定性策略梯度（TD3）算法的能量管理策略。与深度确定性策略梯度（DDPG）算法相比，该算法解决了Q值过高估计问题，能耗损失更小。利用电动汽车行驶方程式和复合电源系统等效电路模型，搭建了基于TD3算法的MATLAB/Simulink仿真模型，并进行测试。仿真结果显示，所提出的能量管理策略能降低大电流对蓄电池的冲击，与DDPG算法相比，能量利用率提高了1.36%，蓄电池峰值电流输出降低了14.68%，蓄电池温升降低了3.52%，系统总能耗降低了2.17%。

Abstract

Combining batteries and super capacitors into a composite power system (CPS) with an effective energy management strategy can significantly improve the energy utilization

and increase the service life of the energy storage system.To minimize the energy loss of the system

an energy management strategy based on the twin delayed deep deterministic policy gradient (TD3) algorithm was designed.Compared with the deep deterministic policy gradient (DDPG) algorithm

TD3 algorithm solved the problem of overestimation of Q value and less energy loss.A MATLAB/Simulink simulation model based on the TD3 algorithm was built

and tested with the electric vehicle driving equation and the equivalent circuit model of CPS.The outcomes indicate that the proposed energy management strategy can effectively reduce the impact of high current on the battery

and compared with DDPG algorithm

the energy utilization efficiency is improved by 1.36%

the peak of output current of the battery is reduced by 14.68%

the temperature rise of the battery is reduced by 3.52%

the total energy consumption of the system is reduced by 2.17%.

关键词

Keywords

references

崔淑梅 , 宋贝贝 , 王志远 . 电动汽车动态无线供电磁耦合机构研究综述 [J ] . 电工技术学报 , 2022 , 37 ( 3 ): 537 - 554 .

CUI S M , SONG B B , WANG Z Y . Overview of magnetic coupler for electric vehicles dynamic wireless charging [J ] . Transactions of China Electrotechnical Society , 2022 , 37 ( 3 ): 537 - 554 .

张文爽 , 李键 , 余文涛 , 等 . 基于新一代电源控制器的锂电池在轨管理分析 [J ] . 电源技术 , 2022 , 46 ( 1 ): 109 - 112 .

ZHANG W S , LI J , YU W T , et al . Analysis and verification of in-orbit management of lithiumion battery based on the new generation of power control unit [J ] . Chinese Journal of Power Sources , 2022 , 46 ( 1 ): 109 - 112 .

靳聪 , 高申 , 孙世光 , 等 . 基于无损检测的生命周期锂离子动力电池安全性分析 [J ] . 储能科学与技术 , 2019 , 8 ( 2 ): 259 - 263 .

JIN C , GAO S , SUN S G , et al . Analysis of safety performance of lithium-ion power battery during life cycle based on non-destructive testing [J ] . Energy Storage Science and Technology , 2019 , 8 ( 2 ): 259 - 263 .

魏东涛 , 黄之杰 , 孔华 , 等 . 蓄电池 SOC 的研究及预测方法 [J ] . 电源技术 , 2016 , 40 ( 6 ): 1321 - 1323 .

WEI D T , HUANG Z J , KONG H , et al . Battery SOC research and prediction methods [J ] . Chinese Journal of Power Sources , 2016 , 40 ( 6 ): 1321 - 1323 .

ZHAI C J , LUO F , LIU Y G . A novel predictive energy management strategy for electric vehicles based on velocity prediction [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 11 ): 12559 - 12569 .

ZHANG L J , YE X M , XIA X H , et al . A real-time energy management and speed controller for an electric vehicle powered by a hybrid energy storage system [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 10 ): 6272 - 6280 .

周美兰 , 冯继峰 , 张宇 . 纯电动汽车复合储能系统及其能量控制策略 [J ] . 电机与控制学报 , 2019 , 23 ( 5 ): 51 - 59 .

ZHOU M L , FENG J F , ZHANG Y . Composite energy storage system and its energy control strategy for electric vehicles [J ] . Electric Machines and Control , 2019 , 23 ( 5 ): 51 - 59 .

胡杰 , 刘迪 , 杜常清 , 等 . 电动汽车复合能源系统能量管理策略研究 [J ] . 机械科学与技术 , 2020 , 39 ( 10 ): 1606 - 1614 .

HU J , LIU D , DU C Q , et al . Study on energy management strategy of hybrid energy storage system for electric vehicles [J ] . Mechanical Science and Technology for Aerospace Engineering , 2020 , 39 ( 10 ): 1606 - 1614 .

孙永健 . 某型纯电动客车复合电源系统优化设计与控制 [D ] . 长春:吉林大学 , 2017 .

SUN Y J . The optimization design and control of hybrid energy storage system for A certain type electric bus [D ] . Changchun:Jilin University , 2017 .

CHEN Z Y , XIONG R , CAO J Y . Particle swarm optimization-based optimal power management of plug-in hybrid electric vehicles considering uncertain driving conditions [J ] . Energy , 2016 , 96 : 197 - 208 .

ZHANG S , XIONG R , SUN F C . Model predictive control for power management in a plug-in hybrid electric vehicle with a hybrid energy storage system [J ] . Applied Energy , 2017 , 185 : 1654 - 1662 .

VÁZQUEZ-CANTELI J R , NAGY Z . Reinforcement learning for demand response:a review of algorithms and modeling techniques [J ] . Applied Energy , 2019 , 235 : 1072 - 1089 .

XIONG R , CAO J Y , YU Q Q . Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle [J ] . Applied Energy , 2018 , 211 : 538 - 548 .

WU J D , HE H W , PENG J K , et al . Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus [J ] . Applied Energy , 2018 , 222 : 799 - 811 .

KONG H F , YAN J P , WANG H , et al . Energy management strategy for electric vehicles based on deep Q-learning using Bayesian optimization [J ] . Neural Computing and Applications , 2020 , 32 ( 18 ): 14431 - 14445 .

LI W H , CUI H , NEMETH T , et al . Deep reinforcement learning-based energy management of hybrid battery systems in electric vehicles [J ] . Journal of Energy Storage , 2021 , 36 : 102355 .

TAN H C , ZHANG H L , PENG J K , et al . Energy management of hybrid electric bus based on deep reinforcement learning in continuous state and action space [J ] . Energy Conversion and Management , 2019 , 195 : 548 - 560 .

WU Y K , TAN H C , PENG J K , et al . Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus [J ] . Applied Energy , 2019 , 247 : 454 - 466 .

SCHULMAN J , WOLSKI F , DHARIWAL P , et al . Proximal policy optimization algorithms [J ] . arXiv preprint,2017,arXiv:1707.06347 .

SONG Z Y , HOU J , HOFMANN H , et al . Sliding-mode and Lyapunov function-based control for battery/supercapacitor hybrid energy storage system used in electric vehicles [J ] . Energy , 2017 , 122 : 601 - 612 .

QI X W , WU G Y , BORIBOONSOMSIN K , et al . Data-driven reinforcement learning–based real-time energy management system for plug-in hybrid electric vehicles [J ] . Transportation Research Record:Journal of the Transportation Research Board , 2016 , 2572 ( 1 ): 1 - 8 .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [J ] . arXiv preprint,2015,arXiv:1509.02971 .

FUJIMOTO S , VAN HOOF H , MEGER D . Addressing function approximation error in actor-critic methods [J ] . arXiv preprint,2018,arXiv:1802.09477 .

丁镇涛 , 邓涛 , 李志飞 , 等 . 基于安时积分和无迹卡尔曼滤波的锂离子电池 SOC 估算方法研究 [J ] . 中国机械工程 , 2020 , 31 ( 15 ): 1823 - 1830 .

DING Z T , DENG T , LI Z F , et al . SOC estimation of lithium-ion battery based on ampere hour integral and unscented Kalman filter [J ] . China Mechanical Engineering , 2020 , 31 ( 15 ): 1823 - 1830 .

Views

622

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Review: the application of deep reinforcement learning to quantitative trading in financial market

Research on OAC model for quantitative trading of digital currency

HVAC model-free optimal control method based on double-pools DQN

Overview of intelligent game:enlightenment of game AI to combat deduction

Emergence measurement of robot swarm intelligence based on swarm entropy

Related Author

LI Xiangxia

WEN Jiancheng

HE Yijun

XU Bo

LI Xiangxia

HE Yijun

XU Bo

Fan FENG

Related Institution

Guangdong Intelligent Business Engineering Technology Research Center

School of Information, Guangdong University of Finance & Economics

School of Information Science, Guangdong University of Finance & Economics

Key Laboratory of Performance Evolution and Control for Engineering Structures of Ministry of Education, Tongji University

School of Mechanical Engineering, Tongji University

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰