基于双池DQN的HVAC无模型优化控制方法

马帅; 傅启明; 陈建平; 冯帆; 陆悠; 李铮伟; 裘舒年

doi:10.11959/j.issn.2096-6652.202208

您当前的位置：

首页 >

文章列表页 >

基于双池DQN的HVAC无模型优化控制方法

学术论文 | 更新时间：2024-06-05

- 基于双池DQN的HVAC无模型优化控制方法
- HVAC model-free optimal control method based on double-pools DQN
- 智能科学与技术学报 2022年4卷第3期页码：426-444
- 作者机构：
  
  1. 苏州科技大学电子与信息工程学院，江苏苏州 215009
  2. 江苏省智慧建筑节能重点实验室，江苏苏州 215009
  3. 苏州移动网络及应用技术重点实验室，江苏苏州 215009
  4. 德州农工大学，美国得克萨斯州，TX 77843
  5. 同济大学机械与能源学院，上海 200092
  6. 同济大学工程结构性能演化与控制教育部重点实验室，上海 200092
- 作者简介：
  
  [ "马帅（1997- ），男，苏州科技大学电子与信息工程学院硕士生，主要研究方向为强化学习在智慧建筑中的应用" ]
  [ "傅启明（1985- ），男，博士，苏州科技大学电子与信息工程学院副教授，主要研究方向为强化学习、深度学习和建筑节能" ]
  [ "陈建平（1963- ），男，博士，苏州科技大学电子与信息工程学院教授，研究方向为大数据分析、建筑节能和智能信息" ]
  [ "冯帆（1993- ），男，德州农工大学博士生，主要研究方向为HVAC优化控制" ]
  [ "陆悠（1977- ），男，博士，苏州科技大学电子与信息工程学院副教授，主要研究方向为下一代网络、网路管理和机器学习" ]
  [ "李铮伟（1981- ），男，博士，同济大学机械与能源学院教授，主要研究方向为建筑能源系统的优化运行及故障诊断等" ]
  [ "裘舒年（1995- ），男，同济大学机械与能源学院博士生，主要研究方向为强化学习在HVAC中的应用" ]
- 基金信息：
  
  国家重点研发计划基金资助项目;The National Key Research and Development Program of China(2020YFC2006602);国家自然科学基金资助项目;The National Natural Science Foundation of China(62072324);国家自然科学基金资助项目;The National Natural Science Foundation of China(61876217);国家自然科学基金资助项目;The National Natural Science Foundation of China(61876121);国家自然科学基金资助项目;The National Natural Science Foundation of China(61772357);江苏省重点研发计划基金资助项目;The Research and Development Program of Jiangsu Province(BE2020026)
- DOI：10.11959/j.issn.2096-6652.202208
  中图分类号： TP273
- 网络出版日期：2022-09，
  
  纸质出版日期：2022-09-15
- 稿件说明：
移动端阅览
马帅, 傅启明, 陈建平, 等. 基于双池DQN的HVAC无模型优化控制方法[J]. 智能科学与技术学报, 2022,4(3):426-444.

Shuai MA, Qiming FU, Jianping CHEN, et al. HVAC model-free optimal control method based on double-pools DQN[J]. Chinese journal of intelligent science and technology, 2022, 4(3): 426-444.
马帅, 傅启明, 陈建平, 等. 基于双池DQN的HVAC无模型优化控制方法[J]. 智能科学与技术学报, 2022,4(3):426-444. DOI： 10.11959/j.issn.2096-6652.202208.

Shuai MA, Qiming FU, Jianping CHEN, et al. HVAC model-free optimal control method based on double-pools DQN[J]. Chinese journal of intelligent science and technology, 2022, 4(3): 426-444. DOI： 10.11959/j.issn.2096-6652.202208.

摘要

在 HVAC 控制领域，基于模型的最优控制方法得到了学者的广泛研究与验证，但是该方法高度依赖模型的准确性、大量历史数据的收集以及传感器的部署。针对上述问题，结合 EnergyPlus、实际系统参数以及历史数据，构建 HVAC 优化控制模型，并提出一种改进的双池 DQN 算法，最后将其应用于 HVAC 系统中不同型号冷机的负荷分配、冷却塔风机频率以及冷却水泵频率的组合优化控制。基于所构建的问题模型，针对决策优化过程中存在的样本不平衡的问题，该算法在 DQN 的基础上，建立两个独立的经验池，分别存储负荷分配与非负荷分配样本，在训练过程中，遵循一定的比例从经验池中进行采样，以加快算法收敛。将所提出的方法与基于模型的控制方法及基线方法进行比较，实验结果表明，与基线方法相比，基于模型的HVAC控制器能够节能11.5% （最优节能效率），而基于双池DQN的HVAC控制器在第1年就能够节能7.5%，同时，随着系统运行，该控制器在第8年左右即可获得接近最优节能效率的结果。此外，与基于模型的HVAC控制器相比，该控制器不依赖于系统模型，且在在线控制过程中，所需的先验知识及传感器较少，在实际的工程应用中更具价值。

Abstract

In the field of HVAC (heating

ventilation and air conditioning) control

the model-based optimal control method has been extensively studied and verified by scholars

but this method highly depends on the accuracy of the model

the collection of a large amount of historical data

and the deployment of sensors.In response to the above problems

combined with EnergyPlus

actual system parameters and historical data

the HVAC optimized control model was constructed

and an improved double pools-based DQN (DPs-DQN) algorithm was proposed.Finally

it was applied to the load distribution of different types of chillers

the combined optimal control of cooling tower fan frequency and cooling water pump frequency in HVAC system.Based on the constructed problem model

aiming at the problem of sample imbalance in the decision-making optimization process

the algorithm established two independent experience pools on the basis of DQN to store load distribution and non load distribution samples respectively.During the training process

followed a certain ratio to sample from the experience pool to speed up the algorithm convergence.The proposed method was compared with the model-based control method and the baseline method.The experimental results show that compared with the baseline method

the model-based HVAC controller can save 11.5% (optimal energy-saving efficiency)

while the DPs-DQN can save energy by 7.5% in the first year.At the same time

as the system runs

the controller can obtain results close to the optimal energy saving efficiency in the eighth year.In addition

compared with the model-based HVAC controller

the controller does not depend on the system model

and requires less prior knowledge and sensors in the online control process

which is more valuable in actual engineering applications.

关键词

Keywords

references

PÉREZ-LOMBARD L , ORTIZ J , POUT C . A review on buildings energy consumption information [J ] . Energy and Buildings , 2008 , 40 ( 3 ): 394 - 398 .

WANG J Q , HUANG J J , FENG Z B , et al . Occupant-density- detection based energy efficient ventilation system:prevention of infection transmission [J ] . Energy and Buildings , 2021 ,240:110883.

HOU J , XU P , LU X , et al . Implementation of expansion planning in existing district energy system:a case study in China [J ] . Applied Energy , 2018 , 211 : 269 - 281 .

PANG Z H , O’NEILL Z , LI Y F , et al . The role of sensitivity analysis in the building performance analysis:a critical review [J ] . Energy and Buildings , 2020 ,209:109659.

WANG J Q , HOU J , CHEN J P , et al . Data mining approach for improving the optimal control of HVAC systems:an event-driven strategy [J ] . Journal of Building Engineering , 2021 ,39:102246.

MA Z J , WANG S W . Online fault detection and robust control of condenser cooling water systems in building central chiller plants [J ] . Energy and Buildings , 2011 , 43 ( 1 ): 153 - 165 .

YAO Y , LIAN Z W , HOU Z J , et al . Optimal operation of a large cooling system based on an empirical model [J ] . Applied Thermal Engineering , 2004 , 24 ( 16 ): 2303 - 2321 .

HUANG S , ZUO W D , SOHN M D . Improved cooling tower control of legacy chiller plants by optimizing the condenser water set point [J ] . Building and Environment , 2017 , 111 : 33 - 46 .

ZHU N , SHAN K , WANG S W , et al . An optimal control strategy with enhanced robustness for air-conditioning systems considering model and measurement uncertainties [J ] . Energy and Buildings , 2013 , 67 : 540 - 550 .

SUN Y J , WANG S W , HUANG G S . Chiller sequencing control with enhanced robustness for energy efficient operation [J ] . Energy and Buildings , 2009 , 41 ( 11 ): 1246 - 1255 .

LI Z W , HUANG G S , SUN Y J . Stochastic chiller sequencing control [J ] . Energy and Buildings , 2014 , 84 : 203 - 213 .

QIU S N , LI Z H , LI Z W , et al . Model-free control method based on reinforcement learning for building cooling water systems:validation by measured data-based simulation [J ] . Energy and Buildings , 2020 ,218:110055.

BAGHAEE S , ULUSOY I . User comfort and energy efficiency in HVAC systems by Q-learning [C ] // Proceedings of 2018 26th Signal Processing and Communications Applications Conference . Piscataway:IEEE Press , 2018 : 1 - 4 .

FADDEL S , TIAN G Y , ZHOU Q , et al . Data driven Q-learning for commercial HVAC control [C ] // Proceedings of IEEE Southeast Con 2020 . Piscataway:IEEE Press , 2020 : 1 - 6 .

刘朝阳 , 穆朝絮 , 孙长银 . 深度强化学习算法与应用研究现状综述 [J ] . 智能科学与技术学报 , 2020 , 2 ( 4 ): 314 - 326 .

LIU Z Y , MU C X , SUN C Y . An overview on algorithms and applications of deep reinforcement learning [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 4 ): 314 - 326 .

刘莹莹 , 王占山 . 异构多智能体系统的输出同步:一个基于数据的强化学习方法 [J ] . 智能科学与技术学报 , 2020 , 2 ( 4 ): 394 - 400 .

LIU Y Y , WANG Z S . Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 4 ): 394 - 400 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

AHN K U , PARK C S . Application of deep Q-networks for model-free optimal control balancing between different HVAC systems [J ] . Science and Technology for the Built Environment , 2020 , 26 ( 1 ): 61 - 74 .

CHANG Y C . A novel energy conse11rvation method—optimal chiller loading [J ] . Electric Power Systems Research , 2004 , 69 ( 2/3 ): 221 - 226 .

LIAO Y D , SUN Y J , HUANG G S . Robustness analysis of chiller sequencing control [J ] . Energy Conversion and Management , 2015 , 103 : 180 - 190 .

ARDAKANI A J , ARDAKANI F F , HOSSEINIAN S H . A novel approach for optimal chiller loading using particle swarm optimization [J ] . Energy and Buildings , 2008 , 40 ( 12 ): 2177 - 2187 .

CHANG Y C , LIN J K , CHUANG M H . Optimal chiller loading by genetic algorithm for reducing energy consumption [J ] . Energy and Buildings , 2005 , 37 ( 2 ): 147 - 155 .

LEE W S , LIN L C . Optimal chiller loading by particle swarm algorithm for reducing energy consumption [J ] . Applied Thermal Engineering , 2009 , 29 ( 8/9 ): 1730 - 1734 .

QIU S N , LI Z H , LI Z W , et al . Model-free optimal chiller loading method based on Q-learning [J ] . Science and Technology for the Built Environment , 2020 , 26 ( 8 ): 1100 - 1116 .

CRAWLEY D B , LAWRIE L K , WINKELMANN F C , et al . EnergyPlus:creating a new-generation building energy simulation program [J ] . Energy and Buildings , 2001 , 33 ( 4 ): 319 - 331 .

ZHU D D , HONG T Z , YAN D , et al . A detailed loads comparison of three building energy modeling programs:EnergyPlus,DeST and DOE-2.1E [J ] . Building Simulation , 2013 , 6 ( 3 ): 323 - 335 .

BENTON D J , BOWMAN C F , HYDEMAN M , et al . An improved cooling tower algorithm for the CooiTools TM simulation model [C ] // Proceedings of ASHRAE Annual Meeting .[S.l.:s.n. ] , 1999 .

浏览量

611

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

深度强化学习应用于金融市场量化交易研究综述

面向数字货币量化交易的OAC模型研究

基于TD3的电动汽车复合电源能量管理策略研究

智能博弈综述：游戏AI对作战推演的启示

基于群体熵的机器人群体智能汇聚度量