Evaluating Machine Learning Models for HVAC Demand Response: The Impact of Prediction Accuracy on Model Predictive Control Performance
Abstract
:1. Introduction
2. Methodology
2.1. Principle of MPC for Demand Response
2.2. Prediction Model for MPC
2.2.1. Model Principle and Hyperparameters
2.2.2. Feature Selection
2.3. Cost Function
2.4. Periods/Horizons Selection
3. Test Platform
4. Performance Comparison among Different Models
4.1. Accuracy
4.2. Training and Prediction Time
5. Impact of Model Accuracy on MPC Control Performance
5.1. Determination of the Model Type and the Metric Type
5.2. Artificial Modification of the Model Prediction Accuracy
5.3. Results Comparison and Analysis
5.3.1. Case 1 with the Highest Prediction Accuracy
5.3.2. Case 4 with the Lowest Prediction Accuracy
5.3.3. Comparison of Four Cases
6. Conclusions
- This study demonstrates that the proposed model predictive control strategy efficiently manages indoor air temperature setpoint for demand response. While the traditional control in the baseline case results in higher costs, the proposed control strategy reduces economic costs by up to 21.61%, with only a minimal increase of 0.10 K in the weighted indoor temperature. In addition, the humidity is also well-managed. The PMV ranges from 0.08 to 0.71, with the highest PMV occurring at 17:00, and the PDD at that time is 15.6%;
- From the perspective of prediction accuracy, the XGBoost model achieves the highest prediction accuracy compared with SVM, ANN, and LightGBM. For the XGBoost models developed in this study, the obtained R-squared values are 0.978 and 0.983 for predicting power use and indoor temperature in the upcoming 10 min;
- From the perspective of hyperparameter tuning, using a relatively low learning rate and a large number of trees is effective in enhancing the performance of the XGBoost model. For the ANN model, two hidden layers are sufficient for predicting HVAC power use and indoor temperature. On the other hand, the number of neurons in hidden layers should be large enough to obtain high performance;
- From the perspective of prediction and training time, LightGBM has the shortest prediction time among ANN, XGBoost, and LightGBM, although its prediction accuracy is slightly lower than that of XGBoost. In addition, the training time of LightGBM is one-quarter the time of XGBoost in this study;
- It is valuable to explore advanced prediction models to increase prediction accuracy, even within the high prediction accuracy range. Furthermore, it is also worth implementing MPC control for demand response even if the model prediction accuracy is relatively low (e.g., R2 = 0.76 in this study).
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Arrambide, I.; Zubia, I.; Madariaga, A. Critical review of offshore wind turbine energy production and site potential assessment. Electr. Power Syst. Res. 2019, 167, 39–47. [Google Scholar] [CrossRef]
- Bilendo, F.; Meyer, A.; Badihi, H.; Lu, N.; Cambron, P.; Jiang, B. Applications and Modeling Techniques of Wind Turbine Power Curve for Wind Farms—A Review. Energies 2022, 16, 180. [Google Scholar] [CrossRef]
- National Energy Administration. [EB/OL]. Available online: http://www.nea.gov.cn/2021-01/20/c_139682386.htm (accessed on 12 January 2022).
- Wang, H.; Ding, Z.; Tang, R.; Chen, Y.; Fan, C.; Wang, J. A machine learning-based control strategy for improved performance of HVAC systems in providing large capacity of frequency regulation service. Appl. Energy 2022, 326, 119962. [Google Scholar] [CrossRef]
- Kolokotsa, D.; Rovas, D.; Kosmatopoulos, E.; Kalaitzakis, K. A roadmap towards intelligent net zero- and positive-energy buildings. Sol. Energy 2011, 85, 3067–3084. [Google Scholar] [CrossRef]
- Fan, C.; Xiao, F.; Zhao, Y. A short-term building cooling load prediction method using deep learning algorithms. Appl. Energy 2017, 195, 222–233. [Google Scholar] [CrossRef]
- Yin, R.; Kara, E.C.; Li, Y.; DeForest, N.; Wang, K.; Yong, T.; Stadler, M. Quantifying flexibility of commercial and residential loads for demand response using setpoint changes. Appl. Energy 2016, 177, 149–164. [Google Scholar] [CrossRef]
- Drgoňa, J.; Arroyo, J.; Cupeiro Figueroa, I.; Blum, D.; Arendt, K.; Kim, D.; Ollé, E.P.; Oravec, J.; Wetter, M.; Vrabie, D.L.; et al. All you need to know about model predictive control for buildings. Annu. Rev. Control 2020, 50, 190–232. [Google Scholar] [CrossRef]
- Afram, A.; Janabi-Sharifi, F. Theory and applications of HVAC control systems—A review of model predictive control (MPC). Build. Environ. 2014, 72, 343–355. [Google Scholar] [CrossRef]
- Serale, G.; Fiorentini, M.; Capozzoli, A.; Bernardini, D.; Bemporad, A. Model Predictive Control (MPC) for Enhancing Building and HVAC System Energy Efficiency: Problem Formulation, Applications and Opportunities. Energies 2018, 11, 631. [Google Scholar] [CrossRef]
- Bianchini, G.; Casini, M.; Vicino, A.; Zarrilli, D. Demand-response in building heating systems: A Model Predictive Control approach. Appl. Energy 2016, 168, 159–170. [Google Scholar] [CrossRef]
- Bacher, P.; Madsen, H. Identifying suitable models for the heat dynamics of buildings. Energy Build. 2011, 43, 1511–1522. [Google Scholar] [CrossRef]
- Adhikari, R.; Pipattanasomporn, M.; Rahman, S. An algorithm for optimal management of aggregated HVAC power demand using smart thermostats. Appl. Energy 2018, 217, 166–177. [Google Scholar] [CrossRef]
- Hu, M.; Xiao, F. Price-responsive model-based optimal demand response control of inverter air conditioners using genetic algorithm. Appl. Energy 2018, 219, 151–164. [Google Scholar] [CrossRef]
- De Coninck, R.; Helsen, L. Practical implementation and evaluation of model predictive control for an office building in Brussels. Energy Build. 2016, 111, 290–298. [Google Scholar] [CrossRef]
- Li, P.; Vrabie, D.; Li, D.; Bengea, S.C.; Mijanovic, S.; O’Neill, Z.D. Simulation and experimental demonstration of model predictive control in a building HVAC system. Sci. Technol. Built Environ. 2015, 21, 721–732. [Google Scholar] [CrossRef]
- Huang, S.; Katipamula, S.; Lutes, R. Experimental investigation on thermal inertia characterization of commercial buildings for demand response. Energy Build. 2021, 252, 111384. [Google Scholar] [CrossRef]
- Ma, J.R.; Qin, J.; Salsbury, T.; Xu, P. Demand reduction in building energy systems based on economic model predictive control. Chem. Eng. Sci. 2012, 67, 92–100. [Google Scholar] [CrossRef]
- Ma, J.; Qin, S.J.; Salsbury, T. Application of economic MPC to the energy and demand minimization of a commercial building. J. Process Control 2014, 24, 1282–1291. [Google Scholar] [CrossRef]
- Hilliard, T.; Swan, L.; Qin, Z. Experimental implementation of whole building MPC with zone based thermal comfort adjustments. Build. Environ. 2017, 125, 326–338. [Google Scholar] [CrossRef]
- Hilliard, T.; Swan, L.; Kavgic, M.; Qin, Z.; Lingras, P. Development of a whole building model predictive control strategy for a LEED silver community college. Energy Build. 2016, 111, 224–232. [Google Scholar] [CrossRef]
- Wang, H.; Wang, S. A hierarchical optimal control strategy for continuous demand response of building HVAC systems to provide frequency regulation service to smart power grids. Energy 2021, 230, 120741. [Google Scholar] [CrossRef]
- Mohandes, S.R.; Zhang, X.; Mahdiyar, A. A comprehensive review on the application of artificial neural networks in building energy analysis. Neurocomputing 2019, 340, 55–75. [Google Scholar] [CrossRef]
- Chen, T.; Guestrin, C. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
- Ke, G.; Meng, Q.; Finley, T.; Wang, T.; Chen, W.; Ma, W.; Ye, Q.; Liu, T.Y. Lightgbm: A highly efficient gradient boosting decision tree. Adv. Neural Inf. Process. Syst. 2017, 30. [Google Scholar]
- Wang, H.; Chen, Y.; Kang, J.; Ding, Z.; Zhu, H. An XGBoost-Based predictive control strategy for HVAC systems in providing day-ahead demand response. Build. Environ. 2023, 238, 110350. [Google Scholar] [CrossRef]
- Shen, P.; Wang, M.; Liu, J.; Ji, Y. Hourly air temperature projection in future urban area by coupling climate change and urban heat island effect. Energy Build. 2023, 279, 112676. [Google Scholar] [CrossRef]
- Zhao, J.; Liu, X. A hybrid method of dynamic cooling and heating load forecasting for office buildings based on artificial intelligence and regression analysis. Energy Build. 2018, 174, 293–308. [Google Scholar] [CrossRef]
- Yousaf, S.; Bradshaw, C.R.; Kamalapurkar, R.; San, O. Investigating critical model input features for unitary air conditioning equipment. Energy Build. 2023, 284, 112823. [Google Scholar] [CrossRef]
- Fan, C.; Wang, J.; Gang, W.; Li, S. Assessment of deep recurrent neural network-based strategies for short-term building energy predictions. Appl. Energy 2019, 236, 700–710. [Google Scholar] [CrossRef]
- DBJ 15-51-2020; Design standard for energy efficiency of public buildings in Guangdong Province. Department of Housing and Urban-Rural Development of Guangdong Province: Guangzhou, China, 2020.
- Chen, Y.; Xu, P.; Gu, J.; Schmidt, F.; Li, W. Measures to improve energy demand flexibility in buildings for demand response (DR): A review. Energy Build. 2018, 177, 125–139. [Google Scholar] [CrossRef]
- Li, W.; Xu, P.; Wang, H.; Lu, X. A new method for calculating the thermal effects of irregular internal mass in buildings under demand response. Energy Build. 2016, 130, 761–772. [Google Scholar] [CrossRef]
- Johra, H.; Heiselberg, P. Influence of internal thermal mass on the indoor thermal dynamics and integration of phase change materials in furniture for building energy storage: A review. Renew. Sustain. Energy Rev. 2017, 69, 19–32. [Google Scholar] [CrossRef]
- Li, W.; Wang, S. A multi-agent based distributed approach for optimal control of multi-zone ventilation systems considering indoor air quality and energy use. Appl. Energy 2020, 275, 115371. [Google Scholar] [CrossRef]
- TRNSYS. TRNSYS 18 Documentation; 2017. Available online: https://sel.me.wisc.edu/trnsys/ (accessed on 12 January 2024).
- Anuntasethakul, C.; Banjerdpongchai, D. Design of Supervisory Model Predictive Control for Building HVAC System with Consideration of Peak-Load Shaving and Thermal Comfort. IEEE Access 2021, 9, 41066–41081. [Google Scholar] [CrossRef]
- GB/T18049-2017; Ergonomics of the thermal environment—Analytical determination and interpretation of thermal comfort using calculation of the PMV and PPD indices and local thermal comfort criteria. General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China and Standardization Administration of China: Beijing, China, 2017.
Model | SVM | ANN | XGBoost | LightGBM | ||||
---|---|---|---|---|---|---|---|---|
Target | Power (W) | T (°C) | Power (W) | T (°C) | Power (W) | T (°C) | Power (W) | T (°C) |
MAE | 25,653 | 0.124 | 18,081 | 0.068 | 11,354 | 0.056 | 15,827 | 0.078 |
RMSE | 35,189 | 0.145 | 24,704 | 0.084 | 16,149 | 0.070 | 24,218 | 0.094 |
R2 | 0.930 | 0.932 | 0.966 | 0.977 | 0.978 | 0.983 | 0.967 | 0.972 |
CV-RMSE | 8.63% | 0.57% | 6.06% | 0.33% | 4.29% | 0.28% | 5.94% | 0.37% |
Training time (s) | 0.0030 | 2.0414 | 0.3918 | 0.1078 | ||||
Prediction time (ms) | 0.1509 | 0.5159 | 0.8042 | 0.4057 | ||||
Optimal hyperparameter for predicting HVAC power use | kernel function = ‘rbf’ C = 8192 gamma = 1 × 10−5 | activation = ‘relu’ number of hidden layers = 2 number of neurons in hidden layers = 250 alpha = 0.1 | max depth of a tree = 10 subsample = 0.8 lambda = 0.4 learning rate = 0.04 tree number = 200 | Min data in leaf = 20 Subsample = 0.5 learning rate = 0.06 tree number = 250 | ||||
Optimal hyperparameter for predicting indoor temperature | kernel function = ‘rbf’ C = 8192 gamma = 1 × 10−5 | activation = ‘relu’ number of hidden layers = 2 number of neurons in hidden layers = 230 alpha = 0.1 | max depth of a tree = 10 subsample = 0.8 lambda = 0.5 learning rate = 0.08 tree number = 200 | Min data in leaf = 15 learning rate = 0.08 tree number = 220 |
Case | R2 (HVAC Power) | R2 (Indoor T) |
---|---|---|
1 | 0.967 (≈0.97) | 0.972 (≈0.97) |
2 | 0.90 | 0.90 |
3 | 0.83 | 0.83 |
4 | 0.76 | 0.76 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, H.; Mai, D.; Li, Q.; Ding, Z. Evaluating Machine Learning Models for HVAC Demand Response: The Impact of Prediction Accuracy on Model Predictive Control Performance. Buildings 2024, 14, 2212. https://doi.org/10.3390/buildings14072212
Wang H, Mai D, Li Q, Ding Z. Evaluating Machine Learning Models for HVAC Demand Response: The Impact of Prediction Accuracy on Model Predictive Control Performance. Buildings. 2024; 14(7):2212. https://doi.org/10.3390/buildings14072212
Chicago/Turabian StyleWang, Huilong, Daran Mai, Qian Li, and Zhikun Ding. 2024. "Evaluating Machine Learning Models for HVAC Demand Response: The Impact of Prediction Accuracy on Model Predictive Control Performance" Buildings 14, no. 7: 2212. https://doi.org/10.3390/buildings14072212