Infrared-Fused Vision-Based Thermoregulation Performance Estimation for Personal Thermal Comfort-Driven HVAC System Controls
Abstract
:1. Introduction
2. Literature Review
2.1. Physiological Sensing of Thermal Comfort
2.2. Fundamentals of Infrared Thermography for Thermal Comfort
3. Infrared-Fused Computer Vision for Capturing Thermoregulation Performance
3.1. Design of the Infrared-Fused Computer Vision System
3.2. Evaluation of Face Alignment Algorithms
4. Comfort Estimation and Setpoint Selection
5. Experimental Design and Procedure to Simulate Exposure to Warm and Cool Environments
6. Validation Results for Sensor/Controller Integrated System
Quantification of Thermal Comfort Efficacy
7. Discussion and Limitations
8. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Standard 55-2004; Thermal Environmental Conditions for Human Occupancy. American Society of Heating, Refrigerating and Air-Conditioning Engineers: Atlanta, GA, USA, 2004.
- Karjalainen, S. Thermal comfort and gender: A literature review. Indoor Air 2012, 22, 96–109. [Google Scholar] [CrossRef] [PubMed]
- Brager, G.S.; De Dear, R.J. Thermal adaptation in the built environment: A literature review. Energy Build. 1998, 27, 83–96. [Google Scholar] [CrossRef]
- Jendritzky, G.; Dear, R. de Adaptation and thermal environment. In Biometeorology for Adaptation to Climate Variability and Change; Springer: Berlin/Heidelberg, Germany, 2009; pp. 9–32. [Google Scholar]
- Ning, H.; Wang, Z.; Ji, Y. Thermal history and adaptation: Does a long-term indoor thermal exposure impact human thermal adaptability? Appl. Energy 2016, 183, 22–30. [Google Scholar] [CrossRef]
- Schellen, L.; van Marken Lichtenbelt, W.D.; Loomans, M.G.; Toftum, J.; De Wit, M.H. Differences between young adults and elderly in thermal comfort, productivity, and thermal physiology in response to a moderate temperature drift and a steady-state condition. Indoor Air 2010, 20, 273–283. [Google Scholar] [CrossRef]
- Uğursal, A.; Culp, C.H. The effect of temperature, metabolic rate and dynamic localized airflow on thermal comfort. Appl. Energy 2013, 111, 64–73. [Google Scholar] [CrossRef]
- Karmann, C.; Schiavon, S.; Arens, E. Percentage of commercial buildings showing at least 80% occupant satisfied with their thermal comfort. Indoor Environ. Qual. 2018. Available online: https://escholarship.org/uc/item/89m0z34x (accessed on 11 July 2022).
- Ghahramani, A.; Dutta, K.; Yang, Z.; Ozcelik, G.; Becerik-Gerber, B. Quantifying the influence of temperature setpoints, building and system features on energy consumption. In Proceedings of the 2015 Winter Simulation Conference (WSC), Huntington Beach, CA, USA, 6–9 December 2015; pp. 1000–1011. [Google Scholar]
- Ghahramani, A.; Zhang, K.; Dutta, K.; Yang, Z.; Becerik-Gerber, B. Energy savings from temperature setpoints and deadband: Quantifying the influence of building and system properties on savings. Appl. Energy 2016, 165, 930–942. [Google Scholar] [CrossRef]
- Yang, B.; Li, X.; Hou, Y.; Meier, A.; Cheng, X.; Choi, J.-H.; Wang, F.; Wang, H.; Wagner, A.; Yan, D. Non-invasive (non-contact) measurements of human thermal physiology signals and thermal comfort/discomfort poses—A review. Energy Build. 2020, 224, 110261. [Google Scholar] [CrossRef]
- Huizenga, C.; Zhang, H.; Arens, E.; Wang, D. Skin and core temperature response to partial-and whole-body heating and cooling. J. Therm. Biol. 2004, 29, 549–558. [Google Scholar] [CrossRef]
- Takada, S.; Matsumoto, S.; Matsushita, T. Prediction of whole-body thermal sensation in the non-steady state based on skin temperature. Build. Environ. 2013, 68, 123–133. [Google Scholar] [CrossRef]
- Ghahramani, A.; Castro, G.; Karvigh, S.A.; Becerik-Gerber, B. Towards unsupervised learning of thermal comfort using infrared thermography. Appl. Energy 2018, 211, 41–49. [Google Scholar] [CrossRef]
- Newell, A.; Yang, K.; Deng, J. Stacked hourglass networks for human pose estimation. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 8–16 October 2016; Springer: Berlin/Heidelberg, Germany, 2016; pp. 483–499. [Google Scholar]
- Ghahramani, A.; Tang, C.; Becerik-Gerber, B. An online learning approach for quantifying personalized thermal comfort via adaptive stochastic modeling. Build. Environ. 2015, 92, 86–96. [Google Scholar] [CrossRef]
- Ghahramani, A.; Castro, G.; Becerik-Gerber, B.; Yu, X. Infrared thermography of human face for monitoring thermoregulation performance and estimating personal thermal comfort. Build. Environ. 2016, 109, 1–11. [Google Scholar] [CrossRef]
- Guo, J.; Deng, J.; Xue, N.; Zafeiriou, S. Stacked dense u-nets with dual transformers for robust face alignment. arXiv 2018, arXiv:1812.01936. [Google Scholar]
- Charkoudian, N. Skin Blood Flow in Adult Human Thermoregulation: How It Works, When It Does Not, and Why. Mayo Clin. Proc. 2003, 78, 603–612. [Google Scholar] [CrossRef]
- Charkoudian, N. Mechanisms and modifiers of reflex induced cutaneous vasodilation and vasoconstriction in humans. J. Appl. Physiol. 2010, 109, 1221–1228. [Google Scholar] [CrossRef] [PubMed]
- Riganello, F.; Garbarino, S.; Sannita, W.G. Heart Rate Variability, Homeostasis, and Brain Function: A Tutorial and Review of Application. J. Psychophysiol. 2012, 26, 178–203. [Google Scholar] [CrossRef]
- Mansi, S.A.; Pigliautile, I.; Porcaro, C.; Pisello, A.L.; Arnesano, M. Application of wearable EEG sensors for indoor thermal comfort measurements. Acta IMEKO 2021, 10, 214–220. [Google Scholar] [CrossRef]
- Yang, B.; Cheng, X.; Dai, D.; Olofsson, T.; Li, H.; Meier, A. Macro pose based non-invasive thermal comfort perception for energy efficiency. arXiv 2018, arXiv:1811.07690. [Google Scholar]
- Wilke, K.; Martin, A.; Terstegen, L.; Biel, S.S. A short history of sweat gland biology. Int. J. Cosmet. Sci. 2007, 29, 169–179. [Google Scholar] [CrossRef]
- Reilly, T.; Waterhouse, J. Circadian aspects of body temperature regulation in exercise. J. Therm. Biol. 2009, 34, 161–170. [Google Scholar] [CrossRef]
- Yao, Y.; Lian, Z.; Liu, W.; Shen, Q. Experimental study on physiological responses and thermal comfort under various ambient temperatures. Physiol. Behav. 2008, 93, 310–321. [Google Scholar] [CrossRef]
- Yang, B.; Cheng, X.; Dai, D.; Olofsson, T.; Li, H.; Meier, A. Real-time and contactless measurements of thermal discomfort based on human poses for energy efficient control of buildings. Build. Environ. 2019, 162, 106284. [Google Scholar] [CrossRef]
- Li, D.; Menassa, C.C.; Kamat, V.R. Non-intrusive interpretation of human thermal comfort through analysis of facial infrared thermography. Energy Build. 2018, 176, 246–261. [Google Scholar] [CrossRef]
- Baker, F.C.; Siboza, F.; Fuller, A. Temperature regulation in women: Effects of the menstrual cycle. Temperature 2020, 7, 226–262. [Google Scholar] [CrossRef] [PubMed]
- Aryal, A.; Becerik-Gerber, B. A comparative study of predicting individual thermal sensation and satisfaction using wrist-worn temperature sensor, thermal camera and ambient temperature sensor. Build. Environ. 2019, 160, 106223. [Google Scholar] [CrossRef]
- Aryal, A.; Becerik-Gerber, B. Skin temperature extraction using facial landmark detection and thermal imaging for comfort assessment. In Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, New York, NY, USA, 13–14 November 2019; pp. 71–80. [Google Scholar]
- Pavlin, B.; Pernigotto, G.; Cappelletti, F.; Bison, P.; Vidoni, R.; Gasparella, A. Real-Time Monitoring of Occupants’ Thermal Comfort through Infrared Imaging: A Preliminary Study. Buildings 2017, 7, 10. [Google Scholar] [CrossRef]
- Wu, Y.; Liu, H.; Li, B.; Kosonen, R. Prediction of thermal sensation using low-cost infrared array sensors monitoring system. IOP Conf. Ser. Mater. Sci. Eng. 2019, 609, 032002. [Google Scholar] [CrossRef]
- Smith, J.W.; Aston, S.J.; Beasley, R.W.; Thorne, C. Grabb and Smith’s Plastic Surgery; Lippincott-Raven: Philadelphia, Pennsylvania, 1997; ISBN 0-316-32255-5. [Google Scholar]
- Bach, A.J.E.; Stewart, I.B.; Disher, A.E.; Costello, J.T. A Comparison between Conductive and Infrared Devices for Measuring Mean Skin Temperature at Rest, during Exercise in the Heat, and Recovery. PLoS ONE 2015, 10, e0117907. [Google Scholar] [CrossRef] [PubMed]
- de Andrade Fernandes, A.; dos Santos Amorim, P.R.; Brito, C.J.; de Moura, A.G.; Moreira, D.G.; Costa, C.M.A.; Sillero-Quintana, M.; Marins, J.C.B. Measuring skin temperature before, during and after exercise: A comparison of thermocouples and infrared thermography. Physiol. Meas. 2014, 35, 189–203. [Google Scholar] [CrossRef]
- Priego Quesada, J.I.; Martínez Guillamón, N.; de Anda, R.M.C.O.; Psikuta, A.; Annaheim, S.; Rossi, R.M.; Corberán Salvador, J.M.; Pérez-Soriano, P.; Salvador Palmer, R. Effect of perspiration on skin temperature measurements by infrared thermography and contact thermometry during aerobic cycling. Infrared Phys. Technol. 2015, 72, 68–76. [Google Scholar] [CrossRef]
- Bulat, A.; Tzimiropoulos, G. How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 1021–1030. [Google Scholar]
- Deng, J.; Zhou, Y.; Cheng, S.; Zaferiou, S. Cascade multi-view hourglass model for robust 3d face alignment. In Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an, China, 15–19 May 2018; pp. 399–403. [Google Scholar]
- Zhang, K.; Zhang, Z.; Li, Z.; Qiao, Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 2016, 23, 1499–1503. [Google Scholar] [CrossRef]
- Ghahramani, A.; Karvigh, S.A.; Becerik-Gerber, B. HVAC system energy optimization using an adaptive hybrid metaheuristic. Energy Build. 2017, 152, 149–161. [Google Scholar] [CrossRef]
- Kaushik, A.; Arif, M.; Tumula, P.; Ebohon, O.J. Effect of thermal comfort on occupant productivity in office buildings: Response surface analysis. Build. Environ. 2020, 180, 107021. [Google Scholar] [CrossRef]
- Xu, Q.; Goh, H.C.; Mousavi, E.; Nabizadeh Rafsanjani, H.; Varghese, Z.; Pandit, Y.; Ghahramani, A. Towards Personalization of Indoor Air Quality: Review of Sensing Requirements and Field Deployments. Sensors 2022, 22, 3444. [Google Scholar] [CrossRef]
- Ghahramani, A.; Galicia, P.; Lehrer, D.; Varghese, Z.; Wang, Z.; Pandit, Y. Artificial intelligence for efficient thermal comfort systems: Requirements, current applications and future directions. Front. Built Environ. 2020, 6, 49. [Google Scholar] [CrossRef]
- Aryal, A.; Ghahramani, A.; Becerik-Gerber, B. Monitoring fatigue in construction workers using physiological measurements. Autom. Constr. 2017, 82, 154–165. [Google Scholar] [CrossRef]
- Ghahramani, A.; Pantelic, J.; Vannucci, M.; Pistore, L.; Liu, S.; Gilligan, B.; Alyasin, S.; Arens, E.; Kampshire, K.; Sternberg, E. Personal CO2 bubble: Context-dependent variations and wearable sensors usability. J. Build. Eng. 2019, 22, 295–304. [Google Scholar] [CrossRef]
- Bhattacharya, A.; Pantelic, J.; Ghahramani, A.; Mousavi, E.S. Three-dimensional analysis of the effect of human movement on indoor airflow patterns. Indoor Air 2021, 31, 587–601. [Google Scholar] [CrossRef]
- Saeed, U.; Shah, S.Y.; Ahmad, J.; Imran, M.A.; Abbasi, Q.H.; Shah, S.A. Machine learning empowered COVID-19 patient monitoring using non-contact sensing: An extensive review. J. Pharm. Anal. 2022, 12, 193–204. [Google Scholar] [CrossRef]
- Ghahramani, A.; Jazizadeh, F.; Becerik-Gerber, B. A knowledge based approach for selecting energy-aware and comfort-driven HVAC temperature set points. Energy Build. 2014, 85, 536–548. [Google Scholar] [CrossRef]
Alg. | Up to the right | Up | Up to the left | |||||||
Precision | Recall | Precision | Recall | Precision | Recall | |||||
C1. Bulat et al. [38] | 90.0% | 95.0% | 100.0% | 100.0% | 95.0% | 100.0% | ||||
C2. InsightFace [39] | 90.0% | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% | ||||
C3. FaceNet [40] | 0.0% | 30.0% | 35.0% | 60.0% | 70.0% | 80.0% | ||||
Alg. | Right profile | Oblique Right | Front | Oblique Left | Left Profile | |||||
Precision | Recall | Precision | Recall | Precision | Recall | Precision | Recall | Precision | Recall | |
C1. Bulat et al. [38] | 45.0% | 80.0% | 80.0% | 95.0% | 95.0% | 95.0% | 90.0% | 90.0% | 85.0% | 90.0% |
C2. InsightFace [39] | 21.4% | 92.9% | 80.0% | 100.0% | 90.0% | 100.0% | 95.0% | 100.0% | 94.7% | 100.0% |
C3. FaceNet [40] | 0.0% | 15.0% | 0.0% | 30.0% | 30.0% | 45.0% | 65.0% | 70.0% | 10.0% | 10.0% |
Alg. | Down to the right | Down | Down to the left | |||||||
Precision | Recall | Precision | Recall | Precision | Recall | |||||
C1. Bulat et al. [38] | 10.0% | 65.0% | 15.0% | 35.0% | 45.0% | 70.0% | ||||
C2. InsightFace [39] | 5.9% | 35.3% | 0.0% | 58.3% | 41.7% | 91.7% | ||||
C3. FaceNet [40] | 0.0% | 0.0% | 0.0% | 0.0% | 5.0% | 10.0% |
Alg. | Light-On (450 lux) | Light-Off (70 lux) | ||
---|---|---|---|---|
Precision | Recall | Precision | Recall | |
C1. Bulat et al. [38] | 66.4% | 81.8% | 59.1% | 78.2% |
C2. InsightFace [39] | 73.6% | 84.6% | 62.7% | 81.8% |
C3. FaceNet [40] | 21.8% | 36.4% | 17.3% | 27.3% |
Subject | Gender | System Response after Warm Exposure | System Response after Cool Exposure | ||
---|---|---|---|---|---|
Personalized Model | Generalized Model | Personalized Model | Generalized Model | ||
1 | M | Cooling | Cooling | Heating | Heating |
2 | F | Cooling | None | Heating | Heating |
3 | M | Cooling | None | Heating | Heating |
4 | M | Cooling | Heating | Heating | Heating |
5 | F | – | – | Heating | Heating |
6 | F | Cooling | Heating | Heating | Heating |
7 | M | Cooling | Cooling | Heating | Heating |
8 | M | Cooling | Cooling | Heating | Heating |
9 | M | Cooling | Cooling | Cooling | Cooling |
10 | M | Cooling | Cooling | Heating | Cooling |
11 | M | None | Cooling | Heating | Heating |
12 | M | Cooling | None | Heating | None |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ghahramani, A.; Xu, Q.; Min, S.; Wang, A.; Zhang, H.; He, Y.; Merritt, A.; Levinson, R. Infrared-Fused Vision-Based Thermoregulation Performance Estimation for Personal Thermal Comfort-Driven HVAC System Controls. Buildings 2022, 12, 1241. https://doi.org/10.3390/buildings12081241
Ghahramani A, Xu Q, Min S, Wang A, Zhang H, He Y, Merritt A, Levinson R. Infrared-Fused Vision-Based Thermoregulation Performance Estimation for Personal Thermal Comfort-Driven HVAC System Controls. Buildings. 2022; 12(8):1241. https://doi.org/10.3390/buildings12081241
Chicago/Turabian StyleGhahramani, Ali, Qian Xu, Syung Min, Andy Wang, Hui Zhang, Yingdong He, Alexander Merritt, and Ronnen Levinson. 2022. "Infrared-Fused Vision-Based Thermoregulation Performance Estimation for Personal Thermal Comfort-Driven HVAC System Controls" Buildings 12, no. 8: 1241. https://doi.org/10.3390/buildings12081241