Exploring the Potential of Emerging Digitainability—GPT Reasoning in Energy Management of Kindergartens

Abstract

One of the barriers to the rapid transition of societies toward a more sustainable future is a scarcity of field experts. Members of scientific and professional communities believe that this obstacle could be overcome by supplementing the decisions of non-experts with artificial intelligence. To examine this opportunity, this study examines the viability of GPT-3.5 as an expert adviser in the energy management of kindergartens. Thus, field experts investigated the deductive and inductive reasoning potential of GPT-LLM (Large Language Model). The first task was conducted on a sample of kindergartens in the Western Balkans. The LLM was instructed to provide the buildings’ specific heat consumption (SHC) by relatively detailed building descriptions and building occupancy. The second task involved kindergartens in various European locations, and the LLM was tasked with estimating energy savings using limited data about the renovation process. The study found deductive reasoning to be insufficient for estimating SHC from the building envelope details, with average accuracy below the least predictive model (R² = 0.56; MAPE = 48%). Including the factor of occupancy, the SHC estimates were relatively accurate, wherein the first deductive test proved precise (MAPE = 27%), but it was less so in the opposite case (MAPE = 67%). In terms of inductive reasoning, the LLM assumptions were relatively consistent with practice.

1. Introduction

Due to ever-increasing scientific progress, members of modern societies must adapt to social and technological changes (STCs) faster than previous generations [1]. In the preceding saeculum, the pace of technological change was predetermined by society’s ability to automate industries and establish diverse service sectors [2]. In contrast, contemporary saeculum STCs are driven by networking, digitalization, and the ever-increasing presence of artificial intelligence (AI) [3]. The key disparity between the two periods is that the former’s dynamic was determined by infrastructure development (e.g., roads, railroads, and the internet), whereas the latter is not. Furthermore, previous technological advancements primarily impacted working-class jobs, while novel technology focuses on decision making, influencing mainly white-collar occupations [4]. As a result, future human progress should be faster than in the past [5], and decisions affecting it will be made with less effort [6]. Ideally, this should enable shared prosperity for humanity, prevent global conflicts, and promote overall well-being [7]. By harnessing the synergy of sustainable ideas and digitalization, defined as digitainability [8], the prospects for a more sustainable future should be brighter than they were previously.

1.1. Subject of Research

One of the relatively significant technological advances in the AI field pertains to the development of Large Language Models (LLMs)—algorithms specifically designed to simulate conversations with human users [9]. Although chatbots have been in use since 1966 [10], they have only recently gained widespread attention due to significant improvements in their usability [11]. These advancements were made possible by progress in natural language processing (NLP) algorithms [12] employing unsupervised learning techniques. Unlike the supervised NLPs utilized by some chatbots before, the latter does not require explicit human instructions or data labeling for LLM chatbot training. This allows prompt learning on large amounts of textual data, and this approach was applied to OpenAI’s Generative Pre-trained Transformer (GPT) [13]. To be as exhaustive as possible, GPT was trained on vast amounts of data (

Table 1. GPT-3 knowledge base [17].

Dataset	Number of Tokens	Training Mix
Common Crawl (filtered)	490 billion	60%
WebText2	19 billion	22%
Books1	12 billion	8%
Books2	55 billion	8%
Wikipedia	3 billion	3%

) gathered from different sources (Common Crawl [14], WebText2 [15], Wikipedia [16], and two separate sets of books available on the internet (Books1 and Books2).

The text on which the GPT was trained was divided into smaller units of words or sub-words (tokens). Each of the tokens had embedding that allowed the model to understand the context and the relationship between the words. In this context, the text output LLM provides is based on predictions of the tokens that follow the textual input sequence. To reduce harmful bias and factual inaccuracies, the data LLM was trained on were thoroughly cleaned and filtered. This could include techniques such as identifying and removing harmful stereotypes, flagging potential misinformation, and maintaining data quality standards [17]. Upon the training, the model underwent a fine-tuning process, i.e., adaptation of the pre-trained model to a new task. This can be accomplished by prompt-based fine-tuning, in which the user provides directions for the LLM on how to come to output; or few-shot learning, in which the LLM adapts to a new task following given examples [17].

As a result of the new technology’s development, members of the scientific and professional communities began to investigate the opportunities for GPT application in augmenting (non-)experts’ knowledge, highlighting the novel technology’s strengths and weaknesses. Table 2 provides a brief overview of the studies that have addressed this topic.

Table 2. Short overview of the studies examining GPT usability in a variety of professions.

Field	Ref.	Country	Study Aim	Study Outcome	Stated Concerns/Downsides
Industry	[18]	United Kingdom	To investigate how GPT can be used to reduce waste generation, improve product quality, and achieve sustainability in the textile industry.	By utilizing GPT, companies in the textile industry can improve the customer experience and make their services more efficient, cost-effective, and prompt.	Not stated.
	[19]	United Arab Emirates	To evaluate GPT output by a pool of participants (experts); to gather feedback regarding the overall interaction experience and the quality of the GPT output.	The participants had an overall positive interaction experience and indicated the potential of such a tool in automating many preliminary and time-consuming tasks.	The response is not reliable; generic and boilerplate statements; not connected to real-time internet data.
	[20]	United Kingdom	To explore what users anticipate from AI; to gain insight into GPT’s applications and the potential effects they may have soon.	GPT can improve interactive learning, simplify collaborations between students and teachers, and provide a more efficient way to store and access course materials.	Privacy and data security; potential to replace human jobs.
Environment and Sustainable Development	[21]	Brazil	To examine the usability of five LLM models in natural resources management decision making.	In the context of water management, it is possible to support human decisions by the use of conversational agents.	Not stated.
	[22]	Austria	To evaluate contributions and the potential impact of AI on sustainable development in the society domain.	AI has the potential to significantly aid in achieving sustainable development goals.	Lack of transparency concerning AI decisions; bias built into the algorithms; overreliance on automated solutions rather than human intervention.
	[23]	Austria	To investigate the benefits of AI for digitalization, urbanization, globalization, climate change, automation and mobility, global health issues, and the aging population.	GPT-3 provides easily understandable insights into the complex and cross-sectional matters of megatrends.	AI systems can make mistakes or generate wrong output.
	[24]	India	To investigate how GPT can be used to spread the concept and benefits of nearly-zero-energy buildings through the academic community.	GPT can contribute to activities aimed at spreading the benefits of sustainable development.	Not stated.
	[25]	Germany	To investigate the political reasoning, biases, and limitations of GPT.	GPT argues for pro-environmental, left-libertarian ideology. It would impose taxes on flights, restrict rent increases, and legalize abortion.	The study examined just two political orientations, i.e., Germany’s Wahl-O-Mat and the Netherlands’s Stem Wijzer.
Education	[4]	Singapore	To discuss the potentials of GPT in education and research; discuss student-facing, teacher-facing, and system-facing applications; and analyze opportunities and threats.	Despite the challenges that GPT poses for traditional assessments, it will not necessarily lead to their extinction. Instead, it will encourage educators to use AI tools to create diverse assessments that evaluate deeper understanding and critical thinking.	Academic dishonesty; superficial understanding; overreliance on chatbots.
	[26]	Kenya	To explore the possibility of implementing a constructivist learning environment using chatbot technology.	Chatbot technology can contribute to education through active and social learning.	Not stated.
	[27]	United Kingdom	To establish an understanding of the ethics of AI applied in educational contexts.	While initial indicators suggest a lack of interest in the ethics of AI in education, the community recognizes its significance. To improve ethical engagement, discussions and frameworks are required to ensure ethical principles for meaningful real-world impact.	Uncertainties in equity, fairness, confidentiality, and anonymity.
	[28]	United States	(Not directly stated) Conversation was aimed to explore complex issues and propose solutions and strategies.	Not directly stated.	(Not directly stated) Limited access to external resources (references).
	[29]	United States	To evaluate the abstracts using an AI output detector, plagiarism detector, and blinded human reviewers trying to distinguish whether abstracts were original or generated.	Most generated abstracts were detected using the AI output detector. Blinded human reviewers correctly identified 68% of generated abstracts as being generated by GPT.	GPT writes believable scientific abstracts, though with completely generated data.
	[30]	India, Zambia	To understand the perceptions and opinions of academicians toward GPT by collecting and analyzing social media comments, and a survey was conducted with library and information science professionals.	While some academicians may not accept GPT-3, most are starting to accept it.	GPT reduces critical thinking and raises ethical concerns.
	[31]	United States	To evaluate the performance of GPT on questions within the scope of the United States Medical Licensing Examination Step 1 and Step 2 exams, as well as to analyze responses for user interpretability.	By performing at a greater than 60% threshold, the model achieved the equivalent of a passing score for a third-year medical student.	GPT training data were not up to date.
	[32]	China	To evaluate GPT capabilities in open-ended question answering, factual modeling, and following instructions. The study highlights the strengths and weaknesses of the bot in comparison with human experts.	Although GPT demonstrated impressive capabilities, it still cannot replace human experts.	The study findings were based on unbalanced data.
	[33]	Slovakia, UAE, Czech Republic	To provide an up-to-date overview of upcoming changes and advancements in the use of AI in dental education.	GPT can facilitate communication between healthcare providers and patients.	Ethical and legal implications.
	[34]	Germany	To assess the quality of radiology reports simplified by GPT. The evaluation was performed by 15 radiologists.	Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient.	Instances of incorrect statements; missed key; medical findings.
Computing	[35]	China	To provide an overview of GPT, its features, benefits, and challenges.	GPT is a promising AI technology that can be used to automate conversations and generate more accurate responses.	Security and limited capabilities.
	[36]	United States	To assist researchers and developers in enhancing future language models and chatbots.	Despite its impressive capabilities, GPT improvement is necessary for it to excel in areas such as reasoning, mathematical problem solving, and reducing bias.	Unsatisfactory context comprehension; weak math and arithmetic skills; perception of ethics and morality; difficulty using idioms.
	[37]	United States	(Not directly stated) Highlighting potential limitations of GPT, such as its ability to generate inaccurate or meaningless content as well as raising concerns about the technology’s potential harm.	(Not directly stated) GPT has limitations.	Overreliance on AI is harmful.

Table 2. Short overview of the studies examining GPT usability in a variety of professions.

Field	Ref.	Country	Study Aim	Study Outcome	Stated Concerns/Downsides
Industry	[18]	United Kingdom	To investigate how GPT can be used to reduce waste generation, improve product quality, and achieve sustainability in the textile industry.	By utilizing GPT, companies in the textile industry can improve the customer experience and make their services more efficient, cost-effective, and prompt.	Not stated.
	[19]	United Arab Emirates	To evaluate GPT output by a pool of participants (experts); to gather feedback regarding the overall interaction experience and the quality of the GPT output.	The participants had an overall positive interaction experience and indicated the potential of such a tool in automating many preliminary and time-consuming tasks.	The response is not reliable; generic and boilerplate statements; not connected to real-time internet data.
	[20]	United Kingdom	To explore what users anticipate from AI; to gain insight into GPT’s applications and the potential effects they may have soon.	GPT can improve interactive learning, simplify collaborations between students and teachers, and provide a more efficient way to store and access course materials.	Privacy and data security; potential to replace human jobs.
Environment and Sustainable Development	[21]	Brazil	To examine the usability of five LLM models in natural resources management decision making.	In the context of water management, it is possible to support human decisions by the use of conversational agents.	Not stated.
	[22]	Austria	To evaluate contributions and the potential impact of AI on sustainable development in the society domain.	AI has the potential to significantly aid in achieving sustainable development goals.	Lack of transparency concerning AI decisions; bias built into the algorithms; overreliance on automated solutions rather than human intervention.
	[23]	Austria	To investigate the benefits of AI for digitalization, urbanization, globalization, climate change, automation and mobility, global health issues, and the aging population.	GPT-3 provides easily understandable insights into the complex and cross-sectional matters of megatrends.	AI systems can make mistakes or generate wrong output.
	[24]	India	To investigate how GPT can be used to spread the concept and benefits of nearly-zero-energy buildings through the academic community.	GPT can contribute to activities aimed at spreading the benefits of sustainable development.	Not stated.
	[25]	Germany	To investigate the political reasoning, biases, and limitations of GPT.	GPT argues for pro-environmental, left-libertarian ideology. It would impose taxes on flights, restrict rent increases, and legalize abortion.	The study examined just two political orientations, i.e., Germany’s Wahl-O-Mat and the Netherlands’s Stem Wijzer.
Education	[4]	Singapore	To discuss the potentials of GPT in education and research; discuss student-facing, teacher-facing, and system-facing applications; and analyze opportunities and threats.	Despite the challenges that GPT poses for traditional assessments, it will not necessarily lead to their extinction. Instead, it will encourage educators to use AI tools to create diverse assessments that evaluate deeper understanding and critical thinking.	Academic dishonesty; superficial understanding; overreliance on chatbots.
	[26]	Kenya	To explore the possibility of implementing a constructivist learning environment using chatbot technology.	Chatbot technology can contribute to education through active and social learning.	Not stated.
	[27]	United Kingdom	To establish an understanding of the ethics of AI applied in educational contexts.	While initial indicators suggest a lack of interest in the ethics of AI in education, the community recognizes its significance. To improve ethical engagement, discussions and frameworks are required to ensure ethical principles for meaningful real-world impact.	Uncertainties in equity, fairness, confidentiality, and anonymity.
	[28]	United States	(Not directly stated) Conversation was aimed to explore complex issues and propose solutions and strategies.	Not directly stated.	(Not directly stated) Limited access to external resources (references).
	[29]	United States	To evaluate the abstracts using an AI output detector, plagiarism detector, and blinded human reviewers trying to distinguish whether abstracts were original or generated.	Most generated abstracts were detected using the AI output detector. Blinded human reviewers correctly identified 68% of generated abstracts as being generated by GPT.	GPT writes believable scientific abstracts, though with completely generated data.
	[30]	India, Zambia	To understand the perceptions and opinions of academicians toward GPT by collecting and analyzing social media comments, and a survey was conducted with library and information science professionals.	While some academicians may not accept GPT-3, most are starting to accept it.	GPT reduces critical thinking and raises ethical concerns.
	[31]	United States	To evaluate the performance of GPT on questions within the scope of the United States Medical Licensing Examination Step 1 and Step 2 exams, as well as to analyze responses for user interpretability.	By performing at a greater than 60% threshold, the model achieved the equivalent of a passing score for a third-year medical student.	GPT training data were not up to date.
	[32]	China	To evaluate GPT capabilities in open-ended question answering, factual modeling, and following instructions. The study highlights the strengths and weaknesses of the bot in comparison with human experts.	Although GPT demonstrated impressive capabilities, it still cannot replace human experts.	The study findings were based on unbalanced data.
	[33]	Slovakia, UAE, Czech Republic	To provide an up-to-date overview of upcoming changes and advancements in the use of AI in dental education.	GPT can facilitate communication between healthcare providers and patients.	Ethical and legal implications.
	[34]	Germany	To assess the quality of radiology reports simplified by GPT. The evaluation was performed by 15 radiologists.	Most radiologists agreed that the simplified reports were factually correct, complete, and not potentially harmful to the patient.	Instances of incorrect statements; missed key; medical findings.
Computing	[35]	China	To provide an overview of GPT, its features, benefits, and challenges.	GPT is a promising AI technology that can be used to automate conversations and generate more accurate responses.	Security and limited capabilities.
	[36]	United States	To assist researchers and developers in enhancing future language models and chatbots.	Despite its impressive capabilities, GPT improvement is necessary for it to excel in areas such as reasoning, mathematical problem solving, and reducing bias.	Unsatisfactory context comprehension; weak math and arithmetic skills; perception of ethics and morality; difficulty using idioms.
	[37]	United States	(Not directly stated) Highlighting potential limitations of GPT, such as its ability to generate inaccurate or meaningless content as well as raising concerns about the technology’s potential harm.	(Not directly stated) GPT has limitations.	Overreliance on AI is harmful.

According to Thurzo et al. [33], ChatGPT can prompt quick decisions with reasonably accurate diagnoses and solutions, resulting in increased operational effectiveness. In terms of sustainable development, Rathore [18] explored the opportunities of ChatGPT utilization in the textile industry, indicating that technology can mitigate waste generation, improve the quality of products, and contribute to sustainability goals. Alves et al. [21] had a similar conclusion, confirming that chatbots can contribute to decision making in natural resource management. Prieto et al. [19] demonstrated that GPT can generate a coherent construction schedule for a simple construction project. According to the authors, the platform used a logical approach to completing the task scope. Other research found that AI platforms can facilitate intelligent traffic management systems [20] and improve the efficiency of supply chains [22]. The Internet of Things and artificial intelligence, in that regard, can be combined to create the AIoT (artificial intelligence of things), improving building and process performance [35]. AI-driven analytics can also be used to identify the impact of climate change on certain communities [23]. Jungwirth and Haluza [22], for example, note that ChatGPT could be useful in addressing social megatrends, though they warn that much work on the platform and its proper use is required before tangible results can be seen. This can be of particular use for both developed and developing countries’ educational systems [24,26].

In contrast to just positive aspects, Holmes et al. [27] see the prior opportunities as a threat to humanity, as AI may not always reflect the values of society as a whole. Hartam et al. [25] provided converging evidence on ChatGPT’s pro-environmental, left-libertarian orientation. Borji [36] created a categorical archive of ChatGPT failures, referring to false information as bot hallucinations. These errors were observed in other studies as well [4], some of which emphasized the absence of [38] or incorrectly stated references [28] as a particular issue. Marcus and Davis declared GPT to be a “not reliable interpreter of the world” [37], whereas Gao et al. [29] stated the platform can generate realistic scientific abstracts, but the data could be completely made up. Because of all this, Subaveerapandiyan et al. [30] indicate that ChatGPT should aid decisions rather than generate ideas. Consequently, the confidence in ChatGPT as an expert adviser has been examined in several professional and scientific domains.

Guo et al. [32] created a dataset of 40,000 questions and an appropriate mixture of expert and artificially generated answers to test how closely ChatGPT resembles human experts. The question–answer pairs were provided to a pool of experts and non-experts to characterize them. In comparison with expert reports, the study found that the machine writing style was relatively weak, which has also been shown in some other studies [39,40]. Because of this, successfully contrasting different styles was not as difficult a task for experts as it was for non-experts. However, non-experts understood the artificially generated answers better than the expert responses because the former were plainer and simpler. Other studies have proven that ChatGPT has sufficient “knowledge” and adequate reasoning to pass graduate exams in law and business schools, score in the top 10% on a law exam [41], and assist juristic decisions [41]. A study conducted in Turkey showed that ChatGPT performed better than anatomy students [39], while a similar study found that the bot would pass the third year at the faculty of medicine in the US [31]. Even more, Jeblick et al. [34] suggest using ChatGPT in addition to expert opinions. Moving on to more complex intellectual analyses, Borji [36] subjected the bot to a series of challenging logical tests to determine the overall potential of ChatGPT reasoning. He found it to have relatively good physical reasoning skills and particular challenges when dealing with spatial, temporal, psychological, and commonsense tasks. To summarize the reviews: ChatGPT has proven its worth both in the hands of experts (discussing the challenges of modern humanity) and in the hands of non-experts (as an advisor). However, due to the challenges that still exist in terms of AI reliability, governments of countries and regions are treating AI innovations with particular caution [42,43]. Final decisions recommending the use of technology would require years of professional and scientific evaluations to prove the technology is useful and compliant with the ethical principles present in the Data for Humanity Initiative [39]. To contribute to these efforts, this study aims to examine the usability of GPT as an advisor tool in the domain of kindergarten energy management. In this context, experts in the field of energy management evaluated the usability of ChatGPT as an advisor for non-experts. There is no similar study in the available literature. The study findings should fill existing knowledge gaps by answering the following research questions: how successfully GPT can deal with the topic of energy management in kindergartens and how useful the bot could be for energy managers. The novelty of the study lies in the exploration of ChatGPT as an advisory tool in the specific context of energy management in buildings. The study aims to inform and influence AI practice in educational and professional settings.

1.2. Object of Research

The object of the research in this study is a sample of educational buildings, i.e., kindergartens. These buildings were chosen for analysis because they accommodate the youngest population, require strict comfort control, and are prioritized in renovation efforts, making them ideal starting points for research into energy management and comfort in buildings. Depending on the latitude and level of industrial development, buildings in the EU are responsible for 60–80% of countries’ final energy consumption [44], and public buildings consume about 50% more specific heat (SHC) (kWh/m²/a) than residential buildings [45]. Because of this, buildings are the focus of modern initiatives dealing with a more sustainable future and better-organized societies [46]. One of the obstacles to the anticipated level of advancements in the field of public building energy management is a lack of subject matter experts [44]. To address this issue, scientists and professionals in the field developed a variety of simple-to-use models that enable non-experts to monitor and predict building energy consumption. Jurisevic et al. assessed the performance of various predictive models to target energy [47] and water [48] consumption in public preschool buildings, achieving up to 92% accuracy. Similar models were developed in other studies for a variety of building types, including school buildings (86% accuracy) [49], educational buildings (60%) [50], university campuses (89%) [51], banks (up to 69%) [52], and supermarkets (86% to 95%) [51]. Although the models perform relatively well, their limitation is the fact that they were developed on relatively small building samples. Consequently, the models would not accurately describe the energy performance of buildings out of the sample. Apart from this, users of the models need to have at least some field knowledge, as the results of the model are simply numbers representing either the building’s energy consumption or its potential for energy savings. On the other hand, LLMs allow building operators to consult the AI platform for advice and potentially receive the right answer. No prior programming or statistical knowledge is required from the operator. Unlike predictive models, GPT answers are generated for a single building, which is an advantage over models that were developed using a sample of multiple buildings. Additionally, the GPT response would not be a number, but rather a clear and concise written response, which is another benefit for non-experts [32]. This could be one of the positive effects that novel technology could have on contemporary challenges that modern humanity encounters in pursuing a more sustainable future. In addition, GPT should execute deductive and inductive reasoning to respond to this specific challenge. This should help the scientific and professional public to gain a better understanding of the platform’s reasoning skills. Contributions made by this study are consistent with the Data for Humanity Initiative [39].

2. Materials and Methods

The study used two building samples to assess the effectiveness of GPT (gpt-3.5-turbo) reasoning in managing energy in kindergartens: (1) buildings situated in the same region—a city in the Western Balkans—and (2) buildings distributed across various locations in Europe. The first building sample was used to evaluate GPT’s effectiveness in predicting the energy consumption of buildings with different floor areas and different construction periods. The second set of buildings was utilized to assess the ability of GPT to estimate energy savings upon building renovation in various locations (Figure 1). The first building sample was relatively well described (

Table 3. Details of the first sample of educational buildings—public kindergartens located in the same city.

		Building Thermal Envelope Details (i)
		1	2	3	4	5	6	7	8	9	10
		Built Year	Number of Floors	External Walls Gross Area	Heated Floor Area	Gross Heated Volume	Gross Glazing Area	External Walls U-Value	Glazing Elements U-Value	Ceiling U-Value	Roof Type
		[-]	[-]	[m2]	[m2]	[m3]	[m2]	[W/m2K]	[W/m2K]	[W/m2K]	[-]
Kindergarten No (kn)	1	1947	3	468	484	1382	92	1.38	4.01	0.37	Flat
	2	1948	1	740	452	1429	98	1.28	3.68	1.75	Pitched
	3	1968	2	1121	862	2888	548	0.5	1.59	0.52	Flat
	4	1973	2	738	860	2580	236	1.38	3.6	0.25	Flat
	5	1974	3	1036	1174	3745	270	0.46	3.21	0.35	Pitched
	6	1974	1	764	1370	4482	499	2.0	4.26	1.4	Pitched
	7	1974	2	1942	537	5199	453	0.46	3.52	0.34	Pitched
	8	1974	2	685	807	2598	273	1.16	2.88	1.4	Pitched
	9	1980	2	2708	1321	4057	461	1.38	3.52	1.53	Pitched
	10	1982	2	2480	2379	7636	755	0.34	3.11	0.34	Pitched
	11	2008	1	311	387	1136	68	0.16	2.71	0.35	Pitched
	12	2010	1	230	464	1508	80	0.16	2.9	0.35	Pitched

), whereas the second was not as much (Table 4). Consequently, one will be used to test GPT precision (deductive reasoning), while the other to test the LLM’s ability to assess building performance from a relatively subpar building description (inductive reasoning). Figure 1 depicts the locations, images, and basic information about the analyzed buildings, such as the year of construction and heated floor area. Buildings from the first study sample are shown in blue, while those from the second are shown in red squares.

Details describing the first set of buildings (Table 3) were taken from Jurišević’s doctoral dissertation [44]. Twelve kindergartens were described in great detail in the building information section. Inputs used in the study were sufficient for accurately estimating the buildings’ SHC, achieving performance metrics comparable to those reported in state-of-the-art approaches from the literature (R²: 0.92; MAPE: 14%) [47]. Henceforth, this study considered the selected inputs sufficient for drawing reliable deductive conclusions when estimating the SHC of the chosen building sample.

A second set of building details to which this study refers was gathered from energy reports and scientific papers. These publications gave different and less thorough descriptions of buildings than they did of energy-saving techniques and energy savings realized. As a result, the available data were unsuitable for drawing deductive conclusions. Nevertheless, these limitations did not hinder the use of inductive reasoning, which involves deriving conclusions from a limited or insufficient set of information. Table 4 lists the available details of four kindergartens in a relatively comparative manner before and after renovation.

Table 4. Details of the second sample of educational buildings—public kindergartens distributed across Europe.

				Building Location (l)
				l1	l2	l3	l4
				Vejtoften, Denmark [53]	Wolgast, Germany [54]	Graz, Austria [55]	Tver, Russia [56]
Before Renovation	Data Label (k)	6	Built year	Not stated	1973	1970	Not stated
		5	Heated floor area	221 m2	2339 m2	992 m2	632 m2
		4	Number of stories	1	2	2	2
		3	Fenestration details	Traditional double-glazed windows	Unknown	Unknown	Wooden frame windows with a total surface of 151 m2
		2	External walls details	With 95 mm thermal insulation (not stated what type)	Unknown	Unknown	Building brick, plastered and painted, the percent of wear makes 64%
		1	Roof details	Pitched, with 145 mm thermal insulation (not stated what type)	Flat	Pitched	Pitched roof is on rafters and an obreshetka
			Energy consumption	167.4 kWh/m2/a	158 kWh/m2/a	Not stated	Not stated
Upon Renovation	Data Label (j)	5	Modernization completed in	Before 2015	2009	2010	Before 2014
		4	Fenestration details	Triple-glazed windows	Double glazing with insulating protection (U-value including frame 1.4)	Replacement of windows	Metaplastic-framed windows with a total surface of 151 m2
		3	External walls details	With 390 mm thermal insulation (not stated what type)	Exterior wall insulation with mineral wool (15 cm, U-value 0.22)	Additional thermal insulation of external walls	Not renovated
		2	Roof details	Pitched, with 145 mm thermal insulation (not stated what type)	Roof insulation (30 cm, U-value 0.12)	Not stated	Not renovated
		1	Additional measures	In order to reduce/remove thermal bridge effects at the uninsulated base/foundation of the building, 200 mm of insulation was added on the outside to a depth of 400 mm.	Not stated	Thermal insulation of heat pipes	Not stated
			Energy consumption	91.7 kWh/m2/a	116 kWh/m2/a	Not stated	Not stated
			Energy or CO2 savings	45.2%	70 t/a	70%	40%

Table 4. Details of the second sample of educational buildings—public kindergartens distributed across Europe.

				Building Location (l)
				l1	l2	l3	l4
				Vejtoften, Denmark [53]	Wolgast, Germany [54]	Graz, Austria [55]	Tver, Russia [56]
Before Renovation	Data Label (k)	6	Built year	Not stated	1973	1970	Not stated
		5	Heated floor area	221 m2	2339 m2	992 m2	632 m2
		4	Number of stories	1	2	2	2
		3	Fenestration details	Traditional double-glazed windows	Unknown	Unknown	Wooden frame windows with a total surface of 151 m2
		2	External walls details	With 95 mm thermal insulation (not stated what type)	Unknown	Unknown	Building brick, plastered and painted, the percent of wear makes 64%
		1	Roof details	Pitched, with 145 mm thermal insulation (not stated what type)	Flat	Pitched	Pitched roof is on rafters and an obreshetka
			Energy consumption	167.4 kWh/m2/a	158 kWh/m2/a	Not stated	Not stated
Upon Renovation	Data Label (j)	5	Modernization completed in	Before 2015	2009	2010	Before 2014
		4	Fenestration details	Triple-glazed windows	Double glazing with insulating protection (U-value including frame 1.4)	Replacement of windows	Metaplastic-framed windows with a total surface of 151 m2
		3	External walls details	With 390 mm thermal insulation (not stated what type)	Exterior wall insulation with mineral wool (15 cm, U-value 0.22)	Additional thermal insulation of external walls	Not renovated
		2	Roof details	Pitched, with 145 mm thermal insulation (not stated what type)	Roof insulation (30 cm, U-value 0.12)	Not stated	Not renovated
		1	Additional measures	In order to reduce/remove thermal bridge effects at the uninsulated base/foundation of the building, 200 mm of insulation was added on the outside to a depth of 400 mm.	Not stated	Thermal insulation of heat pipes	Not stated
			Energy consumption	91.7 kWh/m2/a	116 kWh/m2/a	Not stated	Not stated
			Energy or CO2 savings	45.2%	70 t/a	70%	40%

Due to the different nature of the available data, the instructions provided to GPT for the first and second sets of buildings differ. However, to make the GPT responses suitable for fair analysis, the prompt commands were issued in the same way for all buildings from the same set. Commands to the GPT were instructed throughout OpenAI’s playground [57] platform, where parameters such as temperature, maximum response length, diversity, wording frequency, and text presence penalties could be set. The parameter values this study utilized are presented in

Table 5. OpenAI playground settings.

GPT Parameters	Parameter Value	Parameter Role [17,58]
Model	“gpt-3.5-turbo”	A deep learning model that generates text employing a neural network.
Temperature (ranging from 0 to 1)	1	Determines the randomness of the response. The more closely the temperature approaches 0, the less erratic the result will be.
Maximum length (ranging from 0 to 2048)	200	Caps a number of tokens that are allowed for a response. This varies according to the type of model.
Stop sequences (user input)	-	Makes responses end at the desired point, such as the end of a sentence or list.
Top probabilities/Top P (ranging from 0 to 1)	1	Controls which tokens the model will consider when generating a response. Setting this to 0.9 will consider the top 90% most likely of all possible tokens.
Frequency penalty (ranging from 0 to 1)	0	Controls the repetition of the same tokens in the generated response. The higher the penalty, the lower the probability of seeing the same tokens more than once in the same response.
Presence penalty (ranging from 0 to 2)	0	Reduces the chance of repeating any token that has appeared in the text. It is stricter than the frequency penalty, so it increases the likelihood of introducing new topics in a response.

.

2.1. GPT-3.5 Deductive Reasoning Test

To examine the usability of GPT as an adviser in kindergarten energy management, a deductive reasoning test was conducted. To evaluate GPT reasoning, the study utilized input-based prompting to initiate the bot’s deductive reasoning. In this regard, the prompt instructions included: (a) building description section (D) and (b) questioning sections (Q). The order and content (italic text) of the instructions were as follows:

D1: 

The public kindergarten is located in Kragujevac, Serbia. It was built in year i1 (Table 3) and has not been renovated since. The other details of the building are i = from 2 to 11 (all inputs were entered together with their units available in Table 3). The building is heated and naturally ventilated from 6:30 am to 9:30 pm.

Q1: 

How much heat is expected for the building to consume during the heating seasons [kWh/m²/a] with the following number of heating degree days: (a) 2133 K∙Day; (b) 2349 K∙Day; and (c) 2510 K∙Day.

Conclusions on the quality of deductive reasoning were drawn from expert judgment based on a comparison of the GPT and mathematically based assessments in [47]. In addition, the study examined the potential of GPT to account for the impact of occupancy (i.e., occupant behavior) on building energy performance. This factor is difficult to quantify and is therefore often overlooked in the field of predictive analytics [44]. Potential advances in novel technologies that can address this challenge could enhance the calibration of predictive models and make predictions more accurate. Because the influence of occupancy on a building’s energy performance is better measured in relatively small time steps, this study tested GPT deductive reasoning on a monthly rather than annual time frame. In this context, GPT was provided with the number of HDDs, calculated following Equation (1):

$$\mathrm{H}\mathrm{D}\mathrm{D}={\sum }_{j=1}^{DHS}{Tm}_j-Tr$$

(1)

where HDD [K∙Day] is the number of heating degree days, Tm is the mean outside temperature [K], Tr is the room temperature [K], j is the day of a heating season [-], and DHS is the duration of a heating season [day]. Room temperature for the examined building was set to 24 °C (297.15 K), while the monthly or seasonal HDD did not include the days with an average daily temperature higher than 12 °C (285.15 K). In addition to HDD, GPT was provided with the number of building monthly visits for two consecutive heating seasons. The task assigned to the prompt was as follows:

Q2: 

Having in mind D1, assess the monthly heat consumption of the same building by adding the influence of monthly visits of the building users (children), and the number of heating degree days (HDD). The number of visits nv. How much heat building will consume that month?

Appendix A and Appendix B contain the hdd and nv values used for each month of the studied period for the buildings analyzed.

2.2. GPT-3.5 Inductive Reasoning Test

In addition to Section 2.1, the study performed an inductive reasoning test to evaluate GPT’s usability in energy management tasks with insufficient building details. In this regard, a second set of buildings was used. The GPT was instructed by contextual template-based prompting to answer the questions concerning each of the buildings individually. The order and content (italic text) of the instructions were as follows:

D2: 

The public kindergarten is located in: li (Table 4). It was built in: ki (Table 4). The details of the building envelope are k1, …, k7 (all inputs were entered together with their units available in Table 4). The building was renovated in the year j6, and considered following improvements of the thermal envelope: j1, …, j5.

Answer the following questions by relying on inductive reasoning:

Q3: 

How much specific heat [kWh/m²/a] did the building consume before renovation?

Q4: 

How much specific heat [kWh/m²/a] does the building consume upon renovation?

3. Results and Discussion

The responses GPT provided to the instructions are presented visually to make them easier to interpret. To measure the accuracy of the assessments, the study used two accuracy indicators: mean absolute error (MAE) [59] (Equation (2)) and mean absolute percentage error (MAPE) [60] (Equation (3)). In addition to MAPE, the study used the coefficient of determination (Equation (4)) [61] to compare the GPT assessments made in this study with the assessments from another study.

$$MAE={\displaystyle \frac{{\sum }_{i=1}^n\left|{(y}_i-{\widehat{y}}_l)\right|}{n}}$$

(2)

$$MAPE={\displaystyle \frac{{\sum }_{i=1}^n\frac{\left|(y_i-{\widehat{y}}_l)\right|}{y_i}·100\%}{n}}$$

(3)

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{({\widehat{y}}_l-\overline{y})}^2}{{\sum }_{i=1}^n{{(y}_i-{\widehat{y}}_l)}^2}}$$

(4)

where n is the number instances (sample size), y_i the true value of the instance, ${\widehat{\mathit{y}}}_{\mathit{l}}$ the assessed value of the instance, and $\overline{\mathit{y}}$ is the mean value of the sample.

3.1. GPT-3.5 Deductive Reasoning Test

GPT responses to the Q1 set of questions are presented in Figure 2. The actual SHCs for buildings are represented by bars, while the corresponding GPT assessments are represented by dots. The bar and dot colors represent three HDD scenarios. The units used are kWh/m²/a. As can be seen from the figure, the number of HDDs did not have a decisive influence on the buildings’ SHCs. This means that relatively small changes in HDD during the heating season (~200 K∙Day) do not necessarily follow seasonal changes in SHC. This could be explained by the fact that variable behavior of building occupants (as determined by the number of monthly visits and activities within the building) has a greater influence on SHC than relatively minor changes in HDD. On the other hand, the order of the GPT-assessed SHCs mainly followed the order of the heating seasons’ HDDs, thus neglecting the influence of occupant behavior. This is a shortcoming of deductive reasoning, which was solely based on the data instructors provided to the prompt. On the positive side of deductive reasoning, GPT presented a comprehensive approach by listing the approach segments as bullet points (listing the inputs and calculating the total heat demand for each building and the SHC of each building). The method was systematic and simple to follow. However, the formulas used in the calculation method were oversimplified and inaccurate. The heat consumption was calculated based on the heated floor area rather than the thermal envelope area. The formula did not contain units, but rather dimensional notations. The formula “Heat Demand (kWh/m²/a) = Heating Degree Days × Gross Heated Floor Area × U-values” used to calculate SHC was oversimplified and incorrect, both dimensionally and formally. In this context, GPT proved unable to replicate the accuracy of traditional calculations, even though the formal approach appeared systematic and logical.

When compared with the actual data, the GPT-assessed SHCs are mainly underestimated (two-thirds of the cases). The greatest underestimation in terms of MAPE was measured in the case of kn12: 469% (MAE: 88.9), and the greatest overestimation in the case of kn10: 60% (MAE: 188.4). Moreover, errors in predicting building SHC were higher when GPT underestimated the value (MAPE: 199%, MAE: 107.3) than when it overestimated it (MAPE: 37%, MAE: 117.4).

Figure 3 depicts the distribution and accuracy of the buildings’ actual and GPT-assessed SHCs across different consumption ranges. The x-axis represents the actual SHC, while the y-axis represents the GPT-assessed SHC. Each dot represents the SHC of a building over one heating season. In terms of SHC consumption ranges, the MAE indicators for scenarios with less than 150 kWh/m²/a and those between 150 and 400 kWh/m²/a were relatively similar (108.5 and 123.3, respectively). MAPE values for the two same-span categories were 97% and 52%, respectively. Regarding the SHCs greater than 400 kWh/m²/a, GPT overestimated all the consumptions by 17% on average. The overall coefficient of determination (R²) between real and GPT-assessed data was 0.38, with a MAPE of 67%. In this context, the most intuitive and least precise statistical model (simple linear regression (SLR)) developed on the same set of buildings [47] outperformed GPT by around 55% in terms of R² and 51% in terms of MAPE. Moreover, SLR required only the HDD and building heated floor areas to provide estimations, whereas the LLM was given five times as many inputs. This performance was significantly lower than the performance of more advanced predictive algorithms developed for the same building sample (multiple linear regression (R²: 0.88; MAPE: 31%), Decision Tree (R²: 0.84; MAPE: 25%), and Evolutionary assembled artificial neural network (R²: 0.92; MAPE: 14%))

To investigate GPT’s ability to use occupancy as a factor affecting SHC, this study examined the cases of buildings where the LLM previously assessed the SHC with (1) highest (kn7: MAPE = 13%, MAE = 76.9) and (2) lowest accuracy (kn10). Although most predictive models dealing with energy management in public buildings neglect occupancy as a factor affecting heat consumption, there is no doubt this feature influences the SHC. By Q2, a comparison of two buildings’ real and GPT-assessed heat consumption is presented in Figure 4 (due to data availability and data filtering, Figure 4a,b do not represent the same consecutive heating seasons). The bottom axis of both graphs represents the month to which the measurements (SHC, number of visits) relate, while the upper axis shows the number of HDDs for each corresponding month. The data for kn10 and kn7 are available in Appendix A and Appendix B, respectively. The blue dots in the graph indicate the real SHC of kindergartens, while the green dots are GPT-assessed SHC. SHC values are shown on the left y-axis, while the number of monthly visits, (represented by red crosses on the graph), is indicated on the right y-axis.

Variations in GPT-assessed heat consumption (HC) relatively fairly followed the variations in the real data. The coefficient of determination between real and LLM-assessed values was the same (0.59), although the assessed values provided a much better fit in the case of kn7 than in the case of kn10, with just two dots being out of the ground truth pattern. As for MAPE, the average error of the GPT estimates for kn10 was 67% (MAE: 39,067), while for kn7 it was 27% (MAE: 730). This suggests that LLM algorithms can reasonably predict the influence of occupancy on HC, but only in kindergartens where they have previously proven to be reliable at predicting SHC. To respond to Q1 and Q2, GPT applied formulas, explaining them step by step. The approach was not entirely correct, nor were the formulas used. In this sense, some of the formulas were dubious and incomplete. Because of this, GPT proved unsuitable for comparison with engineering students. This contradicts the findings of papers dealing with the interpretation of theoretical knowledge such as medicine [31,39] and law [41].

3.2. GPT-3.5 Inductive Reasoning Test

The GPT responses to the Q3 set of questions are presented in Figure 5. Figure 5a compares the actual and GPT-assessed SHCs using side-by-side comparable bars, with the actual SHC shown in red and the GPT-assessed SHC in green. Similarly, Figure 5b shows the actual and GPT-assessed savings in SHC. Due to the relatively weak data describing the building and the actions taken, LLM was unable to provide any details before being instructed to rely on inductive reasoning. After this instruction, it began to assume the missing data and the expected energy savings. It was interesting to see that the assumptions were relatively good and in line with practice. When evaluating the building HC before renovation (Figure 5a), LLM overestimated the value by 7% (in the case of the building in Vejtofen, Denmark) and underestimated it by 5% (in the case of the building in Wolgast, Germany). When comparing the energy savings achieved after the renovation (Figure 5b), the errors were higher, between 10% and 40%, when compared with the actual SHC improvements.

For the buildings in Graz (Austria) and Tver (Russia), SHC consumption before renovation was not reported in the source literature. However, according to the information provided (Table 4), values were assumed against which energy savings were evaluated. In the case of the kindergarten in Graz, the savings were underestimated by 17%, while in the case of the kindergarten in Tver, they were overestimated by 15% (

Table 6. Comparison of buildings’ real and GPT-assessed SHCs and buildings’ real and GPT-assessed energy savings.

	Vejtofen (Denmark)	Wolgast (Germany)	Graz (Austria)	Tver (Russia)
Real SHC [kWh/m2/a]	167.4	158	Not stated	Not stated
GPT-assessed SHC [kWh/m2/a]	180	150	150	200
Real SHC savings [%]	49%	23%	70%	40%
GPT-assessed SHC savings [%]	55%	53%	53%	55%

).

Assessments based on LLM inductive reasoning were relatively fair, particularly those dealing with SHCs before renovation. This is particularly interesting given the weak data input (Table 4).

3.3. Study Contributions and Directions for Future Research

This was the first study in the field of energy management of public buildings to provide a comprehensive analysis of the applicability and reliability of GPT in real-life scenarios. In addition to the provided results, the study could guide future research by indicating what positive outcomes to expect and what advances to look for. By increasing community evaluation of LLM usability, studies like this contribute to the knowledge base that can provide valuable feedback for future advancements in LLM reasoning.

Future research will assess the reliability of LLM recommendations in shaping decisions related to building renovations. The research will compare the usability of competing technologies in the field. This study will investigate the variety of LLMs’ inductive reasoning abilities, emphasizing a thorough analysis of their strengths and limitations. This would encompass assessing the GPT capability to differentiate between construction periods, understand legislation governing building energy efficiency, and recognize changes in building envelope characteristics over time.

4. Conclusions

This study examined the viability of employing GPT as an expert adviser in the field of energy management of kindergartens. The research was conducted on two groups of buildings: (a) 12 public kindergartens in the city of Kragujevac (Serbia) and (b) 4 kindergartens in different cities in Europe. The first group of buildings provided a comprehensive set of data dealing with building physics that facilitated the evaluation of GPT’s deductive reasoning potential. The second group of buildings was poorly described, and therefore was used to test GPT’s inductive reasoning potential. Concerning deductive reasoning, GPT was tasked to assess the buildings’ SHC [kWh/m²/a]. The response was relatively inaccurate, with an average MAPE of 67%. This outcome can be considered unsatisfactory, especially considering that a simple linear regression, using a single input, outperformed GPT on the same dataset [47]. When dealing with deductive reasoning in assessing the kindergartens’ SHCs, GPT proved incapable of performing correct calculations and providing satisfactory accuracy of scores. This aligns with Borji’s findings [36], which identified earlier versions of GPT as incapable of math and arithmetic skills. Hence, the success of LLM in this sort of energy management task cannot be compared with that in medicine, where GPT provides the knowledge of a student [31] or even an expert [34]. When dealing with the estimates of monthly heat demand considering the occupancy as an influential factor, LLM proves a promising technology. The average MAPE on this task was 48%. In terms of inductive reasoning, the LLM bot was instructed to assess the building’s HC and energy savings by following the renovation procedure. When dealing with missing details in this context, GPT assumptions were in line with practice. As a result, SHC assessments for two of the buildings analyzed indicated MAPE between just 5% and 7%, while energy savings were estimated with poorer performance (15% and 17% error). After analysis, GPT deductive tasks can be considered to be ineligible as an adviser in the field of energy management of kindergartens. This conclusion is based on GPT’s weak and unreliable mathematical approach rather than the accuracy of its assessment. Moreover, made-up formulas and false explanations can lead non-experts to make wrong decisions. In the case of inductive reasoning, the technology shows promising potential in augmenting non-experts. Unlike similar studies examining the usability of GPT assessments in other domains, the energy management domain analyzed in this study did not encounter challenges related to the need for real-time internet data (as in [19]), privacy and data security (as in [20]), political bias (as in [25]), or issues of equity and fairness (as in [27]). Therefore, continued advancements in LLM technology could pave the way for practical applications of GPT in addressing energy management challenges in kindergartens.