Energy Consumption Prediction in Residential Buildings—An Accurate and Interpretable Machine Learning Approach Combining Fuzzy Systems with Evolutionary Optimization

Abstract

This paper addresses the problem of accurate and interpretable prediction of energy consumption in residential buildings. The solution that we propose in this work employs the knowledge discovery machine learning approach combining fuzzy systems with evolutionary optimization. The contribution of this work is twofold, including both methodology and experimental investigations. As far as methodological contribution is concerned, in this paper, we present an original designing procedure of fuzzy rule-based prediction systems (FRBPSs) for accurate and transparent energy consumption prediction in residential buildings. The proposed FRBPSs are characterized by a genetically optimized accuracy–interpretability trade-off. The trade-off optimization is carried out by means of multi-objective evolutionary optimization algorithms—in particular, by our generalization of the well-known strength Pareto evolutionary algorithm 2 (SPEA2). The proposed FRBPSs’ designing procedure is our original extension and generalization (for regression problems operating on continuous outputs) of an approach to designing fuzzy rule-based classifiers (FRBCs) we developed earlier and published in 2020 in this journal. FRBCs operate on discrete outputs, i.e., class labels. The experimental contribution of this work includes designing the collection of FRBPSs for residential building energy consumption prediction using the data set published in 2024 and available from Kaggle Database Repository. Moreover, the comparison with 20 available alternative approaches is carried out, demonstrating that our approach significantly outperforms alternative methods in terms of interpretability and transparency of the energy consumption predictions made while remaining comparable or slightly superior in terms of the accuracy of those predictions.

1. Introduction

Effective approaches allowing to model the electric energy use and to predict the level of energy consumption in residential buildings are very useful for various reasons related, in general, to building energy performance analysis including, e.g., detecting abnormal energy use patterns [1], determining correct photovoltaics sizing [2], supporting energy management systems [3], modeling predictive control applications using the loads [4], etc.

Significant amounts of data describing various aspects of building energy consumption and their prediction are available now—see, e.g., the most recently published data set [5] that is used in our experiment presented later in this work. It is obvious that a meaningful knowledge on building energy consumption is buried within such data sets. Thus, the development of effective tools for an automatic discovery of such knowledge in the available data sets has a strong rationale; moreover, these data sets constitute a good platform for the application of various machine learning (ML) approaches for performing knowledge discovery tasks. ML was initially defined in 1959 by A. L. Samuel as a “field of study that gives computers the ability to learn without being explicitly programmed” [6]. Presently, ML—partially overlapping the notion of data mining (DM) introduced in the 1990s—also addresses the earlier-mentioned tasks of knowledge discovery in data including an automatic revealing (through learning from data) of valid and understandable decision mechanisms, trends, and patterns hidden in the considered data sets.

Many ML methods have been applied to prediction of energy consumption. A brief review of them is carried out in the next section of this work. Unfortunately, their essential drawback is their non-transparent, non- or hardly interpretable, black-box-type nature that focuses almost exclusively on the accuracy of predictions and does not provide any deeper (or any) explanation, justification, or insight into mechanisms governing predictions they generate. This work is an attempt to address the above-outlined problem by providing an original knowledge-based ML prediction system coming from the field of computational intelligence (CI). CI—a modern extension of traditional artificial intelligence systems—covers “the theory, design, application, and development of biologically and linguistically motivated computational paradigms” [7]. The proposed solution is characterized both by high accuracy and by high interpretability and transparency. To be more specific, it is characterized by genetically optimized accuracy–interpretability trade-off of generated predictions.

As far as the representation of the knowledge learned from data is concerned, among the most transparent and useful structures are conditional IF-THEN-type rules and, in particular, linguistic fuzzy conditional rules [8] due to their high readability, high modularity, and easy-to-grasp interpretation and comprehension by humans. On the other hand, however, methods generating excessive numbers of complex (with many antecedents) rules significantly limit the transparency of the rule-based systems obtained. Therefore, both the accuracy and the interpretability and transparency should be the main performance indices in designing (fuzzy) rule-based systems from data. Considering this, in this paper, we present an original approach to designing fuzzy rule-based prediction systems (FRBPSs). This approach uses separate measures of system accuracy and system interpretability as optimization objectives in the process of building an FRBPS from data. Both optimization objectives have a complementary/contradictory nature. For this reason, a multi-objective evolutionary optimization algorithm (MOEOA) is employed in the process of the FRBPS’s structure and parameter optimization. Such process is equivalent to the FRBPS’s accuracy–interpretability trade-off optimization (see [9] for a review of related work).

The contribution provided by this work is twofold including both methodology and experimental investigations. As far as methodological contribution is concerned, in this paper, we present an original designing procedure of fuzzy rule-based prediction systems (FRBPSs) for accurate and transparent energy consumption prediction in residential buildings. The proposed FRBPSs are characterized by a genetically optimized accuracy–interpretability trade-off. Trade-off optimization is carried out by means of multi-objective evolutionary optimization algorithms. The proposed FRBPSs’ designing procedure is our original extension and generalization (for regression problems) of an approach to designing fuzzy rule-based classifiers (FRBCs) we developed earlier and recently published in this journal [10]. The generalization proposed here in the form of FRBPSs operates on continuous outputs (typical for regression tasks), whereas FRBCs of [10] process only discrete outputs, i.e., class labels (typical for classification tasks). Therefore, from the methodological point of view, the present work can be treated as a continuation and earlier-mentioned generalization of [10], although with different areas of applications ([10] for smart-grid stability prediction and this work for building energy consumption prediction). The experimental contribution of this work includes designing the collection of FRBPSs for residential building energy consumption prediction using the most recently published data set [5]. Moreover, the comparison with as many as 20 available alternative approaches is carried out to demonstrate the advantages and possible drawbacks of the proposed solution.

The remaining part of this work is organized in the following way: First, a review of related work on the application of different ML methods to prediction of energy consumption is presented. Next, the components of the proposed FRBPS design methodology from data through MOEOA-based learning and optimization are briefly presented. In turn, the aforementioned data set used in the reported experiments is characterized. Then, the designing of accurate and interpretable FRBPSs for building energy consumption based on the aforementioned data set as well as a comparative analysis with 20 available alternative methods are carried out and discussed.

2. Related Work

In this paper, we propose an FRBPS (an ML approach) based on multi-objective evolutionary optimization technique and its application to the prediction of energy consumption. For this reason, in this section, we briefly review several alternative ML approaches as well as some multi-objective optimization methods applied to the considered problem. We consider six distinct groups of ML approaches including (1) adaptive neuro-fuzzy inference systems, (2) various types of artificial neural networks, (3) support vector machines/regressions, (4) decision trees and random forests, (5) ensemble approaches, and (6) hybrid approaches. For each of the considered ML types, we briefly review three representative (in our opinion) works; only for a very broad group of artificial neural networks, we review seven works. We also briefly review three representative (in our opinion) papers on the optimization of predictive models for electricity consumption forecasting using multi-objective optimization algorithms. Moreover, we concentrate on the most recently published works, i.e., the works (except for three papers) published since 2021. Reviews of earlier-published papers can be found in [11,12,13].

Adaptive neuro-fuzzy inference systems (ANFISs): As far as ANFISs are concerned, in [14], they are applied to predict electricity consumption of public buildings based on weather conditions and human activities. A set of data samples representing the relationships between weather conditions (seven continuous input attributes), human activities (two categorical input attributes), and electricity consumptions of the library of the Providence University in Taichung, Taiwan (continuous output attribute) and registered between December 2017 and February 2019 (one sample per day) is considered as the benchmark problem. In turn, paper [15] proposes the application of an ANFIS to predict the cooling load in the sector of residential, energy efficiently built buildings. A total of 768 building cases described by eight continuous input attributes (the properties of buildings like overall height, relative compactness, wall area, surface area, roof area, orientation, glazing area, and glazing area distribution) and one output attribute (the cooling load/demand) is considered. In [16], an ANFIS is applied to energy consumption prediction in multi-storey residential buildings. This time, the experimental data set contains measurements of electricity consumed per hour and recorded from January 2010 to December 2010 (8760 measurement samples for 365 days and 24 h per day with four input attributes representing a point in time of measurement: hour of the day, day of the week, day of the month, and month).

Artificial neural networks (ANNs): As already mentioned, various types of ANNs are used in energy consumption prediction tasks. Paper [17] addresses the problem of residential electricity demand drivers in Cameroon using a prediction model based on an ANN. A sample of 1013 questionnaires describing household electricity consumption surveys is used as the empirical context for the application. A single questionnaire contains responses on a total of 40 predictive factors (input attributes) organized into five groups: building factors (number of rooms, rural or urban area, etc.), sociodemographic factors (head of household age, household size, etc.), appliances and lighting factors (number of lamps, the use frequency of fan/air conditioner, etc.), weather factor (average thermal amplitude is only used), and behaviors and attitudes of electricity use (e.g., awareness about energy savings practice). In turn, paper [18] presents an approach to predict electricity consumption for residential buildings where highly irregular human behavior plays a significant role. A Gaussian mixture clustering to detect behavior clusters and an extreme gradient boosting model for classification problems to predict the behavior patterns are combined with an ANN to predict electricity consumption by over 500 residential users placed in a southeastern region of Spain. Input attributes used for prediction cover daily electricity consumption statistics (e.g., mean consumptions during the morning, night, afternoon, and evening), weather statistics (mean, max., and min. temperatures), and calendar data (weekday or holiday and season: summer, winter, or midseason merging autumn and spring). As far as extreme learning machines (ELMs) and wavelet neural networks (WNNs) are concerned, in [19], a multilayer ELM for the prediction of annual building energy consumption is proposed. The approach fuses stacks of autoencoders with an ELM. The data set consists of 5000 residential buildings in the UK described by meteorological metadata and building metadata. In turn, in [20], a type-2 fuzzy WNN is proposed for modeling the energy consumption prediction in residential buildings. The system implementing type-2 fuzzy reasoning using WNN technology is used for the prediction of energy consumption in residential buildings of Northern Cyprus. In [21], a broad survey of many applications of ANNs can be found, including feedforward ANNs (e.g., multilayer perceptrons, radial basis function networks, ELMs, WNNs), recurrent ANNs (e.g., Elman neural networks, long short-term memories, echo state networks, restricted Boltzmann machines), and convolutional ANNs. In turn, paper [22] presents the application of deep neural networks (DNNs) to predict annual building energy consumption with particular emphasis on the effect of building clusters upon the prediction performance. A data set of residential buildings in the UK (5000 cases collected from the Ministry of Housing Communities and Local Government repository) is applied in comparative analysis of DNNs with several alternative ML techniques (see [22] for details). A total of 22 input attributes covering building metadata (e.g., postcodes, floor levels, roof and walls descriptions, number of rooms, window types, etc.), and meteorological data (temperature, wind speed, and pressure taken from Meteostat repository) are considered as predictive factors. In [23], a data collection of 100 buildings described by a total of 20 input attributes including weather conditions (temperature, humidity, wind speed, solar radiation), building properties (number of floors, building area, orientation, window-to-wall ratio, etc.), and environment features (indoor temperature, occupancy, fan and pump efficiency, etc.) is considered as a benchmark problem. An approach based on the rough set theory is used to reduce the dimensionality of data (to select important input attributes), and then an DNN is applied to predict energy consumption in the building. In [11], an overview of selected DNN approaches to forecasting energy consumption in buildings published since 2015 can be found.

Support vector machines/regressions (SVMs/SVRs): As far as SVMs/SVRs are concerned, in [24], the impact of eight input factors (i.e., surface, wall, roof and glazing areas, relative compactness, overall height, orientation, and glazing area distribution) on two output variables (heating and cooling loads) for residential buildings is explored using the SVR-based approach supported by several meta-heuristic optimization algorithms. Paper [25] also applies an SVR combined with some meta-heuristic algorithms to predict heating energy consumption in residential buildings. The prediction is based on the building architectural characteristics and also outdoor temperature (eight input variables are considered). In [26], several prediction modeling approaches (including SVRs) are applied to estimate energy use intensity for US commercial office buildings as well as the individual energy end-uses of HVAC (heating, ventilation, air conditioning) systems, plug loads, and lighting, based upon the Commercial Building Energy Consumption Survey (CBECS) 2012 microdata. The CBECS administered by the US Department of Energy belongs to the most comprehensive publicly available data sets for energy use in commercial buildings in the United States. A sample subset of 1024 office buildings described by 19 continuous and 56 categorical attributes is selected from the original CBECS data and used in the experiments.

Decision trees (DTs) and random forests (RFs): DTs and RFs constitute another class of ML approaches applied to energy consumption prediction problems. In [27], nine ML approaches, including DTs, are compared for energy performance assessment at the stage of residential buildings design. The data set contains 4500 various types of buildings (house, flat, bungalow, and maisonette) collected from the UK Ministry of Housing Communities and Local Government repository. The data set consists of only input attributes (a total of 23 attributes are included) that can be detected and modified at the early design stage (e.g., glazed area, floor description, windows energy efficiency, number of heated rooms) as well as three weather conditions (temperature, wind speed, air pressure). In turn, paper [28] presents the design and implementation of a data-driven building energy benchmarking (BEEM) system for buildings in Singapore. An ensemble tree algorithm for accurately modeling building energy use as well as for identifying the most influential factors is proposed in [28]. The energy disclosure data set (for 1145 buildings) was released by the Building and Construction Authority for the year 2017. A total of 618 buildings with 10 properties as input attributes of the prediction system (e.g., age of building, occupancy, type of air conditioning system, number of rooms) are considered in the experiment. In [29], a particle swarm-optimized RF classification algorithm is proposed to identify the most important factors that contribute to residential heating energy consumption. A data set from smart meters in residential buildings in Tomsk city in Russia is used to test the approach (it contains a total of 18 input attributes covering sensor data, building attribute features like number of floors, wall material, year of construction, etc., as well as weather records).

Ensemble models: An ensemble approach combining long short-term memory and Kalman filter to predict the short-term energy consumption of multi-family residential buildings in South Korea is applied in [30]. The benchmark data set consists of electric consumption measurements for apartments in four residential multi-family (33 floors) buildings and recorded between January and December 2010. Data samples are expressed by 25 input attributes (24 hourly consumption readings and time stamp). In turn, paper [31] performs an overall analysis of energy consumption in smart homes by deploying various ML models including an DT-RF-XGBoost-based ensemble model proposed in [31] (XGBoost denotes eXtreme Gradient Boosting classification algorithm). Two data sets incorporating a total of 503,910 readings from smart meters (with a time span of 1 minute) with 32 features (as input attributes) of house appliances and weather conditions are considered.

Hybrid approaches: Three hybrid approaches are considered in [32], combining an ANN with an evolutionary algorithm (a heap-based optimizer, a multiverse optimizer, and a whale optimization algorithm are considered) to predict building energy consumption in the residential sector. In turn, [33] presents a hybrid approach combining a deep neural network, fuzzy logic, and wavelet transformation yielding the so-called deep neural network-based fuzzy wavelet, supported by particle swarm optimization and a gradient-based learning algorithm, to predict energy consumption in urban buildings of Iranian cities (Tehran, Mashhad, Ahvaz, and Urmia) in the time period from 2010 to 2021. Paper [34] proposes a hybrid neural network prediction model combining convolutional neural networks, a bidirectional gate recurrent unit, an attention mechanism, and residual connection. The energy consumption data of an office building in Guangzhou, China is considered as a case study. The data set includes samples of hourly registered measurements (from 00:00 on 1 September 2020, to 23:00 on 31 August 2021) and described by 10 input attributes (four time factors and two weather, human, and energy factors each).

Multi-objective optimization of electricity consumption predictive models: In [35], the authors indicate that their multi-objective approach can be used to optimize the selection of predictive models by considering objectives such as forecast accuracy, model complexity, and robustness to outliers. However, they do not specify what optimization criteria they use in their experiments. Paper [36] presents an adaptation of a single-objective gray wolf optimization to its bi-objective version (a prediction accuracy and the number of input attributes are considered as two optimization objectives). Finally, in [37], a multi-objective optimization algorithm is used to minimize two objectives, i.e., cooling and heating load prediction errors in buildings.

Concluding, the overwhelming majority of the above-reviewed studies presents the exclusively prediction accuracy-oriented techniques, i.e., black-box, non-transparent systems not providing any insight into the internal mechanisms governing the prediction of energy consumption. It is worth emphasizing that interpretability and transparency are included in the list of future directions of research on energy consumption prediction. For instance, according to the most recently published work [38], “Future research in energy forecasting should aim to enhance data preprocessing methods, develop more interpretable and less computationally demanding models, and increase the adaptability of forecasting techniques” (a quote from [38]). Our paper is an attempt to address that problem and to fill that knowledge gap.

3. Methodology: Designing Fuzzy Rule-Based Prediction Systems (FRBPSs) from Data Using Multi-Objective Evolutionary Optimization Algorithms (MOEOAs)

The proposed approach employs MOEOAs to develop design from data FRBPSs that are characterized by genetically optimized accuracy–interpretability trade-off. Such FRBPSs can provide both accurate and interpretable predictions in a given domain—in particular, in the building energy consumption domain. As we already mentioned in the Introduction of this work, the proposed FRBPS design procedure is our original generalization (for regression problems usually operating on continuous outputs) of the earlier-developed by us and recently published in this journal [10] fuzzy classifier (operating on discrete outputs, i.e., class labels typical in classification tasks). Therefore, in this section, we particularly concentrate on all newly introduced elements of the FRBPS designing procedure, briefly addressing the remaining ones. First, the FRBPS structure, attribute representation, knowledge base, and fuzzy approximate inference engine are presented. Then, FRBPS genetic learning and MOEOA-based optimization are outlined including the learning data set, objectives of the FRBPS’s evolutionary optimization, and the MOEOA used.

FRBPS’s structure and attribute representation: Consider a system (an FRBPS) with n input attributes $x_1,x_2,\dots ,x_n$ ($x_i\in X_i$, $i=1,2,\dots ,n$), including both numerical and categorical (qualitative) attributes. and an output attribute y ($y\in Y$).

We let $F\left(X_i\right)$, $i=1,2,\dots ,n$ and $F\left(Y\right)$ denote families of all fuzzy sets defined in universes $X_i$, $i=1,2,\dots ,n$, and Y, respectively. As far as numerical input attributes are concerned, each attribute $x_i\in X_i$, $i\in \{1,2,\dots ,n\}$ (see such attributes in

Table 1. Details of particular records of the data set used in our experiment.

No.		Attribute Name	Attribute Type	Attribute Domain	Attribute Description
1.	input attributes	Timestamp	date, time	[1 January 2022, 11 February 2022]	The chronological record of each data point, providing a time-based context.
2.		Temperature	numerical-real	[20, 30]	Randomly generated values representing ambient temperatures in degrees Celsius.
3.		Humidity	numerical-real	[30, 60]	Randomly generated values reflecting the humidity level as a percentage.
4.		SquareFootage	numerical-real	[1000, 2000]	Simulated values representing the size of the environment in square footage.
5.		Occupancy	numerical-integer	[0, 9]	Randomly generated integer values indicating the number of occupants.
6.		HVACUsage	categorical	On, Off	Categorical variable denoting the HVAC system’s operational state (‘On’ or ‘Off’).
7.		LightingUsage	categorical	On, Off	Categorical variable indicating the lighting system’s operational state (‘On’ or ‘Off’).
8.		RenewableEnergy	numerical-real	[0.01, 30]	Randomly generated values representing the contribution of renewable energy sources as a percentage.
9.		DayOfWeek	categorical	Monday, …, Sunday	Categorical variable indicating the day of the week.
10.		Holiday	categorical	Yes, No	Categorical variable denoting whether the day is a holiday (‘Yes’ or ‘No’).
11.	output attribute	Energy Consumption	numerical-real	[53.2, 99.2]	Output (target) attribute.

later in the paper) is characterized by $a_i$ fuzzy sets $A_{i{k}_i}\in F\left(X_i\right)$, $k_i=1,2,\dots ,a_i$. $A_{i1}$ denotes an S-type fuzzy set (representing linguistic term “Small”), $A_{i{a}_i}$ denotes an L-type set (representing linguistic term “Large”), and $A_{i2},A_{i3},\dots ,A_{i,a_i-1}$ denote M-type sets (representing linguistic terms “Medium 1”, “Medium 2”, …, “Medium $a_i-2$”). Similarly, a numerical output attribute (see such an attribute also in Table 1) is characterized by b fuzzy sets $B_{k_{out}}\in F\left(Y\right)$, $k_{out}=1,2,\dots ,b$. $B_1$ denotes an S-type fuzzy set, $B_b$—an L-type fuzzy set, and $B_2,B_3,\dots ,B_{b-1}$—M-type fuzzy sets (particular fuzzy sets represent appropriate linguistic terms as in the case of input attributes). For simplicity, $A_{i{k}_i}$s and $B_{k_{out}}$s also denote the corresponding linguistic terms. Membership functions of S-, M-, and L-type fuzzy sets used in our experiments are of the following form (variable v stands for any input attribute $x_i$, $i=1,2,\dots ,n$ and output attribute y; see Figure 1):

$${\mu }_S\left(v\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{for} v\le e_S,\hfill \\ f_G\left(\frac{v-e_S}{{\rho }_S}\right),\hfill & \mathrm{for} v>e_S,\hfill \end{array}\right.$$

(1)

$${\mu }_M\left(v\right)=\left\{\begin{array}{cc}{f}_G\left(\frac{v-d_M}{{\sigma }_M}\right),\hfill & \mathrm{for} v\le d_M,\hfill \\ 1,\hfill & \mathrm{for} d_M<v\le e_M,\hfill \\ f_G\left(\frac{v-e_M}{{\rho }_M}\right),\hfill & \mathrm{for} v>e_M,\hfill \end{array}\right.$$

(2)

$${\mu }_L\left(v\right)=\left\{\begin{array}{cc}{f}_G\left(\frac{v-d_L}{{\sigma }_L}\right)\hfill & \mathrm{for} v\le d_L,\hfill \\ 1,\hfill & \mathrm{for} v>d_L,\hfill \end{array}\right.$$

(3)

where $e_S<d_M\le e_M<d_L$, ${\rho }_S>0$, ${\sigma }_M>0$, ${\rho }_M>0$, ${\sigma }_L>0$, and $f_G\left(\tau \right)$ is a Gaussian-type function, i.e.,

$$f_G\left(\tau \right)=e^{-{\tau }^2}.$$

(4)

As mentioned earlier in this section, one S-type, one L-type, and usually several M-type fuzzy sets can be considered for a given attribute. We emphasize that a given linguistic term for a given input/output attribute is represented by the same fuzzy set in all fuzzy rules in which it occurs.

As far as categorical input attributes are concerned (see attributes of that type in Table 1 later in the paper), each attribute $x_i\in X_i=\{x_{i1},x_{i2},\dots ,x_{i{a}_i}\}$, $i\in \{1,2,\dots ,n\}$ is characterized by $a_i$ fuzzy singletons $A_{i{k}_i}=A_{i{k}_i}^{(singl.)}\in F\left(X_i\right)$, $k_i=1,2,\dots ,a_i$ defined for particular “values” $x_{i{k}_i}$ of $x_i$ as follows: ${\mu }_{A_{i{k}_i}^{(singl.)}}\left(x_i\right)=1$ for $x_i=x_{i{k}_i}$, and 0 elsewhere.

FRBPS knowledge base: It contains R genetically optimized fuzzy IF-THEN rules discovered during the evolutionary learning and optimization process presented later in this section. We propose the following form of the rth rule, $r=1,2,\dots ,R$ (the number of rules R changes during the learning process):

$$\begin{array}{c}\begin{array}{cc}\mathbf{IF}\hfill & {[{x_1 \mathrm{i}\mathrm{s} \left[\mathrm{n}\mathrm{o}\mathrm{t}\right] }_{(s{w}_1^{\left(r\right)}<0)} A_{1,|s{w}_1^{\left(r\right)}|}]}_{(s{w}_1^{\left(r\right)}\ne 0)}\mathbf{AND}\dots \hfill \\ & {[{x_i \mathrm{i}\mathrm{s} \left[\mathrm{not}\right] }_{(s{w}_i^{\left(r\right)}<0)} A_{i,|s{w}_i^{\left(r\right)}|}]}_{(s{w}_i^{\left(r\right)}\ne 0)}\mathbf{AND}\dots \hfill \\ & {[{x_n \mathrm{is} \left[\mathrm{not}\right] }_{(s{w}_n^{\left(r\right)}<0)} A_{n,|s{w}_n^{\left(r\right)}|}]}_{(s{w}_n^{\left(r\right)}\ne 0)}\hfill \end{array}\hfill \\ \mathbf{THEN} y \mathrm{is} B_{k_{out}^{\left(r\right)}},\hfill \end{array}$$

(5)

where

(i)

${\left[expression\right]}_{\left(condition\right)}$ in (5) represents conditional inclusion of $\left[expression\right]$ into a considered rule if and only if $\left(condition\right)$ is fulfilled,

(ii)

$|·|$ return the absolute value,

(iii)

$s{w}_i^{\left(r\right)}$, $i=1,2,\dots ,n$, $r=1,2,\dots ,R$ are switch-parameters (set and modified by an MOEOA) that control the presence/absence of the ith input attribute in the rth fuzzy rule. $s{w}_i^{\left(r\right)}\in \{0,\pm 1,\pm 2,\dots ,\pm a_i\}$, where $a_i$ is the earlier-defined number of fuzzy sets (and the corresponding linguistic terms) that represent the ith input attribute. $s{w}_i^{\left(r\right)}$ are defined as follows:

-

for $s{w}_i^{\left(r\right)}=0$, the ith input attribute is excluded from (not active in) the rth rule,

-

for $s{w}_i^{\left(r\right)}>0$, the component $\left[x_i \mathrm{is} A_{i{k}_i}\right]$ ($k_i=s{w}_i^{\left(r\right)}$) is included in the rth rule,

-

for $s{w}_i^{\left(r\right)}<0$, the component $\left[x_i \mathrm{is} \mathrm{not} A_{i{k}_i}\right]$ ($k_i=|s{w}_i^{\left(r\right)}|$) is included in the rth rule (not $A_{i{k}_i}={\overline{A}}_{i{k}_i}$ and ${\mu }_{{\overline{A}}_{i{k}_i}}\left(x_i\right)=1-{\mu }_{A_{i{k}_i}}\left(x_i\right)$; ${\mu }_{A_{i{k}_i}}\left(x_i\right)$ and ${\mu }_{{\overline{A}}_{i{k}_i}}\left(x_i\right)$ are membership functions of fuzzy sets $A_{i{k}_i}$ and ${\overline{A}}_{i{k}_i}$, respectively).

The FRBPS knowledge base is implemented using two separate modules, i.e., a rule-base-structure module $RBS$ and a rule-base-data module $RBD$. We introduce a simple, direct, and thus computationally efficient $RBS$ representation in the following form:

$$RBS={\{s{w}_1^{\left(r\right)},s{w}_2^{\left(r\right)},\dots ,s{w}_n^{\left(r\right)},k_{out}^{\left(r\right)}\}}_{r=1}^R.$$

(6)

In turn, the $RBD$ module contains tunable and non-tunable parts. The tunable part contains parameters of membership functions of fuzzy sets representing linguistic terms for particular numerical input and output attributes. These parameters are subject to tuning during FRBPS learning and MOEOA-based optimization. The considered parameters include (see Figure 1) $e_S$ and ${\rho }_S$ for S-type fuzzy sets, ${\sigma }_M$, $d_M$, $e_M$, and ${\rho }_M$ for M-type fuzzy sets as well as ${\sigma }_L$ and $d_L$ for L-type fuzzy sets. The non-tunable part of $RBD$ contains domains of categorical input attributes $X_i=\{x_{i1},x_{i2},\dots ,x_{i{a}_i}\}$, $i\in \{1,2,\dots ,n\}$.

We develop original crossover and mutation operators for the processing of the population of $RBS$s as well as some specialized crossover and mutation operators for the transformation of the population of $RBD$s; see, for details, the earlier-mentioned work [10] we recently published in this journal.

FRBPS’s approximate inference engine: During the genetic learning process, an evaluation of particular individuals (fuzzy rule bases) competing with each other in the framework of Pittsburgh-type genetic approach used, cf. [39,40], must be performed in each generation. For this reason, a fuzzy set theory-based representation of linguistic IF–THEN rules (5) must be defined and a fuzzy approximate inference engine must be employed. Various definitions of fuzzy implications (see, e.g., [41]) can be implemented in our approach. The most often applied is the so-called Mamdani’s model (see, e.g., [42] for details), which uses min-operation for combining many rule antecedents (input attributes) and rule antecedents with rule consequents (input and output attributes) as well as max-operation for rule aggregation. Employing the aforementioned most often used Mamdani’s approach, we obtain, for input numerical data ${\mathit{x}}^{\prime }=(x_1^{\prime },x_2^{\prime },\dots ,x_n^{\prime })$, an FRBPS fuzzy-set response $B^{\prime }\in F\left(Y\right)$ characterized by its membership function ${\mu }_{B^{\prime }}\left(y\right)$, $y\in Y$:

$$\begin{array}{cc}{\mu }_{B^{\prime }}\left(y\right)=\hfill & \underset{r=1,2,\dots ,R}{max}{\mu }_{B^{\prime \left(r\right)}}\left(y\right)=\hfill \\ & \underset{r=1,2,\dots ,R}{max}min[{\alpha }^{\left(r\right)},{\mu }_{B_{k_{out}^{\left(r\right)}}}\left(y\right)],\hfill \end{array}$$

(7)

where

$${\alpha }^{\left(r\right)}=\underset{\begin{array}{c}i=1,2,\dots ,n,\\ s{w}_i^{\left(r\right)}\ne 0\end{array}}{min}{\alpha }_i^{\left(r\right)},$$

(8)

and

$${\alpha }_i^{\left(r\right)}=\left\{\begin{array}{cc}{\mu }_{A_{i,s{w}_i^{\left(r\right)}}}\left(x_i^{\prime }\right),\hfill & \mathrm{for} s{w}_i^{\left(r\right)}>0,\hfill \\ {\mu }_{{\overline{A}}_{i,|s{w}_i^{\left(r\right)}|}}\left(x_i^{\prime }\right),\hfill & \mathrm{for} s{w}_i^{\left(r\right)}<0.\hfill \end{array}\right.$$

(9)

${\alpha }^{\left(r\right)}$ is the activation degree of the rth fuzzy rule by input numerical data ${\mathit{x}}^{\prime }$, whereas ${\alpha }_i^{\left(r\right)}$ for i such that $s{w}_i^{\left(r\right)}\ne 0$ is the activation degree of the ith input attribute in that rule.

If an FRBPS’s non-fuzzy response $y^{\prime }$ is required, it is calculated using a “half of the field” approach as follows:

$$y^{\prime }=arg\underset{y_0\in Z}{min}|\underset{-\infty }{\overset{y_0}{ʃ}}{\mu }_{B^{\prime }}\left(y\right)-\underset{y_0}{\overset{\infty }{ʃ}}{\mu }_{B^{\prime }}\left(y\right)|.$$

(10)

FRBPS’s learning data set: The proposed prediction system is designed during the genetic learning and MOEOA-based optimization process from learning data set L that contains K input–output samples:

$$\mathit{L}={\{{\mathit{x}}_k^{\left(lrn\right)},y_k^{\left(lrn\right)}\}}_{k=1}^K,$$

(11)

where ${\mathit{x}}_k^{\left(lrn\right)}=(x_{1k}^{\left(lrn\right)},x_{2k}^{\left(lrn\right)},\dots ,x_{nk}^{\left(lrn\right)})\in \mathit{X}=X_1\times X_2\times \cdots \times X_n$ (× stands for Cartesian product of ordinary sets) is the collection of input attributes and $y_k^{\left(lrn\right)}$ is the corresponding output attribute ($y_k^{\left(lrn\right)}\in Y$) for the kth data sample, $k=1,2,\dots ,K$.

FRBPS’s evolutionary optimization objectives: Two separate optimization objectives are employed, i.e., the accuracy and the interpretability of the system. The FRBPS accuracy measure (subject to maximization) is defined in the following way:

$$Q_{ACC}^{\left(lrn\right)}=1-\frac{1}{y_{max}-y_{min}}{Q}_{RMSE}^{\left(lrn\right)},$$

(12)

where

$$Q_{RMSE}^{\left(lrn\right)}=\sqrt{\frac{1}{K}\sum _{k=1}^K{(y_k^{\left(lrn\right)}-{y^{\prime }}_k)}^2}.$$

(13)

$Q_{ACC}^{\left(lrn\right)}\in [0,1]$, $[y_{min},y_{max}]$ is the range of values of $y_k^{\left(lrn\right)}$, ${y^{\prime }}_k$ is the system’s non-fuzzy response (10) to the kth learning data sample ${\mathit{x}}_k^{\left(lrn\right)}$, and $y_k^{\left(lrn\right)}$ is the desired response for that sample; see (11); $Q_{RMSE}^{\left(lrn\right)}$ is the root mean square error of our approach in the learning data set.

The FRBPS interpretability embraces two aspects: a complexity-related interpretability and a semantics-related interpretability. As far as the FRBPS’s complexity-related interpretability is concerned, its measure (subject to maximization) is defined as follows:

$$Q_{INT}=1-Q_{CPLX},$$

(14)

where

$$Q_{CPLX}=\frac{Q_{RINP}+Q_{INP}+Q_{FS}}{3},$$

(15)

and

$$\begin{array}{c}{Q}_{RINP}=\frac{1}{R}{\sum }_{r=1}^R\frac{n_{INP}^{\left(r\right)}-1}{n-1}, Q_{INP}=\frac{n_{INP}-1}{n-1}, Q_{FS}=\frac{n_{FS}-1}{{\sum }_{i=1}^n{a}_i-1}, n>1.\hfill \end{array}$$

(16)

The FRBPS complexity measure $Q_{CPLX}$ (15) ($Q_{CPLX}\in [0,1]$; 0 and 1 represent minimal and maximal complexities, respectively) is an average of three sub-measures that evaluate

(i)

an average complexity of particular rules $Q_{RINP}$ (16) ($n_{INP}^{\left(r\right)}$ in (16) is the number of active input attributes in the rth rule),

(ii)

the complexity of the whole system related to the number of its active inputs $Q_{INP}$ (16) ($n_{INP}$ in (16) denotes the number of active inputs in the whole system), and

(iii)

the complexity of the whole system related to the number of its active fuzzy sets $Q_{FS}$ (16) ($n_{FS}$ in (16) is the number of active fuzzy sets (linguistic terms) in the whole system).

The FRBPS semantics-related interpretability is implemented by imposing, as optimization constraints, the so-called strong fuzzy partitions (SFPs) [43] upon domains of all numerical input and output attributes. SFPs are fuzzy partitions in which the sum of the values of membership functions of all fuzzy sets (fuzzy partitions) for any domain value are roughly equal to 1; it means that neighboring fuzzy sets overlap each other on a roughly 0.5-level, which means a good coverage of the considered domain. Straightforward implementation of SFP requirements for the case of Gaussian-like membership functions can be formulated in the following way (see Figure 2 for illustration of the implementation of the SFP-requirements for the three-set SFP of the v domain):

$${\sigma }_{i{k}_i}={\rho }_{i,k_i-1}=d_{i{k}_i}-e_{i,k_i-1},k_i=2,3,\dots ,a_i$$

(17)

and

$$\begin{array}{c}{e}_{i1}<d_{i2}\le e_{i2}<\cdots \le d_{i,a_i-1}\le e_{i,a_i-1}<d_{i{a}_i}, i=1,2,\dots ,n.\end{array}$$

(18)

The FRBPS accuracy and interpretability measures used as evolutionary optimization objectives guide the optimization process carried out by an appropriate MOEOA. In our experiments reported in this work, we use our generalization of one of the well-known MOEOAs, i.e., the strength Pareto evolutionary algorithm 2 (SPEA2) [44]. That generalization, referred to as SPEA3, is introduced in [45] and briefly presented in the earlier-mentioned paper [10] published recently in this journal (see also [46]). The essence of our generalization of SPEA2 consists in changing the original mechanism responsible for the selection of non-dominated solutions during the optimization process. In our SPEA3, this mechanism is replaced by our original environmental selection procedure, which aims at maintaining the best possible spread (i.e., the longest possible distance between extreme solutions in the objective space), the best possible balance (i.e., keeping possibly equal distances between adjacent solutions in the objective space), and the high diversity of solutions occurring in Pareto-front approximation. As a result of that, the proposed SPEA3 is characterized by better performance than SPEA2. SPEA3 generates sets of solutions that are more accurate and characterized by higher spread and better balanced distribution in the objective space than alternative solutions generated by SPEA2. The details concerning the operation of our SPEA3 can be found in [45].

4. Energy Consumption Prediction Data Set Considered

As already mentioned in the introductory section of this paper, the most recently published data set, available (on 15 April 2024) from https://www.kaggle.com/datasets/mrsimple07/energy-consumption-prediction/data [5] and describing various aspects of building energy consumption and its prediction, is considered in our experiments. It is designed to emulate real-world scenarios related to energy usage encapsulating the influence of a diversified list of attributes including temperature, humidity, HVAC (heating, ventilation, air conditioning), and lighting usage, renewable energy contribution, occupancy, square footage, etc., of residential buildings upon energy consumption (see [5] for comments). The considered data set contains $1.000$ records (instances) without missing values. Each record is characterized by 10 input attributes and 1 output (target) attribute. Input attributes include five numerical ones (four real and one integer), four categorical (qualitative) ones and the timestamp; the output attribute (to be predicted), i.e., the energy-consumption level, is numerical (real) one; see Table 1 for details. According to some psychological findings (see [43] and particularly [47]), “the number of conditions in the antecedent of a rule must not exceed the limit of $7\pm 2$ distinct conditions, which is the number of conceptual entities a human being can handle” (quotation from [43]). Extending that principle for the number of linguistic terms (fuzzy sets) for numerical attributes as well as having in mind a satisfactory performance of the prediction system obtained, we use seven linguistic terms (fuzzy sets) for each numerical attribute; initial shapes of membership functions of those fuzzy sets are shown in Figure 3. A bigger number of fuzzy sets out of which our approach selects the most appropriate ones for a given fuzzy rule (see the next section) improves the overall performance of the prediction system (the rule precision, accuracy, and transparency).

5. Experiments (Application of Our Approach to Prediction of Building Energy Consumption) and Comparative Analysis

This section presents the results of several experiments (for the data set characterized in the previous section of this work) regarding the application of the proposed approach to designing from the considered data an accurate and interpretable FRBPS for building energy consumption. A comparative analysis with as many as 20 available alternative approaches (their software implementations are available from [48]) is also carried out.

The main part of experiment evaluating our approach consists in performing its cross-validation-based verification. However, before the cross-validation-based experiments are performed and presented, we demonstrate some details of the operation of our approach during the process of the FRBPS designing from data. For this reason, the SPEA3-MOEOA-based genetic learning and optimization experiment for a single learning test data split is presented and discussed. The split ratio of the original data, according to comments of [5], is 660:340. Figure 4 presents a flowchart showing sequences of activities implementing our methodology for both groups of experiments. These activities include, in general, the partitioning of the original data set into the learning and test data sets, the genetic learning and optimization process, and the calculation of the accuracy and interpretability measures of the solutions (FRBPSs) obtained.

Figure 5 presents a 10-element collection (front) of non-dominated solutions (optimized FRBPSs) generated in a single run of our FRBPS designing technique. Each solution is characterized in Figure 5 by its complexity measure $Q_{CPLX}$ (15) as well as its accuracy measures, $Q_{RMSE}^{\left(lrn\right)}$ (13) in the learning data set and $Q_{RMSE}^{\left(tst\right)}$ in the test data set ($Q_{RMSE}^{\left(tst\right)}$ is calculated for the test data in an analogous way to $Q_{RMSE}^{\left(lrn\right)}$ for the learning data). The collection of Figure 5 represents the best available approximations of Pareto-optimal solutions generated by our approach. The considered collection of solution begins with solution No. 1 (of highest interpretability and lowest accuracy) and ends with the solution No. 10 (of lowest interpretability and highest accuracy in the learning data set). Particular solutions from the considered collection are characterized by different levels of optimized accuracy–interpretability trade-off. Therefore, the user can select a single solution (a specific FRBPS represented by its knowledge base) characterized by a desired level of trade-off between its accuracy and interpretability.

In principle, the user’s choice can be made in any way, starting with the FRBPS of highest interpretability (but accepting its typically low accuracy) and ending with the FRBPS of highest overall accuracy in learning and test data sets (but accepting its generally worse interpretability); see such extreme solutions No. 1 and No. 10, respectively, in Figure 5. However, it seems that the FRBR characterized by the highest ability to generalize knowledge, i.e., the FRBPS of the highest accuracy in the test data set (data not seen during the learning process), is the best choice in the first step of the FRBPS selection procedure; see such solution No. 6 in Figure 5. If there are more than one such FRBS (i.e., characterized by the same and highest test-data accuracy), the FRBPS of highest interpretability is chosen out of the earlier-selected FRBPSs—it is the second step of the FRBPS selection procedure. It does not occur in our case since in the first step we selected only one system—the earlier-mentioned solution No. 6.

The numerical details of accuracy and interpretability of all solutions (FRBPSs) from Figure 5 are collected in

Table 2. The accuracy and interpretability measures of solutions from Figure 5.

No.	Objective Function Complements			MAPE-Based Accuracy Measures		Interpretability Measures
	${\mathit{Q}}_{\mathit{CPLX}}$	${\mathit{Q}}_{\mathit{RMSE}}^{\mathbf{\left(}\mathit{lrn}\mathbf{\right)}}$	${\mathit{Q}}_{\mathit{RMSE}}^{\mathbf{\left(}\mathit{tst}\mathbf{\right)}}$	${\mathit{Q}}_{\mathit{MAPE}}^{\mathbf{\left(}\mathit{lrn}\mathbf{\right)}}$	${\mathit{Q}}_{\mathit{MAPE}}^{\mathbf{\left(}\mathit{tst}\mathbf{\right)}}$	$\mathit{R}$	${\mathit{n}}_{\mathit{INP}}$	${\mathit{n}}_{\mathit{FS}}$	${\mathit{n}}_{\mathit{INP}\mathbf{/}\mathit{R}}$
1.	0.0062	5.841	5.973	6.25%	6.38%	1	1	2	1
2.	0.0810	5.706	5.637	6.14%	6.12%	2	2	4	1.5
3.	0.0802	5.454	5.551	5.95%	6.09%	3	2	5	1.3
4.	0.1219	5.269	5.498	5.50%	6.06%	3	3	5	1.3
5.	0.2203	5.142	5.494	5.31%	6.04%	4	4	8	2.2
6.	0.3117	5.073	5.459	5.24%	5.96%	5	5	11	3.2
7.	0.4021	4.956	5.520	5.19%	6.02%	7	6	16	3.4
8.	0.4942	4.875	5.600	5.08%	6.10%	8	7	22	3.7
9.	0.6768	4.791	5.592	4.97%	6.06%	10	9	31	4.8
10.	0.8498	4.624	5.718	4.84%	6.14%	21	9	49	6.3

, in which $n_{INP/R}$ is the averaged number of input attributes per rule and $Q_{RMSE}^{\left(tst\right)}$ is the root mean square error (RMSE) of our approach in the test data set. It is calculated in an analogous way to $Q_{RMSE}^{\left(lrn\right)}$ (13). Bearing in mind that our approach is also an interpretability-oriented one, in Table 2, we also include the MAPE (Mean Absolute Percentage Error) accuracy measures for all FRBPSs from Figure 5. The MAPE measure is scale-independent and easier to interpret in comparison with RMSE. The MAPE measure expresses the system response error as a percentage of the desired output response. For the learning data set, it is defined as follows (for the test data set—in an analogous way):

$$Q_{MAPE}^{\left(lrn\right)}=100\%·\frac{1}{K}\sum _{k=1}^K\left|\frac{y_k^{\left(lrn\right)}-{y^{\prime }}_k}{y_k^{\left(lrn\right)}}\right|;$$

(19)

$y_k^{\left(lrn\right)}$ and ${y^{\prime }}_k$ are the same as in Formula (13). It should be emphasized that MAPE cannot replace RMSE as the optimization objective. MAPE generates undefined errors for $y_k=0$ and relatively large errors for small $y_k$. The RMSE measure does not have these drawbacks and is universal (it can be used in both classification and regression problems). The remaining parameters in Table 2 are defined in Section 3 of the paper.

Solution No. 6 of Figure 5 and Table 2 characterized by the highest test data accuracy and high interpretability (only five rules with a 3.2 averaged number of input attributes per rule) is of particular interest (see the cross-validation experiment later in this section). Linguistic fuzzy rule bases of solutions (FRBPSs) from Figure 5 and Table 2 are presented in

Table 3. Fuzzy rule bases for solutions (FRBPSs) Nos. 1–9 from Figure 5 and Table 2.

No.	Fuzzy Prediction Rules
Solution No. 1:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
Solution No. 2:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND HVACUsage is On THEN EnergyConsumption is Medium4
Solution No. 3:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND HVACUsage is On THEN EnergyConsumption is Medium4
3.	IF	Temperature is Medium1 THEN EnergyConsumption is Medium2
Solution No. 4:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND HVACUsage is On THEN EnergyConsumption is Medium4
3.	IF	Occupancy is Medium5 THEN EnergyConsumption is Medium4
Solution No. 5:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND HVACUsage is On THEN EnergyConsumption is Medium4
3.	IF	Temperature is Small AND Occupancy is Medium5 THEN EnergyConsumption is Medium4
4.	IF	Temperature is Medium2 AND Occupancy is Small AND HVACUsage is Off AND LightingUsage is Off THEN EnergyConsumption is Medium2
Solution No. 6:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND HVACUsage is On THEN EnergyConsumption is Medium4
3.	IF	Temperature is Small AND Occupancy is Medium5 THEN EnergyConsumption is Medium4
4.	IF	Temperature is Medium2 AND Occupancy is Small AND HVACUsage is Off AND LightingUsage is Off THEN EnergyConsumption is Medium2
5.	IF	Occupancy is Medium1 AND HVACUsage is Off AND LightingUsage is On AND RenewableEnergy is Small THEN EnergyConsumption is Medium2
Solution No. 7:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND HVACUsage is On THEN EnergyConsumption is Medium4
3.	IF	Temperature is Small AND Occupancy is Medium5 THEN EnergyConsumption is Medium4
4.	IF	Temperature is Medium2 AND Occupancy is Small AND HVACUsage is Off AND LightingUsage is Off THEN EnergyConsumption is Medium2
5.	IF	Occupancy is Medium1 AND HVACUsage is Off AND LightingUsage is On AND RenewableEnergy is Small THEN EnergyConsumption is Medium2
6.	IF	Temperature is Small AND Humidity is not Medium5 AND HVACUsage is On AND LightingUsage is On THEN EnergyConsumption is Medium4
7.	IF	Temperature is Small AND Humidity is Small AND Occupancy is Large AND HVACUsage is On AND RenewableEnergy is Large THEN EnergyConsumption is Large
Solution No. 8:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND SquareFootage is not Medium2 AND HVACUsage is On THEN EnergyConsumption is Medium4
3.	IF	Temperature is Small AND Occupancy is Medium5 AND RenewableEnergy is Large THEN EnergyConsumption is Medium4
4.	IF	Temperature is Medium2 AND SquareFootage is not Large AND Occupancy is Small AND HVACUsage is Off AND LightingUsage is Off AND RenewableEnergy is not Large THEN EnergyConsumption is Medium2
5.	IF	Occupancy is Medium1 AND HVACUsage is Off AND LightingUsage is On AND RenewableEnergy is Small THEN EnergyConsumption is Medium2
6.	IF	Temperature is Small AND Humidity is not Medium5 AND HVACUsage is On AND LightingUsage is On THEN EnergyConsumption is Medium4
7.	IF	Temperature is Small AND Humidity is Small AND Occupancy is Large AND HVACUsage is On AND RenewableEnergy is Large THEN EnergyConsumption is Large
8.	IF	Temperature is Medium1 AND Humidity is Medium1 AND SquareFootage is not Medium5 THEN EnergyConsumption is Medium3
Solution No. 9:
1.	IF	Temperature is Small THEN EnergyConsumption is Medium2
2.	IF	Temperature is Small AND SquareFootage is not Medium2 AND Occupancy is not Medium1 AND HVACUsage is On THEN EnergyConsumption is Medium4
3.	IF	Temperature is Small AND Occupancy is Medium5 AND RenewableEnergy is Large THEN EnergyConsumption is Medium4
4.	IF	Temperature is Medium2 AND Humidity is not Medium5 AND SquareFootage is not Large AND Occupancy is Small AND HVACUsage is Off AND LightingUsage is Off AND RenewableEnergy is not Large THEN EnergyConsumption is Medium2
5.	IF	Occupancy is Medium1 AND HVACUsage is Off AND LightingUsage is On AND RenewableEnergy is Small THEN EnergyConsumption is Medium2
6.	IF	Temperature is Small AND Humidity is not Medium2 AND SquareFootage is not Small AND LightingUsage is On AND RenewableEnergy is not Medium2 AND DayOfWeek is Friday THEN EnergyConsumption is Medium5
7.	IF	Temperature is Small AND Humidity is Small AND Occupancy is Large AND HVACUsage is On AND RenewableEnergy is Large THEN EnergyConsumption is Large
8.	IF	Temperature is Medium2 AND Humidity is Medium4 AND HVACUsage is On AND RenewableEnergy is Medium4 AND DayOfWeek is Tuesday AND Holiday is No THEN EnergyConsumption is Medium2
9.	IF	Temperature is Small AND Humidity is not Small AND SquareFootage is not Small AND Occupancy is not Large AND HVACUsage is Off AND DayOfWeek is Wednesday AND Holiday is No THEN EnergyConsumption is Small
10.	IF	Humidity is Medium3 AND Occupancy is Small AND LightingUsage is Off AND RenewableEnergy is Medium2 AND DayOfWeek is not Friday THEN EnergyConsumption is Medium2

(Figure 6 shows membership functions of fuzzy sets used in those rule bases); solution No. 10 of the lowest interpretability (21 rules) is not included in Table 3.

Table 3 reveals an interesting regularity. We can see that the rule base of solution No. 2 contains the one-rule base of solution No. 1 (marked by red font in the rule base of solution No. 2). In turn, the rule bases of solutions Nos. 3 and 4 contain both rules (marked by red font) from the rule base of solution No. 2). Next, the rule base of solution No. 5 contains all three rules (marked by red font; one slightly modified) from the rule base of solution No. 4. In what follows, the rule base of solution No. 6 contains all four rules (marked by red font) from the rule base of solution No. 5, etc. In general, the rule base of solution No. i contains the rule base of solution No. $i-1$ (marked by red font in Table 3), $i=2,3,\dots ,10$. Therefore, if higher FRBPS accuracy is required, then our approach either introduces some additional fuzzy rules or extends the earlier-discovered rules. In such a way, our approach provides a more detailed (more accurate) description of the prediction problem we consider. It ts worth emphasizing that such a regularity confirms an internal integrity of our approach.

In order to demonstrate some practical aspects regarding FRBPS interpretability, an example based on solution No. 6 from Table 3 is considered. Its five rules can be aggregated into four, easier-to-interpret sentences, in which the technical names of attributes and the labels of fuzzy sets are replaced with more friendly and semantics-related terms:

• If temperature is small, then energy consumption is slightly smaller than average.
• If temperature increases slightly but the number of occupants is small and if HVAC and lighting systems are turned off, then energy consumption is still slightly smaller than average (as in the case of rule No. 1).
• If the lighting system is turned on but the number of occupants is relatively small, the HVAC system is turned off, and renewable energy sources provide a small amount of energy, then energy consumption is still slightly smaller than average (as in the case of rule No. 1).
• If temperature is still low (as in the case of rule No. 1) but the HVAC system is turned on or the number of occupants is relatively large, then energy consumption increases slightly (in comparison with the case of rule No. 1).

The first and most general rule confirms the quite obvious direct dependence of energy consumption on temperature. The next two rules can be interpreted as a recipe for maintaining low energy consumption in houses with (relatively) few inhabitants, in two cases: (i) when the temperature rises or (ii) when the lighting system is turned on (e.g., in the evenings). In the first case, HVAC and lighting systems should be turned off. In the second case, the HVAC system should be turned off and an additional source of renewable energy (providing at least a small amount of energy) should be turned on to compensate the energy consumption of the lighting system. The last rule indicates the reasons for increased energy consumption despite low temperatures (in comparison with previous rules), i.e., the energy consumption rises when the HVAC system is turned on or a large number of residents live in the house. Naturally, the interpretation of rules for more complex problems (with many input attributes) may not be so obvious as in this example.

The proposed approach has some limitations regarding the interpretability of generated rules. On the one hand, systems with the lowest complexity (top-left extreme part of the Pareto-front approximation) usually contain a very small number of rules with a small number of input attributes. These rules describe quite simple and usually obvious relationships between data. For this reason, they often do not bring anything groundbreaking to the considered regression problem (see, e.g., solution No. 1 in Table 3 indicating only a simple, direct dependence of energy consumption on temperature). On the other hand, systems with very complex structures (bottom-right extreme part of the Pareto-front approximation) contain many rules with a large number of input attributes. Some of them are often of low strength (i.e., they are activated by a small number of data samples) and represent unique, exceptional, and special cases of a given regression problem. Due to the small number of such cases in the learning data set, they cannot be considered as representative ones. Therefore, general conclusions drawn on the basis of rules describing such specific data may be false. Ultimately, the most valuable systems are located in the middle part of the Pareto-front approximation (like the considered solution No. 6 in Figure 5). They contain not very complex rules with relatively high strength and, because of this, they have a high ability to generalize knowledge, which is confirmed by their high accuracy for test data. Drawing conclusions based on such rules and their interpretation is devoid of the above-mentioned shortcomings.

An important part of this work is the cross-validation-based evaluation of our approach. Following the earlier-mentioned original learning test data split ratio equal to 640:340, i.e., roughly equal to 2:1, we performed the k-fold cross-validation experiment with $k=3$ (i.e., with 2:1 learning test data split ratio). A single learning experiment begins with generation of a Pareto-front approximation as shown earlier in this section (see Figure 5, Table 2 and Table 3). In turn, a single solution characterized by the highest test data accuracy is selected from that front approximation. If more than one such solutions occur, then one solution characterized by the highest interpretability is selected (it is solution No. 6 in Figure 5, Table 2 and Table 3). Next, the results from all $k=3$ partial experiments are averaged. Such an experiment is then repeated 10 times for different initializations of our approach. The final averaged results are shown in the last row of

Table 4. Results of our approach and comparison with alternative methods (k-fold cross-validation with $k=3$; averaged values from 10 runs; n/ap stands for not applicable).

No.		Method	Averaged Accuracy Measures for Learning and Test Data				Averaged Interpretability Measures
			$\overline{{\mathit{Q}}_{\mathit{RMSE}}^{\mathbf{\left(}\mathit{lrn}\mathbf{\right)}}}$	$\overline{{\mathit{Q}}_{\mathit{RMSE}}^{\mathbf{\left(}\mathit{tst}\mathbf{\right)}}}$	$\overline{{\mathit{Q}}_{\mathit{MAPE}}^{\mathbf{\left(}\mathit{lrn}\mathbf{\right)}}}$	$\overline{{\mathit{Q}}_{\mathit{MAPE}}^{\mathbf{\left(}\mathit{tst}\mathbf{\right)}}}$	$\overline{\mathit{R}}$	$\overline{{\mathit{n}}_{\mathit{INP}}}$	$\overline{{\mathit{n}}_{\mathit{FS}}}$	$\overline{{\mathit{n}}_{\mathit{INP}\mathbf{/}\mathit{R}}}$
1.	Clasical linear regressors	LinearRegression	5.5 × 10−12 ± 2.3 × 10−12	5.07 ± 0.16	5.38 × 10−12 ± 2.9 × 10−12	5.37 ± 0.17	n/ap	10	n/ap	n/ap
2.		Ridge	2.50 ± 0.02	5.07 ± 0.16	2.64 ± 0.02	5.37 ± 0.17	n/ap	10	n/ap	n/ap
3.		SGDRegressor	3.6 × 10−4 ± 1.1 × 10−5	5.92 ± 0.9	3.63 × 10−4 ± 3.5 × 10−6	6.25 ± 0.18	n/ap	10	n/ap	n/ap
4.		LGBMRegressor	1.97 ± 0.05	5.54 ± 0.30	2.02 ± 0.06	5.86 ± 0.42	3046.4 ± 12.8	15 ± 0.0	n/ap	7.91 ± 0.52
5.		XGBoost	3.01 ± 0.09	5.54 ± 0.30	3.20 ± 0.11	5.89 ± 0.43	556.4 ± 19.8	15 ± 0.0	n/ap	5.87 ± 0.02
6.	Regressors with variable selection	ElasticNet	5.37 ± 0.04	5.41 ± 0.15	5.69 ± 0.05	5.73 ± 0.19	n/ap	10	n/ap	n/ap
7.		Lars	2.20 ± 0.06	5.28 ± 0.21	2.71 ± 0.09	5.55 ± 0.25	n/ap	10	n/ap	n/ap
8.		Lasso	5.49 ± 0.05	5.52 ± 0.16	5.81 ± 0.06	5.85 ± 0.21	n/ap	10	n/ap	n/ap
9.		LassoLars	8.14 ± 0.05	8.15 ± 0.21	8.68 ± 0.06	8.69 ± 0.26	n/ap	10	n/ap	n/ap
10.		LassoLarsIC	5.17 ± 0.07	5.21 ± 0.17	5.44 ± 0.08	5.46 ± 0.18	n/ap	10	n/ap	n/ap
11.		Orthogonal-MatchingPursuit	3.59 ± 0.02	5.07 ± 0.17	3.74 ± 0.02	5.37 ± 0.16	n/ap	10	n/ap	n/ap
12.	Bayesian regressors	ARDRegression	9.65 × 10−3 ± 2.8 × 10−3	5.16 ± 0.19	7.78 × 10−3 ± 3.4 × 10−3	5.47 ± 0.20	n/ap	10	n/ap	n/ap
13.		BayesianRidge	4.77 ± 0.05	5.07 ± 0.16	5.05 ± 0.05	5.37 ± 0.16	n/ap	10	n/ap	n/ap
14.	Outlier-robust regressors	HuberRegressor	5.42 ± 0.06	5.49 ± 0.25	5.70 ± 0.06	5.79 ± 0.28	n/ap	10	n/ap	n/ap
15.		TheilSenRegressor	3.3 × 10−12 ± 1.1 × 10−12	5.07 ± 0.17	4.01 × 10−12 ± 1.4 × 10−12	5.38 ± 0.17	n/ap	10	n/ap	n/ap
16.	Generalized linear models	GammaRegressor	7.81 ± 0.64	7.91 ± 0.56	8.32 ± 0.72	8.43 ± 0.51	n/ap	10	n/ap	n/ap
17.		PoissonRegressor	7.61 ± 1.04	7.77 ± 0.81	8.12 ± 1.14	8.27 ± 0.81	n/ap	10	n/ap	n/ap
18.		TweedieRegressor	5.36 ± 0.06	5.43 ± 0.13	5.68 ± 0.06	5.75 ± 0.16	n/ap	10	n/ap	n/ap
19.	Decision trees and random forests	DecisionTree-Regressor	0.0 ± 0.0	7.04 ± 0.28	0.0 ± 0.0	7.41 ± 0.37	800.0 ± 0.0	17.8 ± 0.4	n/ap	11.7 ± 0.1
20.		RandomForest-Regressor	2.29 ± 0.03	5.51 ± 0.28	2.10 ± 0.03	5.71 ± 0.35	505.2 ± 6.4	17.9 ± 0.24	n/ap	10.9 ± 0.24
21.		Our approach	5.091 ± 0.15	5.46 ± 0.21	5.26 ± 0.16	5.97 ± 0.23	5.3 ± 0.6	5.6 ± 0.6	11.6 ± 0.6	3.3 ± 0.06

, which also collects the averaged results of as many as 20 alternative methods applied to the considered data set. The results of Table 4 are presented in the form of “mean value ± standard deviation” except for $\overline{n_{INP}}$ for non-rule-based systems which operate on all 10 input attributes; in such cases, $\overline{n_{INP}}=n_{INP}=10$. For reader convenience, the results of Table 4 (the mean values of averaged accuracy and interpretability measures) are also presented in a graphical form (using horizontal bar graphs) in Figure 7 and Figure 8.

We consider six groups of alternative approaches. The first group, referred to as “Clasical linear regressors”, includes (i) ordinary least squares linear regression (in brackets, we offer the corresponding software implementation name; see [48]; in the considered case—LinearRegression), (ii) linear least squares with L2 regularization (Ridge), (iii) linear model fitted by minimizing a regularized empirical loss using Stochastic Gradient Descent, SGD (SGDRegressor), (iv) Light Gradient Boosting Model (LGBMRegressor), and (v) eXtreme Gradient Boosting for regression (XGBoost). The second group, labeled “Regressors with variable selection”, includes (i) linear regression with combined L1 and L2 priors as a regularizer (ElasticNet), (ii) Least Angle Regression model, LARS (Lars), (iii) linear model trained with L1 prior as regularizer (Lasso), (iv) Lasso model fit with LARS (LassoLars), (v) Lasso model fit with LARS applying Bayes Information Criterion or Aikake Information Criterion for model selection (LassoLarsIC), and (vi) orthogonal matching pursuit model (OrthogonalMatchingPursuit). The third group of “Bayesian regressors” covers (i) Bayesian Automatic Relevance Determination, ARD, regression (ARDRegression), and (ii) Bayesian Ridge regression (BayesianRidge). The fourth group of “Outlier-robust regressors” includes (i) L2-regularized linear regression model that is robust to outliers (HuberRegressor) and (ii) Theil-Sen estimator: robust multivariate regression model (TheilSenRegressor). The fifth group of “Generalized linear models (GLM) for regression” covers (i) a generalized linear model with a Poisson distribution (PoissonRegressor), (ii) a generalized linear model with a Tweedie distribution (TweedieRegressor), and (iii) a generalized linear model with a Gamma distribution (GammaRegressor). The sixth and last group of “Decision trees and random forests” includes (i) decision trees (DecisionTreeRegressor) and (ii) random forests (RandomForestRegressor). The DecisionTreeRegressor, RandomForestRegressor, LGBMRegressor, and XGBoost methods of Table 4 can process only numerical attributes. For this reason, each categorical attribute of the considered data set is converted to $a_i$ binary numerical attributes ($a_i$ is the number of “values” of the ith categorical attribute; see Section 3 of this work); as a result of that, the overall number of input attributes processed by those four methods increases from 10 to 18.

The overwhelming majority of the alternative techniques of Table 4 are the black-box, non-transparent, and non-interpretable approaches focusing only on prediction accuracy. Also, the four earlier-mentioned and rule-generating methods can be included in the black-box class of methods since they generate huge numbers (hundreds) of complex rules. Therefore, all alternative methods are favored, in a way, with regard to our approach, which aims at optimizing the accuracy–interpretability trade-off of prediction systems. However, despite that, the proposed method generates the predictions that are not only highly accurate but also highly interpretable (i.e., using, on average, only 5.3 fuzzy rules with 3.3 input attributes per rule). Concluding, the approach we propose significantly outperforms alternative methods in terms of interpretability of the energy consumption predictions made while remaining comparable or slightly superior in terms of the accuracy of those predictions.

6. Conclusions

The problem of an accurate as well as interpretable and transparent prediction of energy consumption in residential buildings is considered in this paper. The solution that we propose in this work to address the considered problem employs the knowledge discovery machine learning approach combining fuzzy systems with evolutionary optimization. The proposed approach is characterized by both high accuracy and high interpretability and transparency or, more specifically, igenetically optimized accuracy–interpretability trade-off of generated predictions. The trade-off optimization is carried out by means of multi-objective evolutionary optimization algorithms. For that purpose, we use our generalization of the well-known strength Pareto evolutionary algorithm 2 (SPEA2). The accuracy (i.e., the ability to make correct predictions) and the interpretability and transparency (i.e., the ability to formulate compact, explicit, and understandable explanations of mechanisms governing those predictions) are important features of many prediction systems including energy consumption predictions. Fuzzy linguistic rules are characterized by easy-to-grasp interpretation and comprehensibility. Compact sets of such rules generated by our approach belong to the most effective knowledge representation schemes in various domains, including the domain of energy consumption prediction. The most recently published in [5] energy consumption prediction data set is used in the reported experiments.

The contribution of this work includes both methodology and experimental investigations. From the methodological point of view, the designing procedure of fuzzy rule-based prediction systems proposed in this work is our original extension and generalization (for regression problems operating on continuous outputs) of our approach to designing fuzzy rule-based classifiers (operating on discrete outputs, i.e., class labels), which we developed earlier and recently published in this journal in [10]. The experimental contribution of this work includes designing the collection of fuzzy rule-based prediction systems characterized by optimized accuracy–interpretability trade-off levels for residential building energy consumption prediction. The earlier-mentioned and most recently published in [5] data set was used in those experiments. A comparison of our approach with 20 available alternative methods shows that our approach significantly outperforms alternative methods in terms of interpretability and transparency of the energy consumption predictions made while remaining comparable or slightly superior in terms of the accuracy of those predictions.

Our future research will concentrate on improving the effectiveness of algorithms for the optimization of system accuracy–interpretability trade-off. Such algorithms are essential in designing highly accurate and highly interpretable modern classification and regression systems from data in various areas of applications including electricity consumption prediction. Such systems can be classified as explainable artificial intelligence systems (see, e.g., [49,50]) or interpretable machine learning systems (see, e.g., [51,52]).