Modeling and Validation of Residential Water Demand in Agent-Based Models: A Systematic Literature Review

Abstract

Current challenges, such as climate change or military conflicts, show the great importance of urban supply infrastructures. In this context, an open question is how different scenarios and crises can be studied in silico to assess the interaction between the needs of social systems and technical infrastructures. Agent-based modeling is a suitable method for this purpose. This review investigates (i) how agent-based models of residential water demand should be validated, (ii) how such models are commonly built and (iii) validated, and (iv) how these validation practices compare to the recommendations drawn from question (i). Therefore, a systematic literature review using the PRISMA framework is conducted. Out of 207 screened papers, 35 models are identified with an emphasis on highly realistic models (i.e., highly detailed and representing specific real-world systems) for planning, management, and policy of urban water resources. While some models are thoroughly validated, quantified validation distinct from calibration data should be emphasized and used to communicate the confidence in results and recommendations drawn from the models. Pattern-oriented validation, validation on multiple levels and on higher moments of aggregated statistics should be considered more often. These findings expand prior literature by providing a more extensive sample of reviewed articles and recommending specific approaches for the validation of models.

1. Introduction

An increasing majority of the global population lives in urban areas [1]. The needs of these people must be met with a limited amount of resources available. Linking resources (supply) and demand is the task of infrastructures. Due to urbanization, the number of people and thus their demand for resources, such as water or energy, is steadily increasing in cities. At the same time, the climate crisis or military conflicts lead to a decrease in supply. The increasing demand and the decreasing supply result in increased stress that the infrastructures are exposed to.

Since the function of infrastructures is vital for the population, many political entities, such as national, regional or local governments, have implemented policies to protect them and improve their resilience, referring to them as “critical infrastructures” [2,3]. As Scharte [4] points out, the study of the resilience of infrastructures should always account for the socio-technical nature of infrastructure systems. Socio-technical systems are comprised not only of technical components, but also social factors. Such social factors are essentially the interaction among humans and of humans with the infrastructures. These socio-technical interactions foster the emergence of complex behaviors [4,5]. For example, for water infrastructures, the time and location of water demand by end users has been shown to be an important factor for resilience [6]. Yet, modeling disaggregated water demand for cities is not a trivial pursuit [7]. Therefore, models for spatially heterogeneous water demand are highly relevant to assess the resilience of water infrastructures and for the planning or management of these systems.

One paradigm that has become increasingly popular in recent years to model the behavior of such complex and spatially heterogeneous socio-technical systems is agent-based modeling [8]. In this technique, the behavior of sub-systems (called agents) is modeled, and the interaction between those sub-systems is simulated [9]. Macal and North [10] define four distinct features of an agent: autonomy, modularity, sociality, and conditionality. Autonomy refers to the agents ability to perform actions based on the inputs and states it receives from the other aspects included in the model, which are called its environment. The agent is further clearly separable from its environment (modularity) and interacts with the other agents in that environment (sociality). Those interactions can change the agent or its behavior (conditionality). The resulting interactions between agents, other agents, and their environment are simulated and result in an aggregated system behavior which is then assessed.

Therefore, agent-based modeling is often referred to as a “bottom-up” approach to modeling aggregated systems, as it makes no (or very limited) assumptions about the aggregated behavior of the whole system at hand [11,12]. In these aggregated simulations, complex behaviors can emerge, which lead to the use of agent-based models for the simulation of complex systems [13]. The ability of agent-based models to represent complex systems has peaked the interest of researchers of various scientific disciplines, including economics, political sciences, engineering and ecology [13]. Most importantly for this paper, agent-based models are also of high interest for the modeling of water infrastructure systems [14,15], as is discussed in depth in Section 1.1. Nevertheless, agent-based modeling has often been criticized and and is sometimes not accepted by scientific communities, as agent-based modeling presents challenges in the validation of the resulting models [9], due to issues such as the high number of parameters and assumptions of underlying agent behavior [16,17,18].

The results of this paper provide researchers with an overview of agent-based models of residential water demand that are described in the scientific literature. In this overview, special consideration is paid to the behavioral models and assumptions used in these models and the used validation approaches. This provides a deep insight into the possible modeling decisions one can take when designing an agent-based model of residential water demand and further provides guidance for the validation of such models.

1.1. Modeling Urban Water Demands: State of the Art

Given the importance of modeling urban water demand, a multitude of authors have reviewed this topic. In the following, relevant literature on the modeling of urban water demand is summarized to highlight the research gap addressed by this review. Focusing on econometric methods, Arbues et al. [19] reviewed the findings of studies assessing the relation of economic factors, such as household income, water price and the design of the water billing structures on the water demand of households, aggregated over billing periods. They included other variables, such as climatic effects and the structure of housing. Besides reporting estimates of the effects observed in the reviewed studies, they point out limitations in the econometric methods for the estimation of water demand and the application of the findings in the design of policy decisions.

While the studies assessed by Arbues et al. [19] use water billing data, which normally reflect annual or monthly water demand, Donkor et al. [20] focused on models used to forecast water demand in various time horizons. In contrast to the focus on policy design chosen by Arbues et al. [19], they focused on the forecasting of water demand to decrease uncertainty in water utility management. They found that artificial neural networks were prominent for short-term water demand forecasting, i.e., forecasting periods shorter than one month. For longer forecasting periods, econometric methods and scenario-based forecasting methods were used more frequently.

Differing from the approach of forecasting or predicting water demand chosen by the aforementioned studies, Creaco et al. [21] reviewed models aimed at the reconstruction of water-demand profiles based on stochastic pulses. Their findings show that different models are able to generate time series of stochastic water demand that recreate the statistical properties of real water demand time series.

In contrast, Nauges and Whittington [22] assessed the literature on water demand in the Global South. While the surveyed statistical models are similar to the regression models discussed by Arbues et al. [19], water demand in the Global South is often characterized by distinct challenges. The challenges assessed by Nauges and Whittington [22] include the trade-off of water sources differing in water price, cost of water collection (i.e., ease of access) and water quality.

House-Peters and Chang [23] reviewed literature on modeling urban water demand over various spatial and temporal scales, including influences on water demand and the used modeling methods. Their findings show an increase in spatially explicit models, such as agent-based models in the years leading up to the review. Spatially explicit models are models that describe the behavior of a system involving a spatial dimension, i.e., spatially disaggregating water demand. They saw great potential in such data-rich models, while pointing out the potential shortcomings of the transparency of agent-based models.

In the context of spatially explicit models, Cominola et al. [14] gave a comprehensive review on the advances and challenges in residential water-demand modeling due to the use of smart meter data. They discussed agent-based models and highlighted challenges in the validation of agent-based models due to the large number of assumptions and parameters in such models.

A review dedicated to agent-based models in the context of water resource planning and management was performed by Berglund [15], outlining the potentials and limitations of the approach. The findings are illustrated on two exemplary models. (The models are also part of the review starting in Section 4). They found that the use of agent-based models was limited by the confidence and applicability of model predictions and that validation remains an important challenge for further research on the topic.

This overview shows a multitude of approaches to model water demand, ranging various spatial and temporal scales and incorporating diverse independent variables and model structures [14,15,19,20,21,22,23]. Although the importance of validation and transparency of agent-based models has been highlighted by many of the aforementioned reviews [14,15,23], no specific recommendations have been proposed. In addition, no review has systematically surveyed the literature of agent-based models of water demand regarding the used behavioral assumptions.

Therefore, the aim of the review at hand is to fill these research gaps by giving modelers an overview of common practices when building and validating agent-based models of residential water demand and contrasting this overview with recommendations regarding said validation.

1.2. Outline

To facilitate this review, Section 2 sharpens the scope by specifying the research questions and outlining the used methodology. Following this, Section 3 discusses different approaches for the validation of agent-based models that include human behavior and summarizes the emerging recommendations for the validation of agent-based models. This section follows the approach of a narrative literature review to condense relevant literature in the field and summarize key methods and patterns in the literature. Next, Section 4 uses the “Preferred Reporting Items for Systematic reviews and Meta-Analyses” (PRISMA) [24,25] method to perform a systematic literature review, surveying agent-based models of residential water demand regarding their used behavioral assumptions and the methods used for their validation. Following this, Section 4.5 compares the findings of the applied methods for the validation of agent-based models of residential water demand and the recommendations drawn in Section 4. The logical connections between the Section 3, Section 4 and Section 4.5 are visualized in Figure 1. Finally, Section 5 discusses the results of the paper, touching on the implications, potential biases and possible further research directions. Section 6 then concludes the paper with a brief summary.

2. Research Questions and Methodology

As briefly mentioned in the previous section, agent-based models of residential water demand can be constructed and validated in various ways. In order to provide a deeper understanding of such approaches and methods, a deep investigation, identification and comparison of the most relevant scientific contributions regarding the validation of agent-based models is needed. This review investigates which recommendations regarding the validation of agent-based models can be drawn from the literature and how these recommendations are adopted for agent-based models of residential water demand. The following research questions (described below) were used to narrow down and clearly define the scope of this paper.

(i)

Which approaches for the validation of agent-based models, including human behavior, are described in literature?

Models seek to represent the behavior of one or multiple real-world systems. Since models are always abstracted from reality, this representation is always inaccurate [26]. The first research question seeks to understand how data should be used to test whether an agent-based model represents the real-world system in a useful way (i.e., validation). Validation is generally done by comparing the output of a model to the output of the modeled real-world system or knowledge about the behavior of the real-world system. With this, the fit of the models behavior to real-world systems behavior is assessed. To answer this question, the literature on the validation of agent-based models is surveyed. A review is given on the recommendations of which data and specific methods should be used to approach the challenge of validating agent-based models that include human behavior.

(ii)

How is residential water demand modeled using agent-based modeling?

Here, the functions used to describe the behavior of residential water consumers in agent-based models are assessed. The goal is to understand which different theories and assumptions are made regarding the consumption of water by humans. Specifically, this question seeks to isolate which main drivers of water consumption are respected and how the functional relationships between those drivers and water consumption are assumed. To answer this question, the focus is put on agents that represent the behavior of individuals or groups characterized by human consumption and household use in urban environments. This especially excludes such models that primarily model the agricultural use or water use in remote rural areas since such models are deemed of lower relevance for the scope of this paper.

(iii)

How are the proposed models validated?

As explained in the description of question (i), the fit of a model to the real-world systems behavior represented by the model can be validated in multiple ways. Question (iii) seeks to answer which methods are used in the surveyed literature to assess the fit of the agent-based model to the real-world systems behavior, i.e., which validation methods are used.

(iv)

How are the recommended principles for the validation of agent-based models (discussed in (i)) applied in the validation of the surveyed models (discussed in (iii))?

Finally, the methods identified and described in (iii) are compared to the practices or guidelines described in (i). Therefore, this question answers if the model validation in the literature of agent-based models of residential water demand is performed in line with the recommendations for model validation drawn in the literature on the validation of agent-based models.

Question (i) is answered in Section 3 by surveying some frequently cited literature regarding the building and validation of agent-based models. The perspectives of multiple scientific disciplines and authors are compiled in order to obtain a holistic view on the practices and challenges of validation in agent-based modeling and simulation. Most notably, sources from sociology, engineering, ecology and economics are included. This provides an overview of highly relevant literature for readers seeking guidance and deeper understanding of the subject of validation of agent-based models.

Questions (ii), (iii) and (iv) are investigated by performing a systematic literature review in accordance with the PRISMA guidelines [24,25]. The PRISMA approach ensures a comprehensive overview of models and modeling approaches present in the literature.

The goal of the review at hand is to give modelers an overview of common practices when building and validating agent-based models of household water demand. It can also serve as a basis for the validation of such models or for researching methods to do so. The used methods are presented in

Table 1. Research questions and used review methodologies.

Number	Question	Review Method
(i)	Which approaches for the validation of agent-based models including human behavior are described in literature?	Semi-systematic review/narrative review
(ii)	How is residential water demand modeled using agent-based modeling?	Systematic descriptive literature review
(iii)	How are the proposed models validated?	Systematic descriptive literature review
(iv)	How are the recommended principles for the validation of agent-based models applied in the validation of the surveyed models?	Synthesis from (i), (ii), and (iii) (Not in review format)

while detailed information on the choice of review methods can be found in Appendix A.

We base this review on agent-based modeling for residential water demand on the guidelines of the PRISMA 2020 statement [24]. The PRISMA framework is chosen to provide the necessary rigor and comprehensiveness of the review at hand. The methodology has to be adjusted for this review. The adjusted PRISMA method for this systematic descriptive review is presented comprehensively in Appendix A.1. In the following, the most unique adjustments and chosen variables are presented.

An important variable for the definition of a search strategy is the chosen search query for the data base search. In this review, the scope includes all studies that use agent-based models of residential water demand. Therefore, the terms “agent-based”, “water”, “household”, “residential” and “demand” were identified as relevant for the literature search. Following a preliminary screening of literature, relevant synonyms for the search queries were collected. To ensure a comprehensive search, the synonyms presented in

Table 2. Initial search queries and identified synonyms.

Initial Query	Synonyms
agent-based	agentbased, multi-agent, multiagent
water	-
resident *1	domestic, household, home *
demand	consum *, use, need

Note(s): ¹ The use of the asterisk (*) includes multiple endings or suffixes of a word into the search query. It does not indicate additional explanations.

were included, resulting in the search queries reported in Appendix A.2 and Appendix A.3. The terms where different suffixes impacted the search were shortened to their stem and an asterisk (*) was added to include possible word forms (e.g., the terms consume, consumption and consumer were all included by searching for the query “consum*”).

Another approach specific to this review was the handling and automated exclusion of duplicates. The data obtained by the data base search were exported into .csv files and imported into MS-Excel. Excel functions were used to exclude duplicates based on the title of the study. The title was chosen since it provides a unique identifier while also allowing fast comprehension and error-checking by human readers. Other unique identifiers, such as the digital object identifier (DOI), were found to be less feasible for manual handling. Further columns were added in the Excel spreadsheets to allow the notation of findings during the review process. The further selection of records and the data collection and evaluation was done by the main author of this review.

Another important detail of the review process in this paper is that no synthesis of findings of the reviewed records was performed. Since the scope of this paper is to assess methods rather than evaluating the findings of the surveyed literature, the synthesis consisted of the qualitative description and comparison of the studies. A quantitative assessment of study findings, reported effects and potential biases was, therefore, not performed.

3. Narrative Review of Validation Techniques for Agent-Based Models

While for technical models and systems the term “validation” and the related term “verification” are considered to be well defined, a clear definition and distinction of those terms is harder to find in the case of models in the social sciences [27]. To ensure the consistent use of the term validation in this paper, the definition of validation according to Fagiolo et al. and Frisch [17,28,29] is used. Therefore, validation is seen as the process of evaluating to which degree a model represents a real-world system. While this general definition of validation can be interpreted to include the verification [27], these terms are separated in this paper, and the concept of Fagiolo et al. and Frisch [17,28,29] is followed, which can be seen as the approach to “external validity” as described by Hahn [27].

3.1. Differing Approaches to the Validation of Agent-Based Models

Since the validation of agent-based models has been thoroughly discussed from different perspectives, different approaches will briefly be presented in the following. The presented approaches are summarized in

Table 3. Approaches to the validation of agent-based models in various scientific disciplines.

Ref.	Approach	Description
[27,31]	Expert knowledge	Expert judgment, whether the model represents the real-world system behavior correctly or usefully
[18,31,32]	Time series validation	Comparison of the model output to specific empiric time series data of the modeled system
[17,28]	Validation on stylized facts	Comparison to general behaviors observed repeatedly in multiple systems
[33]	Pattern-oriented validation	Comparison to general patterns observed in the modeled system and characteristic for its behavior

. Differences in the approaches are mainly rooted in the differing empirical data that the model outputs are compared to. The approaches presented include the validation on expert knowledge (Section 3.1.1), time series validation (Section 3.1.2), validation on stylized facts (Section 3.1.3), and pattern-oriented validation (Section 3.1.4).As this paper is focused on socio-technical models, the review of validation methods focuses on the validation of agent-based models that include human behavior. The results might therefore not be fully applicable for purely technical agent-based models. Such models are often referred to as models of “multi-agent systems” but are not within the scope of this review. For an example, see Logan et al. [30].

3.1.1. Validation on Expert Knowledge

One approach to agent-based modeling of socio-technical systems stems from the perspective of classical engineering disciplines, such as environmental, civil or mechanical engineering. In the tradition of these disciplines, validation is often carried out through experiments, which are hard or impossible to perform on social systems [27,31]. Therefore, the literature tends to recommend the validation utilizing the knowledge of experts, such as stakeholders in the modeled system or scientists in academic fields. These experts can either be asked to assess the models output [31] or to participate in structured review and role-playing experiments [27] to judge the validity of the assessed models. These techniques benefit from the knowledge of those experts and allow a nuanced perspective of the models’ performance and might be used to increase stakeholder acceptance [27]. Further, this technique has the potential to assess the qualitative and structural features of a model that might be insufficiently assessed by other approaches [34] due to the rejection of models that are able to explain phenomena on the basis of false assumptions. Despite these benefits, this approach to validation is prone to bias and subjectivity [31]. Therefore, validation on the basis of expert knowledge is often seen as a low level of evidence for the validity of a model [16,28].

3.1.2. Time Series Validation

Another approach for the validation of agent-based models of socio-technical systems is the comparison of model outputs to historical time series [31]. In this approach, the model which is validated is instantiated with conditions resembling a historic state of the real-world system. Afterward, the models’ ability to replicate historically observed time series data is judged. Some authors see this approach as superior to others [32]. Yet, the comparison of model outputs to historical time series can be criticized on the ground of the complex properties of socio-technical systems. Two challenges emerging with these complex properties are of special importance for this review: over-parametrization and the uniqueness of time series. Over-parametrization refers to the fact that most agent-based models are defined by a large number of independent variables. Those variables can, in theory, be fitted in such a way that the model output replicates a wide range of possible system behaviors. This in turn reduces the assured validity of the model structure since any model could be fitted to the data in a similar way, regardless of the accuracy of the underlying structural modeling assumptions [17]. Secondly, socio-technical systems often show stochastic and path-dependent behaviors. Therefore, a singular time series should not be assumed to be a typical trace of the systems behavior. Since every historic time series is potentially unique, one should not measure the validity of modeling assumptions or modeled behavior on the basis of such a singular trace [16,17]. This criticism is especially relevant since the historic state of a system might not be comparable to the current system such that a model accurately representing the historic system might not be appropriate or useful for the description of the current system [31].

3.1.3. Validation on Stylized Facts

While the validation on historic time series compares the models’ output to a specific real-world system behavior, a different approach seeks to validate the model behavior against typical system behavior, so-called stylized facts. Stylized facts refer to statistical properties or regularities that can repeatably be observed in different economic systems [28]. Examples of properties that are stylized facts are distribution shapes (e.g., fat-tailed distributions in financial asset returns) and correlations (e.g., between production and unemployment in macro economies) [28]. While the study of stylized facts is common in economics, other social sciences do not emphasize the concept as much [35]. Therefore, many economic agent-based models seek to replicate such stylized facts and hence also are validated against such facts [28]. The drawback of this approach lies in the fact that the use of such stylized facts does not guarantee a proper validation [28] and is subject to many of the weaknesses of the aforementioned validation approaches [17]. Further, the lack of established stylized facts in the domain of socio-technical systems and social sciences [35] might make it harder to find facts to be used as a measure of validity.

3.1.4. Pattern-Oriented Validation

An approach that is not only used in social sciences but also ecology is pattern-oriented validation [33]. This approach validates a model by assessing its ability to replicate multiple behavioral patterns that characterize the observed behavior of the real-world system. Examples for such patterns include spatial or temporal distributions. Pattern-oriented validation reduces the possibility of alternative parametrizations or model structures (i.e., equifinality) by using multiple relevant patterns and supporting adequate granularity and detail in the models [33].

3.1.5. Distinction between the Validation Approaches

Even though these approaches are distinct in the literature, a clear separation between the approaches can be challenging. An example of this is the distinction between pattern-oriented validation and validation on stylized facts. Gilbert [11] emphasizes the distinction between the approach of Fagiolo et al. [17], which is based on stylized facts, and the pattern-oriented validation described by Grimm et al. [33]. Other authors, such as Meyer, [36] use the work of Grimm et al. [33] as an example of validation using stylized facts.

Hirschman [35] defines stylized facts as “simple empirical regularities in need of explanation”. Grimm et al. [33] define patterns as “defining characteristics of a system”. A detailed comparison of those definitions is not within the scope of this paper. Yet, the similarity between the terms and hence the described discrepancy in the literature addressing the distinction between the approaches seems understandable. Therefore, the distinction between the two terms “stylized fact” and “pattern” is not emphasized in this paper, in line with the description of Meyer [36].

Further, as Casti [37] points out, a pattern (and therefore a stylized fact) in a time series can merely be seen as the existence of a correlation between current and past observations. Therefore, the statistical analysis of a time series could be interpreted as a mathematically formalized way of finding patterns in that time series. Some approaches of validation on time series validate models against time series on the basis of statistical moments [18,32]. The quantified validation on time series data could, therefore, be seen as a validation against temporal patterns, according to the argument of Casti [37], thereby further blurring the differentiation between time series validation, pattern-oriented validation, and validation on stylized facts. This is also pointed out by Tieleman [38], who refers to approaches such as the one described by Guerini and Moneta [32] as “ultimately extensions” of the approach of validation by stylized facts in [28]. Similarly, as Hirschman [35] describes, patterns or stylized facts are often part or even the basis of expert knowledge. Hence, parallels between the aforementioned approaches and the validation on expert knowledge could also be drawn.

3.2. Recommendations for the Validation of Agent-Based Models

As described in the previous subsection, the literature on the validation of agent-based models is diverse yet presents significant commonalities. The following section aggregates relevant commonalities to draw recommendations for the validation of agent-based models. The section does not aim to present a framework or process for the validation of agent-based models. Its goal is rather to lay out general concepts and recommendations that should be kept in mind when validating agent-based models. This answers the first research question and provides the basis for the following assessment of agent-based models of residential water demand and their validation in the systematic review in Section 4. The resulting recommendations are summarized below in

Table 4. Recommendations for the validation of agent-based models.

Recommendation	Description
Objectivity	Validation on objective data should be preferred over the validation by expert knowledge
Quantification	Quantitative metrics should be chosen to perform and report the validation
Detail and abstraction of models	The more detailed a model is, i.e., the more parameters and assumptions are included, the higher is the need of validation
Multiple levels	if possible, multiple levels of system and model behavior should be compared
Purpose of the model	Models aiming to provide a basis for real-world interventions should be held to high standards of validity.
Distinct test data	Validation should be performed on data sets that are not used for building, parametrization or calibration of the model.
Sensitivity analysis	modeling assumptions, including parameters and functional structures, and interventions should be subject to thorough quantitative sensitivity analysis.

.

As explained above, the validation of simulation models seeks to compare the output of a model to an observed behavior of the modeled real-world system. Marks [39] summarizes the properties that the formal expression of the validation results should adhere to. For the review at hand, four properties are of special importance: the results of the validation should be expressed using a metric that is (i) quantitative in nature to provide objectivity, (ii) inclusive of higher statistical moments of the results to include multiple (all) differences between the model outputs and real-world behavioral data, (iii) using the same metric as the model output itself, and (iv) non-negative and symmetric. These properties provide objectivity and comparability for validation [39].

Another stipulation for the validation of models is the use of distinctive data sets [34]. This means that the data used to validate, i.e., “test” the model should be different from the data used to build the model, e.g., choose its mathematical structure or set the models parameters through parametrization and calibration. Using distinct validation data ensures that the model is not only able to describe or reproduce the data that are fit to, but also has the ability to predict data accurately [34].

While the aforementioned recommendations are of general relevance for the validation of simulation models, some recommendations can be drawn from the specific structure of agent-based models. As previously explained, one property of agent-based models is their distributed structure. Each agent-based model consists of a multitude of autonomous agents, whose actions and interactions constitute the the aggregate and emergent behavior of the model. This leads to a structure that should be assessed in multiple levels [38]. The validation of agent-based models should therefore not only assess the aggregate emergent behavior, i.e., the simulated behavior of the whole system. Rather a validation should also assess the behavior on micro- and meso-levels of the model. Micro-level validation refers to the validation of individual agent behavior and meso-level behavior to the validation of distributions arising through the interactions of said agents. Therefore, Tieleman [38] proposes a validation on the basis of three levels of system behavior: micro-behavior, i.e., single-agent behavior; macro-behavior, i.e., the behavior of the aggregate system; and meso-behavior, i.e., distributions of properties or behaviors of said agents that lead to the macro-behavior. The approach of validating an agent-based model on several levels is also shared by other authors [16,33].

Another important aspect in the validation of agent-based models is the degree of realism in a model [16]. In this review, models are referred to as high- or low- detail or realism. While this is congruent with the wording of some literature, such as that of Bruch and Atwell [16], one could also choose to refer to the models’ degree of abstraction. The terms high-realism model and high-detail model are equivalent to the term low-abstraction model. To avoid confusion, this review always refers to the “level of realism” or “highly realistic models” rather than the “level of abstraction”, even though both terms could be used. As Bruch and Atwell [16] point out, the thorough validation on multiple levels is especially important for models that are highly realistic. Highly realistic models aim to resemble a specific real-world system closely, in contrast to stylized or “low-dimensional” models that seek to model a typical system behavioral mechanism in isolation. The increased importance of validation for highly realistic models stems from two reasons:

First, highly realistic models are typically built to draw conclusions for specific real-world systems. This also makes the outputs more comparable to real-world data since the highly realistic model should also resemble the real-world system closely [16]. Low-realism models, on the other hand, typically seek to replicate general behaviors (i.e., stylized facts) and should also be validated against such general or reoccurring patterns or stylized facts [11]. Another aspect of the validation of highly realistic model is their “complicatedness”, i.e., they include a large number of aspects of the real-world and therefore pose a larger parameter space, hence increasing the potential model uncertainty [16]. Another example of this recommendation is given by Ormerod and Rosewell [40]. Here, the authors point out the importance of the size of the parameter space for the modeling of agents. They argue that modeling decisions in learning agents need to be justified and validated to a higher degree than for simpler agent structures. This is also due to the higher number of variables needed to represent a learning agent. The stipulation that a model with a larger parameter space needs more validation ultimately traces back to the problem of over-parametrization discussed in Section 3.1. The problem of over-parametrization is especially relevant if recommendations regarding political interventions are drawn from the results generated with the model [41] since this increases the need for confidence in the results of the models. Conclusively, models of higher realism and greater parameter space have a higher need for validation. The validation of stylized (low-realism) models should focus on stylized facts or knowledge, while high-realism models should also include real-world data of the specific modeled systems.

Role of Sensitivity Analysis in the Validation of Agent-Based Models

One frequently described approach to combat the uncertainty stemming from a large parameter space and the multitude of assumption of a highly realistic agent-based model is sensitivity analysis [16]. Sensitivity analysis is carried out by varying the modeling assumptions of an agent-based model and observing the following changes in model outputs [11] by varying the parameters of modeled equations, or varying the mathematical structure of those equations. The results can be used to explore the influence of input assumptions on simulation outcomes and uncertainty in model outputs [16]. A rigorous understanding of the relevance of input uncertainty on output uncertainty helps to understand which assumptions are important for the output of the model and should therefore be subject to validation efforts. Further sensitivity analysis decreases the input uncertainty by reducing the dimensions and ranges of possible parameters or equations used in the model by only selecting those that are able to reproduce the behavior of the modeled system adequately. For this review, it is important to understand that sensitivity analysis is an important part of the validation of agent-based models and can be incorporated into the validation procedure in multiple ways [17]. A detailed description of sensitivity analysis and parameter estimation in agent-based models can be found in the works of Thiele et al. [42].

While these recommendations can be seen as general guidelines, they do not present a general process for the validation of agent-based models. As many authors point out, the validation of simulation models should always be tailored to the specific model at hand and the real-world behavior it seeks to reproduce [16,28,38,39].

4. Systematic Review of Agent-Based Models of Water Demand

This section answers the research questions (ii) and (iii) presented in Section 2. For this, a systematic literature review according to the PRISMA framework is carried out, as explained in Section 2. First, the process and results of the data collection are presented, i.e., the number of records identified and filtered through the review are given (cf. Figure 2).

A detailed description of the search results and the selection process can be found in the Supplementary Material of this study. From 298 initially identified records, 207 individual records written in English were screened based on their title and abstract. After the exclusion in this step, 122 of those records were sought for retrieval, 44 of which could not be retrieved, leading to an assessment of 78 records. This screening found 52 records as appropriate for the inclusion criteria presented in Section 2. To provide a concise overview of the models in the literature, records that present a common model are aggregated based on their model for this review. For example, the model constructed in the seminal paper by Zechman [43] was used and extended by multiple publications later on [44,45,46]. Therefore, all these records were reported as one model, considering all extensions and uses proposed in the connected records. Hence, not all traits collected for a model can necessarily be found in a single record, e.g., if the building and the validation of a model are reported in separate records. In the 52 included records, 35 different models were identified and reported on in this review. (In total, 35 records were found to report on a model that was not previously covered, yet two records [15,47] reported on two new models to compare them. In the case of Berglund [15], the second model is not applicable for the inclusion criteria. In the case of Rixon et al. [47], both models were reported, yet provide similar features.) A complete list of all models can be found in Section 4.2.

The following subsections answer the research question: (ii) how is residential water demand modeled using agent-based modeling? This allows the reader to understand and find a multitude of approaches presented in the literature and builds a basis for the evaluation of validation approaches in Section 3. Therefore, the description of the models will focus on the properties of the models that are especially relevant for the appropriate validation approaches. These properties are derived of the recommendations in Section 3. Additionally, some properties are added to the review that might be of relevance for researchers searching for socio-technical models of water demand since some reviewed models are mainly focused on social perspectives (such as economic or sociological models) or the technical perspective. If a model in a screened record also models other dynamics, such as residential energy demand, those dynamics are not the subject of this review. Therefore, behaviors described in a model and the validation of such behaviors are not described in this review if the aspects are not part of the scope of this review. In the following, the relevant traits for which the models were screened are laid out. Afterwards, the manifestations found in the surveyed models are described.

4.1. Nature of Models

As discussed in Section 3, it is important for the validation of a model which phenomenon or real-world system the model seeks to replicate. In this review, such qualities of the model are called the nature of the model. To give the reader an overview of the surveyed literature,

Table 5. Purpose of the surveyed models and whether they are predictive or descriptive.

Purpose of Models	Ref.	Models	Share 1
Predictive models		31	89%
Demand side management & Policy	[15,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72]	18	51%
Infrastructure & resource planning	[73,74,75,76,77,78,79,80,81,82]	7	20%
Theoretical advancement & modeling	[83,84,85,86,87]	3	9%
Water trading	[88,89]	2	6%
Crisis response	[43,44,45,46]	1	3%
Descriptive models		4	12%
Demand reconstruction	[90,91]	2	6%
Theoretical advancement	[92,93]	2	6%

Note(s): ¹ Percentages do not add to 100% due to rounding errors.

displays what purpose the models found in the literature serve. A detailed and comprehensive list of the models can be found later in Section 4.2.

As can be seen, most models (51%) are used to predict or plan demand-side management and policy interventions. Another common purpose (20%) is using models for the planning of infrastructures and allocated resources. A smaller portion of models primarily seek to advance theoretical understanding or methods for modeling (15%), investigate water trading schemes (6%), the planning of crises response measures (3%), or to reconstruct demand profiles for further research (6%).

Beyond the insights that a model seeks, the nature of the model also plays a major role for its applicable validation. The following subsections present a deeper understanding of such qualities. Specifically, three qualities are assessed:

• Whether the model seeks to predict or merely describe behavior;
• The degree of detail or realism of the model;
• The time steps or granularity on which temporal aspects are included in the model, as this might be of interest for researchers searching for adequate models in a research endeavor.

4.1.1. Prediction and Explanation

As performed by Cominola et al. [14], the models are classified as predictive and descriptive models. As discussed by Troitzsch [34], the goal of scientific modeling is ultimately the prediction of previously unknown data. Such models are called predictive models in this review. Other models seeking to reproduce or describe existing data are called descriptive models. It should be noted that this definition of “prediction” should be used with caution. Other authors refer to “prediction” only when using the models output to give exact estimates of a systems output or state under specific circumstances (e.g., Berglund [15]). An example for such predictions in the narrow sense is the estimation of water availability in a water distribution system in the future. The definition used in this review is broader. Prediction in the presented sense also includes models that seek to understand how measures or circumstances could influence the state of a system, even if this estimate is not supposed to be seen as an accurate forecast of the system’s state.

The majority of surveyed models are predictive (89%). Such models seek to predict the development of urban water demand subjected changes, such as the adoption of water-saving technology, climate change, or water-saving behavior. Often, these predictions also include the effects of policy or management interventions, which are discussed in further detail in Section 4.2. A smaller number of models seek to describe or reproduce water demands (12%). The classification of all surveyed models into predictive and descriptive models is also presented in Table 5.

4.1.2. Level of Detail

To categorize the level of realism in agent-based models, Gilbert [11] identifies three distinct classes, which are utilized in this review: abstract models, middle-range models and facsimile models. In contrast to Gilbert, Bruch and Atwell [16] classify agent-based models into only two groups. “Low-realism models” are similar to the class of abstract models by Gilbert, and “high-realism” models are close to the description of facsimile models. For this review, the classification of models in regard to their level of realism is based on the more detailed classification of Gilbert. As previously explained, this review refers to a model’s realism instead of its abstraction. Therefore, the classes used in this review are high-realism models, middle-range models and low-realism models.

According to this classification, low-realism models are aimed at demonstrating a basic process of the modeled system that is not necessarily connected to any particular case or observable data. This class represents the lowest number of surveyed models, with only three models falling in this category. It should be noted that for these models, some of the further classifications are not easily applicable, due to their stylized nature. An example is the paper of Oh et al. [86] in which the real-time equivalent of the simulated time steps is not clearly defined. To classify the model, this review assumes that the time steps are rather long (i.e., longer than monthly) since the modeled human migration patterns would probably fall under this time frame, rather than short durations, such as hourly decisions. A deeper discussion about time steps can be found in the next Section 4.1.3.

Middle-range models are such that they represent a specific phenomenon yet stay general enough to be applied to multiple specific instances, such as different industries or geographies. Such a model might seek to reproduce the general behavior of people, e.g., an increase in water demand in the summer, but not pay attention to the detailed heterogeneity of a real society. Often, such models are used to study the relation or interdependence of multiple factors influencing the behavior or systems while staying general enough to not be associated with a specific real-world system.

Thirdly, facsimile or high-realism models are specific to a real-world instance. They are created to assess or predict the behavior of a specific real-world system as closely as possible. Such models seek to model the world accurately so that the model can be used as a “laboratory” for the real-world behaviors of a system. Examples of these models may seek to simulate the reaction of people under a specific policy or intervention. These models are often used to estimate specific results of interventions in real-world systems for management or policy evaluation or planning.

In line with the purposes described in the beginning of Section 4.1, most models seek to reproduce the behavior of specific systems or behaviors and are therefore in the classes of high-realism and middle-range models. Only three models are low-realism models [68,86,89]. A summary of all models’ degree of realism can be found in Table 6.

4.1.3. Time Steps

As digital simulation models, agent-based models work in discrete time steps. These time steps are the increments in which the agents behavior is calculated. Monthly [15,49,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,68,73,75,78,80,81,83,84,85,87,88,92,93] or longer [66,69,82,86,89] time increments are common in the surveyed models. This can be attributed to two main reasons: the timely dynamic of the behavior of interest and the granularity of data available for model building. First, many models investigate water consumption subjected to long-term trends, such as climate change, market adoption of water-saving technologies, or demographic developments. Secondly, many models rely on water billing data for model building. Since these data are typically recorded on a monthly or quarterly time frame, this granularity of the data is also used in the constructed models [59,61,62].

Shorter time increments are used less often. While daily water demand sometimes is modeled [47,50,67,70,71,72,76,77], those models typically do not include factors that differ between days, such as weekly fluctuations in water consumption. These models instead use factors similar to models of monthly water demand but calculate that demand for daily demands. A deeper insight into the used behavioral variables is provided in Section 4.2.

Models of higher dynamics include variations in the human behavior over different times in the day. The influence of daily work- or sleep schedules can be included [43]. A deeper analysis of the factors included in the water behavior models is given in the next Section 4.2.

The length of the time steps and the degree of realism are displayed in Table 6 below. As can be seen, there are no general tendencies with regard to this relation, yet only middle-range models are used for hourly or shorter time steps.

Table 6. Time steps and degree of realism used in the surveyed models.

Time Steps	High-Realism	Middle-Range	Low-Realism
Hourly	-	[43,44,45,46,48,79,90,91]	-
Daily	[50,67,70,71,72,76,77]	[47]	-
Monthly	[49,51,53,54,55,58,59,60,61,62,63,64,73,75,78,80,81,83,84,85,87,88]	[15,52,56,57,65,92,93]	[68]
Yearly	[66,69,82]	-	[86,89]

Table 6. Time steps and degree of realism used in the surveyed models.

Time Steps	High-Realism	Middle-Range	Low-Realism
Hourly	-	[43,44,45,46,48,79,90,91]	-
Daily	[50,67,70,71,72,76,77]	[47]	-
Monthly	[49,51,53,54,55,58,59,60,61,62,63,64,73,75,78,80,81,83,84,85,87,88]	[15,52,56,57,65,92,93]	[68]
Yearly	[66,69,82]	-	[86,89]

4.2. Agent Behavior

As explained in the introduction of this paper, the basis of any agent-based model is the decisions and interactions of the agents that compose the model. This section will provide a description of these agent behaviors for the surveyed models. As any (sub-)system, the agents can be defined by their input, their output and the structure or process that interconnects these inputs and outputs, i.e., the internal logic or functions of the agent. Therefore, this section describe the following aspects of the agents used in the agent-based models surveyed in this review:

• The output variables of each agent, i.e., the decisions they make.
• The input factors of each model agent, i.e., the information they use for their decisions.
• The decision structure, i.e., which kind of formula or logic is used to derive the output from the input.

The results of this subsection are summarized in

Table 7. Output variables, input variables and structure of agent decisions. Output variables: AD = aggregated water demand, CS = consumption, AP = appliance, LO = location. Input variables: Pol. = policy, Wth. = weather, Sea. = seasonal or diurnal changes, Soc. = societal influence, Hou. = housing characteristics, Out. = outdoor demand, SEC = socio-economic factors, No.P. = number of people/inhabitants aggregated per agent (S = single person, H = household, A = aggregated demand/larger area), Min. = minimum demand. Decision structure: Dis. = discrete (if not marked, the output variable is continuous), Sto. = stochastic, Mult. = multiple decision structures in one model.

Model	Output Variable				Input Variable											Decision Structure
	AD	CS	AP	LO	Pol.	Wth.	Sea.	Soc.	Hou.	Out.	SEC	Eco.	Min.	Oth.	No.P.	Dis.	Sto.	Mult.
Aguirre et al. [90]		×		×											H	×	×
Alvi et al. [48]		×				×					×				S	×	×
Athanasiadis et al. [49]	×				×			×			×	×	×	×	S
Barthel et al. [83,84,85]		×	×		×	×		×			×			×	A	×		×
Berglund [15]	×				×					×			×	×	H	×
Chu et al. [50]		×	×					×	×		×	×		×	H	×	×	×
Daniell et al. [73,74]	×										×				H		×
Darbandsari et al. [51]	×				×	×		×				×	×		H	×	×
Downing et al. [52]		×	×		×			×						×	H	×
Edmonds et al. [92]		×			×	×		×						×	H	×
Edwards et al. [93]		×						×						×	?	×	×
Faust et al. [75]	×							×				×			S	×
Galan et al. [53,54,55]	×	×	×	×				×	×		×				H		×
Giacomoni et al. [56,57,58,59,60]		×	×		×	×		×	×	×	×			×	H	×	×
Hung et al. [88]	×								×		×	×	×		H
James et al. [61]		×	×		×			×	×	×				×	H	×	×
Kandiah et al. [76]			×					×	×	×				×	H	×	×
Klassert et al. [77,78]		×							×		×	×	×	×	H
Koutiva et al. [62,63]		×				×	×	×				×	×	×	H	×	×
Linkola et al. [91]		×			×						×			×	S	×	×
Liu et al. [79]		×		×			×		×		×			×	S	×	×
Nikolic [80,81]				×										×	S
Oh et al. [86]	×			×								×		×	S			×
Ozik et al. [64]	×				×	×	×	×				×		×	H		×
Perello-Moragues et al. [65]		×	×				×	×	×		×			×	H	×
Perugini et al. [89]	×										×	×			H
Ramsey et al. [87]			×			×		×						×	A	×
Rasoulkhani [66]			×					×	×	×	×	×			H	×	×
Rixon et al. [47]	×	×			×	×	×	×	×	×	×	×		×	H/S	×
Sampson et al. [82]	×			×	×				×	×	×	×			A		×
Searle et al. [67]	×					×		×	×	×	×	×		×	H	×	×
Shafiee et al. [43,44,45,46]		×		×	×		×	×			×			×	S	×	×
Tourigny et al. [70,71,72]		×	×		×			×	×	×	×			×	H	×	×
Xiao et al. [68]	×				×							×		×	A
Yuan et al. [69]	×				×			×			×	×	×		H	×	×
Quant. Summary	43%	51%	31%	20%	40%	29%	17%	63%	40%	26%	57%	42%	20%	69%	n.a.	69%	57%	9%

. This review does not assess whether agents are learning or non-learning since none of the surveyed models focused on learning about residential water consumers in a meaningful way (in the surveyed models, there are two exceptions to this in which agents adjust their price expectations, which can be seen as a simple form of learning [64,92]). As pointed out in Section 3, this distinction might be highly relevant in a different context or future work, as more extensive learning behaviors introduce a larger number of independent variables into a model.

4.2.1. Output Factors of Agent Decisions

The output factors of the agents effectively define “what the agent does” or “what the agents decide on”. Therefore, the outputs are substantial for the description of the agent behavior. Since this review surveyed models of water demand, all models of agent behavior describe or predict the amount of water consumed by the humans represented by the agents.

In total, 43% of models describe this relation directly, i.e., predicting an agent’s aggregated water demand as a function. Often, the water demanded by an agent is only calculated as a result of the decisions each agent makes (51%). These decisions include agents choosing when or if to engage in water-consuming activities and agents deciding to adopt water-saving behaviors, such as reducing outdoor water demand for lawn irrigation. A further common output is agents’ decision to adopt water-saving end-use devices (e.g., toilets or shower heads requiring less water) (31%).

Another decision made by agents in some models is their location (20%). By choosing their location, the agents affect the spatial domain, where their individual demand is located. For an agent to choose its location, these models are also spatially explicit as indicated in Section 4.3. Yet, not all spatially explicit models also model agents’ decisions to move in this spatial dimension, i.e., those models display spatially distributed but stationary agents.

4.2.2. Input Factors of Agent Decisions

The biggest diversity of classes in this review are provided by the input factors of the agents’ decision. The input factors are variables that agents “consider” when making decisions about their water consumption or demand.

Since many models seek to assess the effect of policy interventions on the behavior of socio-technical systems, the agents’ reaction to policy is often modeled (40% of models). Examples of such policy reactions are agents reducing their water consumption if public announcements to save water are made by governments. This class does not include indirect influences of policy on consumer agents’ decisions, e.g., agents reacting to the price of water if it is set by a government agent.

Other common inputs are weather factors, such as temperature or rainfall quantity (29%). Similarly, some models include seasonal patterns (such as lower water consumption in the winter) or diurnal patterns (such as lower water consumption during the night) (17%).

The majority of models include some degree of social influence on the behavior of agents (63%). Social influence means that the behavior and decisions of each agent directly depends on the decisions made by other agents. Any model that includes such a social impact on the behavior of agents effectively models some sort of “diffusion” through the social network, i.e., the spreading of behaviors or decisions in the population of agents. Other terms used to describe such phenomenon are “adoption”, “contagion” or “memetics”.

Further input factors were identified in the surveyed literature: the structure of housing, such as the building type or building volume (40%); the use of water in outdoor areas, such as lawn irrigation (26%); socio-eoconomic factors, such as income or social class (57%); economical considerations, such as the monetary savings provided by water-saving decisions (42%); and minimal demand thresholds (20%). Beyond the classified factors, 69% of models also include various other factors that would be too diverse to present here in detail.

Lastly, this review found three different levels of aggregation in terms of the number of people that are represented by one agent. Some models represent one person by one agent, allowing a transparent linking of human behavior and decisions to agent behavior. Yet, most models (63%) aggregate all the people in one household to an agent. This is done since the decisions of individuals within a household are often hard to separate since water-billing data are aggregated for households as well. Further, some decisions, such as the installed water fixtures, affect the water demand of all inhabitants. Lastly, some models aggregate all people within a larger geographical district to agents. This is often done to reduce the computational complexity of models representing larger geographical areas. For the model by Edwards et al. [93], it could not clearly be defined which aggregation level is picked since it is not clearly stated (or necessary for the model) whether singular humans or a group of people is represented.

4.2.3. Structure or Logic of Agent Decisions

The diversity of input and output factors is also represented in the myriad of functional structures linking those in- and outputs, i.e., the logic on which agents base their decisions. Generally, two structures were identified: decisions, which represent an agent’s ability to choose between distinct actions and, secondly, agents that are able to choose a value of continuous variables.

One common logic behind agents’ (discrete) decisions found in the literature are based on agents deciding whether to cooperate with other agents [51,53,54,70,71,72,87,93]. Here, agents chose between the incentives to cooperate, such as peer pressure, and the benefits of exploiting other agents’ willingness to cooperate. (Similar models are often referred to as “game theoretical” since cooperation/non-cooperation decisions are often the subject of game theoretical assessments). These models are mostly based on the work of Young [94] and the extensions thereof by Edwards et al. [93]. Another decision logic is based on the works of Ajzen [95] and utilizes the “theory of planned behavior” [65,83,84,85,90]. This theory combines the societal norms and attitudes of an agent with the agent’s perceived behavioral control. Another functional form that does not rely on psychological models is the model by Blokker et al. [96], which is used to calculate stochastic water demand profiles [70,71,72,90]. While these functional forms are predominant, also other decision structures are used, e.g., the social impact theory by Nowak [97] used by Koutiva et al. [62,63] and the Bass model [98] used by Galan et al. [53,54].

Continuous variables are mostly used for the decisions of aggregated water demand discussed in Section 4.2.1. These continuous variables are typically calculated by regression. The functional form of these can vary, e.g., linear or logarithmic regressions. A thorough discussion of these functional forms is outside of the scope of this review.

Lastly, in some cases, multiple decision rules are used within a model [50,83,84,85,86]. This can be done to assess how policy interventions affect populations of heterogeneous decision structures. While similar models are prominent in economics (most notably, the “Santa Fe Artificial Stock Market” based on the seminal work by Arthur et al. [99]), this field is not extensively covered in the context of residential water demand.

Regardless of decision structure, most models include stochastic aspects. This can be modeled in two ways: either the output of a decision is a probability, or the decision structure is a probabilistic equation, i.e., if the decision is represented as $d=f\left(x\right)$, either d could be a probability or f could include a probabilistic term. As discussed in Section 3.1, stochastic models are subject to the uniqueness of time series. In total, 31% of the surveyed models do not implement any stochastic aspects into the decision structures of agents, although sometimes other stochastic elements are included in the models.

4.3. Environment

In models, every aspect external to a system is typically called its environment. In agent-based models, the environment therefore refers to the aspects that are modeled, yet not part of the description of the agents. Every influence on the behavior of an agent can be described as the environment of the agent. To avoid redundancies with Section 4.2, this section focuses on such aspects that are not implied by these input factors. For example, social influences on the agent’s behavior imply that there is some kind of social network implemented in the model, agents’ reactions to rainfall imply that there is some variable of “rain” implemented, etc.

Therefore, two aspects of the environment are assessed for this review: how spatially explicit the model is (Section 4.3.1), and to which degree a technical environment is included (Section 4.3.2). The results of this assessment are summarized in

Table 8. Traits of the modeled environments. This table references the specific records, where the models environments are reported on.

Type of Environment	Spatially in-Explicit	Unspecifically Spatially Explicit	Specifically Spatially Explicit	Share
Extensive technical environment	-	-	[43,44,45,46,62,63,64,70,71,72,76,79]	17%
Simple technical environment	[15,48,68,93]	[52,55,92]	[56,57,58,59,60,73,77,78,80,81,83,84,85]	34%
No technical environment	[47,65,67,75,82,87,88,89,90,91]	[47,49,66,69] 1	[50,51,53,54,61,86]	51%
Share	40%	20%	46%

Note(s): ¹ Rixon et al. [47] are cited twice since two models are reported in this record. Additionally, Galan et al. [53,54] are cited disconnected from the record of López-Paredes et al. [55] since the first are ultimately extensions of the latter, but report different traits of the environment. Therefore, the percentages add to more than 100%.

by referring to the records in which each type of environment has been reported on. Figure 3 presents the number of models in each category of modeled environment.

4.3.1. Spatial Explicitness

The models can be distinguished into three groups of spatial explicitness. The first group is spatially inexplicit (40%). In this group, no influence of spatial aspects on the behavior of agents is included in the model. Secondly, there are spatially explicit models with a generic spatial dimension (20%). In this review, a generic spatial dimension refers to regular geometric structures along which the agents can interact with each other or their environment. While rare in the surveyed literature, examples of such generic spatial structures are lattices (“check-boards”) or tori (“doughnut-shaped spaces”) on which the agents are positioned. Lastly, fully spatially explicit models exist (46%). Since this structure is common in highly realistic models, models of this group typically display all agents on a map of a geographic region. This region is often mapped according to real geographic regions, or exemplary maps modeled to portray the properties of typical real geographical regions.

4.3.2. Technical Environment

As this review focuses on socio-technical models, special emphasis is put on the modeling of technical components in the surveyed literature. The majority of surveyed models do not explicitly model a technical environment, focusing solely on the change of the human components in the socio-technical system, i.e., the change of water demand. In models that do explicitly model technical components, two groups were identified: The first group exclusively focuses on the mass balance of water reservoirs (34%) to model the total consumption of water resources by a socio-technical system (i.e., a urban community). Those models are referred to as including a “simple technical environment”. More detailed approaches with an “extensive technical environment” use hydrologic models of the water distribution systems (17%) to represent the technical environment. These models often aim to provide detailed insights in the management or function of water distribution systems and are typically connected with shorter time steps discussed in Section 4.1.3. This seems plausible since the hydrologic behavior of a water-distribution system is typically characterized by the dynamics of shorter time scales, and the models used operate on time scales that are shorter than a day.

In this review, appliances are not considered a technical environment since they are typically modeled as a trait of the agent mediating its water demand. In contrast to this, an environment is clearly separate from the agent and interacts with it. This trait is not portrayed by the appliances.

4.4. Validation of Surveyed Models

As discussed in Section 3, the validation of agent-based models can be approached in various ways. Section 3 provides a discussion of the scope of this review regarding validation. The following section presents which validation approaches are most common in the surveyed literature. The sections results are aggregated in

Table 9. Validation reported for the surveyed models.

Validation Approach	Expert Knowledge	Time Series	Patterns and Stylized Facts
Qualitative	[50,52,55,67,73,75,83]	[47,49,64,66,75,91]	[52,62]
Quantified	-	[50,51,53,54,60,61,62,69,77,87,90]	-
Multiple levels	[83,84,85]	[53,54,60,90,91,92]	[86]
Distinct test data	[83,84,85]	[50,51,60,61,62,64,90,92]	-
Sensitivity analysis	[77]	[15,44,47,50,55,62,64,65,66,71,72,76,82,87,89,92,93]	[86]

.

In this section, the models are classified in two dimensions according to Table 3 and Table 4. Regarding the validation approach, the models are classified into validation based on expert knowledge, time series, and pattern or stylized fact. The last class consolidates the approaches of pattern-based validation and the validation on stylized facts. This is done due to the similarity of the approaches described in Section 3 and the fact that it is the least common class found in the surveyed literature. In the dimension of recommendations, the models are classified on whether they are validated utilizing qualitative validation, quantification, multiple levels, distinct test data, and sensitivity analysis. The class of models validated by “qualitative validation” aggregates such models that are reported to be validated, yet none of the other classes applied to the reported validation.

The recommendations to objectively validate, paying attention to the level of realism and purpose of the model, are not used in the classification since adherence to these recommendations cannot be judged on a polar criterion. Rather than a criterion that validation should adhere to, these recommendations are considered general goals of validation in this review.

For some of the surveyed models, no form of validation is reported [48,68,79,80,81,88]. For those models that are formally validated, many are validated using expert knowledge. However, only a small portion of the models relied solely on this form of validation [52,67,73,83,84,85]. As explained in Section 3, there are multiple approaches to perform and report on the validation of expert knowledge. One common way in the surveyed literature is to ask stakeholders in the modeled systems to judge the models’ micro- and macro structures (i.e., the agents decision rules and the produced model outputs). However, there are also other approaches, such as interviews, or utilizing games or interactive simulations [47].

Another common approach is time-series validation. In Table 9, also the validation on other aggregated statistics is described as “time series validation” since the methods for this are comparable to the time-series validation and cannot be seen as a validation of stylized facts or expert opinion. In some cases, time-series validation is performed by qualitatively comparing model outputs to real time series [47,49,64,66,75,91]. Those models quantifying the comparison use measures such as the mean absolute error [51,62], root mean squared error [51], the Nash–Sutcliffe coefficient [62,100], the R² and statistical tests, such as the F-Test and p-value [53,54] or various other measures [90].

Only a few of the surveyed records describe some kind validation using characteristic patterns, i.e., stylized facts [52,86]. In the case of Downing et al. [52], the leptokurtosis (heavy tailedness) of the distribution of water demands is used as a standard for the water-demand models. Oh et al. [86] assess the influence of different parameter sets (factor configurations) on the emergence of patterns in migration and water demand. Yet, validation in the sense of a comparison to empiric data is not the focus of their paper. They report that the described models did create the chosen patterns while not providing quantitative numbers. The further validation is performed by expert knowledge, as described above.

Sensitivity analysis is extensively used in the surveyed literature. Sensitivity analysis is used to vary the parameters of evaluated interventions on the socio-technical systems, and in many cases, to vary some of the relevant parameters assumed in the building of the model. A variation of the underlying structural assumptions of the models is rarely performed. Exceptions to this are the assessment of alternative social structures (e.g., Rasoulkhani et al. [66]) and the assessment of the sensitivity of results to differing agent decision structures [46,86,93]. It should be noted that in some cases, a sensitivity analysis is performed without comparing the outcomes to empirical data [86]. While sensitivity analysis is not a validation in the classic sense, it can still serve to improve confidence in the results of the study by providing insights into the robustness of the results.

4.5. Comparison to the Recommendations for Model Validation

To answer the last research question (iv) How are the recommended principles for the validation of agent-based models applied in the validation of the surveyed models?, the results of the survey reported in Section 4.4 are compared with the recommendations outlined in Section 3. Therefore, the recommendations are briefly summarized:

• Models of higher realism need more validation, especially if they aim to provide the basis for real-world interventions.
• Time-series validation can be used for high-realism models, pattern-oriented validation should be used for stylized models and can also be used for high-realism models to some degree.
• Validation on expert knowledge tends to be subjective.
• Validation should be quantitative on multiple levels, and data should be distinct from the calibration data.
• Sensitivity analysis should be used to test the confidence of the results and assess the sensitivity to alternative modeling assumptions and parameter sets.

In contrast, for some models, no validation is reported. Those models on which validation is reported, validation on expert knowledge or time-series validation are common approaches. Pattern-oriented validation is used in three cases, yet only in the case of [86], more than a qualitative evaluation of the model is conducted. Not all models are validated on multiple levels using distinct test data and reporting quantified validation results. Sensitivity analysis is often used, mostly to assess differences in the effects of interventions. An assessment of parameter sets is less common, and sensitivity analysis for the assessment of structural modeling assumptions is rare.

Figure 4 presents the relation between the chosen approach of validation and the degree of realism in models. As can be seen, the majority of models are middle-range and high-realism models validated by expert knowledge and time-series validation.

5. Discussion

A variety of different models of water demand and underlying psychological theories are identified in this paper. The complexity of human behavior can be expected to produce such a multitude of functional relationships and include various independent variables. Therefore, no “best practice” models for residential water demand can be identified easily. The models appropriate for a specific socio-technical model of water demand are therefore highly context-dependent, highlighting the need for the validation of each model. This heterogeneity of approaches has been described for water-demand modeling in general [14] and specifically for agent-based models [15]. The presented paper extends these prior works by a more comprehensive survey of the literature and by detailing which variables and modeling approaches are most common in the surveyed literature.

Approaches for the validation of agent-based models described in this paper include the validation on the basis of expert knowledge, time series, stylized facts, and reoccurring patterns. While these approaches present a significant overlap and similarities, general recommendations can be drawn for the validation of agent-based models. To obtain objective perspectives on the validity of models, the specific approach of model validation should be tailored to the goal and degree of detail of the model. Therefore, a quantitative approach should assess multiple levels of the model on data sets distinct from the data used for building the model. Further, a sensitivity analysis should be used to identify and discuss the effects of modeling assumptions and certainty of the results. The need for a more thorough validation of agent-based models is often pointed out in various works [14,15,23]. This paper extends this call by providing specific and more comprehensive recommendations for how such a validation might be approached.

A multitude of agent-based models of water demand are presented in the scientific literature. While rigor is applied in the building of the models, validation often is either not reported, not performed or done on the basis of qualitative expert knowledge. Yet, some examples of quantitative validation on time series data exist. On the basis of these assessments, three recommendations can be identified for future research:

(i)

Validation should be prioritized to increase confidence in models and results. Exploring approaches to validation such as the pattern-oriented approach could be promising for agent-based models of water demand. Validation should be quantitative, on multiple levels of the model, on data distinct from calibration data, and include sensitivity analysis. These recommendations underline the indications of previous research of a need for more rigorous validation and assessment of equifinality in agent-based models of water demand [14]. While the aims of scientific modeling can be diverse [101] and validation need might vary for different goals [102], the confidence and associated uncertainty in model results should be considered and communicated when research is the basis of policy recommendations [41].

(ii)

Incomplete reporting was a challenge to this review. Therefore, reporting on the building and validation of models should be aligned with standardized reporting structures. In conducting this review, the use of standards, such as the “Overview, Design concepts, and Details (ODD) protocol” [103], was found to increase the transparency of modeling choices and the ease of handling papers.

(iii)

The interdependencies of human behavior and technical systems should be taken into consideration. While socio-technical perspectives are present in some studies, many models focus either on the technical or social science perspective. As Scharte [4] points out, the socio-technical perspective is vital for the understanding of infrastructure systems, and as Nikolic et al. [31] underline, agent-based modeling is a promising approach for such socio-technical investigations. However, no standardized or “best” model of the social components in a model can be identified. The appropriate models will likely always be highly context and case specific. Therefore, it is indispensable for future research to include interdisciplinary discourse and perspectives of non-technical academic disciplines, such as economics, sociology and psychology.

However, the comprehensiveness of the assessments in this paper is potentially limited by the queries and strategies of the literature search. A significant number of the records found during the search in databases were not recovered by the authors. Incomplete recovery of records found in the database search could have introduced a bias into the assessments of this study by potentially excluding relevant literature. Additionally, a significant portion of the assessed records was found by snowball search. This indicates that the search queries may not sufficiently cover the body of literature of the surveyed topic. However, since also more than half of the identified records were excluded during the screening of the abstract or full-text, the search queries also show to not be very restrictive, potentially qualifying this limitation.

Another potential limitation stems from the narrow definition of the term “validation”. As previously discussed, some authors claim that the external validation of agent-based models of human behavior is unnecessary or impractical. Therefore, the results of this review should be seen as a basis for future research, rather than a judgment of the validity of the results of the surveyed records.

While the surveyed literature might be biased, this review still merits a comprehensive overview of the models and validation approaches in the literature that was assessed. Therefore, the results and implications are still relevant for future research by pointing out missing links to scientific disciplines that address the highlighted issues.

6. Conclusions

The results of this paper (i) compile recommendations for the validation of agent-based models, survey agent-based models of residential water demand in the scientific literature regarding (ii) the models used to describe water-demand behavior and (iii) the utilized validation strategies, and (iv) assess how the recommendations regarding the validation (compiled by question (i)) were applied in the validation of the surveyed models.

For this, the paper compiles recommendations for the validation of agent-based models from several academic disciplines. It therefore emphasizes the need for quantitative assessment of models on multiple levels of the model. Further, data relevant to the behavior of interest and distinct from the data used for building the model should be used. Lastly, sensitivity analysis should be used to provide confidence and estimate the certainty of the results and the sensitivity to alternative modeling assumptions and parametrizations.

Agent-based models of residential water demand in the literature are diverse. Models predicting water demand on longer time increments (equal to or greater than a month) are most common. The models are mostly used to support the planning of policy and management interventions to decrease water demand or to plan the allocation of resources and infrastructure. The models are often highly realistic, meaning that they seek to model specific real-world systems. These findings extend the prior literature by the use of a larger sample of surveyed literature.

Further, this review deepens existing reviews by focusing on the used validation approaches and giving specific recommendations for the future validation of models. The (external) validation of these models was, in many cases, not extensively reported. Further, approaches based on face validation and expert knowledge were common. While more extensive validations were also presented, stronger validation and sensitivity analysis of findings should be encouraged for future research in the area of agent-based models of residential water demand.