5.1. Validation Results
The model itself uses data from the real-world system it focuses on. Indeed, all shapefiles are from the SITG portal for geomatic data about the canton of Geneva, and the model incorporates the spatial positions and other variables of all the objects it uses. Key values, such as the spontaneous job change rate and the travel time by public transport from one stop to another, are derived from the real-world data, such as the OCSTAT statistics and the Swiss public transport API available to all passengers. These design choices enhance our model’s conformity with the real system’s functioning.
Furthermore, we compare the dynamics of simulation 6 in terms of relocation rate, with the real-world data (for methodology, see
Section 4.5). We chose to use this simulation as the validation of our model since it incorporates the population growth factor. In this simulation, five commuter agents are added to the simulation every cycle. This corresponds to 500 new people in the system every month, or 6000 per year, when factoring in the commuter granularity default value of 100. This figure is in line with the range of values for demographic growth in the canton of Geneva: between 5500 and 6500 new residents per year (data from OCSTAT). The validation results are shown in
Figure 4.
After running the simulation for three full years (36 months), despite some differences in the shape of the curves, the orders of magnitude are coherent between both curves. The differing shape of the simulation data curve can be explained by two peculiarities in the model’s functioning. First of all, the initial patience of all commuters in the system is set to at least 10 at initialisation. This patience decreases by 1 each cycle if the commuter is unhappy (see Algorithm 2), but nothing happens until the patience value becomes negative. This explains the flat curve from cycle 0 to cycle 9. Then, at cycle 10, the relocation requests from unhappy commuters are triggered, and commuters start relocating, which is when the simulated data curve starts adopting the same behaviour as the validation data curve. In parallel, the population growth induces a reduction in the amount of available housing and job places. The government agent reacts by creating 10 homes in all PDC areas suitable for housing (see Algorithm 5), which prompts a massive increase in the number of relocation requests. This is visible at cycle 15 on the plot. Finally, from cycle 16 onwards, the simulation is characterised by a steady increase in the number of relocations, which is driven by the regular influx of new commuters in the system.
On the other side, the validation data assume a constant relocation rate that is not influenced by any modelling assumptions or behavioural gimmicks. When discarding the peculiarities pertaining to our model’s behaviour, the orders of magnitude of the simulated data and the real-world data are coherent, and the average amount of relocations over three full years in both datasets are in line with each other.
While our model’s components are based upon real-world data, the elements presented in our validation process are not sufficient to guarantee a complete validation of our model, but rather a partial one. They ensure our model is in general agreement with reality, and that it has been built on the basis of real-world data, but leave out some margin of error that prevents the model from providing a flawless and accurate quantitative analysis of some phenomena (especially future ones). Strengthening our validation process would require further studies and resources that lie outside the scope of this paper. We discuss possible works in this direction in
Section 5.4.
5.3. Simulation Data Analysis and Discussion
All plots and maps presented in this section are directly derived from the web application for data analysis and exploration. For each simulation output dataset, the app may be launched by running the
app.py python script in the corresponding folder. All simulation output datasets are included in the ‘
OutputDatasets’ folder in the shared repository (
https://gitlab.unige.ch/cas/Flann_ABM/mdpi-dpsir-model-1, accessed 12 June 2024) for this paper.
We present and discuss an overview of the possibilities offered by the data analysis web application to paint a clearer picture of how well it blends in and complements our agent-based modelling endeavours. In
Figure 5, we present a map of the average rent prices across the canton, obtained at month 1 of simulation 6. For each address, the size of the circle represents the number of commuters residing there in the simulation, and the colour of the circle represents the average rent price, expressed in CHF/m
2. This choice of symbology is freely left to the user, who can easily change what they wish to observe on the map thanks to the radio items menu pictured on the left side of
Figure 5.
As the commuter granularity factor was set to 100 (on average, one agent represents 100 people in real life), only addresses with a substantial amount of residents are represented on the map. We observe a general trend of the rent price and the population density increasing as one approaches the city centre, as indicated by the large yellow and orange circles. The rent prices are generally the lowest at the western peripheral part of the canton, in communes such as Vernier and Onex.
The inclusion of this particular map highlights one of the strengths of agent-based modelling and the associated simulations. We started with aggregated statistical data, and spatialised this dataset by using geomatic data processing tools. We then ran the simulation and used its self-organisation properties to obtain the spatial distribution of commuters with our modelling assumptions. Thus, agent-based modelling is able to both spatialise and enrich statistical datasets, which can have a myriad of follow-up applications.
Displaying the spatial distribution of and relationship between other key indicators of the model is as simple as changing the selection of items in the radio items menu pictured on the left panel of
Figure 5. For instance,
Figure 6 represents the amount of commuters having moved in and out of each address for simulation 4 (month 36). From this map, it is possible to have a glance at which spatial locations experience the most relocation dynamics. For instance, addresses located towards the centre of Geneva are generally preferred when moving in, indicated by the circles being generally larger towards the city centre and smaller at the peripheries. However, housing availability and high rent prices may severely limit the amount of commuters who are able to move into these addresses, despite initially wishing to do so. From the yellow and orange colours of the circles, one can deduce that the homes from which the most commuters decided to move out of are located in Meyrin (west, towards the airport) and Versoix (north), with two specific addresses towards the city centre displaying the highest count.
As pictured in
Figure 7, a map of the canton also exists for the commuters themselves. This map offers the same possibilities for a user-defined symbology as with the addresses map (as seen on
Figure 5).
Figure 7 represents the happiness status of commuters at month 5 (left) and month 36 (right) of simulation 4. The first observation that can be made is that the overall satisfaction of commuters in the system increases, with visibly fewer unhappy commuters at month 36 than at month 5. This is directly correlated with the relocation dynamics of the model, which implies unhappy commuters systematically seek a suitable home and move out when they find one. These two maps also give context to the observations made above for
Figure 6. Meyrin, Versoix and two particular addresses in the city centre saw the most commuters moving out. While this fact is correlated with the high count of people residing in these areas, the analysis of the happiness map at these specific locations allows us to identify a spatial correlation between the happiness status at the start of the simulation and the move-out probability. Indeed, the town of Versoix is initially marked by an extremely high proportion of unhappy commuters, as is the case with Meyrin and the city centre, albeit at a lower degree. This highlights another strength of building spatially explicit plots as part as our model’s results analysis process: the ability to pinpoint spatial correlations between key indicators of our model.
The data analysis app also quantifies the evolution of these indicators through time.
Figure 8 shows the evolution through time of the average travel time (left) and happiness status (right) of commuters in the system for simulation 4. While the average travel time steadily decreases after the commuters start relocating to a more suitable home, the proportion of satisfied commuters increases in a similar manner. This behaviour may be observed in all other simulations, and quantifies the trend identified above for the amount of happy commuters on the map (see
Figure 7) in terms of the function of time. As a side note, this interactive plot module on the web application is also equipped with a radio items menu, which lets the user choose and cycle through the various key indicators in order to easily visualise their evolution though time.
We now analyse, for each simulation, the evolutions of the amount of relocations in the system and contextualise them with the discrepancies between each set of initial conditions. This will let us appreciate the sensitivity of our model to changes in the initial conditions. For all six simulation output datasets, we plot the relocation rate in the system in terms of the function of time (
Figure 9,
Figure 10 and
Figure 11). The real-world data curve is also represented in green for context.
Simulation 1 corresponds to the most basic setup, with no population growth, and no desire for a low built density environment. Once the patience of the unhappy commuters runs out at month 10, a rapid increase in relocation rate is to be seen, which eventually settles to a much lower rate.
Simulation 2 divides the average initial patience by 2. As a result, the relocations happen as early as month 5, which can be seen on the plot.
Simulation 3 reinstates the minimum initial patience value of 10, and introduces a new profile of commuters: 10% of the commuters will prefer living in an environment with a built density strictly less than 10% of the maximum built density in the canton:
In return, the travel time threshold for these commuters is set to 60 min instead of 35 min. The plot of the relocation rate for this simulation shows a smoother curve than for simulation 1. The initial increase happening at month 10 onwards is not as sharp, and there are overall fewer relocations than in simulation 1. This is because of the threshold rise for part of the population: some commuters that were unhappy because their travel time exceeded the low threshold may become happy if their threshold value is raised.
This behaviour is further shown on the plot for simulation 5. In this simulation, the proportion of commuters preferring a low built density but accepting a higher travel time threshold is brought to 20%. The low density threshold is also brought to 20%. The plot shows that the increase in relocations is much smoother, and the total amount of relocations is the lowest of all simulations.
In simulation 4, as the proportion of commuters preferring a low built density but accepting a higher travel time threshold is kept to 20%, but the low built density threshold is brought back down to 0.1%, the increase in relocations is sharper and the amount of relocations is higher. This is because commuters may be unhappy not only because of a high travel time but also because the surrounding built density is too high. This cause is more likely to happen than in simulation 5 because of the lower threshold value. This results in a higher relocation likelihood.
Finally, the behaviour of simulation 6 has already been discussed in the validation section of this paper (see
Section 5.1). The large step that can be seen is linked to the sudden creation of a large amount of homes in the PDC areas by the government agent, and the steady increase in relocations that happens afterwards is linked to the continuous influx of new commuters in the system. The establishment of these new residential buildings impacts the sustainability of these regions as they influence metrics about the built-up area.
This section would not be complete without taking a look at the results of simulation 6. As a reminder, the goal of this experiment was to determine which addresses are the most attractive for incoming commuters (a result of a steady population growth). The government agent was tasked to create an arbitrarily large amount of new homes in all PDC areas.
Figure 12 shows, in yellow circles, which locations were chosen by the arriving commuters. Only the newly created addresses are shown. The eastern part of the canton around Chêne-Bourg, the northwestern axis towards Meyrin and the southern part of Lancy appear to be the most attractive. Housing in these areas is on average cheaper than around the city centre of Geneva, as can be seen on
Figure 5. Since, in our model, the commuters’ happiness and willingness to move into a given home is heavily tied to the travel time to work by public transportation, the attractiveness of the first two regions can be explained by the presence of the trams 12 and 14/18, respectively, and for the southern part of Lancy, the proximity of the two train stations of Lancy-Pont-Rouge and Lancy-Bachet, which all allow for fast and convenient access to the city centre, where the most workplaces and overall businesses and shops are located.
To answer our third research question, a visually appealing presentation of results and data is highly valuable for the agent-based modelling field [
9]. We have developed and used a data exploration platform, and showed that compelling data visualisation tools may help achieve multiple tasks, such as the spatialisation of aggregated data and the identification of spatial correlations. Efficient visualisations are capable of showcasing and explaining key emergent behaviours observed in the model—a remarkable trait when considering the critical and exigent need for an accurate assessment of the sustainability of urban systems [
4,
5,
6]. Creating a web application for data visualisation and analysis is one possible way to ensure an effective dissemination of our study’s results.
5.4. Moving Forward: Shortcomings and Possible Improvements
To enrich our closing discussion, we point out some shortcomings of our study and steps towards addressing them, and discuss possible works that would improve this study.
Much like a double-edged sword, one limitation of the model and its subsequent simulations resides in the use of the API for computing journey travel times. These data are fixed at a particular recent date, which is adequate for representing the system in recent timelines and grounding our model in the reality of the system. However, this facet makes it difficult to use the model to represent past trends. For this accomplishment, the use of the model as is would come with the flaw of an inaccurate representation of the public transportation system state at the given date of the study. Since the public transportation API does not allow a lot of freedom in the dates used for computation, a recreation of the public transportation network at different points in time, and the computation of the travel times at different points in time based on this remodelling, would have to be systematically undertaken for the model to be able to accurately accommodate different timelines than the present.
As discussed in
Section 4.5, we could only achieve a partial validation of our model. The discrepancies between the simulated data and the real-world data were explained but subsist nevertheless. Integrating stochasticity in the agents’ behaviours could smooth out the large jump in the amount of relocations observed in the simulation, and place the model in better agreement with reality. For instance, people agents may have a probability of moving out in each cycle they are unhappy (i.e., their patience value is negative), instead of automatically doing so as soon as their patience runs out. Another interesting behaviour to investigate would be the government agent’s: instead of making a large amount of homes appear at every possible address, a smaller amount of new homes would be spawned randomly in a limited selection of addresses.
However, this would add more parameters to consider when building the model, which highlights another difficulty: the sheer immensity of the parameter space that usually comes with any agent-based model, and the daunting task of calibrating the values of all of these parameters [
15]. One way to tackle this challenge is participatory modelling. The involvement of experts on the topic, policy makers and any other party capable of providing valuable insights, right from the start of the agent-based modelling project and during the entire model development phase, can help us reduce this parameter space to focus on processes that garner the most interest [
27]. They can suggest the inclusion of this or that parameter and set it at such a value, or of some key behaviour, and help design relevant what-if scenarios. Calibration is greatly facilitated and backed up by knowledge and evidence. It follows that the validation process can also be enhanced by the guidance of experts, which further grounds the model in the reality of the terrain.
Participatory modelling can be seen as a major keystone of efforts to increase the policy-making guidance powers of agent-based models. In that context, opportunities arise from embedding agent-based models into digital twins by providing the necessary accompanying infrastructure. Digital twins may represent concrete manifestations of an agent-based model being used to inform governmental parties, develop new policies and support urban planning [
10,
21,
28,
29,
30,
31,
32,
33]. We plan to investigate this matter in future research. On a side note, the works conducted in the scope of this study have piqued the interest of local authorities (more specifically, the GE2050 consortium, an initiative from the canton of Geneva to tackle sustainability by 2050), which could lead to future collaboration of this sort.
The pipeline also shows the possibility of directly querying the API during the simulation runtime for building a connected model that could be used to represent real-time movements of commuters in the system.
Another improvement for our model would be the inclusion of more attributes for our people agents, such as age, gender or job type. These factors can considerably influence personal decisions such as which suburb to live in or the preferred transportation method for daily commutes. Such attributes have been integrated in our previous study, which focuses on a particular axis of the Geneva transportation network [
19].
Finally, transferring the model to other use cases would be a compelling axis of improvement, which is made possible by the high modularity of agent-based models. It is entirely possible to keep a model’s rules of evolution (or tweak them to account for the peculiarities of a given use case), and replace existing datasets by those of another area. For instance, to transfer the study to the city of Dijon (France), the geometry of the virtual world and datasets such as the spatial distribution of the population and travel times by public transportation would be those of the city of Dijon (instead of Geneva), while the people agents would still exhibit the same behaviours as the model for Geneva (such as relocating when unhappy, and becoming unhappy when commute travel time is above a given threshold). Such endeavours would not only be interesting for the new systems studied but also for the model itself: if a sound analysis can be provided by the model for other use cases as well, then its validity is further proven.