Where Maps Lie: Visualization of Perceptual Fallacy in Choropleth Maps at Different Levels of Aggregation

Abstract

This paper proposes a method for quantitative evaluation of perception deviations due to generalization in choropleth maps. The method proposed is based on comparison of class values assigned to different aggregation units chosen for representing the same dataset. It is illustrated by the results of application of the method to population density maps of Lithuania. Three spatial aggregation levels were chosen for comparison: the 1 × 1 km statistical grid, elderships (NUTS3), and municipalities (NUTS2). Differences in density class values between the reference grid map and the other two maps were calculated. It is demonstrated that a perceptual fallacy on the municipality level population map of Lithuania leads to a misinterpretation of data that makes such maps frankly useless. The eldership level map is, moreover, also largely misleading, especially in sparsely populated areas. The method proposed is easy to use and transferable to any other field where spatially aggregated data are mapped. It can be used for visual analysis of the degree to which a generalized choropleth map is liable to mislead the user in particular areas.

1. Introduction

Choropleth maps are compiled and used in order to comprehend the spatial patterns of distribution of different social and natural phenomena. The obvious motivator is the rapid and easy interpretation of a map image compared to statistical data tables intended more for the specialist. Moreover, special software and specialized knowledge is necessary for customized spatial analysis and the generalization of data. People today become used to choropleth maps at school; they trust cartographers and are accustomed to regarding the represented visual information as generally speaking to be authoritative. The presumed facts are well memorized. Spatial patterns and differences are quickly noticed. Maps are therefore powerful tools for forming public opinion and impacting decision making. However, cartographic information is in one way or another misleading [1]. Apart from instances of intentional manipulation, there are many ways in which representation of information can lead to possibly incorrect interpretation of the facts. Users are rarely aware of the risks that may result from cartographic overgeneralization. Descriptive statistics, such as dispersion of represented values, may or may not reveal the actual extent of the impact of these errors on users’ perception of maps.

Different methods are used for the mapping of population and other point data, for example, locations of objects, crime or fire events, morbidity, and mortality [2]. If there is some uncertainty of location or the data are sensitive, the data are usually aggregated by spatial units. Depending on the characteristics of the represented phenomenon (level of uncertainty of location, spread, sensitivity, etc.) and on the purpose of the visualization, the size of the units can vary from very small to very large. Small units such as cells of a regular grid that are often used for aggregation of events allow for relatively precise representation. However, the cells of a regular grid can only be referred to by abstract identifiers. Thus, such maps are inconvenient, especially for non-professional users who are accustomed to common choropleth maps that represent named administrative units. In addition, in some cases using small aggregation units results in possibilities for the divulging of sensitive personal data. On the other hand, aggregation to a smaller number of standard administrative units results in traditional choropleth maps that are easier to compile and more user-friendly.

However, such aggregated maps convey a distorted image of the phenomena represented. For example, in the case of population maps, fallacies are due to uneven distribution of population over the area—in each administrative unit there may be relatively highly populated settlements as well as large areas of arable land or forest with no residents. Average values in such cases do not represent the actual situation. The less even the spatial distribution, the larger the aggregation unit, and the greater the difference will be between data represented on maps at different aggregation levels. We term the phenomenon in which map users get incorrect information from maps perceptual fallacy (similar to the phenomenon of ecological fallacy described by Robinson [3]). It is illustrated by two pairs of map fragments representing the same five classes of population density at different levels of aggregation. In the first case, the patterns of the 1 × 1-km grid and of the same data aggregated by larger administrative units are approximately the same (Figure 1a). For the second fragment with more heterogeneous distribution, they emerge looking starkly different (Figure 1b) and the generalised visualization is misleading. Some territories with extreme class values at a detailed level are assigned intermediate class values on a generalized map. It is very difficult to predict where such distortions are apt to occur.

Many research papers are devoted to problems of spatial units in density mapping: modifiable spatial units [4,5,6,7,8]; disaggregation of spatial data [9,10]; aggregation of spatial data; and spatial uncertainty [11,12,13]. The impact of the data aggregation level on the accuracy of mapping has been extensively discussed in the field of health geography. It is commonly agreed that the higher the aggregation level, the lower the statistical power of the analysis [9,10,14]. A similar approach is used in the geography of crime [15,16]. Gerell demonstrated that the smallest types of units are of the greatest importance for analysis of arson but found little difference between large and medium-sized geographical units (1.2 and 0.4 square kilometers, respectively) [17]. Shi [12] concluded that there is no effective way to estimate, represent, and present uncertainty resulting from data aggregation. It is recommended that individual data be used whenever possible. Various methods of density mapping such as dasymetric maps and cartograms [18,19,20,21] that have been proposed result in more accurate representation, but there is an obvious lack of studies on the effect of spatial aggregation error on map perception. Cartographic and visualization approaches to representing local spatial statistics are extensively described by Nelson and Brewer who also proposed a method that ‘provides guidance to researchers in selecting the most appropriate scales to aggregate, analyze, and represent data for problem-specific analyses’ [22]. We appreciate the analytical approach taken by these authors; however, we find their cartographic results to be sophisticated to a point that makes them difficult to interpret for inexperienced map makers.

Simple cartographic representations, such as choropleth maps, are not usually created by professional cartographers. Everyone with some knowledge of GIS can represent data on a choropleth map. A small qualitative survey conducted in the form of a semi-structured interview in 2019 revealed that 16 GIS specialists, teachers, data analysts, and a journalist tend to trust in the general correctness of choropleth maps they often create. They all said they have never thought about possible substantial deviations from spatial patterns of primary data due to aggregation of values in large units. It is paradoxical that when asked what is the main purpose of map communication, they all specifically indicated spatial pattern. Moreover, they were all informed about ecological inference fallacy but never related it with significant changes of original spatial pattern. A more extensive survey with 140 participants was conducted in 2021. It is described in Section 3.1. The results are compared to actual pattern of perceptual fallacy of population density map of Lithuania in the Section 3.3.

The pattern seen on a map depends on the chosen method of classification, but users, including professional geographers, generally did not care about the method used. They also expected that the choropleth maps display more or less the same patterns at different levels of aggregation (Figure 2).

The aim of this research was to find a way to evaluate the level of misinterpretation of data for particular areas of choropleth maps that are derived from known detailed data.

In this paper, we focus on the problem of quantitative evaluation of deviation that would somehow relate to human perception:

(a)

Where there are deviations from expected relative (class/rank) values, and;

(b)

How much perception of spatial distribution is distorted due to these deviations.

2. Materials and Methods

We propose a method of visualization that helps to determine to what extent perception of map information would be distorted due to the aggregation of individual data to larger units as compared to small standardized units. The method is based on quantitative evaluation of erroneous information on class values deduced from choropleth maps compared to those of a detailed grid map. The method is hereinafter also referred to as VPF—visualized perceptual fallacy. It allows for visual representation of areas where the user is effectively given wrong information on the class of the represented phenomenon (e.g., population density, crime, morbidity). Visualizations of fallacy very clearly show how misleading decisions can be when they are based on information, derived from points or small-scale choropleth maps, which represent data aggregated into administrative units. They also reveal the spatial distribution of fallacies over the territory of concern.

The variable input data used in the VPF method are:

• The benchmark dataset (point data aggregated into small spatial units sufficient to adequate representation of the visualized phenomenon);
• The dataset of the larger aggregation units;
• The number of classes;
• Break values that depend on the chosen method of classification for both datasets.

The process of calculation is represented in Figure 2 and is rather simple. It consists of the following steps:

• Point data are aggregated into a benchmark grid and into larger aggregation units;
• Both datasets are classified using the chosen method with the chosen number of classes;
• The benchmark grid is intersected with larger aggregation units;
• Class values of the larger aggregation units are compared to the benchmark grid cells. For the benchmark grid cells that intersect several larger aggregation units, class is calculated as weighted average and rounded to integer value;
• The difference between larger aggregation units class value and benchmark grid class value is calculated:

$$P{F}_i={\displaystyle \sum }_j{a}_j{w}_{ij}-c_i; C_i\cap A_j\ne Ø$$

(1)

where $P{F}_i$ is a decimal measure of perceptual fallacy; $c_i$ is the integer class value assigned to cell $C_i$ from the benchmark dataset; $a_j$ is the integer class value assigned to cell $A_j$ from aggregated dataset; $w_{ij}$ is percent of $C_i$ overlapping with $A_j$.

There are several factors that are different for different spatial data and have to be considered.

• The size and shape of a cell. The method works in the same way for cells of any shape, administrative units, or other polygons. The optimal size and shape of the cells depend on the nature of data. Any cartographic projection can be used as long as the maps used for comparison and the map of distortions are in the same projection;
• The number of classes. While interpreting maps, people operate not the values, but the categories, such as ‘average’, ‘low’, ‘high’, or ‘normal’. The results of some pilot tests conducted as a part of the preparation for this research allow for the assertion that most users do well at perceiving and memorizing just three classes: ‘average’, ‘low’, and ‘high’ [23]. As stated in a study on visual perception of choropleth maps by Schiewe [24], ‘the number of classes should be kept as small as possible, […]. Defining a universal upper limit (e.g., 5 or 6 classes) is of course rather difficult […].’ According to previous interviews and experiments with memorizing, seven classes appeared to be too many even for geography students. Five classes allow for representing the data with higher precision than just three, but further studies are needed to find out whether people really distinguish between the two ‘low’ and two ‘high’ categories. The method described can be applied for any number of classes;
• The method of classification. Any method of classification can be used. The method operates with semantic class values. These values remain comparable across choropleth maps that use different number of classes ad different classification methods. Thus, different methods of classification may be applied for benchmark and for the aggregated dataset. An appropriate classification method must be selected depending on the nature of represented data;
• The color scheme. Both diverging and gradual color scales can be used. If they are applied correctly from the viewpoint of cartography and special users’ needs are considered, they would have no major impact on perception of the general spatial pattern.

The main characteristics of other groups of methods and the VPF method are summarized in

Table 1. Comparison of quantitative methods of assessment of problems resulting from spatial aggregation.

Methods	Descriptive Statistics	Evaluation of Data Stability in Aggregation Structures [19]	VPF Method
Approach	Statistical (standard deviation or dispersion index for an aggregation unit)	Statistical–visual (local indicators of spatial association)	Direct comparison of class values assigned to mapped units
Cartographic visualization of fallacies	No	Yes (requires additional interpretation)	Yes (direct)
Takes map perception into account	No	No	Yes
Precision	Precise values	Similarity values	Relative ordinal values
Usability of results	Requires interpretation	Requires additional analysis and interpretation	Usable without specialized knowledge
Main disadvantages	Does not represent spatial distribution of fallacies	Complexity	Shows generalized picture (may depend on the method of classification)
	Evaluates possible impact of fallacy on data analysis, but not on perception
Main advantages	Precise and commonly used	Educates users	Yields immediately usable maps of fallacies

. We did not include visual analytics [25] because (a) it is an approach rather than a method and (b) we did not investigate it in the context of usability (it is worth noting that we have no doubts concerning the utility of this approach).

The VPF method is based on a presumption that ordinary map users intuitively compare class values represented on maps and memorize relative characteristics, for example, ‘low’ values compared to ‘average’ or ‘very high’ values. This observation requires more extensive methodological reasoning that is outside the scope of this paper, but it is obvious that use of ordinal values is very common when population density issues are discussed. The results of the above-mentioned qualitative survey also facilitate the assertion that people perceive and memorize classes of values represented on choropleth maps as a limited number of ranks from ‘lowest’ to ‘highest’.

The VPF method yields maps of differences between representations by relative class values (Figure 3). Negative values (in blue) show that the class of the cell in representation of aggregated data is lower than in the benchmark representation, for example; a cell that was assigned a class value of 3 (‘average’) is within larger area with class value of 1 (‘low’). Positive values (shown in orange color) show how much class values increase after aggregation and reclassification.

3. Case Study

3.1. Rationale for the Case Study

To provide proof for the concept, the proposed method was applied for evaluation and mapping of the perceptual fallacies in population density maps that are widely used in Lithuania. The choice of the population density of Lithuania to exemplify the method is rooted in the repeated misinterpretation of small-scale population density maps in recent years. Depopulation in Lithuania is due to growing emigration and a low fertility rate that does not ensure population replacement. Since 1990, Lithuania has had population loss indicators that are among the highest in Europe. The decline of population will continue in the coming decades [26]. In recent decades, researchers have observed and analyzed sparsely populated territories that are expanding [27,28,29,30,31,32]. The data of population density in Lithuania are compiled from census data and from data of the state registers that adequately reflect the actual situation, but population density maps of Lithuania are usually compiled at small scales (1:1,000,000 or smaller). Censuses of the population and/or registries data normally refer to address points. In order to prevent the leaking of personal information, individual data are aggregated by administrative units and, since 2012, by statistical grid cells. Detailed population information has been used extensively in crime mapping for Lithuania [33,34]. Nevertheless, in 2019, most population density maps for Lithuania used for education and for evaluation of the demographic situation show information that was aggregated by second level administrative units (municipalities of 40 to 2216 square kilometers). Maps that represent administrative units are presented to politicians and the wider public and serve to form ‘the common view’ about territorial differences with respect to population.

Detailed grid maps can be found in the portal of Statistics Lithuania, but it will take time for public users to get used to this method of visualizations. As the majority of users of such maps are not cartography professionals, it is important to ensure that they interpret the maps correctly. This is a sensitive issue in Lithuania, where depopulation due to emigration and the decreasing birth rate has been gaining pace. Planning processes dedicated to preventing depopulation also make use of this information along with the precise individual data. The problem is that the majority of people interested in distribution of the population obtain their information not from databases, but from maps. Small-scale choropleth maps are quickly fathomed and memorized. All possible measures must therefore be taken in order to minimize potential misinterpretation of the information presented.

A pilot survey of 140 GIS/geography professionals in Lithuania was conducted prior to the experiment. The purpose was to understand the peculiarities of their mental maps of distribution of rural population. The respondents were employees of public and private organizations engaged in GIS and mapping: geography teachers (99; 71%), researchers in geosciences and social sciences (17; 12%), and GIS engineers (24; 17%). Out of them, 135 (96.4%) were confident about their ability to use and assess map information and 119 (85%) answered that they were informed about ecological inference fallacy. Then they were asked questions, two of which were relevant to our experiments: (a) about their opinion concerning homogeneity of rural population density in Lithuania and (b) whether they consider the demonstrated map (the same as presented in Figure 5) appropriate for representation of rural population density. There was a possibility provided to add comments as free text. Only 7 respondents (5.7%) stated that rural population density is more or less homogeneous across the country (Figure 4). The majority (102; 72.9%) stated that there are large regional differences and more than a half of them even provided various explanations why—whereas this was actually not the case in point of fact. Thus, the results of the survey supported our assumption that municipality level choropleth charts commonly seen in school atlases and in mass media may result in fallacious mental maps that persist despite availability of accurate data and detailed maps.

3.2. Experiment Design

Data and aggregation units. The method was applied to two aggregation levels of estimated population data of the Republic of Lithuania in 2020.

Our benchmark dataset was a 1 × 1 km rectangular grid. We assumed that people commonly move in the vicinity of their homes within a circle whose radius is at least 0.5 km. It corresponds to about a 5-min walking distance to the nearest facilities [35]. It would therefore not make sense to analyze population data with higher precision, even though the census data is available in 100 × 100 m cells. For the same reason standard rectangular grid cells were used instead of more compact hexagonal cells. This dataset was used as a reference dataset to determine deviations in two population density choropleth maps.

• The eldership level map (encompassing all elderships). Elderships are the smallest administrative divisions of Lithuania with an area from 0.7 to 594.8 km²; average 116.2 km²;
• The municipality level map (rural municipalities with an area from 40.0 to 2213.9 km²; average 1080.1 km²). For this map, urban areas with extreme density values (the seven largest cities) were removed as outliers to the issues at hand (Figure 5). Inclusion of cities would only mean higher levels of distortion. On the other hand, people commonly understand that towns are by their nature populated many times more densely than rural areas (in Lithuania, correspondingly averaging 984 and 31 inhabitants per km², respectively). In the discourse of depopulation, urban areas need to be analyzed and discussed as a separate issue.

The population dataset is publicly available online at the portal of official statistics in the form of an interactive atlas [36] and as a downloadable set of grids for the entire territory of the country. Due to personal data protection requirements, public point data have been aggregated by standard cells starting with the 100 × 100 m.

Classification of data. As the purpose of the study was to demonstrate not the absolute but the perceived difference in population density values, we classified the density values into five classes using the Jenks natural breaks classification method implemented in ArcGIS 10.2.2. It should be noted that this method may not be the best option for population density mapping, but for our purpose the method selected is not of primary importance. It is also important to note that break values differ for different implementations of the same classification method.

The classes were assigned the numeric values: 0—unpopulated (in the reference grid only); 1—relatively lowest population density; 2—low, 3—average, 4—high, and 5—relatively highest density. The actual density value ranges for each class were different in all maps, but the five classes represent the way people actually read maps: the value is either ‘average’ or moderately/significantly lower or higher than the average. The class values of both administrative maps were then resampled to the 1 × 1 km grid and compared to the values of the reference grid.

Visualization. In case of population density, a gradient scale is more popular and semantically justified—the values increase from zero to the highest without a change in any other quality. Pale to dark orange represent low to high density class values.

3.3. Outcomes

Two interactive maps were published on the Carto platform (https://carto.com/, accessed on 28 November 2021). They display PF values for 1 km² grid cells against:

(a)

Eldership level (https://vucarto.carto.com/builder/e86912d2-c776-449d-8a27-4423b718ef55/embed, accessed on 28 November 2021); and

(b)

Municipality level (https://vucarto.carto.com/builder/6f48cac4-5770-482f-aaf2-1b41561553d0/embed, accessed on 28 November 2021) choropleth map layers and statistics of class values for visible area (Figure 6).

Interactive maps can be explored in order to understand peculiarities of deviations. The spatial patterns investigated in this case study are unexpected for people who are used to working with population data and maps.

The existing difference between the class value assigned to a unit on an administrative map and the smaller reference grid values (‘true’ values) means that people would attribute a wrong class value to a grid cell: For example, they might perceive a part of a particular territory as densely populated whereas it is not. Quantitative difference (in units) shows the size of the gap between the estimated and perceived values. Deviations from the reference grid class values were calculated for each reference grid cell. For the municipality level map, the average PF value was +2.27, which is surprisingly high. The maximum positive deviation was five—this was the case for unpopulated cells that were nested in aggregated units with the ‘highest’ population density. The maximum negative deviation was two. Negative deviations were rare and mostly appeared for small towns (mostly less than 10,000 inhabitants) covering up to 4 reference grid cells. Considering that urban areas were not analyzed in this case, they were ignored.

Deviations of the municipality level map clearly show in which areas the perception of density was mostly distorted:

(a)

The areas surrounding the five largest cities (Vilnius, Kaunas, Klaipėda, Šiauliai and Panevėžys, Lithuania) and, surprisingly, several municipalities which have several times smaller population and no bigger towns. It means that although there is a trend of higher positive deviations to be observed near cities and towns, this trend is far from universal. The deviations were determined more by the configuration of administrative districts and by a specific mosaic of land use in each district;

(b)

Central and Western Lithuania with prevailing deviations of +3 and +4 over the entire area. These distortions are responsible for the wrong assumption repeatedly used (and even explained by higher fertility of land) by Lithuanian researchers who contend that rural population density is relatively much higher in middle Lithuania.

Thus, this pattern of distortion may also be the explanation why 46% of respondents of the survey described in Section 3.1 indicated in their free text comments that the density of rural population is highest in precisely these regions (false).

The remainder of the municipalities contains about 97% of cells with positive deviation of 1 to 3 density classes.

There are only two municipalities—Rietavas and Pagėgiai, Lithuania—with prevailing zero distortion. This fact cannot be explained by specificity of population distribution as it is very typical for all rural municipalities.

Almost the entire rural territory of Lithuania in the municipality level map was assigned too high a class value, whereas too low a value was found in about just 0.1% of the territory. The fact that over 97% of the map represents semantically incorrect information qualifies this type of population density map as unusable.

The eldership map distorted population density data to a much lesser extent: the average PF value was only +0.27. Still, class values that are too high were assigned to about 27% of the territory. The distribution of distortions was more even. In the eldership level map, the fallacies depend less on proximity to cities and may also be observed in sparsely populated areas. Further analysis of the distribution of fallacies may lead to assumptions regarding the demographic situation in particular areas.

4. Discussion

The described method appears to be applicable in other fields where point data are aggregated. It can be fully automated and used as an analytic tool by mapping experts. Creating a tool was not a purpose of this study, and appropriate automation of the task would require some efforts. Such a tool should provide a possibility to upload custom datasets for benchmark and for tested choropleth maps; to select the method of classification, ideally with a possibility of applying different methods for benchmark and test units. A graphical user interface also needs to be designed that simplifies the choice and use of the parameters and visualizes the results. Such a tool would make the evaluation of perceptual distortions rather easy for non-specialist users, especially when they have to visualize big data with unknown distribution. Upon implementation, it is intended that usability analysis and more case studies will follow.

In this study, we only demonstrated the results with one classification algorithm. It is worth mentioning that application of two other popular methods—equal interval and quartile—yielded slightly different cartographic results. Differences were due to specificity of concrete data distribution. The impact of the map’s chosen classification method on perception can also be analyzed and evaluated, but it is beyond the scope of this paper.

The described method of comparison and fallacy mapping can be applied for any sets of maps where point data are aggregated to different units: unemployment, crime, health or other maps. Before making the decision to choose a larger aggregation unit we advise visualizing and comparing possible perceptual distortions. If the phenomenon is evenly spread, the differences can be minor and evenly distributed—which means, in effect, that more general maps can be used. This option is usually cheaper and more easily perceived by inexperienced map users. If the resulting discrepancies are unacceptable, it may be a sign that smaller aggregation units should be used.

In this paper, we demonstrated the feasibility of the VPF method using particular spatial aggregation units and the population dataset of Lithuania. Evaluation of perceptual fallacies, perhaps with different initial assumptions and different variables, is applicable to any spatial areals and to any set of individual data that are aggregated to these areals. It is also applicable with any method of classification used for visualizations provided that the same classification method is used for the compared maps.

For any type of phenomenon or pattern of administrative boundaries, the spatial pattern of deviations that may lead to wrong perception cannot be understood and controlled using known parameters. For example, we cannot make any inference about what this pattern would look like for crime or mortality data. However, the pattern is quickly seen in the fallacy maps generated by the VPF method. A quick glance at the distortion map gives a fair idea about the extent and volume of misrepresentations. For deeper understanding, statistics of deviations can easily be calculated for each map areal.

We do not expect that because of demonstrable distortions, choropleth population density maps will anytime soon be replaced by grid or interpolated density maps. But before disseminating small-scale choropleth maps, the extent of the distortions must be evaluated and suitability for purpose assessed. Obviously, the potential damage done by misrepresentation and corresponding quality requirements differs markedly depending on whether the maps are being used for research, decision making, education, or just providing information to the public.

5. Conclusions

In this paper, we proposed a cartographic method that allows for the assessing of impact of aggregation on perception of a spatial distribution pattern seen on a map.

The method is applicable to any data that are aggregated in variable areal units. Perceptual fallacy can be presented visually as a map of distortions, or described by statistics of distortions. The statistics of distortions can be used for correction of automatically assigned class values.

We demonstrated the method with data of rural population density in Lithuania. Based on the results of the test, a decision can be made regarding suitability of an aggregated map for correct perception of the spatial distribution seen on the benchmark map—in the given case, the choropleth map was suitable at eldership level but became misleading at the municipality level. Common use of the municipality level map may be the cause of widespread fallacious beliefs concerning spatial distribution of the rural population.