Spatial Semantics for the Evaluation of Administrative Geospatial Ontologies

Abstract

Administrative geography is concerned with the hierarchy of areas related to national and local government in a country. They form an important dataset in the country’s open data provision and act as the geo-referencing backdrop for many types of geospatial data. Proprietary ontologies are built to model and represent these data with little focus on spatial semantics. Studying the quality of these ontologies and developing methods for their evaluation are needed. This paper addresses these problems by studying the spatial semantics of administrative geography data and proposes a uniform set of qualitative semantics that encapsulates the inherent spatial structure of the administrative divisions and allows for the application of spatial reasoning. Topological and proximity semantics are defined and combined into a single measure of spatial completeness and used for defining a set of competency questions to be used in the evaluation process. The significance of the novel measure of completeness and competency questions is demonstrated on four prominent real world administrative geography ontologies. It is shown how these can provide an objective measure of quality of the geospatial ontologies and gaps in their definition. The proposed approach to defining spatial completeness complements the established methods in the literature, that primarily focus on the syntactical and structural dimensions of the ontologies, and offers a novel approach to ontology evaluation in the geospatial domain.

1. Introduction

Much of all government data have some reference to location, and thus benefit from being mapped to a geographical data framework [1]. The last decade has seen substantial efforts in opening up geospatial datasets by governments. For example, for the city of Manchester in the UK (https://mappinggm.org.uk/, accessed on 13 August 2024) and the Halifax Regional Municipality, Canada (https://catalogue-hrm.opendata.arcgis.com/, accessed on 13 August 2024), provide open public access to map data, including, housing, socioeconomic and demographic data. In the UK, open data policy has led to new data sets being made freely available by the Ordnance Survey (UK’s national mapping agency) (https://osdatahub.os.uk/, accessed on 13 August 2024), including among others, all of the administrative and postal code boundaries [2]. Similar initiatives exist across all of Europe [3], and many other countries in the world [4], driven by evidence of significant social and economic value [1]. Administrative geography open datasets are an important set of data representing a hierarchy of areas relating to national and local government, that are often used as backdrop map layers for locating other geospatial data. These multilayered hierarchies are normally complicated by the differing structures within and across countries. For example, in the UK, Council Areas, Unitary Authorities and London Boroughs are used to represent regions at corresponding levels in Scotland, Wales, and England, respectively. Proprietary ontologies are used by different countries to encode and publish these datasets [5,6,7], as linked open data [8,9]. However, there is a lack of studies that evaluate the effectiveness of these ontologies. There is a need for ontology evaluation methods that consider the specific semantics of the geospatial ontologies. The inherent spatial semantics in the representation of administrative divisions as layered partitions of space by fiat boundaries are common across different geographic places and datasets. The definition and explicit representation of these semantics are important to allow for the development of common ontologies and spatial reasoning frameworks.

On the other hand, much effort has been made in the design and development of standard models for the representation of geospatial data to support their effective discovery, sharing and integration; in particular, the ISO Geographic information standards (https://committee.iso.org/sites/tc211/home/re.html, accessed on 13 August 2024) and the Open Geospatial Consortium (OGC) (https://www.ogc.org/standards/, accessed on 13 August 2024) standards. A prominent example of these standards for geospatial linked data is the ontology underlying the OGC Geographic Query Language for RDF (GeoSPARQL) (https://docs.ogc.org/is/22-047r1/22-047r1.html, accessed on 13 August 2024). The GeoSPARQL ontology provides core vocabulary and concepts for the representation of geographic features and spatial relationships that are fundamental modelling elements in geospatial domains. Adopting and reusing these foundational ontologies will enable the shared use and integration of the administrative geography open datasets.

In this work, we propose a new measure of semantic completeness for geospatial ontologies of administrative datasets. Topological and proximity semantics are distinguished and combined in a measure of spatial semantic completeness. The significance of the proposal is twofold; (a) it provides a homogeneous method of defining spatial semantics in these types of ontologies, and (b) it provides a novel metric of quality for evaluating geospatial ontologies that considers the spatial dimension of the datasets. This core dimension has not been considered before in any ontology evaluation methods. The utility of the proposal is demonstrated by the evaluation of ontologies from four European national mapping agencies used in the provision of their open datasets. It is shown how the proposed metric provides an objective measure of quality of the geospatial ontologies and identifies the gaps in their definition. The contributions of this work are as follows: (a) a study of the nature of administrative open geospatial data and the identification of a uniform set of spatial semantics that are inherent in the data that can be explicitly defined in their representative ontologies, (b) proposal of a uniform measure of spatial completeness as a method for evaluating ontologies of administrative geographies, (c) demonstration of the utility and the effectiveness of the proposed measures through the evaluation of four real world ontologies.

The remainder of this paper is structured as follows. Section 2 provides an overview on geographic ontologies and ontology evaluation methods. Section 3 presents the proposed measures of spatial semantics and concludes with a combined measure of spatial semantics completeness. The proposed spatial semantics are translated into a set of competency questions that can be used for evaluating geospatial ontologies. Section 4 provides a systematic evaluation of four administrative division ontologies against the proposed measures. A discussion of the results and their significance, in comparison to other standard ontology evaluation metrics, is also presented. Conclusions and an overview of future work is given in Section 5.

2. Related Work

An overview is given of geospatial ontologies, and in particular those used for representing open administrative geography data sets. Ontology evaluation methods are briefly reviewed as possible approaches for evaluating geospatial ontologies.

2.1. Geospatial Ontologies

An essential building block to open geolinked data is the underlying ontology that it adopts [10]. An ontology conceptualising geographic knowledge is normally referred to as a geospatial ontology. The development of geospatial ontologies has been the subject of interest for several decades. One notable success is the definition of standard vocabulary for topological spatial relations, adopted by the GeoSPARQL ontology (an OGC standard for querying geospatial information expressed in RDF [11]. The GeoSPARQL ontology can be considered as a foundational ontology for the geospatial domain, as it provides basic concepts and relations that are specific to this domain, but are fundamental for the definitions of any geospatial domain or application ontologies. In particular, it provides a core ontology with top-level classes for modelling geospatial information, namely, an geo:Feature; intended to represent uniquely identifiable geospatial phenomena, such as, rivers, valleys, mountains or a buildings. As a subclass of geo:SpatialObject, a geo:Feature can have multiple geometries, e.g., a mountain can be defined as a point or as a polygon, representing its boundary. The combination of a core, geometry and spatial relations ontologies provides basic constructs for the representation (and querying) of geographic data.

Several other proprietary ontologies exist to support the definition of geospatial concepts on the Semantic Web, including, WikiData [12], DBPedia [13], OpenStreetNames [8], Geonames [14]. For example, Schema.org [15] provides a set of class definitions for geospatial objects and relationships, which is comparable to GeoSPARQL, intended to capture the representation of Place, e.g., sdo:Place, sdo:GeoShape, sdo:GeoSpatialGeometry and sdo:GeoCoordinates. Potential mappings between these ontologies (and several others) and GeoSPARQL are provided in [11].

Ordnance Survey of Great Britain [16] is one of the first national mapping agencies to propose the use of ontologies for the representation of their open datasets. They provided a separate spatial relations ontology and presented detailed ontologies for administrative geography data sets, as well as specific application ontologies (namely, hydrology ontology) [17]. More recently, they have recommended the need for referencing authoritative ontologies and are currently undertaking a redesign of their open data provisions [9]. The latest Open Data Index [4] identifies 19 countries (out of 94 surveyed) that provide full open access to administrative boundary data and 53 others provided partial access. Additionally, thirty countries provided full (n = 8) or partial (n = 22) open location datasets (postal code and coordinate data).

Open geospatial data are also being created through crowdsourcing. The largest platform, OpenStreetMap (OSM) [18], is built by a community of mappers that contribute and maintain data about roads, railway stations, cafes, etc. all over the world. OSM adopts a proprietary geospatial ontology, representing space as a graph of nodes (places), ways (edges) and relations. Notably, this form of mapping is useful for geographic features that can be identified and located by people. Administrative boundaries are, in most cases, not identifiable on the ground, and are defined by authoritative resources.

The above efforts all refer to basic models of space that capture the spatial aspect of the geographic feature, by offering constructs for modelling geometry and shape attributes to allow for mapping applications and basic spatial analysis. Other research efforts have also been carried out that consider fundamental modelling of the concept of place and its different dimensions. For example, the early work of Smith et al. on agglomerations and vagueness of place definition and dynamic spatial ontologies [19,20,21], and the more recent work of studying place facets [22] and the need for capturing, managing and analysing place data in information systems [23]. Bittner and Smith [24] present a formalisation of the definition of granular partitions and provide an example of its application in geographic space for cadastre maps. These formalisms are needed to support the definition of spatial relations and reasoning over these types of spaces, as presented in [25]. Administrative geography regions and their associated divisions and boundaries are subclasses of geographic objects. However, their particular spatial semantics (including geographic space partitioning) have not been studied before, and specifically in the context of their representation in ontologies of geospatial open data.

2.2. Geospatial Ontology Evaluation

Evaluating the quality of ontologies has been the subject of interest for many years. Several works have reviewed the subject [26,27] and quality models have been proposed that attempts to group the proposed metrics into meaningful dimensions to guide the evaluation process. For example, Gangemi et al. [28,29] proposed three dimensions: structural, functional and usability-related. The structural dimension focuses on the syntax and formal semantics of the ontology, e.g., by measuring its graph structure properties, such as, the richness of the inheritance hierarchy or the depth and breadth of the hierarchy. The distributions of classes and the attributes over the classes are also examples of this dimension. The functional dimension essentially aims to measure how far the ontology fits its purpose. Precision, recall and accuracy measures, widely used in information retrieval, are proposed to evaluate this dimension. In other works [30], semantic coverage (completeness); a measure of the information content in the ontology against a gold standard, was proposed to assess this dimension. The usability-related dimension covers aspects related to how well the ontology can communicate its content through documentation. Automated tools have been developed to facilitate the evaluation of the structural and usability dimensions of ontologies, e.g., OOPs [31], OntoQA [32], ONTOCOM [33] and Luzzu [34]. Demonstration of the their application in different domains has been reported [35]. However, measuring the functional dimension is more complex and cannot be directly automated. Human-based (expert-based) approaches can be used here, but this carries the limitation of resourcing the appropriate and sufficient expertise and is a costly process. Other approaches have been proposed including the gold-standard and data-driven approaches, as discussed below.

Gold standard-based evaluation, is the approach of comparing the ontology against an ideal representation of the domain knowledge [26,36,37]. In [30], compatibility and completeness are proposed as the quality attributes to be tested with this method. In particular, syntactic completeness questions how much the vocabulary of the ontology matches that of the standard, and semantic completeness measures how much the vocabulary of the standard can be derived from the ontology. Specific metrics that compute the degree of overlap between the ontology and the standard, in terms of the concepts, instances, attributes and relationships are defined. Gruninger and Fox [38] defined competency questions as a criteria to measure the quality of the ontology, i.e., questions that the ontology should be able to answer, and the ontology is complete if all the required competencies are fulfilled.

The data-driven ontology evaluation method is a technique used for assessing the quality of an ontology [39], based on comparing the ontology to a corpus of datasets in the domain. This approach is effective for evaluating the vocabulary, structure, and semantic layers of the ontology [40]. It is most suited to domains that are text-based, and used natural language processing and machine learning in the evaluation process. Work on these in the geospatial domain is still emerging [41] and is being tested in the area of geographic question answering [42].

Very few efforts have considered the problem of evaluating the quality of geospatial ontologies. A notable related work is [43], which used a linked data quality assessment framework Luzzu [44] to evaluate the Ordnance Survey Ireland (OSI) ontology open linked data provision. The focus is on identifying any errors in mappings that generate RDF data, and thus produce incorrect or inconsistent information and to check if Linked Data best practices are being followed [45]. The same method was also applied to validate a proposal for a linked data vocabulary to represent the form and function of spatial objects, intended to underpin the OSI ontology [46]. This paper is concerned with spatial semantics coverage in administrative geography ontologies. Spatial completeness, and related competency questions, are of particular interest here. To our knowledge, the issue of evaluating completeness of geospatial ontologies has not been addressed in the literature.

3. Measures of Spatial Semantics in Administrative Geography Data

Consider the map of Wales, UK, shown in Figure 1. The map in Figure 1a represents the boundaries of the local authority districts in the UK. In particular, the map is focused on the Welsh districts. In Figure 1b, the boundaries of the Local Health Boards are shown for the same area. The map implicitly provides semantics about the location of these regions in space and their relationships to one another that can be inferred by visual spatial reasoning Some examples of these semantics are as follows, with references to the shaded regions in Figure 1 (Map data are from the Open Geography portal from the Office for National Statistics, UK. https://geoportal.statistics.gov.uk/, accessed on 13 August 2024).

• The local authority district of Cardiff is in Wales.
• Cardiff and Newport are neighbours.
• Swansea is further away from Cardiff than Newport.
• The Isle of Anglesey is farther away to the north from Swansea than Carmarthenshire.
• Pembrokeshire, Carmarthenshire and Caredigion are part of the Hywel Dda University Health Board.

The above statements are examples of qualitative spatial relationships between regions, and in particular, topological relations; describing the degree of connectedness between regions, proximal relations; describing qualitative distance relationships between regions and directional relations; describing the relative position of the regions with respect to a specific frame of reference (e.g., cardinal directions). Qualitative spatial representation and reasoning (QSRR) is an established area of research that seeks to define formalisms for modelling space and spatial relations to allow for the automatic inference of such spatial semantics (see [47] for a detailed review of this subject).

Here, we utilise topological and proximal spatial relationships to define the spatial semantics of administrative geographies. The hierarchical division of space normally adopted for administrative divisions lends itself naturally to connection patterns represented by topological relations. We extend these patterns to also express some coarse representation of proximity and leave out directional relationships for future studies. Figure 2 shows a set of six standard topological spatial relations between simple regions. Generalised containment relationships are used where no distinction is made for regions with touching boundaries.

Administrative divisions, as seen in the examples shown in Figure 1 divide the space into distinct neighbouring regions that together cover the space completely. Several divisions of the same space can be adopted to represent different purposes. For example, health board regions, local authority district regions or postcode regions, etc. Multiple layers of these divisions combine to form administrative hierarchies, e.g., local authority districts contain wards and parishes, and postcode areas contain postcode districts, which in turn contain postcode sectors, and so on. Relationships between regions in administrative division geographies can thus be summarised as follows:

• Spatial relationships between regions in one division of space are restricted to either touches or $disjoint$; e.g., $disjoint(Cardiff,Swansea)$, and $touches(Cardiff,Newport)$.
• Relationships between regions on different layers can be defined by containment relationships; e.g., $inside(Cardiff,CardiffandValeUniversityHealthBoard)$.
• Regions in different divisions may intersect, where $intersects$ is defined as any relationship of connectedness between two regions [48]: $inside$∨$contains$∨$overlaps$∨$touches$. For example, regions representing local districts will intersect with postal code areas in the same geographic region.

3.1. Topological Semantics

Let $L_{ij}$ represents level i in an administrative hierarchy $D_j$. Let a region $r_n$ represents an instance of the set of regions R in $L_{ij}$. A measure of the topological semantics in $D_j$ can thus be formulated in terms of the numbers and types of possible topological relations defined between regions across the levels of $D_j$.

Let $T_n$ be the set of all neighbouring regions to $r_n$. Let $\overline{T_n}$ be the complement of $T_n$ in $(R-r_n)$ (the remaining set of regions in R besides $r_n$). The topological semantics ($TS$) of the administrative division $L_{ij}$ in $D_j$ are defined as follows.

$T{S}_1$:

There exists a $touches$ relationship between $r_n$ and every region in $T_n$.

$T{S}_2$:

There exists a $disjoint$ relationship between $r_n$ and every region in $\overline{T_n}$. Note, that this is a derived relationship from $T{S}_1$.

$T{S}_3$:

There exists one region in $L_{i-1j}$ (for all levels except the root), that $contains$ $r_n$. I.e. every region lies inside one parent region. Transitivity of the $inside$ and $contains$ relationships can then be applied to define all possible containment relationships between regions on all levels.

$T{S}_4$:

Regions on a similar level in different administrative hierarchies for the same extent in space will intersect.

For two administrative hierarchies $D_{j_1}$ and $D_{j_2}$, that represent different divisions of the same space, let $L_{i{j}_1}$ represents level i in $D_{j_1}$ and $L_{i{j}_2}$ represents level i in $D_{j_2}$. There exists at least one instance of the relationship $intersects$ between $r_n$ in $L_{i{j}_1}$ and $r_m$ in $L_{i{j}_2}$.

To determine the correspondence between levels in different hierarchies, we propose the use of a global universal scheme, such as the Global Administrative Areas (GADM) (https://gadm.org/, accessed on 13 August 2024).

Using the above definitions of topological semantics, we can propose a definition of topological semantic completeness ($TC$) for the representation of administrative geographies as follows.

$T{C}_1$:

A level $L_{ij}$ in $D_j$ is considered to be complete with respect to topological semantics (topologically-complete), if for every region $r_n\in L_{ij}$, the four topological semantics above can be defined.

$T{C}_2$:

$D_j$ is considered to be topologically-complete, if every level $L_{ij}\in D_j$ is topologically complete.

3.2. Proximity Semantics

We propose a measure of proximity for administrative geography regions that is based on and extends the semantics of connectedness used in defining the topological semantics above. Two regions in R are either $connected$ or $disjoint$; $\forall r_n,r_m\in R(intersects(r_n,r_m)\vee disjoint(r_n,r_m)),n\ne m$. A measure of the proximity semantics in $D_j$ can thus be formulated in terms of the numbers and types of regions that $intersect$ across the levels of $D_j$. Thus, measures of proximity semantic completeness ($PC$) for the representation of administrative geographies can be proposed as follows.

$P{C}_1$:

A level $L_{ij}$ in $D_j$ is considered to be complete with respect to proximity semantics (proximity-complete), if for every region $r_n\in L_{ij}$, its connectedness with all other regions in $L_{ij}$ is defined, or can be inferred.

$P{C}_2$:

$D_j$ is considered to be proximity-complete, if every level $L_{ij}\in D_j$ is considered to be proximity-complete.

3.3. Spatial Semantic Completeness

We use the above set of topological and proximity semantics to define an overall spatial semantic completeness metric that can be used to evaluate an ontology of administrative geography. Let a geospatial ontology O be a tuple (C, I, A, R), where C represents concepts, I represents instances of these concepts, A represents a collection of finite sets of attributes of these concepts, and R a finite set of binary relations on these concepts. Let $O=(C,I,A,R_s)$ be the ontology O that defines, in addition to R, all spatial relations that are definable through QSR from O ($O⊢R_s$).

Let a set of map layers $\Omega$ represent a hierarchy of administrative divisions in space, against which O will be evaluated. Let $\Omega =(C^{\prime },I^{\prime },A^{\prime },{R_s}^{\prime })$ be the set of concepts, instances, attributes and spatial relationships definable in $\Omega$, where ($\Omega ⊢{R_s}^{\prime }$).

$\Omega$ represents a gold standard ontology with respect to O and contains all concepts, relations, attributes, and instances of the administrative division for a specific space. $\Omega$ is considered to be topologically complete ($T{C}^{\Omega }$) and proximity complete ($P{C}^{\Omega }$); namely, it is possible to define the set of all topological and proximity semantics for $\Omega$. Furthermore, it is assumed that O is semantically complete with respect to $\Omega$ on the three components: C, I and A (since the maps are faithful representations of the ontology). On the other hand, completeness of O with respect to $R_s$ is not assumed and will need to be evaluated as follows.

• Spatial Semantic Completeness: The spatial semantic completeness (spatial completeness for short) of ontology O with respect to $\Omega$ ($SpCo{m}_{R_s}^{\Omega }\left(O\right)$) is defined as follows:

  $$\begin{array}{cc}\hfill SpCo{m}_{R_s}^{\Omega }\left(O\right)& =S_{R_s}/Siz{e}_{R_s}(\Omega )\hfill \end{array}$$

  where,

  $$\begin{array}{cc}\hfill S_{R_s}& =\sum _{r_s\in R_s}\Vert r_s\cap {r_s}^{\prime }\Vert ,\hfill \\ \hfill Siz{e}_{R_s}(\Omega )& =\sum _{{r_s}^{\prime }\in {R_s}^{\prime }}\Vert {r_s}^{\prime }\Vert .\hfill \end{array}$$

$Siz{e}_R(\Omega )$ is the number of definable spatial relations in $\Omega$. $S_{R_s}$ is the ratio between the definable spatial relationships in O to those defined in $\Omega$.

3.4. Spatial Semantic Competency Questions

Competency questions (CQs) play a key role in ontology engineering [49]. They consist of a set of questions that the ontology should be able to answer and thus define the ontology scope and provide a way of evaluating the ontology. Spatial semantics are part of the domain knowledge captured by a geospatial ontology. Capturing these semantics in the form of competency questions can therefore be used to facilitate both the process of defining and evaluating ontologies. When designing the ontology, requirements can be captured by ontology engineers through CQs and expressed in natural language. During evaluation, the questions are expressed formally using Description Logic and posed are posed as queries to the ontology.

The proposed set of spatial semantics can be used to formulate a set of competency questions for administrative domain ontologies as follows. If O is spatially complete with respect to $\Omega$, then O would be able to address the set of “spatial semantic competency questions” for all classes of regions C and instances I in O. Let $x\in I$ be an instance of a region, and $D_n$ be a subset of map layers representing a possible hierarchical division in O. The set of competency questions (SCQ) that can be used to evaluate O are as follows.

SCQ1: 

Which regions are $neighbours$ of ($touch$) region x?

SCQ2: 

Which regions are $near$ region x?

SCQ3: 

Which regions lie $between$ region x and region y?

SCQ4: 

Which regions are not neighbours (do not touch) region x?

SCQ5: 

Which regions are parents of ($contain$) region x?

SCQ6: 

Which regions are contained in ($inside$) region x?

SCQ7: 

Which regions in $D_1$ intersect with region x in $D_2$?

SCQ8: 

Which regions in $D_1$ are near region x in $D_2$?

$SCQ1$ and $SCQ4-7$ directly correspond to topological and proximity relationships in Figure 2. $Near$ relationships ($SCQ2-3$ and $SCQ8$) describe relative distance in natural language communication. They are dependent on the scale of the space and size of objects described and have no precise quantitative definition [50]. Using a graph-based approach to the representation of qualitative spatial relations, semantics of the $near$, $between$ and $far$ relationships can be described as the graph distance between regions (nodes). Some interpretations of proximity relationships can be described as follows on a graph representing regions and $touches$ relationships as described in Section 3.3 above.

• Regions are considered to be $near$ if they are disjoint and there is a path of two (touches) edges between them, as shown in Figure 3a.
• A region is considered to be $between$ two other regions if it on the path between them such that the path does not contain cycles or parts of cycles in the graph. For example, region y is between regions x and z in Figure 3a and regions $\{y,z\}$ are between regions x and n in Figure 3b.
• Regions are considered to be $far$ if $>2$ regions exist $between$ them, as shown in Figure 3b.

Using the above relationships on the map in Figure 3c, we can define some proximity relationships as follows: $between(Butetown,TaffsWell,\{Whitchurch,Gabalfa,Cathays\}$. $near(Whitchurch,Cathays)$, $near(TaffsWell,Gabalfa)$, $far(TaffsWell,Butetown)$, etc.

The above definitions are possible examples of how proximity can be defined in this type of ontologies. Formal definitions of these relationships and other variations (e.g., very near, very far, etc.) using graph theory is possible, but is outside the scope of this work. In the remainder of this paper, it is assumed that there exists an $\Omega$ and an associated set of spatial competency questions, against which an administrative geography ontology can be evaluated.

4. Evaluating the Spatial Semantics of Administrative Geography Ontologies

Four well-established administrative geography ontologies were chosen to be studied here. These are the administrative geographies of the UK, Ireland, France and Greece; offered by their respective National Mapping Agencies (Ordnance Survey of Great Britain, Ordnance Survey Ireland, IGN France and the Ministry of the Interior and Administrative Reconstruction, Greece). The ontologies, provided on open data portals, were downloaded and stored in GraphDB (https://graphdb.ontotext.com/, accessed on 13 August 2024) and Protégé (https://protege.stanford.edu/, accessed on 13 August 2024) for analysis. A summary of the the number of classes and instances in the datasets is presented in

Table 1. Administrative geography ontologies used for evaluation.

	Ontology	Classes	Individuals
$O_1$	Ordnance Survey (OS)	53	2,021,346
	https://data.ordnancesurvey.co.uk/ontology/ (accessed on 13 August 2024)
$O_2$	Ordnance Survey Ireland (OSI)	10	659,333
	http://ontologies.geohive.ie/geoff/ (accessed on 13 August 2024)
$O_3$	Greek Administrative Ontology (GAG)	9	2914
	http://linkedopendata.gr/dataset/greek-administrative-geography (accessed on 13 August 2024)
$O_4$	IGN Ontology France (geofla)	8	132,567
	http://data.ign.fr/def/geofla/20190212.en.htm (accessed on 13 August 2024)

.

The analysis was performed by studying the ontologies to determine the different administrative hierarchies (and corresponding levels) presented in each dataset, and then identifying the spatial relationships defined between the classes and between the instances. In what follows, we present, for each dataset, the structure of the ontology and a summary of the spatial semantics encoded in the data. These are then used to compare with the spatial semantics proposed in the previous section to provide a measure of completeness. Analysis of the Ordnance Survey ontologies is presented here, while analysis of the Irish, French and Greek ontologies is provided in the Appendix A.

4.1. Ordnance Survey, UK (OS)

The Ordnance Survey, UK, has invested much effort in preparing its ontologies, including a specific ontology of spatial relationships [17,51]. Administrative divisions for Wales are shown in Figure 4a,b. Postal code division for the whole of the UK is shown in Figure 4c, and an example class definition of the OS ontology is shown in Figure 4d. Note that the Welsh divisions are used here as an example. Similar representations of divisions, are used to represent different areas of the UK. Note also that postal code divisions are not considered as administrative divisions. However, they are represented with similar spatial hierarchies and can thus be treated in the same manner.

Table 2. Spatial Semantics in OS ontologies.

Domain	Range	Possible Relationships	Defined Relationships	Definable by QSR	Not Defined	Spatial Completeness
Unitary Authority	Unitary Authority	$disjoint$,$touches$	$touches$	$disjoint$	∅	1
Unitary Authority	Unitary Authority Electoral Division	$disjoint$, $within$, $contains$	$contains$	$within$, $disjoint$	∅	1
Unitary Authority Electoral Division	Unitary Authority Electoral Division	$disjoint$, $touches$	$touches$	$disjoint$	∅	1
Unitary Authority	Community	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Community	Community	$disjoint$, $touches$	$touches$	$disjoint$	∅	1
Postcode Area	Postcode Area	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Postcode District	Postcode District	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Postcode Sector	Postcode Sector	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Postcode Area	Postcode District	$disjoint$, $within$, $contains$	$within$, $contains$	$disjoint$	∅	1
Postcode District	Postcode Sector	$disjoint$, $within$, $contains$	$within$, $contains$	$disjoint$	∅	1
Postcode Sector	Postcode Unit	$disjoint$, $within$, $contains$	$within$, $contains$	$disjoint$	∅	1
Overall Spatial Completeness						$0.72$

presents the computation of the spatial completeness measure for the OS ontologies.

In the table, the columns for Domain and Range represent the types of regions explicitly defined in the administrative division in the ontology. Regions in three administrative divisions, corresponding to those shown in Figure 4, are presented in groups, separated by a horizontal divider. For example, the first group in Table 2 consists of three types of regions: Unitary Authority, Unitary Authority Electoral Division and Electoral Division. The column of Possible Relationships lists the set of sound topological relationships between the regions considered (the only types of physically possible relationships between the regions). For example, two Unitary Authority regions can exist only in $disjoint$ or $touches$ relationships. The Defined Relationships column lists the actual relationships that are explicitly defined in the ontology between the regions considered. For example, the OS ontology defines the $touches$ relationship between regions of type Unitary Authority. The Definable by QSR column lists the possible relationships that can be automatically derived by qualitative reasoning from the explicitly stored ones. For example, using $T{S}_2$, all disjoint relationships between region of type Unitary Authority can be deduced from the defined touches relationships, etc. The Not Defined column lists the difference between the possible relationships and the union of defined and definable relations. Spatial completeness is the ratio between the total number of defined and definable relationships to the total number of possible relationships. An overall measure of spatial completeness is given in the last row, as an average of the measures across all the considered regions and relationships.

Based on the defined relationships, some example competency questions that can be answered by the OS ontologies are as follows, assuming they are used within a type of location-based service application.

• Find coffee shops that are located between (the Communities of) Roath and Llandaff in Cardiff?
• Find all the primary schools in the neighbourhood of (the Unitary Authority of) Cardiff?

Some examples of questions that cannot be answered by the ontologies are as follows.

• Find houses for rent near (the Postcode District) $CF24$?
• Which Postcode Units are between $CF158BB$ and $CF243AA$?

Table 3. Coverage of spatial competency questions for the OS ontologies.

Competency Question	Possible between Regions	Not Possible between Regions	Coverage
SCQ1: Which regions are neighbours of (touch) region x?	{Unitary Authority}, {Unitary Authority Electoral Division}, {Community}	{Postcode Area, Postcode District, Postcode Sector, Postcode Unit}	$43\%$
Examples	Find schools in neighbouring regions of Powys (Unitary Authority)?	Find hairdressers close to CF24 3AA (Postcode)?
	Which communities are neighbours of Cathays (Community)?
SCQ2: Which regions are $near$ region x?	{Unitary Authority}, {Unitary Authority Electoral Division}, {Community }	{Postcode Area, Postcode District, Postcode Sector, Postcode Unit}	$43\%$
Examples	Find parks near Cathays?
SCQ3: Which regions lie $between$ region x and region y?	{Unitary Authority}, {Unitary Authority Electoral Division}, {Community}	{Postcode Area, Postcode District, Postcode Sector, Postcode Unit}	$43\%$
Examples	Find bakeries between Cathays and Llandaff?
SCQ4: Which regions are not neighbours (do not touch) region x?	{Unitary Authority}, {Unitary Authority Electoral Division}, {Community }	{Postcode Area, Postcode District, Postcode Sector, Postcode Unit}	$43\%$
Example	Is Roath far from Cathays?
SCQ5: Which regions are parents of ($contain$) region x?	$\left\{All\right\}$	∅	$100\%$
Example	Which authority is Cathays located in?
SCQ6: Which regions are contained in ($inside$) region x?	$\left\{All\right\}$	∅	$100\%$
Example	Which communities are in Powys?
SCQ7: Which regions in $D_1$ intersect with region x in $D_2$?	{Unitary Authority Electoral Division, Postcode Unit}	{Unitary Authority Electoral Division, Community}	$33\%$
		{Unitary Authority Electoral Division, Postcode Unit}
Examples	Which postcode units are within Cathays?
SCQ8: Which regions in $D_1$ are near region x in $D_2$?	{Unitary Authority Electoral Division, Postcode Unit}	{Unitary Authority Electoral Division, Community}	$33\%$
		{Unitary Authority Electoral Division, Postcode Unit}
Examples	Is CF24 3AA near within Cathays?
Overall Coverage			$55\%$

provides a list of all the competency questions that can be answered by the ontology and those that the ontology cannot address.

4.2. Results and Discussion

As can be seen in the four examples of administrative geospatial ontologies, they all capture the containment relationships between levels in their represented hierarchies. These relationships are essential for encoding the semantic structure of the administrative divisions. In all cases, spatial containment is assumed based on a semantic relationship, such as, $belongs\_to$ or $part\_of$. With the exception of the OS ontology, no explicit spatial relationships are defined. In the case of the OS ontology, the definition of the spatial relationships was not homogeneous across all hierarchies, in particular, no $touches$ relationships was defined in the postcode hierarchy. Note also that no explicit relationships were encoded across divisions in any ontologies, (except for postcode units in the OS ontology).

As can be expected, the greater the degree of spatial completeness of the ontology, the more spatial competency questions it is able to address, as shown in the

Table A1. Spatial Semantics in the OSI Ontology.

Domain	Range	Possible Relationships	Defined Relationships	Definable by QSR	Not Defined	Spatial Completeness
Municipal District	County	disjoint, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
County	County	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Municipal District	Municipal District	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
ine
Local Electoral Area	County	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Local Electoral Area	Local Electoral Area	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
ine
Electoral Division	County	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Electoral Division	Electoral Division	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
ine
Townland	Electoral Division	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Townland	Civil Parish	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Townland	Townland	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Civil Parish	Barony	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Civil Parish	Civil Parish	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Barony	County	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Barony	Barony	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
ine
Overall Spatial Completeness						$0.5$

,

Table A2. Coverage of spatial competency questions for the OSI ontology.

Competency Question	Possible between Regions	Not Possible between Regions	Coverage
SCQ5: Which regions are parents of (contain) region x?	$\left\{All\right\}$	∅	$100\%$
Examples	Which regions are parents of ($contains$) Ardkill (Townland)?
	Which regions are parents of ($contain$) Glenties (District)?
	Which regions are parents of ($contains$) Boho (Parish)?
SCQ6: Which regions are contained in ($inside$) region x?	$\left\{All\right\}$	∅	$100\%$
Examples	Which Districts are contained in ($inside$) Cork (County)?
	Which Civil Parishes are contained in ($inside$) Iveragh (Barony)?
SCQ1, SCQ2, SCQ3, SCQ4, SCQ7, SCQ8	∅	$\left\{All\right\}$	$0\%$
between 7 classes of regions
Overall Coverage			$25\%$

,

Table A3. Spatial Semantics in the GAG Ontology.

Domain	Range	Possible Relationships	Defined Relationships	Definable by QSR	Not Defined	Spatial Completeness
Municipality Unit	Municipality	disjoint, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Municipality Unit	Municipality Unit	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Municipality	Municipality	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Municipality	Regional Unit	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Regional Unit	Regional Unit	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Regional Unit	Region	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Region	Region	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Region	Decentralized Administration	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Decentralized Administration	Decentralized Administration	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Decentralized Administration	Country	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Country	Country	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Municipality Community	Municipality Unit	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Municipality Community	Municipality Community	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
local Community	Municipality Unit	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
local Community	local Community	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Overall Spatial Completeness						$0.46$

,

Table A4. Coverage of spatial competency questions for the GAG ontology.

Competency Question	Possible between Regions	Not Possible between Regions	Coverage
Which regions are parents of ($contain$) region x?	$\left\{All\right\}$	∅	$100\%$
Examples	Which regions are parents of ($contains$) Municipality Unit of Bishop?
	Which regions are parents of ($contains$) Kefalonia (Regional Unit)?
Which regions are contained in ($inside$) region x?	$\left\{All\right\}$	∅	$100\%$
Examples	Which Municipality Units are contained in ($inside$) Florina (Municipality)?
	Which Municipalities are contained in ($inside$) Kefalonia (Regional Unit)?
SCQ1, SCQ2, SCQ3, SCQ4, SCQ7, SCQ8	∅	$\left\{All\right\}$	$0\%$
between 7 classes of regions
Overall Coverage			$25\%$

,

Table A5. Spatial Semantics in the geofla Ontology.

Domain	Range	Possible Relationships	Defined Relationships	Definable by QSR	Not Defined	Spatial Completeness
Commune	Canton	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Commune	Commune	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Canton	Arrondissement	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Canton	Canton	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Arrondissement	Department	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Arrondissement	Arrondissement	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Department	Region	$disjoint$, $within$, $contains$	$within$	$contains$, $disjoint$	∅	1
Department	Department	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Region	Region	$disjoint$, $touches$	∅	∅	$disjoint$, $touches$	0
Overall Spatial Completeness						$0.44$

,

Table A6. Coverage of spatial competency questions for the geofla ontology.

Competency Question	Possible between Regions	Not Possible between Regions	Coverage
Which regions are parents of ($contain$) region x?	$\left\{All\right\}$	∅	$100\%$
Examples	Which regions are parents of ($contain$) Montpellier (Commune)?
	Which regions are parents of ($contain$) Ermont (Canton)?
	Which regions are parents of ($contain$) Louvre (Arrondissement)?
Which regions are contained in ($inside$) region x?	$\left\{All\right\}$	∅	$100\%$
Examples	Which Communes are contained in ($inside$) Aveyron (Canton)?
	Which Cantons are contained in ($inside$) Thionville (Arrondissement)?
SCQ1, SCQ2, SCQ3, SCQ4, SCQ7, SCQ8	∅	$\left\{All\right\}$	$0\%$
between 5 classes of regions
Overall Coverage			$25\%$

and

Table A7. Structuredness and Readability metrics used for evaluating the ontologies.

Metric	Definition
AR	Attribute Richness is the average number of attributes (slots) per class.
IR	Inheritance Richness of the schema is defined as the average number of subclasses per class.
RR	Relationship Richness of a schema is the ratio of the number of (non-inheritance) relationships, divided by the total number of relationships defined in the schema.
AD	Average Depth measures the average depth of the vertical hierarchies.
MD	Maximal Depth is the maximum height of the graph.
AB	Average Breadth is the degree to which the ontology has horizontal hierarchies modelling.
MB	Maximal Breadth is the number of elements of each level/generation in the set of levels of the graph.
CR	Class Richness measures the ratio of classes with instances to the total of classes.
C.cmt	Class comment measures the ratio of classes with defined rdfs:comment to the total number of classes.
C.lbl	Class label measures the ratio of classes with defined rdfs:label to the total number of classes.
O.cmt	Object property comment measures the ratio of object properties with defined rdfs:comment to the total number of object properties.
O.lbl	Object property label measures the ratio of object properties with defined rdfs:label to the total number of object properties.
D.cmt	Data property comment measures the ratio of data properties with defined rdfs:comment to the total number of data properties.
D.lbl	Data property label measures the ratio of data properties with defined rdfs:label to the total number of data properties.

in the Appendix A. The OS ontologies are able to handle approximately half ($55\%$) of the possible competency questions, while the three other ontologies address only two of the possible eight questions; coverage score of ($25\%$). Thus, the proposed topological semantics can make a significant improvement in the semantic richness and usability of the geospatial ontologies. Summary of results for spatial completeness (SpCom) and competency questions coverage (CQCov) for the four ontologies is shown in Figure 5.

The measure of spatial completeness proposed here is a special type of general measure of semantic completeness for ontologies; which is a measure of the coverage of the concepts and relationships in the ontology usually in comparison to a gold standard ontology; considered to be complete. A data-driven approach to semantic completeness can also be used where the coverage is measured against a corpus of data, where information retrieval metrics of recall and precision are used for evaluation. In our case, the administrative geography maps provide the gold standard and the structure of the administrative divisions is used to identify the complete set of classes and possible spatial relationships between classes. Hence, the production of a gold standard for comparison is feasible and is systematically applicable to any type of administrative geography ontologies. The power of this metric is that it provides a clear indication of the gaps in the semantic knowledge as well as how to address it by defining the missing spatial relationships. The uniformity of representation is beneficial because it paves the way for the development of universal tools and languages for querying and manipulation of different geospatial ontologies.

Few studies have reported on the correlation between different quality metrics of ontologies, particularly whether completeness of the ontology is related to the complexity of its structure or to its readability, etc. [30]. Here, we present the results of evaluation of some selected metrics that are used in the literature to evaluate the structural quality of the ontologies and its readability [52,53]. The following structuredness metrics were used: schema metrics (Attribute Richness (AR), Relationship Richness (RR), Inheritance Richness (IR)), graph metrics (Average Depth (AD), Maximal Depth (MD), Average Breadth (AB), Maximal Breadth (MB)), and Class Richness (CR) as a knowledge base metric. Readability metrics used are Class comments and labels (C.cmt, C.lbl), Object properties comments and labels (O.cmt, O.lbl) and data properties comments and labels (d.cmt, d.lbl). A brief definition of these metrics is given in Table A7 in the Appendix A. OntoMetrics [54] was used to calculate the graph metrics, schema metrics, and knowledge base metrics from the ontologies directly. As for the readability metrics; these were computed using SPARQL queries over the GraphDB database that stores the ontologies.

Table 4. Results of metrics application over the four ontologies.

Metric	${\mathit{O}}_1$	${\mathit{O}}_2$	${\mathit{O}}_3$	${\mathit{O}}_4$
$SpCom$	0.72	0.5	0.46	0.44
$CQCov$	0.55	0.25	0.25	0.25
$AR$	0.42	0	0.44	0.40
$IR$	0.85	0.88	0.88	0.41
$RR$	0.73	0.2	0.42	0.72
$AD$	2.38	1.9	1.89	1.54
$MD$	3	2	2	3
$AB$	4.33	4.5	5	5.5
$MB$	8	8	9	15
$CR$	0.75	0.9	0.7	0.7
$C.cmt$	0.75	0.93	0	0.75
$C.lbl$	1	1	1	1
$O.cmt$	0.84	1	0	0.85
$O.lbl$	0.84	1	1	1
$D.cmt$	0.6	0	0	1
$D.lbl$	0.8	0	1	1

and Figure 6 shows the values of the metrics as applied over the four ontologies.

4.3. Overall Comparison of Quality

The metrics need to be normalised for comparison between different aspects of quality as some metrics are relative values (e.g., completeness) and some are absolute values (e.g., structural complexity and readability). We mapped the metrics’ values to subranges, where subrange 1 means the poorest quality of a specific metric, and subrange 5 is the highest quality, as shown in

Table 5. Mapping from metrics values to subranges.

Metric	Subrange
		1	2	3	4	5
Completeness	SpCom	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]
	CQCov	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]
Structuredness	AR	(0.8, 1]	(0.6, 0.8]	(0.4, 0.6]	(0.2, 0.4]	[0, 0.2]
	IR	(0.8, 1]	(0.6, 0.8]	(0.4, 0.6]	(0.2, 0.4]	[0, 0.2]
	RR	(0.8, 1]	(0.6, 0.8]	(0.4, 0.6]	(0.2, 0.4]	[0, 0.2]
	CR	(0.8, 1]	(0.6, 0.8]	(0.4, 0.6]	(0.2, 0.4]	[0, 0.2]
	AD	[1.5, 1.8]	(1.8, 2.1]	(2.1, 2.4]	(2.4, 2.7]	(2.7, 3]
	MD	[1.5, 1.8]	(1.8, 2.1]	(2.1, 2.4]	(2.4, 2.7]	(2.7, 3]
	AB	[4, 4.3]	(4.3, 4.6]	(4.6, 4.9]	(4.9, 5.2]	(5.2, 5.5]
	MB	[8, 9.4]	(9.4, 10.8]	(10.8, 12.2]	(12.2, 13.6]	(13.6, 15]
Readability	C.cmt	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]
	C.lbl	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]
	O.cmt	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]
	O.lbl	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]
	D.cmt	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]
	D.lbl	[0, 0.2]	(0.2, 0.4]	(0.4, 0.6]	(0.6, 0.8]	(0.8, 1]

. Figure 7 shows the ontology scores after normalisation. Once the data has been normalised, the completeness, structuredness, and readability metrics can be aggregated into single values by taking the average. A combined result of all quality metrics is shown in Figure 8. As shown in the figure, the OS ontology is more spatially complete in comparison to the three others, which are equally spatially complete. The degree of structuredness or readability of the geospatial ontologies do not correlate with their spatial completeness; e.g., IGN geofla is superior to all others with respect to both measures, but is inferior with respect to completeness. Hence, it can be seen how this new measure of completeness provides a useful complementary metric for evaluating geospatial ontologies.

5. Conclusions

This work presents a novel measure of quality of geospatial ontologies. With a focus on administrative geography open data and their ontologies, this work identified a set of topological and proximity semantics, which can be explicitly defined (and inferred by spatial reasoning) that captures the spatial semantics between their component regions. A uniform measure of spatial completeness is proposed and interpreted as a set of competency questions that, together, can be used to evaluate the completeness of geospatial ontologies in this domain. Four European administrative geography ontologies and datasets were analysed and evaluated. It is shown how the metrics can be homogeneously applied and computed for different ontologies with different levels in their hierarchical divisions. The proposed measures provide an objective view of spatial completeness of the ontologies and explain the gaps in their representations. The proposed spatial completeness measures are novel and significant as they can allow a homogeneous definition of ontologies across different countries and can therefore support data sharing and integration. The measures complement the established methods in the literature, that primarily focus on the syntactical and structural dimensions of the ontologies, and offers a novel approach to ontology evaluation in the geospatial domain. Research points that would be worth studying in the future include: a) methods of encoding the proposed semantics, including spatial reasoning, in the ontologies to allow for the automatic building of ontologies and checking and maintaining their consistency, and b) the reuse of spatial data standards and designing frameworks, that can support their integration with the proposed semantics and metrics, for the sharing of geospatial open data.