The rapid development of geospatial data collection techniques, and the growth of the World Wide Web (web) for a wide range of “geo”-applications, has resulted in recent increases in the availability of geospatial data. The number and variety of sources of geospatial data, exemplified by sites such as the OpenStreetMap project, the (British) Ordnance Survey (OS) OpenSpace service, and the US Census Bureau, show that it is possible to create, revise, download, distribute, embed and add value to, an enormous range of data. Progress in web services has resulted in thousands of Web Map Servers (WMS) providing hundreds of thousands of readily usable map layers, and Web Feature Servers (WFS) supplying raw data of enormous variety and provenance.

The “democratising” nature of the web means that such data can result from efforts of both professional geospatial scientists or engineers, providing authoritative, official, “formal” data, and enthusiastic amateurs with no formal training in collection, processing and analysis, who can contribute “informal” information. The collaboration of such distinct groups in contributing, integrating, archiving and disseminating “geo”-data is not currently undertaken in a regular and systematic way. However, it is clear that, in order for extensive and ambitious strategies such as multi-national spatial data infrastructures to succeed, input from a range of mapping projects and data sources is required, and mechanisms for handling geospatial data of varying sources need to be developed. For example, the INSPIRE (Infrastructure for Spatial Information in Europe) initiative, which initially aimed to create a Europe-wide spatial data infrastructure (SDI) exclusively from governmental and official geospatial datasets [1], could potentially benefit from additionally considering cheaper and more timely contributions from citizen-based sources [2].

Such integration of informal and formal datasets must also be considered, and indeed is increasingly implemented, in other contemporary projects such as development of databases for in-car navigation systems, and rapid mapping solutions in emergency response situations. Further examples of the integration of formal and informal datasets include the use of informal volunteered geographic information (VGI) in site-engineering projects, in developing city-centre pedestrian databases [3] and in utilities projects, notably in developing countries [4]. As critical decision-making may rely on effective integration, it is important to consider its feasibility, notably by assessing the properties and characteristics of the varying contributing types of data, such as its quality [5]. Variability in accuracy of location, attribute coding, time-stamp, geographic scale, and other components of data quality, can be measured and judged and the evaluation of relative geospatial data quality is an important and necessary pre-requisite to successful data integration and effective collaborative mapping.

The development of such mechanisms requires a coordinated approach to data collection, and also a valid method for data integration. This paper presents a customised tool which is intended to help users judge the viability of data integration, and thus improve the frequency and quality of multiple data source conflation for mapping and SDI database creation.

It can be difficult to effectively make such an assessment of the varying value and relevance of a range of data sourced from differing communities. There is a distinct contrast between those who capture and collate geospatial data as members of the public, contributing User Generated Content (UGC) (which is not limited to spatial data, although specifically geospatial UGC is usually called Volunteered Geographic Information, or VGI) and those state-sponsored or commercial organisations which provide official or Formal Data (FD). Differences between the resultant datasets, in terms of measurement source, quality, coverage and purpose, are evident: for example, low cost GNSS receivers and the availability of GNSS signals make it possible for the public to acquire positional information about different locations using standard mobile phone handsets and upload it to local or personal databases; whilst access to virtual reference stations (VRS) and networks of high accuracy correction facilities used as standard by national mapping agencies give significantly different results even for basic point positioning. The mapping based on volunteered efforts, personal computers and the Internet [6] is different in many respects to traditional large-scale topographic mapping projects, long the hallmark of official governmental agencies.

The variable quality (including variation in inherent accuracy) of Volunteered Geographic Information can result from a combination of an untrained data collector, unsystematic approaches to data collection over extensive spatial extents, and low-accuracy capture technology. However, its ready availability, often at no financial cost, can make its use for supplementing, enhancing or updating Formal Data appealing: the result may be the type of collaborative mapping and geospatial database development which can be applied to the types of activities mentioned above.

Geospatial data integration involves the bringing together of discrete items and sets of data which often reveal inconsistency and incompleteness. Such data integration is not just a matter of overlaying a set of layers in a geographic information system, but also assessing how well the positional, geometric and other properties of features in one dataset can be converged with the other [7]. In order for such convergence, and ultimate data integration to assist in the development of SDIs and other projects, both positional and geometric correspondences of datasets, amongst other properties, have to be established. It is these properties of geospatial datasets which are considered in this paper, and in particular, a comparison of their relative positional and geometric accuracies—an area of significant long-standing interest in the fields of surveying and geomatics [8]—has been effected in a tool which can be used to summarise the integration possibilities for informal and formal datasets.

The results of the research described here include a system which can allow those who are attempting to integrate datasets to determine whether they can be completely or partially combined, or not at all. The eventual judgement of this depends on the user, but the tools provided will assist in such judgement being informed by a quantitative assessment and a visual representation of the comparison between sample geospatial datasets. The datasets consist of point, line and polygon features which are examined, using established standards for spatial data accuracy to consider point positions; buffering and overlay methods for comparing linear data; and the calculation of “moments invariant” for comparing areal objects. Further discussion considers the utility of such a system, and its contribution to collaborative mapping projects.

A major example of an extensive informal geospatial data archive is the OpenStreetMap (OSM) project [9]. Anybody can access and contribute data to the geospatial datasets in OSM without any restrictions or limitations. The OSM information, and its underlying map data, can be viewed freely, uploaded by creating a user account, and edited online through a wiki-like interface. The free accessibility of OSM data (licence, cost, sharing) and opening up of a new paradigm of “peoples’ geo-data” can be considered the main profit of this project. “Mapping parties” and systematic data collection efforts involve thousands of amateur geospatial data collectors. Many more web users are able to access current OSM data without any charge. The OSM sources allow visitors to search anywhere on a world map and download different portions remotely. However, the level of detail can vary widely, and in addition, the problem of heterogeneous data quality is evident when handling such highly variable geospatial data.

Several studies have subjected the OSM data to tests of quality, addressing issues such as completeness, topicality, and accuracy. For example, Zielstra and Zipf [10] studied the quality of the route network in OSM data in Germany, comparing it with the commercial Tele Atlas dataset. The comparisons revealed that the positional accuracy of the road lines was sufficient to allow OSM data to be used for many routing applications, but that there was a shortcoming in the completeness of the regional OSM datasets. Haklay [11] used similar techniques of completeness testing to conclude that a comparison of OSM data with smaller-scale OS data is favourable. Girres and Touya [12] considered both the positional accuracy of OSM data compared to French national mapping agency data (the average displacement of point locations was 6.65 m) and the classification of features (which showed consistency for major roads, but less agreement on classification of minor routes).

These studies have tended to address the quality of VGI data at small scales (typically map scales of 1:50,000 and smaller). By contrast, Al-Bakri and Fairbairn [13] assessed geospatial data quality at much larger scales, reporting on positional accuracy testing at scales equivalent to 1:2,500, and also considering the semantic classification accuracy of the feature coding and attribute designation of the objects in the OSM data [14]. The geometrical accuracy testing is a fundamental component of the tool demonstrated here, and is quantitatively reported to the user on-screen. The relative matching of the informal OSM data with the formal data produced by an official topographic mapping agency, the Ordnance Survey of Great Britain (OS) is examined. Another comparison was undertaken of OSM data with official mapping agency data from the Iraqi General Directorate of Survey (GDS). In addition, further datasets have been gathered by trained surveyors using higher-order field survey (FS) to determine whether the introduction of a more rigorous data collection regime can yield positional data which matches the formal, systematic, official data, and which could be integrated with it for the purposes of collaborative mapping.

It was shown in Al-Bakri and Fairbairn [13] that matching of OS and OSM data is more accurate in areas of cultural (anthropogenic) detail, whilst the “fuzzy” nature of some natural topographic features means that geometric discrepancies between OS and OSM data in rural areas are likely to be larger. Two OS datasets were sampled, one of an urban area where “hard” detail (buildings, kerb lines, and street furniture) predominates, another of a more rural area where “soft” detail (vegetation boundaries, water edges, terrain features) is the more common type of object. The former was situated in the town of Cramlington, UK, whilst the “soft” detail was characterised by the environment around the village of Clara Vale, UK. The Iraqi dataset selected was for an urban area, close to the centre of Baghdad, but also covering a more extensive university campus estate. For each area, the formal, accurate, traditionally-derived Ordnance Survey MasterMap data or GDS dataset was tested against the open access data from web-based VGI (i.e., OpenStreetMap), and also against the field survey undertaken by experienced surveyors (Figure 1, Figure 2, Figure 3).

The large scale MasterMap data has been created over many years of successive Ordnance Survey (OS) revision, using a mixture of control survey, photogrammetric plotting, and field and GPS survey updates. The General Directorate for Survey (GDS) dataset is sourced from photogrammetric plotting using rectified and geo-referenced air-survey imagery. By contrast, most of the OpenStreetMap (OSM) data in the UK study areas has been collected by “lower order” data collection, primarily GPS tracking, whilst for the Iraq case study it has been manually traced from Yahoo satellite imagery, not registered by accurate ground control and therefore “geo-referenced” visually. The regulation field survey (FS) was captured by establishing a reference framework using Leica and Topcon GPS receivers, then picking up detail using total stations, or GPS in RTK mode in some open areas. It is suggested that positional variability between these User Generated Content datasets and Formal Data is dependent on associated influences such as the experience of the surveyor, and the type of instrument used to capture data.

This project, therefore, addressed a range of factors including type of detail mapped (“hard”; “soft”), agency undertaking data collection (official suppliers of formal data in Great Britain and in Iraq), data collection method (field survey; handheld GPS unit; tracing from aerial imagery), and experience of the individual surveyor (professional mapper; interested citizen) expecting each factor to have some effect on the results obtained. Tests were undertaken addressing comparative positional accuracy of points (including corner points of polygons, intersections of line features, and individual point features), directional measures of discrepancies between datasets, relative variation in paths of linear features (employing buffering techniques), and a comparison of geometric properties (including moments) of polygons. In each case, standard quantitative testing methodologies have been applied such that the reporting of results, both numerically and visually, shows valid and useful summaries of relative accuracy and allows for an assessment of possible integration.

When topographic line features such as complex rivers and boundaries, or even simple roads or railways, are considered, comparison using point accuracy may be insufficient to capture the spatial complexity and variability of the dataset objects. Point measures may be straightforward to understand and calculate, but they cannot capture all aspects of linear matching. Thus, in addition to considering positional discrepancies of points along the compared lines, linear matching procedures can be usefully examined to assist in developing an integration assessment tool, using techniques of determining shape or curvature similarity between two compared lines. Buffering methods, and assessment of buffer overlay, have been popular techniques [29,30]. Buffers of variable size can be constructed around lines in each dataset (e.g., line X buffer XB, line Q buffer QB), within which statistical calculations can be carried out. The following measurements are taken: (2) (3) (4) (5)

These measures can be used to give the deviation of a line in one dataset from the line of another, or to compare two lines together. The average displacement between the two lines, for example, can be calculated as: (6) where: 2bs_i: The total buffering width that has been generated around the centre of the line.

Iteratively increasing the buffer size will yield differing results when measuring displacement of lines or overlap of buffer areas, but these will stabilise at some appropriate buffer size. Once again, the OS, GDS, FS and OSM datasets were examined, with a comparison carried out primarily for some road centre-lines in the urban Cramlington, UK and Baghdad, Iraq study areas. Buffer sizes have been incremented from 0.5 m to 12.5 m, in 0.5 m intervals for comparisons involving informal OSM data.

Graphs reveal that the average displacement of comparative pairs of lines reaches a sill as buffer size increases. The significant differences between the OSM data and the formal dataset in the same area are considerably larger than any comparison of the more rigorous field survey (FS) with formal data: the average displacement for the comparison of OS- OSM (solid line in Figure 6, Cramlington) is approaching 3.5 m when the buffer size is 12.5 m. Figure 7 (Baghdad) similarly shows that the FS data is very close to GDS datasets (average displacement about 0.7m for buffer size of 0.5 m beyond which the graph remains constant) but comparing the OSM data to GDS shows a displacement of about 5 m for a buffer size of 5 m.

It is possible to also interpret the slope of the graphs, with steeper gradients indicating greater variability and confirming the lower levels of matching. Further investigation of displacement information can be undertaken with reference to the buffer measures already taken. For example, the areas outside the buffers of line features in one dataset can be compared to the areas inside the buffers of the second dataset. The characteristic curve for such graphs is downward towards the x-axis.

The conclusion with reference to the linear data is similar to that obtained from the point position comparison. The path of major line features in the OSM dataset can diverge significantly from the official dataset. Although the displacement statistics and the overlap percentages might favour the possible integration of the formal data with the informal, but rigorously collected field survey data, they would caution against integration of the OSM data with the formal data.

The measures of matching of datasets detailed above were incorporated into a visual tool which can be used to assist in judging the potential for data integration. The intention was to create a user-friendly interface incorporating quantitative and visual analysis of dataset comparison that could be used to assess geometrical similarity of tested datasets. The processing steps in point, line and area matching were developed in the MATLAB software package, and a graphical user interface (GUI) was created also to report on the results of calculations and visualization of the comparisons [35]. With this user-controlled interface, data can be input as text-based files of planimetric coordinates from the datasets, computations are applied using the formulae and methodology indicated above, and the graphical representation of the input data and the analysis can also be exported in a report. Each dataset, formal (FM), official data (consisting of OS and GDS data), rigorously collected field survey data (FS), and the informal OpenStreetMap (OSM) data, has been examined using this interface, examples of which are shown below. Point, linear and areal comparisons are made for each combination of datasets, although the most important is the direct comparison of the formal and the informal data.

Although these user-friendly interfaces are shown to be effective in reporting the input, computation, mapping and output of data, there is no specific decision-making function attached to the system. No thresholds of acceptability are presented to guide the user, as it is felt that considered judgment should be the main determinant of whether integration of data should be recommended. The major factor, in any data integration exercise, is, of course, the eventual application of any combined dataset. As has been indicated at the start of this paper, the range of potential application areas for the integration of formal and informal data is enormous, and each will make demands on the datasets used in their execution. What is appropriate for one task may not be so for another. This is particularly true of a core characteristic of a geospatial dataset, its scale. The focus of this work has been on large scale data and it is suggested here that significant attention be paid to this aspect before any dataset integration is considered.

Figure 9 shows an example MATLAB GUI reporting the comparison between the point positions, in the formal Ordnance Survey data (here taken as the “reference data”), compared to the OpenStreetMap data for the rural study area, in Clara Vale, UK. The positional, descriptive and directional statistics are reported numerically, along with graphical rendering of the magnitude of discrepancies, plotted on a scaled map and a summary vector diagram.

Similarly, Figure 10 shows the GUI which was designed for assessing linear matching, in this case comparing the formal data (FM), with the rigorous field survey (FS), and the informal OSM data, in the urban study area (Cramlington, UK). Here, calculations of the percentage buffer overlap are presented in numerical and graphical form, along with an overlay of the linear features (in this case, road centre-lines). The calculated average displacement graph shows all three datasets simultaneously, whilst the linear features are accurately mapped in the final window in this interface.

Finally, Figure 11 demonstrates the polygon-based interface, summarizing the moments calculations, and again providing a mapped overlay, for visual judgment of discrepancies. Here, the seven moments invariant are calculated for each dataset: FM, the formal data, is part of the GDS dataset for metropolitan Baghdad, Iraq, whilst the field survey (FS) and OSM data are compared alongside. The overall position of the summary moments value for each dataset is plotted in seven-dimensional space, and the bar chart in the GUI shows the distance from one dataset to another. The simple datasets considered here are shown in map form, also.

A methodology and system has been developed to derive measures for assessing shape and positional quality among datasets for integration purposes. Different methods have been used for obtaining valuable descriptions of geometric matching (based on positional accuracy and shape fidelity). The NSSDA has been chosen to select and analyse tested points, then calculate the RMSE values, yielding comparative measures of positional discrepancy. For error direction analysis, circular statistics have been adopted. Linear geometric similarity has been tested by establishing buffers around the lines in the relevant datasets, with analysis of the relative uniformity of such buffer zones (union, intersection, etc.). Shape metrics, including moments invariant, have been applied to evaluate polygon matching.

The results of this analysis show that comparing official datasets (the OS and GDS data) with rigorous FS datasets gives good matching, especially for areas of “hard” detail. However, the positional and shape measurements of the informal OSM data do not match the formal data in any of the case study areas examined. The conclusion is that, when examined from a geometrical matching perspective, there would be significant problems in trying to integrate the OSM data with official data, at the large scales considered here. The major intention of this paper has not been to assess the accuracy of OSM against a perceived “truth”, rather to undertake some comparison. The measured discrepancies which have resulted from the work described here clearly militate against integration. A major reason for the discrepancies noted is obviously the varying data collection procedures: the VGI presented in the OSM dataset can be sourced from GPS devices of significantly varying precision, from satellite images or aerial photographs, or simply from local knowledge. The “source” tag attached to each feature in the OSM dataset is commonly used to specify the origin of the data [36]. This can be used to gain a preliminary view of the accuracy of the geometry: an OSM feature, tagged with source “landsat”, will usually be less accurately captured than one obtained by “GPS tracking”. Even for those features captured in the field using GPS, the majority have been sampled using low-accuracy GPS receivers such as Garmin, Holux GPSlim236 and iPhone GPS. Occasionally, features in OSM will be averaged by eye from many GPS positions and tracks recorded by several individuals over time, and often detail is inferred rather than actually measured, due to access problems or lack of experience in field survey. For those OSM features digitized from Yahoo aerial or satellite imagery (the standard data source for those contributors remote from the mapped area), lack of familiarity may further degrade quality. The recognition of features such as one way streets or dual carriageways can be difficult from small-scale imagery. A local contributor is likely to have better knowledge about an area of interest, as well as having access to the locality, but much OSM data is remotely mapped (for example, some of the OSM data in Baghdad has been created by an OSM contributor living in UK). In addition to local knowledge, it may be that further independent datasets and expert opinion can assist in enhancing the OSM dataset’s quality: the addition of a field survey dataset in the studies reported in this paper could be equated to an extra moderating input into the comparison process. Comber et al. [37] have shown how such expert knowledge can enhance the accuracy of VGI data captured for land use survey. There are further personal attributes, such as experience and abilities of the VGI collector, which will affect the OSM dataset’s integrity and value.

The nature of the feature type is also a determinant of the observed discrepancies. Al-Bakri and Fairbairn [13] have shown that rural and natural VGI data (soft detail) has a lower positional accuracy and completeness compared to the “hard”, built environment, data collected in urban sites. Integrating formal and informal datasets in urban areas could therefore be done with greater confidence than in rural locations.

The measurement of accuracy of geospatial datasets and the assessment of possible integration goes beyond positional and geometric issues. The meaning of symbols and the categorisation of objects in the real world are addressed by the semantic data added to, or derived from, the measured coordinated dataset. A further study of the semantics of each dataset has been undertaken [14], and a measure of “closeness” of attribute information was attempted, measuring the semantic similarity of XML schema elements in order to additionally assess the possibilities of data integration from a range of “crowdsourced” and official sources. The measurement of semantic similarity between OSM and official datasets in this study yielded mismatching similar to the tests on geometric properties. Further extensions to this methodology could address temporal accuracy, completeness and lineage, all comparable characteristics.

It should also be recognised that VGI can consist not only of measured data from the field or from remotely-sensed imagery, but also user created photographs (e.g., Flickr), textual descriptions and user updating of geospatial datasets (e.g., TomTom MapShare, Google MapMaker): these can also contribute to “crowdsourced” UGC and their accuracy can be examined using techniques outlined here.

As the potential use of VGI is given increasing prominence, and “crowdsourced” data is presented as both an initial dataset, for example in areas where large-scale mapping does not exist (such as in remote areas hit by disasters), and as a contributor to integrated spatial data infrastructures (including updating of official datasets), it is important to urge caution. A valid implication of this study is that OSM data could be described as being of lower accuracy that the formal datasets. The positional and geometric accuracy of VGI can, therefore, give cause for concern: its presentation as a panacea for the improvement of out-dated, large-scale, official datasets should be viewed with scepticism in most cases, and with rejection in many. In some of the applications suggested at the start of this paper, including in-car navigation systems, on site-engineering and utilities projects, and any systems based on accurate land-parcel delineation, it is probably better to have no mapping at all, rather than some inaccurate, possibly incomplete, user generated content. Although the involvement of local communities and enthusiastic individuals and projects in extending the range of geospatial datasets is to be welcome, it is clear that such involvement must be considered as part of a wider and systematic programme of mapping activity, exhibiting consistency, care and checking.

The MATLAB GUI code which was developed to produce the interface used in undertaking the assessment is presented in Al-Bakri [38].