Usability of WebXR Visualizations in Urban Planning

Abstract

Extended reality (XR) technology is increasingly often considered in practical applications related to urban planning and smart city management. It offers many advantages as a new visualization technique that gives its users access to places that are not available in material space and a unique perspective on existing objects. It can provide immersive multi-sensory experience that can induce emotional response in participatory planning. However, standard mode of implementation that relies on mobile phone applications and VR headsets has a disadvantage when it comes to availability and accessibility. Here we test the WebXR solution that can mitigate those problems. We have created six AR and VR environments that resembled common urban planning scenarios and conducted usability tests with people having planning and GIS background. Results indicate that WebXR can provide useful solution in urban planning when the interface and environment resemble common practices and situations encountered in real life. Environments that have introduced new digital affordances like AR measurements or semi-transparent walkable scale models were rated lower. Users evaluated presented environment as having high usability and expressed their positive attitude toward using XR in their professional practice mainly as a participatory and visualization tool.

1. Introduction

For a long time, urban planning has been tightly connected to the theoretical and practical developments within the domain of geographical information systems (GIS) [1,2]. New technologies that deal with spatial data are readily incorporated into planning frameworks [3,4] and the reliance on smart city paradigms and the overall increase in smartness of the cities make it even more important than before [5,6,7]. However, the smart city approach and the use of GIS has been also strongly criticized before as solutionism and technocratism [8,9,10]. This is often quite true with numerous examples of big capital using the drive for smart city to take advantage of the data that cities and urban planners are willing to give out in return for a vision of efficient management and planning [11,12]. Recent conceptualizations of the smart city therefore put more emphasis on participation and human capital and not on algorithmic decision making [13,14]. In this paper we take a look at the WebXR as a relatively new addition to the spectrum of extended reality (XR) technologies, one that could be used in urban planning, especially in the context of public participation in the smart city. This development has been made possible by current technological advances and increase in computing power, especially in mobile devices. WebXR possesses certain characteristics that make it a promising tool for engagement with the public in such circumstances. It provides the environment for a wide range of interactive spatial visualizations, the possibility of creating Virtual Geographic Environments (VGE) [15], the engagement factor resulting from its newness and specific set of affordances that allows for immersive experience, and finally—the unrivaled accessibility. The last two features on this list—immersiveness and accessibility, are especially interesting. Immersion is a unique selling point of VR [16]. The capability of inducing the feeling of presence or tele-presence [17] where through multi-modal stimuli a person experiences ‘being in a different place’ is compelling withing domains of geography and GIS that deal with space and place on a daily basis. Mainly due to the potential of giving users the feeling of immersion virtual reality (VR) and augmented reality (AR) were previously considered to be valuable tools for communication in urban planning [18,19,20] and in GIS in general [21]. Especially VR for a long time has been seen as a promising tool for geographical inquiry and visualization and for creation of a new kind of spatial experiences [22,23]. This became even more pronounced in recent years with the advent and rising popularity of technologies such as digital earth [24], digital twins [25] and finally spatial computing [26], that bring the possibility of high fidelity XR experience through mobile devices.

But traditional modes of delivering the content through headset-only applications and even mobile applications have certain limitation from technological, design and human perspectives [27]. WebXR offers at least a partial solution to these issues since it excels in accessibility and interoperability by making possible to use visualizations either in its intended virtual reality or augmented reality modes or simply using internet browser [28]. However, a wide range of output modes means that there are additional challenges in user interface design. Both the interactive web-based visualization and virtual reality environments have specific requirements that need to be met to achieve satisfying levels of usability. Without it, the vision of WebXR being a vehicle for greater participation and engagement will be hard or even impossible to achieve and it will join a number of other failed attempts in this regard.

The main aim of this paper is therefore to shed light on the usability and practical issues connected to the possible use of WebXR within spatial and urban planning context. Our approach was to conduct an exploratory study (i.e., no hypotheses were formed in advance) of the technology through the people that could be using WebXR-based spatial visualizations in their work—urban planners, both academics and practitioners, and other experts actively involved in the planning process. This allowed us to investigate the possible use scenarios and gather information with a relatively small sample size. While this study is limited in its scope (see the Study Limitations section at the end of the paper), it nevertheless provides a much-needed start to a discussion on this technology use and development. We ask the following research questions: How could WebXR technology be applied to spatial planning? How could WebXR technology influence current practices in spatial planning? How do academics and practitioners perceive WebXR and what do they know about it?; Is WebXR a viable technology from the usability perspective to be included in the spatial planning process?

In the next two sections of our introduction, we would like to present the current state of discourse on XR and certain terminological issues that arose during its development, as well as our position on this matter. In the last section we will also introduce selected examples of XR applications in urban spatial planning and management that show the importance of the issues we talk about in this paper.

1.1. XR Spectrum and Terminology

Extended reality is an umbrella term encompassing the whole range of technologies and phenomena that describe various interactions and interweaving of digital and material spaces and places. We deliberately choose here to use ‘space/place’ instead of ‘reality’ to avoid making an unnecessary distinction between the real and unreal and to take a position in line with virtual digitalism [29]. The term XR has also been interpreted to be a placeholder for (A)ugmented, (V)irtual and (Mixed) Realities [30] and as cross-reality—the union between physical sensor networks and virtual worlds where machine and human perceptions meet [31]. However, in this paper we interpret XR as extended reality that encompasses AR, VR and MR technologies, which is much more widely accepted [27]. The idea of a spectrum was conceived by Milgram and Kishino [32] as a taxonomy of mixed reality visual displays. On one end of the spectrum there is reality—the material world without digital content. When we add digital overlays (e.g., 3D models of future or buildings) we are augmenting the reality and this can be described as AR. Finally at the other end of the spectrum there are virtual worlds generated by computer simulation—VR (e.g., whole urban neighborhood that is accessible only by using VR headset). While concept of the spectrum is still influential, the advances in both technology and philosophical discourse made it insufficient to describe the current state of the matter. For example, the term mixed reality (MR) has been adopted to describe only a part of the spectrum that is closely associated with technologies like Microsoft Hololens [33] or Magic Leap [34] that utilize spatial referencing to connect digital object (holograms) to material spaces. And while this is far from uniquely accepted definition of MR [35], it shows how the understanding changed with time. Similarly, there are many more definitions of the other parts of the continuum. Most closely related to MR is AR, which is sometimes defined very similarly to MR. In one of the classic definitions [36] AR combines the real space and objects with computer-generated (virtual) objects, registers them together and runs interactively in real time. The main difference between MR and AR is that in the case of the latter virtual objects are not only superimposed on the material world but they can be also interacted with [37,38]. The difference can also be seen in spatial reference frames used in the former [27]. Digital MR objects are referenced to material space—for example a hologram can be seen overlaying a specific building while in the AR case a hologram model of a building can be positioned on any flat surface. Another

The line between VR and other parts of the spectrum is much more clearly defined. Still, there are many definitions of what constitutes the VR itself. It is a technology that has a long history and that recently has re-emerged in the public life and scientific discourse alike [16]. The simplest but nonetheless accurate definition portrays VR as ‘an alternate world filled with computer-generated images’ [39]. This already suggests the experience of ‘being somewhere else’/‘being there’ (in virtual world) as something most important, that became a staple of modern VR definitions [27,40,41]. In addition to this, some authors list the necessary features for a technology to be considered VR. For example, Heim [42] formulated three Is of VR—immersion, interactivity, information intensity and MacEachreen et al. [43] added the fourth I with intelligence of objects. Similarly to this, Sherman and Craig [44] listed four elements that together form VR experience: a virtual world, immersion, sensory feedback and interactivity. While all this adds up to form VR, it is currently unclear what the most important factor is. Evans [16] brings forward the fact that the orientation and activity of the user toward achieving immersion is also crucial. What’s more, there are also non-immersive virtual environments that are considered to be a virtual reality, especially in the field of medicine [45]. It is also worth noting that within the field of GIS, definition of VR closely overlaps with the idea of virtual geographic environments (VGE) that were defined by Lin and Gong [46] as ‘pertaining to the relationship between post-humans and 3-D virtual worlds’ and that are evolving.

WebXR is a technology that does not by itself bring a new type of display to the XR continuum but rather makes it possible to a deploy virtual worlds within web environments. It is the main building block of an immersive web movement, concentrated around W3C Immersive Web Community Group 1 [47]. Currently the main standard for implementation of WebXR is the WebXR Device API 2 (XRDA) that provides the platform, independent interface and which unites both AR and VR devices around the core capabilities common to both of these display types [28]. Therefore, it makes it possible to develop an application that will work either in VR and AR depending on the type of device that the end-user will have and it will work in most of the available platforms. And while the idea of delivering GIS VR content via internet media has already been proposed and developed before by Huang and Lin [48,49] recent advances are much more accessible and have a potential to become widespread standards.

1.2. XR in Urban Planning and GIS

Technologies that belong to the XR spectrum for a long time have been seen as a promising possibility in GIS and spatial planning [50]. In his review of trends and directions of virtual reality GIS (VRGIS) Haklay [51] identifies already a rich landscape of applications and projects relating to various attempts of merging those two domains. Haklay also points out to urban planning as the most dominant field in VRGIS which stems from the long tradition of visual scale models used as a means of communication. Despite high expectations VR failed to deliver tangible results and in the next few years no major developments were seen. Still, ideas like VGE were actively pursued as their potential for use in geography is high [15,48,52]. Only in recent years, when VR technology matured enough and became popular once more—in what Evans [16] called re-emergence of VR, the renewed interest became visible in the fields of urban planning and GIS. This coincided also with advances in AR which were seen as equally promising in urban planning [53] but were held back even more by the lack of proper devices to deploy it on. It all changed when VR headsets such as Oculus Quest became consumer technology and smartphones became powerful enough to smoothly run AR environments. There are also more advanced devices like Hololens and Magic Leap that offer custom functionalities for more demanding environments. This all resulted in various projects in urban planning and connected disciplines like architecture [54] and collaborative design [55]. XR can offer to stakeholders of the planning process numerous new opportunities: access to places that are not available to visit in material space [56], immersive multi-sensory experience that can induce emotional response in participatory exercises [57,58], new ways for creative interventions [59], ability to compare existing and planned buildings in the same time and space [60] and possibility of better visualization of elements of the urban tissue such as underground infrastructure [61], air quality [62], or even whole models of urban ecology [63]. XR environments in comparison to non-immersive 3D visualisation often used in urban planning offer some advantages. The most important one is that through immersion they can provide the correct circumstances to induce the feeling of spatial presence—being in another location. This can in turn emotional engage the user, which is crucial in participatory planning and help with understanding of the changes in the urban space. For example it can be much easier for users to perceive scale in XR.

However, apart from the technology itself, there are other obstacles and limitations that stand in the way of a more widespread adoption of XR in practice. One of them is the lack of incentives to produce and maintain good quality 3D models [64] that could form the basis for virtual representations of planning projects. Current practices are simply ‘good enough’ and there is motivation to try an unproven new approach. That being said, we think that the participatory paradigm in spatial planning is the domain that would most readily benefit from the introduction of WebXR technology. It has already been shown in multiple studies that XR can be used to foster better mutual communication between stakeholders [65], to gather valuable insights for planners [66], to serve as a multisensory community planning platform [57], to improve useful engagement in the process compared to traditional methods [67] and that it is an overall effective tool for participatory planning [68]. And the unique characteristics of WebXR mode of XR development seems to be fitting for this particular case of application even more because of accessibility and ease of use. This is the assumption we test in the following study through the usability test and self-ethnographic process of software development.

2. Materials and Methods

To answer our questions, we adopted two methods. Firstly we created a WebXR application that included both VR and AR environments as case studies. This allowed us to observe, using auto-ethnography method, the process of the software development itself as a practice that is beneficial in HCI studies [69]. Neither of us had had any previous experiences in developing virtual environments and this experience confronted us with the current state of the art in this particular technology and unveiled the limitation of its use in various fields. It was important to be able to test VR and AR as they reside in different places on the virtuality continuum and thus provide different digital affordances for developers and users alike. The final product was used for usability test with a group of potential end-users and to encourage them to share their views and experiences using WebXR as well as perception of its potential as a tool in spatial planning and management. As our case study we used our home city Poznan since it has a public database of 3D models of all buildings within its city limits. Below we present details of each method.

2.1. Development of the WebXR Application

The web application was designed to consist of six separate XR environments (

Table 1. XR environments created for usability tests.

Name	Description	Screenshot Example
AR1—Display	Displaying a detailed 3D model on a flat surface
AR2—Measuring	Measuring distance on any flat surface and recalculation in a given scale
VR1—Display	Virtual gallery of four different 3D scale models. Users can navigate freely through the environment
VR2—Zoning map	Virtual 3D display of a small area in Poznan with buildings and the zoning map with a legend
VR3—Area renovation proposition	3D visualization of proposed changes in one of Poznan parks. Users can walk and run through the model.
VR4—Virtual walk	Poznan Old City 3D model rendered with textures. Users can walk and run through the model.

), four in VR and two in AR, created using mainly three tools: three.js library [70], WebXR API [71] and Canvas UI library [72]. The first was used for creating, rendering and animating 3D models in a browser through WebGL. The second one added VR and AR capabilities and finally Canvas UI provided XR user interface.

Each environment was different in regard to its content, models that were used to create it, provided modes of interaction and the initial frame of reference for the user. For 3D models of buildings, we used data provided by the official Poznan Spatial Information System [73] which included LoD2 data. Model VR3—Area renovations was kindly provided separately by Poznan Board of Geodesy and City Cadastre GEOPOZ and it is based on real data used in public participation process and it supplemental 3D models and sky textures were added to provide more context. For model VR1—Display and VR4—Virtual walk we also used Google Earth data imported to Blender. In all cases models needed to be corrected using Blender and BlenderGIS plugin [74] and exported to GLTF format. All the code can be accessed on Github repository and the working application can be found under the following URL: https://moonshroom.github.io/Spatial_WebXR/ (accessed on 23 October 2021) [75].

Each environment was created as a separate entity to test different capabilities and affordances of WebXR. Apart from AR2, which provides basic interaction capability, all the cases present some form of visualizations of the urban environment. They were created to place users in different situations—in AR1, VR1 and VR2 they observe 3D analogues of physical scale models (Figure 1) while in VR3 and VR4 they are placed inside the model and can gauge its spatial scale from the “frog perspective” (Figure 2). Those are main modes of access that in our opinion can be seen practical to include within spatial planning process. In each stage of the software development process, we made notes about choices that were made and limitation imposed on the application design by the available technology. Our insistence on using Open Source Software influenced the whole process but this kind of environment is preferred over closed source alternatives in all processes where public participation is one of the aims—as is the case in modern urban planning [76].

2.2. Usability Testing and Survey

For the usability test we gathered non-representative, purposive sample of 15 people. Our main criterion for selection was practical involvement with spatial planning process which included architects, urban planners, spatial planning students and GIS specialists. We also wanted a diversified sample regarding age and gender of the participants (

Table 2. Participants of the usability tests.

Participant	Gender	Age	Profession
P1	Male	18–25	researcher/academic teacher
P2	Female	31–50	architect/urban planner
P3	Male	31–50	architect/urban planner
P4	Male	18–25	researcher/academic teacher
P5	Female	18–25	urban planning student
P6	Female	18–25	urban planning student
P7	Male	31–50	architect/urban planner
P8	Male	26–30	researcher/academic teacher
P9	Male	18–25	GIS specialist
P10	Male	18–25	GIS specialist
P11	Male	18–25	GIS specialist
P12	Female	26–30	researcher/academic teacher
P13	Female	26–30	GIS specialist
P14	Female	31–50	architect/urban planner
P15	Male	31–50	architect/urban planner

).

While the sample size is small, this is not uncommon in usability studies and can help uncover most of the problems with a careful choice of participants [77]. In our case it was more important to have people with first-hand experience in spatial planning. We prepared six tasks, one for each environment in the application. They were designed to be short and simple and reflected tasks that could be encountered during real-life deployment of a WebXR spatial planning software (

Table 3. Tasks presented to usability tests participants.

Environment	Task Description
AR1—Display	Try to place the model on a surface. After loading the model, try to locate the main entrance to the WNGIG building.
AR2—Measuring	Measure the distance between the points indicated on the map (from 1 to 2; from 2 to 3; from 1 to 4). Try to verify its correctness.
VR1—Display	When you are in a virtual reality environment: 1. Try to locate the models depicting the old town, then locate the town hall and check if you can see it from all sides. 2. Try to locate the models showing Stary Browar and Półwiejska Street, then locate Stary Browar and check if you can see it from all sides.
VR2—Zoning map	When you find yourself in a virtual reality environment, approach the legend and check how the areas called fortifications are marked on the map, then try to locate the largest of them on the map.
VR3—Area renovation proposition	When you are in a virtual reality environment, try to locate the areas marked in yellow and blue, and then reach them by moving along the park’s paths.
VR4—Virtual walk	Once you are in a virtual reality environment, try walking and seeing the buildings located in the central part of the Poznan market square.

). Each test started with signing an informed consent form and the pre-test survey followed by the introduction to the VR headset (Oculus Quest 2) through the application “First Steps” and VR browser navigation. The participants were allowed to take a sitting or standing position depending on their perceived comfort. To make sure there were no adverse physical affects, there was a 15-min break from the VR environment and during that time we asked users to resolve tasks 1 and 2 that involved the AR application. Then participants were once more asked to put on the VR headset and resolve tasks 3–6. Finally, the participants filled out a post-test survey and we had a short interview about their behavior. During the whole duration of the test, the participants were encouraged to talk loudly and describe their experiences. We also observed their actions on a separate monitor using video casting function of the VR headset. The final survey consisted of several open questions aimed at gauging overall impression of the technology and the usability of our particular applications. The participants were also asked to rate both general software experience and separately its potential application in the field of urban planning using simple 1 (worst)—5 (best) scale. For the quantitative measure we used the widely applied Brook System Usability Scale [78] adopted for Polish language.

3. Results

The post-test survey consisted of two parts in which the participants evaluated their experience—numerical evaluation using simple ratings and SUS scale and open questions about most important usability issues and practical potential of the software. Results are presented separately below together with description of our own observations from the developers’ perspective.

3.1. Numerical Evaluation

The first part of the evaluation was the series of questions that allowed us to construct a SUS score [79] for the applications. The results are presented in

Table 4. System usability scale scores and general usability ratings (1-worst, 5-best).

		General Rating (Perceived Usefulness in Urban Planning)
Participant	SUS Score	AR1	AR2	VR1	VR2	VR3	VR4
P1	85	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)
P2	90	4 (5)	4 (5)	5 (5)	4 (5)	5 (4)	5 (5)
P3	85	4 (5)	4 (4)	5 (5)	4 (5)	5 (5)	5 (5)
P4	73	5 (5)	3 (5)	5 (5)	3 (4)	5 (4)	5 (3)
P5	83	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)	5 (5)
P6	67	3 (4)	4 (4)	4 (5)	4 (5)	4 (4)	4 (5)
P7	70	4 (4)	5 (4)	5 (4)	2 (2)	5 (5)	4 (4)
P8	80	4 (4)	3 (2)	5 (4)	4 (4)	4 (5)	4 (4)
P9	65	5 (5)	4 (3)	5 (5)	4 (4)	4 (5)	5 (4)
P10	70	3 (2)	2 (1)	4 (3)	2 (1)	2 (4)	4 (3)
P11	90	3 (4)	2 (2)	5 (4)	2 (2)	4 (5)	4 (5)
P12	70	4 (4)	5 (4)	5 (5)	3 (5)	5 (5)	4 (4)
P13	67	3 (5)	3 (3)	5 (5)	3 (5)	5 (5)	4 (4)
P14	73	5 (5)	4 (5)	5 (5)	4 (5)	3 (5)	5 (5)
P15	83	3 (4)	3 (3)	4 (5)	5 (5)	4 (5)	5 (3)

.

The higher the SUS score, the better it is in terms of usability, while the scale is not linear and cannot be interpreted as such. It is generally assumed that the score above 68 points means that the tested application has the appropriate usability level. Only in 3 cases our application received SUS score lower than this threshold and the lowest score was 65. It seems that our participants perceived using our software as relatively easy. This was also reflected in our observations and the users’ comments during the tests. Most of the users had no problems with movement controls and navigating through virtual environments. Main issues that came up were related to the process of starting the application. It was difficult to some users to identify the need for entering immersive environment—by clicking “Enter VR” button on a web page displayed in Firefox Reality window, since for them they were already “in VR”. This caused some confusion. Another issue was the interface of the browser itself. In some cases, the navigation panel obstructed the browser window and it required more spatial awareness of the user to be able to reach beyond it to interact with the web page. While not frequent, this can be treated as a serious limitation since the popularity of a web browser interface is one of the main strengths of WebXR.

In the general, the users rated VR1 most favorably, much higher than other environments (Table 4). Our observations and conversations revealed this to be the product of implementing clear spatial metaphors. In VR1 we used VR world that resembled a showcasing of a set of physical scale models—the real-life situation with which almost all our participants were familiar either from a perspective of an organizer and creator or as an audience and which is a deeply embedded practice in urban planning and development. This environment can also be seen as being the least immersive VR from all the other VR cases. That means that it is also least straining and most accessible. This is mainly why it was rated as the most favorable. On the other hand, the worst ratings were given to VR2 and AR2. The VR2 environment seemingly caused confusion and bad experience in two aspects. Firstly, it introduced users with the necessity of walking on a scale model itself, which is unfamiliar experience and one that to be appreciated requires some practice and skill (e.g., crouching). Most importantly however, the choice of semi-transparent textures made it difficult to recognize familiar places and majority of the users complained that they were unable to locate themselves at first glance. Also, when applied to semi-transparent structures the colors were much harder to compare with the legend. However, the legend itself was regarded favorably as easy to read, being separated clearly from the content and described as accessible with just a movement of a head. In case of AR2 its low rating was the result of an unfamiliar web experience with mobile browser and frequent errors when it was not started properly—which proved to be a challenge for most of the users. Mobile phone users are much more accustomed to interactive and more advanced tasks being relegated to specialized apps rather than being carried out inside the browser. This means that when they encountered an error they for example restarted the app and not refreshed the page. Confusion also resulted from the necessity of using a special physical object (the map). Participants expected experience rather to be self-contained as in the AR1 case.

It is also interesting to take a look at the differences between general rating and perceived usefulness in spatial planning (

Table 5. Comparison between mean general usability rating and mean perceived usefulness in urban planning. Standard deviation values in brackets.

	AR1	AR2	VR1	VR2	VR3	VR4
Mean general rating (SD)	4.00 (0.85)	3.73 (1.03)	4.80 (0.41)	3.60 (1.05)	4.33 (0.90)	4.53 (0.51)
Mean perceived usefulness in urban planning (SD)	4.40 (0.82)	3.67 (1.29)	4.67 (0.62)	4.13 (1.36)	4.73 (0.46)	4.27 (0.80)

). In cases of AR1, VR2 and VR3, despite the fact that these environments received lower general rating, they were perceived as being very useful in real life scenarios. This is especially visible with VR2 where despite the fact that the actual experience was stressful and cognitively confusing, our participants recognized the potential of this form of cartographic communication. Data that were used in our example are frequently encountered in real-life situations and traditional maps are regarded as not easy to use by the public (the same can be said about web maps [80]). Participants see our VR2 example as a case of badly designed but promising tool. Similar but not as drastic differences are visible in AR1 and VR3. The latter was regarded as being the most useful in urban planning as it portrayed the situation when an abstract 3D model of proposed reconfiguration of urban space is showed to stakeholders. In those circumstances it is often hard for inexperienced users to situate new physical interventions within some familiar spatial context. Urban planners that we asked to participate in our test regarded the task of visualizing 3D models in one’s mind with a proper scale as particularly difficult. Immersive VR world is seen as a way to mitigate this issue as it allows for easy adoption of the “frog perspective”.

3.2. Problems with WebXR Identified by the Users

Our participants were able to point out the most serious problems they encountered while using XR environments either verbally during the test or afterwards in the survey. While there were differences between users, there were also differences between the types of environments. In the AR case, the users struggled with spatial clues and anchoring. They did not know what to expect, where the models ‘should’ be placed or, for example, if they could turn around in a different direction. This was much more prevalent in AR2 which shows that even slightly increased complication in user interface is harming user interface in augmented reality. In case of VR, problems were mainly related to the data and visualizations—especially to the features that were missing. Models were either too distorted (VR4), lacking enough context and content (VR3), distracting and confusing (VR2) or lacking more advanced interaction (VR1). Much more rarely the users had problems with navigation but those were brief moments. It seems that VR is much more accessible, with people being much more familiar with the concept of a virtual world. They want the experience to be fuller and more immersive with more data and interactions.

3.3. Possibilities of Using WebXR in Urban Planning

In the final part of the survey, we asked our participants to try to imagine the potential possibilities of using XR in urban planning. The prevailing opinion (n = 8) was that XR could help visualize either planned or historical buildings and planning concepts. It was mainly positioned as a new way to engage viewers, something to spark the interest of the public, which is often not very keen on participating in the planning process. The second often mentioned possibility (n = 5) was explicitly participation potential and the ability of XR (especially VR) to represent the scale of the models. Those of the participants with architectural background also saw XR as a tool that can be used in their everyday work.

3.4. Observations from the Software Development Process

As the final result of our empirical research, we propose a few observations regarding the viability of WebXR as a general tool for creating XR environments (

Table 6. Pros and cons of implementing WebXR for creating virtual environments in urban planning.

Pros	Cons
Access to developing tools and software within GIS and web development ecosystems	Weak graphical optimization
Easy debugging using tools like Chrome Dev Tools	Current lack of examples and ready to use scripts
Ability to share virtual environments with wide range of devices and users	Necessity of extensive manual optimization and simplification of 3D models.
Extensive catalog of software libraries that can be used in the development process	Physical limitations of the development process—e.g., motion sickness
Lack of the necessity to rely on gaming engines like Unity 3D and Unreal Engine	Anchoring problems with AR solutions

). Our experience was surprisingly positive given the relative newness of the technology. We were able to develop software with free and available tools and since it does not require using gaming engines, the learning curve is moderately steep. Thanks to various plugins like Qgis2threejs [81] it is possible to connect web developing ecosystem with GIS. WebXR emulator extensions for Chrome and Firefox make it possible to develop software without the need of a headset. Most importantly WebXR present the ability to share virtual environments with a wide range of devices and users with almost full interoperability.

On the downside, creating successful virtual environment with WebXR requires additional steps necessary to achieve the appropriate level of graphical optimization. As with all web-based solutions, one must assume that the final result will be viewed using a wide range of displays, including small mobile devices, and there will be limit on processing power. Therefore, 3D models need to be manually modified to reduce the number of vertices and objects. Development of VR environments also require some level of practical experience with the technology. Motion sickness can be a serious problem limiting time spent using VR headset and hindering the possibility to test the software in real-life situations. Also, there are not as many ready-to-use recipes as with other web technologies which require more trial-and-error development. The most serious technical limitation that we encountered is the glitchiness of AR anchoring that is unreliable and vary between devices.

4. Discussion

In light of our results, we think that it may be safely assumed that WebXR can have its place in the urban planning process. Web applications can be developed relatively easily without much investment in hardware and software and the 3D urban data are increasingly available (although the coverage between countries in uneven [64]), as our efforts in building various XR environments have shown. Results of the usability test indicate that even at this basic level of development, useful and usable application can be built with available tools. Our users had mostly positive experiences and they were successful in carrying out the tasks that were given to them with few exceptions. However, it is also clear that at least some of the positive sentiment toward XR in urban planning is the result of the newness of the technology. While some of the users had previous experiences with some type of XR, it was never in the context of urban planning. They often expressed their engagement in terms of surprise and enjoyment. Therefore, it is safe to assume that scores our apps received in numerical evaluation would be much lower should this not be a first time experience.

Problems we came across during the test suggest that even when immersive environment is used, it should tap into the reservoir of familiar spatial and domain-related experiences of the user. This can help with navigation and the overall feel of the purpose of the exercise and task. It was clearly visible in the VR1 Display case which was almost universally accepted as the best among the participants. It was highly related to real-life situation all our participants had encountered before. The AR2 environment was the illustration of the opposite. Users were not sure what to expect and how this could be useful in real life. These examples could serve as a guideline for design of accessible XR environments. We think that the technological difficulties with WebXR are not as important as implementation dilemmas. The main issue with WebXR technology application, and in reality with XR in general, is finding the proper balance between possibilities that are given and capabilities and expectation of the human participants. It is true that XR brings new ways of interaction and potentially of engaging, immersive experiences. It is however equally true that most of these interactions are not really needed. While our participants were optimistic in the way they expressed their views on the future of XR in urban planning, they painted a picture of a very modest set of possible usable cases, limited mainly to the new way of visualizing models and urban plans. From this point of view WebXR seems even better suited for those tasks. Main limitations of implementing WebXR applications are concerned with its lack of graphical optimization and simplicity of the interface related to the wide range of possible displays and devices being used to access the content. But it seems that this would not pose a significant problem in the case of urban planning implementations.

Also, as a general rule, it seems that when designing XR as a part of an already existing process it would be better to focus on one end of the XR spectrum to avoid confusion among the users. While our results can be with some caution used in the design of XR environments, it is clear that to fully appreciate the accessibility potential of WebXR, tests that focus on this aspect shsould be conducted. We tested our applications in a comfortable environment with VR headset and capable smartphone. The next step would be to test usability of those solutions in real-life street scenario and without the headset on a desktop browser.

5. Study Limitations

It is necessary to highlight some limitation of the study as they impact the overall tone of our works. We adopted an approach to usability testing described by Nielsen [82] as the “Guerrilla HCI”—when the number of participant is relatively small and the sample not-representative. It is however used in the field of GIS [77] and it has been shown that a group between five and eight people are enough to detect most of the usability issues [83] and our sample was larger than that. It also need to be noted that the order of the tasks that were presented to the users was fixed and not randomized which could induce the learning effect. However since a strict comparison between environments was not the main goal of the study it was thought that fixed order will help users to share a more in-depth analysis. Another possible limitation is that the results may be dependent on the equipment that we have used. We are assuming here that since Oculus Quest 2 is known for being user friendly comparing to other such devices the results represent the best case scenario with a good balance between cost, fidelity and user interface complication. This study signals both the potential advantages and risks involved in pushing the WebXR and XR agenda in urban planning and in this regard we think that it can be useful. What cannot be incurred from our study are the ever present and important divides and differences between age and gender groups. This we think should be the next logical step in pushing forward the XR research agenda in GIS and urban planning.

6. Conclusions

Our study indicate that WebXR can provide useful solution in urban planning. We are also suggesting that XR environments that resemble common practices and situations encountered in real life of people participating in the planning process provide the most suitable framework for seamless integration. It also important to note that despite it shortcomings and simplicity the technology we have used in the study was perceived very positively by the test users. XR technology in its various iteration has been present within the GIS science for a long time. And yet with both technological advancements and the ever widening range of possible applications the constant interaction between the two present additional challenges and research opportunities. From the perspective of urban planning the most welcomed development would be the more seamless mode of integration between GIS datasets and XR, especially with the use of web technologies, that would allow the latter to become standard mode of visualization of the results. In addition to this additional research is needed to fully understand the capabilities of new generation of XR systems to generate virtual representation of geographical environments in terms of immersion and presence, especially in relation to non-immersive visualisations. Another point that is worth further consideration is the level of detail in 3D models that can be used in web applications. It can be assumed that higher fidelity can result in more immersive and natural looking environments. And yet our participants were not particularly concerned with this aspect of the provided test applications. This however may be just a result of the specific conditions of the test and lack of comparison to high fidelity virtual environments. Therefore, the empirical study would be necessary to gauge the appropriate level of detail in 3D models for urban planning in its many forms. Answering those questions would help with the integration of GIS and XR fields of study with positive consequences for practice and scientific research alike.