Iterative Prototyping of Urban CoBuilder: Tracking Methods and User Interface of an Outdoor Mobile Augmented Reality Tool for Co-Designing

Abstract

This research presents results from a study developing a smartphone app, Urban CoBuilder, in which citizens can collaboratively create designs for urban environments using augmented reality technology and game mechanics. Eight prototypes were developed to refine selected design criteria, including tracking strategies, design elements, user experience and the interface with game mechanics. The prototypes were developed through an iterative design process with assessments and incremental improvements. The tracking was especially challenging and using multiple bitonal markers combined with the smartphone’s gyroscope sensor to average the user position was identified as the most suitable strategy. Still, portability and stability linked to tracking need to be improved. Design elements, here building blocks with urban functions textures, were realistic enough to be recognizable and easy to understand for the users. Future studies will focus on usability tests with larger user groups.

1. Introduction

International and regional policies often emphasize the importance of multi-stakeholder inclusion in urban planning processes [1,2]. There are promising efforts to develop new forms of engagement, participation, and collaboration to shape public services [3] based on digital technologies, such as visualization, gamification, cloud computing, social media, and mobile technologies [4,5,6]. Still, attracting and engaging citizens to participate in urban design and planning is a challenge [7,8]. It is also a challenge to expand the level of participation beyond information and consultation into collaboration and co-creation [9]. To overcome these challenges and to engage the public into a deeper level of participation, research suggests providing a ‘fun’ factor with the creative use of innovative technologies [4,9].

Augmented reality (AR) is one such promising innovative technology. Considering that visualization and representation is critical for better public participation [10,11], the use of AR can enhance the co-creation experience for participants by providing more intuitive visualization and representation [12,13,14,15]. Already in 1999, Höllerer et al. [16] described an experimental testbed for mobile augmented reality (MAR) systems, both for indoor and outdoor collaborative tasks. Since then, the technology for mobile devices has advanced considerably. Smartphones equipped with a global positioning system (GPS), Wi-Fi, and 3rd-party apps are widely available today, increasing the accessibility of the MAR technology. MAR systems have also developed but, as Liberati [17] (p. 17) points out, “from a technological point of view, augmented reality is still in its infancy”. Remaining technical challenges include the lack of accurate tracking methods to localize augmented 3D objects [18], especially in outdoor urban environments [19,20,21,22,23,24], which is critical for use in a public urban context with multiple users [21,25]. In addition, the lack of digital collaborative design tools using AR technologies to engage the public in an urban design context limits the implementation of these technologies in public participation settings [26,27].

This paper presents the results from a study developing a mobile digital tool in which citizens can collaboratively create designs for urban environments, using AR technology and game mechanics. The objective is to test and discuss multifaceted design solutions with a focus on tracking methods for outdoor MAR intended for real-time interactive use in an urban context, on identifying relevant design elements, and on developing a user interface that enables the concept of co-design. Based on iterative prototyping, the process of developing the first version of an outdoor MAR tool, the Urban CoBuilder, is discussed. Various tracking methods are tested to find tracking strategies that are suitable for an urban co-designing tool with consideration to simplicity and availability in addition to the accuracy and stability required for collaborative actions. By doing so, the paper contributes to increased knowledge of outdoor MAR technologies for urban planning.

The study is closely linked to the work presented in [15] but with a focus on design criteria, technical prototype development and the iterative steps as such, whereas [15] focus on the development of a theoretical framework, tool specifications, prototype requirements, and an overarching presentation of results from the prototyping and the assessment process.

State of the Art

Previous research indicates that a broad representation of participants increases fairness in the outcomes of public participatory decision making [28]. Collaborative platforms and possibilities to actively design and create urban environments can improve participation but new innovative technologies are needed [9,15,29,30]. The use of game mechanics and gamification in non-game contexts [31] has shown to boost civic engagement [31,32,33], especially when it comes to engagement of youth in urban planning discourses [34,35,36]. Apart from gamification, the use of mixed-reality technologies, such as virtual reality (VR) and augmented reality (AR), has the potential to create more attractive platforms for citizen participation [27,37,38,39]. Combining mixed-reality technologies with game mechanics [40,41] can create synergies and encourage diverse civic groups to participate in dialogue processes concerning urban matters.

In urban dialogue processes, the potential of using AR has been recognized for its immersivity into the complex urban context on site, including, e.g., climatic perception, acoustic perception, and changing light conditions, experiences that a medium, such as VR, cannot provide [39,42]. AR can also visualize future projects and enhance the quality of representation of what is ‘to be built’ to better inform the affected citizens [14,21,42]. In combination with the increasing accessibility of hardware, such as smartphones with embedded GIS functions, gyro sensors, accelerometers, and fast Internet connectivity, this potential brings forth the use of mobile AR technologies [17,27] as an accessible and convenient platform that may stimulate broader participation in citizen dialogues regarding urban development. Additionally, the use of mixed-reality mediums, including VR [6,9] and indoor AR applied to downscaled models [12,13], have shown fruitful results in support of collaborative design involving citizens. Still, there is a lack of outdoor AR tools for collaborative design with multiple users, a deficiency that could be related to the challenges related to outdoor AR tracking.

There are two major outdoor AR tracking strategies: sensor-based and vision-based tracking. Sensor-based outdoor tracking relies on the combination of GPS or Wi-Fi with smart devices’ inertial sensors [22,43], such as gyroscopes, accelerometers, and compasses. In these applications, only a smartphone is usually required, and typical applications are outdoor AR games [44], such as Pokemon Go [45]. Although such sensor-based tracking could work for general public participation, the general lack of accuracy of GPS coupled with an eventual drift of the inertial sensors [21,23,24] is a problem for outdoor AR functions that require stable tracking [25].

Outdoor AR tracking using vision-based tracking can be grouped into two categories: marker-based and marker-less visual tracking [22,43]. While marker-based visual tracking uses an artificial marker, often a printed bitonal marker, as reference for the location and orientation of 3D augmented objects [46], marker-less visual tracking uses elements that are already present in the environment as tracking reference. Extracting images from existing elements, such as the facade of a building or a 3D object, and using those as markers facilitates outdoor AR in urban contexts when site preparation needs to be minimized [22]. However, the unpredictability of certain factors, such as sun casting a shadow on the objects [22,47] or parked vehicles, might render these markers unstable [22]. Furthermore, some of the marker-less tracking systems still require extensive preparation, such as finding available 3D databases of the site and pre-imposing the virtual augmentation offline [48,49]. Potentially, to manually align the position of built objects in the real urban environment onsite with a preloaded 3D database [50] could provide a more stable projection.

Real-time tracking is important for collaborative design with multiple users, but difficult for a strategy that integrates 3D virtual objects offline for projection on site. To accommodate collaborative multiple user AR actions, simultaneous localization and mapping (SLAM) is suggested [25]. However, the presented SLAM system needs dedicated MAR devices for tracking and mapping [25]. Likewise, specific AR hardware sets, including headsets and portable computers [12,51,52] limit the accessibility for the general public in open collaborative settings [53], where the general accessibility of smart devices is preferable [54]. Thus, when developing a collaborative MAR tool it is important to find the balance between accessibility and simplicity of use, on the one hand, and accuracy and stability of the system, on the other.

2. Materials and Methods

2.1. The Concept of Urban CoBuilder: Specifications and Design Criteria

The basic idea of Urban CoBuilder is to allow citizens to collaboratively design or modify urban sites by placing building blocks representing various urban functions, for example housing, offices, or commercial activities. Based on game mechanics, citizens can choose a stakeholder role, such as a developer or a municipal officer, and in a turn-based manner place available building blocks in an urban development area. Each stakeholder has a limited budget and specific planning objectives. After placement of the building blocks, the actions are saved and the next person is allowed to place building blocks. It is also allowed to modify or remove building blocks placed by the previous person. This continues until the stakeholder’s budget or building blocks are used up.

The implementation of Urban CoBuilder is based on a framework presented in [15] (pp. 123–124). This framework comprised four overarching specifications with 20 sub-specifications: (1) Simulation of built structures and incremental development processes through the multi-stakeholder inclusion (five sub-specifications), (2) immersivity for on-site perception (one sub-specification), (3) on-site AR projection (four sub-specifications), and (4) rule-based process simulation through game mechanics derived from existing simulative urban games (10 sub-specifications). The specifications were subsequently translated into four main design criteria in support of the prototype development: tracking strategies, design elements, UX-I (user experience and interface) including game mechanics, and data storage and retrieval (see Figure 1). However, as data storage and retrieval were decided on early in the prototype development, this criterion was not tested in the presented prototypes. Data storage and retrieval are shortly presented in the method chapter 2.4 to provide a background.

2.2. Iterative Prototyping of the Outdoor MAR Tool

This research has applied ‘research through design’ [55] (p. 5) methods to develop the MAR tool through iterative prototyping. Iterative prototyping is an efficient method for product development, communication, and exploration of concepts [56]. It supports identifying potential problems, evaluating usefulness, analysis of cost-efficiency, and strategizing productions [57]. Especially under time constraints, iterative prototyping excels, not only in developing prototypes that outperform non-iterated prototypes, but also boosts self-confidence among the designers after the development process [58]. In this project, the iterative processes focused on developing a ‘functional prototype’ [59] (p. 533) (see Figure 2) that strategically implements essential parts of both the user interface and the functionality of the planned application. ‘Exploratory prototyping’ was adopted [59] (p. 532) to provide and test potential solutions that could meet the tool’s requirements. Additionally, ‘experimental prototyping’ was used to create quick ‘breadboards’ for testing technical issues linked to MAR tracking methods, and ‘evolutionary prototyping’ was carried out to develop ‘pilot systems’ [59] (p. 532–533). Bäumer et al. [59] (p. 533) define a pilot system as a ‘very mature prototype that can be practically applied’, our system was not mature enough to be referred to as a full pilot system. We have therefore labeled it a ‘semi-pilot system’ based on evolutionary prototypes of practically applicable systems.

The number of prototypes is equal to the number of tested tracking methods. The design criteria were developed iteratively from simple implementation to more sophisticated improvements. To ensure that limitations set by time and budget constraint would not prevent the implementation of a maximum number of concepts, a systematic prioritizing was carried out after each iteration regarding which implementation would be improved with more detail. One consequence of this prioritization was that the iterations for game mechanics were rather limited.

In total, eight distinct prototypes (P1–8) were developed and tested (see Figure 2) with the test conditions described in

Table 1. Implementation of design criteria and the iteration in each prototype.

Prototype	Tracking Strategies	Design Elements	UX-I and Game Mechanics
1	The photo marker used the building facade and 3D data of the site.	3D cube stacking tested. Colored cubes represented building blocks.	Laptop and web camera.
2	3D objects used as photo markers with 3D data of the site.	Urban building blocks with defined urban functions. Block size was defined as 3 m × 3 m × 3 m (scale 1:1).	Android smartphone. Toggling between the camera and 3D view enabled. Planes added for placing 3D objects. Interface for removal and addition of blocks added.
3	Use of smartphone’s GPS, compass, and gyroscope.
4	Gyroscope used to manually align the site environment.
5	Public signs as Photo-marker (see photo P5). Markers linked to 3D geodatabase. Occlusion of existing buildings.	Grid plane of 3 m × 3 m for placement of building blocks. New building block as a semi-transparent box.	Economic concept indicating available money for a design turn and costs for each building block. The model can be viewed from a bird’s eye view perspective.
6	Use of standing, printed bi-tonal marker 0.9 m × 0.9 m.	Box with a dashed line to indicate a not yet built object (see photo P6). Urban green space added as a building block.	Addition of icons with diverse urban functions as well as an interface for addition/removal and up/down buttons.
7	Bitonal frame marker 1.8 m × 1.8 m and markers 0.9 m × 0.9 m. Gyroscope and accelerometer for tracking.	Adding of facade textures to distinguish urban functions with randomized green areas (see photo P7).	Bird’s eye view disabled. Additional economic rules were implemented. Stakeholder turn-based role-playing enabled. Simultaneous use by multiple users.
8	Printed bitonal frame marker of 19 cm × 19 cm.	Facade textures from a real housing project. Each block represents a facade element.	Bird’s eye view enabled. Table-top version (see photo P8).

.

Breadboard (P1) through experimental prototyping: This first breadboard tested if 3D cube stacking on-site would be a feasible method for creating a collaborative design on an urban scale. For this test, a simple set-up using a laptop and an external web camera was chosen for a better quality of image processing.

Functional prototypes (P2–6) through explorative prototyping: P2, P3, and P4 shared the same crude design elements (3D building blocks with urban functions) and UX-I but tested three different methods of tracking. P5 tested a public sign as photo markers for tracking linked to a geodatabase providing on-site data but also implemented an improved UX-I for adding and deleting 3D building blocks. A basic economic concept was tested to prepare for gamification. P6 tested an improved UX-I and used bitonal marker-based tracking.

Semi-pilot systems (P7–8) through evolutionary prototyping: P7 had significant improvements based on the previous prototypes relating to tracking, design elements, and UX-I. Three bitonal markers of two different sizes were used, randomly generated textures on green area 3D building blocks were introduced, and new interaction buttons were added. Birds-eye view was disabled in this test in order to make the system more stable and focus on the street view. P7 also tested the potential for turn-based role-playing where stakeholders could interact with each other. P8 shared identical mechanisms for operation of the system as P7 regarding how to collaboratively design buildings and the urban environment, for example selection of and building with the blocks. P8 is a down-scaled, table-top version of P7 but with new textures for building blocks representing facade elements from real projects. A table-top version was chosen in order to facilitate a large-scale test with 20 citizens. The birds-eye view was enabled again because it was assumed that larger freedom switching between views would be easier to manage on a smaller scale. Game mechanics were not in focus in P8.

2.3. Evaluation of Iterations

The prototypes were developed and gradually modified based on the defined design criteria concerning tracking strategies, with a focus on the simplicity, accuracy, and stability required for collaborative actions; on the design elements and their level of detail necessary for understanding their content; and on a user interface that would enable easy navigation and co-design. Feedback was gained from different groups of evaluators, i.e. developers, researchers, students and citizens (see

Table 2. Evaluators of each prototype. P = Prototype; core team = 1–2 urban researchers and two software developers.

Nr P	Evaluators	Evaluation Method
P 1–4	Core team	Speak out loud Team reflections Notes and discussions
5	Core team	Speak out loud Team reflections Notes and discussions
6	Core team, 3 UX-I designers (not earlier involved)	Observations Speak out loud Team reflections
7	Core team, 2 urban researchers (not earlier involved)	Observations Speak out loud Short interviews Team reflections
8	Core team, 5 master architecture students, 20 passing-by citizens (Age: Male/Female) Under 12: 5/2 12–20: 3/7 20–30: 0 30–40: 2 40–50: 0 50–60: 1	Observations and short interviews with passing-by citizens and master students Team reflections

). From the start, only the core team were involved in the evaluations but from Prototype 5 and onwards there was a progression in the number and competence background of evaluators.

Prototypes 1–4 were thus evaluated by the core team including two software developers and two urban researchers. During the tests, a ‘think-aloud method’ was applied [60] (p. 6), meaning that the participants talk about their thoughts as they are performing a set of specified tasks. After the tests, the team reflected together over the different observed issues, and notes were taken. The evaluation of Prototype 5 followed the same procedure, except that the core team now consisted of just one urban researcher and the two software developers. For the evaluation of P6, three additional UX-I designers without prior knowledge of the tool were invited. In the evaluation of the semi-pilot Prototype 7, two additional urban researchers participated, and this evaluation also included short interviews.

The evaluation of semi-pilot Prototype 8 was carried out at an urban square with five master students in architecture engaged in the testing with groups of passing-by citizens. The portability of the marker was achieved through each facilitator bringing a printed bitonal frame marker mounted on a filing paper folder to facilitate a scaled-down use of the tool. In total, 20 citizens had the time and accepted to stay to test the prototype by performing a design task in a neighborhood block. After finishing the task, the students and citizens were asked to reflect on how it was to use the prototype and short interviews were conducted.

2.4. Technical Specifications, Data Storage and Retrieval

Urban CoBuilder uses the 3D game engine Unity as a developing platform and the Vuforia platform for AR. Vuforia provides easy integration with the Unity 3D engine and supports tracker recognition. In Urban CoBuilder, the user can create 3D models and the models are, in turn, uploaded to a central server with an online database. The central server enables synchronization and storage of players’ 3D models and other related data as well as allowing rules and scores to be tracked and distributed to other users. With an online saving of the models created by the users, the inheritance of derived designs can later be visualized, showing how the initial designs evolved through time by different users. This function was implemented to respond to the MAR tool specifications for multiple-stakeholders’ incremental input during a session.

3. Results

The prototype iterations resulted in solutions for tracking, design elements, UX-I and game mechanics. In relation to tracking, the iterations went from using elements in the existing environment as markers, via flat surface markers and bitonal markers, to using multiple bitonal markers with user-location averaging. Regarding the design elements, the level of detail was gradually increased from uniformly colored small blocks to larger volumes with urban functions textures, and finally facade textures from a real housing project was used. When it comes to UX-I and game mechanics, different views were enabled and disabled in the iterations (bird-eye view vs. street view), elaborations with the placement of 3D-object were carried out, and the interface was improved by adding buttons for interaction and icons for urban functions. The prototypes were also iterated from single use to simultaneous, multiple use. Game mechanics were first implemented through design choices related to available funds and then with turn-based stakeholder role-playing.

3.1. Tracking Methods

3.1.1. Photographs of Facades and Existing Built Objects as Markers

The starting point for the first iteration (P1) was the assumption that using photographs of existing facades or other built landmarks as markers would allow the tool to be used in locations where other forms of marker placements are not possible. However, this proved insufficient due to the general instability of recognition of a photographed building as a marker. The sun’s movement resulting in changes of shadow patterns on protruding facade elements, e.g., balconies, window frames, and roof overhangs, created problems for recognition after a certain time had passed. In addition, moving clouds and reflective surfaces, e.g., windows, made these photo-markers unrecognizable. Changing the viewing perspective of the same building also resulted in unstable image recognition.

3.1.2. Smartphone Functions for Motion Detection and Manual Alignment of User Position

In the next iterations (P2–P4), built-in smartphone functions, such as GPS, compass, and gyroscope, were used for motion detection and manual alignment of user position. The tests showed that the tracking ability depended heavily on the capacity of the smartphone’s compass and gyroscope function, making it hard to work in a vertical ‘camera mode’ when the compass and gyroscope functions were not working accurately. The GPS also proved to have too low accuracy in handling the movements of the user for tracking and aligning the 3D scene with the real-world environment. Since the gyroscope drifted, especially when turning around more than 90 degrees, manual re-alignments were constantly needed. However, when the alignment was sufficient, this method worked better than in the prototypes not using the gyroscope. The advantage of this method, apart from constant manual re-adjustment to align the scene, was that there was no need for the setup of AR trackers, as long as a geographic 3D database of the surrounding scene is available.

3.1.3. Flat Surface Markers and Bitonal Markers

The flat surface, in our case a public sign placard used in P5 (see Figure 3), worked with more stability compared to using photographs of facades and existing built objects as markers or using the smartphone’s sensors for motion detection. However, it was still not stable enough to be used for a collaborative design application.

The use of printed bitonal frame markers (P6) generated by Vuforia (see Figure 4) for optimal location tracking simplified the process of preparing the setup for the use of the tool. However, due to the size of the marker (0.9 m × 0.9 m), problems occurred when the distance increased between the marker and the users, and from approximately 20 meters the grid was experienced as unsteady. Difficulties were also experienced when distinguishing horizontal grids further away from the user. This issue is discussed more in detail in Section 3.2.2.

3.1.4. Use of Multiple Bitonal Markers with User-Location Averaging

Improvements were made in P7 regarding the user-location averaging by triangulating the user-location whenever two or more markers were visible simultaneously on the smartphone’s screen. In comparison to using only Vuforia that tracks one marker at a time, this user-location averaging provided more stability by reducing the glitch while switching between the markers, and lessened impact from drifting of the gyroscope.

The large size of the markers provided better visibility on site (see Figure 5) which seemed to contribute to the stability of the tracking. Especially the combined use of gyroscope and accelerometer extended tracking range when the marker was invisible on screen only for a short duration of time. This delivered much better usability of the tool by limiting the frequency of disconnection when turning the smartphone to look upwards or into the farther distance. Some jitters and lag were still observed, but it was stable enough to be tolerated by the users to continue to build with the tool. Still, better tracking strategies and reducing the size of the markers would make the tool more accessible to be set up in different locations.

The table-top version marker with scaled-down projection (P8) had no observable tracking issues apart from that the presence of the white edges provided by an A4 paper around the black marker border was essential for the marker to be recognized.

3.2. Design Elements

3.2.1. Building Blocks

In the beginning (P1), scale-less cubes with different colors were used as building blocks to represent various unspecified urban functions. In the second test (P2), the building cubes were defined by size, representing 3 m × 3 m × 3 m in a 1:1 scale and different colors were associated with different urban functions (see Figure 6).

Thereafter, a semi-transparent box locating the building block was introduced with dashed lines to indicate that it is not a yet built object. The semi-transparent box visually enhanced the perception of the scale of the building block in the scene in relation to the existing built environment. Later during the semi-pilot system tests (P7), detailed facade textures (see Figure 7) were added to distinguish urban functions such as a residential, commercial, and office use, also including a randomized green area building block for variability. These more detailed facade textures helped participants to understand the scale of the built object better than the mono-color building blocks. However, the block dimension 3 m × 3 m × 3 m was seen as being too small as a building block to be able to design a whole neighborhood. Therefore, 6 m × 6 m × 6 m blocks were suggested to represent the size of an average dwelling per person, according to Statistics Sweden [61].

In the final semi-pilot system test (P8), textures were replaced with building facade elements from an ongoing housing project from IKEA, also known as BoKlok (see Figure 8). The blocks now represented facade elements, such as doors, windows, balcony and staircases. The changed facade textures were received intuitively, meaning that it was natural for the participants to start using the blocks of walls, doors, and windows to assemble a building.

3.2.2. Grid

During the functional prototype tests (P2–8), a building plane was introduced as a grid of 3 m × 3 m for localization and placement of building blocks. Initially, the grid for the building plane was perceived as problematic due to the indistinguishable horizontal lines further from the user (see Figure 9). Thus, a larger grid size was suggested to improve the visibility of buildable grids further away from the user’s position. However, instead of modification of the grid size, which would only increase the buildable distance from the user slightly, the users were encouraged to physically move around the 3D model on site, and to approach the space where a building block should be placed.

3.3. UX-I and Game Mechanics

3.3.1. Technical Implementations

In the breadboard phase (P1), the hardware used were a laptop and a web camera. During the functional prototype testing (P2–6), an Android smartphone environment for rapid testing and development was set up due to cost efficiency and ease of use compared with an iOS development environment.

In the functional prototype test, toggling between the view of the site through camera mode and the 3D model of the site mode was enabled in the beginning (see Figure 6), and mechanisms to view the scaled-down model from top-down was removed so that the participants are encouraged to create the urban design only from street view. The building ground planes were implemented for placing 3D objects in the scene. This plane operates by a user moving back and forth and turning the smartphone until the plane flips to the desired axis between xy, xz, and yz for vertical and horizontal planes for placing cubes.

3.3.2. Building Block Placements Mechanisms

An interface for removal and addition of cubes was implemented in the first iteration of the functional prototypes (P2). In the last of these iterations (P6), an urban function icon was added on the top-left corner (see Figure 10) with a long press to unfold available urban function categories that the users can choose from. Additional and removal buttons were introduced, as well as up and down buttons indicated with arrows to either build on top of or below the selected box. User tests resulted in several suggestions for user interface improvements. For example, to aid decision making, information pop-ups could be provided with details of the total built area or available rooms, as well as prices of each block for available plots.

3.3.3. Switching Perspective between Birds-Eye View and Street View

All iterations of the prototypes except for P8 were projected as full-scale models (see Figure 11). In P8, a table-top version with reduced-scale building modules was tested. The viewports were changed as evaluators of P7 wished to view the birds-eye view of the built 3D model while building from the street level. This enabled users to switch between views and even design larger scales of a neighborhood and buildings from a birds-eye view. In the P8 test, the participants did not use this functionality and the models were only viewed from the birds-eye perspective.

3.3.4. Game Mechanics

In P5, game mechanics were incorporated at a crude stage in order to test the concept of gamification. An economic concept was introduced by indicating how much money was available for a design turn, and how much each building block would cost depending on the block’s urban function. When an urban block was placed, the total available fund was reduced with the price of the used block. In P7, a turn-based multi-player role-playing mechanisms with specified goals for different stakeholder roles was incorporated. The evaluators of P7 could interact physically on-site, for negotiations and discussions to better achieve their goals, e.g., increase green areas or design a tall office building. The concept of a limited economy and expenditure based on the price of the building block was found easy to play with. These game mechanics, including turn-based role playing, were not tested in P8.

4. Discussion

We have presented the iterative prototype development and testing of an outdoor MAR tool, Urban CoBuilder, developed as a collaborative urban design smartphone app with basic notions of gamification. The implemented and assessed design criteria were tracking methods, design elements, and UX-I and game mechanics.

4.1. Tracking Methods and other Design Criteria in an Urban Outdoor Context

Initially, the iterative prototyping mainly focused on identifying and developing accurate tracking methods, as those are very important for an outdoor MAR tool where users collaboratively create an urban space [15,21,23,25,26,27,39,40,48]. We found that inaccurate or unstable tracking and localization make the whole experience of co-building confusing and laborious, eventually leading to users giving up on using the tool further. Various methods of tracking were tested, not only using different tracking marker types, including photo markers taken from the building facade, a flat surface placard from the site, and bitonal frame markers, but also quick iterations were made using GPS and manual alignment of 3D environment and the site.

Research on outdoor AR has pointed out various methods for tracking to facilitate projecting a 3D model on-site, using GPS [14,21,22,43], marker-less vision-based AR with pre-constructed 3D urban structures [40,48,49], and building contours as tracker [23,39]. However, we found that collaborative design of an urban site by moving 3D building blocks on site needs more stable and simpler approaches for tracking. Perceived difficulties of tracking method using manual alignment of 3D environment with the site environment (P4) and a UX-I using manual flipping of the building plane between the xy, xz and yz axes for placing building blocks (P4) highlighted the importance of simplicity, even though the accuracy of tracking might be compensated through such measures. Manual alignment by pointing, dragging and rotating the virtual environment on a smartphone screen to the real environment would correct the effect of gyroscope drifting. However, even though the user might need to point the smartphone screen at a visual marker repeatedly to reorient the position when drift happens, this is still preferred before having to align using pointing, dragging and rotating on the screen.

The Urban CoBuilder prototype with printed bitonal marker showed an improved tracking accuracy that was tolerable and sufficiently sustainable when viewing the built 3D structure from different angles and when building on the structure by placing a block on a grid space. The tracking method developed in the semi-pilot system P7 incorporated multiple marker tracking combined with the smartphone’s gyro sensors and accelerometer, which improved the continuity by averaging the user location when more than two markers were visible on the screen. The table-top version of the tool confirmed stability and portability of the printed bitonal markers. Even though this smaller scale modelling lacks immersivity, the stability of tracking contributed significantly for ease of use of the tool among non-expert citizens, emphasizing again the importance of accurate and stable tracking. However, more improvements are still needed for even better stability and portability of the markers when working in scale 1:1.

While the tracking method development was given priority in developing the MAR tool, the user interface (UX-I), the urban design elements, and the game mechanics were also developed and tested from rough outlines to more detailed components based on the feedback from each prototype testing. Suggestions were made to improve the UX-I, so that manipulation of the variables of the game mechanics, such as rules, prices of building blocks, budget, and building block designs could be changed during the gameplay. In this way, the dynamics and complexities of real urban contexts could be better reflected, which would also motivate further play by the participants. By developing the prototypes, we could learn that a certain level of detail in the 3D building blocks improves the perception of the scale of the virtual built objects on-site. The basic structure of game mechanics of turn-based role-playing could be carried out by using a central server. However, more complexity needs to be incorporated to make the game more interesting and to provide better motivation to play.

4.2. Reflections on the Research Methods

Pais [62] (p. 490) argues that design work is ‘a thinking reflexive action that presupposes an activity in which data, practice and knowledge are aggregated, used as a basis for new assumptions, actions, and [where] new findings can be generated, tested and consolidated’. This research is based on such reflective action based on iterative prototyping, using a smartphone as a platform for development of an outdoor MAR tool for urban citizen co-design. The iterative prototyping proved efficient for quick testing with uncertainty about boundary conditions on how much the tool would be able to be developed within the limitation of available time and budget. Testing out the design criteria for tracking methods, design elements (building block texture, scales), and UX-I and game mechanics allowed the team to develop criteria to a ‘good-enough’ level (as defined by us), adequately developed to allow testing, and within the budget and time limits. The development of quick explorative prototypes helped to overcome initial misunderstandings between the researchers and the developers regarding the expectations of which urban research concept should be prioritized and what the technology could deliver.

4.3. Urban CoBuilder as a Collaborative Urban Design Tool

In the research, we integrated and tested basic concepts of gamification, tracking methods, and UX-I with a limited number of evaluators. However, for Urban CoBuilder to be a fully developed collaborative urban design tool, it requires more user and usability tests with a larger number of participants representing a broader spectrum of demographics. Earlier research has shown that citizen inclusion in urban planning might run into problems of public participation, where the representation of participant demographics usually is limited [63,64]. Thus, it is important to design evaluation sessions with ample time provided for learning the mechanics of the tool to encourage the participation of broader set of citizens.

The evaluation of P8 with citizens opened up for a group of users who were not at all familiar with urban planning and with varying experiences of augmented reality. When approaching the random local residents passing by to test the tool and being interviewed afterwards, some discrepancies of willingness to participate were observed. The perceived ease of use differed between the younger generations (younger than 12 years old) perceiving the use of the tool as being easy, while youth (from 15 years up) perceived it as not being intuitive enough to just start using the tool without instructions. Some participants over 50 years old expressed being overwhelmed by the complicatedness of the use of the tool even after instructions and demonstrations.

5. Conclusions

In this research, the outdoor MAR tool Urban Co-builder has been developed based on iterative prototyping. The focus has been on tracking methods for outdoor MAR in an urban context, identification of relevant design elements, and development of a user interface which enables the concept of co-design. The iterative prototyping with incremental improvement selected design criteria was an efficient method to advance multifaceted design solutions of the tool. As the most suitable tracking method has been identified as the use of multiple printed bitonal frame markers in conjunction with the smartphone’s gyroscope function to average the user position when multiple markers are visible, even though portability and stability still need to be improved.

From a perspective of design elements, it has been shown that the design of the building blocks with urban functions textures have a realistic enough appearance to be recognizable and easy to understand for the users. The concept of gamification has only been tested in a rudimentary way through an economic concept and through stakeholder role-playing and needs to be further developed. In relation to the UX-I, to benefit fully from the immersive qualities of AR visualization technology, it is crucial to find ways to encourage the users to move around the projected 3D objects for a better experience of the designed environment. Moreover, future studies should focus on further development and testing of the tool with larger user groups, especially citizens, as well as the on implementation of the tool in real projects. An extensive evaluation of the final version of Urban CoBuilder will be carried out in order to evaluate how the tool supports collaboration and how far the aspect of gamification influences the co-design process.