Digital Preservation and Reconstruction of Old Cultural Elements in Augmented Reality (AR)

Abstract

This research aimed to preserve traditional elements and urban fabric with enabled interaction in augmented reality (AR). Cultural elements and fabrics are mutually influential in Lukang, Taiwan. Evolved routes for tourism and religious activities have created characteristic elements and activity-based fabrics and facilities. The sustainable promotion of digital cultural assets started from photogrammetry modeling of alley space and shops. The application of AR enabled situated learning of 68 objects, including decorated façades, jar walls, the Lukang Gate, beggar seats, and other creative cultural elements. The heritages were promoted under a new interactive measure of feasibility that facilitated cultural sustainability in a remote site. A mobile interface with a convenient smartphone configured certain settings that were sufficiently flexible and easy to apply. The study presented an effective and efficient remote and situated learning process that correlated the development or setting of both locations. Correlation was achieved with a high fidelity of appearance and utilizing a flexible transformation interface. An approach, which recreated the background and formerly reconstructed objects during AR simulation, was used to verify the outcome of the situated study with conflicting qualitative and quantitative findings.

1. Introduction

One of the typical efforts conducted in cultural sustainability is to preserve old city elements in a digital form for display, inspection, and/or interaction. Indeed, the cultural entities need to be interacted with on the Internet, so that a wider group of people can share knowledge and appreciate craftsmanship under a realistic visualization experience.

Lukang, which is located in central Taiwan, is a small town replete with rich historical memories and heritages [1,2]. Its distinguishing historical characteristics make it one of the most important sightseeing locations in Taiwan, with well-preserved landscape, humanity, and building styles (Figure 1). The famous Old Streets mainly comprise Zhongshan Road, Zhongzheng Road, and peripheral districts. A large number of cultural heritages exist around these areas. In order to promote local tourism, a Heritage Preservation District was established by the government, and it constitutes a unique commercial pattern of cultural industry. The district features a linear old alley located in the middle of it. Historical blocks also create a compelling texture of space infills, including unique food booths, handcraft studios, youth hostels, and religious spaces.

The fabrics and circulation systems are interconnected for religious tourism and general sightseeing. The former formulates a spatial structure using temples as nodes and pilgrimage as paths. The latter has heritages connected by streets. Both structures share the same fabrics and circulation systems under diversified visiting experiences. This shared structure presents a sustainable relationship between the heritages and urban fabrics in a unique pattern of streets, temples, heritages, shops, and event activities.

Urban morphology analyzes how the urban form is a created for a city. It can be conducted by quantitative and comparative analyses [4], through a typomorphological approach [5], or from an urban design point of view [6]. Lukang was developed from a township located next to a harbor to the region around Zhongshan Road, where the model in Figure 1 is illustrated. The circulation system was created in a radical pattern. The streets parallel to the old river front are full of historic buildings. Images to the right of Figure 1 illustrate how the Old Street and Zhongshan Road appear today. A similar developing process was reported from Bengkulu City, Indonesia, where the center of the city was transferred from the beach of the bay area to more inland region [7]. Moreover, Lukang was developed from the river and harbor, which was either improved or did not exist (Figure 1, top).

2. Related Studies

Digital preservation is closely connected to sustainability, tourism, cultural heritages, augmented reality (AR), virtual reality (VR), situated learning, UAV, and photogrammetry. Under the assistance of UAV and photogrammetry technology, retrieved digital data can be employed to promote tourism, preserve cultural identity, fulfill cultural dimensions of sustainability, and enable architectural situated learning through AR or VR simulation.

Culture and tourism are two powerful drivers of growth [8]. Tourism in Lukang is closely associated with the promotion of culture, heritage, local identity, and sustainability. Culture contributes to the sustainable development of any society [9]. The combination of cultural heritages and activities are critical to accurately understand history [10]. Indeed, culture and heritage are the foundation of local identity, which comprises a core element of tourism.

Heritages, which served various functions in the past, represent ancestors’ creativity in handling forms, styles, materials, and structures. A heritage-related tour can promote cultural awareness through visiting physical entities. However, the travel experience has recently been limited by the current COVID-19 pandemic. Fortunately, new technology has been applied to create virtual experiences, such as AR, for situated learning in different contexts using predocumented models. AR has been successfully utilized in tourism [11,12] and cultural tourism education [13]. To promote understanding of local identity, AR has also been applied to enhance the tourist experience [14], as a marketing tool [15], and as a model to promote tourist experience [16]. The four-stage conceptual model established in AR heritage tourism can also be implemented in other domains [12].

As a tool, the simulation of AR or VR also contributes to the communication and engagement of 3D scenes or entities with enhanced understanding or improvement through interaction. Since tourism information systems have been established on the conceptual foundation of experience [17], multimedia contents, personal stories, and/or virtual communities should be applied. UAV and AR were utilized to monitor deformation of historical sites [18], as well as the structure of architectural heritage [19], archaeological elements [20], and castles [21]. Cultural heritage data were represented, shared, visualized, and monitored in AR using photogrammetry modeling [22,23] or involving Historic Building Information Modeling (HBIM) [24,25]. For specific types of subjects and locations, AR was employed in prehistoric rock art paintings [26], underwater archaeological sites [27], prehistoric sites [28], and cultural heritage sites [29].

In architecture, AR has also been implemented in education of steel details [30], sustainable design with better overall performance [31], and construction to augment engagement and interaction with learning environments [32]. Different urban forms contribute to sustainability in dissimilar ways [33]. The interaction between urban fabric and cultural elements should be documented and reconstructed for preservation purposes. Previous AR or VR studies can be applied to digital preservation of old cultural elements as an extension of architecture education for the sustainable development of local culture. This effort will raise awareness of local residents and tourists with a comparison to similar cultural developments in different locations. Indeed, this correlation offers a broader perspective of preservation.

Markerless AR and smartphones also possess great application potential. Marker-based AR [34,35] and markerless AR [36,37] are frequently used in AR. The former requires users to maintain a position relative to the markers, and fatigue frequently occurs after hours of operation [38]. Moreover, due to the complexity of architectural construction sites, it can be challenging to place AR markers properly without system recognition issues [39].

HoloLens-based mixed-reality has been implemented in remote maintenance assistance [40], medical surgical navigation systems [41], enhanced learning environments in construction [42], maintenance and training services [43], cross-platform collaborative support of AR applications [44], and 3D collaboration [45]. Tablet computers and wearable devices have been reported to constitute superior AR devices to smartphones with great potential in architectural engineering as hand-held AR devices [46]. Compared to other devices, smartphone-based AR can be more convenient for immediate responses to various situations and events [47]. A HoloLens provides a scanned 3D mesh description of an environment, and the relative location of an inserted model under a smartphone environment is correct for feasible manipulation. A smartphone is a cheaper, more popular, and more convenient device that can also receive remote instructions through certain applications, such as Skype. In the case that an instructor wants to contact a user or vice versa, two-way communication is achievable. Moreover, third-party apps provide streaming and cloud access functions to support work in remote sites.

3. Research Goal

This research aimed to preserve traditional elements and urban fabric under a situated scenario of interaction in AR. Cultural elements and urban fabrics are mutually influential in Lukang, Taiwan. Both subjects, which have evolved together over 300 years, are enhanced upon cultural events. The elements represent local identity, and the fabric creates spaces to accommodate identity. The evolvement constitutes a local culturally sustainable effort, which should be documented and promoted with user-friendly interfaces, such as AR or VR. The element and associated interaction should be conducted and re-examined in remote sites as correlation, verification, or promotion of local identity. The promotion of heritages in a remote site should present a new measure of feasibility that enables cultural sustainability in an effective and efficient situated learning process.

This study started from reviewing recommended sightseeing routes and their overlapped regions to correlate elements and fabric. The elements, which inherit traditional roles from the past, represent important culture identity. The supporting fabric provides an environment that is highly valuable for situated learning interactions. For the re-examination of original designs and settings, AR or VR interaction should be implemented with situated simulation opportunities. The interaction should also be applicable to sightseeing locations in different cities with similar or varied backgrounds in terms of architecture history.

4. Research Methodology

This study began from a review of local development and 3D documentation of urban fabric to incorporate different types of routes for tourism and pilgrimage. A 3D photogrammetry application was applied to create 3D models of objects and fabrics using images taken on the ground level and UAV from the sky. The preservation was made in an entity’s as-built form for the reference of structural and visual details. The as-built setting of the surrounding environment was also reconstructed to support estimation or an inspection that vector drawings, survey images, and field drawings are unable to achieve. Digital objects can be interacted with on the same, or different, city for situated study with other historical heritages for a more profound experience of culture evolvement. The whole documented length was approximately 2500 m.

The correspondent changes made between different periods of development were explored qualitatively or quantitatively through plans, elevations, or sections made to spaces, such as streets, village, installations, or architecture elements. The development of urban fabric was achieved by comparing cadastral maps, satellite images, historical maps, and field surveys. The maps and images assisted to define how the township boundary and circulation system evolved. For details not sufficiently defined in the maps or drawings, 3D models were utilized to define the volume and relative location of buildings, streets, and temporary fabrics.

Interactions made to 3D data should be explored to a level that is feasible for pedagogy or tourism application. To achieve this, AR apps should be applied to integrate smartphones and a 3D photogrammetry model to facilitate the representation of urban fabric in a dependent and coexisting relationship with cultural elements. Its importance in historical development and the combinative nature of local characteristics should be explored using a markerless AR system for an intuitive interaction experience. Also, 3D contents must be applicable to diversified urban contexts in different locations.

To preserve cultural elements, an augmented environment can be utilized to represent the spatial structure and elements for real-time interaction. Zhongshan Road in Lukang was selected, modeled, and applied, associated with peripheral cultural elements in an AR or VR system, for inspection and situated study with different contexts as an extension to tourism and pedagogical experience. The interaction is crucial in understanding the development of local history upon the original setting of the elements.

Two markerless smartphone AR app systems, Augment^® and Sketchfab^®, were applied for the interaction between an object and 3D heritage background. The expected outcomes outline new metarelationships between elements and contexts, with arrangements that enlighten a new tourism-like experience between people, installations, temporary fabrics, and permanent fabrics.

5. Evolved Routes for Tourism and Religious Activities

Tourism routes are usually categorized by specific subjects or traditional religious customs as a way to link sightseeing locations (Figure 2). The management of connections makes each characteristic route representative of a long-term development of local identities, architecture vocabularies, foods, folk stories, or religions. Part of the route is even indexed by pavement made of consistent color and material. Some of the characteristic routes were arranged during a specific period of time with the support of numerous interesting installations, which reflect local identity and should be documented as part of a holistic cultural image of the township.

The existing circulation system represents a long-term development of permanent fabric infilled by many complicated activities. The building vocabularies, chain business types, and associated second skins make streets into flexible and characteristic structures to accommodate different events. Based on the traffic flow of traditional festivals and folk activities, religious activities are closely associated with the three pilgrimages of Dragon King, Village God, and Matsu. As one of the main streets for tourism, Zhongshan Road is also a very important pilgrimage route. After inspecting the spatial structure and associated business types, it is remarkable how all of the elements were integrated into a structure of sustainability. For example, although the Dragon King Festival was initially canceled in the early pandemic period of 2020, the opening ceremony of praying was held with authentic religious respects in May, followed by dragon boat racing at a smaller scale. During the weekend of the festival, because concern about the pandemic had at least partially abated, many tourists continued to come.

6. Activity-Based Temporary Fabric and Facilities

The facilities (Figure 3) are temporary settings of fabrics for occasional needs.

• Transportation system: Lukang is not connected by a train station or the Taiwan High Speed Rail (THSR), nor is it directly connected to main bus routes in Taiwan. Most of the tourists arrive by automobiles or motorcycles. Parking spaces are allocated near main streets or popular attractions within a short walking distance.
• Traffic control: Traffic control is usually provided, with restricted vehicle access during Chinese New Year or national holidays. Although Zhongshan Road is one of the major connection streets to Lukang, an attractive pedestrian zone is normally defined to accommodate a large number of visitors and activities.
• Traffic signage: In order to promote local development, jointed government resources contributed to a signage system, sidewalk pavement, and sightseeing maps at major street entrances and junctions in 2016. Historical heritages can be connected by following the pavement system or the direction of maps or road signs. The lack of visual penetration was significantly improved within the current circulation system.

7. Characteristic Elements, Facilities, and Fabrics

The 2015 Township Remodeling Project around the Artist Village specified the style of street elements, such as lanterns, house number plates, and street lamps. The elements must be designed with integration of local cultural characteristics. A number of specification-related elements were documented regarding public facilities, signage, decorations, characteristic constructions, temples, and historical buildings (Figure 4). The elements are commonly designed with cultural icons of enhanced imagery identity of Lukang.

• Public facilities: an icon of deer was applied on pavement, road blocks, and manhole covers.
• Signage: two kinds of house number plates were designed in the Old Street and the Artist Village, same was the design of the tour guide.
• Decorations and characteristic constructions: Two kinds of lanterns were designed. The lantern and spring couplet represent a typical entrance image deployed on shop fronts. In addition to the physical layout, the arrangement and the words of spring couplets for Chinese New Year, located on the top and both sides of the main entrance, present the business identity of the owner. A similar connection between the architectural components and traditional custom was also established by where the couplets were posted. Graffiti was created by residential artists in the Artist Village, after it was remodeled from a former Japanese police dormitory. The Jar Wall is also a unique type of wall constructed in the past.
• Temples: Lukang temples have attracted tourists worldwide. The Longshan Temple is famous for its beggar seats and octagon wood wells, which are unique in Taiwan. The Dragon King is the main god worshiped in pilgrimage.
• Historical buildings: A number of Japanese and Quin architectures were well-preserved and 3D documented. Nevertheless, a few street houses were either abandoned or demolished with only archways remaining.
• Revitalized old house: Old buildings were remodeled and reprogrammed for new purposes. For example, the Lukang Tourist Center, which was a Martial Temple, was renewed with well-decorated lanterns to become a very popular ‘must-visit’ spot. The Artist Village and residential artists have also attracted many tourists and local visitors during Lunar New Year and the Dragon Boat Festival.

8. Sustainable Promotion of Digital Cultural Assets

The application of 3D models in AR represents an exploration of cultural sustainability and pedagogy of architecture history.

8.1. Photogrammetry Modeling

The temporary fabrics, which were created during festivals, were made of installations, supporting facilities, crowds, or pilgrimages. In total, 78 3D models were created using photogrammetry to represent the daily and festival scenes of Zhongshan Road and the Artist Village (Figure 5). The models contribute to the temporary and permanent urban fabrics of Lukang. There were a total of 12 street models, 23 building models, and 43 single object models. Old houses were reconstructed as a digital preservation of historical heritages. Most of the models were created with the photogrammetry applications, Photoscan^® and Autodesk Recap^®, using a series of images taken by a digital camera or unmanned aerial vehicle (UAV) with sufficient details. Reconstruction of as-built objects and scenes was accomplished by intensive cloud computation. Each model can hide visual details to reveal structural details, such as the icons of Lukang images (Figure 6, top).

The size of the physical model varied from an entire street to a house number plate. The most significant modeling cases involved the maximum/minimum number of images and the period of time for cloud computing. The model of Zhongshan Road in the evening took 12 days, 12 h, and 43 min to complete (Figure 6, bottom), using only 219 UAV images at night. In contrast, it took only 14 min to create a tour guide. The pilgrimage in the Tianhou Temple was created by 1228 images, in contrast to an old house in Wenkai Road, which was created by only seven images.

The documented setting was integrated with models of current days to display how the new settings had reconfigured the proportion of a street and defined a festival entrance at night. Photogrammetry modeling not only presented photorealistic textures, but were also used to retrieve profiles. The models contributed to the creation and analysis of elevations, sections, and skylines at both locations. Drawings were utilized to compare the differences before and after festivals. Both the models and drawings assisted to configure the evolved process.

The 3D data presented a sustainable application of urban data for analysis, interaction, and promotion in AR. The quality of data provided the user with a realistic situated adaption of cultural elements from different urban contexts for comparison or learning in another city. Applying the model to different contexts prompted us to reconceptualize its original purpose and background of design. This ‘reversed thinking’ process was quite enlightening for implementing the model outside of the original setting. Moreover, the resemblance in appearance makes the situated application a perfect stimulus of cultural induction in architectural history with the assistance of reconstructed configuration.

8.2. AR Application

AR was made by the application of a smartphone-based app, Augment^®. The app provides a ready-made AR environment of interaction and communication, whereby 3D models are prepared and retrieved through the Augment^® webpage and smartphone app. All models were uploaded to the Augment^® cloud database and assigned Quick Response codes (QR codes). A user needs to scan the QR code to download the 3D model to the scene (Figure 7, left). A number of 3D models can be downloaded and worked with simultaneously under a personal account.

8.3. VR Application

VR was made by the application of a smartphone-based app, Sketchfab^® (Figure 7, right). Sketchfab^® constitutes another ready-made communication tool to interact with 3D models. Its VR interface accepts the mesh model in OBJ format. It is worth noting that point cloud data are available for VR only. The 3D models were uploaded to Sketchfab^® through the Internet webpage interface. A user can browse a 3D model using VR goggles or through a general 3D device, such as a smartphone.

9. Remote Instancing and Studies

Situated study is a kind of mobile learning in which a user travels to a site to become immersed in the field reality for the best judgement of comparison. Not only does the original model have to be linked to the subject with which it is to be compared, but a set of adjustment tools must be provided to enhance learning by manipulation. The working process is particularly important in architectural field visits in which a student has to correlate or differentiate elements by shape or composition in order to discern where the region of consistence or difference is located. Situated learning draws a user’s attention, since a real site is made of many supporting fabrics to expand one’s perspective of the problem space.

Models created in this study were feasible for situated comparison. Not only can a model be viewed independently, but it can also be carried to a new site of coherence or diversity for evolvement elaboration. The 3D models were constructed and promoted in terms of AR or VR to fulfill the preservation and pedagogical needs of cultural heritage in Lukang. A markerless AR system was selected for tourism and pedagogy in correlating 3D contents in different urban contexts. A series of exemplifications were made to historical buildings in Taipei to verify its feasibility in correlating the architecture and cultural elements (Figure 8). The comparison moved to the Red House in West Gate, Bopiliou, and Dihua Street to elucidate how the old cultural components of Lukang behave in a different context. The reinstallation involved learning and judgement through AR interaction. Each simulation took one person and two hours to complete using Augment^® on an iPhone XR^®. In total, four elements were situationally applied in six sites.

9.1. Decorated Facade and Jar Wall

The facade and Jar Wall of Lukang were applied to the Red House, Dihua Street, and Bopiliou (Figure 9 and Figure 10), where buildings were constructed almost at the same period of time during the Japanese occupation in Taipei. Similarly, the building façade on Zhongshan Road was also renovated at approximately the same time in a modern style. The way phablet was designed at both locations and represented a significantly different application of building material and vocabulary. For example, the Jar Wall in Lukang is a creative design as a symbolic representation of fortune. The facade was designed in a simple and concise form of moldings and decorated by temple-inspired elements as a symbol of the owner’s wealth. The Red House is one of the important sites operated by the local cultural and creative industry. In contrast, a two-story brick building was designed with decorated with the phablet. Many modernist buildings in Dihua Street are also adorned with family symbols. Bopiliou, which was originally constructed in the Quin Dynasty, was remodeled from a main street to become a small alley. It is also worth noting that the preserved Quin style is different from that of the Japanese style.

AR facilitated a direct situated comparison of the differences between the two sites. For example, two building facades may have dissimilar widths. The initial allocation of the AR model in the context may have made the façade width fit into the new fabric, irrespective of the original proportion (Figure 9, right), and a further adjustment needed to be made by referring to the existing floor height. The adjustment revealed that, although the façade composition shared a certain similarity, the parapet featured a significantly different form of decoration as the model was made of jars.

9.2. Lukang Gate

Gates were installed in Lukang to ensure security in the past. The same type of gate was installed in Moncha, Taipei, for the same purpose. Since most of the gates were demolished, AR simulation was carried out in the same district to reinstall the old fabric setting (Figure 11) to separate an alley into two isolated zones.

9.3. Beggar Seat and Creative Objects

Exemplification of the elements was made to the building fronts (Figure 12). Stone door support (also known as begger seats), old water pumps, and half wells were typical facilities for daily needs. The beggar seat is the term for the door frame support, which was usually occupied by beggars as seats in front of a temple. The Red House, which was designed as a market in the past, most probably had beggars, as well. As a consequence, beggar seats in AR were reinstalled in the old scene in front of public buildings.

The half well had a curved perimeter facing outside, and was held against the wall along the location near the diameter. Originally, it was configured as a full circle, and was then divided by a wall in half so that the neighbors could share the same well with the owner and access it from outdoors. Although this design restricted user access from all directions, the against-wall design presented an efficient application of space and the wall to hold hinges. The arrangement of space, wall, and hinges is one of the aspects that a user needs to be able to explore. This need is similar to a test held in another city in a different context for a proper selection of installed location and orientation. For example, should it be located in the center of a plaza or close to an entrance for easier access?

The installation revealed what may have been designed in the past for the convenience of local residents. Although the old buildings were remodeled as a showroom of cultural industry with new expectations, the AR-based recall of old facilities capitivated the researchers’ attention with an effective pedagogical illustration in a real environment.

10. Validation

A novel approach was applied to validate the situated study in terms of the accuracy and the appropriateness of AR simulation. This approach is termed “reverse engineering” of the reconstructed objects and space (Figure 13), and comprises the following steps:

• Step 1: download the 3D model in AR and transform it to an appropriate size and location (Figure 14);
• Step 2: take pictures from all orientations, similar to the creation process of photogrammetry (Figure 15, top);
• Step 3: apply these images in a photogrammetry application to create a 3D model of this simulated object and the background context (Figure 15, bottom);
• Step 4: inspect the final model to determine the accuracy and appropriateness of the study.

Inspection of the situated simulation began from the examination of the cultural context by referring to indications or dimensions of context. In this process, the following issues were identified that are worth noting:

1. 

Qualitative conflict of the installation of cultural element.

• The interrelationship between the well and the wall: The well was allocated with approximately two-thirds of it extruded outward from the wall, which was used to separate access of the owner from indoors and neighbors from outdoors. The current setting does not separate this well in a perfect half. Whether and how this heritage was remodeled remains to be explored. However, the reinstallation should consider three factors: (1) the location of the wall, (2) the space on the other side of the wall, and (3) the size extruded outside of the wall. The missing part of the installation on the other side excluded the inspection of the above mentioned concern.
• The interrelationship between the well and the context: Two cases were tested with the well allocated adjacent to the exterior wall of a resident house and to the brick fence of a flowerbed. The former appeared to be a reasonable selection of the site because its location was located proximal to an area that was accessible by residences. The latter appeared to be out of context in a small park, since the original round design of the well was merged into the Earth without accessibility.

2. 

Quantitative conflict of the scale, size, and context in the reinstallation.

• RP verification of the recessed wall: The secondary reconstructed model illustrated a finding regarding the depth of the well. The AR model is usually overlaid in front of the background. Unless the viewing depth was accurately specified, the wall was actually allocated further into the wall. The result was that the model had a part of the wall recessed compared to the remaining part.
• This was an interesting finding, that the depth of the test object and the depth of the background context were misaligned, in addition to the concern of the accuracy of AR adjustment. The offset was confirmed by 3D printed models at two locations (Figure 15, bottom). A second inspection of the screenshot in different orientations revealed that no large tolerance of alignment existed. This tolerance may be resultant from the drifting problem in AR while moving around to take screenshots. Misalignment also occurred to the tiles on the ground level.
• Unmatched brick scale: The situated study should have the result inspected by referring to the scale of objects in the background. Both the flowerbed and the well were constructed by bricks. The result of AR simulation illustrated a conflict of dimensions between the two brick types. Specifically, the old brick appeared to be thinner and wider, in contrast to today’s brick, which was thicker and narrower. In general, the brick in the Lukang half well should be larger than the one used in the flowerbed.
• Compatibility: The incompatible site of context can be seen in initial tests, where a step prevented a direct attachment of the well as part of the wall (Figure 16, left). A circular fence may not be applicable, both due to the different designated function and the potential conflict in configuration of adjacency (Figure 16, right).

11. Discussion

Sustainable cultural preservation can be achieved through application of AR to facilities, temporary installations, and festival fabrics. The setting or objects can be reinstalled to illustrate the original appearance or utilized as a reinstalled reference of a relic. Digital preservation enables an interaction with artifacts demolished in natural disasters.

It is worthwhile to investigate how to collaborate old architectural history with physical interactive experience in a today’s urban context. Part of the model’s creation in this study aimed to preserve cultural elements. Moreover, the most important role of the models was to reinstall old scenes as a way to mirror how we adapted ourselves in today’s environment or to what extent we changed the existing fabric. The review process, which borrowed subjects from different places at approximately the same time of construction, is the key to sustainable awareness of past development.

One of the issues explored in this study was to correlate the cultural elements and urban fabric as an integrated sustainable evolvement effort. The fabrics are shared by local residents, tourists, and religious events. The same streets, which are used for public traffic and tourism, are also selected by pilgrimages. While the fabrics evolved, second skins were created to meet the needs of daily or seasonal activities. Not only were the heritages or buildings demolished or remodeled, but urban elements were also created or installed to form novel cultural identities.

The interaction between fabric and elements, which presented a profound historical understanding of local development, should be applicable to enlighten similar stories of evolvement under different contexts. Either in a coherent or contrasting manner, the interaction features a situated comparison using old experiences as a reference to explore the development of a new environment. The exploration was considered as a creative discovery process that reached for new findings and, simultaneously, reviewed old cases as a back-and-forth process. With the assistance of AR or VR simulation and photogrammetric modeling, the reviewing process reached a level of reality that both the 3D model and context were unable to achieve previously. The simulation described in this study has wide applications outside of laboratory settings. Indeed, it enables comparisons made in an environment that has to be visited in a peripheral context setting for a broader involvement of culture.

11.1. Contributions

The promotion of heritages in a remote site presents a new measure of feasibility that enables cultural sustainability. Promoting heritage in terms of heterogeneous backgrounds correlated the new space and traditional wisdom in an enriched visualization of interaction. The study presented an effective and efficient remote or situated learning process that correlated the development or setting of both locations. Correlation was made to different locations, with a higher fidelity of appearance and utilizing a flexible transformation interface. The situated comparison also contributed to sustainability of local identity and 3D data.

Cross-site experience of heritage often occurs, with the comparison of similar elements in styles, materials, or construction methods. The experience is closely related to the interpretation frequently applied in architectural pedagogy of history. Comparing the AR application, although both technologies provided a sense of 3D depth and reality, the real background of AR provided a more direct connection between the original function of the element and the supporting environment. This is important to the comprehension of the background, since the original setting can be reviewed or examined during the AR simulation with the background context similar to, or different from, the original one.

Cultural elements, such as the half well, are unique representative symbols of local identity. Whether this element can be found elsewhere remains to be determined. If the well-preserved heritage can be presented from different locations, a new pattern of learning experience can be created without the restriction of geographic limits.

A mobile interface with a convenient smartphone configured settings that were sufficiently flexible and easy to apply. The aim of situated learning can be fulfilled by applying a smartphone with an appropriate app supported by cloud access. A combination of AR and 3D photogrammetry data was made to smartphone apps, Augment^® and Sketchfab^®, for field architecture comparisons through an easy application of high fidelity 3D models.

The safeguarded and enhanced heritage values were mainly contributed by the feasibility for situated comparison. Situated study enabled mobile learning, in which a user was immersed in the field reality as an extension and connection to the value of the original cultural elements. For the best judgement of comparison, a new site of coherence or diversity extended former cultural efforts and enlarged the perspective of safeguard heritage values.

11.2. Research Restrictions

Depending on the limits of the subscribed plan, models may need to be reduced to a smaller size prior to being uploaded for AR interaction. For example, Augment^® needs to be run in an ARkit^®-compatible smartphone with Android^® 8+ or later to have the plane anchored, as found in the latest iOS^® version. The number of published models is also restricted in Sketchfab^®, compared to the larger number of models that can be simultaneously uploaded in Augment^®.

Photogrammetry applications come with different levels of efficiency in terms of uploading, detailing, and smoothness in operation procedures. Four types of images were used in the photogrammetric modeling: images retrieved from UAV video, UAV images, smartphone or digital camera images, and images retrieved from video clips on the Internet. Streaming stability may prevent uploading a large number of photos continuously. Additional management has to be planned to partition the site into smaller blocks under a controllable size and followed by registration of all segments. A 3D model of the Artist Village was created using ReCap Photo^®, Photoscan^®, and 3DF Zephyr^®. It was found that 3DF Zephyr^® created superior details.

This study was subjected to certain levels of constraints during the application of hardware and software. Although UAV was a perfect system to take aerial photos quickly in a large area, the flight plan experienced sudden wind and the blockage of advertisement panels above street level or at low altitudes. In addition, the system was operated by one person. The 15-min battery life also limited the filming range and time. Moreover, the large number of photographs required intensive computation from the CPU and graphic accelerator. The numbers varied from 10 to above 1000. Although certain applications support cloud-based computing, the host-based application took from 15–20 min to several days to compute a since scene. The size of the 3D model was enormous.

12. Conclusions

Lukang is rich in urban context, which also attracts young generations. The promotion of cultural tourism is important and should be considered as a sustainable way of preservation. Interaction with the elements opens up an active involvement of culture and that is how this research was operated—by aiming the preservation of traditional elements and urban fabric with enabled interaction in AR.

The interaction between urban fabric and cultural elements has created a unique cultural character in Lukang. The interaction was promoted by tourism and religious activities to enrich predefined open spaces, such as streets and plazas. The local tourism-related activities have been proven to be effective measures of sustainability for cultural preservation.

This study converted physical artifacts to digital format for documentation of cultural entities and street blocks. We presented artifacts to be applied in an AR or VR environment, serving an architectural pedagogical purpose. The 3D platform and cloud database enabled an Internet-supported study. The interaction platform of AR can now be utilized to facilitate the preservation of Lukang history, even after the urban existing fabric is demolished in the future.

Both of the applications were used as sustainable assistance in cultural preservation. For example, the temporary installation deployed during the annual festival can be displayed at the same location. The 3D documentation can also be used to recover the original configuration upon damages caused by natural disasters. This study did not purposely program AR- or VR-related apps. Instead, open app resources were applied for a low learning curve and higher accessibility on a smartphone platform. This attempt was demonstrated to be feasible in the preservation of cultural entities with more interactions to attract people’s attention as a pedagogical approach for students and visitors.

The situated learning process was conducted using case studies to reflect old settings, correlate old vocabularies, and contextualize the buildings constructed about the same period of time. A new measure of feasibility was proposed in a remote site as an effective and efficient promotion of heritages in cultural sustainability.

Future studies could test a large-sized model, such as a street, to achieve a more integrated situated comparison. Compared to the individual application of a façade, a larger model may be correlated by style or skyline as a reference for more evolving information.