Augmented Reality for Historical Storytelling. The INCIPICT Project for the Reconstruction of Tangible and Intangible Image of L’Aquila Historical Centre ^†

Abstract

The urban landscape of the city of L’Aquila has undergone profound changes because of the 2009 earthquake. In particular, there has been a gap between the tangible immanence of places, outcome of history that brought them to the present, and the dimension of the intangible as the intersection between memory and everyday life. In this context, applications of augmented reality offer themselves as an instrument of intersection between history and memory. Augmented reality becomes a complex storytelling tool, where the balance between the authoritative aspect of history and the participative re-meaning of urban sites can be summarized.

1. Introduction: Between History and Memory

The 2009 earthquake has led to profound changes in the urban landscape of the city of L’Aquila and especially the historic center, the symbol place of the community identity, according to an ongoing process far from being concluded.

In particular, there has been a gap between the tangible immanence of places, result of history that led them to the present, and the dimension of the intangible as the intersection between memory and everyday life.

Transformations are linked to many different factors: Housing emptying; Limitations of use and access; The transfer of activities and the transformation of ways of using the sites; The modification of urban wings—both for the buttresses and for the reconstruction interventions; Reconstruction/restoration times; Last but not least the phenomena of the reappearance of usable spaces, sometimes in continuity but often in discontinuity with the past, also by new generations with limited direct experience of the status quo ante.

The issues relating to memory, identity and authenticity are renewed, concepts highlighted in the Operational Guidelines for the Implementation of the World Heritage Convention [1], and already in the Nara Document on Authenticity [2].

This is a fundamentally cultural problem that focuses on the theme of the places meaning, on the one hand as knowledge of historical and architectural values, on the other as a vision, namely configuration project. So basically, the relationship between “history” and “memory” is evident.

2. The INCIPICT Project: New Opportunities from the Optical Net and 5G Experimentation

The work is part of the INCIPICT project (Innovative City Planning through Information and Communications Technology, http://incipict.univaq.it), funded by the Interministerial Committee on Economic Planning (CIPE) and aimed at the development of solutions oriented to Smart city.

From an infrastructure point of view, the project is supported on the realization of a fibre optic communications network that connects the main points of the city. The construction of such a network is made possible by the intense re-construction activities of the historical center and by the review and reorganization of the entire sub services network (water supply, electricity network, telecommunication network, etc.).

The availability of this communication infrastructure and the potential of ICT technologies give the opportunity to conduct experiments relating to different fields of research. In particular, the project proposes to develop advanced services in three different sectors: structural monitoring of buildings, building automation and energy efficiency improvement, enhancement of cultural heritage.

The INCIPICT project is coordinated by prof. Fabio Graziosi and it makes use of a multidisciplinary Management Committee composed by: prof. Alessandro D’Innocenzo for the modeling and control of communication networks; prof. Massimo Tivoli for the Software Area; prof. Cristian Antonelli for the optical unit; prof. Vincenzo Gattulli, responsible for the structural monitoring sector; prof. Stefano Brusaporci, as the coordinator of activities relating to the enhancement of cultural heritage through ICT.

Within the INCIPICT project, experiments on some areas of L’Aquila historic centre are underway (Figure 1), realizing 3-D reconstructions of urban landscapes as they were in the past.

Augmented Reality applications make it possible to correlate visualizations to the current status of places, layering both didactic and shared information.

The experiments described, particularly useful for the centres of the municipality of L’Aquila, are of general validity, enabling the establishment of procedures for the valorisation of cultural heritage, and offering them like best practices for further and more general applications involving the social, cultural, and cheap fabric.

3. The Cultural Dimension of Places

The “Cultural Heritage” definition of UNESCO highlights the importance and interconnection of tangible and intangible aspects: “the legacy of physical artifacts and intangible attributes of a group or society that are inherited from past generations, maintained in the present and bestowed for the benefit of future generations”.

Regarding the concept of “Places”, Silberman [3] raises the following question: “How can such sweeping, all-inclusive definitions—attempting to cover all material, immaterial, objective, and subjective aspects of heritage places—possibly capture their irreducible essence? How can we identity the distinctive characteristic of heritage places without endlessly extending the list of specific material forms and subjective associations a heritage place may possess? At a time when officially recognized heritage types are splintering into distinct regional, ethnic, and religious variants […], when the UNESCO World Heritage list has exceeded a thousand inscribed properties all said to possess Outstanding Universal Value […], when the possession and control of cultural heritage resources have become symbols of political legitimacy, no single physical definition of a heritage place seems capable of capturing the thoroughly fragmented reality of cultural heritage in the twenty-first century. Thus, focusing on process, rather than object, may be a more useful approach” (p. 30). Silberman highlights the difficulty inherent in the definition of “heritage” and the complexities encountered in its identification. In particular, he thinks about how the recognition of “heritage” in the past was often attributed on the basis of the following topics:

Official monuments with outstanding historical or artistic significance; Religious places; Places of civic manifestation, symbol of resistance or freedom, or on the opposite (or in parallel) of state power; Heritage places as entertainment venue; heritage places as reaction to change, related to self-perception and memory of communities [3] (pp. 29–40). From this consideration comes a reflection on the defining process of the heritage itself, rather than on its authoritative definition.

In this regard, it may be useful to recall how the UNESCO World Heritage Cultural Landscape [4] bases the “Landscape” definition on the concept of culture, according to an interpretative line tending to integrate tangible and intangible values within the Landscape idea itself. The Convention for the Safeguarding of Intangible Cultural Heritage [5] highlights how heritage is constantly recreated by communities. Not only intangible heritage but also tangible one, as the result of the continuous and secular interaction between anthropic actions and nature.

Hence the conception of a heritage-making process that blends the authoritative role of institutions with participatory approaches.

However, the concept of heritage has been expanded over time. In particular, following the Convention on the Safeguarding of Intangible Cultural Heritage [5], there has been a shift from the distinction between the different types of heritage (natural, cultural, artwork) towards tangible/intangible dualism.

The intangible heritage “means the practices, representations, expressions, knowledge, skills”. For example: oral traditions and expressions, including language as a vehicle of the intangible cultural heritage; performing arts; social practices, rituals and festive events; knowledge and practices concerning nature and the universe; traditional craftsmanship (Art. 2).

As indicated for the architectural heritage by The Venice Charter [6], the awareness that buildings and sites are not isolated but inherently related to the context—where the term “context” refers to both the physical and cultural spheres—directed the study also toward experience relevant to cultural, social, economic and environmental spheres.

This has led to new lines of thoughts and conservation strategies based on the fact that a site can not be identified simply by a border but rather taking into account the culture and activities of the locals. Over the years, two lines of study have developed in a substantially converging way: on the one hand, there has been an extension of the heritage concept from monument to landscape; On the other hand, the idea of site has shifted to that of environment intended in an ecological and overall manner.

So, The Burra Charter, in the definition of “Place”, correlates the manifestations of tangible and intangible specific to cultural heritage: “Place means a geographically defined area. It may include elements, objects, spaces and views. Place may have tangible and intangible dimensions” [7] (p. 2). Furthermore, defining the expression “Cultural significance”, it indicates how this “means aesthetic, historic, scientific, social or spiritual value for past, present or future generations. Cultural significance is embodied in the place itself, its fabric, setting, use, associations, meanings, records, related places and related objects” [7] (p. 2). In this sense, the intangible is set to the dimension of the tangible. And also, a Place “may have a range of values for different individuals or groups” [7] (p. 2) that is the interpretation, values, and meaning of a place can vary for different groups of people and cultures.

The merging of natural and anthropic, and of tangible and intangible, arise again in the Recommendation on the Historic Urban Landscape [8] defining urban landscape as: “the urban area understood as the result of a historic layering of cultural and natural values and attributes, extending beyond the notion of “historic center” or “ensemble” to include the broader urban context and its geographical setting” (Art. 8). In particular, the Art. 9 says: “This wider context includes notably the site’s topography, geomorphology, hydrology and natural features, its built environment, both historic and contemporary, its infrastructures above and below ground, its open spaces and gardens, its land use patterns and spatial organization, perceptions and visual relationships, as well as all other elements of the urban structure”. The urban area is intended as the result of an historical stratification of values and both cultural and natural characteristics that extends beyond the notion of historical center including a wider spatial, historical, and cultural background”. The Art. 9 adds: “It also includes social and cultural practices and values, economic processes and the intangible dimensions of heritage as related to diversity and identity”. Tangible and Intangible are both considered, where tangible is the physical expression of intangible; it is the outcome and witness of a complexity of relationships, and intangible is a fundamental aspect.

The Internet and Social Media allow the interactive and continuous practice of remembering and of memory sharing and definition. In conclusion, the concept of heritage has become increasingly wider and more complex. The heritage definition has focused on a cultural context where the relationship between tangible, intangible and digital is an open question.

In this context, advanced applications of augmented reality bring information on reality; therefore, they favour a participative, open-ended and ubiquitous discussion on heritage, including into the dialogue the Internet space, the material buildings, the social sphere, the physical space, tangible and intangible aspects.

4. Augmented Reality as Digital Heritage

The Charter for the preservation of the Digital Heritage [9] defines it as a cultural expression of human work. Therefore, it creates new legacy. In particular, digital works from physical heritage could be significant ways for heritage interpretation, communication, and conservation.

Pescarin writes “Although the accepted UNESCO definition of Digital Heritage concerns any digital material referred to our heritage that has a value and needs to be preserved (UNESCO, 2003), Digital Heritage is used today by the scientific community in a wider sense, referring to ICT applications and technological approaches to our cultural and natural heritage, or, better, to the use of digital media in the service of heritage (Cameron & Kenderdine, 2007) [10]. Digital Heritage is a domain that comprehends several different research fields and disciplines, from museography to computer graphics, from archaeology to design, from art history to engineering, from archives to statistics, etc. It is therefore a general term, which includes many ICT topics and heritage themes, and in most cases most of nowadays research lays in the overlapping and interconnection among them” [11] (p. 1).

Modeling and visualization tools represent applications for knowledge, understanding, interpretation and communication of cultural heritage. In particular, focusing on the built heritage, the 3-D model favors a communicative and multidirectional flow where information of different nature participates in the message creation. In this sense, the concept of model has to be intended in a complex way, where three-dimensional elements are related to databases. Data, metadata, paradata, and information are the core of the model: 3-D models and databases are two sides, and the modeling process has to involve both of them. The 3-D model’s semantization and the “granulosity” of its components are important aspects for the correlation with the informative model [12,13]. The same model allows different and customizable visualizations. Moreover, these multi-layered and multi-windowed views are joined the mobile applications and, therefore, can be accessed anywhere and at any time. This complex informative model offers new non-linear narratives, favoring the knowledge, communication and design [14].

The “Mixed Reality” expression [15] describes the “virtual continuum” between only-real and completely virtual environment perception, identifying six classes of hybrid states. The Mixed Reality offers different ideas to think about. For example, it is particularly interesting not to focus on the presence or absence principle of physical or virtual reality with reference to the operator, but rather on the way information is displayed. Therefore, the concept of Hyperreality [16] does not seem to have been followed, on the contrary, ICT has encouraged reconnection between information and built heritage.

5. The Survey of the Façade of the St. Margherita Church in L’Aquila

The experimentation covers the facade of the church of Santa Margherita in L’Aquila (17th century), whose stone facade was completed only in the basement (Figure 2). The study roots on the analysis of the current state of the church and, therefore, on its survey, outcome of previous campaigns conducted in several phases.

In the first half of the 90’s, the survey of the architectural complex of the church and college was carried out by a working group of the University of L’Aquila coordinated by Mario Centofanti: in particular, the church with its facade was the subject of a direct survey integrated by photogrammetry, led by Gianfranco Ruggieri and Marco Canciani. After the earthquake that struck the city of L’Aquila, the survey of the church is integrated through laser scans made by Romolo Continenza in 2010. In 2014, a new architectural survey of the whole church is made by Castellano [17]. The analysis of the historical documentation refers to the numerous studies carried out on the monument [18], which offer the design drawings of the façade. The façade was designed in 1625, but the church building began in 1636 in a site near but different from the one defined in 1625. Based on the knowledge of the monument in its immanent physicality and the documentary apparatus, the historical-critical analysis was carried out, particularly finding as the project was modified during the construction.

6. Augmented Reality and Virtual Applications

The case study focuses on the virtual reconstruction of the facade never completed, and its representation through augmented reality applications (Figure 3). The aim of the study is the interpretation and communication of the architectural heritage and the massive and shared fruition of the church.

In the preliminary stages to the modelling, a critical comparison was made between the different drawings available. Firstly, the adequacy of the restitution of direct survey was verified by spatial positioning of the individual elaborates. Subsequently, the congruity between direct and photogrammetric survey, and laser scans was verified through the spatial positioning of the elements in a three-dimensional modelling environment. Geometric discordances have been identified and a configuration optimizing misalignments and inconsistencies has been found. This configuration was used to align and dimension the project façade coming from historical documentation.

Thus, a heterogeneous model was obtained through two-dimensional and three-dimensional representations used as starting point and framework for reconstructive hypotheses modelling (Figure 4).

By using mathematical modelling software, a first reconstructive hypothesis of the facade was made. It simultaneously took into account all the dimensional and decorative factors existing in the façade, documented by the survey, and project elements indicated by the historical document. This model had a dual purpose: on one hand, it was used to formulate successive and various reconstructive hypotheses; on the other, it has been a test model for the first experimentation of augmented reality applications.

The experimentation of augmented reality applications has been developed according to the following canonical phases: footage live; Definition of targets in the scene; Setting a three-dimensional model, suitably scaled and oriented, that is hooked to the target and displayed in accordance with the target’s location in the movie.

Then, a photogrammetric campaign of the square and, in particular, of the facade was carried out. Geometric and chromatic data were extracted and an orthophoto of the actual church façade was generated. The orthophoto was used to extract target for augmented reality, while the three-dimensional photogrammetric model was used as the reference basis for the correct positioning, alignment and sizing of the reconstructed model with respect to the extracted target (Figure 5).

7. Conclusions: Enhancing Reality with Information

3-D visualization and 3-D modelling applications find in Information and Communication Technologies the vehicle for reconnection between the virtual dimension and reality, thanks to the ability to interact in real time. In this way, the daily experience of the real is enhanced with information through the ever-present applications offered by mobile devices (Figure 6).

It is thus possible to enrich the architectural significance of information of various nature and origin: multimedia, textual, visual, etc. ICT also promotes participatory approaches, whereby the community—itself bringer of memory values—shares and re-elaborates a “vision” of historic sites, where a reconfiguration process is incorporated into tradition.

Advanced applications favour a synthesis of duality “history”/“memory”, through new multidirectional and nonlinear modes of interconnection, interpretation, presentation, sharing and reprocessing. At the same time, it encourages the preservation of the meaning of places [19].

In this respect, applications of augmented reality offer themselves as complex storytelling tools, where the balance between the authoritative aspect of history (in its scientific issues of relationship with sources, philological interpretation, critical analysis and presentation) and the participative re-meaning of urban sites can be summarized. Augmented reality, therefore, becomes an instrument of intersection between history and memory.