Artistic Methodologies in Forced Migration: Using Body Mapping and Augmented Reality in Syrian Refugees’ Narratives

Abstract

Millions of refugees fleeing war and persecution in Syria have received significant attention in both the public sphere and academic research. This article examines body mapping and augmented reality (AR) as artistic methodological tools that allow refugees to articulate their experiences of forced migration. Body mapping enables the creation of life-sized images which trace the contours of the individual’s body on canvas allowing refugees to express their emotions creatively through their own participation. AR adds another layer to the body map as it animates the artwork and brings it to life. Through body mapping and AR, refugee women from Syria are able to co-create research outcomes and counter gender-biased narratives of vulnerability and victimhood often associated with female Muslim refugees. This project critically engages with the use of art as a sensorialized medium to generate knowledge and examines the impact it shows on viewers during exhibitions. Based on research conducted in the United Kingdom, Germany and Jordan since 2017, this article discusses the use of these new technologies as novel research methodologies in refugee and migration studies.

1. Introduction

I miss my sofa. This sofa was in my living room. When my friends came to visit me in the morning we sat on it and drank coffee after our children had left for school. I wonder whether I will have that sofa again or have coffee with my friends again. We were forced to leave by the Assad regime. We are all in different places now. We have become different people. Now I have a new sofa, but I miss my old one. I miss my old friends, and I miss my old me.

(Interview with a Syrian refugee in Germany, 2018)

This article discusses the different methodological tools that can be used to address and examine contemporary forced migration. Diversifying research methods facilitates the examination of the multidimensional aspects of forced migration and the complex experiences of displacement. As ethnographers, we tend to use interviews and focus group discussions to access participants’ lives and analyze the social contexts of their environment. Visual arts-based methodologies are useful tools to add to such data collection, which have proven to offer researchers new perspectives and insights into the lived experiences of participants (Ball and Gilligan 2010; Shanneik 2018). We applied arts-based methods in our current project on Syrian refugees in Europe and the Middle East to examine the lived realities of displacement from the refugees’ own points of view. In this article, we examine the role of art and new technology as tools that enable refugees to communicate their physical and emotional states: the combination of body mapping and augmented reality (AR) is used for the first time in the context of refugee and migration studies. The application of AR to body maps allows participants to connect further digitized data to the physical artwork to express their feelings in non-verbal and non-linear forms, irrespective of language or literary skills. Such methods are more inclusive and have proven to provide new insights that may not be captured in full by conventional interview methods. In our project, this non-verbal way of expression is tied with verbal information gathered through interviews conducted with refugees before and during the production of the artwork (see Figure 1).

The use of arts-based methods in refugee and migration studies is primarily driven by the intent to create new combinations of research approaches and to inform representations through interdisciplinary collaborations between academic researchers and non-academic stakeholders. Arts-based methods are proven to be an additional tool for data collection, which position the participant at the center of knowledge creation. So far, studies on art and migration consider either migrants who are artists—i.e., artists, who have some form of education or at least experience in art production—or migrants who have joined art groups, such as theatre, dance or music groups. There has been increasing interest in recent years in researching arts, culture and migration (Martiniello 2022) to examine immigrants’ and ethnicized minorities’ cultural and artistic contributions to existing popular culture (Martiniello 2015). This led to the investigation of the role of art as a tool for political mobilization and resistance (Tripp 2012, 2013; Hasso and Salime 2016; Shanneik 2022) as well as political participation (Martiniello and Lafleur 2008) or as means to participate in and belong to a community (Damery and Mescoli 2019). However, the intersection between artistic techniques and refugees’ lives and their displacement experiences on a bodily and emotional level has not yet been investigated. Different from other studies, the refugees we worked with had no prior experience using artistic practices, nor had they created art themselves before. The work they have co-produced using body mapping and AR is part of a traveling exhibition1 that seeks to shape the public perception of refugees and intends to influence public discourses on refugee women in particular.

Body mapping is an artistic technique in which participants draw the contours of another’s body on canvas and then fill them with visual representations of their life stories. Since 2017, we have produced a total of 67 full-body maps of Iraqi and Syrian refugees during numerous workshops organized in the United Kingdom, Germany and Jordan.2 The body maps were created by women and their families of different age groups, diverse socio-economic backgrounds and ethnoreligious belongings.3 They were produced either individually or collectively in mainly private spaces such as their own homes or, on a few occasions, in public spaces such as community centers run by NGOs, the state or local councils.

In the second year of the project, we applied AR to the body maps.4 Through the use of the platform Artivive,5 the body maps become digitized via a computing device such as a tablet or smartphone. The refugees have not only co-produced the artwork but have also played a central part in feeding the digital information into the AR through videos, photos and sounds. Life-sized images of refugees are traced on canvas through tangible paint, colors, writings or scanned images. This is overlayed by intangible digitized visual information. Both worlds, the real and the digitized, need to be present as AR can only be accessed through the body map. The static tangible body map becomes a digitized moving space, thereby impacting the aesthetic experience of its viewer.

Our interest is placed on the ability of arts-based methods to induce corporeal communication since the body plays a central role in the refugees’ experiences of war, persecution and displacement. The involvement of the body in the body mapping process allows refugees to reflect on their displacement and stimulates their embodied awareness of the impact their displacement has had on them. This is conveyed through the use of art but also AR, which offers another layer of information communicated by refugees. Refugees’ individual experiences are made visible through both tangible body maps as well as intangible digitized AR. The bodies on the canvases and the information transmitted through the AR impact the space in which the artwork is exhibited.

The article starts with an illustration of the theoretical framework that discusses both body mapping and AR as a form of corporeal communication embedded within a sensory experience. According to neo-phenomenologist Hermann Schmitz, atmospheres are spaces in which corporeal communications take place through the individual’s engagement with that space (Schmitz [1969] 2019; Schmitz 2014a). The individual may be captured/attuned by the atmospheres produced, which in turn can impact the individual’s mood (Stimmung or Befindlichkeit). Following phenomenological approaches to bodily feelings and atmosphere that distinguish between the subjectively felt body (Leib) and the physical or material body (Körper) (Schmitz et al. 2011, p. 247), we argue in this article that body mapping and AR penetrate the atmosphere impacting both the felt and the physical body. This is achieved through what Schmitz, in another context, refers to as bodily-felt affectedness (leiblich-affektives Betroffensein) (Schmitz [1969] 2019; Schmitz 2014a, p. 75). The main part of this article discusses to what extent body mapping and AR are useful tools to foster deeper reflections on the physical and emotional experiences of refugees. The article concludes by highlighting the benefits of using art and new technology in refugee and migration studies. Using such research methods expands the type of knowledge produced about refugees and provides alternative perspectives on forced displacement articulated by refugees themselves.

2. Phenomenological Approaches: Corporeal Communication through Art

In his New Phenomenology, Schmitz (2014b) discusses the relationship between the body and emotions. He argues that emotions are not an internal subjective state but rather penetrate the physical space in which humans find themselves. For Schmitz, emotions exist at the intersection between the human (subjective body) and its surroundings (Umgebung) (Schmitz 2014a; von Uexküll 2010, p. 43). Atmospheres constitute a space that might generate feelings that capture the body that receive them (Schmitz 2014a). Feelings/emotions are present within a space/atmosphere that is captured (ergriffen) differently by individuals (Schmitz 2014b, p. 86). It is the individual’s experience within that space that defines the feelings that are generated (within that same space) by that particular individual. So, in other words, feelings/emotions are conditioned and dependent upon one’s own experience.

Martin Heidegger argues that human existence is always emotionally conditioned (gestimmt) by our moods (Stimmungen, sg: Stimmung) (Heidegger 1996, pp. 134–42).6 Atmospheres affect us emotionally as humans, impacting our mood and our general affectedness (Befindlichkeit).7 For Heidegger, both moods and affectedness are not only interrelated but also coexistent. Feelings such as joy, hope, fear and anger are examples that affect our mood and our being in the world. Such feelings present what matters to us as human beings and highlight our engagement with the world around us but also with ourselves. In other words, through the individual’s experience of the atmosphere (our involvement in the world), moods are generated that affect our affectedness which in turn demonstrates what matters to us as individuals as being in the world.

In this article, we would like to give an example of how aesthetics and technology enable an alternative form of knowledge production on and by Syrian refugee women and their displacement. In our project, we have used different research methods in the form of body maps and AR to generate multisensory and multimodal human experiences of forced migration. Corporeal communication and its bodily-felt affectedness (leiblich-affektives Betroffensein) are central to our approach (Schmitz [1969] 2019; Schmitz 2014a, p. 75).

The body is used in the artistic production of body maps. The refugee lays down on a canvas for another person to draw around the body. During this process, the person gradually turns from a formerly passive position (lying down on the ground) to a more active one (e.g., standing upright and painting the body map). Such an embodiment process creates an atmosphere that affects the refugee’s own mood and affectedness as it allows the refugee to reflect, remember and articulate experiences artistically on canvas. The painting of the body map can be experienced as an individual but also as a collective atmosphere that is generated through the relationship between the different body maps placed on the same canvas and created collectively. Moreover, the refugees we worked with shared their personal struggles for survival which often entailed feelings of loneliness, isolation and despair. While these emotions are inherently individual and intimate, the use of body maps helps transform them into a collective and intersubjective experience. Through these maps, a context of affective solidarity was created among the refugees who, despite coming from diverse backgrounds and circumstances, could find common ground through shared experiences of war, flight and the search for safety.

The body map paintings exhibited in an exhibition space offer another layer of bodily-felt affectedness (leiblich-affektives Betroffensein), this time between the refugees’ bodies on canvas and the viewer. Here, the affect could be generated from one canvas or from the placement of several canvases within one single space. Body mapping is, thereby, one way of corporeal communication that forms the basic structure of our perception (see Figure 2).

In this article, we would like to problematize the notion of perception and how we as humans, experience the world around us through the discussion of AR. We argue that AR complicates traditional notions of perception by laying a virtual world onto the real world. AR produces a richness of experiences and relatedness that has the ability to generate a form of sensibility and understanding. The goal of AR is ‘[…] to enrich the perception and knowledge of a real environment by adding digital information relating to this environment’ (Arnaldi et al. 2018, XXVI). This information is mediated through a device that forms an ecology between the body and the device. Such an ecological interaction ‘brings virtual objects to life and it brings virtual life forms to the perceived space of the body’ (Gould 2014, p. 26). Whilst viewing the AR, the device appears as a ‘vanishing medium’ (Sterne 2003, p. 218)—disappearing in the sensorialized experience of viewing. The augmented perception redefines, thereby, the spatial dimension that moves from the immediate to the augmented environment. In other words, the real/augmented and the material/immaterial become one for the viewer to thereby deconstruct the boundaries of spaces and places.

Scholarship on visual culture has suggested not to focus solely on the image but on the act and ways of seeing. The act of seeing is a cultural operation in which the viewer enters into a relationship with the object that stretches across time and space (Morgan 2005). For Morgan, images are

[…] forms of mediation, filling the spaces between bodies, places, things, saints, dreams, and nations. Images are the material, moral, and imaginative technologies without which these sometimes very abstract or immaterial realities would remain quite unreal. […] Images deliver information, they touch and move, they intuit what cannot be seen, and they imagine communities and connections that complete the individual by securing his or her relation to a larger reality.

(Morgan 2015, p. 9)

The degree of an individual’s felt-bodily affectedness (leiblich-affektives Betroffensein) depends, however, to what extent the viewer is captured by the augmented perception: the digitized sonic atmosphere produced by AR constructs new environments that generate different moods and affects. Günther Buck’s theoretical framework focuses on the experienced-based conception of learning that has the ability, as he argues, to form the fundaments for understanding (Verständnis) (Buck 2019). He argues that such experiences need to be subjective and personal, achieved by the momentum of self-being there (Selbst-dabei-Sein). Once the device used to enable the AR appears to have vanished, viewers may experience it as self-being-there—i.e., in the world of AR. In such situations, the viewers’ affectedness and degree of engagement with the AR increase. AR is, thereby, able to facilitate the compression of distance and emphasizes connectivity between the viewer and the refugee. The refugee’s environment becomes closer to the viewer’s environment, enhancing, thereby, a sense of shared experience and understanding. AR enables the viewer to be virtually there and, by doing so, leads to the despacialization of the viewer and the refugee. Such despacialization allows the viewer to relate to the refugees and their experiences of displacement.

Situations are, however, perceived differently by a variety of people who make variant associations and links or what Massumi calls the ‘felt reality of relations’ (Massumi 2002, p. 16). In addition, the despacialization process enabled through the AR in which the medium may appear to have vanished collapses once the internet is lost or the connection to the body map is interrupted through, for example, someone walking between the body map and the device. The ecology between the body and the device that once connected both worlds, the direct and the augmented environment, is disconnected: the atmosphere is interrupted, which often affects the mood (Stimmung) and the affectedness (Befindlichkeit) of the viewer.

3. Body Mapping as Research Tool in Refugee Studies

3.1. Methodology

Body mapping is an artistic technique for creating life-sized images that traces the contours of one’s own body on canvas (see Figure 3). It has its origin in art therapy. As an arts-based storytelling technique, it is mainly applied to body-related issues, especially concerning health. First used in the field of health research in 1985 by MacCormack and Drapers while studying the fertility rate in Jamaica in comparison to the UK (MacCormack and Drapers 1987), it is today “formalised as research tool to both create and communicate knowledge” (Solomon 2020) as suggested in the most recent and comprehensive book on this topic (Boydell et al. 2020a). In fact, Solomon’s (2002) earlier fundamental work in the field of HIV/AIDS in South Africa, published in 2002, in which she systematically applied body mapping, was decisive for body mapping to be used predominantly in this domain (de Jager et al. 2016). Body mapping is also used in research with people and marginalized groups impacted by psychosis or an intellectual/cognitive disability (Boydell et al. 2020b), complex trauma (Collings and Smith 2020), or anxiety (Cox et al. 2020).

Thus, it can be said that today in health-related research, the therapeutic potential of body mapping as a process and its outcome’s benefits are increasingly recognized, and the technique itself has been mostly studied with a focus on the method’s principal attributes: representation of first-person narratives that counter the potentially biased narratives of researchers, embodied awareness given the method’s collaborative and reflective process as well as its knowledge translation potential (de Jager et al. 2016; Boydell et al. 2020a).

In this study, body mapping is used for the first time in the context of Muslim refugees from and within the Global South. It allows refugees to express their experiences of displacement through colors, writings, scanned objects such as their own passports and travel documents, as well as handprints. In the following, we will introduce the process of using body maps in our research among Syrian refugees in the United Kingdom, Germany and Jordan followed by an analysis of their content and meaning. The second part of this article will then move to discuss how the use of AR adds another layer of meaning: the immateriality of the invisible digitized data collection is juxtaposed with the visible body map, thereby impacting the viewers’ atmosphere and their act of viewing.

3.2. The Production of Body Maps

Before introducing the method of body mapping to Syrian refugee women and their families, we conducted ethnographic fieldwork with them in the form of interviews and focus groups. Based on the data collected in the field, we invited the women to participate in the art workshops we organized. At the start, everything seemed to be unfamiliar for the participants as the spaces we usually had our conversations in had changed: plastic covering the floors, paints, brushes and buckets of water, as well as large pieces of canvas spread throughout the space. The idea of laying down on a canvas for someone else to draw the contours of the body was also new to them—although the majority enjoyed being able to reflect on their lives in such a creative way. Body mapping and painting, as an artistic means of expression and as a creative process, offered refugee women a new way of expression and thus opened up novel ways of reproducing lived and felt experiences. The women were able to decide in which position to lay down on the canvas and what elements to use to tell their stories in the form of colors, writings and images. Some participated as individuals, others in groups reflecting and discussing the different ways of expressing themselves through art (see Figure 4).

The workshops took several weeks: because of other commitments, some women started their paintings but were unable to finish them, allowing other women to take over. This made the painting project a collective one between women who had never met but shared similar experiences. The use of acrylic paint helped in building layers of narratives, images and colors. The workshops became a space for the aesthetic articulation of displacement in which socialities are formed through shared experiences of forced migration. The role of the artist, Rachel Gadsden, was central in introducing refugees to the technique of body mapping but also to the act of painting more generally as most of the women who participated never had a paintbrush in their hands before.

The body maps’ intrinsic personal and biographical elements culminate in individual choices to include (or not) names and portraits of the participating women. Some women have asked Rachel to draw their own faces and the faces of their family members or showed Rachel pictures of famous personalities such as actresses whom they thought to look very similar to them. The refugees we worked with hail from a broad spectrum of urban and rural areas in Syria, encompassing a wide variety of backgrounds, including Arabs and Kurds with diverse religious, socio-cultural and economic backgrounds. As a result, their experiences as refugees differ greatly, which was effectively conveyed through the diverse symbols they have asked Rachel to draw, such as flowers, refugee camps, forests, dinghies, barbed wires or butterflies. The images that Rachel was asked to draw on the canvases are based on refugees’ own descriptions and imaginations or on their own pictures and videos Rachel took as a basis for her own drawings. The titles of the paintings are based on the central theme of each painting participants decided upon while creating the artwork. An essential aspect of the process of body mapping is the authorship of the refugee women who are significantly involved in the production of the paintings. The end product, however, was a joint collaboration between refugees, the artist and researchers, co-creating knowledge on displacement experiences. The artist and the researchers involved facilitated the artistic production but also contributed to its making by incorporating data collected during the interview and focus group processes.

3.3. Imagery and Content

For the large canvases, a combination of body maps and individual portraits has been used to create intense artworks filled with symbolic objects and meaningful writing, all immersed in expressive colors. Such elements determine the composition and hence the effect of the paintings on the viewer. The body maps correspond to the physical aspect due to embodied participation since body mapping requires the use of the body. The body maps as contour-line-imprints have an indexical status which lies in the practice of this technique itself. It is characterized by an existential relation to the respective woman since she actually has been lying there on the ground while the lines have been drawn around her body, creating different postures and expressions, imprinting her physical existence and experience on the canvas.8 The visual language found here serves to visually reproduce lived and felt experiences and to make individual, first-person narratives visible in images and symbols, written words and phrases and color. Mixing of different genres has created its own visual language, which has timeless and idiosyncratic power.9

The body map below is called The Sisters and tells the story of four sisters who fled persecution and violence in the ongoing war in Syria (see Figure 5). On foot, they were able to cross the border through barbed wires and escape to Jordan. First, they went to the Zaatari refugee camp in Jordan, but due to the insecurity and bad living conditions of the overpopulated camp, the family decided to escape the camp and move to the capital city Amman.

The expressions through postures and gestures, colors and painting style (such as the expression of the brush strokes), the images or symbols as memorial items are linked to the emotional experience of refugees. Thus, the body maps become vessels of visualized items of significance, which are linked to individual memory, experiences and emotions.10 The images have symbolic character, and the imagery creates specific semantics and meanings. The Sisters body map, for example, depicts the refugee journey of four sisters who entered Jordan wearing their traditional Syrian dress (the three sisters on the right). After their arrival in Amman, they felt at home. The feeling of safety and integration is expressed through the sister on the left, who took off her Syrian dress and decided to wear a traditional Jordanian gown. This is supported by words such as ‘security’ and ‘safety’ written on her right arm in Arabic. Each of the sisters also has her own story to tell: the sister in yellow, who entered Jordan with her two sons, is fighting for divorce from her husband, who had returned to Syria and married a second wife.11 The sister in red decided to draw an eye with tears on her body not only to express her pain for the loss of her homeland Syria. The eye is placed on her womb, which sheds tears on the numerous miscarriages she has had to endure over the years, leaving her childless up to this day as she explains: ‘Like me, Mother Syria was unable to protect and keep her children. We both bear the painful scars of losing our children and our loved ones. We share the same grief. We are united in our sorrows and tears.’

During the early years of the Syrian war, public discourses in Europe often represented refugee women and Muslim women as vulnerable, oppressed and in need of help (Kapur 2002; Shanneik and Vahle 2023). During our workshops, refugee women were very keen on creating a different picture of themselves. In our conversations but also through artistic expressions, refugee women positioned themselves as strong, active and powerful women who are in charge of finding a secure place for their families and for themselves. In The Sisters body map, for example, all four women insisted on making it clear that it was their own decision to leave Syria as they no longer thought it was safe for their sons or husbands. In most cases, men did not want to leave but rather wanted to wait and see how the situation in Syria would develop. Fearing for the safety of their male family members of either being killed, arrested or conscripted into the Syrian army, the women decided to leave Syria and find refuge in neighboring countries. Women in war situations have become the protectors of their families by thinking pragmatically, as one of the sisters explains: ‘I did not care about my house, my neighbours, or our shop. All I could think about is my sons and husband need to be safe. I, therefore, convinced my husband to come with us and cross the border to Jordan.’ Leaving the Zaatari refugee camp was also a decision the women made, as the sister in yellow explains: ‘he [my husband] started to become comfortable in the camp. Sitting all day doing nothing. I did not like that. The camp was horrible. Not a place for human beings to live in. We had to leave. So, I dragged him out of the camp, and we left for Amman. But he was never happy in Jordan. Always yearning to go back to Syria. At the end, I told him you go but my boys stay with me here. We are safe in Jordan. We are not going back with you’. The husband ultimately returned to Syria and married another wife there, leaving his first wife (the woman in yellow) struggling to divorce him in Jordan.

Taking over the responsibility of the family was also the narrative of the body map entitled Human Rights (see Figure 6). It depicts a family, a wife, her husband and two children who first fled to Jordan. In Jordan, the wife convinced her husband to apply for the UNHCR resettlement program based on his poor health condition. They qualified and moved to the United Kingdom. There, the wife was granted a scholarship to study law at university. A couple of years later, she informed us that she graduated and is completing a two-year Legal Practice Course to become a solicitor in England and Wales, specializing in human rights. During the time of her study, her husband stood beside her by taking care of their two children, as she explains ‘when I head to university in the morning, I know that my children are taken care of by their father who cooks for them, cleans the house and helps with their homework. This helps me to concentrate on my studies. My goal is to become a human rights lawyer, and once I achieve that, I plan to support my husband in his career plans’. In the family body map below, the wife is at the center with her husband’s body next to hers. Beyond her veil, her hair is symbolically embracing her family as she is the bearer and protector of the family. It is the female body that is depicted here as the main center of power, resistance and protection. In this body map, only the face of the mother was asked to be drawn, which adds another layer to the expression of the power of the woman’s presence and the role she plays within the family.

3.4. Results: Body Maps as Visual Voice

Refugees’ artworks are the creative responses to their specific environment of existential danger, personal trauma, uprooting and new beginnings. The imagery and visual strategies of self-representation found in this process of active participation comprise different levels of meaning and a high degree of complexity. The refugee women we worked with act counter-hegemonically to the public perception of Muslim women in Europe as victimized objects within their societies: they have presented themselves as survivors of war and persecution, as strong women who were able to protect their families during their flight or as main bread-winners through home-based entrepreneurial work.

Refugee women have used our numerous exhibitions organized across Europe and the Middle East as a tool to define their subject positions in public discourses anew. They use aesthetics to articulate agency on various local, national and transnational levels. They express resistance to (1) political authoritarianism through critiquing the regimes of Bashar Al-Assad in Syria by highlighting the violence, human rights violations and persecutions exercised by the Assad regime over the years; (2) systematic discrimination expressed through social inequalities within the countries they have found refuge in; and finally (3) oversimplified representations of gender through public discourses depicting female Muslim refugees as victims that need to be rescued from patriarchal oppression.

The production of the artwork required refugee women to reflect deeper on their emotional and physical experiences to decide what and how to depict and present them through art. Women’s reflections on their own self, gender, family, society, religion, language and culture through visual language were additionally enabled through the use of augmented reality (AR). Refugee women utilized AR to further counter gender-based narratives of vulnerability and victimhood and to position themselves within public discourses on their own terms.12

4. Augmented Reality (AR) as Research Tool in Refugee Studies

4.1. Methodology

Augmented reality (AR) in a museum or exhibition setting is considered a powerful tool to connect the physical with the digital space. Through a quick response code (QR code), users can download the Artivive app on their tablets or smartphones, which gives them access to the AR content (see Figure 7).

Once the app is installed, users—and readers of this article alike—can scan the body map (Figure 1, Figure 5, Figure 6 and Figure 9) through the use of their cameras/tablets. The body map is recognized by the app through its designated coordinates, allowing the viewer to see and hear the inserted overlaying visual and audio information. The viewer is thereby invited into an augmented environment that enables invisible information to become visible.

AR as new technology proved to link viewers to other individuals and their communities (Gordon and de Souza e Silva 2011), work as a medium for knowledge transfer (Georgiou and Kyza 2018), improve mediation (Scholz and Duffy 2018), be a source of inspiration (Rauschnabel et al. 2019) and entertain and improve the perception of the overall viewing experience (Jung et al. 2016). Focusing on the visitor experience of the exhibition space and the function of AR as a communication or mediation tool, researchers have defined four categories of specific mediation goals—object annotation, object visualization, guiding and data visualization: “[…] visitors’ eyes were guided to details of the painting and paired with contextual information to trigger engagement with the work and ultimately support learning, contextualisation and understanding of cultural exhibits” (Ossmann et al. 2021, pp. 249–50).

In our project, we have used all four goals in order to invoke contextualization, engagement and understanding of the topics discussed. In the present study, AR is used on body maps for the first time in the context of refugee studies in academia. We argue that by providing refugee women with additional communication instruments or “voices” on different levels (visual, auditory, intellectual), AR also has the potential to be a research tool in ethnographic research. The use of AR in our project provides viewers with an augmented perception of the lived realities of refugees through the use of technology. Its functionality lies on a bodily and emotional level and leads to felt-bodily affectedness, which will be discussed below.

4.2. Content and Function of AR

At the start of this article, we cited an interview we conducted with a Syrian woman who yearns for her sofa as it symbolizes her former social life in Syria. This information is not visible on the body map itself. Only through operating the AR, the viewer understands the leaf patterns on the body of the woman in the bottom left corner. These patterns represent the woman’s memory of similar patterns she used to have on the fabric of her sofa in Syria. It is through the mediation of technology that an increase in information on both bodily and emotional levels occurs in the perception of the object. The object, represented through the body of the woman with leaf patterns, is embedded within its particular context, i.e., the sofa, that is yet to be seen through the operation of technology (see Figure 8).

Human lives are characterized by complex experiences and ‘networks of relationships’ (Orsi 2013) which cannot be presented as a linear and stable entities. Through the use of AR, we were able to construct layers of information and meaning to represent such complex lived realities experienced by refugees. The body map below is called The Cousins (see Figure 9). It depicts two cousins who fled from Syria and found refuge in Germany. The woman in red on the right had to leave Syria with her family because her husband was an anti-regime activist. She has two daughters and one son. One daughter fled with her husband to Jordan. She was resettled by the UNHCR to the UK, where she completed her degree in law (the Human Rights body map introduced earlier). The other daughter traveled with her family to Turkey. However, with a newborn child, she did not want to make the dangerous crossing to Greece on a dinghy and remains in Turkey to this day.

Together with her husband and son, the woman in red crossed to Greece on a small rubber dinghy before traveling overland to Germany. A few days after crossing the border to Germany, her husband died of a heart attack. The painted rubber dinghy on the body map is, through the use of AR, overlayered by recordings of actual refugees in real dinghies rescued by coast guards, which refugees provided us. The viewer is witnessing real rescue audio and visual recordings through AR (see Figure 10).

Such a sonic experience impacts viewers’ immediate environment and enables them to interact with an augmented environment perceived as real. Anthropological engagements with sonic experiences highlight the ability of sound, music and voice to define new understandings of intersubjectivity and social relations (LaBelle 2018; Williamson Fa 2022). AR challenges ocular-centric understandings of the social: the real rescue recording that is seen and heard through the AR contributes to connecting the viewer closer to the refugees. Modes of seeing and acoustic attending generate, thereby, new ways of viewing, listening and feeling.

The digitized sonic atmosphere influences the formation of socialities that are based on the notion of relatedness (Feld 2015, 2017). In our project, we have engaged in long-durée ethnographic fieldwork research with Syrian refugees in three countries: the United Kingdom, Germany and Jordan. Our meetings with women, in particular, were not limited to conversations but also to other activities such as cooking, eating, celebrating engagements and weddings, as well as painting. This diversity of engagements is reflected in the artwork and AR productions, which depict personalized refugee experiences such as cooking and eating and are easily related to contexts outside of the lived world of refugees.

The Cousins body map includes a scan of a food celebration—a picture taken by us during our dinner at the place of the woman in red. Looking at this body map, one would not necessarily expect to see ten steps of cooking instructions which then the AR provides. Parallel to the displacement narrative of the woman in red above, the viewer is introduced to a famous Syrian vegetarian dish called burning fingers (ḥarā’ ’uṣba‘o). The steps illustrated in the AR are all based on the cooking instructions we received during our time with the family in Berlin (see Figure 11).

Alternating between cooking instructions and refugees’ narratives of displacement, the viewer not only witnesses real-life rescue scenes but also takes part in Syrian’s everyday life through cooking. The AR starts as follows:

• ‘Preheat the oven to 400° Fahrenheit’—sound of the oven being turned on.
• ‘This painting portrays two cousins who came from Syria to Germany.’
• ‘Put lentils in a pot with water over medium high heat and bring to a simmer’—sound of boiling water.
• ‘This woman came with her sons and husband. They fled to Turkey as the husband was an anti-regime activist. They crossed to Greece on a small rubber dinghy, before travelling overland to Germany. A few days after crossing the border, her husband died of a heart attack’—sound of water.
• ‘Cut pita bread with a knife into small squares. Toss in olive oil and salt. Bake for 7–10 min until crispy’—sizzling sound caused by bread fried in oil.

Different strands of narrative but also variant visual and audial experiences are presented in this AR. Sonic expressions are positioned at the center of human experiences: the boiling water in the pot is contrasted to the cold seawater from which refugees are rescued. The sizzling sound of the hot oil is linked to the hypothermia wraps refugees are wrapped into after such water rescue actions. The different layers of narratives infused with multisensory experiences (Bruno and Pavani 2018) in the form of sound, music and voice help the viewer to build a relationship with the perceived images in the AR. Such relations connect experiences and generate within the viewer a form of ‘felt reality of relations’ (Massumi 2002, p. 16). The feeling of that relationship might be as significant as sharing the memory of displacement experiences itself. It could, however, also be small, demonstrated through the act of boiling water in a pot. In the final section of this article, such associations that are generated through AR will be discussed within the aforementioned notion of felt-bodily affectedness (leiblich-affektives Betroffensein, Schmitz [1969] 2019; Schmitz 2014a, p. 75).

4.3. Results: AR as a Despacialization Medium

Moving images, along with music, songs, poetry, and narration, but also religio-cultural specificities, such as the Islamic call for prayer (adhan) or the sound of the oud, are integrated into the AR. Such multisensory or multimodal expressions were partially provided and decided upon by the refugee women and partially by the production team.13 Such multisensory expressions allowed new possibilities for the reproduction of lived experiences by bringing to life the imagery found in the body maps on the canvases. At the same time, involving several senses leads to a deeper immersive experience for the viewer.

According to Schmitz, feelings are not subjective from the outset but rather present within a situation (Schmitz 2014a). Once someone enters a situation, one is captured (ergriffen) by these feelings, which only then become private and subjective. Schmitz continues by arguing that a situation is impacted by its surrounding (von Uexküll 2010, p. 43). In our context, it is the real immediate as well as the augmented surrounding brought to life through an augmented perception via AR. The digitized sonic atmosphere produced through AR constructs new environments that generate different moods (Stimmungen) and affects (Befindlichkeiten, Heidegger 1996, pp. 130–36) among the receivers of the artistic productions. Visitors to our exhibitions were diverse: some were refugees themselves or children or grandchildren of refugees not only from the Global South but also from Eastern or Central Europe. Whereas some were actively involved in refugee issues and had a genuine interest in supporting refugees, others were hostile towards refugees and supported anti-refugee policies.

The AR of The Cousins body map above, for example, generated a feeling of distress but also happiness among a viewer who was a refugee from Syria herself. Her feeling of relation towards the AR was articulated through her own narration of her displacement, explaining:

I also come from Syria and made the same journey as this woman here. I also used one of these dinghies. It was crowded and dangerous. It was awful. Watching this brought me back to my own memories. I remember I was holding the hand of another woman on the boat who I did not know. Holding her hand made me feel safe. It made me feel protected. On this boat, we all kept looking at each other as if to say ‘it will be fine. We will survive,’ but no one dared to say a word. We did not talk directly to each other. This experience today with the painting and this technology did the same: I see the woman’s body, her hand, her head similar to the woman who was next to me on the boat. I felt the woman on the painting talking to me but this time saying ‘we survived.’ It is a nice feeling. The woman in this painting survived. She made it. I made it. We are safe.

The degree of a felt-bodily affectedness (leiblich-affektives Betroffensein) depends on the extent to which the viewer is captured by the augmented perception. The particular viewer above was captured by AR to the degree that she extended the feelings generated through the AR to her real immediate environment: the woman held and squeezed my hand while watching the AR as it brought her back to her own lived memory of holding the hand of the woman next to her on the boat. This triangulation of the augmented perception, the own remembered past, and the now-exercised reaction illustrate how the boundaries between time and space have become porous through the viewers’ own affectedness. The atmosphere in which this woman felt generated a felt-bodily affectedness that created a momentum of self-being-there (Selbst-dabei-Sein, Buck 2019): on the boat with the woman depicted in the artwork holding the hands of the woman next to her. After this experience, the woman felt the urge to go to the artwork and place her hand on the hand of the woman depicted on the body map.

This example illustrates how the degree of engagement with AR can lead to highly visceral and emotional experiences that generate past-lived encounters. The real and the augmented worlds were despacialized. This is further illustrated through the sensation and perception of olfactory stimuli that were not present but imagined: ‘I smelled fried pita bread. Did you bring this with you as well?’ the woman asked while looking around her to see whether the dish described in the AR was present within her real direct environment—she was convinced she smelled the fried bread. The experience of the Syrian viewer above was filled with mixed emotions: She started shaking and crying while watching the AR but was also laughing as she enjoyed the fact that this woman made it to the shore and is safe now in Germany, just like her. The woman’s act of seeing and feeling was dwelling between the real/augmented and the material/immaterial.

The affectedness of a German viewer, based on the same AR generated an entirely different, however similar, memory of their own past and prediction of their future:

My parents came from East Germany. After the fall of the wall, they also left their homes and found refuge in West Germany. They did not have much money too. I can totally relate to this narrative through my own family history: No meat, only pasta (or rather potatoes) and vegetables were often on our plates as well.

The felt bodily affectedness of this German viewer was very similar to the Syrian woman above, who also started shaking when describing similar fears of war, destruction and uncertainty:

Now with Putin coming into Ukraine, war feels so close to us in Germany too. With the Syrian war, we felt it is far away. Something that will never befall upon us in Europe. But now it is on our doorsteps. We can feel the consequences of war ourselves already with the current recession and gas prices: Who knows if we can heat our homes this winter. We might also be shivering like those Syrians fished out of the cold sea.

AR has produced an experience of relatedness and understanding through the construction of shared encounters. The viewer was able to connect with the experiences of the refugee seen in the AR by relating them to her own experiences of moving from East to West Germany. The momentum of self-being there was not generated through the exact same shared experience as was the case with the Syrian viewer, although with related comparable encounters.

5. Conclusions

The concept of bringing art to life is utilized in our project through augmented reality (AR): individual and collective experiences of displacement are first expressed through the body maps, then contextualized and annotated digitally through AR. Refugees have played a central role in producing both the body maps as well as the AR. The multiple steps of production correspond to refugees’ multiple ways of remembering and reliving experiences. Their productions have become a form of documentary repository: the viewer is invited to engage in refugees’ thoughts, ideas, emotions and actions. Through these two tools, an emphasis is placed on the women’s visual voices and their relation to the women’s embodied experiences affected by their displacement journeys. The power of representation is granted to refugee women, as it allows them to co-create knowledge and articulate their narratives of displacement.

In this context, we argue that it is crucial to adopt creative strategies in refugee studies that evoke contextualization, engagement and understanding of the topics and stories centered around refugees’ biographic memories and lived experiences. For this reason, we introduced an analytical approach that reflects the following aspects that were discussed among the refugees, researchers, as well as artistic production team: first, refugees identified topics that relate to the biographies they want to convey. They made decisions about how to tell their stories on canvas and what imageries to use. Second, we discussed what visual means and techniques to include: zoom, highlights and overlaps to enable viewers to witness refugees’ living experiences that will enable them to view how the stories evolve through the paintings. Third, the production team decided on the audible means—such as noise, sound, music, adhan—whereby a mood is created, and its perception becomes a living experience that allows the viewer to be part of the emotions intrinsic to the narrative.

Such multisensory or multimodal expressions provide refugee women with new possibilities to reproduce lived experiences and emotions by bringing to life the imagery found in the body maps on the canvases. At the same time, involving several senses leads to a deeper immersive experience for the viewer. Such representations reduce distance, encourage reflection and ultimately transfer knowledge to the viewers. Of methodical interest is the convergence between lived and living experience and artistic form. We argue that due to this convergence, body mapping and AR can function as research tools to generate and communicate further knowledge. The strength of these methods lies in the co-production of knowledge with refugees and in forms of its mediation which may trigger responsiveness among viewers. These methods, in combination, seek to create an affect among viewers since the communication takes place on an emotional level with the help of sensory expressions that may evoke a form of self-experience among viewers.

The knowledge produced is articulated in our project in different ways through static tangible body maps and then through intangible digitized methods using AR. The viewers’ immediate environment and its spatial dimension are reshaped by various sonic and visual experiences. Such representations have an impact on the viewers’ atmosphere and mood (Stimmung or Befindlichkeit) not only in their felt (Leib) but also on their physical body (Körper) that are felt (and constructed) through their own reality of relations. Viewers perceive these experiences differently and, thereby, generate numerous, different, or similar emotions, reactions and memories. Viewed narratives of others are related to one’s own—contributing to a subjective and personalized understanding of forced migration. This is intensified through the use of AR, which enables viewers to virtually witness displacement and provides them with the feeling of self-being there. With such a process of relatedness and connectivity, viewers are able to identify with refugees’ experiences through their own variant narratives and state of affect.

AR proved to be a successful research tool that reinforces a link between viewers and the often isolated and marginalized refugees. AR becomes a tool for digitally mediated interaction with refugees who are able to move from the margin to the center of attention. This time, however, they are not the object of study per se but rather co-producers of knowledge. We have discussed AR as a research tool to question the act of viewing by illustrating multiple ways of seeing beyond the boundaries of space and time. The viewers’ reality is thereby characterized by an augmented perception through sensory aesthetic experiences. The use of aesthetic forms in research as a methodological tool thereby enhances our understanding of the diversity and complexity of human experiences—both of the other and of ourselves. It further provides an alternative view of forced transnational displacement and expands the type of knowledge produced about refugees with conventional ethnographic research methods.