Connecting the Elderly Using VR: A Novel Art-Driven Methodology

Abstract

Demographic change confronts us with an ever-increasing number of elderly people who face isolation and socialization issues. Background: The main challenge of this study is to inject emotional and aesthetic aspects into the design process of a virtual reality (VR) social space for the elderly. In this context, we asked architects and artists to improve the perception elderly people have of their way of communicating with others. Artists, in collaboration with computer engineers, designed experiences that evoke positive cognitive and emotional feelings and memories by following design trends and aesthetic values likely to be appreciated by older people, which were integrated in VR. Methods: We approached our goal by implementing an innovative art-driven methodology, using a plethora of technologies and methods, such as VR, artificial intelligence algorithms, visual analysis, and 3D mapping, in order to make design decisions based on a detailed understanding of the users’ preferences and collective behavior. Results: A so-called virtual village “Cap de Ballon” was co-created, having a public space inspired by the villages of Santorini and Meteora and a private space inspired by the 3D scanning of an elderly person’s apartment. Conclusions: The overall concept of the VR village‘s utility, design, and interior design were appreciated by the end users and the concept was evaluated as original and stimulating for creativity.

1. Introduction

Challenges posed by demographics are an important point of debate for our future. According to the 2020 report of the United Nations “World Population Ageing” [1], the number of persons over 65 in 2020 was estimated to be around 727 million worldwide. This number is predicted to double by 2050, reaching over 1.5 billion. The percentage of older people in the global population is expected to increase from 9.3% in 2020 to 16.0% in 2050, revealing that by mid-century, one in six people globally will be over 65 years old. Moreover, the number of persons over 80 years is probable to increase, rising from 137 million to 425 million on a global scale (reference period: 2017–2050). Among other consequences, as individuals will generally wish to stay in their homes for as long as possible, as they grow older and older, meaning that the home becomes a place where older people spend most of their time, their social activity will be undoubtedly affected. Consequently, the necessity to increase the social capital—the value that social networks and the types of reciprocity associated with them have—for the elderly becomes crucial [2]. A plethora of studies investigate the psychological impact of social isolation, stating that the anxiety levels of elderly people increase due to the lack of social interaction. As a result, they become lonely, bored, exhausted, and distressed, which increases their depression levels [3,4]. VR stands out as an emerging technology capable of instilling a sense of presence in a virtual environment, achieved through responsive visual and auditory cues that adapt to user actions, overcoming limitations of time and space [5]. Its versatile applications extend to various domains, including sports activities, simulations, and surgical techniques [6]. Notably, VR has found utility in rehabilitation efforts for individuals with diverse health conditions, such as post-stroke rehabilitation [7,8] and addressing cerebral palsy [9]. The application of VR in health contexts has garnered increased attention due to the time, space, and financial constraints associated with traditional rehabilitation methods conducted in healthcare facilities, often requiring the involvement of healthcare professionals [10]. For example, VR applications and headsets are being integrated into care homes to aid older individuals, particularly those dealing with Alzheimer’s, aiming to enhance overall mental health and evoke recollections [11,12,13].

In our study, we introduce art and its practices as a key factor to re-think and re-consider design potentials. Art offers the capability to imagine futures that do not just reflect the current consensus in mathematics, logic, and engineering but also set into motion new ways of seeing, hearing, touching, feeling, transforming, and experiencing spaces, places, and community. Regarding older people, more importantly, we needed to consider the overall symbolic experience by associating historical facts and memory recoveries to the experience, in order for them to be more emotionally connected due to the harmonization of their values and identity. For instance, the objects that are present in their homes may be more valuable to them because of the cultural, life events, and family memories they are associated with. This was taken into account in the design process of the private spaces provided in our virtual village.

Another important point in creating experiences through art and technology is by generating well-being for older people with strategies of “Interactive Art”—since in the co-creation procedures, the users are themselves actors who perform and create, beginning with a blank canvas, along with artists, architects, and software engineers—to stimulate their curiosity and self-expression and support their identity [14,15,16,17]. From the recent past, the Tunnel under the Atlantic [18] was an art installation that was completed in 1995. The tele-virtual project linked the Pompidou Centre in Paris and the Museum of Contemporary Art in Montreal. This project was more than just a technical performance; it was an intercontinental virtual reality artwork, referred to as “tele-virtuality” by Philippe Quéau in 1994. This installation was a unique example of what Maurice Benayoun calls “reactive architecture of communication”, and it was another way to explore the limits of communication after Hole in Space by Kit Galloway and Sherrie Rabinowitz [19]. In addition, over the last thirty years, numerous works of art have showcased the potential of VR as an expressive medium. Opting for VR as a form of artistic creation not only offers boundless creativity but also expands the scope of artistic interaction [20].

In general, there has been ample attention paid to the interaction of cognitive and emotional aspects of engaging with arts. During the last several decades, many studies have shown that experiencing and making art have many positive effects on individual well-being [21]. As art addresses the quest of purpose and meaning, experiencing art may forge social bonds. The positive effects of art experiences are produced by various mechanisms, such as being “carried away” by the art or being prompted to question worldviews and values [21]. Many studies point to positive neurological and physiological changes, such as lower levels of cortisol, a stress hormone, or galvanic skin response [22]. In addition, psychological competencies may improve, such as increased creativity or adaptation to circumstances [23,24] while other studies point out an increased self-reported (subjective) well-being.

Additionally, there are a plethora of studies that advocate how active engagement and physical involvement in a virtual environment (VE) contribute to improved long-term memory abilities, aligning with the enactment effect observed in various memory studies [25]. More specifically, the enactment effect suggests that individuals who physically engage in an action are more likely to remember the corresponding event compared to those who merely listen to verbal descriptions or observe someone else performing the task [26]. This phenomenon extends to various scenarios, such as actively rotating objects versus passively observing their rotation, resulting in a faster recognition speed [27]. Additionally, studies indicate that virtually manipulating body parts, as opposed to observing another individual performing the manipulation, enhances anatomical memory, especially benefiting individuals with lower spatial abilities at baseline [28].

When talking about the elderly, actively encoding information during virtual navigation [29] or engaging in physical activities like walking [30] and actively controlling VR navigation by deciding the itinerary [31,32] have been found to enhance distinctive memory traces, boost source memory, and improve episodic memory. Participating in such exercises that simulate daily life activities has been associated with improvements in visual memory, attention, and cognitive flexibility among older adults [33]. Additionally, even seemingly simple but enjoyable tasks have been linked to increased hippocampal gray volume in both older and younger adults [34,35].

Considering the above, we propose a new methodology in which artists and creative thinkers are invited to propose innovative, art-related ideas concerning the quality of elderly-friendly environments and the level of emotional and functional friendliness with an aim to address challenges of a sensitive audience, such as older people and their domestic isolation. In particular, artists showed interest in the process of memory formation and consolidation as well as the neural connections between toolmaking and communicating.

Consequently, the goal of our paper is twofold:

• Combine art and technology via AI and VR.
• Create a social virtual space for elderly people by proposing a novel art-driven methodology.

With the goal of combining art and technology via AI and VR, while relying strongly on visual data to monitor emotional states, artists and scientists explored the potentialities of art and technology to help preserve and improve neurological, cognitive, and emotional functions, also taking into consideration in their approach the crucial question of mental health. Art and technology representatives can merge and brainstorm on how to help people—by composing an artwork and/or an interactive installation that can function as a tool tailored to seek improvements in well-being.

In the following sections, works in the literature relevant to our study are presented and the methods and materials that led to the creation of a VR village for older people, “Cap de Ballon”, are discussed in detail. Section 4 presents the results of our study while the last section, Section 5, ends our paper with a discussion on the results and possible directions for future work.

2. Literature Review

A variety of VR experiences designed for the elderly have been explored in the literature. For instance, in [36], the authors assessed the effectiveness of 360° immersive VR interventions on the well-being of older adults with and without cognitive impairment. The review of 10 articles found that VR 360° video interventions were feasible, safe, and enjoyable for older adults in community or residential care settings.

In another study [37], researchers explored the use of head-mounted display (HMD) VR with individuals suffering from dementia. The evaluation involved interviews and reports, revealing that users were excited about the application and experienced increased pleasure during and after VR sessions compared to before exposure. Matsangidou et al. [38] proposed an experimental design to investigate the feasibility of using VR for rehabilitating patients with moderate to severe dementia. The authors reported the challenges faced during the design, development, and implementation of the experiment. Furthermore, in [39], authors presented a VR-based approach to address social isolation among elderly users.

Regarding VR technology acceptance among older adults, a study [40] investigated the use of VR as a tool for active aging. Thirty older adults used selected VR applications twice a week for six weeks and completed a questionnaire assessing their acceptance of VR technology. The results indicated that perceived usefulness, perceived ease of use, social norms, and perceived enjoyment significantly influenced their intention to use VR. The study concluded that older adults had positive perceptions of VR, considering it useful, easy to use, and enjoyable for active aging. Another study [41] explored the use of VR in engaging older adults residing in care facilities. The study involved residents and staff members who evaluated a VR system for two weeks, utilizing interviews, research notes, and video recordings. The study revealed that interactive VR technology’s usability was affected by the abilities of aged care residents, particularly those with dementia. Additionally, it identified that VR technology could engage older residents who might otherwise isolate themselves. Overall, the study emphasized the potential benefits of using VR in aged care while highlighting the need for design improvements to ensure effective utilization with older adults.

Furthermore, regarding the social aspect of VR, authors in [42] evaluated a novel social VR platform that connected older adults from different locations, enabling them to engage in virtual travel and activities together. The study suggested that VR social applications could facilitate social engagement among older adults.

On the commercial side, there is Rendever [43], a VR application that allows elderly individuals to immerse themselves in and explore virtual worlds using customized VR hardware and 360-degree videos. The acceptance of Rendever demonstrates the potential of VR for the elderly community, while also emphasizing the need to study and develop evaluation methodologies that best meet the needs of the elderly.

In this article, we propose an art-driven social VR platform implementation methodology, developed as part of a European project, which combines art, technology, AI, and VR, having a goal to preserve and enhance neurological, cognitive, and emotional functions while considering mental health. This social platform represents a novelty in the context of the existing literature on social platforms for elderly people. Artists can utilize interactive installations and design environments that incorporate VR, visual analysis, and AI algorithms to create spaces that elicit positive cognitive and emotional responses and memories. By tailoring these environments to the preferences and behaviors of the end users, artists can intellectually stimulate seniors, evoke positive emotions and memories, and encourage communication to address the emotional and cognitive needs of seniors.

3. Materials and Methods

3.1. Methodology

To address elderly people’s needs, artists used technologies developed or provided by the engineers in order to collect seniors’ topics of interests and create a useful VR tool for engagement with the goal of stimulating them intellectually, instigating further thoughts and communication. As already mentioned in the introduction of this paper, the VR public space for elders was developed with the strong involvement of artists, end users, architects, and software engineers throughout the design and development cycles, which by itself is an innovative design approach. The development methodology, which is depicted in Figure 1, included the following steps: (a) the creation of the first VR installation prototype, as a result of the requirement analysis and the synergy between the end users, artists, and software engineers. In this step, designers and artists were provided with 3D models of elderly people’s living spaces as well as information sourced from interviews about habits their homes, etc.; (b) the implementation of the first prototype; (c) the collection of measurements in order to define the most visited VR areas; (d) analysis of stress and sentiment of subjects in the most visited areas (VR space hotspots); (e) evaluation of prototypes by the subjects via questionnaires; (f) assessment of prototypes by the artists after user feedback and sentiment analysis that led to innovative design proposals, responding to innate user needs; (g) adaption and redesign of the VR installation after assessment to meet end users’ needs.

In essence, artists and creative thinkers were asked to suggest imaginative ideas pertaining to the quality of the virtual reality (VR) social environment and its emotional and functional user-friendliness. The artistic proposal aims to concentrate on themes such as the emotional well-being of seniors, their social connections, personal expression, and memory. Consequently, the VR environment can be viewed as a work of art in its own right, potentially assisting seniors in cultivating resilience and engaging in active social interactions.

In the following subsections, we provide a detailed description of the materials, methods, and data collection that supported our implementation.

3.2. Subjects

Human participants were involved in the project during the implementation and the evaluation of the different prototypes of the VR social environment. Our target group was comprised of ten (10) autonomous elders aged between 60 and 85 years living independently in Paris, France, and did not have particular health issues or mental illness. All participants were mandatorily over the age of majority and considered non-vulnerable.

Informed consent was obtained from all subjects involved in the study. Before conducting our interviews or evaluation/experimentation sessions, we made sure all participants signed informed consent forms, which materialized the decision of participation.

The provided documents were transparent, easy to comprehend, and ensured that participants had a clear understanding of data processing and procedures before engaging in the study. Our research team offered comprehensive explanations and addressed all inquiries from participants prior to their involvement in the research study. Detailed information regarding the collected data, their purpose, and the processes of data processing and storage was explicitly outlined in the consent information sheets distributed before each experimentation cycle.

3.3. Evaluation Approach

3.3.1. Prototype Questionnaire Evaluation

Qualitative interviews and questionnaires were conducted to better understand the feelings, opinions, and reactions of seniors in response to the proposed VR. In the context of the interviews, we employed a discussion and interview approach guided by the 8 + 1 dimensions of quality of life (https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Quality_of_life_indicators, accessed on 27 February 2024), as outlined by the European Commission. This framework served as the foundation for constructing the questionnaire designed to gather user needs (refer to

Table 1. Questions of the user needs collection phase.

1.  Introduction (only for interviews)	Could you tell me a little bit about yourself (where you were born, where you live, your family, your background)
2.  Interaction with art (only for interviews)	In what kind of place would you like to find an art installation that you could interact with? What do you expect from an art installation or a work of art? List three spaces (urban, professional, domestic) that are (in your opinion) focused on art. How do you think an art installation can make a positive contribution in social terms?
3.  Seniors’ domestic space and art (focus groups and interviews)	What do you think of your social environment? How do you feel at home? How do you feel in your town? How to fight against loneliness at home? Could interior design have an influence on the feeling of loneliness? What is your favourite room in your house and in which room do you spend the most time? (Ex: bedroom, living room, dining room, kitchen, bathroom, terrace, garden) What types of feelings/sensations do you have when you are at home? How would you redesign/modify your home to make you feel better and make it more welcoming and possibly age-appropriate? If we are talking about objects that interact with you and react to your movements/emotions. What support would be interesting to use (ex: TV, radio, fridge etc).
4.  Loneliness and social connections vs. isolation (only focus groups)	What are you doing in your free time? What are your hobbies? Do you consider you are isolated? What are the obstacles preventing a person to be socially active?

). Considering the holistic framework of the 8 + 1 dimensions of quality of life as defined by the European Commission, these interactions facilitated discussions spanning material living conditions, leisure activities, social interactions, physical safety, the natural and living environment, and the overall experience of life. Only the dimensions of governance and basic rights, as well as economic security, did not emerge as prominent themes within the conversational context.

We organized several experimentation sessions to test/evaluate the developed “Virtual Village”—a platform dedicated to increasing seniors’ social interaction and reducing social exclusion, which was named “Cap de Ballon”. The main idea of the sessions was to test the (1) usability, (2) emotions, (3) social interaction, and (4) creativity of the seniors throughout experiencing virtual reality. The subjects tested two versions of the platform: (1) the desktop version, which was evaluated in two experimentation sessions, and (2) the VR version, evaluated in one session. Three out of ten participants only navigated freely without evaluating the VR. The remaining seven (7) participants were informed about the objectives of the experimentation sessions, privacy, and personal data protection.

The following table (Table 1) outlines the open questions administered during the user needs collection phase, comprising two focus groups involving a total of 27 seniors in April 2019 and qualitative interviews conducted in May 2020. The primary objective was to discern the overall connection of seniors to art, in conjunction with an exploration of their living conditions and sentiments, both within the confines of their homes and in external urban environments. While the primary emphasis was on eliciting seniors’ expectations regarding art and interior design, the focus groups and, notably, the interviews provided valuable insights into critical facets of seniors’ daily lives and mental well-being.

Table 2. Cap de Ballon evaluation questionnaire.

Your gender	Female Male Other
Your age (in years)	Free text
How easy was it for you to perform the following tasks? - Navigating the village - Use emoticons - Navigate in the apartment - Use the microphone - Teleport to another user - Teleport to a window - Set a meeting at a meeting point - Watch pictures and videos in the windows	Not easy at all (1) Slightly easy (2) Moderately easy (3) Very easy (4) Extremely easy (5)
How much time did you spend performing the following tasks? - Navigating in the village - Using emoticons - Navigating in the apartment - Using the microphone - Teleporting to another user - Teleporting to the windows - Setting a meeting point - Watching pictures and videos in the windows	Very long (1) Long (2) Neither long not short (3) Short (4) Very short (5)
How often have you been interrupted by errors/bugs/while performing the following tasks? - Navigating in the village - Using emoticons - Navigating in the apartment - Using the microphone - Teleporting to another user - Teleporting to the windows - Setting a meeting point - Watching pictures and videos in the windows	Never (1) Rarely (2) Sometimes (3) Often (4) Always (5)
How satisfied were you with the following? - Village design - Interior design - Teleportation function - Speech function - Navigation function - Emoticons - Window function (watching videos and pictures)	Not satisfied at all (1) Slightly satisfied (2) Moderately satisfied (3) Very satisfied (4) Completely satisfied (5)
How fast did you get familiarized with the village?	Classification from 1 to 5 where 1 = Slow and 5 = Fast
To what extent did you feel in control of the system?	Not in control at all (1) Slightly in control (2) Moderately in control (3) Very in control (4) Extremely in control (5)
Did you feel the following while wearing the VR equipment? - Umcomfortable - Dizzy - Disoriented	Yes No I don’t know
Did you find the VR equipment easy to understand and use?	Yes No I don’t know
To what extent do you agree to the following: - I can understand well the organisation of the village (e.g., neighbourhood, arrangements of the objects, etc.) - I can understand well the organisation of the apartment (e.g., rooms, arrangements of the objects, etc.)	Strongly disagree (1) Disagree (2) Neither agree nor disagree (3) Agree (4) Strongly agree (5)
Which landmarks did you find easier to use?	Monkey Balloons Bikes Golf cart Ice cream cart Other I don’t know
What kind of window content did you prefer the most?	Videos Pictures Slide shows I don’t have a preference
What is your opinion on having changing window sizes? Please check all that apply	I found it useful I didn’t fi nd it useful I found it confusing I don’t have an opinion about it I didn’t realize the window sizes changed Free text
Which neighbourhood did you feel attracted to?	Yellow Blue Petrol Green Appartment (interior design) All None I don’t know
Why did this specific neighbourhood attract you?	Free text
How did you feel regarding possibilities of social connection?	Very isolated from others (1) Isolated from others (2) Neither isolated nor connected to others (3) Connected to others (4) Very connected to others (5)
How pleasant was the following: - Navigating in the village - Watching the windows - Interacting with the village elements(landmarks) - Interacting with others - Navigating in the apartment (interior design)	Pleasant Neutral No Pleasant
How intense was your feeling while: - Navigating in the village - Watching the windows - Interacting with the village elements(landmarks) - Interacting with others - Navigating in the apartment (interior design)	High Neutral Low
Did the interaction with village triggered your memories?	Yes No I don’t know
If yes, what kind of memories?	Positive life experience Negative life experience
To what extent do you agree to the following: - The concept is original - The concept is useful to fight social isolation - The concept is useful to stimulate creativity	Strongly disagree (1) Disagree (2) Neither agree nor disagree (3) Agree (4) Strongly agree (5)
Would you like to invite your family and friends to the village?	Yes No I don’t know
How would you rate your overall village experience?	Not at all satisfied (1) Slightly satisfied (2) Moderately satisfied (3) Very satisfied (4) Completely satisfied (5)
What are your suggestions for improvement of the MindSpaces “virtual village”?

outlines the survey questions used to assess the experience of senior users while navigating Cap de Ballon either with a VR headset or in the desktop version. The evaluation targeted effectiveness metrics, ease of use, comprehension, system familiarity, and user satisfaction. Responses were quantified using the Likert scale.

After each experimentation session, the users provided their evaluations via questionnaires, which were processed in order to be anonymized and then analyzed in order to receive the evaluation feedback from our subjects. Personal identification data were not used during the evaluation process. No psychological or emotional analyses were conducted during the sessions. Taking of photos and video materials was agreed upon with the senior participants in a form of consent forms. The photo/video materials gathered follow the GDPR requirements. The consent forms will be stored for a period of 5 years after the end of the study.

3.3.2. Image Sentiment Analysis and Behavioral Analysis

Leveraging image sentiment analysis modules can enable the development of multimodal systems capable of assessing human emotional responses within 3D spaces, influencing architectural design principles. The inherent subjectivity governing human emotions introduces risks in sentiment extraction tasks [44]. The utilization of Convolutional Neural Networks (CNNs) proves to be an effective method for extracting a viewer’s emotional stimuli [45,46,47,48]. The literature has presented various approaches to sentiment expression, categorized primarily into two models for emotion modeling: the Dimensional Model and the Categorical Model [45].

The Dimensional Model conceptualizes emotions as points in a 2D or 3D space, defined by three basic dimensions—valence, arousal, and control. Given the minimal impact of the control dimension (dominance), the existing literature primarily focuses on valence and arousal (Figure 2). Some studies solely consider the valence dimension, predicting sentiment polarity in terms of two levels (positive, negative), three levels (positive, neutral, negative), or more, as demonstrated in [49], where a five-level polarity approach is adopted.

Contrastingly, the Categorical Model identifies specific emotions such as “anger” or “fear”, which can also be mapped within the valence–arousal–control space [50]. Numerous studies explore emotion categories, incorporating frameworks like Plutchik’s Wheel of Emotions [51] and Ekman’s theory [52]. In this study, we embrace the Dimensional Model, considering both valence and arousal dimensions. We employ a three-level sentiment polarity, with three classes for valence (positive, neutral, negative) and arousal (excited, neutral, calm), as illustrated in Figure 2. The methodology proposed in [53] is adopted to assess user sentiment while navigating the VR platform.

Figure 2. Two-dimensional valence–arousal representation [54], based on [55].

Figure 2. Two-dimensional valence–arousal representation [54], based on [55].

For this study, a behavioral data analysis module for stress detection was also implemented. The method is an adaptation of the method presented and validated in [56], where the authors exploit motion trajectory data for stress detection of office workers. The analysis of behavioral data involves a two-step process: feature extraction and the application of a Hidden Markov Model (HMM) to derive stress levels from user positional data. Four distinct features are extracted from the positional data, namely moving time, track spread, wandering style, and hotspot. Moving time represents the total duration the user spends moving above a specific speed threshold. Track spread quantifies the overall distance of each point in the user’s trajectory from the trajectory’s center. For the next two features, we establish square cells with a side length of 0.1 m to analyze movement. Wandering style is determined by dividing the number of unique cells in the trajectory by the total covered distance. The hotspot feature measures the distance from the trajectory’s center to the most frequently visited cell. By utilizing these features and inputting them into an HMM, we calculate stress levels based on the user’s positional data.

After acquiring the most visited cells, screenshots were taken from the cell in every direction, and the screenshots were processed from the visual analysis tool [49]. The results from the visual analysis were fused with those of the behavioral analysis, resulting in a fused sentiment (Figure 3). The data fusion component is a service we implemented (see Figure 3) that receives and combines the outputs of the image sentiment components and the behavioral navigation data. At the first step, image sentiment analysis is performed. The behavioral data are also analyzed to predict stress levels from the navigation trajectory patterns. The stress levels from the behavioral data are finally combined with the results from the image sentiment analysis using weighted average methods, which involves the weighted average of the sentiment analysis and the behavioral data-derived stress level, using a 90/10 ratio.

The result from the fusion component is also accompanied with the position information of the subject in the VR space.

3.4. Methods and Tools

3.4.1. Requirements of Focus Groups and Interviews

During the phase of gathering user requirements, our goal was to gain a deeper understanding of the expectations, technical specifications, and potential usage of the virtual village by elderly individuals. To achieve this, we organized two focus group discussions involving a total of 27 participants. We formed two separate discussion groups, with each session lasting for two hours. These discussions allowed us to obtain valuable insights regarding the elderly’s perspectives on loneliness, their experiences within their own homes, and the various factors that could be improved to enhance their well-being. To further delve into the research on user requirements, we conducted interviews using the Zoom (https://zoom.us/, accessed on 28 February 2024) platform. Through these virtual meetings, we addressed the same topics as in the focus groups but on an individual basis with senior participants. This approach provided a more comfortable environment for the elderly to express their thoughts and feelings about art, while enabling researchers to gather more pertinent data.

The examination of user requirements carried out centered on the well-being of older adults and how the findings of this study could contribute to its enhancement. The addressed societal needs were primarily associated with fostering social interaction, alleviating loneliness, and promoting the recollection of positive memories. By collecting data through focus groups and interviews, the research team comprising engineers, architects, and artists gained insights that led to the conclusion that stimulating the memory and social connections of older adults was of significant importance.

More specifically, subjects’ expectations regarding the VR platform were gathered during an online workshop organized on Zoom. Seniors had heterogeneous profiles, different attitudes to social media, and various expectations. Nevertheless, all of them were open to a new experience. In terms of user requirements for the platform, we can underline the following:

• There were no specific preferences regarding platform device or design. The general demand was to keep the platform environment “warm, cosy and welcoming”.
• Senior users would like to be able to navigate freely and visit different spaces (rooms).
• Content format should not be restricted. They should be able to share photos, videos, texts, and music either produced by them or selected by them. They should also be able to share just their impressions on something (an exhibit, a book, etc.).
• The platform should be interactive. Seniors gave the example of the application “Meet up” (https://www.meetup.com/, accessed on 28 February 2024) where you build online groups and host in-person and virtual events for people sharing the same interest. We could imagine meeting up in different thematic rooms with decorations matching the general theme.
• Some of the subjects appreciated the idea of having an avatar and a personal identity in the platform, for instance, one subject expressed the will to be “the couch historian” or another one “the garden hiker”. Their online identity would be connected to their hobbies and real identities.
• The subjects considered that the online space did not allow for expressing nuanced emotions such as in face-to-face and “live” encounters. Emojis could help, but online communication is not subtle enough. Talking, seeing people face-to-face, or using video chat allow users to share more emotions and capture more nuances.
• The subjects were more reluctant in producing and sharing personal content, but this could be addressed via the platform’s interactivity and restricted access, which would need an adhesion to the platform community and group.

3.4.2. Interiors Sensing for Three-Dimensional Reconstruction

The presented version of the social village outlined in this paper does not prominently integrate the outcomes of the interiors sensing phase in our methodology. Nevertheless, we consider it crucial to incorporate these results as they unveil a fundamental concept behind the initial inception—creating a social village where each house is distinct and notably connected to a participant engaged in social interactions within it.

Due to the COVID-19 pandemic, we were able to conduct a data collection mission in the apartment of only one of our participants, aiming to generate a 3D representation of the apartment and capture small objects and furniture of significant importance to the senior resident. These elements were intended to be incorporated into the design of the virtual reality (VR) space. To elaborate, this concept could be expanded to scan each participant’s actual living space, incorporating recognizable items into their digital homes within the village. The intention is to assess the influence of a sense of familiarity on the overall experience.

Subsequently, artists, designers, and architects proposed renovations and enhancements to create emotionally and functionally senior-friendly private/domestic environments. They also focused on designing objects and spaces that would evoke positive cognitive and emotional experiences and memories, aligning with design trends and aesthetic values that are likely to be appreciated by older individuals residing in such environments. Throughout this process, the analysis of user requirements served as the foundation for all design decisions.

The objective here was to reconstruct geometrically and visually accurate—highly detailed—3D models of indoor environments, to use them in the context of emotionally relevant architectural redesign of spaces. Therefore, addressing the need for a fast and streamlined creation of 3D models for architecture, engineering, and construction processes involved in the proposed art-driven framework for the design of an adaptive VR social space, a mobile mapping platform was developed. This prototype mobile mapping platform mainly consisted of an array of cameras, LIDAR, and a processing unit and could operate in outdoor and indoor environments. During the project, the platform itself and the algorithms used to reconstruct the 3D world from images and sparse point clouds were developed. Furthermore, a workflow was developed to generate parametric/semantically rich 3D models from photogrammetric mesh models using CAD (https://www.rhino3d.com, accessed on 28 February 2024) (Rhino v6) software. As part of the advances, a novel, user-friendly (Robot Operating System) ROS toolbox for camera–LIDAR geometric and temporal calibration was also developed (http://wiki.ros.org/cam2lidar, accessed on 28 February 2024). The library is open and free to access in a public repository (http://wiki.ros.org/cam2lidar/Tutorials/How%20to%20calibrate%20Lidar%20and%20Camera, accessed on 28 February 2024) under the Apache 2.0 license (https://www.apache.org/licenses/LICENSE-2.0, accessed on 28 February 2024). Key contributions in the development of the mobile mapping platform included system geometric calibration using a novel algorithm for temporal calibration to estimate small mis-synchronizations between sensors: c REF _Ref109387537\h camera—LIDAR calibration (Figure 4).

This part of the implementation procedure included (i) the selection and identification of chessboard patterns for camera calibration targets with reflective tape for camera–LIDAR calibration, (ii) multi-camera calibration based on bundle adjustment optimization, (iii) custom algorithms for camera–LIDAR calibration, (iv) a custom reflective target with an Apriltag pattern, designed and detected on the image space, (v) detecting the center of the reflective target in the LIDAR space, (vi) selecting static frames based on target movement thresholds, (vii) developing a robust Perspective-n-Point algorithm to estimate the displacements and orientation between cameras and the LIDAR sensor, (viii) nonlinear optimization to obtain the final calibration parameters, and (ix) a transformation tree—“tf-tree”—which was estimated for use in the ROS-compatible 3D reconstruction module.

The data collection included 20 individual scans of ~3 mm resolution with the Faro Focus (https://www.faro.com/en/Products/Hardware/Focus-Laser-Scanners, accessed on 28 February 2024) 3D terrestrial laser scanner. A survey was carried out in the living room with the first beta version of the mobile mapping platform using two sensors:

• The Intel RealSense T265 Tracking Camera (https://www.intelrealsense.com/tracking-camera-t265/, accessed on 28 February 2024) for relative positioning.
• The Microsoft^® Kinect v.2 Depth Camera (https://learn.microsoft.com/en-us/azure/kinect-dk/depth-camera, accessed on 28 February 2024) to acquire depth images and point clouds.

A photographic survey of small objects and selected furniture was also performed to provide visual data for 3D reconstruction by means of multi-view and single-image approaches. A quality assessment of all acquired data was performed in the field, after each photo capture, 3D scanning, and mobile mapping mission. Finally, the reconstructed 3D models were exploited to manually generate parametric/semantically rich 3D CAD (Rhino) models from photogrammetric mesh models (i.e., Figure 5). The results of the tasks supporting this objective were the 3D models supporting our VR environment.

3D models of specific pieces of furniture were also created from the data collection with the mobile mapping platform. Multiple depth maps captured with a Microsoft Kinect depth sensor were converted to point clouds. These were then registered and combined into a single 3D mesh model. The triangular mesh was then textured from RGB images taken with a separate digital camera.

3.4.3. Exterior Village Design

The exterior of the virtual village was designed using CAD software (23.0) (Rhinoceros). As a starting point, a part of an open-source photogrammetry model (https://sketchfab.com/3d-models/meteora-greece-816105d6666843369c1af8e5427e56cf, accessed on 28 February 2024) of the Meteora rocky context was used. A small part of a specific rock from the whole complex was cut out then duplicated, rotated, and joined in order to create the desired inclination and landscape.

The main road running through the village, as well as the smaller ones leading to specific neighborhoods, were designed from scratch using meshes. The houses have simple parallelogram or trapezoid floorplans and span from 3 m to 9 m in height while all houses have a 0.5 m high parapet. To create the houses, a small-scale master plan of the village was created in 2D, mostly organized around the village’s main road system while also creating neighborhoods. Those 2D lines were later projected onto the 3D mesh of the ground and then extruded to the desired height. The ground mesh was further manipulated in order to accommodate the houses, so inclinations and slopes were changed or created based on specific needs.

All houses had windows on them; the windows consisted of two mesh shutters and a simple mesh plane in the middle so the content could be displayed. All mesh planes had their pivot points in the middle so that the windows were able to be scaled proportionally from the midpoint outwards. All materials aside from the ground mesh were created using Rhinoceros 3D material creation capabilities. Lastly, there are several scattered items in the village called landmarks that consist of monkeys, an ice-cream van, balloons, etc. Those items were obtained from open-source 3D object libraries and were used either directly or after specific manipulations, according to the artists’ suggestions.

3.4.4. VR Tool

Unity (https://unity.com/, accessed on 28 February 2024) was used to create the VR social platform and GRPC was used for communication. Before importing the 3D model into the VR Tool, an optimization procedure was followed because the obtained 3D mesh models from the 3D reconstruction techniques employed were often too complicated and complex to be handled directly in real-time VR applications. They consisted of millions of triangles and multiple texture files were assigned to the geometry to depict all the visual details. To overcome this, a typical workflow from the Computer Graphics Community is adopted, which includes:

• Mesh simplification.
• Physically Based Rendering (PBR) Texture Map Baking.
• Level-Of-Detail (LOD) generation.

To streamline the geometry of the reconstructed 3D mesh models, we utilized the “Instant Meshes” tool [57]. This tool employed a local-based approach to re-mesh the original 3D surface into an isotropic triangular or quad mesh. This approach optimized both edge orientations and vertex positions in the resulting mesh. The initial step involved computing an orientation field, defining the directions to which the edges of the simplified mesh should align. The subsequent step computed a local UV parameterization, where the gradient aligned with the orientation field and exhibited discontinuity over edges. Ultimately, a 3D triangular or quad mesh was generated from these two fields. This process resulted in a simplified version of the original 3D mesh model, comprising a reduced number of triangles. This is usually referred to as a “low-poly” model.

Briefly, the architecture of the VR tool can be described by three main components:

• The VR headsets: These headsets support the software to view and interact inside a virtual environment.
• The VR meeting room server: this server is responsible for creating a collaborative environment for the users using the VR headsets and populating it with the user’s personal digital artifacts.
• The database: the database houses the functionality to allow user authentication as well as the personal digital artifacts of the users.

The collaborative VR application was designed for multiple users to immerse themselves into a single VR environment and collaborate as well as view and manipulate objects inside the VR environment. The VR application was built using Steam VR to provide maximum compatibility with VR devices. The application allowed users to log into their user accounts and create meeting rooms, which other users could join. The users were able to join the session in VR as well as on desktops. In fact, the first two prototypes of the study were evaluated as desktop versions, while the last prototype was evaluated using a VR headset. A “VR Ready” laptop successfully operated the tool. Furthermore, the VR meeting room server had the capacity to support a multiplayer server, enabling concurrent interaction among multiple users. This server was responsible for populating the VR environment with digital artifacts sourced from a backend database. The VR meeting room server was hosted on a dedicated server, facilitating the construction and execution of the service.

3.4.5. System Requirements

The VR platform can operate under the following system requirements:

• Operating system: Windows 10 Professional.
• CPU: i7.
• RAM: minimum 16 GB.
• Disk Space: 10 GB.
• Graphic card: Minimum NVidia GTX 1060 6 GB VRAM.
• Others:

  -

  Oculus Rift or Oculus Rift S attached.

  -

  The system should have at least one HDMI port and one USB 3.0 port.

4. Results

The collaborative VR application, named by the users “Cap de Ballon”, was designed for multiple users to immerse themselves into a single VR environment with the possibility to collaborate as well as view and manipulate objects inside the VR environment. The VR application was built using Steam VR to provide maximum compatibility with VR devices. The application allows users to log into their user accounts and create meeting rooms which other users can join.

4.1. Cap de Ballon VR

The VR world was inspired by Maurice Benayoun’s project Last Life [17,18] conceived in 2008 and exhibited online and in a show in 2009. This metaverse project was based on the economy of attention model. A city-like environment provided virtual citizens with the possibility to settle in a specific home and open windows to the world to share their life. Stopping in front of a window, passers-by could watch users’ created content. The size of the window was determined by the cumulated attention, stimulating citizens to improve their content. While Last Life was an ironic response to Second Life, “Cap de Ballon” addressed very different issues, i.e., how to increase seniors’ level of sociability by inviting them to value their life and sharing the best in it.

In general, “Cap de Ballon” is a 3D VR village that encompasses elements of several real-world places. The main goal of the village is to stimulate seniors’ memory and improve social connection by bringing seniors and their friends and family into a platform where they can share content of common interests and interact with each other (Figure 6).

The VR village, built upon a large-scale rocky context, is surrounded by water and has whitewashed houses that reference Greek Cycladic architecture. All houses were organized around a main road that ran through the whole village. Different neighborhoods exist in the village corresponding to different themes selected by seniors and are organized into four thematic categories: (1) art and architecture and cinema, (2) nature, (3) health, food, and physical activity, and (4) events (history, memory, family, local life). Each neighborhood can be distinguished from one another by color. Each house has different colored windows and parapets according to the neighborhood theme they belonged to. Additionally, big balloons fly above each neighborhood, so that users can easily identify the neighborhood they prefer to visit from a distance.

Content produced by seniors such as videos, pictures, and slideshows are displayed on the windows of the houses (Figure 7). The longer a viewer looks at a window, the more the window enlarges, in response to the viewer’s attention. Visitors can also find several items, named ”landmarks”, scattered in the village, such as monkeys, ice-cream cart, golf cart etc., which are used to signal entrances or exterior meeting places. Furthermore, there is a clear distinction between outside and inside spaces. In the outside space, seniors can explore and navigate through the village, visit different neighborhoods and themes, and watch different content. In interior spaces, one per neighborhood, seniors can gather in smaller groups and discuss specific topics. Seniors themselves contributing to the co-creation aspect of the village required visual elements such as ice-cream carts or the use of emoticons.

Using a VR platform, seniors were able to explore the village they had co-designed with artists, software engineers, and architects. The interiors they could enter were decorated with objects scanned from one of our subjects’ homes along with new objects created during the redesign of the apartment by architects. The platform provided the possibility to the seniors to communicate with others inside the VR environment. Moreover, seniors could teleport to other windows and neighborhoods. To express their emotions, they could use emoticons and were able to interact with a variety of objects (e.g., balls).

4.2. Cap de Ballon: Seniors’ Creativity and Attention Span

One important aspect of “Cap de Ballon” was to be a support for seniors’ own creative processes. As we have stated above, creativity and art are precious tools for health and well-being at all ages and especially at an older age.

We wanted seniors to be actors of the village and not only spectators that passively absorb images and content. Valuing their creativity and their power of creation was, therefore, a key factor. We accompanied seniors in the creation of videos and pictures with their smartphones, enhancing their use of new technologies for self-expression. Each senior sent their individually produced content in the format of pictures and videos taken by them illustrating their daily lives, special events, or holidays. Some of them also produced compilations of content around the same theme. The connection with nature, the exploration of cultural elements, and hobbies shared with their own families were recurrent thematic categories. Such videos and pictures were displayed in the village windows as explained above. Other than creativity, valuing individual attention span was a central feature of the village. The longer the senior looked at the content shown in a windows, the bigger the window became.

The sound of the videos was also integrated into the village environment so we can say that the contribution that seniors provided to the village creation was even acoustic. All these elements enhanced the co-created aspect of the village and contributed to the satisfaction and adherence to the concept by seniors.

4.3. Cap de Ballon VR Evaluation

After the two evaluation experimentation sessions (see Figure 8a–c) of the first two prototypes of the village, which were tested in a desktop environment, we concluded with a final prototype, the VR one, shown in the previous section, also evaluated by our test group of seniors. In these sessions, we were also interested in grasping the main feelings of our subjects and impressions when navigating in the village. The evaluation questionnaires included elements related to social connections, pleasantness, and memory.

The scope of the questionnaires was twofold: (1) to gain a general evaluation of the village’s concept including its originality and its utility to fight loneliness and stimulate creativity and (2) to evaluate the level of satisfaction regarding seniors’ experience in the village. After the last evaluation phase, regarding the VR version of the virtual village, we asked the participants to state using a 5-level Likert scale (1—strongly disagree to 5—strongly agree) if the use of the virtual village would ameliorate the quality of their daily life from a social perspective. More specifically, we wanted to use their normal everyday life as a baseline to observe if the introduction of a VR environment like the one we proposed would make them more socially extrovert. The results are represented in Figure 9 and Figure 10.

Regarding the first, 71% of senior testers strongly agreed and 29% agreed that the village stimulated creativity. A total of 57% of senior testers agreed that the village stimulated social connection between seniors, and 43% affirmed that the village has this potential provided seniors understand well its functionalities and are more comfortable with its usage. A total of 57% of senior testers agreed that the concept of the village is original, while 43% of them strongly agreed. A total of 71% of seniors would invite their family and friends to join the village, and 29% of seniors would recommend the village provided their entourage is familiar with VR. A total of 72% of seniors found the VR easy to use, i.e., there was no dizziness, and only 29% felt some kind of discomfort but 43% felt disoriented. For 43% of seniors, the village triggered direct memories and for 29%, the village could trigger additional memories provided they were able to publish and explore more content. For 86%, the memories triggered were positive. A total of 43% of seniors were completely satisfied or very satisfied with their experience while 57% were moderately and slightly satisfied.

In the following figure and table (Figure 9,

Table 3. Desktop vs. VR evaluation.

Question	Desktop	VR
Originality	86%	100%
Stimulate Creativity	43%	100%
Fight social isolation	57%	57%

), you can see some comparative results between the desktop and VR versions of the village.

Finally, the subjects experienced no dizziness during the VR experimentation session. More specifically, most of the subjects felt comfortable (71%) while some felt disoriented (43%). In general, after appropriate explanations and practice, the VR version of the social platform was easy to use for most of the seniors (72%).

Valence and Arousal

In order to assess the VR environment regarding the sentiment of its use evoked in our subjects during the experimentation phase, we implemented the procedure shown in Figure 5. Arousal (or intensity) is the level of activation that an experience creates and ranges from calm (or low) to normal (medium) and excited (or high). On the other hand, valence is the degree of pleasantness that an experience generates and is defined from negative to positive.

In our experiments results values between [0,0.33) are equivalent to negative/calm valence/arousal, values from [0.33,0.66) are equivalent to neutral valence/arousal, and values in the interval [0.66,1] correspond to positive/excited valence and arousal correspondingly.

Figure 11a–e describes the valence and arousal scores of a single subject in each of the five points he visited more during the experiment. There are also screenshots taken from each point facing four different orientations. Along with that, an interpretation of the valence and arousal scores is also provided, in order to better understand the different scores. The corresponding values for the valence and arousal classes are also given. On the other hand, Figure 12 represents the mean values of valence and fused arousal per subject over the five (5) most visited hotspots. From the representation of valence and arousal in Figure 12, we are able to observe that our subjects, in reference to the most visited places (hotspots), were, in general, activated (subjects 1, 3–5, 7–10) and felt calm during the exploration of the village and also felt pleasant (subjects 2–3, 5–9).

5. Discussion

This article introduces an innovative approach for creating a social virtual reality (VR) platform that seamlessly blends art, technology, artificial intelligence (AI), and VR. Developed as part of a European project, the methodology is designed to safeguard and improve neurological, cognitive, and emotional functions, with a particular emphasis on promoting mental health. Our goal was to create spaces that evoked positive cognitive and emotional responses and memories for seniors. By tailoring these environments to the preferences and behaviors of end users, artists aimed to intellectually stimulate seniors, foster positive emotions and memories, and encourage communication to address the emotional and cognitive needs of the elderly population. With the aspiration to inject emotional and aesthetic aspects into the interior design process, we asked architects and artists to improve the perception seniors had of their home environments and their way of communicating with others.

In collaboration with computer engineers, design experiences that evoked positive cognitive and emotional feelings and memories were applied, by following design trends and aesthetic values likely to be appreciated by the users living there. With the use of VR, visual analysis, AI algorithms, and simulations, design decisions were taken in relation to a detailed understanding of the users’ preferences and collective behavior, stimulating them intellectually and instigating further their communication with others that addressed their emotional and cognitive needs. Increased independence and creativity, to counteract both loneliness and possible lack of daily stimuli of older citizens, was expected as a result. The proposed co-created interdisciplinary methodology of a VR social environment for the elderly, with the support of artists, allowed for a collaborative and inclusive approach to the design and development of the proposed multiuser virtual environment with synchronous interaction among the users.

This approach ensured that the final output was tailored to the specific needs and interests of the end users and led to the development of an art-driven social virtual reality (VR) environment specifically designed for seniors. Additionally, we established a co-creation design methodology, aimed at appealing to seniors and enhancing their sense of independence, comfort, and social connection with friends, relatives, and peers, ultimately improving their overall well-being. The societal benefits of improved social contact are expected to extend to caregivers, family members, and social circles, resulting in enhanced quality time spent together. Several technologies, including AI, VR, visual sentiment analysis, and 3D model design, were presented within the context of this study. Unlike the existing literature on social platforms for seniors, this social VR platform represents a unique contribution. Artists play a key role, utilizing interactive installations and designing VR environments enriched with visual analysis and AI algorithms.

The literature extensively explores various VR applications for the elderly, emphasizing their potential benefits. Studies, such as [36,58,59], highlight the positive impact of VR interventions on the well-being of older adults. The feasibility, safety, and enjoyment reported in community or residential care settings suggest the potential for broader applications. Moreover, investigations into head-mounted display (HMD) VR, particularly in individuals with dementia, as conducted in [37], provide insights into the heightened pleasure experienced during and after VR sessions. This aligns with our findings, indicating the positive emotional and cognitive responses evoked by our social VR platform among seniors.

Challenges faced during the experimental design for rehabilitating patients with moderate to severe dementia, as outlined by Matsangidou et al. [38], underscore the importance of addressing specific group needs, reinforcing the necessity for tailored VR solutions, such as the one we propose. Moreover, addressing social isolation through VR among elderly users, approaches such as those of [39,60] align with our objective of enhancing social connection among seniors through our VR social platform. Understanding the social impact of VR is crucial, and our interdisciplinary methodology, involving artists and computer engineers, aims to address this aspect comprehensively.

Investigations into VR technology acceptance among older adults, such as the studies by Syed-Abdul et al. [40] and Shao and Lee [61], provide valuable insights into the factors influencing intention to use VR. Our implementation aligns with their conclusions, emphasizing the importance of perceived usefulness, ease of use, social norms, and perceived enjoyment.

In the context of enhancing social engagement through VR, Kalantari et al. [42] explored a novel social VR platform connecting older adults, indicating the potential of VR applications to facilitate social engagement. Our social VR platform contributes to this discourse, emphasizing the positive impact on seniors’ social connections and well-being. Furthermore, in regards to the commercial application Rendever [43], our work aligns with the acceptance observed in Rendever, underlining the broader acceptance and potential of VR for enhancing the lives of older individuals.

This approach emphasizes the role of artists in creating environments that evoke positive cognitive and emotional responses through an interdisciplinary co-creation methodology involving computer engineers and designers, setting a precedent for tailoring social VR environments to the specific needs and interests of seniors. This approach also ensures a user-centric design that enhances independence, comfort, and social connection, ultimately improving overall well-being.

6. Conclusions

In conclusion, this research study observed a heightened willingness among seniors to engage with VR and modern technologies. The research team noted their desire to create their own content, which was subsequently incorporated into the virtual village’s outdoor elements. Participants expressed motivation derived from the interactive and socializing aspects of the VR environment, feeling both creative and explorative. Image sentiment analysis was employed to assess their sentiments, revealing high levels of acceptance.

More specifically, the assessment considered aspects such as social connection, pleasantness, memory stimulation, and overall satisfaction. The majority of senior testers expressed strong agreement regarding the village’s ability to stimulate creativity (71%), its originality (57% strongly agreed, 43% agreed), and its potential to enhance social connections (57%). The results also indicated positive sentiments towards the ease of use of the VR platform, with 72% finding it easy to use and none reporting dizziness during the experimentation session. This study further delved into valence and arousal, revealing that subjects generally felt activated and calm during the exploration of the virtual village. These findings suggest promising prospects for the virtual village concept, highlighting its potential to positively impact seniors’ daily lives from both a social and emotional perspective.

It is important to acknowledge that our proposed methodology has limitations and potential threats to validity, including a small participant sample size and the absence of advanced statistical inference for comparing different evaluation stages. As part of future work, we plan to address these limitations by increasing the number of participants and incorporating additional techniques, such as physiological signal processing algorithms, to detect users’ emotional states. This will optimize the adaptiveness of the VR environment by measuring physiological signals like galvanic skin response (GSR). Furthermore, real-time adaptation of the VR environment based on users’ navigation behavior will be achieved through visual sentiment analysis.