Three-Dimensional Creation and Physical Movement in Art Therapy Using Virtual Reality Painting ^†

Abstract

Virtual Reality (VR) painting, an emerging form of artistic expression under 5G technology, showcases a broader range of expressive styles and dynamic visual effects compared to traditional computer graphics. The creative process in VR painting enhances spatial depth, exhibiting different spatial abilities and necessitating more physical movements, including hand controllers and foot movements in the virtual space. Furthermore, VR painting in art therapy encourages users to engage in physical activities, contributing to better emotional expression. This study involved digital-native users in VR painting, using Meta Quest 2 to operate Open Brush for their creations. Through observational methods, we examined user operational behaviors and conducted semi-structured interviews post-experiment to explore their painting performance and usage behaviors in the virtual environment. The results of this study indicate that VR painting enhances the sense of space and dynamic expression in creative work and improves users’ emotional and physical engagement, providing new avenues for artistic expression. These findings contribute to improving the usability and application value of VR paintings.

1. Introduction

Virtual Reality (VR) technology, as an emerging expressive digital art tool, offers unique advantages [1]. It provides a highly immersive environment, allowing individuals undergoing treatment to fully integrate into a virtual world, creating empathetic scenarios that reduce patient self-criticism. Additionally, VR technology simulates real-world situations, offering more personalized and targeted therapeutic approaches [2]. Moreover, the interactivity and real-time feedback within VR environments enable the dynamic and flexible therapeutic process and expand the possibilities for diverse and innovative psychological interventions.

The boundless nature of VR spaces increases physical engagement during VR operations compared to traditional tablet- or computer-based drawing. VR equipment demands the movement of upper and lower limbs when manipulating VR controllers. The immersive environment also influences users’ spatial cognition and operational behaviors allowing users to experience a sense of liberation from real-world constraints, stressors, and tensions in an alternative or imaginary space [3]. Artwork created within VR generates distinct visual effects.

This study aims to investigate the creative behaviors and user experiences of multimedia animation students in virtual spaces. Students majoring in multimedia animation participated in virtual space creation activities. Through observation and in-depth interviews, their creative processes and behavioral manifestations were observed in virtual 3D spaces. We also observed users’ operational behaviors using META QUEST 2 VR headsets, including their adaptation processes to virtual environments and interaction methods. We analyzed changes in students’ physical movements and expressions when transitioning from 2D creation to 3D spaces to elucidate how VR technology influences their creative behaviors and bodily movements. The results provide a basis to enhance the future usability of VR drawings for creative purposes and physical stimulation.

2. Literature Review

2.1. VR Drawing

VR technology offers advantages in art and design: it enhances immersive experiences, enabling users to interact more realistically; facilitates personalized and multi-sensory design, providing visual, auditory, and tactile feedback; improves design efficiency by streamlining processes; and expands artistic expression through real-time interaction and liberation from physical constraints [4]. VR also accommodates diverse artistic textures, including special effects and animated visual elements [5]. Furthermore, users can maintain heightened focus during the creative process.

Once users become familiar with the controllers, they can completely control the device with uninterrupted engagement with the artistic flow. VR provides a novel private space, enabling users to enjoy freedom from real-world constraints, stressors, and tensions and facilitating entry into an alternative or imaginary space [3]. Users can better regulate spatial information and various factors through virtual environments, potentially alleviating anxiety and attention-related issues. VR virtual spaces allow for control over scene objects and their quantity, environmental information, and sky brightness (Figure 1).

2.2. Spatial Perception in Drawing

VR drawing offers a unique spatial experience for users to create freely within a three-dimensional environment. This immersive experience enhances spatial cognition and perceptual abilities. Through VR drawing, users can view and adjust their work from various angles and increase creative flexibility to understand spatial relationships. The sense of immersion and interactivity in VR environments significantly improves users’ spatial navigation skills and creativity [1,2]. VR drawing also reduces the creative anxiety of users to focus more on exploration and creative expression [3]. These advantages highlight the extensive potential for VR technology applications in art and design. VR drawing incorporates spatial depth characteristics, enhancing creative flexibility compared to drawing on tablets and smartphones.

2.3. Physical Expression in VR Creation

In bodily movement, virtual environments offer expanded creative spaces, engaging participants in more extensive physical motions to create within surroundings. Hand movements in the air during VR creation are more physically demanding than traditional media, which may be challenging for stroke patients [6]. However, the full-body engagement required when operating VR equipment is highly enjoyable for users with intact motor abilities. The ability to erase portions of artwork in VR mitigates the sense of permanence associated with traditional artistic media. Consequently, participants can focus on exploration and creative expression without the anxiety of making irreversible mistakes [3]. This feature potentially enhances the therapeutic value of the creative process by reducing inhibitions and promoting more accessible artistic exploration.

3. Materials and Methods

We invited participants for the tree-house-person (THP) test [7]. The THP test is common in art therapy, featuring three distinct objects that allow for the exploration of spatial relationships, such as the distance between objects. In this study, participants completed the test using the Oculus Quest 2. We observed the participants’ behavior during the test, including their spatial cognition and physical movements within the virtual space.

3.1. User Interviews

We conducted post-experiment interviews to understand the users’ feelings about the VR space and gather usability feedback on the user experience with Openbrush v2.4. The user feedback and opinions on the VR drawing software allowed for further analysis and improvement of the operational process and user experience. The interview questions were categorized based on the study’s focus on spatial cognition, operational processes, and user behavior.

• Spatial cognition and rotation behavior: How did you feel about spatial cognition and rotation behavior while operating the VR drawing software? Did you experience any dizziness or difficulty in orienting yourself?
• Drawing in virtual space: Could you easily find where to start drawing in the virtual space? Did you encounter any difficulties or challenges in drawing in a 3D space?
• Constructing 3D objects: Could you accurately construct 3D objects through your strokes while drawing in VR? Did you feel capable of transforming a 2D plane into a 3D space?
• Standing position and mobility: Did you find it difficult or uncomfortable to choose a standing position? Did you need to move your feet frequently?
• Physical comfort and movement: Were your physical movements comfortable during the drawing process? Did you use more physical movements? Did you feel any fatigue in your shoulders or arms?
• Favorite features in Openbrush: What features did you like the most while using Openbrush? These include brush types, colors, etc.
• Overall experience with VR drawing software: What are the strengths and weaknesses of your overall experience with the VR drawing software? Do you have any suggestions for the operational process and interface design?

3.2. Experiment

VR drawing software offers a unique 3D creative experience. Most users can adapt to and enjoy drawing with features such as dynamic brushes, special effects, and three-dimensional creation. However, several users face challenges with spatial cognition and initial operation. The interface layout for tool selection and function distribution has room for improvement: implementing shortcut keys and distinguishing between physical and virtual space boundaries. Additionally, prolonged use can lead to physical fatigue, particularly in the arms and shoulders.

Table 1. Interview results.

Aspect	Positive Feedback	Negative Feedback	Suggestions
Spatial Cognition and Orientation	Most users can orient themselves; the VR perspective is broader than a computer’s	Some users have difficulty orienting; some experience dizziness	Provide more spatial orientation aids
3D Drawing Experience	Able to draw 3D objects; creates a sense of depth	Difficult to draw detailed 3D graphics; requires time to adapt	Improve fine drawing tools
Movement and Standing	Most users feel comfortable; the joystick can be used for movement	Concern about hitting physical objects; some need to move their feet	Enhance safety warning system
Physical Comfort	Overall comfortable	Arms and shoulders can get tired, which is inconvenient when selecting tools	Optimize the tool selection interface; consider adding drawing shortcuts
Feature Preference	Dynamic brushes and effects are prevalent; the rainbow feature is attractive	No significant negative feedback	Further enrich dynamic brush effects
Overall Experience	Well-developed features; helpful prompts	Initial operation is complex; functions are scattered and hard to find	Simplify novice operation process; integrate frequently used functions

summarizes the feedback from the participants.

4. Results

4.1. Adaptability

Several participants adapted quickly to the VR environment. For instance, Participant 2 stated, “I do not experience VR sickness, and the perspective is wider than on a computer. Orientation is easy and stable”. However, Participant 1 reported, “I feel dizzy, and it is somewhat difficult to orient myself. I cannot quite grasp the sense of space”. The responses indicate individual differences in adaptability to the VR space.

4.2. Positive 3D Drawing Experience

Most participants engaged in 3D drawing within the VR environment. Participant 1 mentioned, “I can use the brush to create objects in 3D space, easily drawing items”. Participant 3 also said, “I can precisely draw 3D objects”. The participants’ responses demonstrate the VR drawing software’s strengths in 3D creation.

4.3. Movement and Safety Issues Need Attention

While most participants had no issues with movement, safety concerns remain. Participant 3 noted, “I have to move around to draw the back of objects, and I am afraid of bumping into tables and chairs in the real space”. Participant 6 also mentioned, “I am worried about hitting physical objects when moving”. The participants’ responses highlight the importance of ensuring a safe VR environment.

4.4. Prolonged Use and Physical Fatigue

Several participants reported fatigue after prolonged use. Participant 2 noted, “My arms feel sore because there are no shortcut keys, so I have to flip through tools manually. My shoulders get tired easily”. Participant 3 also said, “My arms feel sore, and since the starting position for drawing is too high, I have to keep lifting my arms to draw objects high up”. The participants’ responses suggest adding shortcut keys to the interface and an undoing/redoing movement.

4.5. Special Effect Brushes

Almost all participants expressed a liking for the special effect brushes. Participant 1 stated, “I like the bubble brush the most,” Participant 2 liked the “bubble brush and dynamic effect brushes,” and Participant 4 mentioned, “I like the moving brushes, such as stars and comets”. The participants’ responses show that special effect brushes are a significant highlight of the VR drawing experience.

4.6. Improvement in User Interface

Although the overall experience is good, the user interface has room for improvement. Participant 1 suggested, “Some functions could be displayed individually instead of hidden within the space. It might be hard for beginners to find them”. Participant 2 also proposed, “Common drawing functions could be integrated”. The participants’ responses indicate that simplifying and optimizing the user interface can enhance the participant experience (Figure 2).

5. Conclusions

Interviews with users of VR drawing software showed that VR drawing software provided a new creative experience in 3D. However, when drawing 3D objects in virtual space, spatial orientation issues were determined. Participants encountered difficulty in positioning objects on the same horizontal plane, and the initial drawing point significantly influenced subsequent actions with the VR controller. To elevate the participant experience and broaden the horizons of VR drawing technology, the following need to be addressed:

• Developing more intuitive spatial orientation aids to introduce additional reference points for 3D space drawing to facilitate participant adaptation and utilization of the virtual space;
• Designing a safety warning system to mitigate the risk of participants colliding with their physical environment while immersed in the virtual space;
• Optimizing the tool selection interface to explore the integration of shortcuts or voice commands to minimize the physical fatigue associated with tool selection and switching;
• Enriching special effect brushes by expanding the variety of dynamic brushes while ensuring the ease of operation;
• Adding shortcut keys and reducing fatigue.

Participant feedback highlighted the importance of incorporating shortcut keys into the interface and introducing an undo/redo movement functionality.

In the long term, conducting a comparative study between virtual reality (VR) drawing and traditional drawing methods is crucial for evaluating their respective impacts on object creation and the enhancement of hand movements during the drawing process. The integration of spatial orientation within VR environments significantly influences physical interaction, presenting novel opportunities for user engagement. The outcomes of this study contribute meaningful insights into the usability of VR-based drawing systems and serve as a foundation for the future advancement of VR drawing technologies. The results of this research offer information on the usability of VR drawing creation and the future design of VR drawing.