On the Use of ROMOT—A RObotized 3D-MOvie Theatre—To Enhance Romantic Movie Scenes

Abstract

In this paper, we introduce the use of ROMOT—a RObotic 3D-MOvie Theatre—to enhance love and sex movie scenes. ROMOT represents the next generation of movie theatres, where scenes are enhanced with multimodal content, also allowing audience interaction. ROMOT is highly versatile as it can support different setups, integrated hardware and content and, thus, it can be easily adapted to different groups and purposes. Regarding the setups, currently, ROMOT supports a traditional movie setup (including first-person movies), a mixed reality environment, a virtual reality interactive environment, and an augmented reality mirror-based scene. Regarding the integrated hardware, the system currently integrates a variety of devices and displays that allow audiences to see, hear, smell, touch, and feel the movement, all synchronized with the film experience. Finally, regarding to content, here we theorize about the use of ROMOT for romantic-related interactive movies. Though the work presented in this sense is rather speculative, it might open new avenues of research and for the film and other creative industries.

1. Introduction

The idea of building multimodal movie theatres is not new. Back in 1962, Morton L. Heilig patented the Sensorama [1], one of the earliest immersive, multisensory (or multimodal) machines. The technology integrated in the Sensorama allowed a single person to see a stereoscopic film enhanced with seat motion, vibration, stereo sound, wind, and aromas, which were triggered during the film. It was also referred to as “the cinema of the future” [2,3], though at that time it did not attract wide audiences. On the other hand, when referring to cinemas in a wider sense, i.e., movie theatres, the film experience is collective rather than individual—as it was the case of the Sensorama. The existing research works dealing with multimodal technologies and environments [4,5] usually involve individual, rather than collective, experiences, and/or refer only to the involved technology.

The rapid technological advancements of recent years have allowed the development of commercial solutions that integrate a variety of multimodal displays in movie theatres, such as in [6,7,8], where these systems are usually referred to as 4D or 5D cinemas or theatres. Some claim that this technology shifts the cinema experience from “watching the movie to almost living it” [9], also enhancing the cinematic experience while creating a new and contemporary version of storytelling, which can be conceptualized as a “reboot cinema” [10].

However, the criterion followed to establish the number of dimensions is not unified. In fact, the naming seems to follow commercial purposes rather than referring to physically-based dimensions. For instance, according to some, 4D cinema expands the 3D cinema by allowing a range of real-time sensory effects, including seat movements, leg and back pulsation, projected wind and mist blasts, fog, lightning, scent perfume discharge, etc., all synchronized with the narrative of the film [9]. According to others, the fourth dimension corresponds to the movement and/or vibration of the seat, whereas the fifth dimension integrates the rest of sensory effects [11]. In [11] further distinctions are made that include some kind of interaction and wearing of virtual reality glasses, justifying more than 7D, though the referred systems seem closer to single-person virtual reality simulators than to movie theatres. Having said this, we prefer to use the term “multimodal/multisensory 3D-movie theatre” when referring to rooms exhibiting stereoscopic films enhanced with sensorial stimuli that can be experienced by a group of persons simultaneously. A step beyond would be to add to the film feedback of the users, leading to interactive multimodal/multisensory 3D movie theatres.

In this paper, we present ROMOT, a RObotized 3D-MOvie Theatre and theorize about its use to enhance romantic scenes, discerning between romantic and sexual content. The construction of ROMOT and its first validation through usability and satisfaction tests has been recently published in [12], for a case study on driving simulation awareness. ROMOT follows the concept of the 3D-movie theatre with a robotized motion platform and integrated multimodal devices. Additionally, it supports audience-film interaction. Based on this, the audience gets some kind of reward by the system. Furthermore, in this sense, the whole system can be perceived as being alive, a kind of large robot around the audience. The robot “plays” with the audience, making them feel part of the (multimodally) displayed movies; at the same time, the audience has some degree of interaction with the system, being able to lead the story at certain points. Additionally, ROMOT is highly versatile as it is prepared to support different types of setups, hardware, and content, including films/animations that could be related to learning, entertainment, love, sex, etc. The following setups are integrated in ROMOT and shown in the paper: a traditional movie setup (including a first-person movie), a mixed reality environment, a virtual reality interactive environment and an augmented reality mirror-based scene. The contents of all of the different setups are based on storytelling and are seen stereoscopically so they can be broadly referred to as 3D movies.

This paper is organized as follows: First, we show the main technical aspects behind the construction of ROMOT and the integrated multimodal devices and interaction capabilities. It is worth mentioning that, differently from other existing commercial solutions, we have used a 180° curved screen to enhance user immersion. Then, we show the different kind of setups. Finally, we discuss the expansion of ROMOT for romantic scenes, also showing the results of a questionnaire and giving some hints for future directions.

2. Materials and Methods

2.1. Hardware

The house (audience) was robotized by means of a 3-DOF motion platform with capacity for 12 people (Figure 1). The seats are distributed in two rows, where the first row has five seats and the second one, seven seats. This motion platform is equipped with three 2.2 kW SEW Eurodrive (Bruchsal, Germany) electric motors coupled with a 58.34 reduction-drive. The parallel design of the robotic manipulator alongside with the powerful 880 N·m motor-reduction set, provide a total payload of 1500 kg, enough to withstand and move the 12 people and their seats.

The design of the robotized motion platform allows for two rotational movements (pitch and roll tilt) and one translational displacement along the vertical axis (heave displacement). The motion platform is capable of featuring two pure rotational DOF, one pure translational DOF (the vertical displacement) plus two “simulated” translational DOF by making use of the tilt-coordination technique [13] (using pitch and roll tilt to simulate low-frequency forward and lateral accelerations). Thus, it is capable of working with five DOF, being the yaw rotation the only one completely missing. It is, therefore, a good compromise between performance and cost, since it is considerably cheaper to build than a 6-DOF Stewart motion platform [14], but its performance could be similar for some applications [15]. The motion platform is controlled by self-written software using the MODBUS/TCP protocol. The software includes not only the actuators’ control, but also the classical washout algorithm [16], tuned with the method described in [17]. Combining different DOF results in a reduction of the amount of reachable linear/angular displacement of each DOF. Nonetheless, this parallel design allows for large payloads and fast motion [18], which were the key needs for this project. In fact, the robotized motion platform is capable of performing a whole excursion in less than 1 s.

In front of the motion platform, a curved 180° screen is placed (Figure 2), with a 3 m height (and a 1.4 m high extension to display additional content) and with a radius of 3.4 m. Four projectors display a continuous scene on the screen, generated from two different camera positions to allow stereoscopy. Therefore, to properly see the 3D content, users need to wear 3D glasses.

Although some smaller setups introduce the display infrastructure on top of the motion-platform (thereby they move together and inertial cues are correctly correlated with visual cues), the dimensions of ROMOT’s screen strongly recommend that the display infrastructure is kept fixed on the ground. Therefore, the visual parallax produced when the motion platform tilts or is displaced with respect to the screen needs to be corrected by reshaping the virtual camera properties so that the inertial and visual cues match. This introduces an additional complexity to the system, but allows the motion platform to be lighter and produce higher accelerations, increasing the motion fidelity [19].

In order to enrich the experience of the users and make the filmic scenes more realistic, a set of multimodal displays was added to the robotized platform:

• An olfactory display. We used the Olorama (Valencia, Spain) wireless aromatizer [20]. It features 12 scents arranged in 12 pre-charged channels that can be chosen and triggered by means of a UDP packet. The device is equipped with a programmable fan that spreads the scent around. Both the intensity of the chosen scent (amount of time the scent valve is open) and the amount of fan time can be programmed.
• A smoke generator. We used a Quarkpro QF-1200 (Madrid, Spain). It is equipped with a DMX interface, so it is possible to control and synchronize the amount of smoke from a computer, by using a DMX-USB interface such as the Enttec Open DMX USB [21] (Melbourne, Australia).
• Air and water dispensers. A total of 12 air and water dispensers (one for each seat) (Figure 3). The water and air system was built using an air compressor, a water recipient, 12 air electro-valves, 12 water electro-valves, 24 electric relays and two Arduino Uno (Ivrea, Italy) to be able to control the relays from the PC and open the electro-valves to spray water or produce air.
• An electric fan. This fan is controllable by means of a frequency inverter connected to one of the previous Arduino Uno devices.
• Projectors. A total of four full HD 3D projectors. Blending techniques are required to merge their corresponding images.
• Glasses. A total of 12 3D glasses (one for each person).
• Loudspeaker. A 5.0-channel loudspeaker system to produce binaural sound.
• Tablets. A total of 12 individual tablets (one for each person).
• Webcam. A stereoscopic webcam to be able to construct an augmented reality mirror-based environment.

It is important that all the multimodal actuators can be controlled from a computer, so that they can be synchronized with the displayed content and with the motion platform (Figure 4).

Within this set of multimodal displays, users are able to feel the system’s response through five of their senses:

• Sight: they can see a 3D representation of the scenes on the curved screen and through the 3D glasses; they can see additional interactive content on the tablets; they can see the smoke.
• Hearing: they can hear the sound synchronized with the 3D content.
• Smell: they can smell essences at certain events. For instance, when a car crashes, they can smell the smoke. In fact, they can even feel the smoke around them.
• Touch: they can feel the touch of air and water on their bodies; they can touch the tablets.
• Kinesthetic: they can feel the movement of the 3-DOF platform.

Apart from that, the audience can provide input to the ROMOT through the provided tablets (one tablet per person). This interaction is integrated in the setup of the “3D-virtual reality interactive environment”, which is explained as part of the following section.

2.2. Software

The software is built on the Unity framework and it is currently prepared to support four different setups, for which some preliminary contents have been created: traditional movies, mixed reality environments, virtual reality interactive environments, and augmented reality mirror-based scenes. These setups are briefly introduced in the following paragraphs.

• Traditional movies: Traditional movies can be displayed in ROMOT, while also having the possibility to see the contents stereoscopically, provided that the videos have been recorded with a pair of cameras. Additionally, with this setup first-person movies can be displayed, enhancing the possibilities to enhance users’ experiences, as they can feel like being the protagonists of the movies. As a first example, and to prove the concept, a set of videos were recorded using two GoPro cameras to create a 3D movie set in the streets of a city. Most of the videos were filmed by attaching the GoPro cameras to a car’s hood in order to locate the audience at the centre of the view, as if they were the protagonists of the journey. At every moment there is audio consisting of ambient sounds and/or a locution that reinforces the images the user is watching. The synchronized soft platform movements or effects, like a nice smell or a gentle breeze, help to create the perfect ambience at each part of the movie, making the experience more enjoyable for the audience. Of course, romantic and sexual content can be recorded and displayed too.
• Mixed reality environment: This setup consists of a combination of recorded videos and virtual content, thus creating a mixed reality movie that helps the audience perceive the virtual content as if it were real, making the transition from a real movie to a virtual situation easier. In this setup, the created 3D virtual content—a 3D virtual character—interacts with parts of the video by creating the virtual animation in such a way that it is synchronized with the content of the recorded real scene. Virtual shadows are also considered to make the whole scene more real. This could have interesting applications for romantic scenes, mixing real filmed characters with virtual content.
• Virtual Reality Interactive Environment: This setup consists of a pure virtual environment that is also interactive, as audiences can give their feedback to the system by means of the integrated tablets for each of the seats. As a first trial, we recreated a city where users could walk and drive, thus going through their streets and meeting other people and situations. Each situation was created using a storyboard that contains all of the content, camera movements, special effects, locutions, etc., so at the end, a set of situations were derived that could be part of a movie. In this case, we want the audience not to just look at the screen and enjoy a movie, but to make them feel each situation, to be part of it and to react to it. That is why platform movements and all of the other multimodal displays are so important. When each situation takes place, the audience can feel that they are driving inside the car or walking through the city, thanks to the platform movements that simulate the real movements. In some of the scenes, the virtual situation pauses and asks the audience for their collaboration. At that moment, the different tablets vibrate and a question appears, giving the individual users some time to answer it by selecting one of the possible answers (related to driving, in this case). When the time is up, they are prompted to report whether the answer was correct or not (as in this case, there are right and wrong answers), and the virtual situation resumes, showing the consequences of choosing a right or a wrong decision. When the deployed situation finishes, the audience can see the final score on the large curved screen. The people having the greatest score are the winners who are somehow rewarded by the system by receiving a special visit, a 3D virtual character that congratulates them for their safe driving (see next paragraph). The application to romantic/sexual content is straightforward, allowing, for instance, the creation of point of view (POV) immersive scenes with fictional characters.
• Augmented Reality Mirror-Based Scene: This setup consists of a video-based augmented reality mirror (ARM) [22,23] scene, which is also seen stereoscopically. ARMs can bring a further step in user immersion, as the audience can actually see a real-time image of them and feel they are part of the created environment. This ARM environment is used in the final scenes of the aforementioned virtual reality interactive environment (previous sub-section), where the user(s) with the highest score get(s) rewarded by a virtual 3D character that walks towards him/her/them. Together with this action, virtual confetti and coloured stars appear in the environment, accompanied with winning music that includes applause. The application of ARM to romantic and sexual content is more controversial in terms of ethical issues, yet completely feasible from a technological standpoint.

2.3. Usability and Satisfaciton Tests

Once the system was built, it was validated by a pair of preliminary tests that were conducted at the laboratory level. The results of such tests have been published in [12], and here a short summary is given.

A total of 14 people tested the system and participated in its evaluation. The participants were some of the research staff of the IRTIC institute that interacted with the system, where seven of them were women and seven were men. The evaluation consisted of filling out a pair of questionnaires that were related to the usability of the system and the individual’s satisfaction. For the first case, the System Usability Scale (SUS) [24] was chosen whereas, for the second case, a total of 12 questions were composed, which were related to the user satisfaction of diverse characteristics of ROMOT (e.g., the 3D movies, multimodal content, etc.). Some recent works using these tests can be found in [25,26,27].

The outcome of the SUS test is a score that goes between 0 and 100 points. For ROMOT, the SUS score was of 84.25 points, which can be considered excellent on the scale of scores provided by the questionnaire and taking into account the fact that a minimum score of 68 would be deemed acceptable as a tool [28,29]. On the other hand, the results of the individuals’ satisfaction questionnaires ranged from 0 to 4 points, meaning: 0: strongly disagree (non-satisfactory) to 4: strongly agree (very satisfactory). After calculating the mean values of all 12 questions we concluded that the results were quite satisfactory, as eight out of the 12 mean scores were over 3 points and none were below 2.5 points.

3. Results

ROMOT is a laboratory system (hardware and software) that has been built from scratch. Due to this, and differently from other commercial systems, ROMOT is highly versatile, being easily adapted to different kinds of groups, purposes, contents, setups, etc., as both the hardware and the software can be modified with relatively little effort. This opens new avenues in research related to, e.g., HCI, robotics, learning, perception, etc. A first case study was previously presented in [12], which was focused on driving safety awareness. In this paper, however, we theorize about the possible expansion of ROMOT for romantic interactive content and/or experiences, discerning between love and sex scenes.

Romantic content can be easily adapted to ROMOT. For instance, users can experience films with alternative endings. At a certain point of the movie, we can ask the audience “will she marry him?” and it will be the public who decides. User studies could be performed on this, on whether the public prefers a happy or an unhappy ending. Gender analysis could be furthermore performed. This might be already done in traditional movies, but with ROMOT this will acquire a further dimension, as the number of multimodal stimuli and the different kinds of setups can expand the possibilities and enhance user immersion. For instance, the audience could smell flowers when he gives them to her as a present; we can simulate the motion of a car when they go on a romantic trip across the country, etc.

In order to have a first evaluation of the intention of the general public to see love or sex scenes within ROMOT, we have conducted a survey where people had to answer a short questionnaire. To that end, a love scene and a sex scene of known movies were selected, and the first setup of ROMOT was used (i.e., the traditional movies setup). In the following subsections, these case studies are introduced and then the results of the questionnaire are outlined.

3.1. Description of the Selected Scenes

For the “love” case study, the scene known as the most romantic from the film Titanic is reported here [30]: Rose gasps. There is nothing in her field of vision but water. It’s like there is no ship under them at all, just the two of them soaring. The Atlantic unrolls toward her, a hammered copper shield under a dusk sky. There is only the wind, and the hiss of the water 50 feet below. Rose: “I’m flying!” She leans forward, arching her back. He puts his hands on her waist to steady her. Jack: “Come Josephine in my flying machine…” Rose closes her eyes, feeling herself floating weightless far above the sea. She smiles dreamily, then leans back, gently pressing her back against his chest. He pushes forward slightly against her. How could this scene be reproduced in ROMOT? The audience would feel fresh air on their faces, the smell of the sea, humidity, and their seats moving, accompanied with the waves. We believe that they could reach a greater immersion in the movie and, thus, feel like they are Rose; they are flying, too (perceiving the scene in first person).

In the same way, sex scenes could be reproduced and enhanced in ROMOT. As the “sex” case study, the non-explicit sex scene in The Twilight Saga: Breaking Dawn (Part 1) is reported here. In that movie, Bella marries Edward, the vampire. They are in their honeymoon. They decide to have sex for the first time (partially taken from [31]): Bella walks into another area of the house. Bedroom. Stares at the bed, touches the curtains around the bed. Can see ocean in background (…). Edward walks outside. Takes off shirt to get into water. (…) Bella walks up to Edward in the moonlight. Bella is naked and walking towards the water. Edward is already in. They kiss each other. Then, back to the bedroom. They have sex. The next day, the bed appears completely broken and Bella covered with pillow feathers. In ROMOT the audience would feel the ocean breeze. Then, in the bedroom, bed movements could be directly transferred to smooth vibrations and platform movements. The scene would end with very abrupt movements, and the audience would feel like something has broken, too. Some pillow feathers could also be projected onto the audience.

3.2. Results of the Questionnaire

A total of 22 persons participated in the survey, 50% woman and 50% men with ages ranging from 20 to 50 years old. Those people were related to the IRTIC lab, as we wanted to have the feedback of people that had already interacted with ROMOT. We have asked them the following question for four different situations: “I would feel comfortable watching love/sex scenes in ROMOT in the following situations: (a) If I don’t know the rest of the audience; (b) If I go with my fiancée/husband/wife, but we don’t know the rest of the audience; (c) If I go with friends/colleagues; (d) If I go with my family/relatives (adults)”. For each of the four situations, and for each case study (love/sex), participants had to give a score from 1 (completely disagree) to 5 (completely agree).

The results of the given answers are depicted in Figure 5 (love case study) and Figure 6 (sex case study), for both men (11 individuals) and woman (11 individuals). As it can be seen, when comparing the overall results of the different case studies, we can say that people would feel more comfortable experiencing love scenes than sex scenes in ROMOT. When looking to the different situations for both scenarios, most people would feel uncomfortable when going with their family or relatives, especially in the sex case scenario. This happens even if the selected scene does not involve explicit sexual content, as it is the case of the current sex case study. However, most people would feel comfortable with the sex case scenario if they go with friends or colleagues. Discerning between men’s and women’s results, it can be seen that more women would feel more comfortable experiencing love scenes than men, though in both cases the most often given score is “4” for all of the situations (a–d). On the contrary, more men would feel more comfortable experiencing sex scenes than women, but there is more variation on this. Both genders would feel especially uncomfortable in situation (d) (for men, mostly with a score of “2” and, for women, mostly with a score of “1”) and both genders would feel more comfortable in situation (c) (for men, mostly with scores of “4” and “5” and, for women, mostly with a score of “4”). Men would also feel comfortable in situation (a) (with scores of “4” and “5”).

3.3. Integrating Other Display Technologies

Apart from the currently integrated displays in ROMOT, other kind of technologies could be added to enhance movies with other stimuli, such as taste or touch. For instance, the Kissenger technology [32] could provide the audience with a kiss synthetically reproduced and/or previously recorded by the actors themselves, and individuals could choose between being kissed by her or by him. Could you image being kissed by Jack or by Rose in the Titanic scene? In a similar way, hugs could be also transferred to the audience with a technology like the Huggy Pajama [33], so the audience could feel the hugs of actors in romantic and sex scenes. This technology could open new avenues in the related creative industries, where actors could collaborate to create a new kind of perception to attract very large audiences in such a way that their fans could virtually feel their actions.

Finally, we would like to highlight that, as ROMOT involves the whole audience, it can be perceived as a collective experience. It could raise ethical issues when related to explicit sexual content and, thus, this could be an interesting field of research. For instance, would the audience immersed in ROMOT feel like they are experiencing a kind of orgy and, perhaps, make them feel uncomfortable? Or would they feel the scene as isolated from the rest of the audience? Alternative setups with a single user (such as the Sensorama) could be built for this case.

4. Conclusions

In this paper we have presented ROMOT, a robotized 3D movie theatre, and theorize about its use to enhance love- and sex-related content, departing from a setup following traditional movies.

Both the setups and the film contents of ROMOT can be changed for different kinds of user experiences. With the different setups, we have shown a traditional movie setup and others related to the technologies of virtual, augmented, and mixed realities. Regarding the content, we have briefly shown samples related to driving safety and added a discussion for the use of ROMOT with romantic and sexual content. To support our statements, a first user evaluation has been given that reveals that people would feel more comfortable when experiencing romantic content with ROMOT, but they would also feel comfortable experiencing sexual content if they go with friends or colleagues.

We have also discussed the further integration of other display technologies within ROMOT that include the possibility to transmit kisses and hugs to the audience. In this way, people could feel like being touched by their favourite actors, becoming a new claim to attract very large audiences.

Though the use of ROMOT with romantic and sexual content, as presented here, is rather speculative, we believe that it might open new avenues for romantically- and sexually-related experiences at a multi-user, multidisciplinary level (robotics, HCI, sociology, psychology, arts, etc.), which remains quite unexplored, so it may be of interest for both the research community and film/creative industries.