Directional Force Feedback: Mechanical Force Concentration for Immersive Experience in Virtual Reality

Abstract

In recent years, consumer-level virtual-reality (VR) devices and content have become widely available. Notably, establishing a sense of presence is a key objective of VR and an immersive interface with haptic feedback for VR applications has long been in development. Despite the state-of-the-art force feedback research being conducted, a study on directional feedback, based on force concentration, has not yet been reported. Therefore, we developed directional force feedback (DFF), a device that generates directional sensations for virtual-reality (VR) applications via mechanical force concentrations. DFF uses the rotation of motors to concentrate force and deliver directional sensations to the user. To achieve this, we developed a novel method of force concentration for directional sensation; by considering both rotational rebound and gravity, the optimum rotational motor speeds and rotation angles were identified. Additionally, we validated the impact of DFF in a virtual environment, showing that the users’ presence and immersion within VR were higher with DFF than without. The result of the user studies demonstrated that the device significantly improves immersivity of virtual applications.

1. Introduction

An immersive system is the ultimate version of the VR as it can be enhanced by audio, haptic, and sensory interfaces [1]. In this respect, establishing a sense of presence has been a key objective of VR [2]. Advancements in the immersivity of VR have allowed researchers to simulate and evaluate human responses under difficult conditions, such as virtual tour through VR [3], pedagogical simulation [4], emergency situations [5], motor conditions of elderly patients [6], body movements [7], physiological reactions [8], hand rehabilitation [9], and even in the case of Alzheimer’s patients [10]. Despite the importance of the effective multimodal interface for VR [11], immersive haptic feedback for VR applications has long been in development. According to Mihelj et al. [12], simulated physical contact in VR affects the complexity of realistic haptic rendering. The shape of the 3D objects will be sufficient for immersive haptic rendering among various physical properties if the VR contents only use the display of the 3D object. However, more information (weight, elasticity, texture) needs to be provided when the user grasps the object [12]. Considering this, researchers have developed multiple approaches to providing haptic feedback from virtual environments, with the aim of generating more immersive experiences [13]. Recent studies on force feedback have used air streams [14], air suction [15], and glove actuators [16]. State-of-the-art literature on haptic devices introduced a plausible handheld controller for virtual environments [13], shape-changing device corresponding to 3D digital objects in VR [17], and even a suit that resists the wearer’s movements depending on the VR contents [18]. However, none of these innovations focused on providing directional feedback during interaction with objects possessing inertia (i.e., free-falling objects from various heights or objects travelling laterally with momentum). Despite the state-of-the-art force feedback research being conducted, a study on directional feedback, based on force concentration, has not yet been reported.

In this study, we focus on developing a method of force concentration in different directions for an immersive experience when interacting with moving objects in VR. We developed a device, directional force feedback (DFF), that uses the rotations of motors to concentrate force and deliver directional sensations to the user. DFF generates directional haptic feedback for VR applications using mechanical force concentrations, and comprises actuators and mechanical structures to provide an immersive virtual experience. The device rotates attached weights at controlled speeds to provide directional sensations. As shown in Figure 1, users hold DFF as they would any other input device before interacting with VR objects. In-depth interviews followed by three user studies were conducted to test the impact of DFF. In the first user study, we validated the capability of force concentrations to generate force feedback with directional sensations. The second user study tested the extent to which the sensation of weight can be delivered via DFF. In the final study, we evaluated the impact of DFF on the immersivity of an experience in a virtual environment. To the best of our knowledge, this is the first study reported that uses force concentrations to generate directional force feedback. The highlights of the proposed research are as follows. A novel method of force concentration was developed for directional sensation we identified the optimum rotational motor speeds and rotation angles, considering both rotational rebound and gravity. We validated the impact of DFF in a virtual environment, showing that the users’ presence and immersion within VR were higher with DFF than without.

2. Related Works

Recent literature on haptic devices to render sensations of weight and directional forces varies includes a software-based haptic interface, shape-changing device, wearable haptic device, and airflow-based haptic interface. Rietzler et al. [19] proposed a software-based method that used visual manipulation of digital content to modify perceptions of weight. Interestingly, they found that the levels of presence, immersion, and enjoyment, as rated by the users, could be improved significantly with perceivable tracking offsets. Unlike the software-based approach, Shigeyama et al. [17] proposed a shape-changing haptic device. They proposed Transcalibur, which changes the input device’s center of mass to generate different haptic feedback corresponding to specific VR objects held by users. Transcalibur drives weights on a 2D planar area to alter the mass property of the device. For example, the weight shift varies when the user holds a sword or a shield in the VR content to provide shape perception. Zenner and Krüger [20] proposed Shifty, a rod-shaped dynamic passive haptic proxy device that changes its center of mass. It also changes its perceived thickness and length based on the VR item held by the user. A study was completed wherein the perception of weight was controlled using only visual information. Unlike the shape-changing devices that impart sensations of weight, researchers also investigated the use of wearable haptic devices to simulate physical contact in VR. Maimani and Roudaut [18] proposed a “jamming suit”, which conveyed a sensation of freezing to the human body. The suit jams joints of the user’s body, such as the elbows and knees, to provide the physical sensations of being attacked by ice ghouls (VR creatures). Gu et al. [16] developed Dexmo, a state-of-the-art wearable mechanical exoskeleton enabling motion capture and force feedback, which was shown to improve the quality of hand motion control in VR applications. Benko et al. [13] developed a haptic 3-dimensional (3D) shaped output using handheld controllers to enable users to feel shapes, surfaces, forces, and textures based on linear actuation. They demonstrated two devices that render the surface, force feedback, shape, and texture of the VR contents through the fingertips. Since the controller only uses the bottom of the fingertip for haptic information rendering, the device cannot provide complete haptic information [13]. Choi et al. [21] introduced Grabity, a wearable haptic interface which simulates sensations of weight and grasping in VR. Grabity is composed of a kinesthetic pad to simulate grasping sensations for various sized VR objects. Furthermore, it stretches users’ skin to simulate sensations of various weights. However, unlike the aforementioned studies that focus on simulating physical contacts in VR through restriction and vibrotactile feedback, there are researchers who have used air flow to generate real force feedback, enabling users to interact in VR environments without handheld devices. Tsalamlal et al. [14] generated haptic feedback using a mobile air jet (HAIR), which generated air jets to stimulate the skin. HAIR generated air jets using a robotic arm that tracked and moved underneath the user’s hand. Suzuki and Kobayashi [22] created a device containing a grid of air nozzles. When a virtual 3D object was placed on the device, air flow generated force feedback corresponding to the location of the user’s hand and the VR object. Hachisu and Fukumoto [15] developed VacuumTouch, a force feedback interface based on air suction. VacuumTouch was connected to a touch screen to create pressure when the user touched the digital input interface. The airflow-based haptic interface provides real force feedback, but requires extra installations. Unlike previous literature on haptic devices, this study focusses on rendering actual force feedback by creating force concentrations to render both directional and weight feedback in VR.

3. Methodology

DFF uses the rotation of weight for force concentration. We generate concentrated force by applying rotational forces to a handheld weight. The weight is fixed to a handle, which sits perpendicular to the motor’s axis of rotation. Force concentration is then created by adjusting the speed of the motor. At faster speeds, the stronger centrifugal force displaces the weights further toward the outer edges. The opposing force against the rotating force causes the motor to move itself in the opposite direction. This repulsive force influences the axis of rotation, causing it to rotate with a cone-shaped movement. If the repulsive force is minimized to a level where the user cannot sense the directional force, the user can feel the directional force based on the force concentrated in the rotational axis of the motor. Figure 2 illustrates the mechanism of force concentration, wherein Pt is the target direction for the concentrated force, Ps is the start position, and Pe is the end position. By controlling the speed of the motor and the rotation angle interval, the force can be concentrated in the predefined target direction. Details are provided in the following section. A series of validation experiments were conducted to test the impact of the proposed methods, which will be discussed in detail in Section 4.

3.1. Rotational Speed Control

While the weight is rotating, the force deviates more toward the weight as the rotational speed of the weight changes. Therefore, the rotation of the weight accelerates before reaching its maximum velocity in the target direction and then decelerating. The graph of rotational speed starting at the angle at which the acceleration starts (Ps) and ending at the angle at the end of the deceleration phase (Pe) can be plotted, as shown in Figure 2. However, in contrast to its behavior in computer simulations, the motor control behaves distinctly in physical world. The unfilled blue circles in Figure 3 (actual motor movement) shows a rotational speed curve that does not reach its target value due to the load (i.e., weight of the device and other assembly components) applied and the limitations of instantaneous acceleration. The rotational speed and acceleration/deceleration directions of the real motor must be identical with the programmed values to maximize the force concentration in the target direction. In the light of these considerations, we developed a control model that optimizes the speed changes of the motor.

3.2. Rotation Angle Interval Control

Control of the rotational angle is as essential as the control of the rotational speed in maximizing the directional sensation conveyed to the user through force concentration. The impact of the directional sensation can vary depending on the rotational angle. When the different acceleration-deceleration interval is set to span Ps to Pe, the length of the interval, the actual motor movement will behave differently depending on the length of the interval (|Pe-Ps|) (Figure 4). In theory, if the rotation angle interval is short, the force concentration will be rendered with more impact than the long interval since the motor will accelerate and decelerate for the shorter term. The actual movement of the rotational angle also behaves differently from the programmed movement. If the interval is set to be wide enough, sufficient time is given for the acceleration and deceleration such that the movement may closely follow the programmed movement, but this may in turn deteriorate the directional sensation due to force dispersion. If the interval is set to be narrower, the directional sensation can be maximized, but may result in a large discrepancy between the programmed movement and the actual motor movement. In this study, we developed a control model and tested how well the movement pattern of the motor mimics the programmed movement over three different rotation angle intervals (±30${}^{\circ }$; ±60${}^{\circ }$; ±90${}^{\circ }$). The three rotation angle intervals in 30-degree increments were used as it is inefficient to test rotation angle intervals per degree. In this study, we aimed to identify the direction of correlation between the rotation angle interval and impact of the directional sensation.

3.3. Motor Control on Gravity and Rotational Rebound

If the motor operates at the programmed velocity while the user is holding the DFF stably, the force concentration in all directions must be identical. However, the weight rotation is also influenced by gravity. Therefore, extra force is required in when it rotates against the gravitational force, and conversely less force is required when it rotates in the same direction as gravity. The performance of the motor has a significant influence on the realization of the programmed motor movement. Thus, the actual motor movement will vary depending on the target direction. We tested and observed the force concentration behavior in four directions (up, down, forward, and backward in relation to the axis of rotation) within a specific programmed movement.

3.4. Degree of Force Concentration

In addition to control of rotational speed and the rotation angle interval control which helps to maximize the directional sensation of force concentration, the degree of force concentration can also be adjusted. A large force generated from a high maximum speed can result in a large discrepancy between the actual and programmed motor movement. We tested 3 modes of force concentration, where V1 delivers the strongest force and V3 the weakest (Figure 5).

4. Implementation

4.1. Device Design

The DFF design consists of two poles connected to actuators (Figure 6). To achieve the research objective, we needed actuators that met the following conditions: (1) accurate speed control of the motor must be possible; (2) tracking of the motor’s actual drive should be done in real time; (3) sufficient torque is required to move the weights (1 newton-meter (N*m) is required when stopping, 5 N*m or more when moving); (4) high motor position encoder resolution is required. As illustrated in

Table 1. The comparisons on motor performance.

	AX-18A	MX-64R	MX-106R
Resolution (deg/step)	0.2930	0.0879	0.0879
Stall Torque (Nm)	1.8	6.0	8.4
Max Baud Rate (Mbps)	1	4.5	4.5

, the Robotis Dynamixel MX product family is a module that includes motor, controller, driver, sensor, reducer, and network functions. It can decompose 360 degrees into 4096 steps (0.088 degrees per step) using a magnetic encoder and has up to 4.5 Mbps high speed communication. Therefore, we used the Dynamixel MX-64R, which has a stall torque of 6 N, can track the rotation angle with high resolution (12 bit), and can rotate in 0.08-degree increments. The performance of the MX-64 model was sufficiently satisfactory to meet the objective of the research. Each pole has joints to adjust the placements of the attachable weights. The current version of DFF allows three different weights (100, 200, and 500 g) to be used. Various weights can be attached to DFF with customized weight clamp designs.

4.2. Device Performance Optimization

We measured the rotation angle interval, the movement direction, and the degree of force concentration of the proposed system using the DFF device. Overall, the motor’s drive speed was shown to be slower than the programmed speed, and the real model was slower in reaching the maximum speed past the target point. A total of 36 datasets were created and analyzed (3 rotation angles × 4 directions × 3 levels of force concentration).

Regarding the effect of adjusting the rotation angle interval, the maximum velocity occurs closer to the original target when a wide rotation angle interval is employed. When the rotation angle interval was set to ±90 degrees, it was observed that the point at which the highest velocity occurred was pushed back by approximately 500 steps, which corresponds to a 45-degree angle difference (4096 steps of the 12-bit encoder; Figure 7). When the interval was ±30 degrees, the difference was approximately 250 steps (22.5 degrees). In considering the directional force concentration, operation within a narrow rotation interval has advantages.

As for the direction of rotation, by design, the acceleration direction is from downward to upward along the axis of rotation. To rotate in all four directions (forward, backward, up, down), additional force is required to compensate for the natural tendency and the weight attached to the device. On the other hand, acceleration is easily achieved in the upward and downward directions. As seen in Figure 8, the forward (Figure 8a) and backward (Figure 8c) directions show similar motor rotation behaviors, as do the upward (Figure 8b) and downward (Figure 8d) directions. Considering this, it is necessary to apply additional force when accelerating in the all four directions.

Finally, in controlling the maximum speed of the rotation, we observed that it was more difficult to maintain higher target speeds of motor rotation (Figure 9). When the target speed was high, the motor acceleration was maximized but the motor deceleration was delayed longer.

During the experiments with the actual DFF device, we identified a delay of 20 to 500 degrees in reaching the maximum velocity. Therefore, the directional sensation from the concentrated force did not correspond to the target direction. An actuator control equation was created to minimize this problem. The required target speed according to the rotation angle was derived as Equation (1). Where $\mathit{f}\mathit{v}$($\theta$) is function of target velocity of rotation angle. Theta is a relative rotation angle value set based on $\overline{P_{\mathit{target}}}$. An additional weighting constant, R, was added to Vmax (the maximum velocity) in Equation (2) to increase the initial acceleration. To solve the motor delay, it is necessary to set the inflection point of the motor’s acceleration/deceleration function slightly ahead of $\overline{P_{\mathit{target}}}$. At this time, the changed P correction value was defined as $\overline{P_{\mathit{offset}}}$. Equation (2) thus reduces the overall values of rotation angle to solve the observed delay. Using this model, we obtained results which corresponded much better to the programmed motor movement in all four directions, the rotation angle interval, and the degree of force concentration, as shown in Figure 10.

$$f_v(\theta )=Vmax\times (1-\frac{\left|\theta \right|}{(RotationInterval)})$$

(1)

$$f_v(\theta )=\left\{\begin{array}{cc}(Vmax+R)\times (1-\frac{|\theta +\overline{P_{offset}}|}{(RotationInterval)-\overline{P_{offset}}}),\hfill & \mathrm{if}\theta <-\overline{P_{offset}}\hfill \\ (Vmax+R)\times (1-\frac{|\theta +\overline{P_{offset}}|}{(RotationInterval)}),\hfill & \mathrm{if}\theta \ge -\overline{P_{offset}}\hfill \end{array}\right.$$

(2)

4.3. User Study

We conducted three user studies to validate the practical impact of the DFF device. The first user study tested the accuracy of the directional sensation generated via force concentration. The second user study tested how well the users could feel various magnitudes of weight generated via force concentration. The last study tested the impact of DFF in the reported immersivity of a VR application. A total of 12 participants, consisting of six males and six females, aged between 20–24 (Mean = 21.33; SD 1.37) were recruited for the three user studies and subsequent interviews. The 3 studies lasted approximately 50 to 60 min.

4.3.1. Participants

A total of 12 participants, comprised of six males and six females aged between 20–24 (Mean = 21.33; SD 1.37) were recruited for the three user studies.

4.3.2. Study 1

The participants were welcomed and thanked for their interest in joining the study. They were told about the device and informed that they could abort the study at any time. The participants were instructed to take hold of the DFF device, and the experimental procedure was provided with clear and straightforward instructions. The participants were instructed to correctly identify the direction of the force concentrated by the DFF device while blindfolded. The study was conducted in a controlled lab setting with an experiment mediator. The combinations of parameters influencing force concentration leveraged three rotation angle intervals (i.e., ±30${}^{\circ }$; ±60${}^{\circ }$; ±90${}^{\circ }$) and four target directions (i.e., up, down, forward, backward). In total, 36 combinations (three tests per direction per interval) were used. The participants audibly spoke the rendered direction when DFF performed each combination. Then, the direction data was recorded by the experiment mediator.

We observed that the DFF’s inertia of weight dragged the force to the course of target direction when we accelerated/decelerated motor velocity. We termed this "dragged direction" and used it as an evaluation measure of directional force concentration with the target direction. The result of the first study indicated that there were directions that could be better sensed than others. The result showed that the 83.8% (SD = 0.104) of the participants felt the directional force generated by DFF (Figure 11). The most effective direction was backward (closest to participants). 44.4% of the participants correctly identified the target direction, whereas the 47.2% answer correctly identified the dragged direction. The next most effective direction was upward (target direction = 32.4%; dragged direction = 57.4%). Next was the forward direction (target direction = 35.2%; dragged direction = 43.5%), and the downward direction (target direction = 23.7%; dragged direction = 50.9%). The reason that the downward direction scored worst was that the weight of the device made the gripping hand force feel like an upward force, resulting in relatively insensitivity to the weight change. In addition, in terms of the upward direction, the participants responded that they felt the forward direction more than the upward direction, the target direction. This may have resulted in similar results from the downward direction. However, the forward and backward directions showed higher corrected answer rates in the target direction than the upward and downward directions. In response to changes to the rotation angle interval, the corrected answer rate about the dragged direction was higher at the shorter interval (±30${}^{\circ }$, 52.1%) than the longer intervals (±60${}^{\circ }$, 48.6%; ±90${}^{\circ }$, 48.6%). In the shorter rotation angle interval, both acceleration and deceleration occurred rapidly. Therefore, it resulted in a strong inertial force.

4.3.3. Study 2

The first study revealed the optimum combinations of the parameters of force concentration, which maximized the directional sensation. The second study tested how well the participants could feel the degree of weight generated from force concentration. We adjusted optimal combinations of rotation angle intervals and rotational speeds to elicit the desired weight sensations. Participants were told that the second test was to evaluate how well they could feel the different weights generated by the DFF. Instructions were provided, explaining that the participant should correctly guess the degree of weight generated by the DFF. The study was conducted in a controlled lab setting with an experiment mediator. There were three degrees of weight used for this study (i.e., 1 = light; 2 = middle; 3 = heavy). Participants received a 2-min tutorial session to learn to feel the weights generated and were blindfolded as in the first study. The combinations of parameters affecting the force concentration were based on three rotation angle intervals (±30${}^{\circ }$; ±60${}^{\circ }$; ±90${}^{\circ }$) and 3 degrees of weight (1 = light; 2 = middle; 3 = heavy). In total, 45 combinations (3 tests per weights per interval) were used (Figure 10). The participants audibly spoke the rendered direction when DFF performed each combination. Then, the direction data was recorded by the experiment mediator.

The result of the test showed that the participants could correctly answer the various degrees of weight generated by the DFF (correct answer = 79.6%; SD = 0.09). The light degree of weight showed the highest correct rate (88%), followed by the heavy degree of weight (76.9%) and the middle degree weight (74.1%). The participants could feel the difference of all the three degrees of weight. Additionally, the rotation angle interval did not have a significant effect on correct answer rates in the light and middle degrees of weight (Mean ±30${}^{\circ }$ interval = 84.3%, SD = 0.11; Mean ±60${}^{\circ }$ interval = 86.1%, SD = 0.13). However, the correct answer rate was less in the ±90${}^{\circ }$ interval with 68.5% (SD = 0.2). It can be seen that when the interval is longer than a certain threshold, the resolving power for distinguishing the force direction is decreased.

4.3.4. Study 3

The third study evaluated the impact of the DFF device in a VR application. We tested the impact of the DFF on immersivity within a virtual environment when interacting with VR objects. A VR game was designed wherein the player was required to catch free-falling objects of three different sizes. The participants’ task was simply to catch the objects. The study comprised two sections. Section 1 entailed catching a free-falling ball without the DFF, and Section 2 entailed catching the ball with the DFF. The objective of the study was to compare the virtual experience of the two section. We used E2I questionnaires [23] and our own to measure the presence, immersion, and enjoyment of the virtual environment. The participants were asked to complete the questionnaires after each section. The overall evaluation of the study was well reflected in the experimental results (Figure 12). A significant difference was noted between the sections. The DFF improved the VR presence and immersion. Compared to Section 1, participants in Section 2 reported higher scores of presence (mean Section 1 = 3.17; mean Section 2 = 5.17; t(11) = −3.25; p = 0.00) and immersion (mean Section 1 = 3.92; mean Section 2 = 5.17; t(11) = −2.32; p = 0.04). However, the enjoyment questionnaire did not show a significant difference. Based on the responses of participants during the interview, this was caused by the simplicity of the VR game.

5. Conclusions and Discussion

In this paper, we presented DFF, a device that generates directional sensations in VR applications through mechanical force concentrations. Our approach was based on inducing force concentrations via controlled motor rotations. A method was developed to optimize and control the rotational speeds and rotation angle intervals to maximize the directional sensations. Three user studies were conducted to investigate the potential impact of the proposed device as a VR force feedback system. Study 1 was conducted to test how effectively DFF can provide directional sensations to the users. The result of this study indicated that the participants can correctly guess the target direction (i.e., up, down, forward, backward) along with the dragged direction. The dragged direction is a direction of force concentration in the target direction. We measured the dragged direction because the “target direction” is difficult to estimate, especially when decomposing 360 degrees to 4 directions. During the interview, the participants commented that it was difficult to distinguish directions if the target direction was not exactly the forward, upward, downward, or backward direction. For example, if the force is concentrated at between forward and downward, it is cognitively difficult to classify the target direction. Also, the current DFF sets the target direction with an absolute value. Thus, a user can feel a discrepancy if his/her hand is not placed at the right coordinate. In this case, the inertial sensor can help set the target direction relative to hand position. Future work on optimizing directions in the cognitive level, and installation of the accelerometer can significantly improve the performance of DFF. In study 2, the participants responded that they could accurately feel the variations in weight. We only tested three weight variations (light, medium, and heavy) in the current study; future work on developing more weight variations can widen the scope of applications of DFF. The result of study 3 showed that DFF increased presence and immersion within the VR environment compared with an experiment without DFF. However, users’ enjoyment within the VR did not show significant differences between the two experiments. This may have been caused by the simplicity of the VR game. During the interview session, the participants responded that the VR game in study 3 was not entertaining. This is because we developed a simple VR game to test the impact of DFF on VR immersivity. As the users indicated that both presence and immersion increased with DFF, we believe that the enjoyment can be significantly improved if more features were added to the game.

We identified several research areas for future work to expand the impact of DFF. As the DFF can generate plausible directional sensations of various degrees via force concentrations, the next step will to investigate the miniaturization of the device. The current version comprises two actuators attached to a cylindrical handle. However, the actuators can be attached to other input devices, including VR joysticks and haptic gloves. Furthermore, safety is an important requirement in VR. The DFF was tested in a lab setting with supervision for safe experimentation. The force concentration requires rotation of weights, therefore protection gear for the DFF needs to be developed. Finally, potential applications for DFF need to be considered in various fields. DFF can be used in educational and professional environments in addition to consumer use. For example, DFF can be used to guide visually impaired people towards the target direction. Additionally, it can be used in VR sport games, and as the interaction interface in social VR.