Towards Shape-Changing Devices: Physical Interface Control with an Active Contour Model

Abstract

This paper proposes a novel tangible interface system to enhance the immersive experience in virtual reality environments. The proposed system allows representing physical properties of the tool, such as the deflection of the elastic rods, which users handle in a virtual and physical environment. This system is composed of two parts; one is an articulated interface to visually represent the physical behavior and the other is a computational algorithm that can compute a set of 6-DOF positions of the links. The proposed computational algorithm extends an active contour model, which is used primarily in computer vision and image processing, incorporating a spring and damping constraint energy functional. An elastic rod is modeled as a series of rigid line segments with a symmetric relationship between neighboring segments, and its shape is modeled to be influenced by energies that are induced by a user, and the external deformation of the spline. The symmetric and sparse properties of the proposed model enable an efficient energy minimization process, and the modification of a number of the line segments. Based on this configuration, we construct an energy generation method based on the positional displacement of the base element to generate the deflection behavior of the contour according to the user’s motion in the space. Therefore, the physical device can simulate a variety of deformable objects by modulating energy parameters during the energy minimization process. Experimental results demonstrate the feasibility of emulating various behaviors of deformable splines, and applying to virtual reality system without interfering with the motion-to-photon latency. We also discuss the method’s limitations and explore its potential.

1. Introduction

The aim of a virtual reality system is to offer a variety of experiences similar to the physical world. Conventional virtual reality systems have focused on visually representing users and their environment using head-mounted displays and arrays of sensors [1,2]. The interface connecting a user to a virtual world requires a large workspace and, thus, reducing a system’s form factor to make it portable and lightweight has been key. A large variety of mobile applications that facilitate gesture-command functionalities within simple 3D worlds and complicated 3D games are being released into the market for mobile platforms. Although these technologies enable enhanced interaction with 3D media, such conventional interfaces offer a limited experience to users by restricting the representation of virtual geometry [3] and producing unnatural physical behavior [4].

To resolve these limitations, haptic feedback is often employed to provide realistic sensations during immersive interactions. In particular, haptic rendering generates a reaction force and torque feedback that prevents the penetration of the user’s fingertip into the object [5,6]. Based on this algorithm, a variety of haptic devices have been developed, which provide a virtual presence to the users. Similarly, the information transferred by mobile devices is focused on the contact between users and virtual objects, and the reaction force is substituted to a cutaneous force [7] by representing a user’s hand to a single point or series of points. This method supports the penetration of a user’s hand into an object, and is acceptable for the selection and manipulation of objects, but makes it difficult to express events lacking contact or collision. To overcome this shortcoming, enhancing the expressiveness of mobile devices has been considered. Studies have focused on changing the form factor of mobile devices from rigid to malleable to express information virtually and physically. This enables bi-directional interaction between the physical and virtual worlds, significantly enhancing usability and the realism of the interaction with a virtual world [8]. This paper proposes an actuated interface system to enrich realism and immersion in virtual environments. This system includes a computational algorithm that models a flexible rod into a series of rigid lines with physical properties. Using the active contour model as a baseline algorithm, the method can generate a variety of physical behaviors according to the user’s position input. Therefore, it can be applied for an intuitive control of a line-shaped graphical object, or for enhancing the level of realism during a mid-air interaction by driving articulated shape-changing interfaces. Specifically, the proposed method enables the simulation of a rod with various mechanical properties; therefore, the graphical object or the articulated device can dynamically change its shape into a fencing sword, a whip, or other shapes. We modeled the interface device as a set of serially-linked splines with spring-damper connections.

This paper is organized as follows: Section 2 outlines mobile interfaces for virtual reality environments. Section 3 describes the proposed method to develop an actuated interface based on active contour model. Section 4 analyzes the possibilities of providing flexible behavior using the proposed method. Section 5 discusses applications of the proposed method, and Section 6 draws conclusions and discusses directions for future research.

2. Related Research

To enhance a user’s immersive experience in virtual environments, several studies have proposed interfaces capable of conveying a variety of information, focusing on information expression via haptic feedback and actuated shape changes.

In haptic feedback, the representation of digital information—especially geometric information—relies on reaction forces and torque feedback generated by mechanically-grounded linkages [9,10,11]. Those linkages can be applied for realistic virtual geometric representation [12,13,14], and for assisting the user’s operation from remote locations by applying virtual fixtures [15,16]. However, since mobile interfaces are mechanically ungrounded, they cannot generate a reaction force that can resist a user’s movement; thus, feedback is, instead, provided by cutaneous and inertial forces. Portable haptic interfaces also aim to simulate the experiences of touching a virtual object with a user’s hand. They convey geometric information relating to virtual objects in terms of cutaneous forces by calculating the penetration depth between a user’s hand and the object. The calculated cutaneous force is transferred on the fingertip via various techniques, such as a push mechanism [17,18] or tilting platform [19]. Inertial forces can also be generated by angular momentum provided by flywheel rotation [20,21]. These methods can deliver information regarding virtual object geometry even when devices are ungrounded. However, they only contact a virtual object, and cannot deliver information when the user is away from the object.

Instead of providing virtual object geometry information during contact, other studies have focused on tool characteristics, such as flexibility and form factor, based on the notion that actuated interfaces can significantly enhance the usability or realism of interaction with a virtual world [8]. Physical shape changes can offer a variety of information to a user [22], and can be achieved by point [23,24] and plane actuation [25,26]. For curved interfaces, Katsumoto et al. [27] proposed an interface that can change its shape and flexibility, composed of multiple passive joints and utilizing a shape locking mechanism to hold the current state. It can switch from a flexible to rigid structure, and vice versa. With this, the user can modulate the interface’s shape into a variety of forms. It has a simple passive structure, but cannot provide information about physical properties other than flexible or rigid states. Nakagaki et al. proposed a shape-changing interface formed by a series of linkages driven by rotary actuators that can form various shapes. This interface can provide a variety of physical affordances [28,29], or mirrors the orientation of the other users for teaching purposes [30] by actively rotating its joints. However, it also did not address the issue of providing information about physical properties or interaction with virtual environments.

3. Proposed Method

3.1. Proposed Concept

Real-life situations often involve touching a rigid object with a rod, leading to a deflection of certain mechanical properties. The rod deforms with respect to the user’s movement, according to its mechanical properties, such as stiffness and damping characteristics. These properties can be perceived as not only inertial forces, but also through the deflection behavior. Virtual reality systems likewise offer scenarios dealing with a variety of tools with different mechanical properties. If an interface system provides the deflection behavior of tools both graphically and physically, enhanced realism and immersion may be achieved. Based on this hypothesis, this paper proposes a mobile interface system to physically and graphically express a variety of tool characteristics (Figure 1). The system is composed of two parts; one is an algorithm that drives the interface system in a flexible manner, and the other is an articulated interface to visually represent this physical behavior.

The proposed method visualizes the physical behavior of the flexible rod according to the user’s position input. Hence, the algorithm should consider the time taken to reflect the user’s motion and to display on the virtual reality system, which is called motion-to-photon latency [31]. To make the user feel the presence in the virtual environment, the value of motion-to-photon latency should not exceed 20 ms. Otherwise, it induces a poor virtual reality experience and may cause motion sickness. The physical rod can be modeled as a mass spring-based discretized model and finite element-based continuum model [32]. Although the finite element-based model can achieve precise results, it requires a large computation time. Studies conducted in the past focused on developing a finite element analysis method to reduce the need for computation resources with increased precision [33,34,35]. However, it is still difficult to obtain a sufficiently low value of motion-to-photon latency. Therefore, in this study, a physical rod is modeled to a mass spring-based discretized structure. Based on this structure, the proposed algorithm to generate the physical response is motivated by an active contour model that is an energy-based spline control method. It is a pseudo-physical model, capable of emulating the physical behavior of the rod. It can also be extended to a variety of scenarios such as geometric information transfer [25] and affective interaction [36,37,38] by assigning additional energy functionals. Based on this idea, a tool is modeled as a serial connection of line segments with torsional springs and dampers, and an energy functional describing deflection behavior is defined in terms of a user’s movement and the angular displacement of neighboring joints. Thus, the deflection behavior is represented by the joint rotation of each line segment. In the same manner, the proposed interface system can have each serial linkage joint actively rotated using an actuator. Therefore, flexibility of a rod can be represented as the rotation speed and phase difference of neighboring joints.

The proposed interface system operates as follows. The user grasps the bottom of the interface, and the position tracker follows the motion of the user. When displacement is detected, the trajectory of each joint is generated based on predefined mechanical characteristics, creating a joint trajectory for the proposed interface generated based on the user’s displacement.

3.2. Active Contour Model

To control each element of the line segments, an active contour model (also known as the snake algorithm) [39] was employed as the baseline algorithm. The main purpose of a typical active contour model is to extract the contour of an object placed in an image. The model develops an energy-minimizing spline to determine the contour of the object. The total energy of the active contour model can be expressed as:

$$E_{snake}^{\ast }={\displaystyle \int }{E}_{int}\left(v\left(s\right)\right)+E_{image}\left(v\left(s\right)\right)+E_{con}\left(v\left(s\right)\right)ds$$

(1)

where $v\left(s\right)$ is the position of each element and s represents space parameters (e.g., the pixel position in image coordinates). The energy includes internal energy (E_int) and external energy, which derives from the image features (E_image) and the external constraint energy (E_con). The internal energy imposes piecewise smoothness among neighboring snake points. This can be calculated as:

$$E_{int}\left(v\left(s\right)\right)=(\alpha \left(s\right){\left|v_s\left(s\right)\right|}^2+\beta \left(s\right){\left|v_{ss}\left(s\right)\right|}^2)/2$$

(2)

where the subscript ‘s’ represents a derivative with respect to ‘s’, and α and β represent the weight of each derivative term. The first term represents the distance between neighboring elements, and the second term represents the curvature between elements. Depending on the weights, the spline acts like a membrane or thin plate. The external energy is responsible for pulling each element of the spline to the contour of the image. This energy was chosen since it relates to image contours such as lines and edges.

The position of spline elements at the local minimum of the energy functional satisfy the Euler-Lagrange equation:

$$\frac{\partial }{\partial s}\left(\alpha \left(s\right)\frac{\partial v}{\partial s}\right)-\frac{{\partial }^2}{\partial s^2}\left(\beta \left(s\right)\frac{{\partial }^{2}v}{\partial s^2}\right)-\nabla E_{ext}=0$$

(3)

Via iterative minimization, the snake elements gradually move towards the positions of the local minima of the energy functional. The minimization process is performed using an implicit Euler method as follows:

$${\mu }_i\frac{{\partial }^2{v}_i}{\partial t^2}+{\gamma }_i\frac{\partial v_i}{\partial t}=\frac{\partial }{\partial s}\left({\alpha }_i \left(s\right)\frac{\partial v_i}{\partial s}\right)-\frac{{\partial }^2}{\partial s^2}\left({\beta }_i\left(s\right)\frac{{\partial }^2{v}_i}{\partial s^2}\right)-\nabla E_{ext}\left(i\right)$$

(4)

where ${\gamma }_i$ is the time step size and ${\mu }_i$ is an inertial coefficient added to impose dynamic effects during the process. Equation (4) can be solved by matrix inversions as described below

$$\begin{array}{l}{𝕧}^{t+1}={\mathbb{B}}^{-1}\left\{4diag(𝕦){𝕧}^t+diag\left(𝕘dt-2𝕦\right){𝕧}^{t-1}-2d{t}^2\nabla E_{ext}\right\}\\ \mathrm{where}\\ 𝕧=\left\{v_0,\dots ,v_{N-1}\right\}, 𝕦=\left\{{\mu }_0,\dots ,{\mu }_{N-1}\right\}, 𝕘=\left\{{\gamma }_0,\dots ,{\gamma }_{N-1}\right\}\\ diag(𝕩)=\left[\begin{array}{ccc}{x}_0& \cdots & 0\\ ⋮& \ddots & ⋮\\ 0& \cdots & x_{N-1}\end{array}\right]\\ \mathbb{B}=\left(2diag(𝕦)+diag(𝕘)dt\right)\mathbb{I}-2d{t}^2\mathbb{A}\\ \mathbb{A}={\left\{A_0^T, \dots ,A_{N-1}^T\right\}}^T\\ A_i=\left[\begin{array}{ccccccccc}0& \cdots & {\beta }_{\mathrm{i}-1}& \left({\alpha }_i-2{\beta }_{i-1}-2{\beta }_i\right)& \stackrel{i-th\mathit{element}}{\overbrace{\left(-{\alpha }_{i+1}-{\alpha }_i+{\beta }_{i-1}+4{\beta }_i+{\beta }_{i+1}\right)}}& \left({\alpha }_{i+1}-2{\beta }_i-2{\beta }_{i+1}\right)& {\beta }_{i+1}& \cdots & 0\end{array}\right]\end{array}$$

(5)

As a result, the matrix $\mathbb{A}$ that determines shape of the spline becomes a pentagonal matrix. If the internal energy parameters are set to constant, the matrix is also bisymmetric. Due to its sparse and symmetric properties, the energy of each element is effectively minimized and the number of the spline elements is easily modified.

3.3. Proposed Method

Our aim is to visualize the behavior of a rod with various physical properties both virtually and physically as described in the previous section. Figure 2 is a block diagram describing the proposed method. It is comprised of a position tracker that follows a user’s motion, and a computational algorithm using an active contour model to represent the rod’s behavior while following the user’s movement. The proposed method includes both graphics and linkage hardware, a geometric constraint to maintain the distance between neighboring elements as constant. To satisfy this constraint, we parameterized the contour in other coordinates to decouple the distance from the effect of the energy functional. In this paper, the spline is modeled in spherical coordinates, and is formed as a series of line segments with constant length (Figure 3). In this configuration, the parameter value is defined as the angular displacement of previous nodes. Each line segment is parameterized as:

$$\begin{array}{l}{P}_i\left({\theta }_i, {\varphi }_i,t\right)=P_{i-1}+R_{i-1}^i{\left[l_i\begin{array}{ccc}\cos {\varphi }_i\sin {\theta }_i& l_i\sin {\varphi }_i& l_i\cos {\varphi }_i\cos {\theta }_i\end{array}\right]}^T, i=2 ~N\\ P_1\left({\theta }_i, {\varphi }_i,t\right)=P_{dev}+R_{dev}^{}{\left[l_1\begin{array}{ccc}\cos {\varphi }_1\sin {\theta }_1& l_1\sin {\varphi }_1& l_1\cos {\varphi }_1\cos {\theta }_1\end{array}\right]}^T\\ R_{i-1}^i=R_{dev}{\displaystyle \prod }_{k=1}^{i-1}ro{t}_x\left({\varphi }_k\right)\cdot ro{t}_y\left({\theta }_k\right), l_i=constant\end{array}$$

(6)

where i is the number of the spline node, and the parameters ${\theta }_i$ and ${\varphi }_i$ are the angles of rotation about the x-axis and the y-axis of each orientation $R_{i-1}^i$.

Based on this configuration, internal energy via the applied change is expressed in Equation (7), as

$$\begin{array}{l}{E}_{int}\left(P\left({\theta }_i, {\varphi }_i\right)\right)=\frac{1}{2}\left({\beta }_i{\left|\left[\begin{array}{c}{\theta }_i\\ {\varphi }_i\end{array}\right]-\left[\begin{array}{c}{\theta }_{i-1}\\ {\varphi }_{i-1}\end{array}\right]\right|}^2\right)\\ {\beta }_i=constant, i=1,\dots ,N\end{array}$$

(7)

Since the proposed algorithm imposes the constant length constraint, the internal energy (Equation (2)) derived from the first derivative becomes zero. Since the spline is parameterized by angles, the second derivative becomes a first derivative. The external energy, which is capable of representing the physical behavior of a rod, is modelled as spring-damper connections between each of the neighboring line segments (Figure 4). These energy functions were expressed as:

$$\begin{gathered} E_{rod}\left(P\left({\theta }_i, {\varphi }_i\right)\right)=E_{spring}\left(P\left({\theta }_i, {\varphi }_i\right)\right)+E_{damper}\left(P\left({\theta }_i, {\varphi }_i\right)\right) \\ \mathrm{where} \{\begin{array}{c}{E}_{spring}\left(P\left({\theta }_i, {\varphi }_i\right)\right)={\left[\begin{array}{cc}\frac{1}{2}{k}_{si}{\left({\theta }_i\right)}^2& \frac{1}{2}{k}_{si}{\left({\varphi }_i\right)}^2\end{array}\right]}^T\\ E_{damper}\left(P\left({\theta }_i, {\varphi }_i\right)\right)={\left[\begin{array}{cc}\frac{1}{2}{c}_{di}{\dot{\theta }}_i^2& \frac{1}{2}{c}_{di}{\dot{\varphi }}_i^2\end{array}\right]}^T\end{array}, i=1,\dots ,N \end{gathered}$$

(8)

where k_s and c_d are coefficients that determine the spring and damping characteristics of each line segment. The external energy functional increases with the angular displacement from the previous spline element, and as its angular velocity increases. Therefore, every spline element becomes aligned with previous line segments having varying stiffness and damping properties.

Applying a user’s positional input to the proposed spline requires changing the linear and rotation displacements by varying the parameter of the spline element (Figure 5). These displacement values affect parameter values of first spline element which propagates through the spline element during the energy minimization process with the rotation displacement is correspond to orientation input. When a new orientation input occurs, the parametric variation is calculated based on angle between the line segment and axes of the orientation (Figure 5a). It can be calculated as:

$$\begin{gathered} {}_t{}^{t+1}P_{1dev}={\left({}^{t+1}{R}_{dev}\right)}^T\left(P_1-P_0\right) \\ d{\theta }_1=-\arctan \left(\frac{{}_t{}^{t+1}P_{1dev.x}}{{}_t{}^{t+1}P_{1dev.z}}\right) \\ d{\varphi }_1=-\arccos \left(\frac{{}_t{}^{t+1}P_{1dev.y}}{L_1}\right) \end{gathered}$$

(9)

where ${}_t{}^{t+1}P_{1dev}$ represents the first line segment vector in the input device coordinate of the subsequent step. In the case of linear displacement, the parametric variation is calculated by using the total derivative relation between the Cartesian and spherical coordinates (Equation (10)). As described in Figure 5b from the linear displacement of the base element, the variation of parameter values occurs.

As a result, the spline with the proposed method operates as illustrated in Figure 6. The translation and rotation of the base element caused by positional input induces an angular displacement of the first element according to Equations (9) and (10). These angular displacements affect the change of internal and external energies of entire line segments (Equations (6)–(9)). During energy minimization, the spline operates according to the defined physical parameters, and continues until the entire energy functional becomes zero. As a result, it reacts as a passively-moving object with mechanical properties. Note that only a single input (base position) is required to control the rest of the joints. The entire operation of the proposed method is also available in the Supplementary Materials.

$$\left[\begin{array}{c}d{\theta }_1\\ d{\varphi }_1\end{array}\right]=\left[\begin{array}{ccc}-\frac{\sin {\varphi }_1\sin {\theta }_1}{L_1}& \frac{\cos {\varphi }_1}{L_1}& -\frac{\cos {\theta }_1\sin {\varphi }_1}{L_1}\\ \frac{\cos {\theta }_1}{L_1\cos {\varphi }_1}& 0& -\frac{\sin {\theta }_1}{L_1\cos {\varphi }_1}\end{array}\right]\left[\begin{array}{c}dx\\ dy\\ dz\end{array}\right]$$

(10)

4. Analysis

As mentioned above, this paper proposes a computational algorithm to represent flexible characteristics with the joint angles of multiple line segments and can be achieved by modulating the stiffness and damping energy coefficients. To investigate its feasibility, the effect of changing the energy coefficient and positional input on the node responses was observed. Initially, the spline was constructed, comprised of four nodes, with each node modelled as 30 mm long with a mass of 0.2 kg. As the base element of the spline was moved by a user, its position was captured by an HTC Vive [40], which is capable of tracking 3D positions and orientations. Although the proposed method can be applied to real-time applications, we used a recorded user’s input to compare the effect of the joint trajectories on the coefficients to understand the effect of energy coefficient modulation only. The recorded trajectory was of an arbitrary circular motion in 3D space, and was sampled at 100 Hz.

At first, we examined the computation time for the proposed method to obtain a low motion-to-photon latency. The computation time is measured from the time that the algorithm receives a new position input to the time taken to calculate the subsequent parameter values of the entire spline for next time step. It is measured under various condition such as type of trajectories and number of spline element. The average computation time according to input trajectories and number of spline element is represented in

Table 1. Average computation time and standard deviation of the proposed method in each time step according to variation of number of spline elements and 3D trajectory. We used an Intel i5-3570 processor for the computation.

Number of Spline Element	Circular (239 Points)	Linear (227 Points)	Rectangular (309 Points)	Shaking (629 Points)	Star (291 Points)
4	194 ± 43 μs	201 ± 46 μs	194 ± 31 μs	199 ± 47 μs	193 ± 39 μs
5	224 ± 35 μs	235 ± 45 μs	229 ± 46 μs	232 ± 41 μs	232 ± 49 μs
6	251 ± 24 μs	260 ± 38 μs	264 ± 42 μs	262 ± 42 μs	261 ± 39 μs
7	286 ± 21 μs	298 ± 37 μs	307 ± 47 μs	306 ± 50 μs	297 ± 39 μs
8	326 ± 32 μs	332 ± 51 μs	332 ± 47 μs	329 ± 39 μs	338 ± 56 μs
9	356 ± 32 μs	366 ± 43 μs	369 ± 39 μs	367 ± 40 μs	366 ± 44 μs
10	389 ± 24 μs	396 ± 36 μs	403 ± 46 μs	403 ± 54 μs	398 ± 44 μs
20	769 ± 34 μs	789 ± 62 μs	787 ± 62 μs	789 ± 60 μs	785 ± 59 μs
30	1257 ± 108 μs	1248 ± 65 μs	1258 ± 72 μs	1255 ± 69 μs	1258 ± 63 μs
40	1823 ± 102 μs	1839 ± 102 μs	1834 ± 116 μs	1846 ± 121 μs	1829 ± 94 μs
50	2463 ± 121 μs	2493 ± 131 μs	2495 ± 117 μs	2496 ± 130 μs	2493 ± 140 μs

. The computation time of the proposed method is affected by the number of the spline element only, and the maximum computation time is 2.5 ms when the number of spline elements is 50. Therefore, it can be shown that the proposed method does not require large computational resources and reliably reflects the user’s input to virtual the environment system without interfering low motion-to-photon latency.

The modulation of these coefficients was performed in two ways, either by modulating the spring constants with a fixed damping constant, or by modulating the damping constant with a fixed spring constant. The responses of the spline while following the recorded trajectory are displayed in Figure 7. These figures show the trajectories of the base element (blue solid line), parametric values of the spline elements as defined in spherical coordinates in (a), the trajectories of each spline element in Cartesian coordinates in (b), and in 3D space in (c), respectively. For the soft rod, with a small spring coefficient, a comparatively large angular displacement and phase delay were observed via the amplitudes of each node compared to the base node. Therefore, the entire spline had a large deflection, growing larger when nearing the rear of the nodes. For the stiff rod, however, the response was the opposite. Since it rapidly adapted to the angular displacement of previous elements, it showed a relatively smaller angular displacement and phase delay. Figure 8 shows the response of the joints based on damping coefficient modulation. The modulation of damping did not affect the angular displacement or phase delay, but the smaller damping coefficients caused ripples during the process. In contrast, for high values of the damping coefficient, the joint angles converged to their rest position without ripples.

In this section, we analyzed the possibility of the proposed method to create various physical rod characteristics. To verify this, an analysis of the response generated by energy coefficient modulation was performed. This showed that the modulation of the spring coefficient effects the angular displacement and phase delay among neighboring nodes, while that of the damping coefficient effects the ripples during the response. Therefore, a variety of responses can be obtained by modulating these energy coefficients.

5. Application

Thus far, we described a novel method to imitate the behavior of a deformable rod with certain mechanical properties based on an active contour model. The proposed method can be applied to various interactive areas, including immersive 3D interaction, especially in augmented reality scenarios. Since the proposed interface can change its mechanical properties dynamically, it can be used by a variety of physical tools. This scenario can be applied to games, such as sword fighting, ribbon gymnastics, and other scenarios which deform its physical shape by directly reflecting the user’s position input. Furthermore, by representing the physical behavior in response to the other user’s position input, the proposed method can be applied to training scenarios conveying dynamic characteristics of the rod. The proposed method can also be extended to various interaction scenarios by defining additional energy functions according to a variety of task objects. For example, by defining an additional energy function to provide geometric information of a virtual object that a user contacts, the interface can lead the users to think that they are touching a virtual object with a deformable tool from the dynamic movement of each link. It can enhance the presence and performance of 3D manipulation tasks in augmented reality by providing enhanced spatial information of virtual objects [23].

The proposed approach could also be applied to control graphical objects. The proposed model can generate trajectories of all spline elements while only measuring base positions. Hence, the method can be applied for the intuitive control of a graphic element that is composed of a series of line segments. For example, it could enhance the process of animation authoring by offering intuitive character control using the animator’s gestures. Note that the trajectory generation of 3D objects that are composed of multiple joints requires a great deal of time and cognitive load on the part of the animator [41]. In fact, a previous effort by Shiratori et al. developed an efficient trajectory generation method based on an animator’s 3D gestures to resolve such issues [42]. Placed in the same context, the method proposed herein can contribute to reducing the time and cost of animating tasks in that the phase and amplitude of each joint is determined only by the base position measurement, implying that an animator needs only to move a 3D positioning device around to simultaneously control the position of multiple joints, regardless of the number of joints.

6. Conclusions

In this paper, we proposed a novel shape-changing interface system to enhance the immersive experience in virtual reality environments. This interface is formed with a serially-linked interface, and its control algorithm is a modified active-contour model. It enables the representation of a tool’s mechanical properties that users handle in virtual and physical environments. By setting the external energy to the spring and damper constraints of each spline element, the proposed method can reflect the physical properties of a rod. One of the main advantages of our approach is that the model can generate joint trajectories of a serially-linked system that consist of multiple joints, based only on the position measurement of the base node, which is normally held by the user’s hand. It can also simulate a variety of flexible objects, such as a fencing sword or a whip, depending on the spring-damper parameters of the proposed algorithm. The proposed method can be applied to not only 3D interaction scenarios, but also to graphic element control, such as during the animation of 3D characters. In addition, it can be applied to other interaction scenarios by defining additional external energy functionals with respect to a user’s intention.

To validate the possibility of various response generation, we analyze the behavior of the spline element by modulating the external energy coefficient according to a user’s arbitrary position input. This showed that the proposed method can provide various responses depending on the external energy coefficients. In the future, we plan to systematically relate the parameter space to the user’s perception by conducting user tests.