**1. Introduction**

General displays have achieved good performance after technical iterations, like liquid crystal display (LCD), which have been widely used in everyday life. Limited to its own structural characteristics, it can only express two-dimensional information and the degree of freedom of interaction is limited to the screen surface [1]. With the introduction of the concept of organic user interfaces (OUIs) [2], the research on deformable user interfaces (DUI) [3,4] and its interaction technology is becoming increasingly extensive. The interactive feature of flexible devices is that we can change their shape in accordance with different functions, such as using deformation as input parameters to open music [5,6], manipulate maps [7], and even prove that it can be used to input passwords, which enhances interactive security during the process of innovation [8], thereby providing an advantage that traditional screen technology does not have. This additional input mode not only greatly increases the bandwidth of human-computer interaction [9] but also inspires researchers to explore greater possibilities of this interaction technology.

The design of the deformation simulation prototype is the premise and foundation of this kind of researches, and the main objective is to simulate the input environment of deformation equipment. Certain researchers use electrical components as deformation sensors [5,8,10–12]. The principle is that when electrical components are deformed, their voltage changes accordingly. Moreover, different degrees of deformation can be simulated by tracking the reflective ball attached to the surface [7,13]. There are some other related work, such as the use of optical sensors [14,15] and the use of fiber-based deformation

**Citation:** Yin, J.; Bai, S.; Han, Y.; Zhang, X.; Deng, S.; Wang, S. The Study of Bending and Twisting Input Modalities in Deformable Interfaces. *Electronics* **2021**, *10*, 2991. https:// doi.org/10.3390/electronics10232991

Academic Editor: J.-C. Chiao

Received: 28 October 2021 Accepted: 28 November 2021 Published: 1 December 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

sensor clusters [16] to detect human intentional deformation, etc. Our implementation scheme is based on the OptiTrack motion capture system. In this system, multiple infrared cameras collect reflective identification points in space in real time and calculate the intersection of multiple direction vectors after completing target correction and recognition in order to determine the world coordinates. In addition, this infrared optical positioning technology can effectively eliminate visible light interference, improve system robustness, and yield a high data transmission rate, which is similar to what has been found in extant research [17].

There are numerous studies on enhancing interactive expressiveness based on prototypes. The works of deformation gesture has attracted the interest of many researchers. The remarkable feature of gesture operation is that it can give users a more intuitive experience and a strong sense of participation in the interaction process. Some researchers directly bind specially designed deformable gestures with functional requirements to improve interactive bandwidth, such as controlling smart TV [18]. A larger trend is that researchers collect user-defined deformation gesture consensus sets [19,20] to understand users' natural interaction habits and solve some practical problems.

In the process of exploring deformation gestures, researchers found that devices using bending gestures are the most effective for rapid response to continuous bipolar variables [21], and suggested to maintain two orders of magnitude [22]. If it is a one handed operation, the deformation gesture in the upper right corner is a better choice [23].

In addition to taking the deformation degree of the equipment directly as the input parameter [5,7,17], many researchers have also discussed the input potential of the combination of deformation and other input methods, such as pressure [24,25], touch [26,27], and even provide new interaction methods for people with visual impairment by combining hearing [28] and kinesthetic [27].

Certain researchers believe that the advantage of deformation input is not only a single command binding. In other words, this interactive technology has a vivid metaphor, which is related to the user's behavior and ideas in a few cases. For example, by reading [13], we can simulate the feeling of actual physical paper on the deformation device. When we need to turn the page [10], it is like using real documents. Scenes in virtual games—such as fishing, archery, or golf games are similar examples [11], and embodied in intelligent wearable healthcare equipment [29].

In this study, we design a prototype called FlexSheet that supports deformation as input and based on the OptiTrack motion capture system. This prototype can provide the interactive characteristics of flexible devices to simulate a deformable input environment (Figure 1). We call this the deformation controller. The deformation controller consists of two thin and soft polyvinyl chloride (PVC) parts of size 20 cm × 20 cm on which the reflective traceable ball is fixed. We use two input forms: bending and twisting. In addition, two visual feedback mechanisms are set up—full vision (FV) and partial vision (PV). Simultaneously, we use an angle to describe the degree of deformation. With the increase in force feedback [20,30], the angle of deformation becomes increasingly larger. For the quantitative analysis of deformation, we mapped the angle to the experimental variables and took the maximum bending angle of 180◦ in the horizontal state. Finally, through a simple input form, we checked the user's ability to control deformation input in order to obtain relevant design guidelines.

The remainder of the paper is organized in the following manner. Section 2 introduces the experimental materials, including input mode, visual feedback, and selection technologies. Section 3 describes controlled experiments used to study deformation input performance. Section 4 analyzes the experimental results from the perspective of three indexes: completion time (MT), error rate (ER), and crossing times (NC); thereafter the best order of magnitude suggestions under the two input modes are provided, respectively. Section 5 discusses and summarizes the paper.

92

*Electronics* **2021**, *10*, x FOR PEER REVIEW 3 of 13

**Figure 1.** Composition and operation of deformation input: (**a**) Deformation input environment diagram; (**b**) Experiment with OptiTrack and FlexSheet. **Figure 1.** Composition and operation of deformation input: (**a**) Deformation input environment diagram; (**b**) Experiment with OptiTrack and FlexSheet. **Figure 1.** Composition and operation of deformation input: (**a**) Deformation input environment diagram; (**b**) Experiment with OptiTrack and FlexSheet.

#### **2. Materials Instruction 2. Materials Instruction 2. Materials Instruction**

*Electronics* **2021**, *10*, x FOR PEER REVIEW 3 of 13

#### *2.1. Input Modes 2.1. Input Modes*

Most flexible screen studies have demonstrated that bending and twisting are two basic types in the design of deformation gestures. Combined with our research content, we provide the following design guidelines for deformation gestures: Most flexible screen studies have demonstrated that bending and twisting are two basic types in the design of deformation gestures. Combined with our research content, we provide the following design guidelines for deformation gestures: *2.1. Input Modes*  Most flexible screen studies have demonstrated that bending and twisting are two basic types in the design of deformation gestures. Combined with our research content,


According to the above design principles, we examined two kinds of deformation gestures: bending and twisting. The basic operation example is depicted in Figure 2. According to the above design principles, we examined two kinds of deformation gestures: bending and twisting. The basic operation example is depicted in Figure 2. 4. The deformation gestures must have powerful powerful metaphors [19]. According to the above design principles, we examined two kinds of deformation

**Figure 2.** Two deformation input models: (**a**) bending; (**b**) twisting. **Figure 2.** Two deformation input models: (**a**) bending; (**b**) twisting. **Figure 2.** Two deformation input models: (**a**) bending; (**b**) twisting.

#### *2.2. Visual Feedback 2.2. Visual Feedback 2.2. Visual Feedback*

As depicted in Figure 3, we used two visual feedback conditions: FV and PV. Under FV conditions the cursor is always visible, as illustrated in Figure 3a,b. Under PV conditions, the cursor is only displayed at the initial state of the selection task and is hidden once the selection task is initiated, as depicted in Figure 3c,d. Under PV conditions, the user must depend on previous experience in FV conditions to complete target selection. This simulates a situation in which expert users may learn to use deformation input intuitively and without excessive cursor support or visual cues, which is similar to the behavior that expert users display when tagging menus [31]. As depicted in Figure 3, we used two visual feedback conditions: FV and PV. Under FV conditions the cursor is always visible, as illustrated in Figure 3a,b. Under PV conditions, the cursor is only displayed at the initial state of the selection task and is hidden once the selection task is initiated, as depicted in Figure 3c,d. Under PV conditions, the user must depend on previous experience in FV conditions to complete target selection. This simulates a situation in which expert users may learn to use deformation input intuitively and without excessive cursor support or visual cues, which is similar to the behavior that expert users display when tagging menus [31]. As depicted in Figure 3, we used two visual feedback conditions: FV and PV. Under FV conditions the cursor is always visible, as illustrated in Figure 3a,b. Under PV conditions, the cursor is only displayed at the initial state of the selection task and is hidden once the selection task is initiated, as depicted in Figure 3c,d. Under PV conditions, the user must depend on previous experience in FV conditions to complete target selection. This simulates a situation in which expert users may learn to use deformation input intuitively and without excessive cursor support or visual cues, which is similar to the behavior that expert users display when tagging menus [31].

**Figure 3.** Target selection experiment interface: (**a**) The initial state with full visual feedback; (**b**) The target selected state with full visual feedback; (**c)** The initial state with partial visual feedback; (**d**) The target selected state with partial visual feedback. **Figure 3.** Target selection experiment interface: (**a**) The initial state with full visual feedback; (**b**) The target selected state with full visual feedback; (**c**) The initial state with partial visual feedback; (**d**) The target selected state with partial visual feedback.

#### *2.3. Selection Technologies 2.3. Selection Technologies*

Once the cursor enters the target, the user must have a mechanism to identify and select options in the general graphical user interface (GUI), as general determination measures are usually performed by clicking the buttons on the mouse. Consequently, we tested three alternative technological solutions to replace the traditional mouse functions: Once the cursor enters the target, the user must have a mechanism to identify and select options in the general graphical user interface (GUI), as general determination measures are usually performed by clicking the buttons on the mouse. Consequently, we tested three alternative technological solutions to replace the traditional mouse functions:


#### sor appears in the target, quickly remove the FlexSheet from the deformed state. **3. Controlled Experiment**

**3. Controlled Experiment**  Through this experiment, our objective was to study the human ability to complete the task of selecting discrete string targets when using deformation inputs. This mainly includes the influence of visual feedback on the selection process and selection results, actual performance of three target selection strategies as well as the control level under Through this experiment, our objective was to study the human ability to complete the task of selecting discrete string targets when using deformation inputs. This mainly includes the influence of visual feedback on the selection process and selection results, actual performance of three target selection strategies as well as the control level under different input modes.

#### different input modes. *3.1. Participants*

tion;

*3.1. Participants*  We recruited 12 participants, including ten males and two females, aged 23–31 years, with an average age of 26 years. All participants are right-handed and have some user We recruited 12 participants, including ten males and two females, aged 23–31 years, with an average age of 26 years. All participants are right-handed and have some user graphical interface experience, but have no experience of using deformation input.

#### graphical interface experience, but have no experience of using deformation input. *3.2. Apparatus and Environment*

*3.2. Apparatus and Environment*  The experiment was conducted on Lenovo computers running Windows10, 3.8 GHz, screen size of 27 inches and resolution of 1920 by 1080 pixels. The experiment also used a motion capture device, OptiTrack, which is composed of eight cameras with Prime13/13W specification and connected through Cisco switch. The deformable plate is made of a plastic sheet (20 cm × 20 cm), and markers accessories is fixed to the prototype. The products The experiment was conducted on Lenovo computers running Windows10, 3.8 GHz, screen size of 27 inches and resolution of 1920 by 1080 pixels. The experiment also used a motion capture device, OptiTrack, which is composed of eight cameras with Prime13/13W specification and connected through Cisco switch. The deformable plate is made of a plastic sheet (20 cm × 20 cm), and markers accessories is fixed to the prototype. The products in the experiment were compiled by the NatNetSDK3.0.1 provided by OptiTrack official website and run in the Visual Studio 2015 environment.

#### in the experiment were compiled by the NatNetSDK3.0.1 provided by OptiTrack official website and run in the Visual Studio 2015 environment. *3.3. Task*

*3.3. Task* 

Our long-term goal is to explore the general design criteria for deformation interac-Our long-term goal is to explore the general design criteria for deformation interactions, with a particular emphasis on developing design guidelines for deformation interactions based on bending and twisting modes.

tions, with a particular emphasis on developing design guidelines for deformation interactions based on bending and twisting modes. We used a serial target selection task. Control the cursor to move vertically by deforming the FlexSheet. The deformation value of 180◦ is uniformly mapped to 256 pixels. Within the distance of 256 pixels, we draw a set of continuous rectangular targets. In each

experiment, a target rectangle to be selected will be highlighted in purple. The user's task is to apply appropriate deformation and move the cursor to the target rectangle with different distances and different widths. When the cursor enters the range of the target rectangle, the target rectangle changes from purple to green to give visual feedback to the subject.

It should be noted that the cursor is displayed in the initial state of the task. Since we provide two different visual feedback mechanisms, when the cursor starts to move, the full visual feedback supports the full display of the cursor, while the cursor under partial visual feedback will be hidden.

After the target to be selected gives color feedback, the subject needs to use the three selection technologies we provide to quickly complete the target selection, and the experimental task is over. Subjects will then repeat this task many times under different combinations of conditions.

#### *3.4. Performance Measures*

MT is the dependent variable of the experimental task, which represents the time it takes to select each target, defined as the time (milliseconds) from moving the cursor after the target appears until the target is selected; ER means the number of incorrect selections in the target selection task accounts for the total number of selections the ratio; NC represents the cumulative number of crossovers that enter the target and then leave the target when selecting each target. The functions of these evaluation indicators complement each other. ER and MT provide us with an overview of the overall success rate, and NC provides us with an indication of the degree of control of the user's application of deformed input.

#### *3.5. Procedure and Design*

A within-subjects full factorial design with repeated measures was used. The independent variables were selection modes (Bending and Twisting), selection technologies (Click, Dwell and Quick Release), visual feedback condition (FV, PV), the distance from the starting point to the target (D = 37, 72, 109, 144) and the target width (W = 15, 18, 22.5, 30, 45). The units of distance and width are represented by the deformation angle.

For each visual feedback, each participant will be exposed to two input modes. In each input mode, participants need to complete the experiments of all three selection technologies. For each selection technology, 20 D-W combination conditions are included and presented in random order. The experiment order was counterbalanced among participants using a Latin Square design.Each participant repeated the entire task three times to complete a total of 8640 individual tasks. In summary, the experiment consisted of:


Before the experiment, we explained the task, and performed a brief demonstration. Participants were given a short warm-up set of trials to familiarize themselves with input modes, visual feedback and selection technologies. Participants were instructed to perform the task as quickly and accurately as possible. Participants can take a short break during the experiment. The experiment lasted approximately 1 hour for each participant. To analyze the experimental results, we recorded the relevant parameters of each target selection task, including input modes, target size, target distance, time spent, errors, and crossing times.

#### **4. Results**

We analyzed the collected measures by conducting a repeated measures ANOVA (α = 0.05) that used the three-interaction technique and two-visual feedback condition as independent variables. Further, we used Greenhouse-Geisser corrected values in the analy-

sis when the assumption of sphericity was violated (tested with Mauchly's test). Pairwise t-tests with Bonferroni corrections were used for post hoc tests. The test was considered an outlier if the time required to complete the task exceeded two standard deviations of the average completion time of the task. A total of 282 outliers were eliminated, which accounted for 3.3% of the collected data. sis when the assumption of sphericity was violated (tested with Mauchly's test). Pairwise t-tests with Bonferroni corrections were used for post hoc tests. The test was considered an outlier if the time required to complete the task exceeded two standard deviations of the average completion time of the task. A total of 282 outliers were eliminated, which accounted for 3.3% of the collected data. sis when the assumption of sphericity was violated (tested with Mauchly's test). Pairwise t-tests with Bonferroni corrections were used for post hoc tests. The test was considered an outlier if the time required to complete the task exceeded two standard deviations of the average completion time of the task. A total of 282 outliers were eliminated, which accounted for 3.3% of the collected data.

selection task, including input modes, target size, target distance, time spent, errors, and

selection task, including input modes, target size, target distance, time spent, errors, and

We analyzed the collected measures by conducting a repeated measures ANOVA (α = 0.05) that used the three-interaction technique and two-visual feedback condition as independent variables. Further, we used Greenhouse-Geisser corrected values in the analy-

We analyzed the collected measures by conducting a repeated measures ANOVA (α = 0.05) that used the three-interaction technique and two-visual feedback condition as independent variables. Further, we used Greenhouse-Geisser corrected values in the analy-

#### *4.1. Time 4.1. Time 4.1. Time*

crossing times.

crossing times.

**4. Results** 

**4. Results** 

As depicted in Figure 4, in bending mode, we found that different selection techniques had no significant effect on performance time (F1.342,14.764 = 2.504, p = 0.129). Further, we found no significant interaction between interaction technique and visual feedback (F2,22 = 3.286, p = 0.056), as shown in Figure 5. As depicted in Figure 4, in bending mode, we found that different selection techniques had no significant effect on performance time (F1.342,14.764 = 2.504, p = 0.129). Further, we found no significant interaction between interaction technique and visual feedback (F2,22 = 3.286, p = 0.056), as shown in Figure 5. As depicted in Figure 4, in bending mode, we found that different selection techniques had no significant effect on performance time (F1.342,14.764 = 2.504, p = 0.129). Further, we found no significant interaction between interaction technique and visual feedback (F2,22 = 3.286, p = 0.056), as shown in Figure 5.

*Electronics* **2021**, *10*, x FOR PEER REVIEW 6 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 6 of 13

**Figure 4.** Elapsed time with diverse selection techniques in bending mode (error bars indicate 95% confidence intervals). **Figure 4.** Elapsed time with diverse selection techniques in bending mode (error bars indicate 95% confidence intervals). **Figure 4.** Elapsed time with diverse selection techniques in bending mode (error bars indicate 95% confidence intervals).

**Figure 5.** Elapsed time for each selection technique under diverse visual feedback in bending mode (error bars indicate 95% confidence intervals). **Figure 5.** Elapsed time for each selection technique under diverse visual feedback in bending mode (error bars indicate 95% confidence intervals). **Figure 5.** Elapsed time for each selection technique under diverse visual feedback in bending mode (error bars indicate 95% confidence intervals).

Although there was no significant difference in the performance of time between the three interactive technologies, it can be found that the most time was spent in Dwell, followed by Quick Release, and the then Click. Although there was no significant difference in the performance of time between the three interactive technologies, it can be found that the most time was spent in Dwell, followed by Quick Release, and the then Click. Although there was no significant difference in the performance of time between the three interactive technologies, it can be found that the most time was spent in Dwell, followed by Quick Release, and the then Click.

As illustrated in Figure 6, in twisting mode, we found that different selection techniques had no significant effect on performance time (F1.165,12.818 = 2.284, p = 0.154). As de-As illustrated in Figure 6, in twisting mode, we found that different selection techniques had no significant effect on performance time (F1.165,12.818 = 2.284, p = 0.154). As de-As illustrated in Figure 6, in twisting mode, we found that different selection techniques had no significant effect on performance time (F1.165,12.818 = 2.284, p = 0.154). As depicted in Figure 7, we found a significant interaction between technology and visual feedback (F1.171,12.882 = 4.749, p = 0.044). The visual feedback condition has a significant effect on Click time (p = 0.002) as well as on Dwell and Quick Release times (p < 0.001, p = 0.002). The overall trend is that the Quick Release time under PV is significantly higher than that under FV (p = 0.004, p = 0.034); there was no significant difference among all technologies (p > 0.1).

*Electronics* **2021**, *10*, x FOR PEER REVIEW 7 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 7 of 13

**Figure 6.** Elapsed time with diverse selection techniques in twisting mode (error bars indicate 95% confidence intervals). **Figure 6.** Elapsed time with diverse selection techniques in twisting mode (error bars indicate 95% confidence intervals). **Figure 6.** Elapsed time with diverse selection techniques in twisting mode (error bars indicate 95% confidence intervals). **Figure 6.** Elapsed time with diverse selection techniques in twisting mode (error bars indicate 95% confidence intervals).

picted in Figure 7, we found a significant interaction between technology and visual feedback (F1.171,12.882 = 4.749, p = 0.044). The visual feedback condition has a significant effect on Click time (p = 0.002) as well as on Dwell and Quick Release times (p < 0.001, p = 0.002). The overall trend is that the Quick Release time under PV is significantly higher than that under FV (p = 0.004, p = 0.034); there was no significant difference among all technologies

picted in Figure 7, we found a significant interaction between technology and visual feedback (F1.171,12.882 = 4.749, p = 0.044). The visual feedback condition has a significant effect on Click time (p = 0.002) as well as on Dwell and Quick Release times (p < 0.001, p = 0.002). The overall trend is that the Quick Release time under PV is significantly higher than that under FV (p = 0.004, p = 0.034); there was no significant difference among all technologies

picted in Figure 7, we found a significant interaction between technology and visual feedback (F1.171,12.882 = 4.749, p = 0.044). The visual feedback condition has a significant effect on Click time (p = 0.002) as well as on Dwell and Quick Release times (p < 0.001, p = 0.002). The overall trend is that the Quick Release time under PV is significantly higher than that under FV (p = 0.004, p = 0.034); there was no significant difference among all technologies

**Figure 7.** Elapsed time for each selection technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals). **Figure 7.** Elapsed time for each selection technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals). **Figure 7.** Elapsed time for each selection technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals). **Figure 7.** Elapsed time for each selection technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals).

#### *4.2. Accuracy 4.2. Accuracy 4.2. Accuracy*

(p > 0.1).

(p > 0.1).

(p > 0.1).

In bending mode, we found that selection technology has a significant effect on the accuracy (F1.243,13.673 = 7.9, p = 0.01), as illustrated in Figure 8. The post hoc tests revealed no significant differences among the technologies except Click and Quick Release (p = 0.021) (p > 0.07). *4.2. Accuracy*  In bending mode, we found that selection technology has a significant effect on the accuracy (F1.243,13.673 = 7.9, p = 0.01), as illustrated in Figure 8. The post hoc tests revealed no significant differences among the technologies except Click and Quick Release (p = 0.021) (p > 0.07). In bending mode, we found that selection technology has a significant effect on the accuracy (F1.243,13.673 = 7.9, p = 0.01), as illustrated in Figure 8. The post hoc tests revealed no significant differences among the technologies except Click and Quick Release (p = 0.021) (p > 0.07). In bending mode, we found that selection technology has a significant effect on the accuracy (F1.243,13.673 = 7.9, p = 0.01), as illustrated in Figure 8. The post hoc tests revealed no significant differences among the technologies except Click and Quick Release (p = 0.021) (p > 0.07).

**Figure 8.** Error rate with diverse selection techniques in bending mode (error bars indicate 95% confidence intervals). **Figure 8.** Error rate with diverse selection techniques in bending mode (error bars indicate 95% confidence intervals). **Figure 8.** Error rate with diverse selection techniques in bending mode (error bars indicate 95% confidence intervals). **Figure 8.** Error rate with diverse selection techniques in bending mode (error bars indicate 95% confidence intervals).

As illustrated in Figure 9, there was a significant interaction between selection techniques and visual feedback (F2,22 = 3.609, p = 0.044).

The level of visual feedback had no obvious effect on the error rate for Click (p = 0.604); however, a significant effect was found on the error rates for Dwell and Quick Release (p = 0.006, p = 0.005). The general trend was that the error under PV was significantly higher than that under FV. The post hoc tests revealed that under FV conditions, there were no significant differences between technologies; under PV conditions (p > 0.1), Click and

Quick Release were significant (p = 0.012). In addition, there are no significant differences among other technologies (p > 0.1). and Quick Release were significant (p = 0.012). In addition, there are no significant differences among other technologies (p > 0.1). higher than that under FV. The post hoc tests revealed that under FV conditions, there were no significant differences between technologies; under PV conditions (p > 0.1), Click and Quick Release were significant (p = 0.012). In addition, there are no significant differ-

As illustrated in Figure 9, there was a significant interaction between selection tech-

The level of visual feedback had no obvious effect on the error rate for Click (p = 0.604); however, a significant effect was found on the error rates for Dwell and Quick Release (p = 0.006, p = 0.005). The general trend was that the error under PV was significantly higher than that under FV. The post hoc tests revealed that under FV conditions, there were no significant differences between technologies; under PV conditions (p > 0.1), Click and Quick Release were significant (p = 0.012). In addition, there are no significant differ-

As illustrated in Figure 9, there was a significant interaction between selection tech-

The level of visual feedback had no obvious effect on the error rate for Click (p = 0.604); however, a significant effect was found on the error rates for Dwell and Quick Release (p = 0.006, p = 0.005). The general trend was that the error under PV was significantly higher than that under FV. The post hoc tests revealed that under FV conditions, there were no significant differences between technologies; under PV conditions (p > 0.1), Click

The level of visual feedback had no obvious effect on the error rate for Click (p = 0.604); however, a significant effect was found on the error rates for Dwell and Quick Release (p = 0.006, p = 0.005). The general trend was that the error under PV was significantly

As illustrated in Figure 9, there was a significant interaction between selection tech-

*Electronics* **2021**, *10*, x FOR PEER REVIEW 8 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 8 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 8 of 13

niques and visual feedback (F2,22 = 3.609, p = 0.044).

niques and visual feedback (F2,22 = 3.609, p = 0.044).

ences among other technologies (p > 0.1).

niques and visual feedback (F2,22 = 3.609, p = 0.044).

**Figure 9.** Error rate for each selection technique under diverse visual feedback types in bending mode (error bars indicate 95% confidence intervals). **Figure 9.** Error rate for each selection technique under diverse visual feedback types in bending mode (error bars indicate 95% confidence intervals). As depicted in Figure 10, in twisting mode, we found a main effect on the error rate of different selection techniques (F1.167,12.839 = 11.812, p < 0.001). **Figure 9.** Error rate for each selection technique under diverse visual feedback types in bending

As depicted in Figure 10, in twisting mode, we found a main effect on the error rate of different selection techniques (F1.167,12.839 = 11.812, p < 0.001). As depicted in Figure 10, in twisting mode, we found a main effect on the error rate of different selection techniques (F1.167,12.839 = 11.812, p < 0.001). The post hoc tests showed significant differences among all selection technologies (p = 0.02, p = 0.00, p = 0.047). Further, as illustrated in Figure 11, we found no significant effect between selection techniques and visual feedback (F1.305,14.35 = 1.406, p > 0.2). mode (error bars indicate 95% confidence intervals). As depicted in Figure 10, in twisting mode, we found a main effect on the error rate

The post hoc tests showed significant differences among all selection technologies (p

**Figure 10.** Error rate with different selection techniques in twisting mode (error bars indicate 95% confidence intervals). **Figure 10.** Error rate with different selection techniques in twisting mode (error bars indicate 95% confidence intervals).

**Figure 10.** Error rate with different selection techniques in twisting mode (error bars indicate 95% confidence intervals). The post hoc tests showed significant differences among all selection technologies (p = 0.02, p = 0.00, p = 0.047). Further, as illustrated in Figure 11, we found no significant effect between selection techniques and visual feedback (F1.305,14.35 = 1.406, p > 0.2). **Figure 10.** Error rate with different selection techniques in twisting mode (error bars indicate 95% confidence intervals).

(error bars indicate 95% confidence intervals). **Figure 11.** Error rate for each selection technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals). **Figure 11.** Error rate for each selection technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals).

#### *4.3. Indication of Control*

As illustrated in Figure 12, in bending mode, we found that this technique had a significant impact on the number of performance crosses (F1.254,13.791 = 8.527, p = 0.008). The post hoc tests revealed that there was no significant difference among all technologies, except Dwell and Quick Release (p = 0.022) (p > 0.05).

except Dwell and Quick Release (p = 0.022) (p > 0.05).

except Dwell and Quick Release (p = 0.022) (p > 0.05).

except Dwell and Quick Release (p = 0.022) (p > 0.05).

*Electronics* **2021**, *10*, x FOR PEER REVIEW 9 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 9 of 13

and visual feedback (F1.769,19.464 = 2.771, p = 0.092).

and visual feedback (F1.769,19.464 = 2.771, p = 0.092).

*4.3. Indication of Control* 

*4.3. Indication of Control* 

*4.3. Indication of Control* 

**Figure 12.** NC rate with diverse techniques in bending mode (error bars indicate 95% confidence intervals). **Figure 12.** NC rate with diverse techniques in bending mode (error bars indicate 95% confidence intervals). **Figure 12.** NC rate with diverse techniques in bending mode (error bars indicate 95% confidence intervals).

As illustrated in Figure 12, in bending mode, we found that this technique had a significant impact on the number of performance crosses (F1.254,13.791 = 8.527, p = 0.008). The post hoc tests revealed that there was no significant difference among all technologies,

As illustrated in Figure 12, in bending mode, we found that this technique had a significant impact on the number of performance crosses (F1.254,13.791 = 8.527, p = 0.008). The post hoc tests revealed that there was no significant difference among all technologies,

As presented in Figure 13, there was no significant interaction between technology

As presented in Figure 13, there was no significant interaction between technology

As illustrated in Figure 12, in bending mode, we found that this technique had a significant impact on the number of performance crosses (F1.254,13.791 = 8.527, p = 0.008). The post hoc tests revealed that there was no significant difference among all technologies,

As presented in Figure 13, there was no significant interaction between technology and visual feedback (F1.769,19.464 = 2.771, p = 0.092). **Figure 12.** NC rate with diverse techniques in bending mode (error bars indicate 95% confidence intervals).

As depicted in Figure 14, in twisting mode, we found that different techniques had a significant effect on the NC (F1.353,14.885 = 17.716, p < 0.001). **Figure 13.** NC rate for each technique under diverse visual feedback in bending mode (error bars indicate 95% confidence intervals). **Figure 13.** NC rate for each technique under diverse visual feedback in bending mode (error bars indicate 95% confidence intervals). As depicted in Figure 14, in twisting mode, we found that different techniques had a significant effect on the NC (F1.353,14.885 = 17.716, p < 0.001).

There was no significant difference between Click and Dwell (p = 0.032), as revealed by the post hoc tests, but there was no significant difference between Click and Dwell (p As depicted in Figure 14, in twisting mode, we found that different techniques had a significant effect on the NC (F1.353,14.885 = 17.716, p < 0.001). As depicted in Figure 14, in twisting mode, we found that different techniques had a significant effect on the NC (F1.353,14.885 = 17.716, p < 0.001). There was no significant difference between Click and Dwell (p = 0.032), as revealed by the post hoc tests, but there was no significant difference between Click and Dwell (p = 0.066, p = 0.069).

= 0.066, p = 0.069).

intervals).

**Figure 14.** NC rate with diverse techniques in twisting mode (error bars indicate 95% confidence **Figure 14.** NC rate with diverse techniques in twisting mode (error bars indicate 95% confidence intervals). **Figure 14.** NC rate with diverse techniques in twisting mode (error bars indicate 95% confidence intervals).

**Figure 14.** NC rate with diverse techniques in twisting mode (error bars indicate 95% confidence intervals). There was no significant difference between Click and Dwell (p = 0.032), as revealed by the post hoc tests, but there was no significant difference between Click and Dwell (p = 0.066, p = 0.069).

As depicted in Figure 15, there was no significant interaction between technology and visual feedback (F1.218,13.397 = 5.783, p = 0.026). In addition, the level of visual feedback has no significant effect on the NC of Click (p = 0.084), Dwell (p = 0.169), and Quick Release (p = 0.009). The general trend was that the NC under PV condition is significantly higher than that under FV condition. In FV condition, Click and Quick Release (p = 0.011), Dwell and Quick Release (p = 0.007) differed significantly among different technologies, except

*Electronics* **2021**, *10*, x FOR PEER REVIEW 10 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 10 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 10 of 13

Click and Dwell (p = 1). Under PV conditions, Click and Quick Release (p = 0.01) differed significantly from Dwell and Quick Release (p = 0.007), except for Click and Dwell (p = 1). and Quick Release (p = 0.007) differed significantly among different technologies, except Click and Dwell (p = 1). Under PV conditions, Click and Quick Release (p = 0.01) differed significantly from Dwell and Quick Release (p = 0.007), except for Click and Dwell (p = 1). and Quick Release (p = 0.007) differed significantly among different technologies, except Click and Dwell (p = 1). Under PV conditions, Click and Quick Release (p = 0.01) differed significantly from Dwell and Quick Release (p = 0.007), except for Click and Dwell (p = 1). and Quick Release (p = 0.007) differed significantly among different technologies, except Click and Dwell (p = 1). Under PV conditions, Click and Quick Release (p = 0.01) differed significantly from Dwell and Quick Release (p = 0.007), except for Click and Dwell (p = 1).

As depicted in Figure 15, there was no significant interaction between technology and visual feedback (F1.218,13.397 = 5.783, p = 0.026). In addition, the level of visual feedback has no significant effect on the NC of Click (p = 0.084), Dwell (p = 0.169), and Quick Release (p = 0.009). The general trend was that the NC under PV condition is significantly higher than that under FV condition. In FV condition, Click and Quick Release (p = 0.011), Dwell

As depicted in Figure 15, there was no significant interaction between technology and visual feedback (F1.218,13.397 = 5.783, p = 0.026). In addition, the level of visual feedback has no significant effect on the NC of Click (p = 0.084), Dwell (p = 0.169), and Quick Release (p = 0.009). The general trend was that the NC under PV condition is significantly higher than that under FV condition. In FV condition, Click and Quick Release (p = 0.011), Dwell

As depicted in Figure 15, there was no significant interaction between technology and visual feedback (F1.218,13.397 = 5.783, p = 0.026). In addition, the level of visual feedback has no significant effect on the NC of Click (p = 0.084), Dwell (p = 0.169), and Quick Release (p = 0.009). The general trend was that the NC under PV condition is significantly higher than that under FV condition. In FV condition, Click and Quick Release (p = 0.011), Dwell

**Figure 15.** NC rate for each technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals). **Figure 15.** NC rate for each technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals). **Figure 15.** NC rate for each technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals). **Figure 15.** NC rate for each technique under diverse visual feedback in twisting mode (error bars indicate 95% confidence intervals).

#### *4.4. Optimum Number of Deformation Levels 4.4. Optimum Number of Deformation Levels 4.4. Optimum Number of Deformation Levels 4.4. Optimum Number of Deformation Levels*

able number in bending mode.

One of the intentions of this research is to ascertain the number of independent bending levels (nLevels) a user can distinguish under a given level of device performance. Except the Quick Release, when nLevels ≤ 8, all selection technology error values between 1–2%, and there was no significant difference in the error rate (Figure 16). Except Quick Release, when nLevels ≤ 6, the NC of FV was stable around 1.1 in each selection technology; moreover, all the selection technologies of NC had no significant differences (Figure 17). In bending mode, these results indicate that six levels of deformation were a reason-One of the intentions of this research is to ascertain the number of independent bending levels (nLevels) a user can distinguish under a given level of device performance. Except the Quick Release, when nLevels ≤ 8, all selection technology error values between 1–2%, and there was no significant difference in the error rate (Figure 16). Except Quick Release, when nLevels ≤ 6, the NC of FV was stable around 1.1 in each selection technology; moreover, all the selection technologies of NC had no significant differences (Figure 17). In bending mode, these results indicate that six levels of deformation were a reasonable number in bending mode. One of the intentions of this research is to ascertain the number of independent bending levels (nLevels) a user can distinguish under a given level of device performance. Except the Quick Release, when nLevels ≤ 8, all selection technology error values between 1–2%, and there was no significant difference in the error rate (Figure 16). Except Quick Release, when nLevels ≤ 6, the NC of FV was stable around 1.1 in each selection technology; moreover, all the selection technologies of NC had no significant differences (Figure 17). In bending mode, these results indicate that six levels of deformation were a reasonable number in bending mode. One of the intentions of this research is to ascertain the number of independent bending levels (nLevels) a user can distinguish under a given level of device performance. Except the Quick Release, when nLevels ≤ 8, all selection technology error values between 1–2%, and there was no significant difference in the error rate (Figure 16). Except Quick Release, when nLevels ≤ 6, the NC of FV was stable around 1.1 in each selection technology; moreover, all the selection technologies of NC had no significant differences (Figure 17). In bending mode, these results indicate that six levels of deformation were a reasonable number in bending mode.

**Figure 16.** ER with nLevels under FV condition (bending mode). **Figure 16.** ER with nLevels under FV condition (bending mode). **Figure 16.** ER with nLevels under FV condition (bending mode). **Figure 16.** ER with nLevels under FV condition (bending mode).

**Figure 17.** NC with nLevels under FV condition (bending mode).

Except the Quick Release, when nLevels ≤ 8 and when all selection technology error values are between 1% and 3%, there were no significant differences between the error rates of the technology selected (Figure 18). In addition to Quick Release, when nLevels ≤ 8, of

all stable technologies of NC in 1.1%, there was no significant difference (Figure 19). In twisting mode, these results indicate that eight levels of deformation were a reasonable number for humans to comfortably control in twisting mode. rates of the technology selected (Figure 18). In addition to Quick Release, when nLevels ≤ 8, of all stable technologies of NC in 1.1%, there was no significant difference (Figure 19). In twisting mode, these results indicate that eight levels of deformation were a reasonable number for humans to comfortably control in twisting mode. rates of the technology selected (Figure 18). In addition to Quick Release, when nLevels ≤ 8, of all stable technologies of NC in 1.1%, there was no significant difference (Figure 19). In twisting mode, these results indicate that eight levels of deformation were a reasonable number for humans to comfortably control in twisting mode.

Except the Quick Release, when nLevels ≤ 8 and when all selection technology error values are between 1% and 3%, there were no significant differences between the error

Except the Quick Release, when nLevels ≤ 8 and when all selection technology error values are between 1% and 3%, there were no significant differences between the error

*Electronics* **2021**, *10*, x FOR PEER REVIEW 11 of 13

*Electronics* **2021**, *10*, x FOR PEER REVIEW 11 of 13

**Figure 17.** NC with nLevels under FV condition (bending mode).

**Figure 17.** NC with nLevels under FV condition (bending mode).

**Figure 18.** ER with nLevels under the FV condition (twisting mode). **Figure 18.** ER with nLevels under the FV condition (twisting mode). **Figure 18.** ER with nLevels under the FV condition (twisting mode).

**Figure 19.** ER with nLevels under the FV condition (twisting mode). **Figure 19.** ER with nLevels under the FV condition (twisting mode). **Figure 19.** NC with nLevels under the FV condition (twisting mode).

## **5. Discussion and Conclusions**

**5. Discussion and Conclusions**  In this study, we examined the interaction technology of a flexible screen and designed a simulation prototype called FlexSheet, which can support deformation inputs. Among with the OptiTrack system, a deformation input environment was constructed. In this context, we summarized two basic deformation models: bending and twisting. In order to study the ability of humans to control deformation, we introduced the deformation angle as a reflection of the degree of deformation. Further, we designed a discrete object selection experiment using three selection techniques (Click, Dwell, and Quick Release) and two types of visual feedback FV and PV to simulate novice and expert users. According to the experimental performance of participants, a series of performance indicators were analyzed in detail. The results revealed that in two input modes, less time was spent on simple Click than Dwell and Quick Release, a finding that is in keeping with our hypothesis. The main reason is that the Click selection is more in line with a user's operating habits. Another reason is that the other two selection methods need to be implemented with a certain time delay, thereby reducing the selection efficiency. In terms of accuracy, both models had similar performance, with the lowest error rate for Click, followed by Dwell, and then Quick Release. In terms of stability (NC), from the input mode, the number of twisting intersections was lesser than that of bending intersections, but the overall **5. Discussion and Conclusions**  In this study, we examined the interaction technology of a flexible screen and designed a simulation prototype called FlexSheet, which can support deformation inputs. Among with the OptiTrack system, a deformation input environment was constructed. In this context, we summarized two basic deformation models: bending and twisting. In order to study the ability of humans to control deformation, we introduced the deformation angle as a reflection of the degree of deformation. Further, we designed a discrete object selection experiment using three selection techniques (Click, Dwell, and Quick Release) and two types of visual feedback FV and PV to simulate novice and expert users. According to the experimental performance of participants, a series of performance indicators were analyzed in detail. The results revealed that in two input modes, less time was spent on simple Click than Dwell and Quick Release, a finding that is in keeping with our hypothesis. The main reason is that the Click selection is more in line with a user's operating habits. Another reason is that the other two selection methods need to be implemented with a certain time delay, thereby reducing the selection efficiency. In terms of accuracy, both models had similar performance, with the lowest error rate for Click, followed by Dwell, and then Quick Release. In terms of stability (NC), from the input mode, the number of twisting intersections was lesser than that of bending intersections, but the overall In this study, we examined the interaction technology of a flexible screen and designed a simulation prototype called FlexSheet, which can support deformation inputs. Among with the OptiTrack system, a deformation input environment was constructed. In this context, we summarized two basic deformation models: bending and twisting. In order to study the ability of humans to control deformation, we introduced the deformation angle as a reflection of the degree of deformation. Further, we designed a discrete object selection experiment using three selection techniques (Click, Dwell, and Quick Release) and two types of visual feedback FV and PV to simulate novice and expert users. According to the experimental performance of participants, a series of performance indicators were analyzed in detail. The results revealed that in two input modes, less time was spent on simple Click than Dwell and Quick Release, a finding that is in keeping with our hypothesis. The main reason is that the Click selection is more in line with a user's operating habits. Another reason is that the other two selection methods need to be implemented with a certain time delay, thereby reducing the selection efficiency. In terms of accuracy, both models had similar performance, with the lowest error rate for Click, followed by Dwell, and then Quick Release. In terms of stability (NC), from the input mode, the number of twisting intersections was lesser than that of bending intersections, but the overall performance is similar. From the perspective of different selection technologies, the average number ofintersections to Dwell is the lowest, followed by Click, and then Quick Release. This is consistent with the feedback of participants that rapid release is more difficult to control than other technologies, and it is easier to remain within the target range using bending input, which leads to higher time cost, error rate, and crossing times. According to our

measurement and analysis, the maximum deformation grade can be divided into six levels in bending and eight levels in twisting. The division basis of deformation levels comes from the mapping of deformation angle to width variables in the task, and the division results are reflected by user feedback. It is worth noting that in each index analysis, the overall performance of FV is better than that of PV.

Further, although we discussed the specific performance of the two deformation input modes, respectively, we believe that the research on the combination of the two input modes has good potential in future research on flexible screen technology. In addition, we believe that the research on ergonomic models based on deformation input is of great significance. We are optimistic regarding the future of flexible interactive technology, and our study can provide designers a few useful guidelines for designing interactive techniques based on flexible screen.

**Author Contributions:** Conceptualization, funding acquisition, project administration, supervision, J.Y.; methodology, data analysis, experimental verification, writing, S.B.; investigation, supervision, Y.H.; investigation, supervision, X.Z.; software, data curation, S.D.; investigation, supervision, S.W. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was supported by the National Natural Science Foundation of China (61741206).

**Acknowledgments:** The authors would like to thank the editor and anonymous reviewers for their useful comments for improving the quality of this paper.

**Conflicts of Interest:** The authors declare no conflict of interest.
