5.1.5. Haptic Key Results

In [30], the actuator design of our haptic prototype (Figure 11) was exposed and its ability to reproduce the force on the piano key was tested. In the present section, we analyze in further detail the haptic force felt by the pianist in various conditions. For this purpose, we take advantage of the ROS-ROBOTRAN coupling presented above in order to investigate various settings of the piano action like the position of the button height or the mass of the hammer.

The force felt by the pianist is Fhaptic of Figure 10, which depends of course of Fmod and Fact but also on the physical components of the prototype. To characterize Fhaptic, an external actuator *Faulhaber LM1247* has been used to apply the same position-driven profile that corresponds to one full key dip at 10 mm/s. The Fhaptic force can be deduced from the external actuator current measurement on both the action demonstrator and the prototype, see Figure 23.

**Figure 23.** Piano haptic key: measurements of the force felt by the pianist Fhaptic through an external linear actuator for the Renner® demonstrator (**up**) and the haptic prototype (**bottom**) [30].

We first reproduced and checked the test in [30] with the additional sensors presented and validated above. Figure 24 shows Fhaptic versus the key vertical position measured at its tip. A key at rest refers to position zero and position 9.5 mm to a fully depressed key, as proposed by [53]. We clearly retrieve the phases A - B - C - D as described in [30].

**Figure 24.** Haptic piano key force validation: prototype comparison with the Renner® demonstrator (results confirming those obtained in [30]).

In addition, similar behaviors can be observed in the profiles of Figure 24, despite some differences due to the physical components that differ between the prototype and the demonstrator. For instance, the blue curve shows the action behavior, without the damper, illustrating its role. The measure on the action demonstrator has been performed three times in a row to highlight the consistency between experiments.

In short, the crucial *escapement* phenomenon occurs during phase C in Figure 24. The difference in the force amplitude between the prototype and the reference may be due to residual inaccuracies in the MBS model.

To illustrate the interest of having a multibody model included in the haptic prototype, variations of the action parameters can be done with the prototype. Their effects on the haptic feedback can be compared with the Renner® demonstrator for which these values have been physically modified.

Figure 25 presents the haptic force for the Renner® (left) and the haptic prototype (right). The parameter is the height of the let-off button [43], see also Figure 9. In the legend, *Normal* means that the setting value is nominal and *Low* (resp. *High*) that the let-off button is lower (resp. higher) than its nominal height, by approximately 0.5 mm.

**Figure 25.** Haptic piano key force validation: variation of the button height, physically on the Renner® demonstrator (**left**) and virtually on the haptic key prototype (**right**).

In Figure 25, the trends are similar between the prototype and the demonstrator: a higher (resp. lower) button causes the escapement phase to have a higher (resp. lower) force value around [8–9] mm. The escapement is even more stressed in the prototype. Figure 26 shows the effects of the hammer mass variation, i.e., meaning that a punctual

mass of 0.003 kg has been added once (resp. twice) for the *High* (resp. *Very high*) case.

**Figure 26.** Haptic piano key force validation: variation of the hammer mass, physically on the Renner® demonstrator (**left**) and virtually on the haptic key prototype (**right**).

Again, the impact is similar for the *High* and *Very high* cases, with an increase of the haptic force because the hammer is heavier.

Besides, one advantage of the multibody model is that it can virtually perform many interesting investigations, for instance, the hammer mass can easily be lowered. Doing so in real life would require to manufacture a whole new hammer. This variation is illustrated with the *Low* curve in Figure 26, for the haptic prototype only, resulting with a lower haptic force until the key-bottom contact (phase D in Figure 24).

Despite some discrepancies, the above results (Figures 24–26) show that the ROS-ROBOTRAN coupling proposed in this paper allows to develop a haptic key for digital pianos able to reproduce the Fhaptic action dynamic force quite faithfully, i.e., the *goldstandard touch* of a grand piano. Moreover, the symbolic multibody modeling approach makes it very easy to modify any physical parameter of the action or even the action itself in the haptic device, to modulate the haptic force accordingly. This feature was actually appreciated by some pianists and piano tuners who were consulted for this haptic keyboard project.

### *5.2. Haptic Driving Simulator*

Simulators for vehicle driving are nowadays a common tool and used for many applications [54]. Using a real-time multibody model including all the suitable physical parameters, allows to deal with the highly dynamics effects of a vehicle behavior. To analyze the dynamic performances of a vehicle, other approaches consider for example object-oriented programming of autonomous virtual drivers [32], instead of a real human. A prototype has been built that aims at reproducing the handling torque feedback in the steering wheel. Figure 27 shows the experimental set-up with the block schematics. As in all simulators, the driver has a real-time visualisation of the moving environment on a front screen.

In this haptic demonstrator, mainly developed at UCLouvain for educational purpose, a direct drive actuator acts on the steering wheel rotation. It contains an absolute angular encoder *SinCos Hiperface SKM36 Multiturn* which measures the position *q* and the velocity *q*˙ of the steering wheel. This information is sent through the *data transfer module*—in this case, an embedded processor *Raspberry Pi 3*—to the multibody model of the a full 3D vehicle. Afterwards, the corresponding torque *T*—computed by the inverse dynamics Equation (13)—is applied by the actuator to the human arms.

**Figure 27.** Haptic steering wheel: experimental setup with its real-time visualization and its corresponding block diagram below.

In the following illustrative experiment, in which a driver handles a virtual vehicle, the situation represents an obstacle avoidance while driving on a straight line at constant speed of 75 km/h. During the simulation, an obstacle suddenly and randomly appears, and the driver needs to avoid it by turning on the left and then coming back on the straight line. Meanwhile, for each of the three simulations, the caster distance of the front suspensions is modified without notifying it to the driver.

Figure 28 shows the corresponding lateral versus longitudinal vehicle displacements, as well as the obstacle. Without going into a detailed analysis, one can see that the behavior clearly differs depending on the trials and on the setting of the caster.

**Figure 28.** Haptic steering wheel: vehicle displacement on the ground with various caster.

The steering wheel angle, visible in Figure 29, illustrates in a different way the reaction of the driver to the obstacle apparition.

**Figure 29.** Haptic steering wheel: angle with various caster.

Finally, the torque given as a feedback is shown in Figure 30. Note that this value is taken from the model output, directly in the loop, not measured with an external sensor. For the first negative peak and for a very close angle value, the feedback torque is higher for a higher caster, as expected. After that, the driver counter-steers and tries to stabilize the vehicle towards its initial trajectory.

While a wide range of experiments could be achieved to relate the caster to the driver behavior, this illustrative example clearly highlights the capabilities of a multibody-based haptic steering wheel.

The prototype currently lacks the possibility to control the vehicle velocity. Simulations are done at a constant speed. Current work is ongoing to add *Penny*&*Giles HLP190* potentiometers to measure the position of the existing brake and accelerator pedals. Adding these sensors would allow to instantaneously adapt the vehicle velocity. Thanks to the ROS-ROBOTRAN coupling described in the previous sections, adding this hardware is quite straightforward in this software environment.

In the future, more experiments can be envisaged with several types of drivers, to analyze their feeling and torque feedback. This way, a wide range of vehicle dynamic parameters can be analyzed, such as the wheel toe-in/toe-out, the tire friction, the roll center height or the anti-roll bar stiffness, among others.

**Figure 30.** Haptic steering wheel: torque with various caster.

#### *5.3. Other Implementations*

The two projects presented before constitute the main current applications of the ROS-ROBOTRAN approach. However, other implementations involving a coupling between multibody models with real-time environments have been developed. In this scope, various sensors have been customized and utilized, showing our growing interest in coupling multibody dynamics with sensors for validation or haptic purposes.

For instance, during the Ph.D. thesis of Docquier [55] at UCLouvain, a small-scale demonstrator of a Narrow Tilting Vehicles (NTV) has been designed. The demonstrator allows to test embedded controllers thanks to inboard sensors that are used in real-time via ROS to update and analyze the vehicle behavior whose multibody model was built in parallel. Figure 31 presents the prototype and zooms on the two main sensors.

**Figure 31.** NTV demonstrator and its two main sensors: a rotating potentiometer and a binocular strain gauge for torque measurement.

Besides a classical potentiometer connected to the titling shaft to measure the vehicle tilt, a torque sensor based on a binocular strain gauge provides information associated with the actuated tilting degree of freedom of the vehicle. This information is thereafter used for control and validation purposes. Let us note that the vehicle also carries an IMU on board, to combine the tilting value from both IMU and potentiometer with a Kalman filter.

The second project concerns a so-called "kart" bench designed by and for students, to enable them to visualize and understand the road handling behavior of a four-wheel vehicle (Figure 32), and to compare the real system with its multibody model. The kart is laterally held on a conveyor belt and can be human-driven from a remote steering wheel linked to the front suspension via Bowden-type cables. Real-time multibody model can show live the four lateral forces on the tires, for instance, for different types of driving behaviors and/or kart suspension settings.

**Figure 32.** Sensors of the kart bench.

Apart from a classical optical sensor measuring the conveyor belt velocity, see Figure 32, a longitudinal and transverse force measurement through a homemade binocular load cell with strain gauges is placed between the frame and the moving kart to quantify the lateral force on the kart. It allows to analyze in real-time the effects on the tire-belt interaction forces and also to compare them with those computed live by the multibody model, while modifying suspension settings such as the toe-in/toe-out.

Let us note that this prototype does not run with ROS. Instead, it exploits *Labview* to communicate with the dedicated electronics and to retrieve the necessary sensors information, as for the above-mentioned applications.

#### **6. Conclusions and Prospects**

The main objective of this work was to highlight that the current multibody modeling formulations had reached a sufficient maturity to be exploited within haptic devices applied to mechanical systems involving high dynamics.

In a first step, we presented a multibody formalism in relative coordinates whose constraint equations are eliminated by a proper reduction of the equations. This formalism lends itself perfectly to its programming via the symbolic approach whose capacity of equation manipulation and simplification allows to produce very compact models, i.e., perfect candidates to real-time computation.

The interaction and coupling of these models with the world of sensors is essential, on the one hand, to validate the models themselves with respect to their underlying physics and, on the other hand, to allow a reliable coupling between the model, the sensors, and the actuators of a haptic device. The ROS platform has been chosen as the meta architecture to ensure these couplings successfully: two applications illustrate our developments. The first one, presented in detail, concerns the development of a haptic piano key and demonstrates the capabilities of the approach for a very high dynamic system. The second one refers to a driver simulator under development in our laboratory, whose interest—above all pedagogical—is to show the appeal of multibody models for this kind of application. In particular, it allows students to observe the impact of different parameters on the system dynamics, as taught in the academic courses but through multibody equations that can be a little bit abstruse for some of them.

In terms of perspectives, it would be interesting to continue the investigations on the model side, in particular through the fine-grain parallelization of the symbolic equations. Preliminary tests in ROBOTRAN have indeed shown that recursive equations can be reorganized in very few sequential vectorial steps: this asset could be exploited in order to further reduce the computation time of models to be embedded in devices requiring real-time computation.

Besides, we think that the potentialities of coupling real-time multibody dynamics with sensors and actuators within a middleware platform (e.g., ROS) are very promising for the future. It will enable to develop new concepts of haptic systems that are generic, user-friendly, and efficient. In particular, we see a real pedagogical interest in the use of such an architecture for the fields of multibody modeling, sensor implementation and mechatronic design of haptic systems with a very pronounced dynamic character, such as those presented in the present work.

**Author Contributions:** Conceptualization, methodology, software N.D. and P.F.; validation, S.T.; formal analysis, investigation, writing N.D., S.T., and P.F.; All authors have read and agreed to the published version of the manuscript.

**Funding:** Sébastien Timmermans is FRIA Grant Holder of the Fonds de la Recherche Scientifique-FNRS, Belgium.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** Authors would like to thank Thierry Daras, Alex Bertholet, Antoine Bietlot, Quentin Docquier and Aubain Verlé from UCLouvain, as well as Théo Tuerlinckx, François Huens, and Sébastien de Longueville for their help. Figure 9 is adapted from the paper published in Mechanism and Machine Theory, Vol 160, Timmermans, S.; Ceulemans, A.E.; Fisette, P., Upright and grand piano actions dynamic performances assessments using a multibody approach, 104296, Copyright Elsevier.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

#### **Abbreviations**

The following abbreviations are used in this manuscript:


#### **References**

