Multimodal Technol. Interact., Volume 5, Issue 5 (May 2021) – 6 articles

Technology plays an essential role in shaping youth’s communication and social interactions in online multiplayer games. Due to physical distancing restrictions during the COVID-19 global pandemic, online multiplayer games like Minecraft and Roblox are well-positioned to amplify healthy communication/social connections and mitigate the impact of social isolation. Research so far has been focused on how these gaming environments support youth development from the perspectives of individual stakeholders (e.g., caregivers, educators, designers, and developers). However, features of these games, such as communication and parental controls, are often misaligned with the ways in which children develop communication and social skills. Using a series of case studies of popular virtual worlds and online games, this paper provides an analysis of critical design features that serve youth throughout different stages of childhood and early adolescence. We offer three main contributions: (a) a comparison matrix of similarities and differences in communication and control features between platforms; (b) an evaluation of design features in selected virtual world platforms that promote safe and positive social interactions; and (c) a method for cross-platform comparison aimed at helping researchers, designers, and practitioners examine specific dimensions of social communication and play experience in virtual worlds. Full article

Paving the way to future mobility, teleoperation of vehicles promises a reachable solution to effectively use the benefits of automated driving as long as fully automated vehicles (SAE 5) are not entirely feasible. Safety and reliability are assured by a human operator who remotely observes the vehicle and takes over control in cases of disturbances that exceed the vehicle automation’s skills. In order to integrate the vehicle’s automation and human remote-operation, we developed a novel user-centered human-machine interface (HMI) for teleoperation. It is tailored to the remote-operation of a highly automated shuttle (SAE 4) by a public transport control center and based on a systematic analysis of scenarios, of which detailed requirements were derived. Subsequently, a paper-pencil prototype was generated and refined until a click-dummy emerged. This click-dummy was evaluated by twelve control center professionals. The experts were presented the prototype in regular mode and were then asked to solve three scenarios with disturbances in the system. Using structured interview and questionnaire methodology, the prototype was evaluated regarding its usability, situation awareness, acceptance, and perceived workload. Results support our HMI design for teleoperation of a highly automated shuttle, especially regarding usability, acceptance, and workload. Participant ratings and comments indicated particularly high satisfaction with the interaction design to resolve disturbances and the presentation of camera images. Participants’ feedbacks provide valuable information for a refined HMI design as well as for further research. Full article

High-fidelity localized feedback has the potential of providing new and unique levels of interaction with a given device. Achieving this in a cost-effective reproducible manner has been a challenge in modern technology. Past experiments have shown that by using the principles of constructive wave interference introduced by time offsets it is possible to achieve a position of increased vibration displacement at any given location. As new interface form factors increasingly incorporate curved surfaces, we now show that these same techniques can successfully be applied and mechanically coupled with a universal actuation plate. Full article

Composers do not usually collaborate with other composers but, for the last half century, open works were created that invite performers to implement details left undetermined or even decide the order in which various sections of the composition are to be played. Chance operations were also used in the writing of musical pieces and, in music generated with the assistance of computers, controlled randomness found its place. This article proposes a platform designed to encourage collaborative and interactive composition on high-performance computers with DISSCO (Digital Instrument for Sound Synthesis and Composition). DISSCO incorporates random procedures as well as deterministic means of defining the components of a piece. It runs efficiently on the Comet supercomputer of the San Diego Supercomputing Center and uses the Jupyter notebook environment to integrate the end-to-end processes with a user. These tools, the implementation platform, and the collaboration management are discussed in detail. Comments regarding aesthetic implications of the partnership between one or more humans and computer—considered a bona fide collaborator—are also provided. Possible future developments are supplied at the end. Full article

The history of algorithmic composition using a digital computer has undergone many representations—data structures that encode some aspects of the outside world, or processes and entities within the program itself. Parallel histories in cognitive science and artificial intelligence have (of necessity) confronted their own notions of representations, including the ecological perception view of J.J. Gibson, who claims that mental representations are redundant to the affordances apparent in the world, its objects, and their relations. This review tracks these parallel histories and how the orientations and designs of multimodal interactive systems give rise to their own affordances: the representations and models used expose parameters and controls to a creator that determine how a system can be used and, thus, what it can mean. Full article

Misconceptions of vehicle automation functionalities lead to either non-use or dangerous misuse of assistant systems, harming the users’ experience by reducing potential comfort or compromise safety. Thus, users must understand how and when to use an assistant system. In a preliminary online survey, we examined the use, trust, and the perceived understanding of modern vehicle assistant systems. Despite remaining incomprehensibility (36–64%), experienced misunderstandings (up to 9%), and the need for training (around 30%), users reported high trust in the systems. In the following study with first-time users, we examine the effect of different User Onboarding approaches for an automated parking assistant system in a Tesla and compare the traditional text-based manual with a multimodal augmented reality (AR) smartphone application in means of user acceptance, UX, trust, understanding, and task performance. While the User Onboarding experience for both approaches shows high pragmatic quality, the hedonic quality was perceived significantly higher in AR. For the automated parking process, reported hedonic and pragmatic user experience, trust, automation understanding, and acceptance do not differ, yet the observed task performance was higher in the AR condition. Overall, AR might help motivate proper User Onboarding and better communicate how to operate the system for inexperienced users. Full article