Internet of Tangible Things (IoTT): Challenges and Opportunities for Tangible Interaction with IoT

Abstract

In the Internet of Things era, an increasing number of everyday objects are able to offer innovative services to the user. However, most of these devices provide only smartphone or web user interfaces. As a result, the interaction is disconnected from the physical world, decreasing the user experience and increasing the risk of user alienation from the physical world. We argue that tangible interaction can counteract this trend and this article discusses the potential benefits and the still open challenges of tangible interaction applied to the Internet of Things. After an analysis of open challenges for Human-Computer Interaction in IoT, we summarize current trends in tangible interaction and extrapolate eight tangible interaction properties that could be exploited for designing novel interactions with IoT objects. Through a systematic review of tangible interaction applied to IoT, we show what has been already explored in the systems that pioneered the field and the future explorations that still have to be conducted. In order to guide future work in this field, we propose a design card set for supporting the generation of tangible interfaces for IoT objects. The card set has been evaluated during a workshop with 21 people and the results are discussed.

1. Introduction

Thanks to technological advances, electronic devices are spreading in our environments, seamlessly integrating in our everyday life. When in 1991, Mark Weiser, considered nowadays the father of ubiquitous computing, imagined that technology would disappear to our sight and integrate in our daily routines [1], probably he was not expecting that technology would have modified so profoundly our lives. Smartphones have brought ubiquitous computing to the mass and indeed, nowadays, interaction with technology is becoming more frequent and intuitive. In parallel, in the context of Ubiquitous Computing, since 1999, researchers working in the domain of Internet of Things (IoT) are overcoming technological barriers and are investigating novel applications for connected objects in healthcare, transportation and smart-environments, which could bring many potential benefits for the human well-being [2]. For 2020, Intel is forecasting 200 billion IoT objects, i.e., 26 connected devices per person [3].

However, until now most IoT research focused on the technological challenges, often overlooking the important research issues of how humans should interact with this multitude of IoT objects [4]. Moreover, today typical interfaces for IoT objects are based on smartphone or web apps, which exploit few of the innate human skills that we would normally use to interact with physical objects. As a result, humans are often spending most of their time interacting with what Victor called a “Picture under Glass” [5], sometimes with severe social consequences [6]. Richer interactions that goes beyond the screen and that better exploit our bodily human skills can be imagined for our future [5,7,8]. In particular, richer interaction paradigms for IoT could also help the user understand and trust connected objects, in order to exploit better IoT potentialities and benefits. To this purpose, we propose to study tangible interaction applied to the IoT, what we call “Internet of Tangible Things” (IoTT). The nature of the tangible interface for IoT objects could be twofold: either it can support immediate interaction in the periphery of the user’s attention [9], freeing cognitive resources for the user’s everyday activities, or it can support meaningful and unexpected interactions that stimulate reflection and the understanding of the system [10]. Possibly, these different interaction paradigms could exploit different levels of attention of the user, switching according to the context of the interaction.

In this context, few works have explored tangible interaction principles that can drive the design of IoT interfaces that are situated in the physical world. This article aims at providing a reference framework for the Internet of Tangible Things, through two different main contributions:

(1)

The identification in literature of eight tangible properties that can be exploited for designing interactions with the internet of things;

(2)

A card set for supporting the design of tangible interaction with IoT objects, composed by 16 cards that illustrate the eight tangible interaction properties and eight additional IoT properties.

For the first contribution, after an introduction of the research context (Section 2), we present a first narrative review conducted for identifying eight tangible interaction properties that are relevant for the interaction with the Internet of Things (Section 3). To get deeper insights into the opportunities that these properties offer for designing the interaction with the IoT, we conducted a systematic literature review that analyses which of these properties have been suggested or implemented in previous work and which are yet to be explored. In particular, Section 4 presents the methodology of the systematic literature review, Section 5 presents the results of the review and Section 6 discusses the opportunities and open challenges emerged from the review. As a second contribution to the emerging field of the Internet of Tangible Things, we present in Section 7 a card set that aims at helping designers conceiving the interface of new IoT objects or improving the interaction with existing ones. The design card set has been evaluated with 21 participants, mostly researchers in the field of tangible interaction, who designed 6 IoTT objects during a workshop organized at the third European Tangible Interaction Studio [11]. Section 8 presents the results of the evaluation of the card set. Section 9 concludes the paper and presents future work.

2. Context

Atzori et al. individuated four typical application domains for IoT: transport and logistics, healthcare, smart environments and personal and social applications [2]. Since this article focuses more on the interaction with IoT, where humans have an active role and IoT objects can offer useful services during daily routines, the explored application domains are typically related to smart environments and personal and social applications rather than to logistics and healthcare. Domestic environments, work environments and public environments are those that are mostly relevant for human-computer interaction with IoT. In these scenarios, not only objects can communicate through the Internet and their behaviour can change according to network information but the object physical states and behaviours can also be affected by other people interacting at distance. This is particularly evident whenever the Internet of Tangible Things is used to strengthen social relationship at distance [12,13]. As an example of an IoT system, Figure 1 presents a scenario were a user interacts with an IoT object, which is connected not only to a similar remote object through the Internet but also to other objects that can be located in proximity of the first object. It is worth noting that the behaviour of an IoT object can be affected by other people interacting with connected objects, or by external services (e.g., weather services) that are difficult to identify and materialize for the user. Therefore, the typical scenario faced by a user of an IoT object is often very complex. The purpose of this study is therefore to understand if tangible interfaces are able to facilitate the understanding and control of these complex scenarios, increasing trustiness in IoT systems and supporting long-lasting interactions, even beyond technology obsolescence. As a starting point, we aim at looking at existing tangible properties identified and used in previous work and that could increase the user experience with IoT objects.

3. Related Work

This section presents related work in the two research fields that are closely related to the Internet of Tangible Things: The Internet of Things (IoT) and Tangible Interaction. In particular, our initial analysis will focus on the current research on Human-Computer Interaction applied to IoT (Section 3.1) and current trends of tangible interaction that seems promising for the internet of things (Section 3.2). In particular, through the analysis of tangible interaction literature, we identify eight tangible interaction properties that are promising for designing the interaction with the Internet of Things. We introduce then the new research field of the Internet of Tangible Things and discuss similar or related work (Section 3.3).

3.1. Human-Computer Interaction with IoT

The US National Intelligence Council defined the Internet of Things (IoT) as one of the six most “Disruptive Civil Technologies” [2]. The European Community is investing 192 million euros in several different domains of IoT research through the Horizon 2020 programme for the years 2014–2017 [14]. Since the first definition of IoT, different technological platforms generated different related visions of what the IoT is. Atzori et al. identified three main visions: the first one is more “things-oriented” and respond to the initial trend of equipping everyday objects with Radio-Frequency Identification (RFID) tags; the second vision, defined as “Internet-oriented”, relies on smarter objects, capable of fulfilling the full Internet Protocol (IP) stack; the third vision highlights the semantic reasoning that can be performed on IoT data [2]. The authors suggest that IoT stems from the intersection of these three visions. The Internet of Things is still a very actual research topic and many efforts are being devoted in order to standardize this broad ecosystem of interconnected objects: the Internet Architecture Board (IAB) has recently proposed a document for the definition of interconnected smart objects [15]. Similarly, in 2012, the ITU has proposed a first Recommendation for the definition of the Internet of Things [16].

A literature review of IoT shows that so far, most of the research on the Internet of Things focused on improving the hardware and software architecture that allows IoT objects to better manage power consumption, optimize data collection and sharing as well as seamlessly communicate to each other, often without human intervention [17].

A search in the ACM Digital Library with the keywords (+(“human-computer interaction” HCI) +(“internet of things” iot)) produced 152 results, most of them concentrated in the last 3 years (from 2 results in 2010, to 14 results in 2013, up to 49 in 2016).

As is also pointed out by Koreshoff et al. [4], the human is often neglected in IoT design and only recently has research been conducted on the human-IoT interaction.

It is worth noting that interaction in smart environments and with the ubiquitous computer [18] has been already explored in depth. However, although these research fields share several challenges with the Internet of Things, the interaction with IoT introduces peculiar challenges, because of the worldwide interconnectivity between the different objects, people and virtual [19] and physical sensors and because of their autonomous evolution according to exchanged information. In Designing Connected Products [20], Rowland et al. discuss the main challenges for improving the consumers’ User Experience with IoT products. Rowland suggests that, because of the novelty of IoT in the consumer market, it is difficult to communicate to the user the conceptual model that shows how the device works and the related interaction model [20]. As a result, many IoT products fail to conquer a broad market, remaining niche products for geeks or experts because of functions that do not appeal to the mainstream user or configuration processes and interaction models that are not straightforward.

In order to design IoT objects that better meet users’ needs and expectations, it is fundamental to also analyse IoT from a human-centred point of view. Koreshoff et al. readapted Atzori et al.’s IoT framework to address relevant research issues for Human-Computer Interaction (HCI) that are still open [4]. Koreshoff et al. conducted a literature review of HCI research and a review of IoT products [17]. The authors denote a lack of explicit involvement in HCI for IoT: most academic research focuses on specific application issues or in object interconnectivity and out of the 89 papers reviewed from the CHI conference and the Personal and Ubiquitous Journal, only 5 dealt with user inputs. For the 93 IoT commercial products reviewed, the authors show that most user interfaces are based on smartphone or web apps, or on screens integrated in the object.

Other attempts to promote user-centred design of the interaction with IoT devices can be found in Soro et al.’s article [21], who also propose an adaptation of Atzori et al.’s framework and in Fauquex et al.’s article [22], who propose a user-centred methodology for creating “people-aware” IoT applications.

Rowland et al. points out that IoT interaction models often suffer from their underlying technological complexity [20]. A typical property of IoT objects is to be connected to the Internet and to have a behaviour that can depend from a dynamic network of remote connected entities. Therefore, the interaction designer for IoT should be able to cope with connectivity issues (Is the object still able to work when disconnected from the network? Will it respond to user inputs?) as well as with interaction models that depend on the autonomous communication with other connected objects (Did the object state change because of the sensed local environment or because of the object interaction with the networked objects? Is this object sending data to other objects? Which objects? Which data? For which people?).

Moreover, IoT objects are able to sense the external environment, collect sensitive information about the user and share it through the Internet. Therefore, a reliable control on sensed and shared data is particularly important for the user, who can otherwise lose the trust in the product or service. Bellotti and Sellen addressed these privacy issues with a framework for designing feedback and control in ubiquitous computing environments [23]. In particular, the framework addresses four system behaviours concerning feedback and control: when and which information is captured, how the information is elaborated, who has access to the information and for which purposes. While Bellotti and Sellen’s framework supposes that actors that can access the information were humans, in IoT we face to a more complex scenario where also other IoT entities (e.g., cloud software or other IoT objects) can access, elaborate and further share this information.

In this article, we argue that tangible interaction can help addressing these challenges. Therefore, the next subsection reviews the perspectives of tangible interaction applied to IoT.

3.2. Tangible Interaction and Perspectives for IoT

We present here an analysis of current research in tangible interaction, conducted to individuate properties that can be useful for designing the interaction with the internet of things. We summarize the results of this analysis in the form of a narrative literature review. In the analysis, we highlight 8 tangible interaction properties (T1 to T8), which are resumed at the end of this subsection. The eight properties will be used as a framework for the meta-analysis of the papers selected in the systematic review and as themes for the design card set presented in Section 7.

In Chapter 8 of Designing Connected Products, Charlier analyses how different outputs and inputs can be combined in connected products to design appropriate interfaces and interactions [20]. Among the different interaction paradigms that can be explored for IoT, Charlier cites Tangible User Interfaces (TUIs) suggesting as advantages the usefulness for learning applications and music and, as main disadvantages, the need of several tokens that can be lost [20]. A recent research on tangible interaction, published at CHI’2016, suggests that richer interactions, beyond token manipulation, can be designed for IoT [24].

Since Fitzmaurice’s Graspable User Interfaces [25] and Ullmer and Ishii’s TUIs [26], Tangible Interaction has evolved much, embracing new disciplines and interaction paradigms [27]. Hornecker and Buur [27] individuated several advantages of tangible interaction over traditional interfaces: facilitates collaboration thanks to a shared interaction space and multiple access points (T4), exploits the spatiality and the user’s proprioception (T4) as well as the interaction through the full-body (T2), makes use of expressive representations that help cognition (T1). Hoven et al. individuate three main aspects that characterize tangible interaction: the interaction with the physical world, the exploitation of the human skills and digital computation [8]. Although most digital interactions involve to some extent these aspects, the purpose of interacting with everyday objects exploiting human skills that are often ignored by traditional user interfaces (T2) makes tangible interaction different from other interaction paradigms and particularly interesting for interacting with IoT objects. Hoven et al. individuated particular qualities of tangible interactive systems: they generally offer direct, integrated and meaningful representation and control of the digital information (T1) [8].

Since digital information is represented by a physical representation, which can be also used as a user control to manipulate such information, a particular property of Tangible User Interfaces is the persistence of the information even in case of power outage (T3). This property is particularly interesting for IoT, since IoT objects can run out of power or can be disconnected suddenly from the Internet (cf. [20]). Moreover, the directness of the interaction and the coupling between action and perception [28] supports immediate interactions that are generally easy to learn and understand (T5). At the same time, objects can embody meaningful concepts and can support mindful interactions that make the user step back and reflect (T7) [10].

Indeed, among the different tangible interactive systems, it is possible to distinguish between two main categories: those that support immediacy and those that support mindful interactions. This distinction stems from the Heidegger’s definition of Ready-to-Hand and Present-at-Hand [29], which has been resumed by Dourish to introduce the principles of embodied interaction [30]. Tanenbaum et al. [31] extend this notion and map the two Heideggerian concepts to the Bolter and Grusin’s concepts of transparent immediacy and hypermediation [32]. Tanenbaum et al. [31] discuss the typical Heidegger’s example of the hammer, a tool that is typically Ready-to-Hand, until it breaks down and it appears to the user attentions, becoming Present-at-Hand: the hammer can also attract our attention without necessarily break down, for example because a detail of the grip reminds the user of past memories. Therefore, the authors suggest that the objects, besides their functional role, can be categorized according to a semantic line because they can become, present-at-mind. Also Hornecker [33] argued that tangible interaction has a hybrid nature and can support two different approaches, one that exploits affordances and user’s previous knowledge to provide intuitive and seamless interactions and another that breaks the immediacy typical of direct manipulation in order to support reflection and the understanding of the system.

Tangible Interaction is particularly interesting for supporting peripheral interactions. Bakker and Niemantsverdriet [34] suggest that to better integrate digital interactions in our daily routines, the designers of interactive systems should exploit the full interaction-attention continuum. Indeed, the system should support both focused interactions that exploits all the users’ cognitive and motor resources and lighter interactions that happen in the users’ peripheral attention, which are more suitable when they are occupied in other tasks. Indeed, an important challenge that interaction designers have to face in ubiquitous computing is the overwhelming quantity of digital information and request for user input that is continuously presented to the user. This often disrupts the attention and concentration of the user, especially at work. Peripheral interaction deals with this problem by designing interactions that can be performed through the user’s peripheral attention, freeing user’s cognitive resources for other tasks (T6) [9]. Bakker et al. suggests that peripheral interaction is particularly useful for all those situations where the interaction could be integrated into the daily routine, without necessarily being in the centre of the attention [9]. The designer of peripheral interfaces should also be able to take into account the context of the interaction, as well as the personal preferences of the user [9]. While direct manipulation in touchscreen has been proved to be particularly intuitive and easy to learn, it requires visual attention and lacks of haptic feedback; tangible manipulation, instead, can be often performed without visual attention, relying on proprioception and haptic feedback for evaluating the result of the interaction [5].

The opposite approach to design valuable tangible interactions for long-lasting objects is bringing them to the centre of the user’s attention. In this case, the interaction should engage the user, stimulating reflection or emotional response [34]. Schmitz suggests that objects with life-like, often unpredictable, behaviours can enhance the longevity of the interaction, especially if coupled with zoomorphic or anthropomorphic affordances (T8) [10]. As in [10], Chapman proposes a framework for emotionally durable electronic devices [35]. The framework suggests that durable devices should have their own consciousness and should not be fully understood by the user (at least in the product exploratory phase), should make the user develop both attachment and detachment to the product and should carry signs of ageing and stories associated to the product itself. The purpose of Chapman framework is coping with the increasing waste of domestic electronic products, which get often abandoned by the user after few usages (T8). The association of stories to physical objects is a point that is often explored in tangible interaction to create attachment to a product. Hoven and Eggen [36] proposed an extension of Ullmer and Ishii’s [26] Tangible Interaction Framework to take into account personal objects that are associated to personal memories (T7). Indeed, personal objects facilitate the association with digital media thanks to the pre-existing mental models. Tanenbaum et al. explored the relationship between narrative and objects through the Reading Glove, a wearable device equipped with a RFID reader to support storytelling with RFID tagged objects [31]. The ultimate challenge would be to combine both interaction paradigms (peripheral interactions and long-lasting focused interactions) in the same interface, allowing the user to switch between the two paradigms according to the context of use.

We resume the 8 properties found in our analysis of tangible interaction research in the following list:

T1.

Meaningful representations and controls of the single IoT object connectivity status and IoT object interconnections, as well as of information capture, elaboration and sharing.

T2.

Rich interactions that exploit the natural human skills, in particular exploiting haptic and peripheral interactions with IoT objects that are situated in the physical world.

T3.

Persistent physical representations that could last in case of power or connectivity outrage, allowing the user to control the state of an IoT object even when no Internet connection is available.

T4.

Spatial interactions that support collaborative setups with multiple IoT objects.

T5.

Immediacy and intuitiveness of the interaction, facilitating the understanding and control of IoT objects with minimal learning time.

T6.

Interactions with IoT objects that are integrated in the daily routines, which free users’ cognitive resources and do not disrupt attention.

T7.

Facilitated reflections on IoT object meaning and working principles, as well as support for associating and sharing memories.

T8.

Long-lasting interactions with IoT objects exploiting emotional durable designs, to cope with electronic waste due to technological obsolescence.

This analysis of the tangible interaction literature according to its two opposite and complementary approaches, which we identify in this article with the Heideggerian terms of Present-at-Hand and Ready-to-Hand suggests that tangible interaction offers many interesting possibilities for the design of new Human-IoT interfaces. The tangible interaction properties described in this section and summarized in the previous list are promising for providing an easier understanding of the IoT objects and an increased trust in the IoT system. Through the systematic literature review, we aim at evaluating which of these properties have been explored until now and which properties still deserve further investigations.

3.3. Internet of Tangible Things

Tangible interaction in IoT is still mostly unexplored. Embedding the interaction in IoT objects is not a novelty and already in 2010, Kranz et al. [37] showed several examples in this field. The authors evidenced some challenges that are still actual nowadays, such as the risk of changing the nature of existing objects in order to add interactive capabilities, or vice-versa, the risk of hiding too much the interactive capabilities of the objects to the user eyes. Nevertheless, their analysis did not focus on the important implications of interacting with a network of objects that communicate to each other.

The Web of Things, proposed by Guinard and Trifa [38] can be considered another close related domain. While in their first idea the Web of Things aimed at building web services for IoT objects, Mayer et al. [39] proposed a framework that associates semantic interaction primitives to different IoT object characteristics, with the purpose of supporting both web GUI and tangible interfaces, such as physical knobs and sliders. Although this was a valuable effort for bringing back IoT interfaces to the physical world, the richness of tangible interaction was still poorly exploited and the approach was mostly technology-oriented.

The novelty of the field of research explored in this article may be evidenced by the fact that “Internet of Tangible Things” is a new term introduced by Sarah Gallacher [40] only in January 2016 during the Second European Tangible Interaction Studio [41], with no previous reference in scientific literature. This term is adopted in this article to promote a shift towards the design of physical interactions with IoT. Indeed, the potential advantages and the related challenges are mostly unexplored until now in current research.

As an outcome of the systematic literature review that will be presented in this article, we can highlight some theoretical work that has been conducted recently in this field. Among the most relevant papers found in the literature, it is worth citing the work of Ambe et al. [42], who analysed three particular tangible IoT systems for connecting distant people and encourage social connection according to a selection of properties from Hornecker and Buur’s [27] tangible interaction framework. In particular, they highlighted the importance of supporting user appropriation and personalization of IoT devices, in order to support long-term use of these devices. These properties are without doubt valuable for IoT objects that should foster social connection but might not apply to other categories of IoT objects. For example, in VoxBox [43], a tangible questionnaire for collecting feedback during public events, appropriation was discouraged. Indeed, the purpose of the tangible questionnaire was to collect valuable feedback with intuitive but meaningful interactions related to users’ event perception. In this case, while interactions should be playful, they should not be transformed just in a game. Instead, a tangible questionnaire should stimulate reflection on the proposed questions and facilitate discussions with other people.

The tangible properties explored or proposed and the approaches for exploring them might vary consistently among different application domains and research fields. In our literature review, we individuated different kinds of papers, from those having a user-centred approach, for example for understanding the importance of personal objects for older adults [44] and derive guidelines for IoT system implementations, to more technology-driven approaches where the design and development results were not even assessed with users [45]. The vision might also be slightly different between researchers with a computer science background and those with a product design background. For example, Knutsen et al. [46] through a review of commercial IoT products analyse the hybrid nature (tangible/intangible) of current IoT products, suggesting the term “Internet of Hybrid Products.” Indeed, some of the commercial IoT objects reviewed presented a mix between a tangible interface and digital services based on smartphone or web apps. Although the idea is closely related to the IoTT concept, the paper of Knutsen et al. [46] was more focused on product and service design than on interaction design, discussing few of the aforementioned tangible interaction properties. For this reason, this paper was not included in the meta-analysis.

We found in literature interesting attempt to facilitate the interaction with IoT objects through tangible interfaces. Van der Vlist et al. [47] presented a system that allowed to create connections between IoT objects in the smart home through simple manipulations of tangible tokens that represent the IoT objects and of a central tile that represents the network hub. Domaszewicz et al. [48] presented a similar system where “an application pill” object allows interconnecting different IoT objects and determining whether an IoT function can be obtained combining the IoT objects that were in the range of the application pill. Ideally, different pills can be used to assess different functions. Those papers were considered as borderline since IoT objects were not augmented with tangible interfaces; instead, an additional tangible object was introduced to manipulate IoT object connections. Although this a valuable first attempt towards tangible interaction with IoT we did not include the papers in the analysis because, according to our eligibility criteria (cf. Section 4), the physical interaction was not embodied in the IoT object.

Besides all the papers presented in Section 5, to the best of our knowledge, the only other relevant work that was not included in the literature review, is Physikit [26], a toolkit for the physical visualization of ambient data collected by an IoT sensor platform. Unfortunately, this article was not detected through our queries in the digital libraries (cf. Section 6.3).

4. Methodology of the Systematic Literature Review

The systematic literature review presented in this article aimed at identifying previous work on tangible interaction and tangible user interfaces applied to the Internet of Things. The sources of the articles have been selected among the most popular digital libraries for Computer Science and in particular Human-Computer Interaction. The queries have been performed on three digital libraries: ACM Digital Library, IEEE Xplore and Science Direct.

A first research query aimed at individuating any article containing the keywords “tangible interaction” or “tangible user interface” and “internet of things” (Q1). Since several issues typical of the interaction with IoT were already addressed in previous work under the umbrella of “smart objects,” we included in the search also articles containing the keywords “smart object” and “tangible interaction” or “tangible user interface” (Q2). Performing the queries on full texts available on the three aforementioned databases, we obtained 56 distinct results for Q1 and 29 distinct results for Q2 (as of May 2017). After duplicated removal, we obtained 81 distinct papers.

The screening phase aimed at eliminating non-relevant papers based on objective criteria: book chapters without available full-text (5), as well as keynote (1) and workshop proposal (6) papers, were discarded, in order to keep only articles with enough content. Moreover, papers containing “tangible interaction” or “tangible user interfaces” keywords only in citations or as related work were also discarded. Finally, to select only papers dealing with relevant content, the keywords “physical” or “tangible” and “connect”, “network”, “communication” or “relationship” were searched in the full text and analysed in their context in order to discard all papers that were not dealing with physical objects and connected objects. After the screening phase, 34 papers were retained.

The eligibility phase aimed at individuating relevant papers that describe tangible interfaces for IoT systems or that discuss theoretical issues of tangible interaction with IoT. The eligibility criteria were defined as follow:

“The article talks about one or more physical objects, which are augmented with sensing, actuating, or information processing (storage, elaboration, communication) technologies and connected to other objects/people through a form of network technology (e.g., BT, WiFi, NFC, RFID). The user interacts with the object or system through a tangible interface, which exploits the physicality of the object”.

Two of the authors of this article evaluated independently the eligibility of each paper. The two authors further discussed papers that received discordant eligibility ratings. If no agreement could be found on the eligibility rating, a third author read the paper. At this point, eligibility was assigned according to a majority vote rule. After the first discussion, 19 papers were declared as eligible, 12 as not eligible and 3 required the analysis of a third reviewer. After this phase, 18 papers were declared as eligible and 14 as not eligible. One paper [49] met the eligibility criteria but an extended version [50] of the same article was also present in the pool; therefore, it has been excluded from the meta-analysis. This process has been conducted following a lightened version of the PRISMA review protocol [51]. The process is summarized in Figure 2.

18 papers have been analysed to individuate which of the 8 tangible interaction properties presented in Section 3.2 have been explored already and which properties are still unexplored. The analysis allowed also individuating additional properties that were not listed among the aforementioned properties that we have previously individuated in tangible interaction literature, as well as open challenges that were highlighted in the papers. Each paper has been analysed by two authors, who highlighted the phrases that were mentioning tangible properties. The author assigned each phrase to one of the eight properties. Then they indicated for each article whether each of the 8 properties was present or not. Out of 144 attributions, only 19 cases (in average about one property per article) resulted with disagreement between the two reviewers and required further discussion. After discussion, reviewers reached agreement for all property attributions.

5. Results of the Meta-Analysis

Since the purpose of the meta-analysis was mainly to individuate which tangible interaction properties have been suggested or implemented for the Internet of Things, this section presents how each property has been cited by previous work in this field. Additionally, we classified the articles found in the literature review according to different categories in order to understand whether these properties were merely suggested or they have also been implemented and tested with users.

Table 1. Results of the meta-analysis.

Paper	Type	T1	T2	T3	T4	T5	T6	T7	T8
TangiSense (Traffic) [50]	System, no user evaluation	X			X	X
Active Forms [52]	Theoretical	X	X			X	X
AwareKit [53]	System, user evaluation	X	X		X	X	X
CapNFC [45]	System, no user evaluation	X	X		X	X
Cognitive Objects [54]	Theoretical	X		X		X	X
Smart-home environment [55]	System, user evaluation	X				X	X
TANGerINE [56]	System, no user evaluation	X	X		X	X
Expressing Intent [57]	System, no user evaluation		X		X		X	X	X
VoxBox [43]	System, user evaluation	X			X	X		X
Invisible connections [44]	Theoretical, user interviews	X					X	X	X
IoT Owl [58]	System, no user evaluation	X						X	X
Iyagi [59]	System, no user evaluation		X				X
TangiSense (Kitchen) [60]	System, user evaluation				X	X
Projected interfaces [61]	System, no user evaluation	X	X		X	X
RapIoT [62]	Toolkit				X		X
T4Tags [63]	Toolkit, end-user insights		X		X	X	X	X
Technology Individuation [42]	Theoretical, analysis of user-evaluated systems	X	X		X	X	X	X	X
Tiles [64]	Toolkit	X	X		X		X	X
Total: 18 papers	4 theoretical, 3 toolkits, 11 systems. 7 papers include user insights	13	10	1	12	11	11	7	4

resumes the results of the meta-analysis for the 18 selected papers.

From the table, it is possible to notice that the most cited property is T1, with 13 occurrences out of 18 articles, while T3, i.e., exploring persistency of tangible interfaces, is the less mentioned (in only one paper and marginally). Such sentences citing tangible interaction properties are supported in different manners: in some cases, they are just reported as possible benefits of tangible interaction, while in other cases they are supported by working implementations and in fewer cases by user evaluations. Indeed, we found that 4 articles are mostly theoretical (although one of them analysed three existing systems), 3 articles present development toolkits and all the other articles (11) present a system at different level of completion (5 of these papers are published as work-in-progress). Insights from users are presented in 7 papers but only 2 of the systems were properly tested with users [43,55]. Insights from users are presented also in two theoretical papers, i.e., Vaisutis et al. [44] (with user interviews) and Ambe et al. [42] (with the analysis of system that were tested with users) and in a toolkit paper [63] (end-users tested a preliminary version of the toolkit, providing important insights).

Two theoretical articles, Active Forms [52] and Cognitive Objects [54], present concepts of particular IoT tangible objects. Active Forms are shape-changing physical objects that act as “gateways,” i.e., user interfaces, for a Responsive Environment, i.e., a physical space permeated of ubiquitous, context-aware technology. Cognitive Objects, instead, aim at supporting humans and robots in their everyday routines, by means of specific functions embodied in their physical shape.

The other two theoretical articles focus on IoT objects for supporting social connection at distance. Vaisutis et al. [44] present an interview to 6 older adults, conducted to investigate the possibility of augmenting everyday objects with IoT capabilities, in particular, for fostering social connection with distant relatives. This article also elicits human factors that should be considered in the design of such IoT objects. Ambe et al. [42] analyse three existing IoT systems for fostering social connections according to some tangible interaction properties, with a focus on adoption, appropriation and habituation.

3 articles present toolkits for developing IoTT objects. While Bellucci et al. [63] present a toolkit for end-user programming of physical smart-tags (T4Tags) that can be attached to everyday objects; Mora et al. present two toolkits (RapIoT [62] and Tiles [64]) for facilitating the implementation of physical IoT objects (as well as the supporting cloud and embedded software) based on customizable hardware.

The other 11 papers present systems of various types: six papers present custom IoT objects (AwareKit, IoT Owl, Iyagi, Expressing Intent; Voxbox; TANGerINE); 3 papers present systems based on tabletops (2 systems based on TangiSense; TANGerINE); 6 use RFID or NFC tagged objects (2 systems based on TangiSense; CapNFC; AwareKit; Expressing Intent; Smart-home environment); one uses a hybrid approach for augmenting IoT physical objects with light projection (Projected Interfaces).

More in detail, AwareKit [53] is a set of IoT objects connected to digital calendars that aims at facilitating meeting scheduling with colleagues. IoT Owl [58] is an IoT object with an owl shape that retrieves the frequentation of a canteen through another physical object (a stump) and suggests to office employees the right moment to go to eat. Iyagi [59] is a set of tangible objects to enhance bedtime routines for children through storytelling. Expressing Intent [57] presents a set of IoT objects animated by an own soul. The purpose of this article is discussing the importance of considering object animism, rich interactions and heterogeneous ecosystems while designing IoT objects. VoxBox [43] is a tangible questionnaire for collecting feedback during public events (e.g., attendants’ appreciation). TANGerINE [56] is a system for supporting collaborative interactions on table tops where users can transport information and make special actions through a custom-designed IoT tangible cube. Two other systems are based on the TangiSense platform, an RFID-enabled table top that augments tangible tagged objects with virtual agents. The two papers present two different application scenarios: for collaborative traffic management [50] and for suggesting recipes [60]. De la Guía et al. [55] present a system designed to facilitate the management of smart homes through tangible NFC cards associated to different functions. Molyneaux and Gellersen present Projected Interfaces [61], a system for augmenting smart tangible objects through environmental camera detection and light projection on the physical object. Grosse-Puppendahl et al. [45] present a particular Capacitive NFC technology for augmenting everyday physical objects with IoT capabilities and enabling tangible interaction with these augmented objects.

In the following subsections, we will deepen the analysis of each property reporting the most relevant sentences from the 18 analysed papers.

5.1. T1: Meaningful Representations and Controls

The first tangible interaction property proposed for the Internet of Tangible Things refers to the possibility of having meaningful representations and controls of the IoT object functions and/or parameters. The concepts of physical affordances and interaction embodiment are well known in the field of tangible interaction and, not surprisingly, this is the property with more occurrences among the selected papers. The importance of digital content embodiment into meaningful physical representation has been highlighted in Active Forms [52], Cognitive Objects [54] and TANGerINE [56]. In particular, Scott-Harden [52] suggested that interactive products or devices could render digital information content (such as the internal state of the product or of related services) thanks to perceptible changes to their physical form, e.g., shape or size and appearance, e.g., colour or temperature. Authors of both Cognitive Objects [54] and TANGerINE [56] highlighted the concept of semantic embodiment of the interaction, in respect of object abilities and possible user interactions with the object. Also, Mora et al. [65], whose objective is providing interaction primitives that are automatically recognized by their IoT development toolkit, highlighted the importance of designing interaction primitives that can make non-experts familiar with IoT concepts.

We found several concrete examples in the reviewed systems of a meaningful form of the object exploited to communicate to the user the IoT object main function. Hannula et al. [58] chose the form of an Own to display the presence of people in the canteen, an information collected by a remote sensor. The authors specify that “An owl was chosen as a character hence it is an animal, which is traditionally thought to be a symbol of intelligence. As a representation of a virtual friend it is so wise that it knows when people are present.” For the Awarekit [53], Matviienko et al. used different shapes to suggest which action or information the module presents. Indeed, a bell was used to inform about reminders, a round clock for selecting hours, a rectangular block with five boxes for selecting the day of the week and a magnifying lens for the tooltip module. Meaningful representations and controls can also be embedded as part of more complex objects. For example in VoxBox [43] a telephone handset was used for answering open questions, since it is a very familiar interface that every user knows how to operate, while a physical progress bar was implemented by dropping rubber balls in a tube.

Such meaningful tangible representation can be useful to provide users of IoT objects with more intuitive controls on privacy settings. Indeed, Ambe et al. [42], provide as an example the Messaging Kettle, a system for supporting social interactions at distance through a device that detects and notify when someone is preparing tea with the distant kettle. Indeed, the Kettle Mate, an IoT separate object with a sort of eye for detecting the kettle temperature, can be simply turned away, if the user does not want to share the kettle activity. At the same time, the enchanting glowing light of the Messaging Kettle for indicating that the distant kettle is boiling was well appreciated by the users, not only for its intrinsic semantic value but also for its hedonic added value.

As an alternative to complex smart objects, several everyday physical objects or artefacts can also be augmented with tags in order to give physical forms to different functions or behaviours. For example, Grosse-Puppendahl et al. [45] augmented objects with special capacitive NFC-tags to give them different functions, such as a rubber to turns on/off a lamp and a lighter to trigger a special fire-like lighting profile. Although limiting the physical embodiment to cards with icons, de la Guía et al. [55] proposed task actions embodied in physical cards.

Vaisutis et al. [44] give a different perspectives on the meaning that can be associated with tangible objects: “Early work in tangible interaction emphasized the possibility of coupling digital and physical representations, while later work recognized the interweaving of the material/physical and the social aspects of interaction from a design perspective.” Ambe et al. [42] suggest also allowing users to personalize the meaning associated to physical representation, supporting device appropriation with design of controls and representations that leave space to users’ interpretation.

5.2. T2: Rich Interactions and Human Skills

This tangible interaction property aims at promoting the exploitation of natural human skills, in particular, haptic interactions and rich manipulations with IoT objects that are situated in the physical world. While most interactions with nowadays IoT objects are constrained behind a screen and can be operated with a tap or swipe of one finger, our body can be used for much more complex interactions. Designers should be aware of such possibilities and try to exploit all our sensorimotor skills, both for perception (for example, using peripheral vision or haptics) and execution (for example, with full-body or bimanual interactions). Indeed, Ambe et al. [42] highlighted the importance of the whole body in the human understanding of the objects, stressing that: “While objects are often manipulated with the hand, they are experienced by the entire body.”

In their work Expressing Intent, Ng et al. [57] deserved a strong attention to rich interactions, criticizing the limitations of digital interfaces for interacting with the physical world. In particular, the authors used Frens’ framework for Rich Interaction [65] for integrating form, interaction and function into the proposed interactive objects, exploring in particular audio outputs and spatiality in their object designs. Also the authors of Iyagi [59] stressed the importance of rich interactions for their interactive system for bedtime routines, citing as example the Exploratorium of San Francisco, where children explore complex concepts through physical interactions. Scott-Harden suggested that an example of possible implementation of Active Forms [52] would use most of users’ senses, such as “sight, hearing, smell, taste, touch, balance and acceleration, temperature, kinaesthetic sense, pain and also other internal senses”.

Several IoTT object implementations provide concrete examples of rich interactions. Matviienko et al.’s Awarekit [53] exploits different feedback modalities such as light and sound embodied in IoT objects “without need for a smart phone or desktop application” and rich user interactions, such as rotating, touching and connecting modules. Grosse-Puppendahl et al. [45], while initially designing IoT connected plastic cards with braille text for blind users, later proposed 3D objects with audio hints that can be activated by shaking the object. In TANGerINE [56], integrated LEDs and haptic vibrations were provided as feedback modalities. Finally, Molyneaux and Gellersen [61] elicited three different rich interactions that users can perform with their projected interfaces: “Direct manipulation of object (e.g., shaking); Manipulation of object morphology; Manipulation of physical interaction components.”

The reviewed IoT toolkits also aim at supporting the definition of rich user interactions. Bellucci et al. [63] suggested that rich motion gestures can be used to interact with their T4Tags using a programming-by-demonstration approach, i.e., personalized gestures that the system can learn from users. Nevertheless, “shake”, “swipe”, and “up-down” predefined gestures were also provided. Similarly, Mora et al.’s [64] toolkit, provided a set of standard input/output interaction primitives that are easily supported by their technology, including touch, shake and rotate, as inputs and LED and haptic feedback, as outputs.

5.3. T3: Persistency

This property stresses that persistent physical representations could last in case of power or connectivity outrage, allowing the user to control the state of an IoT object even when the object is turned off or no Internet connection is available, while, in comparison, smartphone interfaces always require connectivity to control IoT objects. Surprisingly, we could not find any specific reference to this property in the selected articles, although some systems (for example those based on table tops) might inherently support this property. The closest argument to this property was provided by Möller et al. [54] while discussing the physicality property of Cognitive Objects (CO): “CO are no agents or digital representations but physical objects incorporating affordances for humans and robots.” suggesting that the interaction with cognitive objects could last beyond their numerical representation. Unfortunately, no practical implementation of Cognitive Objects has been presented to support this property.

5.4. T4: Spatial Interaction and Collaboration

Spatial interactions are typical in tangible interaction, since they exploit the physical environment to support not only collaborative setups with multiple IoT objects but also spatial reasoning.

Example of spatial arrangement of different IoT objects can be found in AwareKit [53]. Indeed, plastic figures representing colleagues and containing RFID tags can be approached to Awarekit modules to obtain contextualized information. Moreover, modules can be interfaced together through magnetic connections, for example to combine the Day and Week modules. Grosse-Puppendahl et al. [45] designed similar spatial interactions, for example changing the colour of a lamp when the user approach and lean a bottle containing the corresponding coloured liquid in its proximity, as to pour the bottle content over it. Similar spatial arrangements based on proximity are suggested also for Projected Interfaces [61].

The support for collaboration provided by interaction in space is often stressed. In TANGerINE [56], authors highlighted the importance of providing multiple access to the interaction to support distributed participation. To this purpose, each child had at her disposal a SMCube to interact with the system and different levels of interaction involvement were defined thanks to nested areas surrounding the table tops. Lebrun et al. [61] also stressed that in table top setups “tangible objects […] can be manipulated by one or several users”.

The innate support for collaboration was evident also in VoxBox [43], a tangible questionnaire that was originally conceived for a single usage at time. Surprisingly, VoxBox was used instead mostly (61% of usages) by groups of 2 to 5 people. The possibility to access the tangible questionnaire in groups facilitated discussions among group members, avoided isolation of people waiting and supported participation and engagement of all the group members. Ng et al. [57] suggested also that including several users in the interaction with IoT objects can enrich the overall experience: “By including multiple human actors in the mix, the objects form an interesting matrix for each actor’s preferences, intent and interactions to be interpreted and harnessed in multimodal outputs”.

Spatial interactions are supported also by all the reviewed toolkits [62,63,64]. Indeed, RapIoT [62] supports the interaction with several devices placed in the same environment and therefore users can collaborate with different devices towards a common goal. TILES [64], instead, allows using square modules as physical pixels that can be scattered in space or moved side by side.

It is worth noting that the networking capabilities of IoT objects enable new scenarios for tangible interaction. For example, Ambe et al. [42] showed how spatial interactions can occur also across distance. Indeed, the Messaging Kettle supports the connection not only with a distant IoT connected kettle but its messaging capabilities allow also connection between geographically distant people during their daily routines (in this case, boiling water in the kettle). In particular, they stressed the importance of leaving the possibility to the user to customize the spatial arrangement of IoT objects: “The Messaging Kettle was then conceived to take advantage of spatial interaction strategies already in use in the elderly person’s home”.

5.5. T5: Immediacy and Intuitiveness

Another property of tangible interaction that is often cited in literature is the ability to reduce learning time thanks to the immediacy and intuitiveness of the interaction. Indeed, the affordances provided by physical objects enable interactions that can be considered more “natural” than traditional digital interactions.

The authors of Active Forms [52] suggested that designers of connected objects should take inspiration from our surroundings, e.g., nature, to ensure that the interaction is intuitive and the physical properties of the object can provide hints about its usage. The authors of Cognitive Objects [54] have a similar purpose, since they suggested to embody the necessary a-priori knowledge to use and interact with an object into the respective cognitive object. Also Grosse-Puppendahl et al. [45] stressed the importance of allowing the user to directly identify and combine the objects, exploiting their knowledge of object manipulations and movements. As an alternative way to design IoT natural interactions, Ambe et al. [42] suggested using existing household devices and augmenting them with lightweight and intuitive interfaces. As Ambe et al. and Bellucci et al. [63] consider augmenting existing objects with sensing capabilities and exploiting affordances of existing objects as a means to facilitate learning of T4Tags possible interactions.

Intuitiveness of the interaction is particularly important for older adults, as shown by the authors of the Easy Smart-Home System [55]. The result of the experience conducted with older adults confirmed that users did not need prior knowledge of the system or device to use it and they intuitively performed their tasks without focusing on the system. Similar positive results were shared by Matviienko et al. [53] (“Overall, participants found AwareKit [...] easy to use. The shapes of the modules were functionally intuitive. For example: “Tipping was easy and intuitive to do.””) and by Baraldi et al. [56] (“our initial experience with users highlighted that the approach is very lightweight: especially on collaborative systems we found users to promptly understand the paradigm and use the cube in a natural way after some trial attempts.”).

The authors of VoxBox [43] also acknowledge the benefits provided by tangible inputs with clear affordances to “help people to immediately know how to use the device and reduce worries of not using it correctly”, while for Projected Interfaces [61], authors speak about “serendipitous bi-directional user interaction with smart tangible objects”.

5.6. T6: Peripheral Interaction

In an overwhelming world of digital technologies that might continuously compete for the attention of the user, there is an increasing need of interactions with IoT objects that are integrated in the daily routines, that free users’ cognitive resources and do not disrupt attention. For this reason, peripheral interaction [9] is gaining considerable attention in the HCI community and it deserves attention also for designing interactions for the Internet of Things.

Concerns about reducing the users’ cognitive load were raised by Scott-Harden [52] for Active Forms, who underline the challenge of trading-off the cognitive load on the user, selecting the correct interaction modalities with appropriate media bandwidth and handling user attention.

Also Ng et al. [57], while proposing objects that behaves autonomously and that can interact with the user according to their personality, highlighted an important concern about the possible disruption of attention. Several questions are raised by the authors: “Should all objects be able to interact and interrupt human actors at the same level, or are there those which might only be limited to a periphery? If so, which ones and why? Will the eventual information overload discourage human actors from having these animistic objects inhabit their physical space, or does the organic interaction between objects remain attractive to users?” One possible answer can be found in Matviienko et al. [53], who suggested that tangible interaction is able to ease switching between peripheral and foreground attention. In AwareKit, this is explored through lights as ambient feedback in office environments, notifying the users without disrupting their attention.

To this purpose, it is fundamental to integrate interactions with IoT objects in users’ daily routine. For example, Scott-Harden [52] highlighted that “The integration of the Active Form in its environment is very important, in line with the work on calm and ambient technology”. Also the goal of Cognitive Objects [54] is supporting the user (and possible robotic systems) in daily routines, collaborating with them and with the environment, “proactively and situated”.

The integration of IoT objects in daily routines is not only important in work environments but also in domestic contexts. Vaisutis et al. [44] analysed the role of everyday objects in older adults’ routines to understand which could be the role of IoT objects for fostering social connections. The authors discovered that some objects, which they identified in the category of “Objects of routine and comfort,” are profoundly interlaced with users’ life-stories and their daily routines. To address better these needs, they also suggest a paradigm shift in tangible interaction research: “research has largely been based on discrete tangible and embodied systems and has not examined multiple networked objects embedded in everyday lives”.

Ambe et al. [42] stressed further the concept of technology individuation during daily routines, citing as an example the Messaging Kettle. The author also argue that new routines can be created around these new connected objects and these new routines should be integrated into existing practices and places. Ambe et al. highlighted also that feedback integrated in everyday objects (which could be noticed through peripheral vision) might help establishing these routines. As an example, they cite the cases of the Whereabouts Clock (which shows information of people location in a clock-like device and users can be informed just at a glance) and the Messaging Kettle (which informs of distant activity on the connected kettle with a glowing light). Krishnaswamy et al. [59] also tried to integrate their IoT system in domestic daily routines. In this case, the Iyagi system should integrate voice interaction and everyday objects into bedtime routines, providing immersive stories and engaging activities that could support parents in these routines.

The three development toolkits also tried to support interactions integrated in everyday routines. The creators of RapIoT [62] argued that IoT enables seamless interconnection of people and everyday objects, bringing collaboration off the screen into our everyday routines and environments. As an example, their toolkit can also support visual warnings with green-red LEDs. Mora et al.’s TILES [64], thanks to their small form factor, are also designed to be attached or embedded in everyday objects and can be used “as ambient interfaces for non-intrusive information awareness.” Finally, Bellucci et al. [63] reported encouraging insights from their user studies, showing that T4Tags were easy to integrate with everyday objects already used for managing information, e.g., calendars, to-do lists, notebooks, or recipes.

5.7. T7: Reflection and Memories

This property highlights a different nature of tangible interfaces. Indeed, some tangible interfaces can stimulate user reflections, eventually promoting behaviour change. In other cases, tangible interfaces can facilitate recollecting memories associated to objects. In both cases, the interaction is mindful and at the centre of the user’s attention.

Sometimes this property emerges even in systems that are developed to be intuitive and easy to use. For example, Gallacher et al. [43] noticed that users that engaged with the VoxBox filled very seriously the questionnaire and take their time for reflecting before answering a question, although the system was mainly designed to be playful. The possibility that users have to play with sliders and spinners before setting the final answer might also have supported a different way of reflecting on questions, based on physical thinking. Moreover, the possibility to discuss the questions with other people in their group before selecting their answer could have facilitated reflection. Authors report even of individual users that were seen talking aloud to themselves about the current question.

A different type of user reflection was evidenced by Hannula et al. [58] for the IoT Owl. Indeed, the designed interactions aimed at fostering reflections towards behaviour change. The Owl can force the user (an employee sitting in the office) to stand up by starting to move its wings and to light up LEDs. Since this performance can be ended by the user touching owl’s claws, the interface forces the user to move, remembering him of the negative health effects of sitting too long. Mora et al. [64] also suggested that their platform can be used to build objects that support behaviour change, for example placing a square module next to a water faucet to display over-average water consumption.

Gallacher et al. [43] elicited another important aspect from Voxbox utilization, i.e., how simple tangible designs might easily support memory recollection, especially in the current trends of a world that is more and more digitalized. Indeed, many users reported feelings of nostalgia and fond memories of tangible toys and objects from childhood. In relation to this aspect, Vaisutis et al. [44] stressed that designing IoT for supporting social connections should take into account “how memories associated with objects might be enhanced with technology”. Indeed, from older adults’ interviews, they noticed that some objects could harbour important memories for the users. For example, for an interviewed older adult, “The stereo was not simply a device on which to play music but a reminder of the speech he gave at his daughter’s wedding”. In this manner, objects get a special significance not only for their function but also for the stories that are attached to them.

Also Ambe et al. [42] stressed that not only simple objects but also IoT device can obtain special value after long-term use and reflection. Therefore, a technological device can become a new object with unique and personal values, can reflect the owner’s self-identity and, at the same time, the use of technology can modify the owner’s self-identity. As an example, the authors describe the particular usage that a user did of an emergency care pendant, putting it on a crucifix (instead that wearing it on the neck) to remind her about mortality and relying on faith to relieve the stress about this thought. Also for the Messaging Kettle, the authors suggest that its simple aesthetics evoked memories, allowing the user to associate the object with special meanings. Interactions that facilitate memory recollection were supported also by the T4Tags of Bellucci et al. [63], who noticed how users used T4Tags in long-lasting scenarios, for example, “linked to media content to augment physical objects, kept as memories, given as a gift, or used as a password”.

5.8. T8: Long-Lasting Interactions and Emotional Bonding

This last property highlights the need of long-lasting interactions with IoTT objects. Designing sustained interactions has not only the aim of increasing the user experience but has also an ecological value for avoiding electronic waste related to IoT object obsolescence. While integrating objects within everyday routines could help to this purpose, sometimes a stronger emotional relationship with the object can be supported through tangible interactions. This can be achieved in different manners, for example giving to the object a personality, an own will or associating to an object a personal value, for example the relationship with a distant relative.

Vaisutis et al. [44] note that in general “people have strong emotional attachments to their things.” Such emotional attachment often goes beyond the object itself since is related to stories or people that have some relationship with the object. The authors end up with recommendations for supporting the design of IoT objects that foster social connection, which should take into account the emotion and the social relationships attached to the object and the type of communication that such IoT object should facilitate for older adults.

The purpose of Ambe et al.’s article [42] was directly linked to this property, since they aimed at investigating how tangible interaction with everyday objects and augmented with IoT capabilities can build on the emotional value that people attach to their cherished objects and routines. From the analysis of three previous systems, they could individuate several elements that allow developing emotional attachment with IoT objects. In particular, they introduced the concept of technology individuation, which “makes the technology distinctly personal as it embodies personal history, emotional attachment and time and effort spent in personalization that reflects the owner’s identity and relations.” They also suggested that the strength of this emotional bond is proportional to the time spent personalizing the object. The analysis of the Messaging Kettle and Whereabouts Clock systems shows that the object acquires an additional function, which is creating a communication opportunity and an emotional bonding with their beloved. For this reason, the technology makes the everyday object acquire a new value, which is generally linked to the daily routines that it supports, rather than to the object itself. The authors highlighted that such routines can support sustained use and, in the end, can help overcoming object obsolescence.

Ng et al. [57] developed significantly the importance of giving IoT objects a personality, i.e., object animism. Indeed, IoT objects that boast anthropomorphic elements are able to enhance the human interaction with them allowing developing a dimension of empathy and enchantment that goes beyond traditional human-interaction for mere functional purposes. Also Hannula et al. [58] tried to incorporate such principles in the design of the IoT Owl. Indeed, the IoT Owl was designed to have a personality, thanks to its particular shape and behaviour. Interestingly, some animistic features were generated almost unintentionally during the design process: “the noise of the servo motors [of the wings] sounds like the IoT Owl is trying to communicate with the user verbally.”

6. Discussion

6.1. Trends and Perspectives for the IoTT

As first discussion argument, we would like to deepen the analysis of articles found during the systematic literature review, in order to determine a general trend for the research field of the Internet of Tangible Things. While the buzzword of the Internet of Things has recently gained traction in research and in the industry, with no exception for the field of Human-Computer Interaction, some of preliminary works we found date back to 2008–2009. Older technological platforms relied often on table top setups (as support for tangible interaction) and on RFID and NFC (as network technology for the Internet of Things), which limited the communication capabilities and the intelligence embodied in IoT objects. Recent technological advancements and the diffusion of prototyping platforms, which make easier embedding technology in small-sized battery-powered everyday objects, is enhancing the capabilities of IoT objects. Big technology players, such as Intel, are also pushing this market and its technological advancement, supporting researchers as well as DIY and maker communities. As a result, not only objects gain sensing capabilities that were impossible with previous RFID and NFC IoT technology but they also enable richer interactions with the user. Thanks to the diffusion of prototyping platforms, we noticed a flourishing of work-in progress papers illustrating the development of IoTT objects. We believe that designing objects that embody tangible interfaces for interacting with the object itself and with the larger amount of information provided by the Internet of Things is the aim of the IoTT field of research. The concepts of Active Forms [52] and Cognitive Objects [54] partially supported this research question. Unfortunately, it seems that no concrete effort has been pursued to implement these two concepts. On the other side, we found three toolkits that aim at facilitating the development of IoTT objects, not only for engineers and designers but also for end-users with no programming skills. Considering the large amount of work found in the last years in this field and the continuous traction of the larger field of the Internet of Thing, we expect an increasing number of researchers approaching the field of the Internet of Tangible Things.

Besides the technological aspects linked to the development of IoTT objects, we would like to encourage a user-centred approach for the design of the interaction with the IoTT. Indeed, few of the works found in the systematic literature review followed a user-centred approach. The work of Vaisutis et al. [44] and Ambe et al. [42] are exemplar in this direction. Starting respectively from user-interviews and from the analysis of long-term user evaluations of previous systems, they derived recommendation for the design of IoT objects that support social connection and that integrate in daily routines. The work of Bellucci et al. [63] shows the benefits of conducting exploratory evaluations with end-users and refining the system according to their feedback. Besides these articles, only two papers [43,55] provided a formal user evaluation and reported user insights, while most other work provided just preliminary evaluations or no evaluation at all. Since this is a new field of research, we can argue that most of the works are still in an exploratory or work-in-progress phase and further evaluations will be published in the future. However, proper user-evaluations should be conducted on a long-term basis and this is a challenging task (even in the context of funded projects) as highlighted by Bellucci et al. [63].

6.2. Discussion of Tangible Interaction Properties

As shown in Table 1, most tangible interaction properties were cited by several papers, with the only exception of the persistency property, which was almost not mentioned at all. It is worth noting that there was a general agreement among reviewers about the presence or non-presence of properties in each paper and disagreement was always solved after discussion (cf. Section 4). Nevertheless, some tangible properties are closely related and nuances between them could be subtle. Indeed, closely related properties were the main cause of disagreement between reviewers’ annotations. For example, T1, i.e., the use of meaningful representations and controls of IoT properties or functions, implies often T5, i.e., an interaction that is intuitive and easy to learn. Both properties rely on physical affordances and interaction embodiment, which are one of the pillars of tangible interaction. The little nuance that can be distinguished between T1 and T5 is that the meaningful representations and controls rely often on metaphors and on the semantics provided by the context of the application, while intuitiveness and immediacy can be achieved through low-level concepts, such as naïve physics [66] or embodied metaphors [67]. Some authors referred to intuitive and immediate interactions as natural interactions. However, “natural” is an ambiguous term especially when referred to interaction, as Norman [68] underlined in a famous paper. Indeed, natural might also allude to the fact that the interaction is exploiting our innate natural human skills, such as our ability to manipulate physical objects. This was also a reason of initial disagreement with one reviewer, who considered natural interactions as related to T2 (rich interactions) instead that to T5. Rich interactions (T2) such as complex object manipulations, possibly performed with two-hands or with the whole-body, or interactions that exploit unconventional human senses, might be not immediate at all and, in many cases, require learning new skills. Users would often prefer immediate interactions such as swipes and taps [69,70], also because they are habituated to this kind of interactions with the omnipresent smartphone. Nevertheless, are not the most engaging human activities, such as playing an instrument, practicing a sport, or driving a car, also those that require the longer time for learning? At the same time, those activities that require much of our attention during the learning phase, after some habituation, start to disappear into our daily routines and we free cognitive resources that can be used for other tasks (T6). In this context, peripheral interaction (T6) has the role to exploit these free resources in an obtrusive manner, allowing interacting with IoT objects while performing other activities. Obtaining such goal is not trivial; it requires a good knowledge of human brain sensorimotor capabilities and may imply exploiting unconventional interaction modalities (T2).

On the other side of the spectrum of tangible interaction’s hybrid nature [33], we have interfaces that might not be so straightforward and that may require all our attention. These interfaces can stimulate reflection (T7) and even promote behaviour change. Physical objects might also have personal meaning and can be linked to memories (T7). In this case, the meaningful representations (T1) are not tied to the intuitiveness (T5) anymore but are rather associated to those personal memories (T7). Moreover, whenever the object is tied to personal memories (T7), there is often an emotional attachment with the object (T8). Nevertheless, a designer might want to make the user develop an emotional attachment to a new IoT object, therefore, with no memory associated to it, or ensure long-lasting interactions with the new IoT object. Schmitz [10] encouraged animating objects with personality and providing unexpected behaviours to help creating an emotional attachment between the user and the life-like object. Long-lasting ephemeral interactions are also explored by the Expressing Intent [57] project, which animated IoT objects with different personalities. Nevertheless, the authors raised important questions. Should all the objects be able to expose such personalities? Should they be able to prompt the user at their discretion, possibly disrupting her attention (T6)? The authors suggested that some objects can expose such behaviours while others should not, depending on their role in our life. Conversely, Bakker and Niemantsverdried proposed to start exploring the full interaction-attention continuum [34]: some objects should be able to catch the user’s attention in few appropriate contexts, while unobtrusively supporting our daily tasks during normal routine, intelligently switching between the two behaviours.

Also, Spatial Interaction (T4) is a nuanced property in the context of tangible interaction. Indeed, it is well known that most tangible interfaces, offering several distributed inputs, can foster collaboration. At the same time, collaboration can facilitate discussion, which is tied implicitly to the ability to make the user reflect (T7). Similarly, interaction in space and, in particular, object manipulation in space facilitate spatial reasoning; therefore, spatial interaction can stimulate reflection even in non-collaborative setups. At the same time, object manipulation in space requires performing rich interactions (T2). This close link between T2 and T4 was also one of the sources of confusion between reviewers. Finally, spatial interaction get even more complicated with the IoT, especially when different users start collaborating and communicating at distance (cf. the Messaging Kettle discussed by Ambe et al. [42]). Can spatial interaction through distant spaces provide the same benefits of co-located interaction? Further studies might be required in this case.

Finally, it is worth discussing the Persistency property (T3), the only one for which we found almost no reference in related work. Although the possibility to interact with an IoT object even without Internet connectivity (or even when the object is not powered anymore) might seem uniquely a technological challenge, this property might have stronger implications on the user experience with IoT objects. Indeed, such property can increase the user trustiness on the IoT objects, allowing complete control even in case of temporary failure. In the end, it can potentially increase the perceived robustness of the system. Therefore, we encourage explorations of this property in future work.

6.3. Limitations of the Systematic Review

Although the purpose of a systematic literature review is including in the analysis all relevant work in a specific field of research, we are aware that some relevant previous works could be missing from the selected papers. In order to limit the number of papers that could be excluded from the analysis, we conducted the queries for the selected keywords on full-text rather than just on meta-data or on the abstracts. We also included a query for smart-object, since the IoT keyword could be missing in non-recent relevant work. Unfortunately, after completing the analysis, we realized that a relevant paper that we already identified during the narrative literature review was missing. Indeed, this paper was not among the 81 papers retrieved through the queries, although it does include in the full-text the internet of things and tangible interaction keywords. After double-checking the query, we concluded that the search on full-text might not work as expected on the ACM Digital Library. As a result, other relevant papers might be missing from our analysis.

The definition and identification of the eight tangible properties could be limited by an a-priori selection of previous work based on our knowledge of the field and on a narrative, non-systematic, literature review. A more formal process would have required to derive the tangible properties from the systematic analysis of the eligible papers but this process would have been particularly difficult without a-priori knowledge of what we were looking for. In the end, all but one tangible property was clearly identified in previous work, showing that most of them were particularly relevant and already cited in the field. As discussed in Section 6.2, we also detected some nuances and interdependence between tangible properties, which caused, in some cases, an initial disagreement after individual property identification (solved after discussion). Nuances and interdependences between properties are still present but we argue that this is an intrinsic characteristic of tangible interaction. Indeed, as highlighted by Hoven et al. [8], tangible interaction has no clear boundary but has many concepts and properties that should be considered to improve the interaction with digital information.

Finally, this article does not discuss the possible associations between tangible interaction properties and IoT properties (e.g., the object connectivity state or the ability to capture information) and how these associations can improve the user experience with IoTT objects. From the analysis of the 18 papers, we can conclude that most interactions were designed for IoT object functions, while few interactions were designed for other IoT properties, for example for improving the understanding and control of privacy setting.

To better explore how tangible interaction properties can be used to improve the user experience in relation to different IoT properties, we realized a design card set, which can be used by IoT interaction designers as tool for inspiration and reflection. The card set is described in Section 7 and evaluated in Section 8. Through the evaluation of the card set, we aimed also at further validating the eight tangible interaction properties for IoTT with researchers and practitioners working in this field.

7. Design Card Set

Card games are powerful tools for inspiring the interaction design process, especially design workshop with participants from different backgrounds. In tangible interaction design, Hornecker’s card set [71] materializes Hornecker and Buur’s framework [27] into cards with different concepts, supported by illustrations and design questions for stimulating reflection during the brainstorming phases or evaluation phases of the design process. Card sets are popular also in IoT interaction design. Mora et al. [72] presented recently a design toolkit for supporting the participatory design of interactions with everyday objects augmented with IoT capabilities. Their card set has been conceived for helping novice users understanding and designing IoT objects and services to but provides little support for designing rich interactions with IoT objects. Indeed, only primitive input and output modalities (e.g., shake, vibrate) are suggested by the card set. Dibitonto et al. [73] recently presented a card deck for supporting the co-design of IoT interactions in early phases of the design process, i.e., from the conception phase to experience prototyping. This deck is still in a development phase. Individuals and design companies on the Internet have released other card sets and toolkits for the design of IoT objects. Interesting examples are Know-Cards [74], Thingclash [75] and IoT Service Kit [76]. All previous card sets in this field aims at supporting the understanding of the Internet of Things and at supporting the design of IoT services but offer limited support for exploiting tangible interaction properties in the IoT.

The card set presented in this article has been developed to support the design of IoTT objects, i.e., IoT objects that embody a tangible interface. The card set (Figure 3 and Figure 4) is inspired partially from Hornecker’s card set [71]. It proposes a selection of 8 tangible properties and 8 IoT properties, which are discussed in detail in the following sections. Each card shows on the front side an illustration and the name of the property, while the back describes the property and presents two questions to drive the reflection on how it can be applied. Inspiring metaphoric images have been selected to illustrate properties, with the purpose to retrieve rapidly the card from the set, without reading the property name but also to stimulate reflection on the property. Image selection has been discussed by three HCI and IoT experts, with the purpose to individuate easy-to-understand and worldwide-known metaphors, avoiding metaphors belonging to a particular religion or culture. Images have been retrieved from Pixabay [77] and are copyright free, under Creative Commons CC0 License. Cards have been printed in A6 format and plasticized.

7.1. Tangible Properties Cards

The 8 tangible properties individuated in the tangible interaction narrative review and analysed in the IoTT systematic review have been used to realize 8 tangible interaction cards. In the following list, we report the 8 tangible properties, with the corresponding description (D) and the two questions (Q1–Q2) that we added for driving the reflection of the IoTT object interaction design. Tangible cards are shown in Figure 3.

T1: Meaningful Representations and Controls

(D)

Meaningful representations and controls of the IoT object functions and parameters.

(Q1)

Can the user understand the IoT object function from its form?

(Q2)

Are the most important parameters of the IoT object accessible through physical props?

T2: Rich interactions and human skills

(D)

Rich interactions with IoT objects that are situated in the real world, exploiting the natural human skills and senses, such as haptics and hand manipulations.

(Q1)

Can interactions be performed on the physical IoT object?

(Q2)

Are users exploiting unconventional sensorimotor abilities, beyond those typically used on GUI interfaces?

T3: Persistency

(D)

Persistent physical representations that could last in case of power or connectivity outrage, allowing the user to control the state of an IoT object even when no Internet connection is available.

(Q1)

Are users able to control the object’ls future behaviour even if it is not connected to the network?

(Q2)

Is it possible to change the future behaviour of the object even if the object is turned off?

T4: Spatial interaction and collaboration

(D)

Spatial interactions that support collaborative setups with multiple IoT objects.

(Q1)

Is the spatial arrangement between object and people relevant for the interaction?

(Q2)

Is the spatial arrangement of IoT objects facilitating collaboration?

T5: Immediacy and intuitiveness

(D)

Immediacy and intuitiveness of the interaction, facilitating the understanding and control of IoT objects with minimal learning time.

(Q1)

Can the user interact with the IoT object without previous learning?

(Q2)

Can interactions with the IoT object be completed in less than 4 s (micro-interactions)?

T6: Peripheral interaction

(D)

Peripheral interactions with IoT objects that are integrated in the daily routines, that free users’ cognitive resources and do not disrupt attention.

(Q1)

Can the user interact with the IoT object while performing another task?

(Q2)

Is the IoT object notifying the user at opportune time, without distracting her from other activities?

T7: Reflection and memories

(D)

Facilitated reflections on IoT object meaning and working principles, as well as support for associating and sharing memories.

(Q1)

Is the IoT object form and behaviour supporting user reflections about socially or individually relevant facts (such as personal health, environmental pollution, etc.)?

(Q2)

Does the IoT object support remembering stories and facts, exploiting its physical form as tangible cue for memories?

T8: Long-lasting interactions and emotional bonding

(D)

Long-lasting interactions with IoT objects exploiting emotional durable designs, to cope with electronic waste due to technological obsolescence.

(Q1)

Can the form and/or unexpected behaviour of the object (e.g., life-like, zoomorphic) help developing an emotional bonding with the user?

(Q2)

Is the IoT object supporting appropriation and personalization?

7.2. IoT Properties Cards

Few works in literature have elicited IoT properties that should be considered from a human-centred perspective, in particular for the design of interactions with IoT objects. At the best of our knowledge, no work until now elicited which IoT properties should be considered for the design of tangible interaction with IoT objects. Our properties are partially grounded on existing IoT literature and in particular on the book of Rowland [20] and on Bellotti and Sellen’s [23] and Bellotti and Edwards’ [78] frameworks. The purpose of our study is also determining which IoT properties are more relevant for the design of IoTT objects and which tangible properties can be associated to them. This will be achieved through the analysis of the workshop results (Section 8). In the following list, as for tangible properties, we report the 8 IoT properties and the corresponding questions. IoT cards are shown in Figure 4.

I1: Function

(D)

The IoT main functions are clear to understand for the user.

(Q1)

Is the IoT object appearance meaningful for the user?

(Q2)

Does the IoT object offer easy-to-understand affordances for its use?

I2: Power

(D)

The IoT object can be easily switched on/off and power consumption is under control.

(Q1)

Can the user easily turn off/unplug the IoT object without any possibility to turn it on from the network?

(Q2)

Is the power consumption easy to monitor and to reduce through the interface of the IoT object?

I3: State

(D)

Current state/activity of the IoT object is under control.

(D1)

Is it easy to understand if the IoT object is turned on and correctly functioning?

(D2)

Supposing that the IoT object can have different states or operation modes, can the user easily understand and control these states?

I4: Connectivity

(D)

IoT object connectivity can be easily switched on/off and the network links are intelligible.

(Q1)

Is it possible to physically disconnect the IoT object from the network (while ensuring its local functioning)?

(Q2)

Is it clear which are the other IoT objects and/or people that are (potentially) connected to this object?

I5: Information capture

(D)

User has full control and understanding of the environmental/personal information acquired by the IoT object.

(Q1)

Has the user a clear understanding of the sensors embedded in the object and of the information collected by these sensors?

(Q2)

Can the user easily control which information is acquired by the object?

I6: Information sharing

(D)

The user knows and controls to whom acquired information is sent.

(Q1)

Does the IoT object inform the user to whom the acquired information is sent, if any?

(Q2)

Can the user choose which information can be shared and with whom?

I7: Information storage

(D)

The user knows and control which information is stored, where and for how long.

(Q1)

Can the user choose which information is stored by the object and erase it at any time?

(Q2)

Can the user know or choose where the information is stored (internally/on another personal device/on the cloud)?

I8: Access control

(D)

The user knows who has access to the IoT object functions and information.

(Q1)

Can the user configure who can use the object functions (locally and/or remotely)?

(Q2)

Can the user easily understand and control who has access to the information collected or elaborated by the IoT object?

8. Evaluation of the Design Card Set

We evaluated the proposed IoTT card set during a workshop conducted with researchers and practitioners from the field of tangible interaction, in the context of the European Tangible Interaction Studio [2] (See Figure 5). 21 people actively participated to the workshop, which lasted two half-days. 6 groups of 3 or 4 people were formed. Before the workshop, participants were informed about the purpose of the activity, with a short introduction to the Internet of Tangible Things and to the IoTT card set. Each group was asked to design one or more IoTT object for an application domain of their choice, using the IoTT card set as a support. The workshop was divided in three parts: (1) definition of the IoTT object idea and creation of a storyboard; (2) creation of a low-fi (cardboard) prototype for illustrating the functions and the interaction with the IoT object and (3) creation of a functional IoT prototype using Kniwwelino [3] as hardware IoT platform. Participants could use the IoTT card deck as a support for reflection during the whole activity. We asked to participants to keep track of the tangible and IoT properties explored during the different phases and to individuate the links among tangible and IoT properties (Figure 5, right). Participants were observed and briefly interviewed during the workshop by two organizers and 15 of them answered an online questionnaire after the workshop.

8.1. Workshop Design Results

Each team developed an original prototype throughout the three phases of the workshop. Each idea is briefly presented in the following subsections. For each project, we report only the most relevant properties and links individuated by participants through the process illustrated in Figure 5 (right). The complete results are presented in

Table 2. Property explored by each system at least in a phase of the workshop.

System/Properties	T1	T2	T3	T4	T5	T6	T7	T8	I1	I2	I3	I4	I5	I6	I7	I8	Total Tangible	Total IoT
Bobby	X				X	X		X	X		X		X		X		4	4
DUC	X		X		X				X		X			X			3	3
Eisenhower Matrix			X	X	X	X	X	X	X		X	X	X	X			6	5
FitFoot	X		X		X		X		X	X	X		X		X	X	4	6
Museum Memories	X			X	X		X	X	X		X		X	X	X		5	5
Tangigotchi				X		X	X	X	X		X	X	X				4	4
Total (Property)	4	0	3	3	5	3	4	4	6	1	6	2	5	3	3	1

.

8.1.1. Domotic Universal Control (DUC)

The DUC project explored the domotics theme. This IoTT object was ideated in order to provide a more intuitive control over domestic appliances, especially for controlling ambient lighting, heating and music, according to different scenes (Figure 6, left). Scenes are associated to the different facets of the cube and the user can simply rotate the cube to change scene. Participants highlighted links between meaningful representation and controls (T1) of IoT state (I3) and immediacy and intuitive interactions (T5) with IoT function (I1).

8.1.2. Bobby

This project explored personal health and domotics themes (Figure 6, centre). Bobby is a companion that monitors users’ physiological parameters such us heart rate and body temperature and adjust the domestic environmental conditions to improve our well-being, for example opening the window when the user is feeling too hot. Participants highlighted several links between tangible and IoT properties. Among them, it is worth noting the design of meaningful representations and controls (T1) for the IoT state (I3) and information capture (I5) and storage (I7), thanks to physical add-ons for physiological sensors, which can be detached to turn-off data acquisition. The purpose of the object was also developing an emotional bonding (T8) thanks to the useful everyday function (I1).

8.1.3. FitFoot

This project explored the personal health theme. FitFoot is a pair of smart shoes that track users’ activities and engage them towards a fitter lifestyle (Figure 6, right). While in the first phase participants noted meaningful representations (T1) of IoT function (I1) and persistency (T3) of power (I2), they highlighted in the following phases reflection and memories (T7) of information capture (I5) and storage (I7).

8.1.4. Tangigotchi

This project explored personal health and environmental monitoring themes (Figure 7, left). Tangigotchi is a companion that suggests to the users when they should go outside for performing physical activities, based on whether and pollution information as well as users’ activity tracking. As for Bobby, participants highlighted a link between long-lasting interactions and emotional bonding (T8) with the IoT function (I1) of the companion as well as peripheral interaction (T6) with the IoT state (I3), mostly conceived through ambient light. Finally, they considered reflection and memories (T7) with information capture (I5).

8.1.5. Eisenhower Matrix

This project explored a different theme, related to coordination and collaboration in work environments. The project is based on the idea of the Eisenhower Matrix for time management, with different quadrants that correspond to different degrees of task urgency and importance (Figure 7, centre). Tangible tokens represent tasks, which can change state according to the activity of other colleagues who have a similar system. Participants noted 6 tangible properties and 5 IoT properties, although they did not note any connection between tangible and IoT properties, probably because analysing all possible interconnections was too time-consuming. However, they highlighted reflections (T7) as main tangible property, together with spatial interaction (T4) across distant connected matrices and persistency of information (T3), typical of table tops setups. Because different systems are connected and tokens represent information of distant users, they also highlighted connectivity (I4) and information sharing (I6).

8.1.6. Museum Memories

This project explored the theme of interaction in museums. This project supported memory sharing through different IoT objects: visitors can leave personal memories to other visitors thanks to tangible cards and a booklet (Figure 7, right). Participants highlighted a link between meaningful representation and controls (T1) and IoT function (I1) as well as between spatial interaction (T4) and information sharing (I6). Long-lasting interactions (T8) was also linked to information sharing and information storage (I7).

8.2. Discussion of Workshop Results

All participants were particularly engaged during the whole workshop, showing interest in the proposed activities and asking often for additional time to complete their prototypes. The card set was used especially in the first phase, where all teams browsed the cards and used them to support the discussion of the idea. Some teams used the cards also in the second session, while cards were barely used during the thirds session, since participants were particularly engaged with the technical development of their prototype. Although most participants enjoyed particularly the iterative refinement process, the idea of the IoTT object changed barely among the different sessions. This insight was confirmed by the annotation of tangible and IoT properties and by the description of ideas during the pitches, which generally were sticking to the original idea developed in the first phase. Nevertheless, two teams reported slight changes across the different phases. The MuseumMemories project benefited from the paper prototyping session, since they found novel functions that could be allocated in their physical prototype: “As we had space on the card-board, we added an extra zone to edit and display information in the centre of the “book””. The hardware prototyping was considered in some manner helpful to obtain more concrete functions of the object (“refinement of functionalities, taking into account what it is really possible”, “Functions got more concrete”) or adding new ones (“We added more feedback, as it was easy on the Kniwwelino”, “Addition of functionalities”). At the same time, the hardware platform restricted some initial ideas (“Limitations from hardware made rethink a detail or two”) and, in some cases, it limited the number of tangible properties initially involved. For example, the MuseumMemories project realized that they “were limited by the number and type of sensors, so instead of saving a memory in one of the spaces only by putting the card inside the corresponding pocket, [they] had to press on a button on the Kniwwelino”. As a consequence, they reported that the T5 property (Immediacy and Intuitiveness) was lost.

A summative analysis (Table 2) of the different prototypes shows that 7 tangible interaction properties have been explored by at least 3 projects. Immediacy and Intuitiveness was the most cited (T5). Surprisingly, no project noted (T2) “richer interactions and human skills”, which is a fundamental property in tangible interaction and was found in 10 papers in our systematic review. The reason could be explained by the comment of a participant (member of the DUC team): “I think that during the design phase I was trying to keep the interaction simple, because I knew that then it would be difficult to implement it with a prototyping platform.” Although we avoided presenting the prototyping platform before the first two sessions and we told participants to do not care about implementation concerns in the first phases, the previous knowledge of existing prototyping platforms might have hindered the ideation of more complex interactions. To avoid this technological bias, workshop organizers should probably omit in the activity presentation that an implementation session would follow to the ideation sessions. All IoT properties have been noted by the 6 projects at least one time. All projects noted the function (I1) and state (I3) properties, while information capture (I5) was noted by 5 projects. Access control (I8) and power (I2) were noted by only 1 project and connectivity (I4) by only 2 projects.

In general, we can argue that both tangible and IoT design space were variously explored, thanks also to the variety of proposed applications. Nevertheless, it is worth noting that Bobby and Tangigotchi, which were both designed as a companion, shared similar properties and tangible-IoT links (cf.

Table 3. Links between tangible properties and IoT properties. B = Bobby, D = DUC, E = Eisenhower Matrix, F = Fitfoot, M = Museum Memories, T = Tangigotchi.

Tangible/IoT Property Links	I1 Function	I2 Power	I3 State	I4 Connect.	I5 I. Capture	I6 I. Sharing	I7 I. Storage	I8 Access C.
T1 Meaningful rep. & ctrl	B M F		B D		B		B
T2 Rich interactions
T3 Persistency		F				D
T4 Spatial interactions				T		M
T5 Immediacy & intuitiveness	D		B		B M		M
T6 Peripheral interaction			B T
T7 Reflection & memories			M		F T		F M	F
T8 Long-lasting interactions	B T					M	M

): both supported long-lasting interactions and emotional bonding (T8) through the IoT function (I1), as well as peripheral interaction (T6) with IoT state (I3). This is an interesting insight that could be applied for most IoT objects that behaves as a smart companion. Indeed, although tangible interactions could be designed for supporting an emotional bonding and long-term use of the device, at the same time, peripheral interactions that do not disrupt the user’s attention should be also supported, for example exploiting ambient lighting for informing subtly of the IoT object state. This would support the integration of the interaction in the daily routines, exploiting the full interaction-attention continuum as suggested by Bakker et al. [34]. Other similar links between tangible and IoT properties have been evidenced in the MuseumMemory and Bobby projects, in particular for meaningful representations and controls (T1) of the IoT function (I1) and for the immediacy and intuitiveness (T5) of the interaction with the IoT information capture (I5). In this case, the two projects belong to different application domains; therefore, we can argue that physical affordances are always important to facilitate the user understanding of the IoT object function (such link was also highlighted by FitFoot in the first phase). Similarly, it is important that users have an immediate and intuitive representation and control of the information acquired by IoT objects, which is a key factor for supporting privacy in ubiquitous computing, as suggested by Bellotti and Sellen [23]. It is worth noting that participants generally considered the proposed tangible and IoT properties as pertinent and complete. Indeed, although the form in Figure 5 allowed to add additional properties, none of the groups considered additional tangible or IoT properties.

From the analysis of the evaluation and after the observation of participants’ interactions during the workshop, some people interacted and appreciated less the cards set, while others were enthusiastic about it. From the observation conducted during the workshop, we could evince that people with a different background were less akin to use the cards during idea generation. Other participants, instead, spent a considerable amount of time reviewing and discussing ideas through the cards, even at the expense of the storyboard completion (for these people, indeed, the first phase was considered as too short). While existing IoT card sets (e.g., Tiles [72]) are oriented to non-experts, especially for supporting participatory design with end-users or stakeholders, our card set, is mostly oriented to people that are already familiar with some principles of tangible interaction, in order to get further insights about how tangible interaction principles can be applied to the IoT.

9. Conclusions

This article presented a systematic literature review of tangible interaction applied to the Internet of Things. We defined this research field “Internet of Tangible Things” (IoTT). This work aims at lying the basis for future research in the field, which seems promising considering the increasing number of work published in the last years. We proposed 8 tangible interaction properties and we analysed the existing work in the field according to these properties. Out of 81 distinct papers containing keywords related to tangible interaction and IoT, we selected 18 relevant papers, which included 4 theoretical papers, 3 papers presenting toolkits for developing IoTT objects and 11 papers presenting IoTT systems. We found that 7 of the 8 properties were well represented in the literature and we provided examples of how the properties were mentioned, either as theoretical citations, as implementation solutions, or as user insights. We believe that such work and, in particular, the tangible interaction properties that we individuated can be inspiring for the design of interaction with new IoT objects. To this purpose, we promote a shift towards interactions that are embodied in everyday IoT objects and that offer richer experiences compared to current graphical user interfaces for IoT.

To support this shift, we presented a design card set that aims at facilitating the adoption of tangible interaction properties during the design of the interface of an IoT object. We evaluated the design card set through a workshop conducted with 21 participants, mostly working in the field of tangible interaction. In general, the participants considered the card set useful and considered the list of properties as complete enough. Indeed, no group individuated additional properties in the provided form for describing the designed interactions. From the analysis of workshop results, we showed also that each group adopted at least 3 tangible interaction properties and 3 IoT properties. It is worth noting that while most systems found in the literature review focused only on the IoT object function; workshop participants explored largely also other IoT properties, especially related to privacy concerns. Moreover, 3 groups explored the persistency tangible interaction property (T3), who was barely discussed in previous work in the field. Conversely, the rich interactions and human skills property (T2) was not noted by any group. Although this result could be driven by chance during workshop activities, the adoption of unconventional interaction modalities can be better supported during IoT design workshops, trying to involve unconventional senses and full-body interaction, as described by Caon et al. [79].

Participants to the workshop found also some difficulties in implementing ideas generated during the first part of the workshop, in some cases changing the initial design. A hardware prototyping toolkit based on Arduino that can support an easy implementation of tangible interaction properties for IoT objects would be definitely helpful for IoTT designers. Our effort is currently focused on this task. A refined version of the design card set and a first version of the prototyping toolkit will be tested in a studio at TEI’18 [80]. The design card set as well as the Arduino examples for the hardware toolkit will be publicly released on the studio dedicated website [81].

Interaction designers working in the field of IoT should benefit of the individuated tangible and IoT properties as a reference framework for exploring a richer design space compared to traditional GUI interfaces and they can use the proposed design card set to take inspiration during brainstorming phases. In particular, the design game could be useful to explore new and richer interactions in relation to particular IoT properties that are key for ensuring a better user experience, beyond the object function property.

To conclude this article, we summarize 7 take-home messages derived from the analysis of previous work and from the workshop experience, which could further inspire and guide the design of the Internet of Tangible Things:

• Tangible interaction affordances can facilitate the understanding and control of IoT objects.
• IoTT objects can support unobtrusive interactions in the periphery of user’s attention or focused interactions that support reflection and/or behaviour change. The right balance between these interaction styles should be found, considering the application type and the context of the interaction.
• Privacy and trustiness in IoT systems can be enhanced through tangible interfaces for IoT key parameters (such as connectivity, power, data collection and sharing), beyond the main system function.
• IoT object obsolescence and abandon could be avoided by building long-lasting interaction and an emotional attachment to the object, through life-like behaviours of the object and designing the IoT object for appropriation and personalization.
• Rich and unconventional interactions based on complex object manipulations might have a longer learning curve but do not exclude the possibility to be executed in the periphery of user’s attention and integrated in daily routines after extensive use.
• Intuitive interactions, in some contexts, might not exclude reflections at some point.
• IoT systems are characterized by working principles that go beyond existing users’ mental models. User-centred approaches and long-term tests are needed to ensure appropriate user-experience with IoTT objects.