*2.4. Cueing Action Sequences*

In previous work, we explored the relationship between LEDs and the state of objects in a simple problem-solving exercise [33]. The objective is to ensure that four boxes had satisfied their goals (Figure 9). The goal of each box was defined by a set of rules known to the box, and defined by the position of the box and its proximity to other boxes. Each box has three LEDs representing its state: one to indicate if the goal has been satisfied, one to indicate 'communication status' (in terms of connection with the table), and one to indicate 'proximity'. We were interested in whether people would try to learn the 'rules' that the boxes were using or whether they would find it easier to learn the pattern, or arrangement of the boxes, and whether the rules or patterns could generalize to new configurations. The argument for this comparison was that the patterns could be considered in terms of 'affordance' [34–37]. Not only were the patterns easier to understand (which suggests that the visual cues provides useful semantic information) but also participants found it easier to generalize patterns than the rules (contrary to our expectations). *Sensors* **2017**, *17*, 2308 11 of 16

**Figure 9.** Visual feedback on networked objects [38]. **Figure 9.** Visual feedback on networked objects [38].

### 2.4.1. Procedure 2.4.1. Procedure

drawer and stirring before pouring from the jug.

to obtain timings. The following timings are obtained:

green, spoon lit, spoon stirring cup.

drawer is moved.

spoon picked up.

cup.

debrief.

The trials used the same participants as trial 1. Cues for stirring and pouring were tested with a combination of lights and sounds (from vibration motor). A sequence of actions similar to making a cup of tea was performed purely from the objects' cues. For a 'team making' task, the experimenter raised the handle of the jug and, once the participant had lifted the jug, the experimenter turned the LED on one of the cups to green. When the participant had poured the jug to the cup, the experimenter turned the LED in the drawer handle green and turned on the vibration motor in the handle of the spoon (inside the drawer). The participant opened the drawer and lifted the spoon, and the experimenter turned the LED on the cup (that had been previously used to pour into) green. The trials used the same participants as trial 1. Cues for stirring and pouring were tested with a combination of lights and sounds (from vibration motor). A sequence of actions similar to making a cup of tea was performed purely from the objects' cues. For a 'team making' task, the experimenter raised the handle of the jug and, once the participant had lifted the jug, the experimenter turned the LED on one of the cups to green. When the participant had poured the jug to the cup, the experimenter turned the LED in the drawer handle green and turned on the vibration motor in the handle of the spoon (inside the drawer). The participant opened the drawer and lifted the spoon, and the experimenter

activations but had them appear in an order that did not feel correct, e.g., taking the spoon from the

Each sequence (logical or illogical) was followed by a memory test which entailed placing cards in the same sequence that was performed. The number of mistakes was recorded and participants were not told that they will perform a memory test at the beginning the sequence. Seven cards were used: jug with handle raised, jug tilted (pouring), cup lit green, drawer opened, drawer handle lit

Video footage from experiment three (with participant's permission) was analysed using ELAN

• Time taken to open drawer: Timer begins when light on the handle is on and timer ends when

• Time taken to pick up spoon: Timer begins when drawer fully opened and timer ends when

• Time taken to stir: Timer begins when spoon is picked up till head of the spoon is inside the

Midway through the experiment, only one cup lit up. There were no other audible or visual cues. In the experiment with lights on the jug, the majority of participants performed no action when the cup lit up. On the other hand, in the no lights experiment, the majority of participants picked up the jug and poured into the cup even when the jug's handle was not raised, so participants picked up the jug with two hands and poured. As with the previous trials, participants were interviewed at turned the LED on the cup (that had been previously used to pour into) green. The participant put the spoon into the cup (perhaps making a stirring motion) and the task was completed. This constituted the 'logical' sequence of tea making, and was repeated four times. Additionally, an 'illogical' sequence was also repeated four times: this employed the same object activations but had them appear in an order that did not feel correct, e.g., taking the spoon from the drawer and stirring before pouring from the jug.

Each sequence (logical or illogical) was followed by a memory test which entailed placing cards in the same sequence that was performed. The number of mistakes was recorded and participants were not told that they will perform a memory test at the beginning the sequence. Seven cards were used: jug with handle raised, jug tilted (pouring), cup lit green, drawer opened, drawer handle lit green, spoon lit, spoon stirring cup.

Video footage from experiment three (with participant's permission) was analysed using ELAN to obtain timings. The following timings are obtained:


Midway through the experiment, only one cup lit up. There were no other audible or visual cues. In the experiment with lights on the jug, the majority of participants performed no action when the cup lit up. On the other hand, in the no lights experiment, the majority of participants picked up the jug and poured into the cup even when the jug's handle was not raised, so participants picked up the jug with two hands and poured. As with the previous trials, participants were interviewed at debrief.
