The questionnaire’s results appeared not reliable and were excluded from the evaluation. In fact, although figures supported the semantic differential scales, the aspects under evaluation were too abstract for children of this age. All the answers resulted in the same, oriented on the right axis of the scales.
7.2.2. Video-Recording Observation
The whole activity with children was video-recorded using an action camera, mounted on a tripod. The video was automatically saved in four parts, lasting twenty-two minutes each, for a total of eighty-eight minutes (1.5 h). The videos do not include the preliminary part in which children were welcomed in the room and the final part in which the filled the questionnaire and the other forms.
The video recordings were edited and subdivided according to the procedure listed in
Table 2 and subsequently coded using Boris [
47], a free and open-source software for video-coding and live observations. Some of the steps of the activity (1, 2, 3, 5, 11, 14, and 15) were excluded from the coding action, after a brief observation. In fact, these parts can be described as transition and propaedeutic steps. Thus, the observation addressed the most salient parts of the activity. The observation was integrated with the transcription and translation of children’s comments.
By observing the recordings, a set of fourteen recurring and relevant behaviours were identified and used for the video coding. Due to the difficulties of observing a group interaction in such a context (e.g., some children cover the others, some parts are too far from the camera, etc.), some of the reported behaviours refer to the overall group’s behaviours. Others, instead, were pointed out when at least one child showed a certain behaviour. These behaviours, listed in
Table 6, differ also in terms of event type and valence. Some events, such as
smiling or
verbal interaction, that are prolonged in time, were coded as a state event, indicating a start and stop time. Conversely, others behaviours, characterised by a limited length, such as
jumping, were coded as point event.
The behaviours are also divided into positive and negative, according to the activity’s valence they manifest.
The output of the video coding, consisting of ten plot graphs and time budget excel files, allowed to notice the overall feel of the experience, as well as some differences among the various phases. By looking at
Figure 5, it is possible to observe the general trend of the four main group behaviours: smiling, focused, concentrated, and silence. Children resulted overall engaged by the playtest. The level of focus on the activity, in fact, was over 90% for almost all phases, while just in phase 12 the focus has fallen sensibly. This decline of attention was reaffirmed by the solo and small groups behaviours illustrated in
Figure 6. However, by cross-checking with the video, the distracting element in this phase was represented by the robot. In fact, after the tutor has introduced Shybo to children, this was left on the central table, while the activity moved to the right table focusing on the introduction of some concept about colour theory. Moreover, the introduction of Shybo in phase 7 had an immediate effect. Children, who were very concentrated during the video about Shybo, when the robot was physically presented to them lost their serious and concentrated expression and started laughing frequently.
Figure 7 shows a peak in the laughing frequency, which grew from 0.5 events/minute in phase 6 to 2.25 events/minute in phase 7.
Less regular was also the level of smiling behaviour. During the first three phases, in fact, children appeared very smiling, while in phase 8, 9, and 10 this behaviour has fallen dramatically, to rise again in phase 12. However, the declining trend of smiling was compensated by the rising trend of concentration, which greatly varied among the phases. This inversion of the two trends manifests the nature of the activities carried out in those parts. In these phases, in fact, the tutor explained to children how to train Shybo, and let them do it. Furthermore, especially in phase 9 and 10, another behaviour rose together with the concentration: silence. Children were making silence and appeared serious because they were paying attention to the tutor’s explanations and because of the robot’s sensitivity to sound. Both silence and concentration decreased considerably in the following phases, after the training of the robot was concluded and children became familiar with its functioning and the game. This is also noticeable by looking at the trends in
Figure 6, where in phase 7 the general verbal interaction decreased considerably while some children started to ask questions about the robot. Nevertheless, the fall of smiling in phase 8 and 9 is not only traceable to the normal robot’s training activity. Looking at
Figure 6, in fact, it becomes evident that, in these two phases, some problems emerged. On the one hand, one of the two tutors did not properly connect Shybo to the software that was running on the laptop to perform the training. As a result, children started to get distracted and to appear bored. On the other hand, especially phase 9 in which children trained Shybo, the activity involved children at turns and required them to be quiet, resulting in a decrease of excitement and a rose of boredom.
The level of verbal interaction, as shown in
Figure 6, experienced a fall in the central part of the activity, in which children were mostly asked to observe and listen to the tutor explanations, and rose again from phase 12 to the end. On the contrary, the level of instrument noise, namely children playing the musical instruments when the activity did not require it, presents an opposite trend. This, in fact, increased significantly from phase 6 to 10, and ended in phase 12, since, in this phase, children had to leave the instruments on the table and in the following phases they were used for the game. Especially in phase 8 and 9, while the tutor was demonstrating how to train Shybo, children were often playing the instruments to see its reactions.
The final phases of the experience, from phase 12 to the end, were characterized by a very joyful atmosphere. In fact, although
Figure 5 shows a slight decrease in the focus towards the activity and in smiling,
Figure 7 presents an increase in laughing and a dramatic rise of jumping behaviours. From the beginning of the activity, in fact, children were sometimes jumping, manifesting excitement and joyful impatience. However, when the final game started, in phase 16, the frequency of this behaviour grew by 600%, moving from an average of 0.9 events/minute to 5.61 events/minute. The final phase (17) also reached a peak regarding the excitements, since it represented the end of the game and the victory of one team.
Despite the general excitement and the joyful atmosphere, by looking at
Figure 6 it is possible to notice some behaviours that might result in a negative valence of the experience. As already mentioned, the training phase and the error in the setup, in phase 8 and 9, resulted in a rise of boredom. From these phases, the level of distraction also increased slightly. In fact, although in phase 12 this was due to the robot, in these phases, some children started to move around the room, not very far but anyhow distracted from the activity. This was mostly happening when children had to wait for their group’s turn to play. Other children, however, approached these waiting phases by collaborating with the other teams to find the right instruments, or just watched and incited the game staying around the table.
Another important point event illustrated in
Figure 7 is “wow”. This represents the actual occurrence of children expressing a positive surprise by saying “wooooow”. This mostly happened in the phases in which new elements were introduced, that are the robot and the game elements. Especially regarding the robot, children expressed curiosity by also asking questions (
Figure 6).
The comments of children that were transcribed and translated, highlighted some children’s expectations toward the robot. In fact, a girl asked why Shybo, as a robot, cannot talk and has no legs. Another girl answered her that Shybo cannot talk at the beginning. Some children say that they would like Shybo to be able to reproduce the sounds they play with the instruments or their voices. Another interesting comment is about Shybo getting scared. Rather than thinking that it is getting scared, some of them thinks that it gets angry. In fact, during phase 12, in which children were constantly getting distracted and making noise to see the robot’s reactions, a boy was playing aggressively by walking fast toward the robot with a fist pointed at it. With this behaviour, he was simulating a sort of fight in which he was acting aggressively to make Shybo calm rather than angry.
Finally,
Figure 7 shows a point event that occurred only in one phase, namely when children started to play the game. A boy, who was entrusted of finding the right instruments to play, started to scratch his head manifesting a difficulty and making evident that he did not remember which object was associated with which colour. In fact, at the beginning of phase 16, some other children also commented that playing the game was challenging and that they forgot the colour–sound associations.
Another difficulty observed in the game regarded the game cards. Some of the cards were more difficult than others because these asked children to reflect on primary, secondary and complementary colours, rather than showing the colours directly. After some children did this kind of cards, the tutors decided to remove them from the game, because of the difficulty and the time required to children to play with these.