**4. Results**

This section outlines the results of the study. It is divided into two sections were the first section outlines a descriptive overview of the data. The second section outlines the results in relation to the research question. It should be noted that three participants in the Game group reported that they did not play the provided game. This is to be expected given previous works suggesting that it is challenging to ge<sup>t</sup> users to participate in training [28]. All statistical procedures have been performed with and without those three participants. Results concerning the Game group are reported as *n(m)* were *n* is the result when the complete group is considered and *m* is the result when participants that did not play the game are omitted.

### *4.1. Data Overview*

Data was collected over a period of about two months and included 41 participants. Two participants were removed from the data set since they reported having formal training in cybersecurity. The data collection period was intended to be longer, but data collections stopped after a security incident where the IT department warned all students and staff at the university about phishing involving attachments. Continued data collection would have risked the validity of the data set. The mean participant age was 37. Twenty-three participants identified themselves as female and 16 as male. Twenty-three participants reported being employees and 16 reported being students. An overview of the mean and median values for the collected variables and the distribution form of the variables is presented in Table 2. Please note that eye-tracking failed for three participants and the participants included for the variable behavior\_tracked is therefore only 36.


**Table 2.** Data overview.

### *4.2. The Effect of Training*

The effect of training was assessed by first examining the proportion of participants that received perfect scores. A perfect score means that the participants used all phishing identifiers or identified all emails correctly. The proportions of perfect scores are presented in Table 3.

Table 3 suggests that participants who received training performed better than participants in the control group for the behavior variables and that the participants in the CBMT

group outperformed the other groups for the variable SCORE. The same tendency is seen in Table 2 where mean and median results for the different sample groups are presented. Table 2 suggests that participants in the group game performed slightly better than the control group while the participants in the group CBMT outperformed the other groups with a bigger margin. The exception is for the variable behavior\_tracked where the groups CBMT and game performed equally when participants who reported not playing the game were omitted from the game group.

**Table 3.** Proportions of perfect scores.


Kruskal–Wallis H test was used to identify variables with statistically significant between-group differences. The results are presented in Table 4.

**Table 4.** Kruskal–Wallis H tests.


The Kruskal–Wallis H tests sugges<sup>t</sup> that at least one sample is different from the others when *p* < 0.05, as is the case for the variables SCORE and behavior\_manual. The same is also true for the variable behavior\_tracked when users who did not play the game are omitted. Pairwise Mann–Whitney U tests with Bonferroni correction was used to test what variables that were significantly different from each other. The results are presented in Table 5.

**Table 5.** Pairwise post hoc tests. Please note that post hoc tests for the variable behavior\_tracked were only computed in the case when participants in the group Game, who did not play the game was omitted because the corresponding Kurskal-Wallis H tests was only significant in that case.


In this case, the difference between two variables is statistically significant when *p* < 0.05. Table 5 shows that CBMT is separated from the groups game and control for the variables SCORE and behavior\_manual while control and game cannot be separated. For behavior\_tracked, game and CBMT cannot be separated but are both separated from control.
