*3.2. Usability*

The interaction of LOA and workload on usability was not significant (F (18.137) = 1.615, *p* = 0.064). However, the main effects of LOA (F (18, 135) = 7.768, *p* < 0.001) and level of workload (F (18, 137) = 11.905, *p* < 0.001) was significant. At high LOA, the usability was higher (M = 4.36, SD = 0.83) compared to the low LOA (M = 4.31, SD = 0.773), in agreement with H2. At LWL (M = 4.37, SD = 0.633), usability was higher compared to HWL (M = 4.25, SD = 0.742). Higher usability was obtained when task complexity increased (MWL1, M = 4.45, SD = 0.959) as compared to when there was a secondary task (MWL2, M = 4.29, SD = 0.835).

There was no difference in the workload groups in terms of the perceived ease of use. However, workload level significantly influenced perceived workload as measured through the aggregated raw NASA-TLX scores (F (3, 152) = 11.767, *p* < 0.001), with the HWL (M = 14.6, SD = 4.337) resulting in higher perceived workload compared to the LWL (M = 12.58, 3.796) as expected. Between the medium workload groups, MWL2 (M = 15.33, SD = 3.318) resulted in higher perceived workload compared to MWL1 (M = 11.18, SD = 2.123).

Workload also had significant effect (F (3, 152) = 3.646, *p* = 0.014) on perceived reliability as assessed through the questionnaire. The reliability was perceived as higher by the participants who experienced the LWL (M = 4.53, SD = 0.687) compared to the HWL (M = 4.5, SD = 0.555). Between the medium workload levels, MWL1 (4.63, SD = 0.628) resulted in higher perceived reliability compared to MWL2 (M = 4.19, SD = 0.634).

### *3.3. User Preferences*

A one-way ANOVA revealed that there was a significant difference between workload groups (F (3, 76) = 9.276, *p* < 0.001). When comparing LWL and HWL, high LOA was preferred. However, when comparing between MWL1 and MWL2, low LOA was preferred for the MWL1 (confirming H3). More details regarding user preferences for the LOA modes between the workload groups are depicted in Figure 7.

#### *3.4. Comparison between Workload Groups for Different LOA Modes*

Multiple comparison made between the different workload groups with details on each LOA mode for groups that were significantly different are presented in Table 2. Results revealed that at low LOA: QoT execution is higher when workload is lower; usability is higher when a secondary task is involved, and user preference tended towards low LOA when complexity increases. However, at high LOA: QoT execution was the same for all workload types except when complexity is involved; usability was higher when a secondary task is involved, and user preference tended towards high LOA when a secondary task is involved.

**Table 2.** Comparison of assessment (with *p*-values) within the workload groups \*.


\* green depicts comparison with statistical significance; similar trends are marked with identical colors.

### **4. Discussion**

The main influences and interacting influences of LOA in HRC in an assembly task context, considering different levels of workload is summarized in Table 3.


**Table 3.** Summary of findings.

#### *4.1. Influence of LOA*

In HWL situations, where additional resources are needed to complete the task in the least possible time and with minimal effort, high LOA is preferred. This corresponds with the observations made in the meta-analyses conducted in [38,39] where several automationrelated data where analyzed. It also agrees with the characteristics of the suggested line of solution in workload demands amidst multiple resources as elaborated upon in [37]. However, in cases where complexity is involved, as seen in the results for the LOA preference of participants in the medium workload category, a low LOA can be considered. Most participants seem to prefer a low LOA when the task complexity is high. This confirms H3, and is also in agreement with previous studies where it was stated that a higher LOA may not always give a positive outcome in situations where uncertainties, and higher probabilities of failure exist [38,39]. In high complex tasks where high component and coordinative complexity increases the probabilities of failure [23,49], humans usually have a higher potential to better manage unknown or unexpected situations [50,51]. This reinforces the significance of evaluating LOA modes alongside different workload situations as emphasized in [52] for various contexts and causes of workload. It also calls for further assessments using these constructs.

#### *4.2. Workload Considerations*

Workload had significant influence on most of the measures. The significant effects were seen in effectiveness and efficiency leading to reduced QoT execution in situations where the workload was high. This is consistent with the literature highlighting the contribution of task-related demands (such as mental, temporal, and physical demands, including complexity demands involved in the HRC task) to workload, which could negatively influence resources available to complete task at hand [15].

The medium workload category more clearly reflects some of the differences in additional workload which can be induced by a secondary task or task complexity. Secondary task inclusion (depicted in MWL2) seems to produce a higher perception of workload compared to complexity in the task (depicted through MWL1). This could explain the reason why most users preferred the high LOA (which autonomously executes more aspects of the task) compared to the low LOA for MWL2. The LOA option seems to provide more mental space for the users to execute other tasks, particularly when the automation functioned well, as suggested in [38,39].

This difference in the medium workload category also brings into prominence the relevance of task complexity, specifically the influence reflected through the perceived reliability where MWL1 (reflecting higher complexity) condition was perceived more reliable compared to MWL2 (reflecting secondary task influence). This could be a result of higher uncertainty and failure probabilities which complexity induces as elaborated in [53,54]. It is therefore understandable that users preferred low LOA to the high LOA in this level of workload (where the task complexity exists) where they seem to have an increased sense of control over the operation [55]. This enables them to better manage the higher uncertainties in this condition (through the low LOA) compared to relying on the robot (through the high LOA). The results reveal that both objective and subjective complexity considerations as noted in [21] should be considered along with the suitable LOA modes for such HRC assembly tasks. This consequently affects the QoT execution and usability of the system.

#### *4.3. Limitations*

Evaluation was performed with users who had experience with computers and robots. We expect these results to be amplified with users who have experience in real industrial setting. We are also cognizant of potential differences in the subjective assessment of the students in comparison to professionals in an industrial setting since this plays a role in the perception of the users working alongside a robot in a work setting [56]. We therefore consider the results obtained with caution, with the perception that these could

be relatively equivalent to assessment with novice operators and different from expert or professional assessments.

The LOA and levels of workload design is simplified for research purpose and not fully representative of the degree of automation, workload levels demanded in more industrial settings. The results obtained, therefore, serve as building blocks and insights for further developments where more detailed automation, workload and complexity conditions are tested in sample industrial settings. Some other social aspects of interacting (such as verbal [57] and non-verbal communication methods [58]) with the robot for the collaborative work were not explicitly investigated in this study. However, further research should also investigate the interplay of the socio-technical aspects of the collaboration while also considering economic and societal issues to understand fuller dimensions of improved HRC in industry [56,59,60].
