Cognitive Interaction Analysis in Human–Robot Collaboration Using an Assembly Task

Abstract

In human–robot collaborative assembly tasks, it is necessary to properly balance skills to maximize productivity. Human operators can contribute with their abilities in dexterous manipulation, reasoning and problem solving, but a bounded workload (cognitive, physical, and timing) should be assigned for the task. Collaborative robots can provide accurate, quick and precise physical work skills, but they have constrained cognitive interaction capacity and low dexterous ability. In this work, an experimental setup is introduced in the form of a laboratory case study in which the task performance of the human–robot team and the mental workload of the humans are analyzed for an assembly task. We demonstrate that an operator working on a main high-demanding cognitive task can also comply with a secondary task (assembly) mainly developed for a robot asking for some cognitive and dexterous human capacities producing a very low impact on the primary task. In this form, skills are well balanced, and the operator is satisfied with the working conditions.

1. Introduction

In the context of the Fourth Industrial Revolution, also known as the Industry 4.0 paradigm, collaborative and autonomous robots are emerging. They are modifying the way that tasks are performed. For instance, collaborative robots modify the safety conditions usually applied to industrial robots [1]; autonomous mobile robots increase mobility task skills by simultaneously following a route and avoiding static and dynamic objects, such as an industrial operator moving inside the plant [2].

A key element in this transformation to Industry 4.0 is the emphasis on a human-centered approach and full automation. This human-based transformation implies a paradigm shift from independent automated and human activities towards a human-automation symbiosis characterized by the cooperation of machines with humans in workplaces, which are designed not to replace (eventually, overcome) the skills and abilities of humans, but rather to co-exist and assist humans in increasing human well-being and production performance [3]. In recent reports, such as the Good Work Charter of the European Robotics Industry [4], fusion skills are defined as an interesting challenge: skills that draw on the fusion of humans and robots within a business process to create better outcomes than working independently.

In manufacturing systems, there have been attempts to completely replace the routine activities of manual assembly tasks with robotics, but these have been unsuccessful many times [5]. In the case of developing this work in the form of human–robot collaboration, an assessment was made of operator stress and is detailed in Reference [6]. In [7], the effects of robot appearance and the relative status on human–robot collaboration are investigated to the extent to which people relied on and ceded responsibility to a robot coworker. Negative impact reduction of integrating human–robot teams is investigated in [8], by maintaining human aspects, such as social interaction, autonomy, problem solving, and task variety. Hence, further studies about the role definition of the human operator in manufacturing applications is required.

Cognitive skills should be not only considered for the operators, but also for the robotics systems. Human–robot interaction design at a cognitive level is a key element for the success of the collaborative workplace [9]. In some cases, the operator could get help from an assistant with cognitive skills to improve the operator’s understanding of technology equipment. Hence, in [10], a cognitive work analysis method is applied for the design of an assistance system to support humans in the control of intelligent manufacturing systems. In [11], an intelligent decision making method is developed that allows human–robot task allocation using the robot operating system (ROS) framework. With the aim to decrease the workload of the human and maximize the user adaptation, a set of cognitive models (task model, truth-maintenance model, interaction model, and intention rule-base) is developed in [12]. A mathematical model is introduced in [13] relating the low workload of humans (physical, mental) to high performance in a human–robot collaboration framework, depending on the complexity of the task and whether the robot task is performed successfully or with errors (human intervention is required).

For cognitive interaction between an operator and a station, the type of interface that allows efficient dialogue should be considered. In [14], a framework (methods, metrics, design recommendations) is developed for the study of effective interface designs in collaborative human–robot interaction. Moreover, when collaborative robots progress towards cognitive robotics, the human operator should be trained to be at that same level of competence. In this context, cognitive architectures for human–robot teaming interaction must be developed and tested [15].

1.1. Role Allocation in Human-Collaborative Robot Workplaces

Human operators and robots can adopt several cognitive roles in collaborative human–robot workplaces. They are listed in the following in order of increasing cognitive workload for both sides; operator and robot.

1.1.1. Role 1: Human Operator/Collaborative Robot

The operator accomplishes a physical activity that involves attending to the flow of material in the station, replacing parts (feeding), taking a product to either another station or to the warehouse, and checking product quality. The adaptability of the station operation can allow the human and robot to either share the task or the task to be performed between them sequentially. The worker uses cognitive skills such as visual processing or attention. For stations endowed with visual indicators, the worker knows the state of the station: “robot working”, “worker intervention is necessary”.

The associated task could produce physical fatigue, so it is necessary to take care of the physical body and reduce stress and repetitive hand and arm movements. In case the worker must move the robot, the robot should have a manageable weight and be suitable for all human anthropometric variability. If the operator asks a technical expert for help, the following can be agreed on: the rhythm of production (i.e., the speed of the robot), the limit force value allowed in the case of physical contact, and the level of cooperation (i.e., human and robots work sequentially or complete a shared task together).

This low-level cognitive skills workplace is the one considered for the assembly task in our work.

1.1.2. Role 2: Human Technical Expert/Programmable Collaborative Robot

The human technical expert is provided with skills in robotic tasks, maintenance, programming and security. This role requires a holistic approach with flexibility to understand at all times which technical aspect is a priority. This expert is responsible for the start-up, operation and maintenance of the robotic station. The technical expert uses cognitive skills, such as reasoning, logic, problem solving programming skills.

This technical expert uses the robot teach pendant for communication with the robot. The robot programming is able to create a pop-up window and offer clear feedback to the human when an activity is finished: “an operator action is required”.

The robot programming software adapted to the human technical expert requires splitting the task difficulty into several levels. A program software must be easy to use, easy to configure and scalable. This software can have palletizing, assembly, or similar semi-configured routines. The programming must be designed with a level for novices, intermediate and expert human operators. The robot user’s manual could have a training program with tutorials, recommendations, examples and flexible possibilities in the development of applications. The start-up time of the robot must be short, i.e., quick unboxing, easy integration of end effector and a graphical programming interface with high usability.

1.1.3. Role 3: Human Supervisor/Robot Recommender

In this operational mode, the human supervisor accesses information on production management (key performance indexes, dashboards). The human supervisor plans, organizes, and analyzes the behavior of an industrial plant. The supervisor uses cognitive skills, such as reasoning, strategy analysis, decision making. Visual processing helps the supervisor effectively visualize representations of data, such as graphs and tables. The human supervisor needs a flexible human–machine interface with access to the database that makes it easier for him/her to understand the performance variables.

A cognitive robot could work as a human partner [16]. The functionality of a cognitive robot could be a recommender system. If the robot is connected to the intelligent manufacturing system, it could give relevant information upon human request. The cognitive robot can understand the complexity of the human ability and adapt to a specific understanding level.

1.1.4. Transition between Roles

Current technology allows a trained operator to easily move between the roles previously described. If the operator has a tablet, he/she can move from one station to another by consulting the relevant production information on the screen and at the same time assess whether a robotic station requires direct intervention.

From the cognitive point of view, workload and attention must be assessed. A priori, semi-automatic systems (collaborative systems) should facilitate effective teamwork between robots and humans. The human task must be adequately balanced so as not to excessively increase the assigned load. With regard to attention, it is necessary to analyze what type of attention is appropriate for the tasks that are being performed (sustained, selective, divided). If a task has high priority, then it has preferable sustained attention, i.e., the human is focused and motivated to continue the task and complete it. The other case is preferable selective attention, i.e., the human is developing the task even when there are distractions around him. In some situations, the operator needs to have attention in multiple places. Divided attention helps the human retain information while successfully completing two or more tasks at the same time.

Having to intervene in various places can cause stress for the operator and make it difficult for the chosen task to be carried out successfully. To understand in detail how humans behave when faced with the challenge of completing a task or interrupting it to carry out direct intervention at the station, the following section shows how to prepare a laboratory scenario in which to perform and evaluate human–robot collaboration tasks.

1.2. Objective

The evaluation of mental workload is a key issue to research and develop for human machine interfaces, as well as to find levels of comfort, satisfaction, efficiency, and security in the workplace [17]. Moreover, some researchers explain in detail how important the cognitive load and the mental workload are in the design of workplaces for assembly tasks [18]. A laboratory scenario in a manufacturing context has been designed to create an environment where humans can work side-by-side with robots in close proximity [19]. The human operator facilitates the assembly task carried out by the robot since it feeds the station with parts, collects the products and attends to any possible malfunction. The variation of the human operator’s mental workload is evaluated when switching from a main task into a secondary task in this human–robot collaboration workspace scenario. The general objective of the test in this scenario is to evaluate the variation of mental workload, when the mental workload of the operator is increased due to regular time-constrained collaborations with a robot. Our hypothesis is that when the operator performs the collaborative task with the robot, the mental workload value is not very far from the mental workload of the main task. Moreover, both values would be in an intermediate comfort zone of mental workload. If tasks are designed in which the perceived mental workload is in an intermediate zone, quality in performance and decision-making in problem solving are guaranteed [20].

The second section of this paper explains in detail the materials and methods used. In particular, the experimentation aims to evaluate the participants’ perceived mental workload through the adoption of the NASA TLX standard questionnaire, after the task development. Section three shows the experimental results, which are discussed in Section four. Finally, conclusions and future steps are presented.

2. Materials and Methods

This section shows the laboratory resources, the task allocation between humans and robots and the preparation of scenarios for the experimental sessions.

2.1. Human–Robot Collaborative Workspace

The experimental study of this work was developed in the laboratory scenario shown in Figure 1. A laboratory scenario has great adaptability for the identification of human–robot solutions, development of new methodologies, application of algorithms and evaluation of collaborative human–robot workstations [21].

The human supervisor plans the activities with one or more human operators, with one or more collaborative robots, tuning the task allocation between human and robot, giving visual feedback of the robot behavior to the human (using visual color lights) and information using pop up windows into the robot teach pendant.

Figure 2 shows the workspace, which is divided into two areas. Area 1 is where the operator executes the Tower of Hanoi game with five disks (TOH5). It is considered Task 1, the main task for the operator. Area 2 is where the operator executes the Collaborative Assembly (CA) with the robot of the product. It is named Task 2, the secondary task. In Area 1, the human is in the loop. The information processing system allows visual feedback of the task developed to the participant. The participant can plan strategies and make decisions. The participant executes physical actions on a touch screen.

If the participant pays attention to Area 2, in this case, the human is in the loop. The visual color lights alert the participant if a human collaborative task is required. The participant can decide whether to leave Area 1 momentarily to go to Area 2. In this case, the human is in the loop attending the assembly station.

Inside Area 1, it is possible to design cognitive tasks with regard to human capabilities. The Tower of Hanoi with 5 disks (TOH5) task is an example of problem solving (high-level reasoning capabilities) where people use mental skills and learning abilities to achieve a successful solution [22,23]. The collaborative assembly task in Area 2 is a good example of the pick and place task in an industrial environment. It shows the human eye–hand coordination to achieve a successful solution. Hence, this work illustrates the idea that a human–robot collaborative task must include physical and cognitive aspects together.

2.1.1. Task 1. Tower of Hanoi (TOH5)

The objective of the task is to move the tower of disks from the first support to another support, see Figure 3, with the help of an intermediate support. The tower is segmented into disks, which are the ones that are actually moved to rebuild it again in the final position. This task must be performed in as few movements as possible and with as few errors as possible. Disk movements are conditioned by two constraints:

• It is not allowed to place a larger disk on top of a smaller one.
• You can only move the disks in the order in which they are placed in the tower, starting with the one at the top first.

In this study, the Tower of Hanoi was only performed by the human, using a digital version of this problem, available at Google Play HANOI 3D. This software allows the manipulation of the disks and records the number of moves and total time in seconds required to complete the task. It is proved that there is no experimental cognitive variation if either wood pieces or a digital version are used in experimental sessions [23]. Following the workload level evaluation scale in [24], this task is classified as appreciated mental workload because a human without previous expertise in this game must develop and plan effective strategies. In fact, as a first contact with the Tower of Hanoi problem, the case with 3 disks is a challenge for many users. The difficulty of this task was previously tuned according to several early participants and a medium level of difficulty with 5 disks was decided.

For the solution of the TOH task, the minimum number of moves is given by the formula $2^{n-1},$ where n is the number of disks on tower number 1 at the beginning of the test [25]. Hence, the TOH5 task can be optimally solved with 31 moves.

2.1.2. Task 2. Collaborative Assembly (CA)

The assembly area (Area 2) is where robotic work takes place in the process. Humans and robots should cooperate in order to simplify the job and make the overall system more efficient and productive [26]. The objective of this task is to collaborate in the assembly of a product composed of 3 components: a base, a bearing and a cap, as shown in Figure 4. The task is a priori classified as a low mental workload from a human-centered perspective because eye–hand coordination skills are most relevant in this task. The collaborative assembly task shows a low level of risk for the human, and no action is required to decrease this risk.

Figure 4 shows the implementation of the assembly process using a collaborative robot from the company Universal Robots, model UR3. The difficulty of the programmed robot task was previously tuned experimenting with early participants, and a medium level of difficulty for robot velocity was decided. The purpose is to allow the participant to approach the station and be able to intervene physically without compromising safety and performance.

2.1.3. Cycle of Work

In the workspace, the distribution of activities between the human operator and the collaborative robot (cobot) is shown in

Table 1. Tasks and activities allocation in the human–robot collaborative workspace’s experiment.

Task	Activity	Operator
Task 1	Solve problem TOH5	Human
Task 2	Get base	Cobot
Task 2	Get bearing	Cobot
Task 2	Get cap	Human
Task 2	Reload storage	Human
Task 2	Assembly product	Cobot
Task 2	Palletize product	Cobot

. Figure 5 shows an instance of the cycle of work for a participant.

2.2. Safety Conditions

In the Human–Robot Collaborative Workplace, safe conditions are considered, different from traditional industrial robots. These specific safety requirements are:

• Universal Robot’s UR3 cobot has been designed to work in direct cooperation with humans within a defined workspace according to the ISO 10218-1: 2011 standard [27].
• Maximum speeds and torque have been reduced, as can be seen in

  Table 2. Motor operating conditions for the UR3 cobot.

  Variable	Normal Mode	Maximum
  speed	1500 mm/s	5000 mm/s
  torque	150 N	250 N

  .
• Virtual safety planes are defined for the cobot to reduce the workspace.
• Visual indicators alert the operator of the robot’s operating condition.

In this approach, the authors are not considering strategies and possibilities of collision between the robot and human operator because the human operator and robot do not physically coincide in the same spatial location in the case study [28]. In fact, the robot can wait until the end of the human operator activity before starting its work [29].

2.3. Case Study

Experiments took place at the Teaching Laboratory of the Automatic Control Department of the Universitat Politècnica de Catalunya, Barcelona Tech (North Campus, FIB Faculty, Barcelona, Spain).

Following the hypothesis explained at the beginning of this paper and the tasks detailed in Table 1, two working scenarios are designed:

• Working Scenario 1 (TOH5): The participant only executes Task 1, without distractions, using a tablet for this end, as seen in Figure 6.
• Working Scenario 2 (TOH5 + CA): The participant executes a combination of the two tasks. The main task is TOH5 and also collaborates with the cobot in activities of Task 2, according to the work cycle in Table 1 (see Figure 7).

2.3.1. Participants

Data in both scenarios were collected from 18 participants. Among the participants there were undergraduate students, vocational students, and teaching staff. Details about them are shown in

Table 3. Participants.

Type	Number	Gender	Age (Years)	Percent
Undergraduate students	10	male	22–26	55%
	2	female	22–24	11%
Vocational students	2	male	18–20	11%
Teaching staff	4	male	40–50	23%
Total	18	-	-	100%

. Nobody had previous experience working with cobots.

2.3.2. Procedure

The participant’s study is composed of two main steps: demo and test.

• Demo. As a first step, a facilitator provides each participant with a brief demonstration, usually about 2–3 min long, about the main functionalities of the experience.
• Test. For each test, the participant works on the two defined working scenarios.

During the test, both scenarios are considered by the participant: firstly, Scenario 1 for 6 min, then Scenario 2 for 15 min, fulfilling the objectives described in

Table 4. Experimental tasks and activities.

Scenarios	Task	Activities	Goals	Time
Scenario 1	TOH5	The participant solves the TOH5 problem	Solve the problem with 31 moves	6 min
Scenario 2	TOH5 + CA	The participant solves the TOH5 problem and responds to requests for collaboration from the robot	Solve the problem with 31 moves and complete 9 cycles of work with cobot	15 min

.

A facilitator supervised the test and took notes about the time expended by the participants during the performance of each task. When tasks were completed, each participant was invited to fill a NASA TLX index questionnaire form about the proposed experience.

2.3.3. Measure

There are several mental workload physiological metrics to be taken into consideration, for instance, heart rate, pupil dilation, respiration rate, skin temperature and fundamental frequency [30,31]. In the participants’ study presented in this paper, the authors used a subjective mental workload measure through the adoption of the NASA TLX index standard questionnaire. It allows deriving the overall workload score based on a weighted average of ratings on six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration level. This questionnaire is an effective tool for the study of the mental workload.

The questionnaire form was derived from the app NASA TLX index v1.8.2 (see Figure 8 and Figure 9). As shown in Figure 8, the participant must fill in the two questionnaires, one for the working scenario 1 (TOH5) and the other for scenario 2 (TOH5 + CA).

A description for each of these subscales was provided to help participants answer accurately. They are rated for each task within a 100-point range with 5-point steps. These ratings are then combined with the task workload index. These scales and descriptions are shown in

Table 5. Rating scale definitions for the NASA TLX index standard questionnaire.

Subscale	Endpoints	Descriptions
Mental Demand	Low/High	How much mental and perceptual activity was required? Was the task easy or demanding, simple or complex?
Physical Demand	Low/High	How much physical activity was required? Was the task easy or demanding, slack or strenuous?
Temporal Demand	Low/High	How much time pressure did you feel due to the pace at which the tasks or task elements occurred? Was the pace slow or rapid?
Overall Performance	Low/High	How successful were you in performing the task? How satisfied were you with your performance?
Effort	Low/High	How hard did you have to work (mentally and physically) to accomplish your level of performance?
Frustration Level	Low/High	How irritated, stressed, and annoyed versus content, relaxed, and complacent did you feel during the task?

.

3. Experimental Results

Descriptive statistics, t-test, and analysis of variance tests were used to analyze the effects of the experience in time. The statistical significance level was set at $p<0.05$, and in all the cases, the confidence level is 95%. For a better understanding,

Table 6. The interpretation score for the NASA TLX index.

Mental Workload Range	Value	np TOH5	np TOH5 + CA
Low	0–9	0	0
Medium	10–29	0	1
Somewhat High	30–49	5	2
High	50–79	12	13
Very High	80–100	1	2

shows the used markers with adjectives that qualify the mental workload (MWL) [32], where np TOH5 and np TOH5 + CA are the number of participants versus the observed qualitative mental workload.

3.1. Mental Workload Results

The data collected from both scenarios were gathered and are presented in the histogram in Figure 10.

The distribution of the data obtained is not symmetric and they tend to accumulate in the areas with high mental workload. Six out of twelve participants perceived that the mental workload of the task TOH5 was high. Eight of twelve participants perceived that the mental workload of the main and secondary tasks TOH5 + CA was high.

Table 7. Results of NASA TLX Index Score.

Statistics	TOH5	TOH5 + CA
Mean value	59.11	60.17
sd	12.20	17.41
Non-Blank	18	18
Interpretation Score	High	High

shows the obtained score for each scenario in terms of mean value and standard deviation. The mean values of the NASA TLX index score for both scenarios are very close, 59.11% and 60.17%, respectively. These scores are at the end of the first third of the range (High).

The hypothesis in this study is that when the operator performs the collaborative task with the robot (TOH5 + CA, second scenario), the mental workload value is not very far from the mental workload of the main task (TOH5, first scenario). That is, a very little increment $\delta$ is expected,

$${\mathrm{MWL}}_{\mathrm{TOH}5+\mathrm{CA}}\sim {\mathrm{MWL}}_{\mathrm{TOH}5}+\delta$$

(1)

Moreover, both values would be in an intermediate comfort zone of mental workload; hence, they are not saturating the mental workload.

To test this hypothesis, a paired t-test was performed on the results obtained. The results in

Table 8. Paired t-test.

Confidence Level	95
Null Hypothesis	Difference > 10
Descriptive Statistics	-	Average Difference	1.06
Mean Difference value	−1.1	Confidence Interval Low	−9.72
Median Difference value	−8.0	Confidence Interval High	7.61
sd	17.43	Margin Error	8.7
n (sample size)	18	p-value	0.02

show that there exists a p-value < 0.02 associated with the probability that the difference is greater than 10, so the null hypothesis is rejected, $\delta <10$, and the alternative hypothesis is corroborated: certainly, there is only a small increase in MWL when the operator is moving from only one task to a dual task with a collaborative robot.

The interpretation scores shown in Table 7 for the associated mental workload are in the low range of the label “High” (59.11 and 60.17) for both scenarios using the scale in Table 6. According to the cumulative frequency distributions of TLX Global Workload Scores by Task by Grier [33] (see

Table 9. Cumulative frequency distributions of mental workload scores by task.

	Min	25%	50%	75%	Max
Cognitive Tasks	13.08	38.00	46.00	54.66	64.90

), a common experimental mental workload score is related to cognitive tasks with a value of 64.90 as the maximum and 54.66 as within 75% of the results obtained in experiments with the TLX-Index. Hence, the results for scenarios TOH5 and TOH5 + AC are in this range, indicating that the proposed tasks are actually high cognitive tasks, but they are not saturating the regular experiments on cognitive tasks.

Figure 11 shows the distribution of the mean values of MWL for the different subscales. The mental demand is the subscale with the highest values for both scenarios, which is in accordance with the working conditions established for this study.

The results obtained for the subscales studied using the NASA TLX index are shown in

Table 10. Paired samples test for subscales of the NASA TLX index, TOH5 and TOH5 + CA.

Subscale	Mean	Standard Dev.	p-Value
Mental	3.39	10.01	0.172
Physical	−2.11	2.66	0.004
Temporal	−1.15	10.03	0.633
Performance	−3.72	7.00	0.038
Effort	−1.07	8.39	0.594
Frustration	3.61	7.26	0.050

. The physical subscale shows a p-value < 0.05, i.e., it is the only one showing significant statistical differences between both scenarios, while the other subscales do not. This is consistent with our hypothesis.

3.2. Task Performance

As a complement to the study of mental load, details on task performance are presented below.

3.2.1. Effectiveness

This subsection shows the results for the effectiveness of the tasks developed in the Human–Robot Collaborative Workplace. The variable Task Completion Rate (TCR) is considered,

$$\mathrm{TCR}=\frac{{\mathrm{N}}_{\mathrm{sucsess}}}{{\mathrm{N}}_{\mathrm{replay}}}\%$$

(2)

Table 11. TCR results for TOH5 in both scenarios.

	Scenario 1 (TOH5)	Scenario 2 (TOH5 + A)
N${}_{\mathrm{sucsess}}$	8	12
N${}_{\mathrm{replay}}$	18	18
TCR	44.44%	66.67%
Difference	22.22%

shows the results for TCR when task TOH5 is solved in both scenarios. For Scenario 1, a low effectiveness is obtained (TCR = 44.44%), while for Scenario 2, there is a 22% of percent increase (TCR = 66.67%) as a result of the experience obtained from Scenario 1.

Table 12. TCR results for TOH5 and CA in Scenario 2.

	TOH5	CA
N${}_{\mathrm{sucsess}}$	12	17
N${}_{\mathrm{replay}}$	18	18
TCR	66.67%	94.44%
Difference	27.77%

shows the results for the effectiveness in the resolution of the TOH5 problem as well as in the collaboration in the CA assembly. The results show that the collaboration is carried out effectively, TCR = 94.44%.

3.2.2. Fluency

A measure able to evaluate the team’s work is fluency. According to [34], fluency evaluation allows determining the performance as a team that the human–robot pair has. These metrics are percentage of concurrent activity (C-ACT), human’s idle time (H-IDLE), robot’s functional delay (F-DEL), and robot’s idle time (R-IDLE). To calculate fluency, parameters shown in

Table 13. Time parameters from robot.

Parameter	Description
Idle_Time	Sum Robot wait time
Cycle_Time	Sum Robot work time
Shared_Time	Sum Shared Time to resolve task

were measured from the robot side. Values obtained are summarized in

Table 14. Mean values of times from robot.

Statistics	Cycle_Time	Idle_Time	Shared_Time
Mean value	626	117	34
sd	176	38	8
n	18	-	-

in the form of mean values.

The fluency values are calculated as follows:

$$\begin{array}{ccc}\hfill \mathrm{Time}\mathrm{to}\mathrm{Task}& =& \mathrm{Cycle}\_\mathrm{Time}+\mathrm{Idle}\_\mathrm{Time}\hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill {\displaystyle \text{H-IDLE}}& =& \frac{\mathrm{Cycle}\_\mathrm{Time}}{\mathrm{Time}\mathrm{to}\mathrm{Task}}\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill {\displaystyle \text{R-IDLE}}& =& \frac{\mathrm{Idle}\_\mathrm{Time}}{\mathrm{Time}\mathrm{to}\mathrm{Task}}\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill {\displaystyle \text{C-ACT}}& =& \frac{\mathrm{Shared}\_\mathrm{Time}}{\mathrm{Time}\mathrm{to}\mathrm{Task}}\hfill \end{array}$$

(6)

Figure 12 allows to visualize the results from the calculation. It can be seen that the value for H-IDLE is much higher than the one for R-IDLE. Moreover, C-ACT gets a low value.

4. Discussion

The use of the NASA TLX-index protocol has allowed us to perform a multidimensional assessment of the variation of the mental workload on the participant, in this case in the role of human operator/collaborative robot As established in the hypothesis of the study, it has been shown that there are no significant differences between the mental workload in the solution of the TOH5 problem and its solution when a collaborative task with a robot is added in the same workspace. Moreover, as the results show, the level of mental workload found in both scenarios is high without overloading the operator. In the results, it can be observed that no participant has indicated a level of underload in the mental workload, and according to the experience found with other experiments, the level of mental workload can be considered normal in cognitive tasks, such as the TOH5 of this study. It could be that a high level of mental workload is necessary to be alert and make decisions in time.

The NASA TLX-index considers that the first three subscales (mental, physical and temporal) correspond to the demands imposed on the operator. In this aspect, the results show that the mental demand is the largest subscale of the mental workload. However, as the results show, no significant differences were found between the scenarios. The physical subscale is the least contributing subscale; however, the results show that there are significant differences in both scenarios, which is consistent with the differences in the physical characteristics of the scenarios. The next three subscales (effort, frustration and performance) refer to the interaction of the participant with the workspace. In this aspect, the results show that there are significant differences in performance between the scenarios, which is a result of the demand of attention that the cobot imposes on the participant. The frustration subscale, according to the results, does not show significant differences; however, it is an important subscale in the resolution of the problem when the operator begins his experience in the experiment. The use of descriptive adjectives and multidimensional graphs allows stakeholders to have a representative perspective of the results obtained.

In relation to the performance of the tasks, the results show that human operators can solve the task TOH5 effectively (TCR = 66.67%). Additionally, the human–robot team working in the collaborative assembly task is also able of effectively solving the task (TCR = 94.44%).

Following the results of the objective metrics and based on [34], some considerations can be presented. The measure H-IDLE relates the subjective perception of the human operator while waiting for the robot, e.g., wasting time or being bored. In this study, H-IDLE value remains at the same level, as shown in Figure 12, when the human operator develops Task 1 while waiting for the robot’s request for collaboration; hence, it could be said that the human operator is taking advantage of his/her time.

Moreover, Hoffman relates the R-IDLE measure with fluency, establishing two possible conditions; either the robot is physically inactive but is doing internal work on its processor, or it is inactive while waiting for an intervention from the human operator. In our use case, the second condition is present, the robot starts its idle time by sending a message to the operators waiting for their collaboration and remains in this state until an operator informs it about the end of the collaboration. This collaboration time of the human operator is variable. In the subjective sense, this could be seen as either an inefficient use of the robot or an imbalance in the distribution of the task. In this study, the R-IDLE value is much lower than the H-IDLE value (see Figure 12), indicating that there is an efficient use of the robot. However, it could also indicate an imbalance in the distribution of the task.

Regarding C-ACT, Hoffman describes that a high value for this measure could indicate a subjective feeling of fluency when considering that the teams are well synchronized and that there is a similarity in the team members, perhaps a fair balance of work. In this study, the value for C-ACT is low from a physical point of view (see Figure 12). It could be considered an imbalance in the work. However, when considering a complete vision of the work capacities for each member, the distribution is fair in the sense that each member does what it does the best, i.e., the human operator undertakes more of the cognitive activity and the robot operator more of the physical activity.

For the measure F-DEL, it is established that a low level is related to the subjective perception of human–robot fluency since it indicates an efficient use of the time of the team members. In this study, it can be seen that such F-DEL is not present (see Figure 12). The human operator’s sensation would be of efficient work; however, it should be taken into account that for this task, the level of collaboration is also low.

5. Conclusions

The cognitive interaction approach allows studying the cognitive load associated with assembly tasks in manufacturing systems. In a first approach, this paper show scenarios in a laboratory environment, taking into account task difficulty, participant skills, and observed performance, in the analysis of mental workload.

For this, the context of collaboration between person and robot has been considered appropriate. One of the most representative tasks in collaborative robotics is the assembly task, and one of the most representative tasks of the human is the problem solving. With these ideas in mind, in this paper, these related tasks are embedded in laboratory scenarios.

The participants in the experimental session show that it is possible to keep the mental workload under control while they are developing primary and secondary tasks. This feature could consider flexibility as an important parameter in the operator’s condition.

In industry 4.0 real scenarios, the operator may be subject to task changes, task difficulty, task shared with a robot, and interruptions (for instance, noise) [35,36]. In these cases, understanding the cognitive load types (intrinsic, extraneous, germane) and the relationship with mental workload could be useful for the improvement of the human information processing system, human performance and the effectiveness of the overall system [37,38].

Understanding this human cognitive load will allow the design of intelligent assistant systems to help and support human operators in future cognitive manufacturing systems. Moreover, the vision of the performance of the human–robot team through fluency could be considered an additional component to improve the design of a help support system from the perspective of the human–robot team.

In future works, the role assigned to the human in this work can be expanded considering that there are two robots available in the laboratory. Each robot could perform a task, and the participant would be asked to attend to one robot or the other based on the quality and performance criteria. In this case, the analysis would focus on human decision making. The current human operator requires that her/his work be analyzed not only from traditional perspectives, such as physical, sensory and cognitive characteristics, but as it is proposed in [39], it is necessary to consider the interaction skills in the human–robot team. Fluency is oriented in this dimension.