1. Introduction
Flow charts are visual representations delineating algorithms, systems, or processes [
1,
2,
3]. They are widely used in different domains (e.g., healthcare, computer science, and business) for the following purposes: documentation, instant communication, effective analyses, and problem solving [
2,
4,
5]. Process models constitute a derivation of flow charts for the graphical documentation of processes [
6,
7]. A process model represents all activities to achieve a specific objective. For example, an order to cash (O2C) process describes all activities for receiving and processing customer orders for goods and services [
8]. An activity consumes resources (e.g., machines) to convert inputs (e.g., data) into outputs (e.g., value). In more detail, a process model depicts all activities, decisions, and involved stakeholders as well as resources in a process [
9]. In order to make use of the merits of process models, the understanding of such models (i.e., process model comprehension) should not pose any difficulties for the involved stakeholders [
10]. Many unresolved issues concerning the factors thwarting the comprehension of process models exist, and therefore the identification of these factors is decisive. For this reason, a prerequisite for an overall comprehension of processes is to ensure that all stakeholders can easily read and comprehend corresponding process models in an efficient and effective manner [
11].
Despite existing research in the field of process model comprehension, stakeholders, both inexperienced and experienced, are still facing challenges on how to properly read and comprehend process models [
12]. Numerous works in the literature exist with respect to process model comprehension. For example, [
13] presents results from a series of experiments about factors having an impact on process model comprehension. An extensive literature overview about empirical work focusing on process model comprehension is consolidated in [
14], in which objective and subjective factors influencing process model comprehension are discussed.
In this context, to foster process model comprehension, the usage of eye tracking has proven to be a suitable methodology that may yield promising insights [
15]. Process model comprehension strategies can be visualized (e.g., depiction of the path followed by the eyes when reading a model) and difficult to comprehend modeling constructs (e.g., loops) can be identified in a process model [
16]. The latter is identified by analyzing modeling constructs to which the eyes repeatedly jump back frequently, or to which the eyes have a longer average dwell time. Different types of eye movements (i.e., fixations and saccades) may serve as an indicator representing emerging cognitive load during process model comprehension (e.g., more fixations during high load) [
17]. In addition, the authors of [
18] discussed experiences and lessons learned from eye tracking studies on process model comprehension. The approach discussed in [
19] explains how model-related (e.g., size of a process model) and person-related (i.e., process modeling experience) factors of process model comprehension are influenced by visual cognition variables (e.g., scan path). The results shown in [
20] indicate that visual cues (e.g., colors) used in process models improve the overall comprehension. Finally, the authors of [
21] investigated the impact of coloring in decision models.
If factors that hamper proper process model comprehension are not properly addressed, the respective processes might not deliver the required outcomes. Failures that happen in the application of such models have been commonly linked to model incomprehension [
22]. As a consequence, the identification of factors, both positive and negative, that influence the comprehension of such models is essential. For the continuation of ongoing research on process model comprehension, this paper presents the results obtained from a second study of the authors (i.e., Study Two), which is part of a three-stage study to foster the comprehension of process models (see
Section 5). In the first study (i.e., Study One), we analyzed eye movements and visualized comprehension strategies of novices and experts while reading and comprehending process models. The results revealed that there were similarities and differences between experts and novices in process model comprehension. For example, all participants tried to find the starting point in the process model when they first gazed at the model [
18]. It was particularly noticeable that experts did not look at all the individual elements in the shown process models juxtaposed with novices. The results further showed that experts comprehended process models more efficiently than novices [
23]. Based on these findings, the question emerged whether visual observation capabilities of experts during the comprehension of process models in Study One can be efficiently conveyed to novices. Based on the eye tracking data obtained from the experts in Study One, we created Eye Movement Modeling Examples (EMMEs) for a second study (i.e., topic of this paper), which shall assist novices in the comprehension of process models [
23]. EMMEs are instruments to teach and improve performance on perceptual tasks [
24]. The basic idea behind an EMME is to convey mandatory visual observation capabilities for perceptual tasks, especially to novices [
25]. Therefore, eye movements of experts were recorded during a perceptual task and, afterwards, their eye movements (e.g., fixations) were displayed during novices’ performance of a task [
26]. EMMEs are usually superimposed in a dynamic multimedia form (e.g., video) [
27]. However, it is also possible to superimpose the eye movements (e.g., fixations) of experts in static pictures. These eye movements could, on the one hand, appear for a short period of time in a picture to attract the gaze of the viewer or, on the other hand, be displayed permanently to guide the viewers’ gaze. For example,
Figure 1 presents an EMME, which depicts an image of cats with three superimposed dots (i.e., dot display condition).
Crucial parts in the image are highlighted with green dots to visually attract the attention of the viewer. Research showed that this kind of teaching method with EMMEs can improve performance on perceptual tasks and fosters learning [
28]. The research presented in [
29] reveals a positive effect of EMMEs on comprehension strategies in medical image diagnosis. The authors in [
30] are using EMMEs for the improvement of information processing and learning from pictures and texts. [
27] provides evidence that EMMEs change the information processing during learning and fosters the performance in learning especially for learners with lower skills. The work presented in [
31] evaluated the application of EMMEs during computer programming, resulting in an improved solving of programming problems. Finally, in [
32], the authors demonstrated that EMMEs can be effectively used in order to guide reading and comprehension strategies. In [
33], the same authors presented how EMMEs raise attention during the critical reading of web pages.
To conclude, the following three research questions (RQ) are addressed in study presented in this paper (i.e., Study Two of the three-stage study described earlier):
RQ 1: Do novices perform better in process model comprehension when the novices are supported by EMMEs, and does this depend on the complexity of the process model?
RQ 2: Do novices perform differently as experts in process model comprehension when the novices are supported by EMMEs, and does this depend on the complexity of the process model?
RQ 3: Do novices perform differently in process model comprehension when they are supported by different conditions of EMMEs, and does this depend on the complexity of the process model?
In RQ 1, the results obtained from novices of Study One were juxtaposed with the results of novices from the current study [
23]. We wanted to investigate with RQ 1 whether the application of EMMEs is beneficial to foster the comprehension of process models.
In RQ 2, the results from novices of the current study were compared with results from experts of Study One [
23]. Therefore, RQ 2 is concerned with the question whether novices supported by EMMEs performed differently (e.g., similar) in process model comprehension juxtaposed with experts.
Finally, inRQ 3, the results of novices being confronted with different conditions of EMMEs were compared to each other. An EMME can reflect different conditions (see
Figure 1), depending on the focus being set. We wanted to reveal with RQ 3 whether different conditions of EMMEs may pose varying effects on process model comprehension.
In all three RQs, we took the level of complexity of the process models into account. Therefore, the participants worked with easy, medium, and hard process models.
To the best of our knowledge, there exist no other works dealing with the application of EMMEs in the context of process model comprehension so far.
The structure of this paper is as follows:
Section 2 provides information about materials and methods of the conducted study. In
Section 3, obtained results of the study are presented descriptively, including significance tests. The analyzed results are discussed in
Section 4, including implications for practice and research as well as limitations. Finally,
Section 5 summarizes the paper and discusses future work.
4. Discussion
In RQ 1, we evaluated whether novices supported by EMMEs show better performance measures than novices not supported by EMMEs and whether this depends on the level of complexity of the process models. We found that novices supported by EMMEs had significantly better performance measures than novices not supported by EMMEs (i.e.,
significant ME 2). For most performance measures, this did not depend on the complexity of the process models (i.e.,
non-significant IE), except for the performance measures average fixation duration and score. Follow-up analyses showed that the interaction effect for both performance measures (i.e., average fixation duration and score) can be interpreted as follows: A more significant decrease in the performance measures is observable in novices not supported by EMMEs than in novices supported by EMMEs between the process models showing a medium and hard level of complexity. This indicates that EMMEs may prevent a decrease in these performance measures when it comes to hard process models. This might be explained that the support of EMMEs in process model comprehension leads to a reduction of the mental load, which is beneficial especially for complex process models with a high number of modeling elements and structures (e.g., loops) [
42]. The shorter fixation durations in the group of novices supported by EMMEs were an additional indication that EMMEs support a reduction in the mental load [
43]. This might be due to the fact that the application of EMMEs mainly target attention during process model comprehension to only relevant information. The working memory does not have to process and interpret irrelevant information allowing for a more efficient comprehension of process models. The attention through the eye movement is targeted by the EMMEs on relevant information, which also reduces processioning time of perceived information allowing for a more effective comprehension of process models. Based on the obtained results, the application of EMMEs in the context of process model comprehension fosters the comprehension of respective models significantly. The placed visual cues in the dot display condition and the visual guidance in the path display condition contribute positively to the comprehension of such models. Similar work conducted by the authors in [
20] confirm the observation that visual cues are beneficial in process model comprehension. For future work, it would be therefore interesting to evaluate the association between the application of EMMEs and the effects on the working memory. Obtained eye tracking parameters (e.g., fixations) might serve as an indicator for different mental load during process model comprehension [
44]. A follow-up study should evaluate whether the application EMMEs (i.e., visual guidance) also show a long-term effect. Do novices benefit in process model comprehension when the visual guidance (i.e., EMMEs) is removed from the process models?
In RQ 2, it was investigated whether novices supported by EMMEs show different performance measures as experts. Except for the performance measure score, novices supported by EMMEs and experts did not differ from each other (i.e.,
non-significant ME 2). With regard to scores, novices supported by EMMEs were worse than experts. A reason therefore might be that for the comprehension questions the process models must be memorized. As a consequence, there is an increasing risk that the answers were guessed due to inaccurate memorization [
45]. Another explanation in this context is that experts are more effective in processing the presented information during process model comprehension due to their prior modeling experience. Research showed that novices and experts differ regarding problem solving and decision making [
46]. It was shown that the experts’ knowledge is better structured in the individual and collective memory through deliberate practice, resulting in a more efficient access of respective knowledge in this context. The level of complexity of the process models did not influence this result as the IE did not reach significance. Further, in Study One, the following three performance measures between novices, which were not supported by EMMEs (i.e., Sample Novice), and experts (i.e., Sample Expert), showed significance: fixation (
p = 0.008), score (
p = 0.001), and duration (
p = 0.013). The results indicated that experts performed better in the comprehension of process models [
23]. However, in the current study, the respective performance measures showed no significance between novices, which were supported by EMMEs (i.e., Sample Both), and experts (i.e., Sample Expert). As a result, an EMME might enable a novice to achieve similar performance as experts in the comprehension of process models. Since performance measures regarding eye movements showed no significant differences, it could be that the novices’ attention was only focused on the visual cues (i.e., dots or path) in the process models. Potential relevant information from the process model was not considered, which would have been advantageous for novices regarding a proper process model comprehension (e.g., an activity before a decision), and were therefore recommended in order to answer a comprehension question. As a result and as shown in studies from other domains (e.g., [
27,
29], we want to emphasize the positive effects of the application of different types of EMMEs. However, there are other explanations as well. For example, the sample size of the novices in Study One (Sample Novice: N = 17) and in the current study (Sample Both: N = 43) were not the same and the sample size affects the probability to detect significant differences. Yet, as the sample size was larger in the current study, the probability would have been higher to detect differences with this larger sample compared to the smaller sample of novices recruited for Study One (see
Section 4.2. Finally, it can also be evaluated in a future study whether experts in general may also benefit from the application of EMMEs in process models.
In RQ 3, we analyzed whether dot or path display conditions result in better performance measures. There were no significant differences between the two conditions (i.e.,
non-significant ME 2) and this result did not depend on the level of complexity of the process models (i.e.,
non-significant IE). The dot display condition mainly refers to process model syntactics (i.e., compliance with process modeling rules) with the provision of only visual cues in a process model, whereas the path display condition refers to process model semantics by denoting a given scan path in order to affect the reading direction. This is to ensure that all relevant information is considered during process model comprehension. However, the results confirm that both EMME conditions ensure that the syntactic as well as semantic dimension of a process model are properly captured and correctly comprehended by novices. A reason might be that both conditions raise awareness regarding the syntactic and semantic dimension in a process model. By directing the gaze of participants only to relevant information, more capacity remains free in the working memory that can be used to correctly interpret model semantics as well as syntactics. Finally, for illustration purposes,
Figure 5 present recorded eye movements (i.e., scan path) of two participants, i.e., dot (see
Figure 5a) and path display condition (see
Figure 5b). Notably, similar and different eye movements can be distinguished in both conditions. In this context, the consideration of other EMME conditions (e.g., spotlight display condition, in which relevant parts in a stimulus are brighter and more visible while the other parts are darkened) may be subject of future work.
Generally, in all three RQs, the ME 1 attained significance in all performance measures except one (see Score in RQ 3) indicating that process models were more difficult to comprehend when they were more complex. The selected performance measures (see
Section 2.4) in this study may be used as appropriate indicators evaluating, for example, confronted mental load during the comprehension of process models with varying complexity. Similar research also demonstrated (e.g., [
14,
47] that with rising level of complexity process model comprehension becomes more difficult. Rationales are, for example, the increasing number of modeling elements that needed to be comprehended or the more ramified structure (e.g., varying process flow direction) in larger process models.
4.1. Implications
The provided insights have implications for practice by demonstrating the applicability of EMMEs as well as for research on process model comprehension.
For practice: Since process models in real world are usually not provided with any visual guidance (i.e., EMME), the results have predominantly an impact on the formal training in the comprehension of such models. Process models may be enriched with visual cues guiding practitioners appropriately throughout a process model in order to ensure a proper comprehension. Depending on the focus set, the visual cues can be provided in such way that process model semantics (i.e., path display condition) or syntactics (i.e., dot display condition) are correctly captured and comprehended. Formal training in process model comprehension can benefit from the advantages of EMMEs to offer a more effective training [
29]. Novices (e.g., doctors) can benefit from the application of EMMEs in order to develop a better understanding of working with process models [
24]. Moreover, by capturing the attention with visual cues, crucial parts in a process model can be emphasized to highlight their importance. In this way, relevant information (i.e., about the who, where, and when) in a process can be extracted more efficiently without drawbacks regarding the mental load. Since EMMEs lead to a reduced load on the working memory (i.e., shorter average fixation duration) [
48], practitioners can use the capacity in their working memory freed up by EMMEs for other tasks (e.g., more effective learning) [
49]. Process modeling tools can be developed more specifically or can be extended with additional features to attract and guide the attention of practitioners on important elements or modeling constructs in a process model. The different EMME conditions (e.g., path or dot display condition) may be displayed permanently and according to the needs in order to ensure an optimized assistance during process model comprehension. In addition, the level of complexity of process models should be reduced in order to ensure a proper comprehension of such models. Approaches like changes in the visual representation (i.e., syntax modification) of a process model or the modularization of specific parts in a process model should be taken into account [
50,
51]. Finally, although the use of EMMEs was investigated with BPMN process models, they can be applied to other notations for process modeling (e.g., Event-driven Process Chains (EPCs)) as well.
For research: The insights from this study confirm the results from relevant other works on how to improve process model comprehension with visual cues [
20], or by emphasizing the use of colors in the secondary notation [
52]. Another question for research based on the obtained results is: does another type of EMME condition (e.g., spotlight display condition [
28]) may have a different effect on process model comprehension? Further, the human brain operates in such way that highlighted visual information is perceived more dominantly compared to non-highlighted information [
53]. It can also lead to circumstances where the non-highlighted information is not perceived at all, although it may be of importance (e.g., availability heuristic) [
54]. Towards cognitive load, it would be interesting to investigate the differences in the cognitive load during process model comprehension, when confronting participants with the application of EMMEs and without the application of any EMMEs. This will allow for potential effects (e.g., availability heuristic) and their implications on process model comprehension to be investigated in more detail. The research question whether domain experts (e.g., juxtaposing doctors and economists) perceive EMMEs and respective condition differently could unravel new insights. For example, a doctor might better cope with the complexity of a process model when using the path display condition. The same applies for demographic characteristics such as age and gender. The question arises whether experts in process modeling may work more effectively by using EMMEs. Moreover, further research using interactive EMMEs may provide a new kind of guidance for practitioners in the comprehension of process models [
55]. For example, colored dots in a process model that disappear once they have been viewed for a defined time and which appear in different positions to draw the attention on further important modeling structures (e.g., decisions in a process) in the model. Finally, another focus should be put on the inherent level of complexity of process models. With increasing process model complexity, more semantic and syntactic information needs to be extracted from a process model and stored in the working memory. As capacities in the working memory are limited, not all information can be stored and may be therefore not reflected properly [
56]. This issue should be addressed on how to ensure a proper comprehension of this kind of information from a process model. Finally, this study showed that the application of EMMEs results in a shorter average fixation duration. According the work presented in [
39], the fixation duration correlates with the cognitive load and longer fixation durations indicate an increased strain on the working memory. Additional research should aim to reduce the fixation duration in order to relieve the working memory during process model comprehension. On the other hand, a reduction of the number of fixations can be investigated as it is an indication for a high confronted cognitive complexity [
57]. The reduction in both, fixation and respective duration, should lead to more available capacity in the working memory, enabling a more effective process model comprehension.
4.2. Limiting Factors
There are several limiting factors in this study that needed to be discussed and addressed in future studies. First, the used process models might not be representative. Process models document often complex procedures from the real world. However, the process models used in the study are of rather simple nature. Large and complex process models pose different demands on mental workload compared to simple process models. Second, participants of the study constitute another limitation. We tried to have a balanced and heterogeneous sample size, but, most participants were recruited from the field of computer science. Participants from other fields (e.g., healthcare) may perceive the application of EMMEs differently. In this context, significant differences in the baseline variables were found (see
Table 1). The baseline variables related to gender and age reflect significant differences between participants. Consequently, obtained significant results in the application of EMMEs could also be the result of differences in these base variables (e.g., participants with an age < 25 years benefit more from the application of EMMEs juxtaposed with an age > 24 years). Third, the categorization of participants in the group of novices and experts with questions about prior modeling experience might be too vague and an additional expertise test might be better for a more precise categorization. Fourth, the documented scenarios in the process models constitute an additional risk. Familiar process scenarios might have a positive influence on the comprehension of process models in comparison with unfamiliar process scenarios. Fifth, the missing possibility to have a glance at the process model, while answering the comprehension questions, represents another limiting factor. The process models as well as documented scenarios must be kept in mind and the risk to guess an answer is growing due to an incomplete or either wrong memorization. Sixth, the sizes of the samples also limit the statistical power and there might be additional significant differences between the samples, which we could not show in this study, but which might become apparent in larger samples. Seventh, the results of the comparisons between the dot and path display conditions (RQ 3) have a higher internal validity than the results for RQ 1 and RQ 2, since the allocation of the participants to the two conditions for RQ 3 was done by the researcher (i.e., round-robin approach), whereas the control condition for RQ 1 (i.e., novices without EMMEs) as well as the control condition for RQ 2 (i.e., experts) were historical controls from a previous study. Differences in these samples might be confounders and might have influenced the results. Eighth, the comparison of data obtained between this study and a previous study (i.e., historical Study One) reflect another limitation. In more detail, although the procedure between the two studies has been kept the same, circumstances of data collection might have changed which could have lead to different results. Ninth, the factor process model complexity is not counter-balanced. The process model were presented in increasing level of complexity. Hence, lower performance measures in the more complex process models might be caused due to exhaustion of the participants. Tenth, the robustness of the provided comprehension benefits by the application of the EMMEs was not evaluated extensively. This means that the participants should have at least comprehended a process model without any visual guidance in order to evaluate the long-term effects of the EMMEs (i.e., may comprehension strategies be successfully transferred).