Diagnosing Faults in Different Technical Systems: How Requirements for Diagnosticians Can Be Revealed by Comparing Domain Characteristics

Abstract

In complex work domains, not all possible faults can be anticipated by designers or handled by automation. Humans therefore play an important role in fault diagnosis. To support their diagnostic reasoning, it is necessary to understand the requirements that diagnosticians face. While much research has dealt with identifying domain-general aspects of fault diagnosis, the present exploratory study examined domain-specific influences on the requirements for diagnosticians. Scenario-based interviews were conducted with nine experts from two domains: the car domain and the packaging machine domain. The interviews revealed several factors that influence the requirements for successful fault diagnosis. These factors were summarized in five categories, namely domain background, technical system, typical faults, diagnostic process, and requirements. Based on these factors, we developed the Domain Requirements Model to predict requirements for diagnosticians (e.g., the need for empirical knowledge) from domain characteristics (e.g., the degree to which changes in inputs are available as domain knowledge) or characteristics of the diagnostic process (e.g., the extent of support). The model is discussed considering the psychological literature on fault diagnosis, and first insights are provided that show how the model can be used to predict requirements of diagnostic reasoning beyond the two domains studied here.

1. Introduction

Faults occur in every technical system, albeit with varying frequency and severity. If system designers can anticipate potential faults or if appropriate data and algorithms are available, many faults can be diagnosed automatically. Whenever this is not possible, faults must be diagnosed by humans who can flexibly apply their previous experience and system knowledge even to novel faults. As domain characteristics influence cognition and behavior (e.g., [1,2,3]), strategies and requirements for successful fault diagnosis are thought to differ between domains. This is illustrated by the following two scenarios.

Scenario 1: Car. A car driver notices that the check engine light is turned on and that the acceleration is inhibited, so they decide to take the car to a repair shop. The diagnostician asks about any additional symptoms and the car’s history. They then read out the fault memory of the car, which indicates a malfunction of the lambda sensor and a lean air–fuel ratio (i.e., too much air entering the combustion process). The expert system also describes the measurements needed to check the lambda sensor. The diagnostician follows these instructions and concludes that the sensor functions correctly. As the expert system does not provide an alternative cause, the diagnostician uses their knowledge about system functioning to derive several hypotheses. To test these hypotheses, they conduct formalized electrotechnical measurements and compare the obtained values to the reference values in the expert system. They identify and exchange the broken component, and take the car for a test drive to ensure that the fault is resolved.

Scenario 2: Packaging machine. In a production hall, individual chocolate bars with marzipan filling are packaged, but the production personnel notice that some of the readily packaged bars are broken. The diagnostician first checks whether this symptom occurs in a particular pattern, but they cannot identify one. As they have previously experienced that broken bars were caused by one particular area of the machine, they thoroughly examine the components in this area visually. They also push one bar through the machine to perceive the strain on the bar, but find it to be acceptable. Therefore, this area is ruled out as a potential cause, and the diagnostician continues to check the rest of the machine. As neither the machine components themselves nor their interaction with the products seem to be responsible for the fault, the diagnostician checks the incoming products and notices some fine cracks. By talking to the production personnel, they find out that deviations occurred in previous process steps. Thus, the products are replaced with a correctly manufactured batch, and all machine components are adjusted to reduce the mechanical load during packaging as much as possible.

While fault diagnosis differs notably in these two scenarios, the differences can be explained by taking the characteristics of the respective domains into account. In the car scenario, the diagnostician must carry out different formalized measurements to obtain information about the system state, and they must know which measurements are appropriate. For interpreting the obtained values, however, they can rely on explicit reference values that are provided by an expert system. In the packaging machine scenario, the diagnostician largely relies on their senses to identify the fault. This is possible because the technical system, namely the packaging machine, can be observed directly. At the same time, explicit reference values are missing and, therefore, the diagnostician must deduce them themself. These scenarios show that by comparing work domains with different characteristics, differences in requirements can be examined. Therefore, domain characteristics such as the task, technical system, physical environment, or social and organizational structure [1] should be included in the study of human behavior [4]. The requirements of a specific task also interact with the knowledge and skills of the person carrying out the task, and hence, different people are able to deal with these requirements to differing extents [5]. Understanding the relations between domain characteristics, task requirements, and successful performance is needed to appropriately support humans working in technical domains.

Especially in the context of process control, many studies have identified the information requirements for dealing with technical systems (e.g., [6,7,8]). For fault diagnosis, previous research has often focused on identifying personal aspects of adequate behavior, such as predictors of diagnostic skill [9], difficulties during the diagnostic process [10,11], or different diagnostic strategies [12]. Only a few studies have dealt with domain-specific constraints for fault diagnosis (e.g., [13,14]), but without specifying cognitive requirements for fault diagnosis in complex work domains. Even though domain-specific cognitive requirements need to be known to adequately support people in their work, they are not sufficiently studied.

The present article examines how domain characteristics influence the diagnostic process and what requirements people have to meet in order to successfully diagnose faults in a specific domain. To this end, fault diagnosis is compared between two realistic work domains, namely the maintenance and repair of cars versus packaging machines. Although relevant characteristics of a domain include not only the technical system but also the work task and the environment [1], both domains will hereafter be referred to by naming the technical system, namely cars and packaging machines.

1.1. Models of the Diagnostic Process

As diagnostic reasoning is relevant in different domains (e.g., medicine, education, or technology), many models of the diagnostic process have been developed. What do these models tell us about differences in diagnostic reasoning between domains? Generally, these models describe the diagnostic process as consisting of several steps, such as seeking information, formulating hypotheses, and evaluating these hypotheses [15,16,17]. Diagnosticians may repeat some of these steps, as they may be relevant at different times [15]. Acquiring and representing information, for example, may be needed to specify the diagnostic problem at hand (e.g., information about symptoms), to carry out diagnostic tests (e.g., information about the location of a specific component), or to evaluate a hypothesis (e.g., information about reference values). In most models, it is assumed that the context of the diagnostic task [17] or the extent to which diagnosticians know their domain [16] influences the efficiency of the diagnostic process. Additionally, it is hypothesized that while the steps of the diagnostic process may be domain general, how they manifest in behavior is not [15]. However, such domain-specific aspects are not included in current models of the diagnostic process.

The models reviewed thus far mostly deal with the cognitive steps of fault diagnosis and omit behavioral ones, such as remedying a fault. However, as the two scenarios in the beginning illustrated, the behavioral actions may differ greatly between domains (e.g., a faulty component being exchanged versus the machine being adapted to address changing process requirements). Consequently, actions necessary to remedy a fault pose distinct requirements for diagnosticians [18]. For example, if repairing a component is not a viable option as the required precision cannot be guaranteed, diagnosticians may only have to locate the faulty component without finding out how the malfunction came about. In this case, the availability of certain options would influence the required depth of diagnosis. Thus, behavioral aspects should also be compared between domains. One model that includes both cognitive (i.e., detecting, locating, and isolating a fault) and behavioral steps (i.e., conducting a functional system test, remedying the fault) has been proposed in the domain of diagnosing and repairing household appliances [19]. However, as household appliances arguably differ from more complex work systems, these findings can presumably not be transferred to the domains in focus here.

In sum, existing approaches for modeling fault diagnosis widely acknowledge the influence of domain characteristics, but do not make these influences explicit. Therefore, domain-specific predictions about the cognitive processes and behavior of diagnosticians, the suitability of certain diagnostic strategies, or the need for support cannot be made. In order to make such predictions, it is necessary to better understand how the diagnostic process depends on the task and work environment, and what exactly is required of diagnosticians.

1.2. Comparing the Requirements for Fault Diagnosis in Two Work Domains

As the scenarios from the beginning of the article have illustrated, comparing fault diagnosis between different domains can give valuable insights into the relations between domain characteristics and diagnostic requirements. Some domain comparisons from the literature will be described next, as well as the conclusions that can be drawn and the questions that remain.

Some existing domain comparisons dealt with considerably different domains, such as process control and medicine [20,21]. Examples of work environments from these domains are gas refineries and hospitals. For both environments, there are overarching goals (e.g., safety) and similar measures to realize them (e.g., redundancy), comparable tasks (e.g., monitoring, testing, adjusting), and different roles (e.g., engineers and operators, doctors and nurses) who must collaborate [20]. The comparisons of such widely differing domains can be viewed as an attempt to describe similarities in order to transfer knowledge and strategies between domains. However, an underlying assumption seems to be that transfer requires similarity and, consequently, that for those aspects in which the domains differ, domain-specific solutions have to be found.

A second group of domain comparisons is based on the assumption that both similarities and differences are informative of the requirements that people face when conducting tasks in these domains. In this group of domain comparisons, some authors provide rather general descriptions of the work domains (e.g., [22,23]), whereas others relate similarities and differences to their consequences [24,25,26]. For example, if production systems have different characteristics (e.g., perishability of raw materials), potential measures for increasing resilience (e.g., the ability to stock raw materials) should also differ [24]. Similarly, depending on the tasks of a specific domain, different skills and knowledge would be needed to deal with them [25,26]. The focus on domain differences is especially helpful as differences in task fulfillment indicate the relevance of underlying domain characteristics. More precisely, domains may differ in many ways, but some of these differences may not be psychologically relevant in the sense that they do not influence cognitive processes and behavior, or that they only do so for certain tasks. Including at least two domains provides additional possibilities, because influencing factors may manifest differently in the different domains. More specifically, a domain comparison can be used to examine whether certain cognitive processes or behaviors are shown when the potentially underlying factor is present (e.g., making direct observations when ongoing processes can be observed) compared to when it is not (e.g., conducting indirect measurements when ongoing processes cannot be observed). Therefore, domain comparisons allow us to explore the effects exerted by variations of influencing factors as they occur in real work domains.

For the present study, two technical domains were compared, namely cars and packaging machines. The main focus was on the respective domain characteristics and on the resulting requirements for fault diagnosis. The car domain and the packaging machine were chosen as several experts were available for extensive discussions. Despite being a choice of convenience, these specific domains can be compared in a meaningful way for the following reasons. First, fault diagnosis is a common task in both domains [26,27], and therefore a suitable starting point for characterizing domain differences. Second, the technical systems in both domains are largely constrained by natural laws instead of intentions or rules [28,29]. Consequently, the systems in both domains share some fundamental characteristics, such as, for example, that they realize mechanical and automated functions. Despite these fundamental similarities, the domains also differ greatly from each other, thereby making it possible to examine domain-specific effects. For example, cars are highly standardized mass products that are well understood, while packaging machines are tailored to a specific production context that is often underspecified [30]. Hence, much more domain knowledge seems to be available for fault diagnosis in the car domain compared to the packaging machine domain. This could lead to different strategies being adopted, such as following pre-determined procedures in the car domain and developing novel solutions in the packaging machine domain.

1.3. Present Study

How is fault diagnosis shaped by the background and characteristics of the domain it is embedded in? And how do these characteristics relate to domain-specific requirements that humans have to meet in order to successfully diagnose faults? In the present exploratory study, these questions were addressed by comparing fault diagnosis in the car domain and the packaging machine domain by conducting scenario-based interviews with multiple experts. For both domains, one realistic diagnostic problem had previously been elicited and analyzed to identify initial similarities and differences (see

Table A1. Similarities and differences in diagnostic reasoning as identified from two case scenarios (one per domain), and possible explanations.

	Car Scenario	Packaging Machine Scenario	Possible Explanations
			Domain Characteristic	Alternative Explanation
Most frequently coded steps	Carrying out measurements Collecting new information and data Representing information and data		–	Studied task is fault diagnosis
Most frequently bypassed steps	No thinking about alternatives (e.g., alternative hypotheses, measurement options)		Domains do not afford alternatives	Experts are not aware of cognitive processes
Type of fault	Electronic fault	Decreased product quality interacts with the machine	High percentage of automated functions for cars; interactions are typical in packaging machine domain	Faults chosen by experts were not representative of the respective domain
Measurements	Formalized electrotechnical measurements	Visual and haptic impressions based on experience	Formalized measurements can be applied for cars due to higher degree of standardization	Formalized measurements are used to identify electronic faults, independent of the domain
Basis for evaluations	State and behavior of system components	Behavior of system components together with products	Process boundaries are well known for cars, but not for packaging machines	Difference occurred between scenarios, but is not representative of the domains
Specificity of hypotheses	High, fault location was known	Low, fault location had to be identified	Fault location is identified by expert system in the car domain	Difference occurred between scenarios, but is not representative of the domains
Mode of information processing	Explicit information processing even for high levels of expertise	Automatic information processing for high levels of expertise	Fault diagnosis requires different modes of processing in both domains	Car expert reported more details than the packaging machine expert
Actions for remedying faults	Exchanging the faulty component	Changing inputs and adjusting machine components	Higher degree of standardization for cars than for packaging machines	Electronic components cannot be repaired; packaging machine expert was more closely involved in machine development

in the Appendix A). These scenarios were used as a starting point for interviews with nine domain experts. More specifically, the following questions guided the interviews: Which faults typically occur in the domain? What does the diagnostic process typically look like? What cognitive processes and actions are possible and necessary in order to deal with faults? How would the diagnostic process and the associated requirements change if the domain was different? From the expert interviews, domain-specific influences on the diagnostic process were identified and summarized in the Domain Requirements Model.

2. Materials and Methods

2.1. Participants

Overall, nine experts (five from the car domain and four from the packaging machine domain; see

Table 1. Specification of the experts’ professional background, and the interviews conducted.

Experts	Domain Experience	Domain Experience (Years)	Interview Setup	Interview Duration (min)	Number of Extracted Statements
Car
C1	Service technician for cars	32	Face to face	91	60
C2	Service technician for cars in own repair shop	20	Face to face	81	48
C3	Service technician for cars, subsequent study of mechanical engineering	7	Face to face	96	73
C4	Production engineer at a major automotive corporation	35	Video call	82	9
C5	Software engineer focusing on cyber security at a large automotive corporation	2	Video call	79	23
Packaging machine
PM1	Service technician for a production line with packaging machines	17	Face to face	144	64
PM2	Mechanical engineer for constructing packaging machines	30	Video call	170	70
PM3	Mechanical engineer for constructing packaging machines	28	Video call	98	52
PM4	Automation engineer for packaging machines	26	Video call	101	57

) were recruited through the second author’s professional network. As the goal of the present study was to relate domain characteristics to requirements for fault diagnosis, the experts needed to have work experience either with fault diagnosis in their domain or with the domain itself (e.g., knowledge about the function and structure of the technical systems). In the car domain, three experts worked in independent repair shops (i.e., in repair shops that do not partner with a certain manufacturer). The remaining two car experts had gained their domain experience by working for large automotive corporations. One of these experts had been involved in production planning, while the other was responsible for developing control units. In the packaging machine domain, one expert was working as a technician in a chocolate factory, overseeing and maintaining an entire production line. The remaining three packaging machine experts were working in different development departments of a company developing and manufacturing packaging machines for small-sized confectionary goods. At this point, it should be highlighted that we are aware that nine experts are not sufficient to accurately represent two complex work domains. Rather, we argue that these experts’ insights provide relevant starting points for characterizing differences between the two domains.

2.2. Procedure

To make the best use of participants’ expertise and to incorporate new insights into the subsequent interviews, the questions differed between the interviews. However, the overall procedure was comparable. The only exception was the first expert interview (i.e., the interview with PM2). During this interview, the packaging machine fault scenario was described and discussed extensively before turning to the car scenario, which led to many repetitions and a notably longer interview. Therefore, for all subsequent interviews, both fault scenarios were described at the beginning of the interview. The expert was then asked to identify aspects that were typical for fault scenarios in their domain and to describe typical strategies for diagnosing faults. Since both fault scenarios were described to each expert, it was also possible to compare the domains during the interview. For example, the experts were asked about potential domain differences and to think of any examples that would confirm or refute these notions. It must be noted that remarks on the domain that was not the domain of expertise (e.g., a packaging machine engineer providing input on the car domain) were compared to what the actual experts on that domain said. In case of contradicting statements, the statements from the “real” domain experts were used. Having said that, contradictions between the experts were not identified but instead (and as would be expected), the domain experts provided more detailed statements than the non-domain experts.

2.3. Data Analysis

All interviews were summarized in written protocols and analyzed qualitatively using the open-source software Obsidian (version v1.4.13). The protocols were reviewed for any statements regarding five categories that were determined prior to data analysis, namely the domain background, characteristics of the technical system, characteristics of typical faults, characteristics of the diagnostic process, and requirements for successful fault diagnosis. Within each of these categories, more specific subcategories emerged from the data (e.g., for the category of domain background: goals, changes during use, serial sizes).

Finally, relations between the different domain-specific aspects of fault diagnosis were established to explain differences in the diagnostic process. Starting with differences in fault diagnosis between the car and the packaging machine domain, the other domain-specific characteristics were reviewed for possible relations with the identified differences. More precisely, factors were extracted that manifested differently in both domains (e.g., the degree of automation and ability for self-diagnosis is high for cars and low for packaging machines) and these factors were connected to characteristics and requirements of fault diagnosis (e.g., the level of support during fault-diagnosis is high for cars and low for packaging machines). These domain-specific influences on requirements for fault diagnosis were summarized in the Domain Requirements Model.

3. Results

Overall, 456 statements were extracted from the interviews (see Figure 1): 213 referring to cars and 243 to packaging machines (see Table 1 for the number of statements extracted from each interview). The statements were summarized in the five categories identified before, namely domain background, characteristics of the technical systems, characteristics of typical faults, characteristics of the diagnostic process, and requirements for successful fault diagnosis.

Table 2. Selection of extracted interview statements regarding the domain background, characteristics of the technical system, characteristics of typical faults, characteristics of the diagnostic process, and requirements for successful fault diagnosis (see Table A2 in the Appendix A for all results).

	Car	Packaging Machine
Domain background
Goals	Transporting people and goods Low fuel consumption and emissions High comfort for passengers	Packaging a pre-defined number of high-quality products in a certain amount of time Ensuring process stability
…	…	…
Characteristics of technical systems
Knowledge of system functioning	Functioning of cars and their interaction with inputs and environment are fully known	Functioning of packaging machines and their interaction with inputs and environment are not fully known
…	…	…
Characteristics of typical faults
Knowledge of location	Entry in fault memory narrows down the location of the fault, but not the cause	Often unknown Symptoms and causes have different locations
…	…	…
Characteristics of the diagnostic process
Type of information	Pattern of fault occurrence Entry in fault memory State and behavior of machine components	Pattern of fault occurrence State and behavior of machine components Quality and behavior of products and packaging material
…	…	…
Requirements for successful fault diagnosis
Functional system knowledge	Processes and influencing factors are well understood	Packaging machine itself is understood, but interactions with inputs often are not Effects of single factors are known, but effects of interactions between factors often are not
…	…	…

shows selected results for each of these categories, while all results are provided in the Appendix A (

Table A2. Summary of extracted interview statements regarding the domain background, characteristics of the technical system, characteristics of typical faults, characteristics of the diagnostic process, and requirements for successful fault diagnosis.

	Car	Packaging Machine
Domain background
Goals	Transporting people and goods Low fuel consumption and emissions High comfort for passengers	Packaging a pre-defined number of high-quality products in a certain amount of time Ensuring process stability
Changes during use	No changes within one produced car	Properties of inputs may change but must be processed nevertheless
Serial sizes	Mass production	Small-batch production
Legal background	Regulations regarding safety and environmental impact	Regulations regarding hygiene and safety
Inputs	Air and fuel Fuel composition is legally defined Characteristics relevant to the combustion process are known and understood	Underspecified natural goods and packaging material Not all characteristics relevant to the packaging process are known and understood
Interaction with factors other than the technical system	Car is constantly interacting with the environment (e.g., temperature, humidity, pavement) Simulations and extensive tests are conducted and include extreme situations	Machine operates in a production hall where some factors (humidity, temperature) can be controlled Complex interactions with products, packaging material, and environment are not fully known and understood
Characteristics of technical systems
Knowledge of system functioning	Functioning of cars and their interaction with inputs and environment are fully known	Functioning of packaging machines and their interaction with inputs and environment are not fully known
Mechanical and automated functions	Proportion of purely mechanical functions is very low and constantly reduced further Ability for self-diagnosis is high	Proportion of purely mechanical functions depends on the type of packaging machine, but often high Ability for self-diagnosis is low
Degree of standardization	High	Low
Adaptability	Car can adapt to all relevant changes (e.g., lower oxygen content in the air)	Packaging machines can adapt to some changes (e.g., varying supply of products or packaging material)
Operator influence	Operators do not influence process efficiency, but wear and tear	Operators influence the process efficiency, but not wear and tear Parameters can be changed to adapt to changing process requirements
Feedback and long-term effects	Immediate feedback Long-term effects are possible and typically detected by control units	Immediate feedback Long-term effects are possible and typically hard to detect
Durability	Medium (15–20 years)	High (40 years and longer)
Characteristics of typical faults
Knowledge of location	Entry in fault memory narrows down the location of the fault, but not the cause	Often unknown Symptoms and causes have different locations
Ambiguity of symptoms	Symptoms and entries in fault memory can be evoked by different causes	Symptoms can be evoked by different causes
Novelty	Faults are typically known	Faults can be unknown
Causes of faults	Broken components, wear and tear Electronic faults often occur at the beginning of the lifespan, mechanical faults occur later	Packaging machine (parameter settings, components), quality of inputs, actions of operators and technicians
Characteristics of the diagnostic process
Type of information	Pattern of fault occurrence Entry in fault memory State and behavior of machine components	Pattern of fault occurrence State and behavior of machine components Quality and behavior of products and packaging material
Obtaining information	Conversation with customer Formalized measurements of components directly or indirectly (via the control unit) Test drive with different situations Control units are only accessible with specific licenses and software	Mostly observations Formalized measurements for sensors Assessment during the actual packaging process with products and packaging material Specific system states may be induced to obtain new information
Interpreting information	Possible causes are evaluated according to their likelihood and the overall system state Interpretation of measured values is typically provided in the expert system (with varying specificity) Measured values may be influenced by control unit	Possible causes are evaluated according to their likelihood and the overall system state Evaluation criteria are often unknown, target values are typically variable
Correcting faults	Components or modules are typically exchanged and not repaired as custom-made adaptions are not covered by the vehicle registration Teaching of sensors is not required	Components or modules are repaired and only wearing parts are exchanged Machine-specific adaptions (e.g., changing parameter settings, individualizing mechanical components) are possible Teaching of sensors is frequently required
Diagnostic strategies	Fault diagnosis typically starts with an entry in fault memory Entry in fault memory identifies the area in which the fault occurs Expert system provides the diagnostic strategy Relevant measurements can be derived using functional system knowledge Strategy for diagnosing mechanical faults depends on previous experience Trial and error (in terms of changing components) is costly and time intensive	Proceeding varies greatly between people, depends on previous experience Diagnosis includes an assessment of product and folding quality Two phases of fault diagnosis are typical: (1) identifying the area in which the fault occurs and (2) identifying the cause Entire machine may be checked several times with each iteration focusing on other aspects Observations are often carried out and combined unconsciously Trial and error (in terms of changing parameters) can be successful
Support during fault diagnosis	Extensive documentation of system functioning Fault memory entries are typically specific and useful Extensive support for electronic faults Low support for mechanical faults	Documentation of system functioning is minimal Some potential faults and how they may be solved are covered by the manual
Requirements for successful fault diagnosis
Functional system knowledge	Processes and influencing factors are well understood	Packaging machine itself is understood, but interactions with inputs often are not Effects of single factors are known, but effects of interactions between factors often are not
Role of functional system knowledge for fault diagnosis	Functional knowledge can be largely replaced by the expert system for all known faults, but diagnosticians need functional system knowledge to diagnose novel faults	Functional system knowledge is needed for targeted observations and for interpreting the information Adapting parameters reasonably is only possible if system functioning is understood
Transferability of functional system knowledge	Overall functioning is comparable between all cars Different types of mechanical buildup, within which different car models are very similar Different types of control units, within which the functioning is comparable	Goals (e.g., consistent packaging speed, consistent supply of products and packaging material) and the functional principles used to achieve the goals are comparable between machines Functions and their implementation vary considerably between machines
Empirical knowledge	Linking symptoms and entry in fault memory to their most frequent causes Expectancy of certain faults depending on the overall system state Perception and evaluation of noise and vibration	Linking symptoms to their most frequent causes Expectancy of certain faults depending on the overall system state Perception and evaluation of noise and vibration Knowledge about the behavior of products and packaging material
Role of empirical knowledge for fault diagnosis	Empirical knowledge speeds up the diagnostic process Necessary for mechanical faults (limited support)	Empirical knowledge is needed to limit the scope of possible causes for one symptom and to develop an approach to fault diagnosis
Transferability of empirical knowledge	Faults can occur systematically for specific models	Depends on how similar functions and their implementations are Causes for faults differ between machines and between production environments

). The following five subsections will describe the results for each category, first presenting the statements relating to the car domain, and then those relating to the packaging machine domain. At the end of each section, both domains will be compared. The sixth section specifically focuses on domain differences in diagnostic reasoning and how they relate to the other domain differences.

3.1. Domain Background

Car domain: Cars are supposed to safely transport people and goods from A to B while reducing fuel consumption and emissions as much as possible, and providing comfort for the passengers. These goals are partly defined in legal regulations (e.g., regarding emissions or passenger safety), but may differ between car models. Once a car has been produced, no new requirements arise for this car. Due to this relative stability of goals and the prevalent use of cars in the private sector, cars are mass products. For the most important process during driving, namely the combustion process, air and fuel are needed as inputs. While the characteristics of these inputs may change, they do so within known limits, and the fuel composition is legally defined. It is important that cars function efficiently and safely in different environments (e.g., at temperatures between −20 °C during winter and 40 °C during summer), but not all combinations of environment and system state can be simulated during development. Relevant effects can, nevertheless, be anticipated and are confirmed through extensive tests.

Packaging machine domain: The goal of packaging machines is to package a defined number of high-quality products in a certain amount of time. As packaging machines are part of a specific production environment, they must fulfill the customer’s requirements and ensure high process stability. Thus, packaging machines are usually customized and only produced in small numbers. Over the course of a packaging machine’s lifespan, the type or composition of products and packaging material may change so that an existing packaging machine may have to meet new demands. The inputs (i.e., products and packaging material) are generally underspecified because they consist of natural goods with partly unknown properties. Therefore, not all characteristics relevant to the packaging process are readily available or can be measured. Additionally, the quality of the inputs often changes during production. While packaging machines are utilized in production halls with controlled conditions, even small changes in the environment (e.g., the temperature in the production hall rising from 18 °C to 23 °C) can trigger complex interactions with products and packaging material. These interactions influence the behavior of the machine, but are not always understood.

Comparing both domains: The goals of cars and packaging machines differ substantially, one enabling private consumers to fulfill their needs for transportation, safety, and comfort, and the other allowing business owners to produce packaged goods for further disposal. In contrast to cars, the requirements for packaging machines may change over their lifespan. The inputs used for combustion in cars are standardized and can be formally described, whereas the inputs for packaging machines are underspecified. Additionally, environmental factors vary considerably for cars, and extensive simulations and tests are conducted to ensure proper functioning in this wide range of conditions. Conversely, packaging machines are only exposed to minor environmental changes, but interactions between the environment and inputs often lead to problems in the packaging process.

3.2. Characteristics of the Technical Systems

Car domain: For cars, automated functions monitor and optimize all ongoing processes. As a consequence, the car can adapt to relevant changes in the environment and the inputs, and the ability for self-diagnosis is high. With increasing automation, the impact of the driver is reduced. Today, drivers have no influence on how efficiently the combustion process is carried out, but they do influence the wear and tear of the car’s components. Since cars are subject to different environmental factors and are not built especially sturdy, they last about 15 years.

Packaging machine domain: Since packaging machines have to ensure operator and process safety, and since the ongoing processes are not fully understood, most packaging machines have a high proportion of purely mechanical functions. Due to high packaging speeds, even small changes can lead to changes in machine behavior, but the machines can typically only react to changes within narrow boundaries. The ability for self-diagnosis is therefore rather low. To compensate for the lack of automated adaptation, operators are important to ensure that the packaging machine functions properly, and thus, they directly influence the efficiency of the packaging process. Since the machines are typically very robust, the operators’ influence on wear and tear is low, and machines can last 50 years or longer.

Comparing both domains: In contrast to packaging machines, most of the relevant influences on the ongoing processes in cars are understood and can be measured. As a consequence, cars are highly automated and the driver’s influence is very low. For packaging machines, ongoing processes, as well as interactions with products, packaging material, and the environment need to be understood better before the degree of automation can be increased further. Cars and packaging machines also differ greatly regarding their lifespan, with that of cars being much shorter compared to that of packaging machines.

3.3. Characteristics of Typical Faults

Car domain: Faults in the car domain are often indicated by an entry in the car’s fault memory. This entry typically narrows down where the fault occurs, but a certain set of symptoms or fault entries can be evoked by different causes. Only a few faults are novel, meaning that they are anticipated by designers to large extents or have occurred before (e.g., during system development), and the expert system typically suggests a possible cause. Faults are often caused by components that have reached the end of their lifespan or by wear and tear. Electronic components can be affected from the beginning of a car’s lifespan, while faults in mechanical components typically occur later on.

Packaging machine domain: One typical fault characteristic for packaging machines is that the symptoms occur at another location than the actual cause, and specific symptoms can be evoked by different causes. Since the production environment and its components are underspecified, novel faults can occur. Oftentimes, the cause of faults is not found in the packaging machine per se, but rather in deviations of process parameters that interact with the packaging machine in an unwanted way. For example, if a chocolate bar’s ability to withstand mechanical loads is reduced due to errors in previous process steps, the bar may break during packaging. Due to the high influence of operators and technicians on the packaging process, their actions can also lead to faults. When multiple actions are carried out simultaneously (e.g., changing different settings), it is hard to trace the effects of each action.

Comparing both domains: In both domains, a specific set of symptoms can be evoked by different causes. In the car domain, the fault location is often indicated by the expert system. Most faults are either covered by this system or have occurred before, whereas faults in the packaging machine domain may be completely novel. Faults in the packaging machine domain may be caused by the machine itself, the inputs, or operators’ and technicians’ actions. In the car domain, inputs or actions of car owners or technicians are only rarely responsible for faults.

3.4. Characteristics of the Diagnostic Process

Car domain: The diagnostic process typically starts with an entry in fault memory and a description of the pattern of fault occurrence by the client. Additionally, the car can be taken for a test drive during which diagnosticians may induce different situations to narrow down the affected subsystem. Since symptoms and fault entries can be evoked by different causes, experts use their experience and the impression of the overall system state to evaluate the likelihood of different causes. Most fault entries are specific and useful, and the expert system also provides information on carrying out specific measurements and interpreting obtained signals and values. Components can either be measured directly or indirectly via the control unit, and the resulting values are raw values or may already be interpreted by the control unit, respectively. Diagnosticians must therefore keep in mind that incorrect values may be caused by a malfunctioning component (i.e., incorrect raw values) or by a fault within the control unit (i.e., incorrect interpretation of correct raw values). Access to the control unit software is only possible with an additional license. If there is no useful fault entry, relevant measurements can be derived systematically using functional system knowledge. For mechanical faults, the diagnostic strategy is highly dependent on previous experience and implicit knowledge. Noise and vibrations, for example, are important information sources for diagnosing mechanical faults.

To remedy a fault, components or entire modules are typically exchanged instead of repaired. This is because exchanging components is often cheaper than repairing them, and some components (e.g., electronic components) can only be repaired using special equipment that is not available in repair shops. For exchanging components, finding out why and how they were damaged is only necessary if there are any signs of unusual impacts (e.g., heat or mechanical deformation). Finally, custom-made adaptations are not covered by the vehicle registration and are therefore avoided if possible. After repair, the car is typically taken for a test drive to confirm that the fault is resolved, and to allow the control units to adapt to the new component.

Packaging machine domain: To identify faults, diagnosticians collect information on the quality and behavior of products and packaging material, the state of machine components, and the pattern of fault occurrence. For a specific symptom, different causes have to be considered and are evaluated according to their likelihood. The operating manual typically only covers the basic functioning of the machine, as well as some faults and how they may be solved. In case of unknown faults, the diagnostic process can be split into two phases. First, the area in which the fault occurs must be identified. For this, the entire machine may have to be checked several times, each iteration focusing on another aspect. Second, the actual cause must be found, but how this is carried out highly depends on the fault. Since the packaging process can be perceived directly, targeted observations are the most important way of obtaining information. In addition, specific system states (e.g., feeding fewer products into the machine) may be induced to check the effects on machine behavior. Interactions between the inputs and the machine are frequent and, therefore, diagnosticians must obtain as much information as possible during production (and not, e.g., during downtime). While observations are often chosen and combined in an intuitive fashion, formalized measurements also exist for diagnosing electronic faults. Target values are often not pre-determined as they depend on the production context, and the means to achieve them (e.g., specific machine settings) can sometimes only be determined through trial and error.

For some faults, machine components must be repaired or exchanged completely. Another frequent action to remedy faults is the teaching of sensors. For all of these changes, consequences for the quality of the packaged products can be observed, but side effects may appear in the long term and are then hard to trace back. For example, the friction between the conveyor belts and the products may increase during a shift due to an increase in product buildup. These changes may be compensated by changing machine settings. However, if the conveyor belts are cleaned at the end of the shift, the settings do not work for the new shift. Finding out which parameters were changed and why is a tedious task for the personnel in the new shift.

Comparing both domains: How faults are identified and corrected varies significantly between the two domains. In the car domain, components are judged independently by conducting formalized measurements, while in the packaging machine domain, it is crucial to observe the actual machine behavior during the production process. During these observations, diagnosticians compare their perception to implicit, internal evaluation criteria. Explicit reference values are only rarely available in the packaging machine domain, but typical in the car domain. Certain system states may be induced in both domains to collect additional information, but this is carried out more frequently in the packaging machine domain. For most faults in the car domain, the expert system provides a guideline on how to proceed during fault diagnosis, whereas diagnosticians in the packaging machine domain often have to develop their proceeding ad hoc. This can include making changes to the system in a trial-and-error fashion and evaluating the consequences. If carried out systematically, trial and error can be a successful strategy in the packaging machine domain, whereas, in the car domain, it is typically more time consuming and costly than other strategies.

3.5. Requirements for Successful Fault Diagnosis

Car domain: Extensive knowledge about the ongoing processes, possible influencing factors, and their effects is available in the car domain. This functional knowledge is also well documented and stored in expert systems that are widely used for fault diagnosis. The expert system can therefore replace the need for functional knowledge on the part of the diagnosticians, but not for novel faults. Regarding the structure of both mechanical and electronic components and modules, a small number of different types exist. Within these types, the functioning is similar, and hence, knowledge can easily be transferred. Empirical knowledge is obtained during fault diagnosis and pertains to links between symptoms, fault entries, and their most frequent causes. As a result, experts form an expectation about the likelihood of certain faults, for which they also take the overall state of the car (e.g., age) or the specific model into account. Especially for diagnosing mechanical faults, unformalized information like noise or vibration is helpful, and therefore, own experience is necessary. Even though many faults could be identified using functional knowledge alone, empirical knowledge can significantly speed up the diagnostic process. Diagnosticians in the car domain additionally need methodological and technical knowledge for operating measurement devices. In the past, most faults could be diagnosed using just a multimeter, but nowadays more complex devices like oscilloscopes have to be used more frequently.

Packaging machine domain. In the packaging machine domain, functional knowledge about the machines is limited since interactions with the inputs or with the influences on the packaging process (e.g., humidity, temperature) are often only vaguely known. Diagnosticians, therefore, must keep a large number of potential causes in mind (e.g., machine components, input characteristics, environmental influences, operator actions), but only have a limited amount of functional knowledge at their disposal. However, they need this knowledge to make targeted observations and draw inferences about why a particular problem may arise. Since guidelines for parameter changes and repairs are missing in many cases, functional knowledge is also needed to derive useful actions. Some of these actions (e.g., physical adaptations of components) additionally require craftsmanship. While the goals of different packaging machines (e.g., consistent packaging speed, consistent supply of products and packaging material) and the functional principles used to achieve these goals are comparable, how they are implemented on a physical level is not. In addition, the peculiarities of each production line and custom adaptations of single machines restrict the transfer of functional knowledge across different packaging machines and may additionally require craftsmanship.

Comparing both domains. The extent to which functional system knowledge is formalized and available for fault diagnosis varies greatly between both domains. While all factors relevant to the ongoing processes in the car are understood and well documented, many interactions between packaging machines and their respective products and packaging materials are not even known. In addition, the transferability of functional knowledge is much higher in the car domain, because there are only a few different types of general functioning and physical structure of cars. Likewise, the extensive support from expert systems reduces the need for diagnosticians to possess functional knowledge in the car domain. This changes once the fault is not covered by the expert system, which is typical of mechanical faults. In these cases, empirical knowledge is needed to plan and carry out the diagnostic process, as it is for packaging machines. An additional requirement for successful fault diagnosis in the car domain is the ability to operate increasingly complex measurement devices. In the packaging machine domain, additional demands are placed on diagnosticians because of the need for manual adaptations and repairs, as they require functional knowledge and a certain level of craftsmanship. Also, knowledge about the other steps of the production process is needed, since each step can influence the characteristics of products and packaging material in a way that changes the behavior of the machine.

3.6. Developing the Domain Requirements Model for Fault Diagnosis

So far, both domains were compared regarding domain-specific aspects of fault diagnosis, but the main aim of the present study was to identify how these aspects relate to each other. Those factors leading to differences in the diagnostic process were selected and summarized in the Domain Requirements Model (see Figure 2). The contents of the model will subsequently be described, starting with the observed difference in the diagnostic process (i.e., column D) and moving along the connections to the domain background, system characteristics, and typical faults (i.e., columns A through C). Finally, the requirements associated with the diagnostic processes (i.e., column E) are explained.

Several of the packaging machine experts highlighted that it is crucial to assess and evaluate a packaging machine within the process environment (D1), because faults may occur in the packaging machine without being caused there (C1; e.g., chocolate bars breaking in the machine, but the cause being located in previous process steps). Conversely, the car and its functioning were assessed independently of the environment. This domain difference corresponds to the large influence of external factors on ongoing processes in the packaging machine domain (A1), while only some of them are known and measurable (A2). In the car domain, external factors influence ongoing processes to a smaller extent and in ways that are well understood, so these influences can be handled automatically.

Another difference between the diagnostic processes in both domains is the applicability of trial-and-error strategies (D2). These strategies are thought to be useful if faults are novel (C2). To anticipate faults, designers must be able to predict the machine’s behavior in various situations (B1). Consequently, changes in input characteristics and influencing factors must be known and measured (A2, A3) to define the boundaries of ongoing processes. If this knowledge is available, novel faults are less frequent (C2), and the measurements used during the diagnostic process seem to be more formalized (D3). Additionally, the better faults can be anticipated, the more can diagnosticians rely on elaborate support and self-diagnosis tools (D4).

The domains studied here also differ regarding the extent to which machine components are repaired and manually adapted (D5). This seems to relate to the number of produced machines of one kind (A4), which is much higher for cars than for packaging machines. When this number increases, a higher degree of standardization (i.e., smaller tolerances in the production of machine components, B3) can be achieved, which reduces the need for manual adaptations. While manual adaptations are still necessary in both domains, they are much more prevalent in the packaging machine domain.

Finally, the described relations between domain background, system characteristics, typical faults, and diagnostic process reveal the requirements that must be met for successful fault diagnosis. Functional system knowledge (E4) seems to be more important when only little support is available during fault diagnosis (D4), when the machine cannot be assessed independently of the process it is embedded in (D1), when trial-and-error strategies are applied (D2), and when repairs and manual adaptations must be carried out (D5). Consequently, diagnosticians in the packaging machine domain are required to know more about the functioning of the machine than are diagnosticians in the car domain. In the packaging machine domain, they also need process knowledge (E1) to make accurate judgments about the behavior of the packaging machine in its specific process environment (D1). The need for empirical knowledge (E3) is thought to be especially relevant if measurements are not formalized (D3), and if only little support is available during fault diagnosis (D4). Again, this implies that empirical knowledge is more important in the packaging machine domain. However, for some faults in the car domain, little or no support is available. While formalized measurements decrease the need for functional and empirical knowledge, they require diagnosticians to successfully carry out the prescribed procedures, which often include the operation of complex measurement devices (E2). These requirements are more pronounced in the car domain. Finally, repairs and manual adaptations are more prevalent in the packaging machine domain. These activities require that the actual cause of the fault is specified (E5), and potentially also craftsmanship to implement the desired changes (E6).

4. Discussion

While fault diagnosis is often supported by automation, automated diagnostic tools cannot be introduced in every domain [31,32]. Consequently, people are confronted with different requirements for successfully diagnosing faults. The present study aimed to identify these requirements and to examine how they depend on the characteristics of specific domains. This was achieved by comparing the diagnostic process between the car domain and the packaging machine domain. Starting from one fault scenario per domain, nine domain experts were interviewed about diagnostic reasoning and how this process is influenced by the peculiarities of their domain. From the interviews, several factors influencing the diagnostic process were identified, such as the extent to which changes in influencing factors, machine behavior, or input characteristics are known and measurable, the ability of the technical system for self-diagnosis, or whether faults are novel (i.e., whether faults can be anticipated during system design or have occurred before). These factors seem to relate to different aspects of the diagnostic process, such as the support during fault diagnosis, the extent to which a trial-and-error strategy is efficient, or the extent to which faults are remedied by manual adaptation and repair. Depending on these characteristics, different requirements for human diagnosticians were derived, such as the need for functional, empirical, or process knowledge. The influences were summarized in the Domain Requirements Model (see Figure 2) that can be used to make predictions about fault diagnosis and the requirements associated with it for different types of faults and in different domains.

4.1. Review of the Domain Requirements Model Considering the Fault Diagnosis Literature

Next, the contents of the proposed model will be reviewed in light of previous studies on fault diagnosis. Figure 3 indicates whether empirical findings could be identified for the respective aspect of the model. According to the Domain Requirements Model, the extent to which faults are novel (C2) influences three different aspects of the diagnostic reasoning process, namely the extent to which a trial-and-error strategy can successfully be applied (D2), the extent to which measurements are formalized (D3), and the extent to which fault diagnosis is supported (D4). Regarding related domain characteristics, the model argues that faults can only be anticipated if the technical system is sufficiently understood (B1). This in turn requires that changes in machine behavior, input characteristics, and influencing factors are known and measured (A2, A3). Several findings support this line of reasoning. For example, the better the environment of a technical system can be specified, the better humans can establish causal relations within that system [33]. Conversely, when people have less or even false knowledge about a system, fault diagnosis becomes more difficult [34]. Further, the more uncertainty there is about the relations between the inputs and outputs of a system, the more knowledge is needed to diagnose faults [35]. Especially for unanticipated faults, functional knowledge is important to connect the individual parts of the faulty system to each other, thereby reducing the number of individual parts to be considered [36]. Thus, more functional and empirical knowledge is needed to diagnose novel faults.

The literature also provides some insight into the situations in which trial-and-error strategies are successful, as well as the requirements associated with this strategy. Trial and error is applied even by highly experienced and successful operators [37], and if used in a systematic fashion, more information about the system and the occurring fault can be obtained [38]. This strategy is therefore more efficient than what most people default to in underspecified situations, namely carrying out well-practiced responses irrespective of whether they are suitable or not [11]. In fact, experts are often reluctant to make changes to the system when they have not yet understood the fault [39]. Another strategy for dealing with novel faults is to judge the system or specific components according to the known, optimal state [40,41]. For both strategies (i.e., applying a trial-and-error strategy or conducting good/bad mappings), functional knowledge is required to determine the criterion (i.e., what a good or bad condition looks like in the current situation), to interpret the available information, and to derive suitable actions.

According to the Domain Requirements Model, the requirements for successful fault diagnosis are also influenced by the degree to which support is available (D4). The data presented here suggest that the more support given, the less diagnosticians are required to possess functional (E4) or empirical knowledge (E3). For situations in which no or only little support is available, other authors especially highlight the need for experts to develop their own strategies [3,42], meaning that they have to decide which information is relevant and how it can be obtained. Thus, diagnosticians need functional and empirical knowledge to acquire relevant data and to evaluate their hypotheses [16].

If the beneficial effects of functional knowledge for fault diagnosis are evident, one important question ensues: What happens if this knowledge is increasingly held by an automated support system, and not by the human diagnostician? The interview data suggest that a lack of functional knowledge can partly be compensated by empirical knowledge. However, drawing conclusions from previous experience also relies heavily on functional knowledge [43,44], especially if a previous case is relevant but not exactly the same as the current one. Thus, successful fault diagnosis depends on functional knowledge after all, and additional requirements are introduced if support systems are used for fault diagnosis. Since support systems typically reduce the demands of the actual task, attention and, ultimately, performance are reduced as well [45]. To prevent such detriments, diagnosticians would have to intentionally exert effort to be more attentive. Also, support systems often act as mediators between the technical system and the human, which means that additional skills are needed to interpret the provided information [46]. Instead of making knowledge on the part of human diagnosticians obsolete, the application of support systems likely changes the importance of different types of knowledge and introduces new requirements [47].

Overall, it can be stated that the pertinent literature supports some contents of the Domain Requirements Model. As these findings mostly relate to the availability of system knowledge, fault knowledge, and support during fault diagnosis, the domain-specific contents need to be studied further. However, the model can already be used to describe the requirements for fault diagnosis within and between different domains, as will be illustrated next.

4.2. Using the Domain Requirements Model to Describe Requirements for Different Fault Types

Within both domains, several experts stated that the diagnostic process differs significantly between different types of faults, and these differences can also be explained using the Domain Requirements Model. One example would be whether an electronic or mechanical fault is diagnosed. In both domains, automated functions are monitored and thus, the ability for self-diagnosis (B2) is high. Consequently, electronic faults can often be anticipated by designers (C2). This means that support is available for diagnosing these faults (D4), and the required functional and empirical knowledge is consequently rather low (E3, E4). Additionally, electronic components are typically produced in large numbers (A4) and are highly standardized (B3). Defective electronic components are therefore exchanged instead of repaired (D5) and, consequently, identifying what has caused the defect (E5) as well as craftsmanship (E6) are less important.

Mechanical faults, on the other hand, can be caused by a multitude of different influences, and not all of them can be predicted and monitored (A2). For example, one of the experts stated that mechanical faults could be caused by loose components, jammed components, stones, bumps in the road, or how much luggage is loaded in the car. In the context of fault diagnosis, this means that mechanical faults are often novel (C2) and that less formalized knowledge (e.g., in an expert system) is available for diagnosing mechanical faults (D4). Consequently, diagnosticians have to rely more on their own functional and empirical knowledge (E3, E4). In sum, the proposed model cannot only describe and predict requirements on the domain level, but also on the level of the technical system or specific types of faults. This may be especially helpful if diagnostic tasks change, for example, due to the introduction of new technology [48].

4.3. Using the Domain Requirements Model to Describe Fault Diagnosis beyond the Domains Studied

The main goal of the Domain Requirements Model is to predict requirements for fault diagnosis in any domains that exhibit the factors included in the model, not just in the car domain or the packaging machine domain. While the present study was conducted, the opportunity arose to cross-check some of the findings with experts from other domains, namely trucks and agricultural machines. Since these domains share some of the previously mentioned characteristics, the model can be applied to them as well. In both of the additional domains, machines represent capital goods and, therefore, requirements regarding safety and reliability are comparable to those in the packaging machine domain. Agricultural and packaging machines further share the requirement of processing underspecified goods. In contrast to packaging machines but similar to cars, both trucks and agricultural machines are mobile and have to function in different environments. Even though these additional interviews were far less comprehensive than those in the original two domains, they revealed first insights into how well the Domain Requirements Model can be used to describe and predict requirements for fault diagnosis in other domains.

For trucks, the factors concerning the domain background manifest as follows: External factors have little impact on ongoing processes (A1), and relevant changes in influencing factors, machine behavior, as well as input characteristics are known and can be measured reliably (A2, A3). Machine functioning in different situations is well understood (B1) and, consequently, only a few faults are novel (C2). In comparison to cars, trucks are even more automated and, therefore, the ability for self-diagnosis is also higher (B2). Trucks are produced in large numbers (A4), and the degree of standardization is high (B3). From the Domain Requirements Model, it would follow that trucks can be assessed outside of their typical environment (D1; e.g., on a testing bench while being completely unloaded, instead of on the road while being fully loaded with goods). Also, support during fault diagnosis should be high (D4), trial and error should not be a useful strategy (D2), and formalized measurements should be frequently used (D3). Finally, the degree to which components are repaired or manually adapted should be rather low (D5). While most of these predictions were confirmed, the extent of repair and manual adaptation was larger than expected. This can be attributed to the high costs for components, even though the number of produced trucks and the degree of standardization are high. Therefore, the cost of components could be another relevant factor for explaining the degree to which repairs are carried out. An additional aspect that is not yet contained in the model but seems to be relevant for trucks is functional integration (i.e., one component realizes several functions). The degree to which functions are integrated and the consequences for the diagnostic process could be an interesting path to study further.

For agricultural machines, the influence of external factors (e.g., humidity) on ongoing processes is rather high (A1), but most of these factors and their effects are known (A2). In contrast to packaging machines but similar to cars and trucks, machine behavior and characteristics of inputs (e.g., density of the crop) are known and can be measured as well (A3). Novel faults are less frequent (C2), and the degree of automation as well as the ability for self-diagnosis are high (B2). The number of produced machines (A4) and the degree of standardization (B3) is much higher than for packaging machines, but there are still substantial differences between different types of agricultural machines (e.g., tractors, harvesters). The positive relation between the degree of automation (B2) and support during fault diagnosis (D4) was confirmed, as both factors are typically high for agricultural machines. Nevertheless, it seems that even though the degree of automation is high, defects in electronic components occur less frequently than what would be expected. Instead, mechanical components are often affected, with wear and tear being the most prevalent cause of faults. Therefore, the degree to which manual adaptations and repairs are necessary (D5) is higher than predicted by the model. Similar to trucks, this can in part be attributed to high costs for exchange parts. A second reason could be high requirements regarding machine availability and process safety (e.g., a harvester might only be needed for one week of the entire year, but during this one week, it needs to function). If components are exchanged preventatively, different types of faults may occur and, thus, the requirements for fault diagnosis will likely be influenced.

4.4. Limitations and Directions for Future Research

Several aspects of the methodological approach chosen for the present study determine what inferences can be drawn from the presented data. These limitations particularly concern the background of the interviewed experts and the exploratory nature of the study.

First, all experts from the packaging machine domain had experience with packaging machines for small-sized confectionery goods. As other packaging machines may differ immensely from the ones in focus here, other characteristics might be relevant for fault diagnosis. While the factors contained in the Domain Requirements Model (e.g., the degree of automation or the degree to which faults are known) may manifest differently in different packaging machines, these machines also share general characteristics [30]. Future research should determine whether the requirements for diagnosing faults in other packaging machines can also be explained and predicted using the Domain Requirements Model.

Second, the interviewed experts were involved in different stages in the machine’s lifespan, and this seems to have been a systematic domain difference in the present sample. More specifically, the majority of the packaging machine experts were involved in the development of new packaging machines, whereas most of the interviewed car experts were involved in repair and maintenance. This may have influenced the experts’ impression of which faults are typical in their domain, with packaging machine experts facing more novel faults than car experts. We concluded that novel faults occur more frequently for packaging machines than for cars, but this finding might not hold true if fault diagnosis during car development is studied further.

Third, the present sample clearly is a convenience sample, and finding suitable experts is not always easy. Experts must not only possess comprehensive knowledge and skills in their respective field, but must also be able to access and verbalize this knowledge (e.g., [35,49]). In the present study, experts were also asked hypotheticals, namely whether they thought that their knowledge and proceeding could be transferred to another domain. While this limitation should be kept in mind, no overt disagreements between different experts were identified. More precisely, minor disagreements between the experts could be attributed to the different contexts they work in and, hence, to differences in their expertise.

Fourth, and on a more general note, it should be emphasized that the characteristics and requirements were elicited bottom-up; that is, they were identified from the interviews. An alternative approach would be to identify potentially relevant domain characteristics in the literature, and then ask the experts about them. The bottom-up approach was chosen for the present study because the previous literature largely deals with domain-general influences on fault diagnosis or with domain-specific aspects across different technical systems and contexts (e.g., [26]). As our focus was on identifying domain-specific influences on requirements for diagnostic reasoning in two distinct technical systems, the literature base was not sufficient. In future research projects, we plan on testing both approaches (bottom-up vs. top-down) to identify how they can be combined to develop the model further. Future research should focus on validating the proposed factors and relations and indicate which factors are most important in the context of fault diagnosis. As the current form of the model presents a collection of relevant domain differences as a starting point, we expect more factors to be added in the future. Among the most promising influencing factors are the simultaneous occurrence of multiple faults and the extent to which symptoms can be masked [14,50], the extent to which faults occur intermittently [14,51], or the consequences of faults for the overall system [3,41]. The extent to which these characteristics occur in different work domains and how this shapes the diagnostic process remains to be studied further. Comprehensive methods such as cognitive work analysis [2,52] could be used for this purpose. While the model currently only allows qualitative judgments about the need for certain requirements, future research should aim to quantify the factors and relations included in the Domain Requirements Model. Finally, more research is needed to derive domain-specific descriptions of the different knowledge types and strategies, including their transferability across domains.

5. Conclusions

The present study examined the relations between domain characteristics and cognitive requirements during fault diagnosis. The Domain Requirements Model was developed that relates the domain background, characteristics of the technical systems, and typical faults to the diagnostic process and the resulting requirements. This model can help to determine whether solutions from one domain can be transferred to another. Providing extensive rule-based or model-based support during fault diagnosis, for example, is only possible if relevant changes in influencing factors, input characteristics, and system behaviors are known and can be measured. As the Domain Requirements Model indicates the resulting requirements, it can be used to develop domain-specific support during fault diagnoses, such as assistance systems or trainings. The findings also stress the importance of functional knowledge for fault diagnosis, especially if these faults are novel. Finally, the present article has focused on fault diagnosis as one of many relevant cognitive processes that humans carry out as part of their work. Identifying the relations between domain-specific characteristics and requirements for other kinds of cognitive tasks remains a promising avenue for future research.