The Need for Multi-Sensor Data Fusion in Structural Health Monitoring of Composite Aircraft Structures

Abstract

With the increased use of composites in aircraft, many new successful contributions to the advancement of the structural health monitoring (SHM) field for composite aerospace structures have been achieved. Yet its application is still not often seen in operational conditions in the aircraft industry, mostly due to a gap between research focus and application, which constraints the shift towards improved aircraft maintenance strategies such as condition-based maintenance (CBM). In this work, we identify and highlight two key facets involved in the maturing of the SHM field for composite aircraft structures: (1) the aircraft maintenance engineer who requires a holistic damage assessment for the aircraft’s structural health management, and (2) the upscaling of the SHM application to realistic composite aircraft structures under in-service conditions. Multi-sensor data fusion concepts can aid in addressing these aspects and we formulate its benefits, opportunities, and challenges. Additionally, for demonstration purposes, we show a conceptual design study for a fusion-based SHM system for multi-level damage monitoring of a representative composite aircraft wing structure. In this manner, we present how multi-sensor data fusion concepts can be of benefit to the community in advancing the field of SHM for composite aircraft structures towards an operational CBM application in the aircraft industry.

1. Introduction

The use of composite structures in the aerospace industry has increased over the past decades with newer aircraft such as the Airbus A350 or Boeing 787 made largely (respectively 53% [1] and 50% [2]) of composite material. This development can be attributed to its mechanical properties, which include a high strength-to-weight ratio and corrosion resistance. Consequently, the use of composite structures in aircraft can lead to reduced operational costs due to less fuel consumption and fewer maintenance activities [3]. Although many advantages can be retrieved by using composite structures for aircraft, their behavior also poses a challenge; in particular, the inspection and monitoring of damage in the aircraft structure become more challenging, as damage may be present internally but not identifiable upon visual inspection of the outside layers of the aircraft structure during maintenance procedures. Furthermore, even if signs of damage are seen upon the surface, for example, in terms of barely visible impact damage, their size might not be representative of the actual extent of damage in the inside layers of the composite [4,5,6,7]. Additionally, the damage mechanisms and their growth behavior are complex, which makes the prediction of damage growth in composite aircraft structures under loading challenging.

In the aircraft industry, such unknowns must be controlled and damage must be detected before it compromises the structural integrity of the composite component in the aircraft, which is commonly achieved by setting predefined inspection intervals as part of the aircraft maintenance strategy. Additionally, structural parts may be over-designed, leading to greater aircraft weight, or are preventively replaced during maintenance after a specified time interval. These fixed-time-interval and preventive maintenance programs in the aircraft industry can lead to unplanned maintenance activities, comprehensive inspections when no damage is present, or unnecessary replacement of undamaged parts. To mitigate these aspects, a condition-based maintenance (CBM) strategy can be implemented which can provide both cost and time benefits with an estimated benefit of over 700 million Euros per year for the European aviation industry alone [8]. In a CBM approach, maintenance actions are decided based on the actual health state of the aircraft structure: only if a structure requires repair or replacement are maintenance actions conducted. Consequently, this requires an indication of the aircraft structure’s health state. Therefore, CBM in the aircraft industry for structural applications relies on structural health monitoring (SHM) techniques, which consider permanently installed sensor networks to collect data for health management consisting of detecting and monitoring damage. Thereby, sensor data is used to assess the presence of damage in the aircraft’s structural part, predict future damage growth, and decide on further maintenance actions for the given aircraft.

Applying SHM in the aerospace industry has not yet fully matured [9] and especially its application to composite structures is challenging. For a realistic aircraft maintenance scenario, one must consider the needs of the maintenance engineer at the final application: what type of health management information is required to make further maintenance decisions for an aircraft? For example, as we will discuss in this work, this requires a full damage diagnostic on all four SHM levels rather than a single level, that is: (1) damage detection, (2) damage localization, (3) damage type identification, and (4) damage severity. For aircraft maintenance considerations, the next step beyond damage diagnostics should also be incorporated, i.e., damage prognostics, in which the remaining useful life (RUL) of an aircraft structure is estimated based on the collected sensor data. Namely, the aircraft maintenance engineer must know whether there is damage present in a given aircraft, as well as its characteristics and future impact on the performance of the entire aircraft. Additionally, in an in-service scenario, large complex composite aircraft structures will be monitored under realistic loading conditions, which requires reliable, probabilistic, and upscalable algorithms. Requirements and intended functions of an SHM system for the aircraft industry have been collected and discussed in detail in SAE standard ARP6461 [10].

In this work, we dive further into the challenges of monitoring complex composite aircraft structures using SHM. We argue that a multi-sensor data fusion approach is required and is beneficial for applications in the aerospace industry. Namely, it can, amongst others, lead to a more complete image of the present damage, reduce ambiguity, and increase confidence in the results [11,12]. Although studies on data fusion in the broader sense of SHM have been performed [13], research into their application to composite structures with an aerospace application has not. In this work, we approach ideas in a conceptual manner and the need for researching multi-sensor data fusion for SHM of composite structures in aerospace applications is argued.

This paper is structured as follows. First, in Section 2, we identify gaps that must be addressed in order to apply SHM methodologies to composite aircraft structures. We assess the concepts of upscaling by addressing several considerations for applications to aircraft components and the concept of an integral health management assessment consisting of diagnostics and prognostics required for maintenance applications. Next, in Section 3, the concept of multi-sensor data fusion is introduced and its opportunities, advantages, and limitations are further elaborated upon. Section 4 considers the application of multi-sensor data fusion techniques for SHM of composite aircraft structures by discussing new opportunities, practicalities, and considerations. In Section 5, we demonstrate the first indications of how multiple SHM techniques and data fusion concepts can be incorporated by presenting a high-level design of a conceptual SHM system for a complex multi-stiffener composite aircraft wing structural component. Lastly, Section 6 concludes this work and provides an outlook.

2. Challenges in Applying SHM in the Aircraft Industry for Composite Aircraft Structures

In the aerospace industry, the composite structures incorporated in aircraft are on a component level. Yet most research focuses on lower-level structures as small research coupons. When contemplating a successful application of SHM to such aircraft structures, several gaps in research can be identified that must be addressed. In this section, we discuss two main aspects limiting the current progress in its implementation in the aircraft industry. The first, covered in Section 2.1, discusses the application of SHM techniques to realistic, larger, and more complex composite component aircraft structures and the requirements on the SHM methodologies emerging from such new applications. The ability to obtain a holistic damage assessment (Section 2.2) is hindered by the second aspect of intrinsic capacities of the SHM techniques that, on their own, do not match the requirements from the application to aircraft composites given in a CBM implementation in aircraft maintenance (Section 2.3). Lastly, in Section 2.4, we commence with our vision on why multi-sensor data fusion is required in the field of SHM for composite aircraft structures.

2.1. Upscaling

Aircraft structures are made from an assembly of a large number of components and subcomponents. Yet most SHM methodologies are developed in research labs on research structures. These works are often comprised of small coupons or panels—either flat or stiffened—while fewer studies consider applications on a scale similar to those structural parts used in the aircraft industry; for example, on a subcomponent level, such as on multi-stiffener panels, or on the component level, such as on a wingbox or in-service application [9,14]. Consequently, this leads to minimal research into applying SHM methodologies developed at composite coupon or element level to composite aircraft structures at the (sub)component level. In addition to application to higher structural levels, more realistic loading conditions must be considered at these higher levels: most research related to aircraft structural (sub)components focuses on quasi-static loads or impacts under room environmental conditions, while studies involving fatigue loading, different environmental conditions, or combined loading cases, such as impact with fatigue loads under elevated temperatures, are limited even though these are precisely the conditions during aircraft operations.

In structural design and testing for aircraft structures, a building-block approach is commonly employed. Such a building-block approach (commonly illustrated using a pyramid) consists of coupons at the bottom level and the complexity of structural elements increases as we move towards higher levels. The pyramid in essence describes a strategy in which many tests are performed on the lower levels involving generic elements and fewer tests are performed on higher levels involving non-generic components and full-scale structures (e.g., a wing structure). Here, the assumption is made that the lower levels are simplified representations of the higher levels and that gathered knowledge can be transferred from one level to the next. Thereby, both costs and risks in the aircraft structural design and testing process can be reduced.

For future applications of CBM in the aircraft industry, health management methodologies must be developed that are considerate of and applicable to larger and more complex aircraft structures and components. Therefore, we propose to also follow a building-block approach for SHM, indicated in Figure 1. Namely, for the implementation of SHM concepts in the aircraft industry, a similar approach must be established: the developed SHM methodologies and results obtained at lower coupon levels should be translatable and representative of its application on the higher levels. Moreover, developments and knowledge gathered on the lower levels are key in the study of the next. In essence, we aim at developing methodologies on the lower levels in laboratories using large sample sizes and at applying these to the higher levels under in-service scenarios with minimal re-development.This can be done by defining homogeneous populations on single levels and heterogeneous populations comprising multiple levels [15,16,17]. For homogeneous populations comprising a single structural type on a single level, current research studies involving sensor and diagnostic and prognostic methodology development are commonly applicable. The definition of a homogeneous population allows for the inclusion of various artificial intelligence techniques as (semi-)supervised learning algorithms to combine knowledge obtained from multiple same-type structures. As such, it involves the development and training of diagnostic and prognostic methodologies on subsets of the population and their subsequent application on similar structures in the same domain [18,19,20]. For heterogeneous populations consisting of multiple structural types and levels, artificial intelligence techniques such as transfer learning can be of aid, allowing the knowledge gathered in a source domain to be utilized in new and different target domains, in this case, knowledge gathered on the lower-level structures and applied to higher-level structures [21,22,23].

As only a small amount of research has been performed into the application and development of SHM techniques to structural aircraft components of the higher levels, we must point out several considerations that must be made before such a building-block approach for SHM for upscaling aircraft structures can be implemented. These are similar to those seen for heterogeneous populations: geometry, material, and topology [22]. In this work, the changes in structural geometry and the differences in structural aspects in higher-level components in the aircraft application and their impacts on SHM applications are of interest as they are variable between the different levels in the building-block approach. Each of these two aspects is discussed separately next.

2.1.1. Changes in Structural Geometry

When considering the application of an SHM technique to a (sub)component aircraft structure, it needs to be recognized that we are dealing with a changing structure with respect to the conditions at its original development. Firstly, there is a change in its dimensional growth: component level structures have larger dimensions than those for a lower-level coupon. This may affect the performance of the SHM methodologies related to both the sensor capacity and the SHM technique.

Originating from the aircraft’s different dimensions, the first point of attention is the spatial coverage of the selected SHM technique. A sensor might be capable to monitor damage only in a limited area; for example, in the case of acoustic emission (AE), its signals attenuate over a distance. Hence, when damage occurs at a large distance from the sensor, it might go unnoticed. Thus, when applying such techniques in a damage monitoring system of an aircraft with a changing structural dimension, it consequently means that one must now also consider whether the chosen sensors are capable of monitoring the full structure of interest. Therefore, the exploration of a sensor placement optimization study must be considered in the design of an aircraft component’s SHM system and is crucial in the upscaling of SHM applications to component-level aircraft structures. For example, in the case of employing AE sensors, one must ensure that the AE sensor network provides sufficient coverage over the full structure, taking into account attenuation aspects and varying AE signal characteristics [24,25].

Similarly, when changing a structure’s geometry, the performance of an SHM methodology can be affected. For example, the damage state can be assessed on a local or on a global level. At a local level, sensors record local changes in structural properties and must therefore be placed near hotspots (e.g., fiber Bragg gratings (FBG) for strain measurements). For its application in (sub)components, this requires identification of these hotspots, for example at joint locations between different parts of a wingbox or at the nose region with high risk of impacts, or the—much less efficient, high-cost, and high-weight—application of many sensors throughout the structural parts if sensor topology is not optimized. At a global level, the response of the entire structural part is considered to assess the presence of damage. Yet, where a small damage on a coupon level might cause changes in the global behavior of that coupon, its effect on the response of e.g., a wingbox segment or multi-stiffener panel can be minimal, resulting in challenges when one tries to detect such minimal changes in an aircraft structural application [26].

2.1.2. Effects of New Structural Aspects

In the upscaling towards a full aircraft structure, not only changes in terms of geometry are observed, but the structures now also contain assembly details and consist of several sub-elements. This results in additional geometrical boundaries, which can influence the effectiveness of the applied SHM methodologies. Hence, when we are considering the application of SHM to aircraft components in the health management system, we need to analyze how such boundaries or assembly details may affect the diagnostic and prognostic results and how their methodologies may require adjustments: a simple and direct application will not always work. For example, when considering damage localization in a structural aircraft component using AE, the presence of an assembly detail such as a bolt may influence the path of an AE wave, thereby potentially affecting the localization results [27]. Similarly, the placement of piezoelectric transducers (PZT) for guided wave (GW) applications on different planes and curved structures in a fuselage segment poses new challenges as it can alter the wave propagation [28]. Based on the structural parts and sensing technologies, a sensor network configuration must be designed accordingly, with sensors being optimally placed by taking into account such structural aspects. Namely, it needs to be considered whether the health management system should be constructed based on sub-elements, each having their own damage monitoring system including separate sensors for each sub-element, or whether the structural elements can be monitored using a single sensor network covering the full aircraft structure. Optimal sensor placement studies that take into account such structural boundaries can improve the design of the sensor network as, for example, shown by Thiene et al. [29] and Soman et al. [30] for GW applications involving various boundaries and structural complexities.

2.2. Integral Damage Assessment

For CBM applications in the aircraft industry, it is of essence that a full view of the health state of the aircraft structure be available and that details of present damage be provided. Airline maintenance engineers require information ranging from whether damage is present up to whether maintenance actions are required [10]. For instance, only knowing that damage exists without having further knowledge of whether it is located in a primary or secondary structural member or on its size and severity is insufficient in the in-service environment of an aircraft. Moreover, having information on the current damage state of the aircraft structure without having the ability to estimate its RUL does not allow one to make effective maintenance decisions when a CBM approach is implemented. Therefore, CBM requires us to look at an integral damage assessment that includes both damage diagnostics and prognostics. In the field of SHM, commonly, four diagnostic levels are distinguished:

• Damage detection;
• Damage localization;
• Damage type identification; and
• Damage severity.

This can be followed by a next step, namely, damage prognostics in which one aims to determine the RUL of the structure up to a predefined state [31]. For implementing SHM for CBM in the aircraft industry, it is thus of interest that systems are developed that address all aspects: from damage detection up to RUL estimation in order to support enriched maintenance decisions.

In recent years, a variety of SHM techniques and approaches have been proposed to monitor composite structures and to perform damage diagnostics and prognostics. Yet an integrated health management framework including both diagnostics and prognostics has not yet been demonstrated. Most studies in which SHM techniques are applied focus on the assessment of a single SHM level. Focus lies on, for example, developing new methodologies using current or new SHM technologies (e.g., as seen in [32,33]) or on improving current methodologies (e.g., as seen in [34,35]). Research devoted to assessment on all SHM levels or on combining diagnostics and prognostics is uncommon [36]. Additionally, studies on damage assessment under realistic combined loading cases with complex failure mechanisms (e.g., impact damage combined with fatigue loads) are limited. Instead, the focus is often narrowed down to a single damage assessment at a single SHM level (e.g., as seen in [37,38,39,40,41]). The focus of these studies is not surprising, as we can argue that each SHM technique has its own inherent capacity that limits a full-covering application including a holistic damage assessment. Yet, as outlined in the SAE standard ARP6461 [10], the latter is precisely what is needed when implementing an SHM system for aircraft structures. In the next section, we will dive further into this by discussing the strengths and limitations of different SHM techniques.

2.3. Intrinsic Capacity of SHM Techniques

Several SHM techniques have been used to perform SHM of composite structures including, but not limited to, vibration-based methods [42,43,44,45], AE [27,36,40,46], optical fiber (OF) sensing [47,48,49,50], and GW [51,52,53]. When considering damage assessment in composite structures, each sensing technique has its own capacities. This can mean that a certain technique can only be employed to gather information on specific diagnostic levels or prognostics, but also that only certain types of damage can be identified. These aspects differ per SHM technique: some are sensitive to matrix cracks, while others only identify larger delaminations; some provide local damage localization with high resolution, while others only provide global monitoring. An extensive review of the different SHM techniques and their advantages and limitations, as well as their intrinsic capacities, was presented by Amafabia et al. [37], and we refer the reader to this work for further details regarding these features. Next, we further discuss and illustrate these aspects and the differences among SHM techniques.

The strength of a certain technique for an individual SHM level can clearly be seen when evaluating vibration-based methods for aircraft applications of SHM. Vibration-based methods are based on detecting changes in the global response of the structure, thereby making them suitable for the detection of damage (diagnostic level 1) [44]. Various vibration-based approaches have been employed to identify changes in SHM datasets for structural damage detection, ranging from modal-based and time-series approaches to machine learning and deep learning techniques, including unsupervised learning for novelty detection and trained classifiers [54,55]. However, the method is not equipped for precise damage localization since it assesses the structure as one entity. This means that vibration-based data can provide information that the aircraft wing is damaged, but not to what extent or in which segment. Another example of level-specific SHM can be seen when assessing AE for damage monitoring. While AE has shown its strengths in damage detection (level 1) and type identification (level 3), additional challenges are faced when using it for precise damage localization (level 2). Although methods are being developed in order to allow for more precise damage localization in composite structures, the inherent anisotropy of composite aircraft structures poses challenges [27].

In addition to SHM techniques having an inherent capacity related to specific diagnostic or prognostic levels, we must also consider that not all techniques can be used to monitor all damage types in composite structures [37]. Damage in a composite can be composed of a variety of different damage types, each with its own impact on the integrity of the structure. In this regard, the detection of matrix cracking using the previously mentioned vibration-based damage detection methods is difficult. Namely, the presence of matrix cracks will only have a minimal impact on the overall structural behavior and likely be undetectable when using a vibration-based damage detection technique. For the previous example of an aircraft wing, this means that the local presence of a matrix crack has no substantial influence on the global wing’s response. Meanwhile, when an AE-based damage monitoring technique is used, it is possible to detect and identify matrix cracks as AE is sensitive to matrix cracking.

Additionally, when considering the final structural application, in our case, composite aircraft structures, certain SHM techniques can be more appropriate than others. Namely, based on its application, different needs are established for each SHM level. A way in which the application influences the requirements can for example be seen when considering the moment of detection. If the damage caused by unexpected impact events, such as a tool-drop during maintenance or a hailstone impact, must be observed at their moment of occurrence, it requires a continuous monitoring approach and one might select the inclusion of a passive SHM technique over an active one. On the other hand, if there is a need to monitor structural degradation over a longer time period, such as after a set number of operational flights, one might choose a different technique that does not require continuous monitoring, but instead relies on fixed-interval measurements to allow for time-wise comparisons. A third example in which the aerospace application poses requirements on the to-be-used SHM technique originates from the minimum damage size that should be detected. Namely, certification documents can indicate at which damage size further maintenance actions are required. Consequently, the SHM methodologies incorporated in the aircraft’s health management system must be able to detect and assess the damage state in the structural part before reaching this critical limit, thereby imposing lower-level requirements for each damage assessment level. In essence, for any SHM methodology to be a viable damage monitoring method for a given application, it must fulfill the corresponding imposed requirements coming from its application [10].

2.4. The Need for Multi-Sensor Data Fusion

In the previous two sections, we elaborated upon and discussed the limitations of sensing techniques for application to composite aircraft structures, as well as the aspects of upscaling techniques towards such aircraft components. We have seen how each sensing technique has an intrinsic capacity and how a CBM application might pose requirements on the to-be-employed SHM techniques for the aircraft industry. Moreover, we took a closer look at the application of SHM techniques and methodologies to larger and complex component aircraft structures, and how these considerations and challenges might affect the application of the SHM techniques. Next, we bring these two aspects together and discuss the complexity of applying SHM for composite structures in aircraft and introduce our vision of the need for multi-sensor data fusion concepts.

The damage formation in composite aircraft structures used in aircraft during operation is complex and may consist of a variety of different damage types, including matrix cracking, delaminations, fiber–matrix interface debonding, fiber pull-out, and fiber breakage. Furthermore, when upscaling to larger composite aircraft structures in realistic and complex loading scenarios as seen during flight, additional failure modes become probable, such as impact damage, damage in joints, or stiffener–skin debonding in stiffened panels. Moreover, the probable failure scenarios might differ between subcomponents in the larger aircraft structure with outer parts being more susceptible to impact damage. Additionally, due to the larger dimensions of the aircraft, the presence of multiple damages at various locations becomes a realistic scenario.

When considering a CBM implementation involving composite aircraft structures, one must consider all of the previously mentioned aspects: a variety of structural components and a variety of damage modes, as well as the need to include a holistic damage assessment incorporating both diagnostics and prognostics. The aircraft’s health management system used in a CBM implementation must be able to address and handle the entire range of mixtures between structural parts and damage types. It must provide its diagnostic and prognostic estimates and conclusions with high reliability, high confidence, completeness, and following certification guidelines. Moreover, due to the high safety levels in the aircraft industry, high requirements are placed on such an SHM system: deviations from the true conditions must be minimal and should certainly not have any fatal consequences.

Based on our rhetoric earlier in this section, we can safely conclude that a methodology based on a single SHM technique will not be able to handle such a broad set of requirements pertinent to the aircraft industry. The design of the SHM system should not be led by the SHM technique but rather by the application [10]. There is no ‘one size fits all’; instead, a combination—a fusion—of SHM techniques and methods must be studied. SHM systems must be designed that are capable of dealing with this wide range of variety: systems in which each SHM technique does what it does best and provides the appropriate results for an aerospace implementation.

For a composite aircraft structure, this might, for example, require an SHM system that consists of multiple sensing techniques: a vibration-based method for global damage detection, OF strain sensing for local and precise measurements of delamination growth, and GW for impact damage sizing. In order to obtain such a system, multi-sensor data fusion concepts are required, in which data from different sensors are combined in order to obtain a comprehensive damage diagnostic and prognostic assessment. Next, before continuing our discussion on the possible application and implementation of fusion techniques in the field of SHM for composite aircraft structures, we first provide a background on the concepts of multi-sensor data fusion in the next section, Section 3.

3. Multi-Sensor Data Fusion Concepts

The concepts of multi-sensor data fusion aim at combining data from multiple sensors to obtain an improved outcome with respect to when each sensor is employed individually. Hereby, advantages such as expanded coverage, increased confidence and reduced ambiguity, and redundancy can be expected [11,12]. Before diving further into the opportunities created by applying multi-sensor data fusion techniques, it needs to be established what we exactly mean by fusion, which will be discussed first in Section 3.1. This is followed by Section 3.2, in which two ways of categorizing fusion approaches are considered. Lastly, Section 3.3 considers the advantages and challenges of applying fusion techniques in more detail, thereby laying the ground for Section 4 in which the application of multi-sensor data fusion in the field of SHM for composite aircraft structures is considered.

3.1. Definition of Fusion

Fusion can occur on a variety of levels and from a variety of sources, leading to a variety of terms and type specifications. Several of these terms, for example, sensor fusion, data fusion, and multi-sensor data fusion, are used by researchers, often in an interchangeable manner. Moreover, a variety of definitions for the term fusion exists in literature, yet each one describes a similar underlying concept [56]. In this work, we define the concept of fusion by combining two definitions, namely that proposed by Bostrom [56] and by Dasarathy [57], and present it as follows:

“Fusion is the study of efficient methods for automatically or semi-automatically transforming information from different sources and different points in time into a representation that provides effective support for human or automated decision making”, “such that the resulting decision or action is in some sense better […] than would be possible, if these sources were used individually without such synergy exploitation.”

[56,57]

In this study, the focus lies on the fusion of sensor data obtained from multiple sensors, which can include sensors from either similar or different SHM techniques. We will refer to this concept using the terms ‘multi-sensor data fusion’ or ‘fusion’ in short. Note that in this regard, we do not focus on the aspects of single-sensor temporal-based fusion, in which data from a single sensor at different time instances are fused, though the application of such methodologies can be inherent to the considered SHM subjects.

3.2. Fusion Characterization

Fusion techniques can be conceptually assessed in different manners and no single course exists due to the variety of applications. For example, one can classify based on sensor relations [58], input–output relations [59], or representation level [60], or by considering different fusion models such as the JDL model [61], Boyd model (OODA loop) [62], or Omnibus model [63]. Where the latter are of less interest, the first two categorizations are highly applicable to the topics discussed in this work and are therefore considered in more detail next.

3.2.1. Fusion Characterization Based on Sensor Relations

Durrant-Whyte [58] presented a classification of sensor fusion techniques based on the relations between sensors. This resulted in three classes, illustrated in Figure 2, and defined as:

• Competitive. In competitive fusion, the sensors are used in a competitive fashion with the sensor outputs providing information on the same part of a system. By comparing them, confidence in the output can be increased and robustness is obtained. An example is two temperature sensors providing independent measurements on the same system.
• Complementary. In complementary fusion, sensors are complementary and their outputs provide information on different parts of a system. Combining them results in a more complete image of the damage state of a structure. An example is the monitoring of strain at different locations using several FBGs; their combination allows for strain monitoring of a complete region of an aircraft structure.
• Cooperative. In cooperative fusion, sensor data from different sensors are combined to obtain information that cannot be obtained when using a single sensor. An example is digital image correlation (DIC) in which two cameras are used to obtain a 3-dimensional field assessment.

3.2.2. Fusion Characterization Based on Input–Output Relations

Dasarathy [59] classified fusion techniques based on the input–output relation. The classification consists of five levels, defined as follows and summarized in

Table 1. Data fusion characterization based on input–output relation.

Name	In	Out
DAI-DAO	Data	Data
DAI-FEO	Data	Features
FEI-FEO	Features	Features
FEI-DEO	Features	Decisions
DEI-DEO	Decisions	Decisions

:

• DAI-DAO. Raw data, as directly obtained from multiple sensors, are combined resulting in new fused datasets.
• DAI-FEO. Raw data from multiple sensors are combined to derive fused features.
• FEI-FEO. A set of features are combined to form one or multiple new features.
• FEI-DEO. A set of features are combined in order to obtain an output set on the decision level.
• DEI-DEO. Multiple decisions are fused to obtain a new output on the decision level.

3.3. Benefits and Challenges of Multi-Sensor Data Fusion

A range of advantages can be obtained by applying fusion concepts rather than using a single sensor. These advantages may include [11,12]:

• Completeness. Different sensors can provide information on different aspects of the considered structure, hence by fusing their measurements, a more complete image of the considered structure can be obtained.
• Redundancy. Using multiple sensors can provide redundancy in case they are employed in a redundant or competitive manner. As such, the negative effects of sensor failure on the results can be mitigated.
• Improved confidence. The confidence in the results can be increased when several measurements confirm one another.
• Reduced ambiguity. Fusing data and results can lead to a smaller number of possible explanations.
• Improved spatial coverage. Spatial coverage can be extended when using multiple sensors to cover a larger area.
• Improved temporal coverage. Temporal coverage can be improved when sensors are used for measurements at times when other sensors are unavailable.
• Increased system reliability. The inclusion of the aforementioned benefits, such as the redundancy of multiple sensors, can result in an increase in the overall SHM system reliability.

Although the application of multi-sensor data fusion can bring several benefits, it also brings additional challenges. These are related to the combination of different datasets or to the consideration of additional fusion processes and aspects in the SHM system design. These challenges may include [12,64]:

• Data heterogeneity. Different sensors may record measurements of different properties.
• Spatial alignment. Measurements may be recorded in different coordinate systems, thereby requiring spatial alignment for fused assessment.
• Temporal alignment. Measurements may be recorded at different time instances, thereby requiring temporal alignment for fused assessment.
• Sensor reliability. Missing or erroneous data from faulty sensors may affect the fused results.
• Conflicting data. Measurements of the same property or information derived from different sensor measurements may be conflicting.
• Data dimensionality. By including additional sensors, the size of the datasets can grow rapidly, which may require the inclusion of processes aiming at dimensionality reduction.
• Propagation of measurement aspects. Each sensor measurement may have different levels of, e.g., accuracy, uncertainty, or noise. The propagation of these aspects into the fused results must be considered.
• Sensor network design. Sensor locations can affect diagnostic or prognostic performance, thereby requiring the inclusion of optimal sensor placement studies.
• Selection of fusion methodology/design of fusion process. A wide range of fusion approaches is available, requiring one to take additional steps into account in the design of the SHM system (e.g., level of fusion, selection of fusion algorithms).

All in all, although the application of multisensor data fusion can lead to a variety of advantages, it can also pose challenges and lead to additional design and processing steps. Nonetheless, with proper preparation and design of the fusion system, one can obtain several benefits with respect to employing a single sensor. Considerations of its application in the field of SHM for composite aircraft structures are discussed in the next section.

4. Application of Multi-Sensor Data Fusion for SHM of Composite Aircraft Structures

In this section, we take the previously introduced fusion concepts and consider them for SHM applications in composite aircraft structures. Hereby, we focus on changes in the considered aircraft structure (i.e., the upscaling to larger and more complex component structures) and on the possibilities for holistic damage assessment using multiple techniques required in the health management system, as discussed in Section 2. The benefits of using multi-sensor data fusion approaches for SHM of composite aircraft structures are unfolded and several important additional considerations are outlined.

This section is organized as follows. First, in Section 4.1, the concepts of fusion are employed to address SHM upscaling for complex aircraft structures. Next, in Section 4.2, we outline additional points of attention induced by applying fusion techniques, such as practical and methodological considerations.

4.1. Upscaling

The upscaling to component structures required for applying SHM in the aircraft industry leads to new requirements for the SHM system, as well as new opportunities in combination with the application of fusion techniques. Here, we discuss these in Section 4.1.1 and Section 4.1.2, respectively.

4.1.1. New SHM Requirements from Upscaling

The application of an SHM system to composite aircraft structures imposes several new requirements on such a health management system, specifically on its sensors and methodologies. These were discussed in Section 2.1, in which we identified aspects such as spatial coverage and presence of assembly details. In upscaling to larger, more complex composite structures including curvature and assembly details, the first aspect to consider is the spatial coverage of the sensor. It is likely that more sensors of the same type will be required to cover the complete area. In a multi-sensor setting, these can form a network in which each sensor monitors a specific area, thereby obtaining an ensemble by fusing their data in a complementary manner. Dependent on the technique and application, it can be chosen to allow single-sensor areas to overlap or be adjacent to one another. When a structure consists of multiple elements, such as a wingbox (shown in Figure 1) consisting of multiple ribs and stringers, it can also be decided to place sensors on each of its elements for individual monitoring and subsequently fuse their data for a complete component assessment.

A second aspect to consider when dealing with component-level aircraft structures are new assembly details such as bolts and new boundaries for which the inclusion of fusion concepts in SHM can be of great benefit. These regions are often considered to be critical areas for the structural integrity of the component and by including multi-sensor data fusion, a design for an SHM system can be made that monitors such hotspots specifically in addition to the damage monitoring of the complete component. One can consider placing additional sensors near such regions, for example, by placing sensors near the wing-to-fuselage joint, or, more specifically, by placing sensors near a stiffener to monitor potential disbonding between a stiffener and skin. Subsequently, by employing a complementary fusion approach, both local regions of interest as such joints and the full structural component can be monitored simultaneously for damage growth as part of the health management system.

Both aspects instill the need to perform optimal sensor placement studies, ensuring that full coverage is achieved in terms of spatial coverage and damage monitoring. The definition of full coverage for a given SHM technique can be fluid in such conditions, dependent on the application case. Namely, in a multi-type sensor setting, the synergy of multiple SHM techniques can provide a full damage image whereas the individual sensing technique only comprises part of the required scope. The need for studies into sensor placement and their optimization in combination with multi-type sensing approaches for SHM is discussed in more detail in Section 4.2.1.

4.1.2. New SHM Opportunities from Upscaling

The upscaling to larger aircraft structures also provides several opportunities for SHM methodologies. A variety of aspects, originating from those discussed in Section 3.3, are identified, including opportunities for redundancy, completeness of the damage assessment, higher confidence, and system reliability. Each of these is discussed next.

Redundancy

Firstly, for component aircraft structures, it is possible to include redundancy in SHM systems. Due to the larger structure, space is available, especially with respect to research type structures such as coupons, to include more sensors than the minimum needed to perform damage diagnostics or prognostics. Thereby, it allows for redundancy: if a sensor fails for whatever reason or provides erroneous data during flight, a diagnostic and prognostic assessment can still be obtained using the remaining sensors. This can be of benefit, especially for in-service applications in an operational aircraft. In the aircraft industry, the safety levels are high and a failing sensor resulting in missing data can be unacceptable. Yet, as the inclusion of more sensors might also lead to increased weight, an optimization study on the number and the location of sensors must be conducted in which a trade-off is made between, on the one hand, the increase in aircraft weight and cost and, on the other hand, the obtained benefits such as increased redundancy, completeness, and confidence for the aircraft health management system. For example, such a study was partially conducted for SHM of a metallic fuselage structure by Dong and Kim [65].

Completeness

Secondly, as the composite aircraft component structure comes with a larger spatial area, an opportunity for improved damage assessment arises. Namely, it allows for the use and implementation of multiple types of SHM techniques. As discussed in Section 2.3, not all SHM techniques have similar capacities. Thus, when considering a holistic damage assessment on all four diagnostic SHM levels and prognostics as required for a CBM application in the aircraft industry, it can be resourceful to combine multiple SHM techniques using fusion approaches rather than spending efforts to improve a single SHM technique to assess all levels, even when their intrinsic capacity might not allow it. In this manner, the fusion of multiple SHM techniques can lead to completeness in the damage assessment, thereby using the strength of each technique to its fullest, and to full coverage of the information needed by the aircraft maintenance engineer during flight operations.

Additionally, it needs to be considered that different SHM techniques monitor different damage types. If the requirements of the given structural part in the aircraft state that multiple damage types must be monitored, an improved assessment can be obtained by fusing the SHM data from two techniques, given the appropriate sensor placement for their function. This is, for example, demonstrated by Broer et al. [36] who fuse AE and OF measurements to obtain a full four-level diagnostic assessment for stiffened composite aircraft wing panels subjected to both impact and fatigue loads. A multi-sensing approach is also shown by Romano et al. [66], who combine GW and OF measurements for a completer diagnostic assessment of impact and disbond damage in a similar stiffened composite panel.

Confidence

The use of multiple sensors—either similar or varying types—also allows for improved confidence in the damage assessment results provided by the aircraft’s health management system. If sensor data collected by independent sensors confirm one another in either the diagnostic or prognostic results for a specific structural part, a higher degree of confidence can be obtained. Moreover, for damage diagnostic level 4 ‘severity assessment’ and prognostic RUL estimation, a fusion of sensor data from two techniques monitoring different damage types into a single health indicator (HI), for example, by means of an FEI–FEO fusion, allows for immediate assessment of both. The benefits of this are shown by Eleftheroglou et al. [67], who combine AE and DIC features into a single metric for RUL estimation.

Reliability

With the previously three mentioned opportunities arising from upscaling from research samples to larger and complex aircraft structures, that is, (1) redundancy, (2) completeness, and (3) confidence, an additional SHM system benefit can be obtained in terms of reliability. Aspects such as the redundancy of multiple sensors and robustness against conflicting results and interference cause the overall SHM system’ reliability to improve. Final diagnostic and prognostic results will be less affected by the failure of a single sensor or by its erroneous data as data from the remaining sensors can be used for substitution, thereby ensuring high reliability over a longer operational period of the aircraft’s in-service time.

4.2. SHM System Complexity: Practical and Methodological Considerations

When upscaling SHM systems using fusion-based techniques in its application to (sub)component aircraft structures, the SHM system’s complexity will generally increase with higher-level structures. Multiple SHM techniques and sensors will be involved that must work together—without interference—in a single health monitoring system of the aircraft. Moreover, the fusion of sensor data for diagnostics and prognostics is accompanied by additional challenges, amongst others, in terms of heterogeneous data and data alignment. In this section, we lay out these considerations and categorize them according to their topic, namely related to (1) practicalities, and (2) methodologies and results, and discussed separately next.

4.2.1. Practical Considerations

When implementing fusion techniques in an SHM system for aircraft structures, several practical aspects must be addressed. A variety of aspects, originating from the challenges discussed in Section 3.3, are identified, including data (mis)alignment, acquisition conditions, interference, sensor topology, and data dimensionality. Each of these is discussed next.

Data alignment

A first consideration to recognize when performing multi-sensor data fusion is data alignment in terms of both spatial and temporal aspects. Sensors take measurements in a local coordinate frame and might consider a different acquisition frequency. Consequently, it is required to transform those measurements to a general reference frame before performing data fusion. Note that this step can be required both for same-type sensors and for different-type sensors. An example of the need for spatial alignment can be seen when sensors are placed on different elements of the component, needing transformation to the common reference frame to assess the entire structural component or even aircraft structure. An example of the need for temporal alignment can be seen when different SHM techniques employ a different measurement interval that requires the adjustment into a common time-based measurement system; for example, when one technique takes measurements every hour during flight while another technique takes measurements only during taxiing. Lastly, any applied windowing procedures as part of the methodological approach must consider the acquisition settings of the individual SHM techniques and their coherence.

Acquisition conditions

A second aspect to consider is that certain SHM methodologies require similar loading and environmental conditions when taking sequential measurements. For example, GW signals are affected by changes in the measurement conditions such as temperature changes, which can have a significant influence on the performance of the damage models [68]. Hence, if the temperature during measurements cannot be controlled, as is ordinarily the case when an aircraft is used in operation, one must implement additional compensation steps in the SHM methodology, such as the one shown by Yue and Aliabadi [69], who developed a GW temperature variation compensation technique. Alternatively, methodologies can be developed that can handle SHM data collected under different environmental or operational conditions, such as the method presented by Vitola et al. [70], who developed a methodology involving principal component analysis and numerous classifiers (nearest neighbor, decision trees, and support vector machines) for damage detection by training the classifiers using GW data collected under various temperature conditions. Furthermore, transfer learning concepts can aid to limit the number of testing conditions by defining suitable populations (Section 2.1).

In addition to having to take this into account when designing their health management system for the aircraft structure including its incorporated methodologies, one should also consider this in the aspects of fusing several SHM techniques. Multiple measurements might have to be made at a similar time or shortly after one another; hence, it might be preferable if the different techniques are designed to work under similar conditions. For example, if one SHM technique is designed to take its active measurements during the taxiing phase of the aircraft’s flight schedule, it can be practical to define similar requirements for the second SHM technique. In this manner, combined with the potential inclusion of compensation techniques into the SHM methodologies, the additional complexity in the SHM methodologies of the health management system can be minimized.

Interference

This brings us to a third point, namely, that it should also be considered how measurements of one technique might interfere with those of another technique used in a joint SHM system of an aircraft. For instance, when monitoring an aircraft component both using GW and AE, the former can be picked up by sensors of the latter system. Hence, when considering a fusion of the two in a single monitoring system, one must consider additional preprocessing steps for AE or a temporal pause in the AE data acquisition during GW measurements. Moreover, physical changes arising from a sensor failure may lead to false damage detection by other sensors. In a multi-sensor approach for SHM, this can be mitigated by including sensor self-diagnosis steps or sensor failure detection algorithms allowing one to recognize the difference between sensor failure and damage presence. This could be done based on the collected sensor data, for example, by directly evaluating signal characteristics or by using unsupervised algorithms for outlier detection [71,72].

Sensor topology

With an increasing number of sensors in an SHM system for larger and more complex aircraft structures, the importance of sensor topology optimization increases. Namely, it is not feasible nor practical to apply a large number of sensors as this will directly lead to greater weight and cost, which is undesirable in applications in the aircraft industry. Moreover, it will lead to additional complexity in diagnostic and prognostic methodology development in various aspects, for example, by imposing even higher demands on data dimensionality reduction techniques, as discussed in the next practical aspect.

Yet, a too low number of sensors, a non-optimized sensor topology configuration, or an ineffective fusion of different sensing techniques, may lead to reduced performance of the diagnostic and prognostic methodologies and thereby of the overall SHM system for the CBM application. Especially in multi-type sensing approaches, complications may arise; for example, if the location of an OF is moved for the benefit of placing a PZT sensor, the OF will no longer be able to indicate the presence of damage at its original location, creating a risk for damage detection performance if not properly compensated for. As such, optimal sensor placement studies are essential, taking into account the effects of multi-type sensing systems and their combined configuration, while ensuring effective damage monitoring capabilities.

Opportunities for effective sensor placement have been demonstrated by various studies, thereby reducing the number of sensors, optimizing sensor locations, while simultaneously ensuring the effectiveness of damage monitoring [73,74]. Compared to single-type sensor applications, the use of multi-type sensors in a fusion setting can complicate the optimization process due to the increased number of potential configurations and additional constraints [75]. Although their number is limited, various research has already shown successful sensor topology optimization in cases involving multi-type sensors while avoiding redundancy and providing accurate damage assessment [76,77,78]. Furthermore, with the increased underlying complexity of these multi-type sensor surveys, the application of data-driven artificial intelligence techniques can be of benefit in terms of higher efficiency, flexibility, and computational costs, compared with traditional approaches [79,80,81].

Data dimensionality

A last practical aspect that needs to be noted is data dimensionality. Including multiple sensors or even multiple SHM techniques, in large aircraft structures, combined with long-term damage monitoring during flight operations, can cause the size of datasets to grow exponentially. Moreover, the high dimensionality of data does not always lead to an improved depiction and may even reduce the performance of the methodologies (curse of dimensionality). In this regard, one has to consider the inclusion of dimensionality reduction into the health management system. During the design of the SHM system, the implementation of data dimensionality reduction can be performed at any instance, dependent on its application. For example, it may occur directly after the sensor data are obtained by extracting and storing only predefined features, or it may occur after some initial processing. Numerous approaches ranging from feature selection to feature extraction are available for this purpose. On the one hand, feature selection minimizes the number of features by selecting the ones that are the best fit for a given objective. It is commonly categorized into filter (based on statistical measures as information gain or Fisher score), wrapping (dependence on the machine learning algorithm, includes various selection and elimination approaches as sequential forward selection or backward elimination and genetic algorithms), or embedded methods (integration of feature selection within the machine learning algorithm, includes LASSO and decision trees as random forest) [82]. On the other hand, feature extraction considers the transformation of features into new representative features in lower-dimensional spaces using, for example, principal component analysis or linear discriminant analysis [20].

Additionally, one needs to consider the storage of historical data—which information is needed for future damage assessments, which parts are of no further interest, and for which parts only limited information extracted from the initial dataset is required. In this regard, the selected diagnostic and prognostic methodologies are of great importance and should be taken into account when designing the data dimensionality reduction approach, as such considerations heavily depend on the employed SHM techniques and methodologies, which also pose requirements on the required stored datasets. We can provide an example by looking at the hyper-feature developed for a prognostic RUL model. Eleftheroglou et al. [67] designed a hyper-feature from DIC strain and AE data for composite coupons. After initial data processing to form the hyper-feature, the originally extracted features from the DIC and AE data are no longer required for RUL estimations in the prognostic model. Instead, only the hyper-feature values have to be stored and, by doing so, dataset sizes can be controlled.

4.2.2. Methodology and Results

In addition to practical considerations, several aspects must be addressed regarding the employed methodologies and results when implementing fusion techniques in a health management system for the aircraft structure. These aspects are related to data heterogeneity, the design of the fusion process, and the differences in data properties from various SHM subsystems. Each of these is discussed next.

Data heterogeneity

Firstly, if one includes multi-sensor data fusion, one might retrieve either homogeneous or heterogeneous data. Methodologies must be selected to fuse these data and adaptations to diagnostic or prognostic methodologies may be needed, potentially requiring the development of new methods and the creation of new features. Especially when considering the fusion of heterogeneous data, this is of importance—a translation into a similar unit might not be viable. In this regard, normalization and standardization steps can be of interest, though this may have a downside of requiring a priori knowledge of the potential value ranges and distributions for each sensor. For certain machine learning algorithms, such as regression or distance-based algorithms, these processing steps are essential as their diagnostic and prognostic performance will otherwise be affected.

The inclusion of multi-sensor data fusion also provides new opportunities for the extraction of enhanced features based on heterogeneous data for diagnostic and prognostic purposes through cooperative fusion compared to single-type technique-based features. By using training and test datasets, (semi-)supervised learning algorithms can be employed to optimize feature extraction methods based on objective functions with respect to manually designed features [83]. The combination of prognostic applications with heterogeneous data from multi-type sensing approaches sees additional complexity, as a fused feature must fulfill three crucial objectives defined as monotonicity, trendability, and prognosability [84]. As such, powerful deep learning tools are receiving increased attention, and various techniques have been applied for feature extraction, including autoencoders and neural networks [85].

Design of fusion process

A design for the fusion approach is required: fusion can be performed on several levels, as characterized by Dasarathy [59] and discussed in Section 3.2.2. If we want to fuse data from similar type sensors throughout an aircraft’s structural component, we can perform the fusion at every level, ranging from raw-level to decision-level fusion. However, when the SHM system involves multiple sensor types of which data must be fused, the raw data fusion levels are no longer feasible. Instead, only a feature or decision level fusion is applicable. Moreover in terms of feature-level fusion, various approaches for designing features are available, ranging from hand-crafted to artificial intelligence-aided solutions. Due to the complexity induced by considering multi-type data sources, powerful mathematical tools such as machine or deep learning are attractive for feature-level fusion, as discussed in the previous paragraph on the consideration of data heterogeneity. Similarly, the complexity of designing decision-level fusion approaches may increase through the presence of additional and potentially conflicting data sources, and can be mitigated through the use of intelligent fusion algorithms, for example, based on fuzzy logic, adaptive neuro-fuzzy inference, or Dempster–Shafer theory, as shown by Mikhail et al. [86] and Sun et al. [87].

One must consider, based on the specific goal and methodology for the given aircraft component, what the best level of fusion for their SHM subsystem case may be. Note that this may deviate between different types of components in the full aircraft structure. Such considerations imply the need for optimization studies that consider the objective or methodological performance to assist in the appropriate selection. In a similar manner as for feature selection and feature extraction, the inclusion of multi-sensor data fusion induces additional complexity and the use of data-driven artificial intelligence techniques may lead to increased effectiveness, simplicity, and computational efficiency.

Data Properties and Conflicting Data

Another aspect of fusing data is that the data from different sensors might be of different quality. For instance, one technique can provide measurements with higher confidence, noise levels might be different, or measurement data might be erroneous. These aspects need to be considered when fusing data because they may affect the diagnostic and prognostic results. Moreover, one might consider the measurements of one sensor to be of higher value, requiring the need for a weight-based fusion method in which measurements of one sensor have a higher influence on the final damage assessment results and consequently the decisions made in the aircraft health management system. The importance of one sensor over the other for a given diagnostic or prognostic function might not always be evident, and the employment of artificial intelligence approaches might provide clarification. For example, it may be observed that value changes in one sensor data feature have a large effect on the result, while other feature data or data coming from other sensors have minimal influence on the outcome of the learning model, and as such are closely related to the concepts of feature selection.

One might also encounter conflicting data or information: a strategy must be designed to approach such cases. For example, data from a sensor might indicate the presence of impact damage in the aircraft nose, while data from a sensor used in a competitive fashion detect no such impact damage in the same aircraft noise region; or data from one sensing technique indicate the presence of damage in a certain region, while data from another type of sensing technique do not. In the design of the aircraft structure’s SHM system, the handling of such cases must be managed by assessing sensor reliability, sensor specifications, uncertainty, and error mitigation processes. In these aspects, the consideration of probabilistic diagnostic and prognostic methods is essential [88].

5. SHM Framework Design for Composite Aircraft Structures

In this section, we aim to showcase what a fusion-based SHM framework might look like for aircraft structural component monitoring by proposing a first framework towards a full-level diagnostic and prognostic damage assessment. We develop a conceptual SHM system for composite aircraft structures assessing all levels using multi-sensor data fusion techniques. In this framework, we employ multiple SHM techniques in a fusion setting, in which each technique is used to its fullest capacity. Note that the proposed framework is discussed on a high level in a conceptual manner, and merely serves to demonstrate the first indications of the enhanced capabilities of a multi-sensor-based SHM system. To fully unleash its potential, in-depth research studies are crucial that consist of experimental campaigns, detailed diagnostic and prognostic methodology development, and validation exercises [8,36,50,72,89,90,91,92,93,94,95,96,97,98], which are beyond the scope of this work.

The case structure of consideration is a multi-stiffener composite panel: a generic representative aircraft wing structure in which skin–stiffener debonding and impact damages are critical damage cases similar to those seen in-service. Next, we first discuss some of the requirements on the SHM systems that stem from the in-service application and embodiment of such a component in the aircraft structure. In Section 5.2, we present the SHM framework and discuss the fusion of selected SHM techniques to allow for a holistic damage assessment that can be feasible for incorporation in a health management system for a full aircraft structure.

5.1. Case Study

In this subsection, we present the case study that will be employed to demonstrate the multi-sensor data fusion SHM framework for composite aircraft structures in the next subsection. The case structure under consideration is a multi-stiffener composite panel, which is a generic representation of a realistic composite aircraft wing structure (of which the design is derived from a wing structure by Embraer) and is shown on the left of Figure 3. It was selected for this work as an example, as it will serve in future works as the main case study for the implementation of such a fusion-based framework in a laboratory setting [8,98]. As part of the full aircraft structure during operation, these types of structures experience fatigue loads and changes in environmental conditions including humidity and temperature changes. For these types of composite wing components, critical in-service damage cases are skin–stiffener debonding, as well as unexpected foreign-object impact damages such as tool drops and weather events [99].

From the case study, several requirements can be derived that have an impact on the design of the SHM framework, mainly stemming from the needs set by the maintenance engineer during operational conditions to enable decision-making on further maintenance actions. Firstly, there is the general requirement for obtaining a full-level damage assessment, which originates from the in-service aerospace application, as discussed in Section 2.2. This means that all four diagnostic levels must be assessed: (1) damage detection, (2) damage localization, (3) damage type identification, and (4) damage severity, as well as the prognostic level: RUL estimation. Secondly, there is a requirement to assess various damage types, namely, impact damage, disbonds, and fatigue degradation. Thirdly, there is a requirement to monitor the full component as fatigue degradation and unexpected foreign object impacts may occur at every location in the panel, given that the skin acts as an outside layer. On the other hand, the skin–stiffener disbond can only occur at the stiffener feet.

5.2. SHM Framework

For the considered case study of a composite wing structural component, we have designed a conceptual SHM framework, presented in Figure 4, that is capable of assessing all four diagnostic levels and prognostics. It employs sensor data from four different SHM techniques, namely, (1) vibration-based method, (2) AE, (3) GW, and (4) distributed strain sensing, which are also indicated in

Table 2. SHM techniques employed for damage state assessment of multi-stiffener composite aircraft wing structure.

SHM Technique	Sensor Type	Sensor Location	Diagnostic and Prognostic Purpose
Vibration	Optical fibers containing fiber Bragg gratings	Skin	Anomaly detection
Acoustic emission	Acoustic emission sensors	Skin	Global location, damage type identification, health indicator (HI) for severity and prognostics
Guided waves	Piezoelectric transducers	Skin and stiffener	Precise skin damage localization and sizing, HI for severity and prognostics
Distributed strain sensing	Optical fibers	Stiffener foot	Precise disbond localization and sizing, HI for severity and prognostics

, including their sensor types, locations, and purpose with respect to the damage assessment, as well as in Figure 3, in which the sensing techniques are linked to the to-be-expected in-service damage types. Combined, the four sensing techniques are capable of providing a complete image of the damage state of the considered structure as prescribed by the requirements of the aircraft structural health management system. Here, multi-sensor data fusion is the key: each technique provides different information that, when combined, leads to a complete damage assessment within the structure. The general steps to obtain a full image of the damage state, following the procedure depicted in Figure 4, are described next.

5.2.1. Step 1: Anomaly Detection

The first step in the SHM system is to detect anomalies using a vibration-based method. In Figure 4, this is shown on the top left of the framework. Such a first step is required to decide on the needs for further, more detailed, analysis: only if there are signs of damage, there is an interest in performing further inspection steps as part of the maintenance procedure. In our case, anomalies are detected using FBGs by subjecting the structure to vibrations. In a laboratory setting, such vibrations are artificially induced using an actuator. However, during service, the natural vibrations of the aircraft, for example, those caused when starting the engines, can be exploited to assess the full wing structure. The measurements during these vibrations are compared to initially made baseline measurements, and significant changes in the structural response can be an indication of the presence of damage. Due to the inherent nature of this technique, it will solely provide an indication, and for more details on the potentially present damage, additional SHM techniques are required, as discussed next.

5.2.2. Step 2: Global Damage Location

Only when an anomaly is detected using the vibration-based technique, it is necessary to consider in more detail the damage characteristics, including where the damage is located, the identification of the damage type, and how severe the damage is. On the other hand, if no damage is detected, no unnecessary further inspections are required. If damage is detected and a larger component is under consideration, such as in our case study, it is of importance to obtain a global indication of the damage location. Namely, a multitude of sensors are attached to the structure (e.g., multiple OF sensors for strain measurements and PZTs for GW assessment), and it is required to know which sensor measurements should be employed for further damage analysis to avoid unnecessary evaluations of undamaged regions and to minimize the size of the collected datasets. In a full-scale aircraft application, one might also consider this as first identifying the damaged (sub)component in the full aircraft structure before more in-depth investigation. In our framework, the damage localization is performed using AE measurements that can be used to provide a global location of the damage, thereby allowing for the selection of the appropriate sensor in the damaged region for follow-up analyses. This can be combined with an AE damage classification step in which the type of damage is identified being, in this case, either damage to the skin or skin–stiffener disbond, thereby obtaining knowledge on both diagnostic level 2 (localization) and level 3 (type identification). Subsequently, this information can be employed to select the appropriate next step in the SHM framework.

5.2.3. Step 3: Detailed Damage Assessment: Skin Damage and Disbond

In the next step, after identifying and approximately localizing the damage in the structure, the appropriate techniques and sensors can be selected for a close-up assessment. On the one hand, in case the damage is classified as being skin damage, the GW technique can be activated to perform a scan using guided ultrasonic waves and thereby detect, localize, and size the damage in the skin. On the other hand, in case the damage is classified as a skin–stiffener disbond, fiber optic strain measurements (OF sensors are located along the stiffener foot) are employed to more precisely localize the disbond and provide a size estimate. There is also a possibility of a combined case: namely, an impact occurring near one of the stiffener locations resulting in both skin damage and skin–stiffener disbond. In the latter case, a fusion of both techniques is required: where the OF sensors provide skin–stiffener disbond assessment, they will miss any damage present in the skin farther from the stiffener. The latter will require a GW assessment. Hence, for a complete impact damage assessment in such cases, a fusion of GW and OF data results is required for impact damage localization and sizing. As such, incorporating both GW and OF measurements in the SHM system allows for the collection of detailed characteristics of the propagating damage.

5.2.4. Step 4: Damage Severity

The penultimate step assesses diagnostic level 4 ‘damage severity’ in which the severity of the damage state of the structure is estimated and, thus, so is its influence on the structural integrity state of the given aircraft component. For this indication of the damage state in our case study, measurement data of the AE, GW, and OF sensors is fused on a feature level to form a new HI. Here, it is hypothesized that a feature-level fusion will lead to a new feature that is more sensitive to damage than a feature extracted from a single type measurement. As each technique assesses a different aspect of the structural damage state, a fusion will allow a full inclusion in the severity assessment. Furthermore, its fusion can result in greater confidence in our diagnostic severity assessment. The advantages of a feature-level fusion into a HI have already been discussed by Broer et al. [36], Eleftheroglou et al. [67], and Galanopoulos et al. [93].

5.2.5. Step 5: Prognostic RUL Estimates

After assessing all diagnostic levels and now having a full image of the damage state in the aircraft structure, a CBM approach requires the following indication: ‘if this is the current damage in the composite component and it is continued to be used during operation of the aircraft, what is its RUL?’ Note that here, RUL is provided with respect to a predetermined state: for example, this state can be a final failure, a given degradation state, or a preset damage size. As input to a prognostic model, HIs can be used, either similar to those used for damage severity or specifically designed for prognostics. For prognostics, the HIs have to fulfill three characteristics, namely, monotonicity, prognosability, and trendability [67]. Similar to the HI of level 4, we hypothesize that a fusion of all three SHM techniques (AE, GW, OF) will allow for an improved and more complete assessment of the health state with enhanced RUL estimates. The RUL estimate can then be used within the aircraft health management system for informed decisions on the need for any further maintenance actions such as repair or replacement, as well as cost-friendly and risk-free scheduling of such maintenance activities.

6. Conclusions

This work assists in the move towards CBM in the aircraft industry by presenting a discussion on how multi-sensor data fusion concepts can aid in the maturing of SHM for composite aircraft structures. Two current challenges needed for CBM implementation in the aircraft industry were considered: (1) upscaling to (sub)component composite aircraft structures and (2) assessing all damage diagnostic levels and prognostics in a holistic damage assessment. The concepts behind multi-sensor data fusion and its benefits for SHM of composite aircraft structures were analyzed and a conceptual case study into its application was presented. The following conclusions can be drawn:

• So far, the current challenges for CBM for aircraft structures have not yet been fully addressed, which hinders its progress towards in-service application. Many points of attention cannot always be assessed using current research approaches in the field of SHM for composite aircraft structures, ranging from a simple feature such as spatial coverage to dealing with new boundaries and assembly details in structural components, and ranging from the maintenance engineer’s needs to have a full damage assessment on each diagnostic level and their interconnection to the intrinsic capacities of each SHM sensing technique;
• Multi-sensor data fusion concepts can be beneficial in addressing challenges in maturing the field of SHM for composite aircraft structures towards CBM inclusion in the aircraft industry. Multi-sensor data fusion can aid in upscaling to more realistic composite aircraft structures by, amongst others, increasing spatial coverage and leading to multi-damage type assessment, and in obtaining a holistic damage assessment on all diagnostic levels and prognostics. Implementing fusion approaches also provides new opportunities, including the possibility of redundancy and diagnostic and prognostic performance increases. Including fusion concepts in an aircraft SHM system also comes with several points of attention, ranging from practical issues, such as multi-type optimal sensor placement and data dimensionality reduction, to methodological aspects, such as handling heterogeneous data and selecting appropriate fusion methodologies;
• The potential of a multi-sensor fusion-based approach towards composite aircraft structures was demonstrated using a conceptual case study on a generic representative composite aircraft wing structure. The capacity of each SHM technique can be used most optimally in a synergetic system consisting of multiple techniques, thereby achieving a holistic damage assessment on all diagnostic and prognostic levels. To fully distill the benefits and implementation of the multi-sensing framework, a detailed exercise including methodology development and experimental validation is essential and should be the topic of future research studies;
• Future research on developing methodologies for SHM with the aim of inclusion in the aircraft industry should strongly consider the final application. When moving to more realistic composite aircraft structures and operational flight environments, fusion concepts can be of assistance in this regard. No single fusion methodology can be recommended since its selection highly depends on the application characteristics, the considered component, the SHM techniques, and the user requirements; instead, a fusion-based SHM system for an aircraft structure must be designed for the problem at hand.

It is foreseen that applying fusion concepts in the field of SHM for composite aircraft structures will pave the way for its move towards a CBM implementation in the aircraft industry.