Next Article in Journal
Drones for Flood Monitoring, Mapping and Detection: A Bibliometric Review
Next Article in Special Issue
Research on Adaptive Prescribed Performance Control Method Based on Online Aerodynamics Identification
Previous Article in Journal
Internet of Drones: Improving Multipath TCP over WiFi with Federated Multi-Armed Bandits for Limitless Connectivity
Previous Article in Special Issue
UAV Fault Detection Methods, State-of-the-Art
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Deep Learning-Based Real Time Defect Detection for Optimization of Aircraft Manufacturing and Control Performance

by
Imran Shafi
1,
Muhammad Fawad Mazhar
2,
Anum Fatima
3,
Roberto Marcelo Alvarez
4,5,6,
Yini Miró
4,5,7,
Julio César Martínez Espinosa
4,7,8 and
Imran Ashraf
9,*
1
College of Electrical and Mechanical Engineering, National University of Sciences and Technology (NUST), Islamabad 44000, Pakistan
2
Institute of Space Technology (IST), Islamabad 44000, Pakistan
3
National Centre for Robotics and Automation (NCRA), National University of Science and Technology (NUST), Islamabad 44000, Pakistan
4
Higher Polytechnic School, Universidad Europea del Atlántico, Isabel Torres 21, 39011 Santander, Spain
5
Project Management, Universidad Internacional Iberoamericana, Arecibo, PR 00613, USA
6
Department of Project Management, Universidade Internacional do Cuanza, Cuito EN250, Bié, Angola
7
Department of Project Management, Universidad Internacional Iberoamericana, Campeche 24560, Mexico
8
Fundación Universitaria Internacional de Colombia, Bogotá 111311, Colombia
9
Department of Information and Communication Engineering, Yeungnam University, Gyeongsan 38541, Republic of Korea
*
Author to whom correspondence should be addressed.
Drones 2023, 7(1), 31; https://doi.org/10.3390/drones7010031
Submission received: 16 November 2022 / Revised: 10 December 2022 / Accepted: 16 December 2022 / Published: 1 January 2023

Abstract

:
Monitoring tool conditions and sub-assemblies before final integration is essential to reducing processing failures and improving production quality for manufacturing setups. This research study proposes a real-time deep learning-based framework for identifying faulty components due to malfunctioning at different manufacturing stages in the aerospace industry. It uses a convolutional neural network (CNN) to recognize and classify intermediate abnormal states in a single manufacturing process. The manufacturing process for aircraft factory products comprises different phases; analyzing the components after the integration is labor-intensive and time-consuming, which often puts the company’s stake at high risk. To overcome these challenges, the proposed AI-based system can perform inspection and defect detection and alleviate the probability of components’ needing to be re-manufacturing after being assembled. In addition, it analyses the impact value, i.e., rework delays and costs, of manufacturing processes using a statistical process control tool on real-time data for various manufactured components. Defects are detected and classified using the CNN and teachable machine in the single manufacturing process during the initial stage prior to assembling the components. The results show the significance of the proposed approach in improving operational cost management and reducing rework-induced delays. Ground tests are conducted to calculate the impact value followed by the air tests of the final assembled aircraft. The statistical results indicate a 52.88% and 34.32% reduction in time delays and total cost, respectively.

1. Introduction

With the rapid improvement of technology, meeting the increasing needs of advanced manufacturing industries such as aerospace has become increasingly complex. The failure of a key component can cause unforeseen damages, and even causes loss of property or personnel. Compared to other industries, the requirements for accuracy of parts is much higher in the aerospace industry. Based on the statistics, failure in mechanical components can result in the failure of aerospace components [1]. The aerospace industry operates under numerous regulatory requirements, driving additional challenges in the required manufacturing and improvement processes [2]. The application of technological tools in manufacturing assembly lines demands precise evaluation of processes concerning the time and cost required to optimize the overall manufacturing processes. Aircraft manufacturing processes are complex due to multiple functional groups working in tandem; therefore, well-defined interaction points need to be articulated in order to maintain effective process control. The areas within assembly lines can be modified to reduce waste and increase throughput. Optimizing the manufacturing processes can improve overall efficiency within the aerospace industry. Process optimization ensures high quality, high efficiency, low cost, environment-friendly targets, eliminates weeks of manual trials, and improves assembly operations, thereby allowing the aerospace industry to save considerable production and personnel resources [3,4]. Machine learning plays an essential role in automating manufacturing processes and ensuring a continuous improvement cycle in aerospace manufacturing, laying a foundation for future predictive maintenance technologies.
Industries have experienced a paradigm shift due to technological changes and novelties with the evolution of smart factories [5]. The prospect of artificial intelligence (AI) technology application in industries has recently gained increasing attention [6]. AI and machine learning (ML) have become progressively applicable in factory operations [7]. Traditional ML-based methods have lately shifted from research to industrial usage, gaining importance with the progressive digitalization of industries. Manufacturing factories see enormous potential in AI-driven systems and techniques to achieve optimization ranging from computer-aided design to manufacturing processes, planning, and control [7]. However, the decorative value and usefulness of the workpieces may become compromised due to snags in the manufacturing process and forging technology. Existing procedures employed for detecting workpiece defects depend on manual practices involving human resources, resulting in a high False Positive/ True Negative rate. In order to unravel the overhead deliberation issue, an AI-driven defect detection system is employed here to augment the correct detection rate of workpiece flaws, minimize waste work, improve quality, and reduce the cost of workpiece production [8].
Considerable research work on the implementation of AI-driven defect detection systems in smart manufacturing factories has been carried out in the context of bulk production [5,6,7,8,9]. However, there is a dearth of research work on aerospace manufacturing factories, as manufacturing factories do not engage in bulk production. In existing aerospace manufacturing factories, defect detection with respect to various components involves labor-intensive scrutiny, causing the achieved results to be inconsistent and subjective. Furthermore, due to manufacturing process-related issues and duplication of technology, workpieces in the workshop may develop defects, affecting the ornamental value and usefulness of the workpiece [8]. The cornerstone of a lean management philosophy is to reduce waste in the value chain in order to reduce total lead time, including all operations [10]. Production line inspectors classify defective products through visual observation or manual measurement methods for specific products. In order to reduce product wastage, interventions and modifications should be made to the manufacturing processes to eradicate equipment failures and reduce the rejection rate. As visual/manual examination significantly upsurges the cost of the manufacturing process and is subject to the high error rate of manual detection, failures are often recognized late, resulting in increased losses. Thus, it is essential to advance defect detection techniques of workpieces, increase production competence, and control quality accuracy.
The major function of an aircraft wing assembly is to transmit and resist aerodynamic forces. It constitutes slender shell-bonded structures supported by longitudinal stiffening members and transverse frames that allow it to resist bending and both compressive and torsional loads. Irrespective of this robust assembly, undesired vibration are generated because of aerodynamic effects due to structural mismatches as well as to stability and balance issues. Structural mismatches due to defective wing assembly can enhance these vibrations. Therefore, one of the most critical parts for vibration and stability is the wing assembly [11]. Improving the aircraft manufacturing process of critical parts such as the wing assembly leads directly to improved stability in flight and achievement of the desired aircraft controllability.
In order to achieve time and cost optimization in the relevant manufacturing processes, the proposed AI-based defect detection framework is able to identify those processes that cause maximum rework delays and optimize processes within the aerospace industry. This research employs a three-step interdisciplinary approach involving AI technology and manufacturing processes, and makes the following contributions:
  • We utilize statistical process control techniques to identify the processes with the highest work waste rate within aerospace manufacturing factorys;
  • We develop an AI-based defect detection algorithm for inspection of images acquired from 150 commercial aircraft from a local aircraft manufacturing factory;
  • We calculate the impact of the proposed AI-driven defect detection algorithm on manufacturing processing optimization.
The rest of this paper is organized as follows. First, we discuss related works in Section 2. In Section 3, the proposed framework is introduced, and we discuss the methodology and implementation. Next, the obtained results are reported in Section 4. Finally, the proposed research is concluded and future work is presented in Section 5.

2. Related Work

The competitiveness to gain enhanced optimization in the modern industrial world has created a push towards large-scale manufacturing, perfection in manufacturing processes, and minimum waste to maximize throughput. A tool condition monitoring method based on CNN has been proposed by Dai et al. in [1] for tool state monitoring under specific working conditions to reduce defects in parts during the manufacturing stage. However, different models need to be trained in order for the proposed method to work under different conditions. Helo and Hao [6] provide an overview of the stage-wise development of models involving collecting and, preparing data and then training models, thereby exploring AI’s impact on operation management. Thumati et al. [12] proposed an implementation scheme utilizing AI-based techniques and big data analytics to reduce human dependability and improve quality control for cost optimization within the manufacturing environment. Another implementation scheme for refining military manufacturing processes by Martinez [13] incorporated a seam validation process with machine learning technology to reduce production costs.
Different data-driven methods have been introduced to monitor the parts under normal and fault conditions. Fault detection indicators monitor faults; several studies have been presented based on statistical process control. This method helps with monitoring and control, and ensures that processes operate efficiently to produce products that conform to specifications and require less rework [14,15]. AI methods have greatly helped in carrying out different tasks, for instance, face recognition, using different sensors [16]. Such methods have specifically contributed to biomedical image processing [17]. Similarly, cell segmentation and tracking can be aided by machine and deep learning models due to their superior performance compared to traditional methods [18]. AI for non-destructive testing (NDT) has improved test reliability management within aerospace setups. Such AI-based methods lead to automatic and objective interpretation of data. However, defect detection accuracy relies heavily on the overall quality of the testing process. For human-independent and objective NDT results, a data-centric approach has been proposed by D’Angelo and Palmieri [19] for reliable defect detection and mimicking of the skills and expertise of human inspectors.
A huge breakthrough in recent years has come in the form of deep learning algorithms. Such techniques are motivated by the capabilities of making more efficient use of both computing resources and time. The common goals of the proposed methods are image enhancement, detection, and classification of the different flaws in machinery parts during the manufacturing stage [20]. Several studies have utilized deep learning-based techniques to identify defects. Mery and Arteta [21] summarize the early automated classification methods involving support vector machine classifiers and other machine learning methods. Huang et al. [22] proposed a smart factory architecture involving manufacturing, data management, and yield analysis, utilizing a recurrent neural network to detect machining defects within steel semi-finished products during the manufacturing process. Dong et al. [23] proposed an automated industrial inspection method based on an unsupervised local deep feature learning method using non-annotated image data for defect detection and classification. Their proposed method allowed for real-time quality evaluation and monitoring. To reduce the time spent on defect inspection, Dogru et al. [24] proposed a hybrid approach combining a pre-defined classifier with a MASK recurrent CNN to improve prediction performance and automate the aircraft maintenance visual inspection process. Tang et al. [25] adopted a CNN for defect detection in composite laminates to improve inspection efficiency.
To broaden the engagement with ML-based tools and enable users without specialized technical knowledge to create their own ML modes, the Teachable Machine [26] has been introduced. This web-based interface allows the training of custom ML classification models without coding. This tool keeps the underlying complexities hidden, and helps to find trends and patterns within image samples, thereby enabling users to create functional and accurate models [27]. However, the learning parameters need to be adjusted in order to achieve optimal performance through from hyperparameter tuning. To evaluate the feasibility of the Teachable Machine web-based AI tool, Jeong [28] investigated the effects of different hyperparameters, i.e., the learning rate, batch size, and learning frequency, on the diagnostic accuracy of a tooth-marked tongue. Their achieved results were better than other AI-based models, confirming the possibility of using Teachable Machine in real-life scenarios. Nesakumar [29] presented a model utilizing a teachable solution and simple interface to detect and recognize different classes of diseases from images of plants. It was found that the teachable solution could provide reduced cost and help to attain greater yields than traditional convolutional networks. Based on the tests carried out by Agustian [30], the Google Teachable Machine can create a machine learning model with up to 100% accuracy, precision value, and sensitivity. The study implanted a machine learning model in an Android smartphone using Teachable Machine. It was found that the tool can be programmed according to the dataset being used. In [31], the authors performed vibration control analysis for aircraft wings with the use of a smart material. Undesired vibrations were eliminated by active vibration control to obtain stability. An embedded piezoelectric sensor and actuator were used, obtaining a fast response.
In most of the studies mentioned above, the prime focus remains on defect detection during the manufacturing processes; less work has been carried out on calculating the impact value of the manufacturing processes. Very few studies have considered manufacturing process optimization. This study proposes an AI-based defect detection system capable of detecting defects prior to the integration phase. This approach can ensure reduced rework delays and better time and cost production optimization. Furthermore, it focuses on improving manufacturing processes by introducing a step toward an advanced cyber–physical system and providing improved model theoretical performance value. A comparison is appended below (Table 1) comparing a number of existing studies with the proposed work.

3. Proposed Framework

The traditional aerospace framework consists of four major phases: design, manufacturing, integration/ assembling, and testing. A detailed discussion of manufacturing phases is appended below.

3.1. Design

The design phase starts with the mathematical modeling of a component (wing assembly and control components, etc.), then cross-validation is carried out using different software and testing equipment (i.e., wind tunnel testing). Subsequently, aerodynamic and structural integrity are checked using specialized finite element methods and computational fluid dynamics techniques. Different 3D diagrams of the process are shown in Figure 1.

3.2. Manufacturing

After completion and successful testing of a component, it is dispatched to the manufacturing bays for its physical transformation. Every component passes through a computer-based numerically control machining process according to its desired material specifications, and is manufactured as per the engineering drawing provided by the design section. Sheet metal work and machine work are extensively used in this process. GO/NO-GO gauges and tools are mostly used for quality control of the components.

3.3. Assembling/Integration

Here, sub-components are matched and integrated into a larger assembly. Problems may occur in this stage when assembling two mismatched components, mainly the wing assembly, canopy, and control surfaces with the fuselage. Due to mismatches, faults may occur in the components, and faulty components must be sent back to the re-manufacturing bay. As a result, rework is performed, causing additional time delays and increasing costs. To avoid this rework delay, the proposed framework incorporates an AI fault detection system in the assembly/integration phase before the parts are assembled. This helps to achieve near-optimized values, and provides the capability of identifying component faults before the integration/assembly phase.

3.4. Tests

After assembly, there are two tests: first, ground tests of all sub-systems are carried out, followed by air tests, including checks of the controls, wing performance, engines and avionics.

3.5. Use of AI Tool in Integration Phase

In order to reduce the problem of mismatches, an AI tool was introduced to the integration phase; the AI tool minimizes errors in the assembly phase and enhances aircraft performance. This AI tool can detect faults in the integration phase and enables the manufacturing factory to optimize its processes at an early timeframe, and is able to reduce the number of test flights required before accepting the aircraft. Without this AI tool, the same defects surfaced repeatedly during air tests. The proposed optimized manufacturing model is shown in Figure 2.

3.6. Methodology and Implementation

An in-depth interdisciplinary literature review involving AI implementation in manufacturing processes alongside a statistical analysis of aerospace manufacturing processes was synthesized to determine its impact on process optimization. This work aims to calculate the time delays and costs required to identify defects during the testing phase, and proposes an AI-based defect detection system to optimize both cost and time during the initial stages before integration. The proposed approach is divided into different phases, as shown in Figure 3.

3.6.1. Data Acquisition

The data are mainly acquired from different sources, including manufacturing bays, statistical process control, planning, and production. The manufacturing time for each aircraft is acquired to identify the sub-assemblies with maximum rework delays during phase II of the proposed framework, as shown in Figure 3. The manufacturing time is recorded and processed for analysis. For example, the normal distribution of the manufacturing time of parts during the manufacturing phase is shown in Figure 4.
For defect identification, images of correct and faulty wing tips are acquired. A sample of 150 small commercial low-speed aircraft from an understudy aerospace manufacturing factory was chosen, and their wing tip images were acquired. Due to faulty wing tips, aircraft can experience stability issues (vibrations, etc.), which need to be addressed during manufacturing. A high-definition camera was installed at assembly lines to take overhead photos of corrected and faulty assemblies for further preprocessing. The captured images were visually inspected for consistency, and 200 good-quality images were selected from each class to develop the labeled dataset. A few sample defects within the wing assembly are depicted in Figure 5. Structural analysis and stability analysis are provided in Figure 6 and Figure 7, respectively.

3.6.2. Calculation of Induced Rework Time Delay

Due to mismatch in the assembly of parts during the integration/assembly phase, faults may occur within the components. These faulty components must be sent back for re-manufacturing. This study aims to determine the rework induced by the design changes and measure the resulting impact on the cost and time required to complete the manufacturing process [33]. Defects add additional rework, making the inspection process more time-consuming and expensive. Furthermore, induced rework and delays significantly impact schedule predictability within manufacturing industries [34]. In this study, induced rework delays within different sections, i.e., wing section, control surfaces, and canopy section are calculated before further analyzing the data using the statistical process control tool.
The rework delays of the aircraft are calculated through a normal distribution for the calculated mean and standard deviation, as shown in Table 2. Furthermore, the induced delays are depicted in Figure 8.

3.6.3. Data Analysis of Manufacturing Processes: Statistical Process Control Analysis

Statistical process control practices help to improve manufacturing performances. This aids in determining the frequencies and time duration of breakdowns. Furthermore, it helps with investigation of the significant causes of breakdowns that affect productivity. In the present research, the statistical process control tool is used to identify the major loss time within the aerospace industry [35]. The quantitative data obtained is further analyzed using different statistical quality control techniques, including plotting data on normal distribution curves and pie charts. The means ( μ ) and standard deviations ( σ ) of manufacturing time and cost are calculated, and analysis is carried out to identify those sub-process with maximum delays and costs. Normal curves are plotted, and the curves are analyzed for the shift of mean and standard deviation values.
Figure 9 shows the induced delays in three sections, i.e., the wing section, control surfaces, and canopy section. It can be observed that wing assembly integration has the highest rework delays. Furthermore, the wing section has the maximum man-hours consumed compared to the other two sections. Table 3 shows the mean and standard deviation calculated in terms of the impact factor for man-hours consumed.

3.7. AI-Based Defect Detection System

Manufacturing processes need to be optimized in order to ensure that the correct parts are assembled during the integration phase to reduce the induced rework delays. The idea is to detect faults before the integration phase automatically without any manual intervention. Here, all three sections are analyzed for defect classification purposes, and images from the section with maximum induced rework delay and consumed man-hours are considered. The aim of this study is to employ and assess the feasibility of a machine learning model extracted from a Teachable Machine. For classification purposes, a machine learning network was built using the Teachable Machine open intelligence platform. This network employs a pre-trained MobileNet model, and transfer learning is carried out by modifying the final layers of the model with the custom dataset. The test platform contains a high-definition camera that takes images of assembly line aircraft components and sends those images to the process control system. The process control system (pre-trained AI model) detects faults and provides results within a few seconds, specifying whether the component meets design requirements or otherwise. A 28-layer CNN-based MobileNet model is used by the Teachable Machine, using depth-wise separable convolution to build a lightweight neural network. The proposed AI defect algorithm is depicted in Figure 10.

3.7.1. Google Teachable Machine

This research uses a web-based artificial intelligence development tool called Google Teachable Machine for defect detection in images [26,28]. This tool supports the development of Python-based code and uploading of training data. Multiple classifications can be performed by adding classes. To gain maximum accuracy, hyperparameters such as epoch, batch size, and learning rate need to be adjusted according to the utilized data. The detection accuracy of the model is evaluated using the unseen test data. The environment setting is carried out by deploying different libraries, such as Jupyter, Panda, Keras, Scikit learns, and Numpy.

3.7.2. Dataset of Images

A total of 1550 images of aircraft wing tip assemblies were taken using a high-definition camera installed in the assembling/integration bay of an aircraft manufacturing factory. An image dataset of faulty and serviceable wings was formulated. Bad-quality images were excluded to improve the accuracy of the results. These images were stored in designated libraries through the sci-kit image library. A few images from the dataset are shown in Figure 11.

3.7.3. Image Preprocessing

Cleaning, balancing, transforming, edge detection, and splitting were performed for pre-processing to eliminate unwanted regions and extract the region of interest. After the preprocessing stage, the dataset was trained on a Teachable Machine-based image classifier. Data were randomly split into 0.8, 0.1, and 0.1 ratios for training, testing, and validation, respectively, to evaluate the accuracy of the proposed approach. Figure 8 shows the development phases of the proposed AI defect detection tool. After the model was trained, the test images were used to test the performance of the model. For pre-processing of missing values, images were further filtered, with good quality images were retained and low-quality images discarded. The dataset was further separately categorized into faulty and correct images for training and testing purposes. Lastly, images were featured and scaled through the maximum absolute scaling technique. This finalized dataset was used for training and testing.

3.7.4. Image Labeling

The dataset of images was labeled as Correct and Defective based on the original design specifications available from the factory. The same images formed part of the dataset for further pre-processing, including the edge detection and cleaning process.

3.7.5. Model Training

After pre-processing of images, the model was trained through the MobileNet pre-training model, and further transfer learning was carried out by modifying the final layers. A 28-layer CNN-based MobileNet model was used by the teachable machine, with depth-wise separable convolution used to build a lightweight neural network. The process of model training is shown in Figure 12.
Figure 13 shows the development phases of the proposed AI defect detection tool.

3.7.6. Hyperparameter Tuning

The characteristics of the image data depend on the nature of the acquisition device, environment, and application field. Therefore, in order to gain increased learning accuracy, the variables need to be adjusted based on the characteristics of the obtained image data [28]. The Teachable Machine comprises three learning parameters, i.e., batch size, learning rate, and epochs. Initially, the batch size was set to 16, the number of epochs to 25, and the learning rate to 0.01. To further improve the performance of the model, the hyperparameters were fine-tuned for this research. The learning rate was optimized and set to 0.001 to improve the accuracy of the training data. Furthermore, the model was optimized, and the model was trained to predict the class of each pixel in the final layer using the cross-entropy loss function.

3.8. Impact Value Calculation

An AI-based system was developed to detect defects prior to the integration phase to ensure that components would not be sent back for re-manufacturing. Time and cost production optimization, which are considered key cardinals of operation management, were successfully achieved in one of the manufacturing factory sub-processes through the application of our AI-based defect detection tool. The impact value of time and cost optimization on the wing assembly section before and after the AI application was calculated by comparing the before and after values of the time and cost factors. The objective of the research effort was to optimize manufacturing processes (time and cost), thereby improving manufacturing processes in the aerospace factory through the improvements realized by the proposing model. In addition, a theoretical performance analysis was carried out to analyze the improved model’s efficacy and output. The performance analysis of the improved model was conducted considering the impact value of AI tools used in the manufacturing process. Salient features of performance analysis include:
i.
The impact of the AI tool in reducing rework delays;
ii.
The impact of the AI tool on reducing costs due to re-work;
iii.
Improvement in manufacturing processes by introducing a step towards an advanced cyber–physical system in the factory;
iv.
The introduction of intelligent production management in the factory.

3.9. Performance Analysis

After obtaining optimized results regarding the time and cost factors of a single process, a proposed model was recommended for adoption by an understudy manufacturing factory that could be customized and enhanced over time. The AI detection tool was implemented theoretically on the old model and the results were analyzed. Based on the achieved results, it is proposed that if the AI tool were implemented on other sub-sections, i.e., the control surfaces section and canopy section, then near-optimized time and cost values for manufacturing processes in these sections could be achieved as well.

4. Results and Discussions

This section presents a performance summary of the proposed AI-based defect detection algorithm implemented using a Teachable Machine image classifier. The proposed AI model positively impacts the operation time management of aircraft factories. The validation results are shown in Table 4.
For aircraft control components and wing tip diagnosis, a dataset of 1550 images was placed in the library. Subsequently, 80% of the images were used for the training dataset and the remaining 20% for the test dataset. To optimize the machine learning parameters, we measured the diagnosis accuracies according to the value of the epoch, batch size, and learning rate. After hyper-parameter tuning, the ROC (receiver operating characteristic) analysis method was used to determine the sensitivity (true positive rate, TPR) and specificity (false positive rate, FPR) of the machine learning model in order to diagnose the defects in aircraft control components. Initially, the batch size was set to 16, the number of epochs to 25, and the learning rate to 0.01. To further improve the performance of the model, the hyperparameters were fine-tuned. The learning rate was optimized and set to 0.001 to improve the accuracy of the training data. Furthermore, the model was optimized and trained to predict the class of each pixel in the final layer using the cross-entropy loss function. Figure 14 shows results regarding accuracy and epochs, batch size and learning rate. The accuracies for the no-fault wing tip and for the tooth-marked tongue/no-marked tongue were 72.5% and 70%, respectively.
Validation was performed using images representing the classes used for network development. Although the AI tool in the improved model was studied for implementation in the wing section, the theoretical impact value dictates that an improved model should improve the optimization of other sections of an aerospace manufacturing factory as well. The cost of AI equipment, human resource (HR) training, and space management aspects have not been considered here, as they are beyond the scope of this research. While remaining within the scope of the present research, the improved model should be able to improve the operational management of smart manufacturing factories.
The impact value comparison in terms of costs and delays is shown in Table 5.
The induced delays improve significantly after implementing the proposed AI defect detection model on the wing section, as shown in Figure 15a. The wing cost due to rework delays is reduced, as shown in Figure 15b. These results indicate that the proposed model has a positive impact on improving the operation cost management of the aircraft factory.
The induced delay before applying the AI defect detection model was 13.01 days, which improved significantly to 6.13 days. Furthermore, the wing cost before applying the AI model was $0.67m which was reduced to $0.23, indicating that the model can improve performance during the manufacturing process.
It can be seen from the plots in Figure 15 that the standard deviation and mean values were improved significantly. Table 5 shows that the induced work delays for all three sections improved. The impact value after implementing the AI defect detection model is depicted in Figure 16.
Dataset analysis and factory survey of rework delays revealed that minor defects caused a delay of less than 7–8 days, wherein defects were repaired/reworked within the same section during the integration phase. However, significant defects caused a delay of more than 7 days and up to 20 days, as the wing assembly needed to be sent back to the manufacturing bay for major rework or re-manufacturing of the complete wing. As the AI tool in our case has been designed/trained to detect major defects, it can enable the factory to detect major defects before the integration phase. In this way, rework delays that recorded after AI tool implementation were within a range of 0–7 days per rework. The same conjecture can be applied to other components where the AI tool could theoretically be applied, with the AI tool impacting a section to ensure that no major rework items are sent back to the manufacturing bay through detection prior to the integration phase.

5. Conclusions

Applying AI technology in manufacturing process optimization is a fledgling area of development. The present research aims to provide a customized approach capable of manufacturing high-quality products that meet customers’ demands. The proposed approach involves using the statistical process control tool for analyzing manufacturing processes and selecting the single manufacturing process that causes the maximum time delays and maximum costs due to defects. After identifying the process, an AI-based defect detection algorithm is applied to identify the defects before the integration/assembly phase. The results show that applying the AI defect detection model helps in manufacturing process optimization. The results before and after the proposed model are compared, with a 52.88% reduction in time and a 34.32% reduction in costs. To add value to the factory’s competitiveness, it is essential to transform its progressions and administrative edifice into AI-driven tools and innovative production management tools. This study aims to help transform the factory into an intelligent factory that is ultimately aimed at optimizing operation management cardinals.
In a continuation of existing research, future research work is envisaged to further enhance the existing model to achieve optimized operation management goals by bringing AI technology into the design phase of manufacturing. In this way, the manufacturing cycle defect rate can be brought to near zero, with the probability of the defect rate mitigated to the lowest level.

Author Contributions

Conceptualization, I.S.; Data curation, Y.M.; Formal analysis, I.S., M.F.M. and Y.M.; Funding acquisition, R.M.A.; Investigation, I.S. and A.F.; Methodology, M.F.M.; Project administration, R.M.A.; Resources, R.M.A.; Software, A.F., Y.M. and J.C.M.E.; Supervision, I.A.; Validation, J.C.M.E. and I.A.; Visualization, J.C.M.E.; Writing—original draft, M.F.M. and A.F.; Writing—review and editing, I.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the European University of the Atlantic.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The dataset and code is available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflict of interests.

References

  1. Dai, W.; Liang, K.; Wang, B. State Monitoring Method for Tool Wear in Aerospace Manufacturing Processes Based on a Convolutional Neural Network (CNN). Aerospace 2021, 8, 335. [Google Scholar] [CrossRef]
  2. Abollado, J.R.; Shehab, E. A systems approach for the definition of lean workflows in global aerospace manufacturing companies. Procedia CIRP 2018, 70, 446–450. [Google Scholar] [CrossRef]
  3. Xiong, D.D.; Yang, H.C.; Wan, N. Research on aerospace equipment machining process optimization based on MBD procedure model. In Advanced Materials Research; Trans Tech Publ.: Zurich, Switzerland, 2012; Volume 538, pp. 2772–2775. [Google Scholar]
  4. Rishardson, M. Optimisation Makes All the Difference. 2012. Available online: https://www.aero-mag.com/optimisation-makes-all-the-difference/ (accessed on 2 March 2022).
  5. Shamim, S.; Cang, S.; Yu, H.; Li, Y. Management approaches for Industry 4.0: A human resource management perspective. In Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC, Canada, 24–29 July 2016; pp. 5309–5316. [Google Scholar]
  6. Helo, P.; Hao, Y. Artificial intelligence in operations management and supply chain management: An exploratory case study. Prod. Plan. Control 2021, 33, 1573–1590. [Google Scholar] [CrossRef]
  7. Fahle, S.; Prinz, C.; Kuhlenkötter, B. Systematic review on machine learning (ML) methods for manufacturing processes–Identifying artificial intelligence (AI) methods for field application. Procedia CIRP 2020, 93, 413–418. [Google Scholar] [CrossRef]
  8. Ye, L.; Xia, X.; Chai, B.; Wang, S.; Yang, B. Application of deep learning in workpiece defect detection. Procedia Comput. Sci. 2021, 183, 267–273. [Google Scholar] [CrossRef]
  9. Russell, R.S.; Taylor, B.W. Operations and Supply Chain Management; John Wiley & Sons: Hoboken, NJ, USA, 2019. [Google Scholar]
  10. Ejsmont, K.; Gladysz, B.; Corti, D.; Castaño, F.; Mohammed, W.M.; Martinez Lastra, J.L. Towards ‘Lean Industry 4.0’—Current trends and future perspectives. Cogent Bus. Manag. 2020, 7, 1781995. [Google Scholar] [CrossRef]
  11. Abdulameer, H.A.; Wasmi, H.R. Vibration control analysis of aircraft wing by using smart material. Innov. Syst. Des. Eng. 2015, 6, 7–42. [Google Scholar]
  12. Thumati, B.T.; Subramania, H.S.; Shastri, R.; Kumar, K.K.; Hessner, N.; Villa, V.; Page, A.; Followell, D. Large-scale Data Integration for Facilities Analytics: Challenges and Opportunities. In Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 10–13 December 2020; pp. 3532–3538. [Google Scholar]
  13. Martinez, J.; MacAllister, A.; Dominguez, E.; Mahlmann, C.; Quinlan, G.; McKee, M. Applying Neural Networks to the F-35 Seam Validation Process. In Proceedings of the 2021 IEEE Aerospace Conference (50100), Big Sky, MT, USA, 6–13 March 2021; pp. 1–11. [Google Scholar]
  14. Fan, W.; Xue, H.; Yi, C.; Xu, Z. TQWT-assisted statistical process control method for condition monitoring and fault diagnosis of bearings in high-speed rail. Proc. Inst. Mech. Eng. Part J. Risk Reliab. 2021, 235, 230–240. [Google Scholar] [CrossRef]
  15. Aykroyd, R.G.; Leiva, V.; Ruggeri, F. Recent developments of control charts, identification of big data sources and future trends of current research. Technol. Forecast. Soc. Chang. 2019, 144, 221–232. [Google Scholar] [CrossRef]
  16. Jin, B.; Cruz, L.; Gonçalves, N. Pseudo RGB-D Face Recognition. IEEE Sens. J. 2022, 22, 21780–21794. [Google Scholar] [CrossRef]
  17. Yao, T.; Qu, C.; Liu, Q.; Deng, R.; Tian, Y.; Xu, J.; Jha, A.; Bao, S.; Zhao, M.; Fogo, A.B.; et al. Compound figure separation of biomedical images with side loss. In Deep Generative Models, and Data Augmentation, Labelling, and Imperfections; Springer: Cham, Switzerland, 2021; pp. 173–183. [Google Scholar]
  18. Zhao, M.; Jha, A.; Liu, Q.; Millis, B.A.; Mahadevan-Jansen, A.; Lu, L.; Landman, B.A.; Tyska, M.J.; Huo, Y. Faster Mean-shift: GPU-accelerated clustering for cosine embedding-based cell segmentation and tracking. Med. Image Anal. 2021, 71, 102048. [Google Scholar] [CrossRef] [PubMed]
  19. D’Angelo, G.; Palmieri, F. Knowledge elicitation based on genetic programming for non destructive testing of critical aerospace systems. Future Gener. Comput. Syst. 2020, 102, 633–642. [Google Scholar] [CrossRef]
  20. Tyystjärvi, T.; Virkkunen, I.; Fridolf, P.; Rosell, A.; Barsoum, Z. Automated defect detection in digital radiography of aerospace welds using deep learning. Weld. World 2022, 66, 643–671. [Google Scholar] [CrossRef]
  21. Mery, D.; Arteta, C. Automatic defect recognition in x-ray testing using computer vision. In Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA, 24–31 March 2017; pp. 1026–1035. [Google Scholar]
  22. Huang, D.C.; Lin, C.F.; Chen, C.Y.; Sze, J.R. The Internet technology for defect detection system with deep learning method in smart factory. In Proceedings of the 2018 4th International Conference on Information Management (ICIM), Oxford, UK, 25–27 May 2018; pp. 98–102. [Google Scholar]
  23. Dong, X.; Taylor, C.J.; Cootes, T.F. Automatic aerospace weld inspection using unsupervised local deep feature learning. Knowl.-Based Syst. 2021, 221, 106892. [Google Scholar] [CrossRef]
  24. Doğru, A.; Bouarfa, S.; Arizar, R.; Aydoğan, R. Using convolutional neural networks to automate aircraft maintenance visual inspection. Aerospace 2020, 7, 171. [Google Scholar] [CrossRef]
  25. Tang, Y.M.; Ip, A.W.; Li, W. Artificial intelligence approach for aerospace defect detection using single-shot multibox detector network in phased array ultrasonic. In IoT and Spacecraft Informatics; Elsevier: Amsterdam, The Netherlands, 2022; pp. 1–27. [Google Scholar]
  26. Carney, M.; Webster, B.; Alvarado, I.; Phillips, K.; Howell, N.; Griffith, J.; Jongejan, J.; Pitaru, A.; Chen, A. Teachable machine: Approachable Web-based tool for exploring machine learning classification. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–8. [Google Scholar]
  27. Phillips, K. Teachable Machine 2.0 Makes AI Easier for Everyone. 2019. Available online: https://blog.google/technology/ai/teachable-machine/ (accessed on 24 May 2022).
  28. Jeong, H. Feasibility Study of Google’s Teachable Machine in Diagnosis of Tooth-Marked Tongue. J. Dent. Hyg. Sci. 2020, 20, 206–212. [Google Scholar]
  29. Nesakumar, C.D. Teachable Machine based Crop Health Monitoring System. Bull. Monum. 2021, 22, 20–33. [Google Scholar]
  30. Agustian, D.; Pertama, P.P.G.P.; Crisnapati, P.N.; Novayanti, P.D. Implementation of Machine Learning Using Google’s Teachable Machine Based on Android. In Proceedings of the 2021 3rd International Conference on Cybernetics and Intelligent System (ICORIS), Makasar, Indonesia, 25–26 October 2021; pp. 1–7. [Google Scholar]
  31. Zheng, Q.; Yang, M.; Yang, J.; Zhang, Q.; Zhang, X. Improvement of generalization ability of deep CNN via implicit regularization in two-stage training process. IEEE Access 2018, 6, 15844–15869. [Google Scholar] [CrossRef]
  32. Lourenço, E.J.; Oliva, M.; Estrela, M.; Baptista, A.J. Multidimensional Design Assessment Model for eco-efficiency and efficiency in aeronautical assembly processes. In Proceedings of the 2019 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), Valbonne Sophia-Antipolis, France, 17–19 June 2019; pp. 1–9. [Google Scholar]
  33. Yap, J.B.H.; Abdul-Rahman, H.; Chen, W. Collaborative model: Managing design changes with reusable project experiences through project learning and effective communication. Int. J. Proj. Manag. 2017, 35, 1253–1271. [Google Scholar] [CrossRef]
  34. Evoy, D. Measuring Construction Rework & Delays in Sustaining Capital Projects. 2021. Available online: https://becht.com/becht-blog/entry/measuring-construction-rework-delays-in-sustaining-capital-projects/ (accessed on 8 September 2022).
  35. Sultana, F.; Razive, N.I.; Azeem, A. Implementation of statistical process control (SPC) for manufacturing performance improvement. J. Mech. Eng. 2009, 40, 15–21. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Aircraft design phase, (a) wing design, and (b) aileron design.
Figure 1. Aircraft design phase, (a) wing design, and (b) aileron design.
Drones 07 00031 g001
Figure 2. Proposed framework for process optimization.
Figure 2. Proposed framework for process optimization.
Drones 07 00031 g002
Figure 3. Flow of the proposed framework.
Figure 3. Flow of the proposed framework.
Drones 07 00031 g003
Figure 4. Manufacturing time for parts during the manufacturing phase.
Figure 4. Manufacturing time for parts during the manufacturing phase.
Drones 07 00031 g004
Figure 5. Examples of wing assembly images obtained from commercial aircraft: (a) correct wing assembly (front), (b) correct wing assembly (wide), (c) faulty wing assembly (side angle), and (d) correct wing assembly (front).
Figure 5. Examples of wing assembly images obtained from commercial aircraft: (a) correct wing assembly (front), (b) correct wing assembly (wide), (c) faulty wing assembly (side angle), and (d) correct wing assembly (front).
Drones 07 00031 g005
Figure 6. Structural analysis of wing assembly due to defective wing tips: (a) front view, (b) isometric view, and (c) side view.
Figure 6. Structural analysis of wing assembly due to defective wing tips: (a) front view, (b) isometric view, and (c) side view.
Drones 07 00031 g006
Figure 7. Stability analysis of wing assembly due to defective wing tips: (a) front view, (b) isometric view, and (c) side view.
Figure 7. Stability analysis of wing assembly due to defective wing tips: (a) front view, (b) isometric view, and (c) side view.
Drones 07 00031 g007
Figure 8. Induced rework delays for different sections within the aerospace manufacturing process: (a) wing section, (b) control surfaces, and (c) canopy section.
Figure 8. Induced rework delays for different sections within the aerospace manufacturing process: (a) wing section, (b) control surfaces, and (c) canopy section.
Drones 07 00031 g008
Figure 9. Consumed man-hours for different sections within the aerospace manufacturing process: (a) man-hours consumed for wing section, (b) man-hours consumed for control surfaces, and (c) man-hours consumed for canopy section.
Figure 9. Consumed man-hours for different sections within the aerospace manufacturing process: (a) man-hours consumed for wing section, (b) man-hours consumed for control surfaces, and (c) man-hours consumed for canopy section.
Drones 07 00031 g009
Figure 10. The proposed framework algorithm.
Figure 10. The proposed framework algorithm.
Drones 07 00031 g010
Figure 11. Wing tip images from different angles.
Figure 11. Wing tip images from different angles.
Drones 07 00031 g011
Figure 12. Block diagram of Mobilenet training model.
Figure 12. Block diagram of Mobilenet training model.
Drones 07 00031 g012
Figure 13. Development phases of AI defect detection technique.
Figure 13. Development phases of AI defect detection technique.
Drones 07 00031 g013
Figure 14. Accuracy of results vs. epochs, batch size, and learning rate: (a) accuracy vs. epochs, (b) accuracy vs. batch size, and (c) accuracy v. learning rate.
Figure 14. Accuracy of results vs. epochs, batch size, and learning rate: (a) accuracy vs. epochs, (b) accuracy vs. batch size, and (c) accuracy v. learning rate.
Drones 07 00031 g014
Figure 15. Induced delays and wing costs before AI and after AI: (a) induced delays and (b) wing cost.
Figure 15. Induced delays and wing costs before AI and after AI: (a) induced delays and (b) wing cost.
Drones 07 00031 g015
Figure 16. Impact value comparison before and after implementation of the AI defect detection model before integration.
Figure 16. Impact value comparison before and after implementation of the AI defect detection model before integration.
Drones 07 00031 g016
Table 1. Comparison of the proposed work with recent research works.
Table 1. Comparison of the proposed work with recent research works.
Performance Analysis Matrix of the Proposed Model with Latest Strategies
Current aerospace manufacturing modelModel from [13]Manufacturing model in [12]Aeronautical Assembly Process (Horizontal Tail Plane Structure) [32]
AI (Deep Learning) Tool for optimization of aircraft manufacturing and enhanced controllability (through Wing tip assembly)Seam validation through artificial neural network aidImage classifier accuracy testing techniquesAI (machine learning) algorithms for energy optimization in manufacturing processesIncrease in operation time efficiency through multidimensional design assessment model
0.23 million $ saving achieved through rework evaluations/fault detectionRework evaluations/fault detection--
52.88% reduction in time delays of a single processAI (ANN) algorithms for rework production--
Improvement in performance/ controllability of aircraft achieved through optimized Wing assembly manufacturingImprovement in performance of aircraft through incorporation of AI-based technique in manufacturing processAerospace manufacturing equipment energy consumption and their optimized energy usage was calculated before and after AI (machine learning) tool implementationOperation time and cost values are calculated using software-based model before and after model implementation
Table 2. Calculated induced rework time delay.
Table 2. Calculated induced rework time delay.
Induced DelaysCalculated MeanCalculated Standard Deviation
Wing section13.024.98
Control surfaces10.885.66
Canopy section8.175.01
Table 3. Calculated man-hours consumed.
Table 3. Calculated man-hours consumed.
Consumed Man HoursCalculated MeanCalculated Standard Deviation
Wing Section102.2457.74
Control Surfaces42.3724.42
Canopy Section16.309.47
Table 4. Validation results obtained on the dataset.
Table 4. Validation results obtained on the dataset.
MeasureScoreDerivation
Accuracy0.7250 A c c u r a c y = T P + T N T P + F P + T N + F N
Precision0.7500 P r e c i s i o n = T P T P + F P
F1 score0.7317 F 1 = 2 × P r e c i s i o n × R e c a l l P r e c i s i o n + r e a l l
Table 5. Comparison of delays and costs before and after AI adoption.
Table 5. Comparison of delays and costs before and after AI adoption.
ProcessesBefore AIAfter AIImproved Model Theoretical Performance Value
Delays (Days)Cost ($)Delays (Days)Cost ($)
Wing13.010.676.130.44–52.88% reduction in time of a single process–34.32% reduction in cost
Canopy8.170.365.660.26–30.7% reduction in time of a single process–27.7% reduction in cost
Control Surfaces10.880.376.570.26–39.61% reduction in time of a single process–29.72% reduction in cost
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Shafi, I.; Mazhar, M.F.; Fatima, A.; Alvarez, R.M.; Miró, Y.; Espinosa, J.C.M.; Ashraf, I. Deep Learning-Based Real Time Defect Detection for Optimization of Aircraft Manufacturing and Control Performance. Drones 2023, 7, 31. https://doi.org/10.3390/drones7010031

AMA Style

Shafi I, Mazhar MF, Fatima A, Alvarez RM, Miró Y, Espinosa JCM, Ashraf I. Deep Learning-Based Real Time Defect Detection for Optimization of Aircraft Manufacturing and Control Performance. Drones. 2023; 7(1):31. https://doi.org/10.3390/drones7010031

Chicago/Turabian Style

Shafi, Imran, Muhammad Fawad Mazhar, Anum Fatima, Roberto Marcelo Alvarez, Yini Miró, Julio César Martínez Espinosa, and Imran Ashraf. 2023. "Deep Learning-Based Real Time Defect Detection for Optimization of Aircraft Manufacturing and Control Performance" Drones 7, no. 1: 31. https://doi.org/10.3390/drones7010031

APA Style

Shafi, I., Mazhar, M. F., Fatima, A., Alvarez, R. M., Miró, Y., Espinosa, J. C. M., & Ashraf, I. (2023). Deep Learning-Based Real Time Defect Detection for Optimization of Aircraft Manufacturing and Control Performance. Drones, 7(1), 31. https://doi.org/10.3390/drones7010031

Article Metrics

Back to TopTop