Next Article in Journal
HPMC/PVP K90 Dissolving Microneedles Fabricated from 3D-Printed Master Molds: Impact on Microneedle Morphology, Mechanical Strength, and Topical Dissolving Property
Next Article in Special Issue
Mechanical Properties and Performance of 3D-Printed Acrylonitrile Butadiene Styrene Reinforced with Carbon, Glass and Basalt Short Fibers
Previous Article in Journal
The Modification of Useful Injection-Molded Parts’ Properties Induced Using High-Energy Radiation
Previous Article in Special Issue
On Cyclic-Fatigue Crack Growth in Carbon-Fibre-Reinforced Epoxy–Polymer Composites
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Polymers
Volume 16
Issue 4
10.3390/polym16040451
polymers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Application of Neural Network Models with Ultra-Small Samples to Optimize the Ultrasonic Consolidation Parameters for ‘PEI Adherend/Prepreg (CF-PEI Fabric)/PEI Adherend’ Lap Joints

by
Dmitry Y. Stepanov
1,
Defang Tian
2,
Vladislav O. Alexenko
1,
Sergey V. Panin
1,2,* and
Dmitry G. Buslovich
3
1
Laboratory of Mechanics of Polymer Composite Materials, Institute of Strength Physics and Materials Science of Siberian Branch of Russian Academy of Sciences, 634055 Tomsk, Russia
2
Department of Materials Science, Engineering School of Advanced Manufacturing Technologies, National Research Tomsk Polytechnic University, 634050 Tomsk, Russia
3
Laboratory of Nanobioengineering, Institute of Strength Physics and Materials Science of Siberian Branch of 9 Russian Academy of Sciences, 634055 Tomsk, Russia
*
Author to whom correspondence should be addressed.
Polymers 2024, 16(4), 451; https://doi.org/10.3390/polym16040451
Submission received: 6 January 2024 / Revised: 29 January 2024 / Accepted: 2 February 2024 / Published: 6 February 2024
(This article belongs to the Special Issue Research Progress on Mechanical Behavior of Polymers)
Browse Figures
Versions Notes

Abstract

:
The aim of this study was to optimize the ultrasonic consolidation (USC) parameters for ‘PEI adherend/Prepreg (CF-PEI fabric)/PEI adherend’ lap joints. For this purpose, artificial neural network (ANN) simulation was carried out. Two ANNs were trained using an ultra-small data sample, which did not provide acceptable predictive accuracy for the applied simulation methods. To solve this issue, it was proposed to artificially increase the learning sample by including additional data synthesized according to the knowledge and experience of experts. As a result, a relationship between the USC parameters and the functional characteristics of the lap joints was determined. The results of ANN simulation were successfully verified; the developed USC procedures were able to form a laminate with an even regular structure characterized by a minimum number of discontinuities and minimal damage to the consolidated components.

1. Introduction

Manufacturing processes, as a rule, are controlled according to several technological parameters, the combined influence of which determines the resulting product quality. Values of these parameters can individually exert opposite (conflicting) effects on each other, so they have to be optimized [1,2]. Solving such problems can be considered as a ‘classical’ design of experiment (DoE), and has attracted considerable attention from many researchers [3]. An example is turning, when it is necessary to simultaneously take into account the spindle speed, the feed rate, and a number of other factors [4]. In additive manufacturing by the fused filament fabrication/fused deposition modeling (FFF/FDM) method, both extruder and bed temperatures, the head speed, the material feed rate, and some additional parameters have different effects on the structure and properties of the 3D-printed products [5].
Since conducting a complete multifactorial experiment is not always possible (or rational), various optimization methods may be applied, for instance, the Taguchi [6] or Box–Behnken [7] techniques. Recently, artificial neural networks (ANNs) have increasingly begun to be used for solving such problems, especially for approximation or classification [8,9]. ANNs are characterized by high efficiency when a large (experimental) data sample is available [10]. However, the reliability of the prediction decreases (or it cannot be considered reliable at all) with limited data sets [11,12,13]. At the same time, numerous ANNs have been developed to date, so their correct selection and learning is a challenge under such conditions [14].
Ultrasonic welding (USW) of laminated polymer composites has been implemented in many high-tech industries (primarily aerospace) [15,16]. To form reliable welds, numerous USW parameters have to be optimized [17,18], including ultrasonic (US) frequency, amplitude of sonotrode vibration, clamping pressure, USW duration, duration of clamping after USW, etc. These parameters are the input data for the USW process [19], while its efficiency can be controlled by one output factor, namely the USW joint thinning, taking into account the need to insert an energy director (ED) between joined plates (adherends) [20].
It should be noted that USW can be used not only for joining laminates for structural components, but for the fabrication of laminates as well [21,22,23,24,25,26,27]. In such cases, their structure is formed due to processes developing at several interfaces, with the unilateral input of mechanical energy converted into frictional heating [28]. Respectively, the number of output parameters increases, since some physical, mechanical, dimensional, and structural characteristics have to be considered [29,30,31]. Their complete assessment is a rather long procedure, also requiring thorough statistical justification. The solution to this problem fully correlates with another production route for the formation of prepregs or laminates via automatic fiber placement assisted with heating via a laser beam [32,33,34,35,36,37].
A previous paper by the authors [38] was devoted to the optimization of the US-consolidation (USC) parameters for the formation of USC lap joints of polyetheretherketone (PEEK) adherends, a carbon fiber (CF) fabric prepreg impregnated with polyetherimide (PEI), and two PEEK EDs [39]. Impregnation of the CF fabric with the polymer, characterized by a low melting point and a melt flow index (MFI) greater than that for PEEK, determined the specific development of the structure formation process. In particular, molten PEI was squeezed out of the prepreg during USC, damaging the reinforcing CF fabric. Based on this experience, only PEI was utilized in this study, so PEI adherends were joined using the CF fabric impregnated with a PEI-based solution as well. A film of the low-melting TECAPEI (PEI-based copolymer) was inserted as an ED in the development of a USC procedure that firstly enables it to melt, ensuring the formation of lap joints with minimum possible damage to the CF fabric-based prepreg [30,40]. Hereinafter, these USC lap joints are designated as the ‘PEI adherend/Prepreg (CF-PEI fabric)/PEI adherend’ samples.
The aim of this study was to optimize the USC parameters by ANN simulation, providing the required functional characteristics of lap joints with a minimum number of full-scale experiments. To achieve this goal, it was firstly necessary to simulate the USC process as a ‘black box’ with many inputs and outputs. Then, two ANNs were trained using an ultra-small sample, which did not provide acceptable predictive accuracy for the applied simulation methods. It was proposed to implement a well-known approach, which consisted in artificially increasing the learning sample by including additional data synthesized according to the knowledge and experience of experts [41].
This paper is structured as follows. Section 1 provides an overview of the implementation of ANNs for optimization purposes, and generally reveals the ideology of their use to solve the problems highlighted in this study. Section 2 presents the sequence of ANN simulation of the USC process with an analysis of both a priori and a posteriori knowledge, as well as the obtained results, while Section 3 is devoted to their verification. Section 4 discusses the prospects for application of the developed approach using an example of laminated composites formed by layer-by-layer USC processing of the prepregs based on the CF fabric impregnated with the PEI solution. All the above results are summarized in conclusions.
A brief overview of the application of machine learning methods using small experimental data samples. As the authors have shown previously [42], the issue of finding an optimal combination of USC parameters cannot be considered as an optimization problem, since the conditions for ensuring the optimality of the functional characteristics of such lap joints are represented by a system of inequalities. This formulation is determined not only by the inconsistency of the requirements (formulation of criteria) for their individual properties, but also by the specifics of the problem. Practical interest is not in the specific values of the USC parameters, but their ranges. So, a solution to this problem should be approached in two stages: (i) approximation of a vector quantity (characteristics of a lap joint) in the multidimensional space of the USC parameters, and (ii) search for such a range of values of the parameter vector within which all the inequalities of the optimality condition are satisfied. In the general case, such areas contain an infinite number of solutions, so calling all of them ‘optimal’ (in the classical sense) is incorrect. More precisely, they should be referred to as areas of ‘suboptimal’ parameters (SOPs), within which the optimal solution is located.
The first stage of approximation can be carried out using (i) linear interpolation algorithms based on triangulation, (ii) inverse distance weighting or polynomial methods, (iii) basis function approach, (iv) Kriging interpolation, (v) piecewise linear function, and (vi) component-wise splines [43,44,45,46,47,48]. Each of these methods has both advantages and drawbacks. Nevertheless, only ANN simulation is recommended for universal approximation of a vector quantity in a multidimensional space, taking into account the relationship of the vector components and the significant non-linearity of the observed patterns [49].
The second stage of searching and constructing SOP areas based on the results of ANN simulation can be carried out using well-known methods of cluster analysis and image processing. The issue of the implementation of one of these is outside the scope of this study since it deserves a separate investigation.
One of the key challenges in ANN simulation is the selection of the ANN type and architecture. For processing static data, feedforward neural networks (FFNNs), radial basis function networks (RBFNN)s, and their modifications are most often used [50,51]. The numbers of layers and neurons in them are determined not only by the complexity of an approximated dependence but also by the learning sample size [52]. For small samples, the complexity of the applied model is typically neglected. In the presence of a large number of factors, reducing the dimension of the problem is achieved by highlighting one or two of the most significant factors, while the rest are not used for simulation. In [53], it was proposed to compensate for the simplification of ANNs by including additional parameters to the significant factors obtained using known output/input relationships. The complexity of ANNs depends on the numbers of their layers and neurons, as well as the justified relationships. In some cases, ANNs are divided into two or more interrelated (but simple) types for clarity. A special approach to the development of ANNs that considers the known physical laws of simulated processes is described in [54,55]. In that case, the ANN architecture was designed according to those laws and includes a hybrid physical–statistical learning method that explicitly embeds the solution of partial differential equations into the loss function of the so-called physics-informed neuron networks (PINNs).
Great attention is paid to numerous techniques for training ANNs, such as (i) learning algorithms for solving direct problems (for example, based on the finite element method), (ii) genetic algorithms, (iii) support vector machines [51,56], etc.
ANNs are characterized by the so-called ‘curse of dimensionality’: as the dimensions of their input vectors rise, the complexity of ANNs increases exponentially [49]. As a result, a learning sample has to be enhanced. For example, the following procedures should be implemented in one of the most common classes of problems related to classification and image recognition [57,58]: (i) generation of surrogate data, (ii) interpolation of experimental data, (iii) algorithms for shifting, permutation, reflection, and rotation of data to achieve system invariance, or (iv) randomization procedures to increase noise immunity.
Due to both the high labor intensity and the cost of full-scale research in materials science, the number of experiments performed is typically negligible [12,59,60]. In these cases, the authors mean by the ‘ultra-small sample’ concept such experimental data arrays that are sufficient to draw a linear or quasi-linear relationship, but significantly less than are necessary to formulate an adequate non-linear one. For example, the number of experiments can vary from nine (for the Taguchi method) to twenty-seven (for the fractional full factorial design) when designing an investigation with three factors and their levels. In such cases, the results of hundreds of experiments are required to train ANNs, but thousands are required for deep learning. Therefore, ANN simulation is characterized by high errors in the approximation region for ultra-small samples, no matter what type of ANN and training method are implemented. This is especially true outside the range of experimentally determined values. Such a phenomenon is referred to by many researchers as ‘the poor ability of ANNs to solve extrapolation problems’ [61]. It would seem an obvious and correct conclusion that using ultra-small learning samples requires abandoning the implementation of ANNs; however, they may be suitable for solving numerous applied problems that do not require high accuracy of simulation results, i.e., when a quality solution is enough.
It should be noted that for ANN simulation of objects and processes that are not characterized by periodicity or high correlation of values in parameter spaces, the task of extrapolating the small experimental data samples cannot be solved correctly without additional information. Therefore, one of the possible ways to minimize errors in the approximation and extrapolation regions is to add a priori known data for the boundary, limit, or special parameter values (Figure 1). It is more correct to designate the latter a priori knowledge rather than data, because they are not obtained experimentally but are rather based on the experience of experts. A priori knowledge can be considered in a broad sense, since it includes both theoretical premises and previously comprehended knowledge about the simulated objects or processes. It is not always possible to formulate such knowledge, so the development of ANNs should be carried out in stages, assessing the adequacy of developed models and the accuracy of obtained results, as well as adding a posteriori knowledge, if necessary.

2. ANN Simulation with Ultra-Small Samples to Optimize the USC Parameters for the Formation of ‘PEI Adherend/Prepreg (CF-PEI Fabric)/PEI Adherend’ Lap Joints

2.1. A General Model of the USW Process

As mentioned above, conventional input (control) USC parameters include US frequency (ω), amplitude of sonotrode vibrations (θ), clamping pressure (P), USC duration (t), duration of clamping after USC (τ), etc. [38]. In this study, it was assumed that the values of the t, τ, and P parameters varied within specified ranges for optimizing the USC process, whereas the other two (ω, θ) were constant. This decision was based on the specifications of the deployed ‘UZPS-7’ ultrasonic welding machine (‘SpetsmashSonic’ LLC, Russia).
The following mechanical characteristics were used as the required properties of USC lap joints [38]:
(a)
tensile strength (σ), MPa;
(b)
elongation at break (ε), %;
as well as the dimensional parameters:
(c)
USC joint thinning (Δd), µm;
(d)
top ED thickness, according to the image analysis (δED top), µm;
(e)
distance between PEI adherends (δED+CF), µm;
(f)
top PEI adherend integrity, assessed qualitatively (C), +/–.
Considering that the USC process was a transformation of the properties of all components of the lap joints due to external impacts, and its law was significantly non-linear and unknown as well, the model of this process can be represented as a ‘black box’ with three inputs and six outputs (Figure 2).

2.2. ANN Architecture

In this study, both FFNN and RBFNN were implemented. According to Figure 2, the inputs were three USC parameters, while six outputs were the functional characteristics of the lap joints. In their feedforward architectures, one hidden neural layer was used (Figure 3). In the case of FFNN, the following parameters had to be studied: the number of neurons in the hidden layer, the type of activation function of each layer, the selection of both learning and testing samples, etc. The RBFN architecture was uniquely defined except for the number of neurons in the hidden layer (Figure 4).
It was determined by its learning algorithm, which possessed two parameters: the ‘spread’ (the radius of the response of neurons to the input stimulus/distance between the input data) and the ‘goal’ permissible error (between the approximation and the values of the learning sample). Both these parameters were determined from an analysis of the distances between the input data of the learning sample and the measurement errors of the output data.
Due to the different dimensions of the USC parameters and the functional characteristics of the lap joints, the values of both input and output data spaces were normalized to dimensionless before the simulation. As the obtained results show, the highest rate of convergence of the ANN learning algorithms was achieved when normalized to the [−1, 1] interval. Formal criteria for the quality of such models included a number of assessments of the difference between the test sample and the predicted values (standard deviation, mean square, determination coefficient, etc.) used in the ANN learning algorithms. However, according to the authors’ best knowledge, no formal criteria have so far been proposed to justify the adequacy and overlearning of ANNs.

2.3. Analysis of a Priori Knowledge and Its Implementation

A preliminary analysis of the USC parameters and the expected results was based on previously acquired knowledge about the process patterns:
  • USC procedures could not be successfully completed due to their short durations and/or insufficient or excessively high clamping pressures. In these cases, the dimensions of the USC lap joints were not changed, while the mechanical properties were minimal.
  • If the USC durations were too long, the components being joined were partially or completely damaged. The USC lap joint thicknesses were minimal (after their maximum thinning), while the mechanical properties were low (but above zero values).
  • Releasing the clamping pressure immediately after USC reduced the mechanical properties of the lap joints to below acceptable levels. Moreover, their dimensions predominantly depended on the USC parameters (duration, clamping pressure) at the first stage, while the mechanical properties were determined by both USC duration and clamping duration after USC.
Since it was not possible to simultaneously achieve extreme (maximum or minimum) values of all the functional characteristics of the USC lap joints, their acceptable ranges were specified, i.e., the optimality conditions (Table 1). The quantitative values of the range boundaries were based on the previous experience of the authors [38] and the results of an analysis of data reported by other researchers. Accordingly, further search for the optimal combination of the USC parameters was carried out taking into account the values given in Table 1.
Trial numerical investigations to determine the optimal USC parameters were performed using only the a priori knowledge (the first two features mentioned above). In this case, the ANN simulation was not enabled to achieve the goal due to patterns in the learning data structure. Separately, the parameter values were the maximum values of the acceptable ranges of the functional characteristics of the USC joints. Their approximation also gave similar levels that did not satisfy the optimality conditions. Therefore, subsequent learning samples included both a priori knowledge and predictive parameters within the acceptable ranges.
Advanced numerical investigations were based on a priori knowledge of the USC parameters at which such procedures could not be successfully completed and the levels required to achieve the optimal functional characteristics of the lap joints (Table 2). Figure 5 presents the results of the ANN simulation and assessment of the SOP area, drawn using the data from Table 2 when varying each range of the USC parameters presented in three rows for condition #2 “Constancy of dimensions’. The learning sample included 29 vectors of the USC parameters and the corresponding ones of the functional characteristics. In this case, the RBFNN (‘goal’ = 0.001) and FFNN (activation functions are hyperbolic tangents) were applied, with variation of their other parameters.
Note that the USC parameters presented in Table 2 limited the search area for their optimal values, but lay only in three sections of the corresponding space. As follows in Figure 5, this turned out to be enough to limit the SOP areas drawn using the RBFNN over their entire volumes. The ‘spread’ parameter exerted a direct impact on the sizes of the SOP areas: increasing the parameter gave rise to enlargement of the SOP areas. At this stage of the analysis using a priori knowledge, only models with ‘spread’ values below 0.5 were recognized as inadequate. In this case, it was accepted that the τ parameter was not dominant [38] and the SOP areas could not be discontinuous. Obviously, such a gap was associated with a great relative distance between the points of the predicted optimal USC parameters and could be eliminated by supplementing the sample with intermediate data. To the best of the authors’ knowledge, the problem of selection of an adequate value for the ‘spread’ parameter has not been formally solved for the RBFNN.
For the FFNN models, the absence of restrictions on three sections of the parameter space led to the unboundedness of the SOP areas in the direction of increasing the t parameter and in all directions of the τ parameter. It was noted that all the studied FFNN models implemented reliable approximation under these conditions but did not correspond to the a priori physical and mechanical concepts of the SOP areas. In this case, increasing the FFNN complexity (the number of neurons in the hidden layer) did not result in any significant changes in the SOP areas.
Additional a priori knowledge in the form of local points were added to the models as new variable parameters, namely the number of values and their distribution in space. Accordingly, it was necessary to use a uniform distribution of a priori knowledge in the parameter space according to Table 2, and to assess the influence of the number of additional points. In each of the three sections of the τ = 100 ms, P = 0.5 atm, and P = 4 atm space of the models described above, a priori knowledge was determined in the amounts of 6, 9, 12, and 18 points. The learning sample sizes were 20, 29, 38, and 56, respectively.
Subsequent numerical investigations included the development of a set of both RBNNs and FFNNs. The ‘spread’ value was 0.5, while the ‘goal’ levels varied from 10–3 to 10–6. The FFNNs were characterized by a single hidden layer (five neurons) and a log tangent activation function. Bayesian regularization was used as a learning algorithm.
The accuracy of the models was assessed by the standard deviation of the data from the a priori values at the outputs. Errors were calculated for three cases: (i) the approximation region was limited to the predicted ranges of the optimal USC parameters, (ii) the extrapolation region was limited to the maximum predicted optimal USC parameters and the maximum boundaries of the acceptable ranges (t = 700 ms, P = 4 atm, τ = 9000 ms), and (iii) the scope of the analysis was limited to the acceptable ranges of the USC parameters. Since the FFNN learning results were multi-valued, the errors in these cases were estimated from the ensemble of synthesized and learned networks. Table 3 and Figure 6 show some of the results of calculating the errors of ANN simulation as instances.
As expected, the errors exhibit complex, ambiguous, and extreme patterns. According to Figure 6a and Table 3, for the RBNNs, the errors of the simulation depended on the ‘goal’ parameter as well as the sample size. The errors in the extrapolation and approximation ranges were comparable and repeated the behavior of the errors in the field of analysis. This indicates that the RBFNNs’ results might be validated only over the data of the latter error. For FFNNs, a substantial difference was found in the behavior of the error with an increase in the sample size. After reaching a minimum, the error in the approximation range changed insignificantly, while in the extrapolation range it began to drastically increase.
In the general case, determining the optimal size of the learning sample could not be formulated as ‘the higher, the better’ for both RBFNNs and FFNNs. This is related to the problem of redundancy of the training sample, which occurs with large sample sizes. However, the use of deep learning can result in decreasing the accuracy of the simulation [10]. This is usually solved by introducing one or more of various methods of reducing the dimensions of the source data or the total sample size. Among them are the principal component method, linear discriminant analysis, random or deterministic sampling, etc. [62,63,64,65]. In the current study, the optimal size of the training sample in terms of minimizing the error of approximation and extrapolation was addressed as the problem of searching for the minimum error. Obviously, this task was of the search type. For the data presented above, the best results of the ANN simulation were achieved with a learning sample size of 38 (9 additional points with a priori knowledge in each section of the parameter space).
Comparing the errors in the approximation and extrapolation regions with the numbers of neurons in the RBFNNs after learning, the following could be concluded. Their training algorithm enabled them to avoid overlearning the model, i.e., an increase in the sample size did not enhance the number of neurons in all cases. Perhaps for this reason, it also did not contribute to a significant increase in extrapolation errors. In contrast to the RBFNNs, increasing the learning sample size of the FFNNs could lead to a significant increase in extrapolation errors (Figure 6b, volume > 29). Consequently, the implementation of FFNNs for ANN simulation using samples of a priori knowledge required the development of criteria for assessing their redundancy and limiting the sizes.

2.4. Analysis of the Experimental Data

In the ANN simulation, experimentally measured levels of the functional characteristics of lap joints obtained using the USC parameters presented in Table 4 were applied as a learning sample. The determined values are shown in Table 5, including nine vectors assessed by the Taguchi method. Both RBFNNs (‘goal’ = 0.001) and FFNNs (activation functions—hyperbolic tangent) were implemented. The distribution of the learning sample in the parameter space is shown in Figure 7a, while the SOP areas are presented in Figure 7b–d. As expected, the SOP area, determined using the RBFNN model with an ultra-small sample (Figure 7b), covered the relevant data and tended to the minimum size with lowering of the ‘spread’ parameter. On the one hand, this fact led to more accurate approximation within the range of the experimental USC parameters, but on the other hand it did not allow solving predictive problems. The implementation of the FFNN gave an infinite number of solutions, from completely unrealistic (Figure 7c) to adequate (Figure 7d). Moreover, all these models met the minimum mean square error criterion (the difference between the calculated values and the data of the learning and testing samples). Note that the drawn SOP areas were not limited to the region of the experimental USC parameters and extended into the predicted (extrapolation) area. Nevertheless, general patterns in their behavior were not revealed. This drawback of the ANNs testified to the inefficiency of their use in learning with ultra-small training samples, requiring the use of additional data to limit acceptable solutions in the extrapolation range. The accuracy of the constructed models was estimated according to verification with additional laboratory tests, as described in Section 3.

2.5. Analysis of a Posterior Knowledge and Its Implementation

Next, within the framework of the ANN simulation, a posteriori knowledge was added to the a priori knowledge (Table 2). The latter was obtained after or in the process of the experimental investigations and was introduced in sections of the parameter space corresponding to high t values but low τ levels (Table 6). In this case, the model was limited to knowledge in five sections of the parameter space, which were determined by three values according to Table 4. The learning sample size was 47 vectors of the USC parameters with the corresponding functional characteristics of the lap joints.
ANN simulation was carried out implementing both RBFNNs (‘goal’ = 0.001) and FFNNs (activation functions are hyperbolic tangent), as well as using the data from Table 2 and Table 6. A distribution of the learning sample in the parameter space is shown in Figure 8a, while the SOP areas, drawn using various USC parameters, are shown in Figure 8b–g.
Comparing the results obtained using the RBFNNs (Figure 4 and Figure 6), a conclusion was drawn that additional knowledge on the boundary sections could lead both to enhancing the SOP areas (Figure 8b) and their downsizing (Figure 8c,d). Changing the ‘spread’ parameter still affected the sizes of the SOP areas and could be considered as a tool for applying the subjective knowledge of experts.
Enhancing the learning sample size through the use of new knowledge in the FFNN models resulted in the limitation of the SOP areas in the directions of those sections where this new knowledge was determined. In this case, increasing the complexity of the FFNN (through the number of neurons in the hidden layer) played an important role in the non-linearity of the model: from a simple quasi-linear law (Figure 8e) to a significantly non-linear one (Figure 8g). Unlike the RBFNN models, the way to express non-formalized knowledge in the FFNN ones was to justify the number of neurons (N) in the hidden layer. In this example, the model had to be considered as coarse (undertrained) at N = 5, displaying an unreasonably complex law (overtrained) at N = 25, but adequate at N = 10.
Subsequent ANN simulation was performed using a sample that included nine experimentally measured values as well as both a priori and a posteriori knowledge about the USC parameters. The total sample size was 54 vectors. A distribution of the experimental parameters in the search area is presented in Figure 9a. The SOP areas drawn using the RBFNN models with different values of the ‘spread’ parameter are presented in Figure 9b,c, while Figure 9d reflects the case for the FFNN.
As a result of comparing the data presented in Figure 8 and Figure 9, the following conclusions were drawn about the impact of the addition of both a priori and a posteriori knowledge on the assessment of the SOP areas:
  • The RBFNN models became more complex. The sizes of the SOP areas increased, including in the extrapolation region.
  • The SOP areas determined using the FFNN models were limited to the extrapolation regions and acquired a general pattern (the uncertainty of the behavior decreased). However, this phenomenon did not solve the problem of multiple solutions.
  • The models drawn with the addition of both a priori and a posteriori knowledge should not be considered as competing, due to their imprecise expression. SOP areas drawn both with and without them had to be jointly analyzed.
  • The problem of choosing the learning sample size of both a priori and a posteriori knowledge had to be justified as a problem of finding the optimal size according to the minimum extrapolation error criterion.

3. Verification of the Results of ANN Simulation

Verification of the results of the ANN simulation was carried out using USC parameters (t = 510 ms, τ = 9000 ms, P = 1.85 atm) from the extrapolation region, for which the predicted values (i) varied most significantly, (ii) could be implemented using the available USW facility, and (iii) were located in the direction of the greatest expected errors of the ANN simulation. Table 7 presents the predicted values of the functional characteristics of the USC lap joints, obtained using the predicted USC parameters:
-
the RBFNNs were trained with only the experimental data (the ‘RBFNN’ row in Figure 8b) and with both a priori and a posteriori knowledge in addition to the experimental data (the ‘RBFNN (+ knowledge)’ row in Figure 9c);
-
for the FFNNs, due to the ambiguity of their training, the predicted values were calculated from a variety of the results (examples shown in Figure 8c,d and Figure 9d,e) after training with only the experimental data (the ‘FFNN’ row) and with both a priori and a posteriori knowledge in addition to the experimental data (the ‘FFNN (+ knowledge)’ row).
Table 7 also includes the experimentally measured values of the functional characteristics of the lap joint, obtained using the optimal USC parameters. Figure 10 shows an optical image of its cross-section. The USC lap joint thinning was 330 ± 10 µm, while the top ED was noticeably (but not uniformly) thinned (δED top = 80 ± 60 µm). At the same time, the bottom ED thickness was changed to a much lesser extent (δED bottom = 170 ± 70 µm). The “CF fabric layer” thickness was 280 ± 40 µm, which was slightly greater than the initial prepreg value (considering possible scattering). Samples of such USC lap joints fractured at rather high stress levels of 62.60 ± 3.13 MPa and values of elongation at the break of 4.22 ± 0.21%.
An analysis of the results presented in Table 7 showed that the RBFNN model trained with both additional a priori and a posteriori knowledge significantly increased its accuracy, so the predicted and experimentally measured values were closer to each other. This fact confirmed the possibility of using the RBFNN models for prediction by extrapolation using ultra-small samples after artificial expansion of their sizes. Respectively, such models should be considered as the most promising ones.
On the other hand, the FFNN models, both with additional a priori and a posterior knowledge and without them, were characterized by average predicted values close to the real ones. However, the scatter of the predicted values with additional knowledge in the learning samples increased significantly (Table 7), bringing into question the applicability of this approach. Additional research is required to determine the reasons for these variations.
A comparison of the predicted and real values (Table 6) showed that mechanical properties were achieved that exceeded the experimental data obtained above (mode 6). In this case, the δED top and δED+CF parameters were outside the acceptable ranges according to Table 1. Nevertheless, these values reflected the optimal USC parameters, enabling improvement of the mechanical properties (falling within the range specified in Table 1), a situation which was not obtained using modes 1–9. On this basis, it was decided to change the SOP areas according to Table 8.
The results of ANN simulation using the updated models trained with both a priori and a posteriori knowledge in addition to the experimental data (Table 5, modes 1–9; Table 7, the optimal USC parameters) are summarized in Figure 11.

4. Discussion

As noted above, most researchers of USW/USC procedures implemented for joining composites based on thermoplastic binders have focused on laminates. The reason for this includes both their high strength properties and the practical relevance of the obtained results. Similar, ANNs have been applied for solving such problems [12]. However, the key advantage of USC procedures is their short duration, expanding the application areas. For example, data on USW patterns concerning particulate composites based on thermoplastic matrices have been reported [66,67,68], in addition to which the results presented above could be adapted to develop such procedures for composites fabricated from polymer blends or hybrid polymer mixtures [69,70,71]. In this way, ANN simulation performed to optimize the USW parameters for obtaining USC ‘PEI adherend/Prepreg (CF-PEI fabric)/PEI adherend’ lap joints enabled an understanding of their complex mutual influence on the functional characteristics.
By analogy with the approach implemented for manufacturing laminates from sequential layers of both thermoplastic and CF fabric [72], a similar material was fabricated from PEI/CF prepregs and EDs in this study. To achieve this goal, the optimal USC parameters were applied, which were determined through ANN simulation. The method for manufacturing the PEI-impregnated PEEK-based prepreg based on the CF fabric (Toray Cetex TC1200, Toray Industries, Japan) was described in a previous paper by the authors [38].
Figure 12 shows cross-sections of the laminates made from (a) PEI-impregnated and (b) commercially available PEEK-based prepregs. The USC parameters justified above were used (clamping pressure of 1.85 atm, USW duration of 510 ms, clamping duration after USW of 9000 ms). The thickness of the PEI prepreg was 250 ± 20 µm, while that of the PEEK-based one was 170 ± 20 µm (PEEK prepregs were US-consolidated without EDs). The number of prepreg layers was five, while it was four for the EDs (for the PEI prepregs only). The USC procedures were carried out using the layer-by-layer method. The total number of passes of the USC instrument (sonotrode) was four. Under such conditions, it was possible to form non-porous USC laminates (joints) with satisfactory quality (minimal damage to the components). Thereby, the correctness of the optimal USW parameters predicted by ANN simulation was verified experimentally. This approach can also be implemented to repair damaged regions of other composites based on thermoplastic binders.
The research areas in which USC procedures have been implemented to form fiber-reinforced composites based on thermoplastic binders were already mentioned above:
-
continuous ultrasonic impregnation and consolidation of thermoplastic matrix composites;
-
ultrasonic-assisted consolidation of commingled thermoplastic/glass fiber rovings;
-
consolidation of composite pipes by in situ ultrasonic welding (thermoplastic matrix composite tape);
-
ultrasonic vibration-assisted automated fiber placement;
-
automated fiber placement and tape laying (thermoplastic composite prepreg);
-
filament winding and automated fiber placement with in situ consolidation.
All of these were characterized by the use of different USC parameters. Moreover, the sizes of the experimental data samples were very limited. Respectively, the authors believe that the approach developed in this study, based on ANN simulation using ultra-small samples, is of undoubted practical interest and can be applied to solve related problems, including the automated tape placement [73,74].

5. Conclusions

  • By ANN simulation, an approach was developed to establish the relationship between the USC parameters and the functional characteristics of ‘PEI adherend/Prepreg (CF-PEI fabric)/PEI adherend’ lap joints. For this purpose, the RBFNN and FFNN models were tested and the most reliable results were selected.
  • The real values of the functional characteristics of the USC joints were measured experimentally and used to train the ANNs. For improving the effectiveness, the experimental sample was significantly expanded by adding both a priori and a posteriori knowledge.
  • The conducted studies into the influence of the ANN simulation parameters and the addition of both a priori and a posteriori knowledge to the learning sample on the accuracy of the drawn SOP areas showed the following outcomes. The RBFNN model trained using the sample with the additional a priori and a posteriori knowledge significantly increased its accuracy, and the predicted data came closer to the experimental results. This fact confirmed the possibility of using RBFNN models for prediction through extrapolation of ultra-small samples after artificial expansion of their sizes. On the other hand, the FFNN models, even without adding both a priori and a posteriori knowledge, gave average values close to the real ones. However, the scatter of the predicted values increased significantly after the addition of both a priori and a posteriori knowledge, so the obtained results sow doubt on the applicability of this approach.
  • The results of ANN simulation were verified, since the implementation of the predicted USC parameters made it possible to maintain the structural integrity of the reinforcing CF fabric and ensure the maximum strength properties.
  • The results of ANN simulation for optimizing the USC parameters were suitable for manufacturing the ‘PEI adherend/Prepreg (CF-PEI fabric)/PEI adherend’ laminates (and those from the commercial PEEK/CF prepregs). Such procedures enable the formation of an even (regular) structure characterized by a minimum number of discontinuities and minimal damage to the joined components.

Author Contributions

Conceptualization, S.V.P. and D.Y.S.; methodology, S.V.P. and D.Y.S.; validation, V.O.A.; formal analysis, D.Y.S. and S.V.P.; investigation, D.Y.S. and D.T.; resources, D.G.B.; data curation, D.Y.S. and V.O.A.; writing—original draft preparation, S.V.P., D.Y.S. and S.V.P.; writing—review and editing, S.V.P.; visualization, D.Y.S.; funding acquisition, D.G.B. and D.T. All authors have read and agreed to the published version of the manuscript.

Funding

The work was performed according to the government research assignment for ISPMS SB RAS, project FWRW-2021-0010.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Data are contained within the article.

Acknowledgments

The authors acknowledge the senior researcher of ISPMS SB RAS, M.V. Burkov, for the providing PEEK prepregs for US consolidation of laminates.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Park, H.Y.; Datta-Gupta, A.; King, M.J. Handling conflicting multiple objectives using Pareto-based evolutionary algorithm during history matching of reservoir performance. J. Pet. Sci. Eng. 2015, 125, 48–66. [Google Scholar] [CrossRef]
  2. Ashby, M.F. Materials Selection in Mechanical Design, 5th ed.; Butterworth-Heinemann: Oxford, UK, 2016; p. 660. [Google Scholar]
  3. Aized, T.; Shirinzadeh, B. Robotic fiber placement process analysis and optimization using response surface method. Int. J. Adv. Manuf. Technol. 2011, 55, 393–404. [Google Scholar] [CrossRef]
  4. Nalbant, M.; Gökkaya, H.; Sur, G. Application of Taguchi method in the optimization of cutting parameters for surface roughness in turning. Mater. Des. 2007, 28, 1379–1385. [Google Scholar] [CrossRef]
  5. Dontsov, Y.V.; Panin, S.V.; Buslovich, D.G.; Berto, F. Taguchi optimization of parameters for feedstock fabrication and FDM manufacturing of wear-resistant UHMWPE-based composites. Materials 2020, 13, 338. [Google Scholar] [CrossRef] [PubMed]
  6. Xie, J.; Qiao, Y.; Wang, Z.; Qi, Y.; Xu, Q.; Yona, K.S.; Chen, P.; Rittel, D. Application of the Taguchi method to areal roughness-based surface topography control by waterjet treatments. Appl. Surf. Sci. Adv. 2024, 19, 100548. [Google Scholar] [CrossRef]
  7. Elkady, E.F.; Fouad, M.A.; Mozayad, A.N. Application of Box-Behnken experimental design and response surface methodology for selecting the optimum RP-HPLC conditions for the simultaneous determination of methocarbamol, indomethacin and betamethasone in their pharmaceutical dosage form. BMC Chem. 2022, 16, 114. [Google Scholar] [CrossRef] [PubMed]
  8. Miller, B.; Ziemiański, L. Multi-Objective Optimization of Thin-Walled Composite Axisymmetric Structures Using Neural Surrogate Models and Genetic Algorithms. Materials 2023, 16, 6794. [Google Scholar] [CrossRef] [PubMed]
  9. Tamur, C.; Li, S.; Zeng, D. Artificial Neural Networks for Predicting Mechanical Properties of Crystalline Polyamide12 via Molecular Dynamics Simulations. Polymers 2023, 15, 4254. [Google Scholar] [CrossRef] [PubMed]
  10. Haykin, S. Neural Networks: A Comprehensive Foundation, 2nd ed.; Prentice Hall: Upper Saddle River, NJ, USA, 1998. [Google Scholar]
  11. Zhang, Y.; Ling, C. A strategy to apply machine learning to small datasets in materials science. NPJ Comput. Mater. 2018, 4, 25. [Google Scholar] [CrossRef]
  12. Sutton, C.; Boley, M.; Ghiringhelli, L.M.; Rupp, M.; Vreeken, J.; Scheffler, M. Identifying domains of applicability of machine learning models for materials science. Nat. Commun. 2020, 11, 4428. [Google Scholar] [CrossRef] [PubMed]
  13. Xu, P.; Ji, X.; Li, M.; Lu, W. Small data machine learning in materials science. NPJ Comput. Mater. 2023, 9, 42. [Google Scholar] [CrossRef]
  14. Gandomi, A.H.; Roke, D.A. Assessment of artificial neural network and genetic programming as predictive tools. Adv. Eng. Softw. 2015, 88, 63–72. [Google Scholar] [CrossRef]
  15. Jongbloed, B.; Teuwen, J.; Benedictus, R.; Villegas, I.F. On differences and similarities between static and continuous ultrasonic welding of thermoplastic composites. Compos. Part B Eng. 2020, 203, 108466. [Google Scholar] [CrossRef]
  16. Bhudolia, S.K.; Gohel, G.; Leong, K.F.; Islam, A. Advances in ultrasonic welding of thermoplastic composites: A review. Materials 2020, 13, 1284. [Google Scholar] [CrossRef]
  17. Kirby, M.; Naderi, A.; Palardy, G. Predictive thermal modeling and characterization of ultrasonic consolidation process for thermoplastic composites. J. Manuf. Sci. Eng. 2023, 145, 031009. [Google Scholar] [CrossRef]
  18. Tao, W.; Su, X.; Wang, H.; Zhang, Z.; Li, H.; Chen, J. Influence mechanism of welding time and energy director to the thermoplastic composite joints by ultrasonic welding. J. Manuf. Process. 2019, 37, 196–202. [Google Scholar] [CrossRef]
  19. Benatar, A. 12—Ultrasonic welding of plastics and polymeric composites. In Power Ultrasonics, Gallego; Juárez, J.A., Graff, K.F., Eds.; Woodhead Publishing: Oxford, UK, 2015; pp. 295–312. [Google Scholar]
  20. Villegas, I.F. In situ monitoring of ultrasonic welding of thermoplastic composites through power and displacement data. J. Thermoplast. Compos. Mater. 2015, 28, 66–85. [Google Scholar] [CrossRef]
  21. Lionetto, F.; Dell’Anna, R.; Montagna, F.; Maffezzoli, A. Modeling of continuous ultrasonic impregnation and consolidation of thermoplastic matrix composites. Compos. Part A Appl. Sci. Manuf. 2016, 82, 119–129. [Google Scholar] [CrossRef]
  22. Lionetto, F.; Dell’Anna, R.; Montagna, F.; Maffezzoli, A. Ultrasonic assisted consolidation of commingled thermoplastic/glass fiber rovings. Front. Mater. 2015, 2, 32. [Google Scholar] [CrossRef]
  23. Dell’Anna, R.; Lionetto, F.; Montagna, F.; Maffezzoli, A. Lay-up and consolidation of a composite pipe by in situ ultrasonic welding of a thermoplastic matrix composite tape. Materials 2018, 11, 786. [Google Scholar] [CrossRef] [PubMed]
  24. Chu, Q.; Li, Y.; Xiao, J.; Huan, D.; Zhang, X.; Chen, X. Processing and characterization of the thermoplastic composites manufactured by ultrasonic vibration–assisted automated fiber placement. J. Thermoplast. Compos. Mater. 2018, 31, 339–358. [Google Scholar] [CrossRef]
  25. Yassin, K.; Hojjati, M. Processing of thermoplastic matrix composites through automated fiber placement and tape laying methods: A review. J. Thermoplast. Compos. Mater. 2018, 31, 1676–1725. [Google Scholar] [CrossRef]
  26. Rizzolo, R.H.; Walczyk, D.F. Ultrasonic consolidation of thermoplastic composite prepreg for automated fiber placement. J. Thermoplast. Compos. Mater. 2016, 29, 1480–1497. [Google Scholar] [CrossRef]
  27. Boon, Y.D.; Joshi, S.C.; Bhudolia, S.K. Review: Filament Winding and Automated Fiber Placement with In Situ Consolidation for Fiber Reinforced Thermoplastic Polymer Composites. Polymers 2021, 13, 1951. [Google Scholar] [CrossRef]
  28. Koutras, N.; Amirdine, J.; Boyard, N.; Villegas, I.F.; Benedictus, R. Characterisation of Crystallinity at the Interface of Ultrasonically Welded Carbon Fibre PPS Joints. Compos. Part A Appl. Sci. Manuf. 2019, 125, 105574. [Google Scholar] [CrossRef]
  29. Takamura, M.; Uehara, K.; Koyanagi, J.; Takeda, S. Multi-Timescale Simulations of Temperature Elevation for Ultrasonic Welding of CFRP with Energy Director. J. Multiscale Model. 2021, 12, 2143003. [Google Scholar] [CrossRef]
  30. Jongbloed, B.; Teuwen, J.; Palardy, G.; Villegas, I.F.; Benedictus, R. Continuous Ultrasonic Welding of Thermoplastic Composites: Enhancing the Weld Uniformity by Changing the Energy Director. J. Compos. Mater. 2020, 54, 2023–2035. [Google Scholar] [CrossRef]
  31. Levy, A.; Le Corre, S.; Poitou, A. Ultrasonic welding of thermoplastic composites: A numerical analysis at the mesoscopic scale relating processing parameters, flow of polymer and quality of adhesion. Int. J. Mater. Form. 2014, 7, 39–51. [Google Scholar] [CrossRef]
  32. Stokes-Griffin, C.M.; Compston, P. The effect of processing temperature and placement rate on the short beam strength of carbon fibre–PEEK manufactured using a laser tape placement process. Compos. Part A Appl. Sci. Manuf. 2015, 78, 274–283. [Google Scholar] [CrossRef]
  33. Chen, J.; Fu, K.; Li, Y. Understanding processing parameter effects for carbon fibre reinforced thermoplastic composites manufactured by laser-assisted automated fibre placement (AFP). Compos. Part A Appl. Sci. Manuf. 2021, 140, 106160. [Google Scholar] [CrossRef]
  34. Tierney, J.J.; Gillespie, J.W., Jr. Crystallization kinetics behavior of PEEK based composites exposed to high heating and cooling rates. Compos. Part A Appl. Sci. Manuf. 2004, 35, 547–558. [Google Scholar] [CrossRef]
  35. Slange, T.K.; Grouve, W.J.; Warnet, L.L.; Wijskamp, S.; Akkerman, R. Towards the combination of automated lay-up and stamp forming for consolidation of tailored composite components. Compos. Part A Appl. Sci. Manuf. 2019, 119, 165–175. [Google Scholar] [CrossRef]
  36. Forcellese, A.; Greco, L.; Pieralisi, M.; Simoncini, M.; Trevisan, G. Mechanical properties of carbon fiber reinforced plastic obtained by the automatic deposition of an innovative towpreg. Procedia CIRP 2020, 88, 451–456. [Google Scholar] [CrossRef]
  37. Struzziero, G.; Barbezat, M.; Skordos, A.A. Consolidation of continuous fibre reinforced composites in additive processes: A review. Addit. Manuf. 2021, 48, 102458. [Google Scholar] [CrossRef]
  38. Alexenko, V.O.; Panin, S.V.; Stepanov, D.Y.; Byakov, A.V.; Bogdanov, A.A.; Buslovich, D.G.; Panin, K.S.; Tian, D. Ultrasonic Welding of PEEK Plates with CF Fabric Reinforcement—The Optimization of the Process by Neural Network Simulation. Materials 2023, 16, 2115. [Google Scholar] [CrossRef]
  39. Li, W.; Frederick, H.; Palardy, G. Multifunctional films for thermoplastic composite joints: Ultrasonic welding and damage detection under tension loading. Compos. Part A Appl. Sci. Manuf. 2021, 141, 106221. [Google Scholar] [CrossRef]
  40. Kim, J.W.; Lee, J.S. The effect of the melt viscosity and impregnation of a film on the mechanical properties of thermoplastic composites. Materials 2016, 9, 448. [Google Scholar] [CrossRef]
  41. Micallef, L.; Sundin, I.; Marttinen, P.; Ammad-ud-din, M.; Peltola, T.; Soare, M.; Jacucci, G.; Kaski, S. Interactive Elicitation of Knowledge on Feature Relevance Improves Predictions in Small Data Sets. In Proceedings of the 22nd International Conference on Intelligent User Interfaces, Limassol, Cyprus, 13–16 March 2017; Association for Computing Machinery: New York, NY, USA, 2017. [Google Scholar]
  42. Panin, S.V.; Stepanov, D.Y.; Byakov, A.V. Optimizing Ultrasonic Welding Parameters for Multilayer Lap Joints of PEEK and Carbon Fibers by Neural Network Simulation. Materials 2022, 15, 6939. [Google Scholar] [CrossRef] [PubMed]
  43. Butyrsky, E.Y.; Kuvaldin, I.A.; Chalkin, V.P. Approximation of multidimensional functions. Sci. Instrum. Eng. 2010, 20, 82–92. (In Russian) [Google Scholar]
  44. Misener, R.; Floudas, C.A. Piecewise-linear approximations of multidimensional functions. J. Optim. Theory Appl. 2010, 145, 120–147. [Google Scholar] [CrossRef]
  45. Berezin, I.S.; Zhidkov, N.P. Computational Methods; GIFML Gosudarstevennoe Izdatelstvo Fiziko Matematicheskoy Literatury: Moscow, Russia, 1962; Volume 1, 464p. (In Russian) [Google Scholar]
  46. Dahlquist, G.; Björck, Å. Society for Industrial and Applied Mathematics. In Numerical Methods in Scientific Computing; SIAM: Philadelphia, PA, USA, 2008; Volume i. [Google Scholar]
  47. Koshel, S.M.; Musin, O.R. Digital modeling methods: Kriging and radial interpolation. Inf. Bull. GIS Assoc. 2001, 28, 23–24. (In Russian) [Google Scholar]
  48. Yepes-Bellver, L.; Brun-Izquierdo, A.; Alcalá, J.; Yepes, V. Embodied Energy Optimization of Prestressed Concrete Road Flyovers by a Two-Phase Kriging Surrogate Model. Materials 2023, 16, 6767. [Google Scholar] [CrossRef] [PubMed]
  49. Haykin, S. Neural Networks and Learning Machines, 3rd ed.; Pearson: Upper Saddle River, NJ, USA, 2009; pp. 246–249. [Google Scholar]
  50. Shan, K.; Zhang, Y.; Lan, Y.; Jiang, K.; Xiao, G.; Li, B. Surface Roughness Prediction of Titanium Alloy during Abrasive Belt Grinding Based on an Improved Radial Basis Function (RBF) Neural Network. Materials 2023, 16, 7224. [Google Scholar] [CrossRef] [PubMed]
  51. Chen, F.; Xu, W.; Wen, Q.; Zhang, G.; Xu, L.; Fan, D.; Yu, R. Advancing Concrete Mix Proportion through Hybrid Intelligence: A Multi-Objective Optimization Approach. Materials 2023, 16, 6448. [Google Scholar] [CrossRef] [PubMed]
  52. Hagen, M.T.; Demuth, H.B.; Beale, M. Neural Network Design; PWS Publishing Co.: Boston, MA, USA, 1996. [Google Scholar]
  53. Hijazi, A.; Al-Dahidi, S.; Altarazi, S.A. Residual Strength Prediction of Aluminum Panels with Multiple Site Damage Using Artificial Neural Networks. Materials 2020, 13, 5216. [Google Scholar] [CrossRef] [PubMed]
  54. Karniadakis, G.E.; Kevrekidis, I.G.; Lu, L.; Perdikaris, P.; Wang, S.; Yang, L. Physics-informed machine learning. Nat. Rev. Phys. 2021, 3, 422–440. [Google Scholar] [CrossRef]
  55. Gui, L.; Wang, B.; Cai, R.; Yu, Z.; Liu, M.; Zhu, Q.; Xie, Y.; Liu, S.; Killinger, A. Prediction of In-Flight Particle Properties and Mechanical Performances of HVOF-Sprayed NiCr–Cr3C2 Coatings Based on a Hierarchical Neural Network. Materials 2023, 16, 6279. [Google Scholar] [CrossRef] [PubMed]
  56. Lian, W.; Du, F. Reliability Prediction of Near-Isothermal Rolling of TiAl Alloy Based on Five Neural Network Models. Materials 2023, 16, 6709. [Google Scholar] [CrossRef]
  57. Burmakova, A.; Kalibatienė, D. Applying Fuzzy Inference and Machine Learning Methods for Prediction with a Small Dataset: A Case Study for Predicting the Consequences of Oil Spills on a Ground Environment. Appl. Sci. 2022, 12, 8252. [Google Scholar] [CrossRef]
  58. Caiafa, C.F.; Sun, Z.; Tanaka, T.; Marti-Puig, P.; Solé-Casals, J. Machine Learning Methods with Noisy, Incomplete or Small Datasets. Appl. Sci. 2021, 11, 4132. [Google Scholar] [CrossRef]
  59. Jordan, M.I.; Mitchell, T.M. Machine learning: Trends, perspectives, and prospects. Science 2015, 349, 255–260. [Google Scholar] [CrossRef] [PubMed]
  60. Seko, A.; Hayashi, H.; Nakayama, K.; Takahashi, A.; Tanaka, I. Representation of compounds for machine-learning prediction of physical properties. Phys. Rev. B 2017, 95, 144110. [Google Scholar] [CrossRef]
  61. Zhou, W.; Ji, X.; Yang, S.; Liu, J.; Ma, L. Review on the performance improvements and non-destructive testing of patches repaired composites. Compos. Struct. 2021, 263, 113659. [Google Scholar] [CrossRef]
  62. Jolliffe, I.T. Principal Component Analysis, 2nd ed.; Springer Series in Statistics; Springer: New York, NY, USA, 2002; ISBN 978-0-387-95442-4. [Google Scholar]
  63. Tille, Y. Sampling Algorithms; Springer Series in Statistics; Springer: Berlin/Heidelberg, Germany, 2006; ISBN 978-0-387-30814-2. [Google Scholar] [CrossRef]
  64. Bolón-Canedo, V.; Remeseiro, B. Feature selection in image analysis: A survey. Artif. Intell. Rev. 2020, 53, 2905–2931. [Google Scholar] [CrossRef]
  65. Kabir, H.; Garg, N. Machine learning enabled orthogonal camera goniometry for accurate and robust contact angle measurements. Sci. Rep. 2023, 13, 1497. [Google Scholar] [CrossRef] [PubMed]
  66. Zhi, Q.; Li, Y.; Shu, P.; Tan, X.; Tan, C.; Liu, Z. Double-Pulse Ultrasonic Welding of Carbon-Fiber-Reinforced Polyamide 66 Composite. Polymers 2022, 14, 714. [Google Scholar] [CrossRef]
  67. Wang, K.; Shriver, D.; Li, Y.; Banu, M.; Hu, S.J.; Xiao, G.; Arinez, J.; Fan, H.T. Characterization of weld attributes in ultrasonic welding of short carbon fiber reinforced thermoplastic composites. J. Manuf. Process. 2017, 29, 124–132. [Google Scholar] [CrossRef]
  68. Wang, K.; Shriver, D.; Banu, M.; Hu, S.J.; Xiao, G.; Arinez, J.; Fan, H.T. Performance Prediction for Ultrasonic Spot Welds of Short Carbon Fiber-Reinforced Composites Under Shear Loading. J. Manuf. Sci. Eng. 2017, 139, 111001. [Google Scholar] [CrossRef]
  69. Natesh, M.; Yun, L.; Vendan, S.A.; Kumar, K.R.; Gao, L.; Niu, X.; Peng, X.; Garg, A. Experimental and numerical procedure for studying strength and heat generation responses of ultrasonic welding of polymer blends. Measurement 2019, 132, 1–10. [Google Scholar] [CrossRef]
  70. Vendan, A.S.; Chinnadurai, T.; Senthil Kumar, K.; Prakash, N. Investigations on mechanical and structural aspects of ultrasonic hybrid polymer mixture welding for industrial applications. Int. J. Adv. Manuf. Technol. 2017, 93, 89–102. [Google Scholar] [CrossRef]
  71. Chinnadurai, T.; Natesh, M.; Vendan, S.A.; Dinek, R.; Kumar, K.R. Experimental studies on thermo-mechanical behavior of ultrasonically welded PC/ABS polymer blends. Silicon 2018, 10, 1937–1948. [Google Scholar] [CrossRef]
  72. Gomer, A.; Zou, W.; Grigat, N.; Sackmann, J.; Schomburg, W.K. Fabrication of fiber reinforced plastics by ultrasonic welding. J. Compos. Sci. 2018, 2, 56. [Google Scholar] [CrossRef]
  73. Qureshi, Z.; Swait, T.; Scaife, R.; El-Dessouky, H.M. In situ consolidation of thermoplastic prepreg tape using automated tape placement technology: Potential and possibilities. Compos. Part B Eng. 2014, 66, 255–267. [Google Scholar] [CrossRef]
  74. Donough, M.J.; St John, N.A.; Philips, A.W.; Prusty, B.G. Process simulation of In-situ consolidated thermoplastic composite by automated fibre placement–A review. Compos. Part A Appl. Sci. Manuf. 2022, 163, 107179. [Google Scholar] [CrossRef]
Polymers 16 00451 g001
Figure 1. An example of changes in the results of one-dimensional approximation of experimental data with the additional use of a priori knowledge.
Figure 1. An example of changes in the results of one-dimensional approximation of experimental data with the additional use of a priori knowledge.
Polymers 16 00451 g001
Polymers 16 00451 g002
Figure 2. A general model of the USC process.
Figure 2. A general model of the USC process.
Polymers 16 00451 g002
Polymers 16 00451 g003
Figure 3. Architecture of the feedforward function for the FFNN: Wn are matrices of synaptic weights, Bn are vectors of bias values.
Figure 3. Architecture of the feedforward function for the FFNN: Wn are matrices of synaptic weights, Bn are vectors of bias values.
Polymers 16 00451 g003
Polymers 16 00451 g004
Figure 4. Architecture of the radial basis function for the RBFNN: Wn are matrices of synaptic weights, Bn are vectors of bias values.
Figure 4. Architecture of the radial basis function for the RBFNN: Wn are matrices of synaptic weights, Bn are vectors of bias values.
Polymers 16 00451 g004
Polymers 16 00451 g005aPolymers 16 00451 g005b
Figure 5. The USC parameters of a priori knowledge (a) and the SOP areas, constructed using the RBFNN (b–d) and FFNN (e,f) models. A priori knowledge data.
Figure 5. The USC parameters of a priori knowledge (a) and the SOP areas, constructed using the RBFNN (b–d) and FFNN (e,f) models. A priori knowledge data.
Polymers 16 00451 g005aPolymers 16 00451 g005b
Polymers 16 00451 g006
Figure 6. The dependences of the model accuracy on the sizes of a priori knowledge learning samples.
Figure 6. The dependences of the model accuracy on the sizes of a priori knowledge learning samples.
Polymers 16 00451 g006
Polymers 16 00451 g007
Figure 7. (a) The USC parameters of a priori knowledge and the SOP areas, drawn using the (b) RBFNN and (c,d) FFNN models. The learning sample was the experimental data from nine points, according to Table 5.
Figure 7. (a) The USC parameters of a priori knowledge and the SOP areas, drawn using the (b) RBFNN and (c,d) FFNN models. The learning sample was the experimental data from nine points, according to Table 5.
Polymers 16 00451 g007
Polymers 16 00451 g008aPolymers 16 00451 g008b
Figure 8. The USC parameters of a priori knowledge (a) and the SOP areas, drawn using the RBFNN (b–d) and FFNN (e–g) models. The learning sample was both the a priori and a posteriori knowledge.
Figure 8. The USC parameters of a priori knowledge (a) and the SOP areas, drawn using the RBFNN (b–d) and FFNN (e–g) models. The learning sample was both the a priori and a posteriori knowledge.
Polymers 16 00451 g008aPolymers 16 00451 g008b
Polymers 16 00451 g009aPolymers 16 00451 g009b
Figure 9. The USC parameters of both knowledge and experimental data (a), the SOP areas drawn using the RBFNN (b,c) and FFNN (d–g) models. The learning sample was the experimental data, as well as both a priori and a posteriori knowledge.
Figure 9. The USC parameters of both knowledge and experimental data (a), the SOP areas drawn using the RBFNN (b,c) and FFNN (d–g) models. The learning sample was the experimental data, as well as both a priori and a posteriori knowledge.
Polymers 16 00451 g009aPolymers 16 00451 g009b
Polymers 16 00451 g010
Figure 10. The optical image of the cross section of the lap joint obtained using the optimal USC parameters.
Figure 10. The optical image of the cross section of the lap joint obtained using the optimal USC parameters.
Polymers 16 00451 g010
Polymers 16 00451 g011
Figure 11. The USC parameter space and the SOP areas, drawn using the RBFNN (a) and FFNN (b) models. The learning sample was the (1–9, ‘optimal’) experimental data as well as both a priori and a posteriori knowledge.
Figure 11. The USC parameter space and the SOP areas, drawn using the RBFNN (a) and FFNN (b) models. The learning sample was the (1–9, ‘optimal’) experimental data as well as both a priori and a posteriori knowledge.
Polymers 16 00451 g011
Polymers 16 00451 g012
Figure 12. Optical images of the cross sections of the five-layer USC laminates of (a) PEI-impregnated and (b) PEEK-based CF fabric-based prepregs.
Figure 12. Optical images of the cross sections of the five-layer USC laminates of (a) PEI-impregnated and (b) PEEK-based CF fabric-based prepregs.
Polymers 16 00451 g012
Table 1. The acceptable ranges of the functional characteristics of the USW joints.
Table 1. The acceptable ranges of the functional characteristics of the USW joints.
PropertyRange
Tensile strength, MPa 30 ≤ σ ≤ 65
Elongation at break, % 1.75 ≤ ε ≤ 4.50
USC joint thinning, μm 100 ≤ Δd ≤ 400
Top ED thickness, μm 120 ≤ δED top ≤ 180
Distance between PEI adherends, μm 450 ≤ δED+CF ≤ 550
Top PEI adherend integrity, +/– 0 ≤ C
Table 2. A priori knowledge, as well as predicted USC parameters and functional characteristics of the lap joints.
Table 2. A priori knowledge, as well as predicted USC parameters and functional characteristics of the lap joints.
ConditionsParametersCharacteristics
USW Duration, msClamping Pressure, atmDuration of Clamping after USW, msσ, MPaε, %Δd, μmδED top, μmδED + CF, μmC
1. Optimal predicted properties5001.53000654.54001204501
5001.57000
2. Constancy of dimensions1001÷21000 ÷ 70000002507501
300 ÷ 70041000 ÷ 7000
300 ÷ 7000.51000 ÷ 7000
Table 3. The errors of ANN simulation using a priori knowledge learning samples.
Table 3. The errors of ANN simulation using a priori knowledge learning samples.
Sample SizeMSE for Learning SampleNumber of Neurons (N)Standard Deviation
Analysis RegionApproximation RegionExtrapolation Region
RBFNN (goal = 1 × 10–3)
209.72 × 10−4100.06360.03360.047
292.3691 × 10−4200.03180.01940.0189
384.1871 × 10−5200.01910.00690.0139
562.6309 × 10−4180.02360.02240.0173
RBFNN (goal = 1 × 10–6)
207.22 × 10−7160.03450.03990.0242
292.852 × 10−31270.00746.0390 × 10−40.0025
382.6257 × 10−10340.00746.0496 × 10−40.0025
562.6669 × 10−7370.00775.6014 × 10−40.0033
FFNN (average of seven models)
207.71 × 10−660.13950.04700.0905
299.45 × 10−760.03250.00010.0134
386.79 × 10−660.02100.00600.0979
566.68 × 10−660.32470.00020.2165
Table 4. The UCC parameters obtained by combining the factors and their levels in the L9 format, according to the Taguchi method.
Table 4. The UCC parameters obtained by combining the factors and their levels in the L9 format, according to the Taguchi method.
Experiment No.Parameters
USW Duration, msClamping Pressure, atm Duration of Clamping after USC, ms
(1)4001.53000
(2)4001.75000
(3)4001.97000
(4)5001.55000
(5)5001.77000
(6)5001.93000
(7)6001.57000
(8)6001.73000
(9)6001.95000
Table 5. The functional characteristics of the joints obtained using the USC parameters presented in Table 4.
Table 5. The functional characteristics of the joints obtained using the USC parameters presented in Table 4.
No.Tensile Strength (σ), MPaElongation at Break (ε), %USC Joint Thinning (Δd), μmTop ED Thickness (δED top), μmBottom ED Thickness
(δED bottom), μm		Mean ED Thickness (δED mean), μmDistance between PEI Adherends (δED+CF), μm“CF layer” Thickness (δCF), μm Top PEI Adherend Integrity, +/–
126.0 ± 1.61.61 ± 0.05110 ± 60140 ± 40220 ± 40180 ± 80620 ± 20230 ± 30+
224.7 ± 1.71.41 ± 0.06160 ± 30160 ± 60200 ± 40170 ± 70620 ± 40230 ± 70+
325.0 ± 1.51.70 ± 0.07230 ± 20170 ± 30180 ± 40180 ± 40540 ± 40210 ± 50+
435.2 ± 1.42.07 ± 0.07160 ± 20150 ± 70160 ± 40150 ± 70540 ± 40190 ± 70+
546.2 ± 3.22.85 ± 0.10120 ± 20150 ± 150115 ± 65150 ± 150550 ± 50250 ± 110+
661.3 ± 3.13.81 ± 0.11400 ± 40160 ± 40175 ± 75175 ± 75500 ± 40190 ± 70+
734.3 ± 2.12.00 ± 0.07360 ± 2090 ± 70130 ± 110130 ± 110420 ± 80200 ± 60–
838.2 ± 2.32.12 ± 0.07290 ± 20110 ± 50170 ± 50140 ± 80490 ± 70210 ± 30–
941.6 ± 1.72.37 ± 0.08420 ± 20100 ± 60170 ± 70140 ± 100420 ± 60180 ± 40–
Table 6. A posteriori knowledge about the functional characteristics of the USC lap joints.
Table 6. A posteriori knowledge about the functional characteristics of the USC lap joints.
ConditionsParametersProperties
USW Duration, msClamping Pressure, atmDuration of Clamping after USC, msσ, MPaε, %Δd, μmδED top, μmδED + CF, μmC
Failure of lap joints during USC (before testing)9001 ÷ 31000÷7000401.540050350−1
Negligible strength due to minimum clamping duration after USC3001 ÷ 30100.75502506501
5001 ÷ 32001605501
7001 ÷ 340050350−1
Table 7. The functional characteristics of the USC lap joints.
Table 7. The functional characteristics of the USC lap joints.
No.σ, MPaε, %Δd, μmδED top, μmδED + CF, μmC
RBFNN−3.520.12661.2489.86316.53−1.04
RBFNN (+knowledge)41.172.2998.08191.29551.460.91
FFNN31.70 ± 6.932.06 ± 0.58268.3 ± 146.0164.6 ± 5.0533.9 ± 53.41.0 ± 0.0
FFNN (+knowledge)45.50 ± 15.782.9 ± 0.9208.3 ± 208.3148.5 ± 94.9567.04 ± 160.500.88 ± 0.10
Optimal62.60 ± 3.134.22 ± 0.21330 ± 1080 ± 60280 ± 401
Table 8. The acceptable ranges of the functional characteristics of the USC joints, justified by ANN simulation and verification of its results.
Table 8. The acceptable ranges of the functional characteristics of the USC joints, justified by ANN simulation and verification of its results.
PropertyRange
Tensile strength, MPa 30 ≤ σ ≤ 65
Elongation at break, % 1.75 ≤ ε ≤ 4.50
USC lap joint thinning, μ m 100 ≤ ∆d ≤ 400
Top ED thickness, μ m 65 ≤ δED top ≤ 180
Distance between PEI adherends, μ m 230 ≤ δED+CF ≤ 550
Top PEI adherend integrity, +/– 0.0 ≤ C
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Stepanov, D.Y.; Tian, D.; Alexenko, V.O.; Panin, S.V.; Buslovich, D.G. Application of Neural Network Models with Ultra-Small Samples to Optimize the Ultrasonic Consolidation Parameters for ‘PEI Adherend/Prepreg (CF-PEI Fabric)/PEI Adherend’ Lap Joints. Polymers 2024, 16, 451. https://doi.org/10.3390/polym16040451

AMA Style

Stepanov DY, Tian D, Alexenko VO, Panin SV, Buslovich DG. Application of Neural Network Models with Ultra-Small Samples to Optimize the Ultrasonic Consolidation Parameters for ‘PEI Adherend/Prepreg (CF-PEI Fabric)/PEI Adherend’ Lap Joints. Polymers. 2024; 16(4):451. https://doi.org/10.3390/polym16040451

Chicago/Turabian Style

Stepanov, Dmitry Y., Defang Tian, Vladislav O. Alexenko, Sergey V. Panin, and Dmitry G. Buslovich. 2024. "Application of Neural Network Models with Ultra-Small Samples to Optimize the Ultrasonic Consolidation Parameters for ‘PEI Adherend/Prepreg (CF-PEI Fabric)/PEI Adherend’ Lap Joints" Polymers 16, no. 4: 451. https://doi.org/10.3390/polym16040451

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop