Optimizing System Reliability in Additive Manufacturing Using Physics-Informed Machine Learning

Abstract

Fused filament fabrication (FFF), an additive manufacturing process, is an emerging technology with issues in the uncertainty of mechanical properties and quality of printed parts. The consideration of all main and interaction effects when changing print parameters is not efficiently feasible, due to existing stochastic dependencies. To address this issue, a machine learning method is developed to increase reliability by optimizing input parameters and predicting system responses. A structure of artificial neural networks (ANN) is proposed that predicts a system response based on input parameters and observations of the system and similar systems. In this way, significant input parameters for a reliable system can be determined. The ANN structure is part of physics-informed machine learning and is pretrained with domain knowledge (DK) to require fewer observations for full training. This includes theoretical knowledge of idealized systems and measured data. New predictions for a system response can be made without retraining but by using further observations from the predicted system. Therefore, the predictions are available in real time, which is a precondition for the use in industrial environments. Finally, the application of the developed method to print bed adhesion in FFF and the increase in system reliability are discussed and evaluated.

1. Introduction

The production of plastic parts by means of fused filament fabrication (FFF) is on its way to becoming established for mass production [1]. The major advantage of mass-produced products from FFF over conventional manufacturing is the greater variability and individuality of the products as well as lower cost for smaller production quantities [1,2]. Furthermore, FFF is an inexpensive and widely used additive manufacturing process [3]. In FFF, a polymer is heated until it reaches a semi-fluid state. Then it is squeezed out of a nozzle, cools down, and becomes solid shortly after. In this way, products are created layer by layer with a 3D printer. The process depends on print parameters that determine, among other things, the number and properties of the layers, speeds, and temperatures. In order to be able to use the FFF process economically in mass production, the machine costs and personnel costs must be reduced [4] as well as the uncertainty in the product quality such as aesthetics, dimensional accuracy, and mechanical properties [5]. Research is conducted to control the uncertainty by using simulations of the whole printing process [6] and by optimizing the printing parameters [7] among others [5]. Three-dimensional printing is a complex process with a minimum of 75 printing parameters and more to be optimized simultaneously. Physically informed machine learning (PIML) combines machine learning using data and physical knowledge in the form of models or constraints to reduce errors of machine learning and physical models [8,9]. PIML has shown promising results in high-dimensional contexts [9]. Through integration of mathematical physics models into machine learning fewer data are needed for the training of the neural network [10].

The method developed in this paper differs from the literature mentioned above by deriving empirical models from domain knowledge (DK), which can be in the form of research results or other sources. These models are then combined with data collected from 3D printers. System reliability is ensured by individually optimizing the input parameters for each 3D printer, paying attention to the disturbance variables.

The summarized contributions of this paper are as follows:

• Development of a new method using physics-informed machine learning to optimize system reliability in additive manufacturing.
• Reliability optimization is implemented by using neural networks to predict the system responses and an optimization algorithm with respect to system specific boundary conditions.
• System behavior of 3D printers is quantified by unsupervised machine learning and is used for predictions of system responses for the knowledge transfer.
• The new method is designed to be used for a number of 3D printers in an industrial environment in continuous operation and to provide calculation proposals for input parameters in real time.

2. Methods

This section gives an overview of the techniques used by the proposed method for reliability optimization. These methods include Latin hypercube sampling, deep neural networks, recurrent neural networks, which have been widely used and closely studied in the scientific field [11], and PIML, which yields promising results.

2.1. Latin Hypercube Sampling (LHS)

The influence that a parameter has on a target value is called the total effect, which is composed of interaction effects and the main effect. The main effect is the influence of the parameter that depends only on the parameter itself. Interaction effects also depend on other parameters. The order of an interaction indicates how many other parameters the influence depends on. To conduct experiments, the research question must be clarified to determine what type of effects should be identified. For the proposed method, detailed knowledge of the effects is required, and therefore the goal of knowing main and interaction effects is pursued. This is achieved using stochastic simulation. In Latin hypercube sampling (LHS), each parameter is divided into equally sized intervals, with the number of intervals corresponding to the number of experiments performed. Then, the experiments are distributed uniformly among the intervals [12]. This method was extended to generate approximately orthogonal, i.e., particularly balanced, experimental designs [13]. Currently, the method is accepted and widely used, although the number of experiments cannot be expanded without changing the overall number of samples. This method was chosen because it does create a pseudo-random experimental design, but it is also approximately orthogonal. These properties are important for quantifying the behavior of 3D printers, as effects can be determined independently of neural networks.

2.2. Artificial Neural Networks (ANN)

Artificial neural networks (ANN) consist of artificial neurons which in turn are composed of a weighted sum connected to an activation function. Therefore, each artificial neuron has one or more inputs and one output. ANN can be constructed from artificial neurons in layers, where a neuron has all the outputs of the neurons of the previous layer as inputs. This setup is called multilayer perceptron (MLP). The value of the neurons in the first layer of the MLP, which is called the input layer, are the inputs of the MLP. The last layer is called the output layer, as the output from the artificial neurons in this layer are the outputs of the total MLP. The layers in between are called hidden layers; see Figure 1 for a graphical representation of an MLP. An MLP with multiple hidden layers is called a deep neural network (DNN) [14].

During the training, the weights of the artificial neurons are altered in a way such that the input from the DNN matches the desired output. This is performed by means of an optimization algorithm. Consider a DNN $A_1$ that is trained by a set of input vectors $\mathit{X}=\left\{X_1, \dots ,X_n\right\}, X_i\in {ℝ}^m$ and a set of output vectors $\mathit{Y}=\left\{Y_1, \dots ,Y_n\right\} , Y_i\in {ℝ}^k$, where each output is assigned to one input $Y_i\to X_i$. The inputs and outputs are combined in the training data $D_{\mathrm{train}}\left(\mathit{X},\mathit{Y}\right)$. If $A_1$ is given an $X_i$ as an input, $A_1$ can be called to estimate ${\widehat{Y}}_i$, which can be written as$A_1\left(X_i\right)={\widehat{Y}}_i$. To quantify how well the estimations are matching with the desired outputs, a loss function $ℒ$ is used, e.g., the root mean square error (RMSE), which is defined as

$${ℒ}_{RMSE}\left(\mathit{Y},\widehat{\mathit{Y}}\right)=\sqrt{{\displaystyle \sum }\frac{{\left(Y_i-{\widehat{Y}}_i\right)}^2}{n}}$$

(1)

For the initialization, random values are assigned to the weights from the weighted sum in all neurons. Consider multiple DNNs $A_j$ using the same architecture and a random, and thus different, initialization. The training is performed using the same training data $D_{\mathrm{train}}\left(\mathit{X},\mathit{Y}\right)$, but the resulting $A_j$ differ. While after a successful training the loss function is minimized and $Y_i\approx {\widehat{Y}}_i$ can be assumed for all $A_j$, different $A_j$ are explained by using a gradient decent optimization algorithm for the training, which minimizes the loss function and can only find local optima. Therefore, different optima are found [15,16]. ANNs show substantial differences in weights in intermediate and higher-level networks with more than six layers, despite similar performance [17].

A DNN with a special architecture can be used as an encoder–decoder for lossy compression of data. The encoder–decoder (ED) has a small central layer and is trained to reconstruct input data $\mathit{X}$ as output $\mathit{X}=ED\left(\mathit{X}\right)$, as shown in Figure 2. The values of the small central layer are called compressed feature vector (CFV) $\mathit{Y}$. The CFV is significantly smaller than the input and output. The size of the CFV is a main factor on how well the output can be recreated from the CFV [18]. The encoder–decoder can be split into two DNNs, the encoder $E$ and the decoder $D$. $E$ uses data $\mathit{X}$ to generate CFV $\mathit{Y}=E\left(\mathit{X}\right)$ and $D$ uses CFV to estimate data $\widehat{\mathit{X}}=D\left(\mathit{Y}\right)$ [19].

The ED is chosen for the proposed method because it contains a way of unsupervised learning using algorithms for supervised learning. This is intended to allow for ease of connection with the other ANNs that use supervised learning.

2.3. Recurrent Neural Networks (RNN)

Recurrent neural networks (RNN) are ANNs that can use information from previous calls. This makes it possible for RNNs to analyze data with respect to past inputs and makes estimations dependent on previous calls. A simple implementation uses an artificial neuron, where the output from the previous call is going to be an additional input for the next estimation. With this implementation, the problem of vanishing gradients occurs. This means that the output from the previous call has to be mostly defining the state of the next call, if the neuron needs to keep its state over a larger number of calls [20]. A type of cell which can solve this problem is the long short-term memory (LSTM) [21]. Here, the cell is replaced by multiple cells with well-defined tasks. One of the cells is the memory cell, where the output is an input for next call with a fixed weight of one. Therefore, the problem of vanishing gradients cannot occur, as the state is transported over various time steps without vanishing or growing exponentially [20]. This property predestines LSTM for the proposed method, since patterns found in the observational data should not be lost due to vanishing gradients.

2.4. Physics-Informed Machine Learning (PIML)

The amount of data needed to train a neural network can be reduced by including physical knowledge in machine learning. This physical knowledge includes laws or observations typically found through extensive scientific work in the form of experimentation, modeling, and other measures. This knowledge is called domain knowledge (DK) [22]. The more DK that is used, the fewer the data that are needed to train a neural network [10]. Several possibilities to include DK into neural networks exist. Physical laws can be added to the loss function as an extra term and can therefore penalize unphysical calls during training, called physics-guided neural networks (PGNN) [23]. Kapusuzoglu et al. [10] compared different approaches of PIML in the context of 3D printing. Based on this comparison, the approach shown in Figure 3 was selected for the developed method in this paper. In this approach, neural networks are pretrained with estimates based on DK of inputs and outputs. The network is pretrained to gain knowledge of how certain input parameters must be combined to obtain the desired results. Physical laws are not enforced in further training and are only an initialization for the neural network to learn fundamental correlations [10]. This feature is important to adapt neural networks to a wide range of 3D printers.

3. Developed Method

In this paper, the developed method aims at optimizing system reliability with changing requirements by predicting outcome and suggesting input variables. Reliability can be ensured without further training by transferring and enriching DK on conducted observations, though neural networks can only generalize from data that were used to train the neural networks. Predictions from neural networks can only be as informed as DK and observations are.

3.1. Prediction Versus Suggestion

Consider a system $S_a$ which uses a vector of input parameters $X$ to generate a set of target values $Y=S_a\left(X\right)$. Essentially, for a reliable operation, it is important that target values $Y$ must be achieved within certain limits. Hence, for an explicit system $S_a$, an explicit vector $X$ that achieves this target value $Y$ needs to be identified. This could be achieved directly by a neural network suggesting $X$. As in most cases some parameters in $X$ cannot be set freely, the presented method is developed to solve this problem. To identify an explicit vector $X$, an optimization algorithm is better suited as it can satisfy constraints in the form of mathematical equations and optimize the target values, see Figure 4. Thus, the aim of this work is to generate predictions $\widehat{\mathit{Y}}$ for the system $S_a$ and freely chosen $\mathit{X}$ that can be used in an optimization algorithm to identify an explicit $X$ with a desired estimation for $\widehat{Y}.$ By using this method, target values and, thus, system reliability can be optimized. Since constraints depend on the application, further considerations can be found in Section 4, where the application to additive manufacturing is discussed.

3.2. Predictions of Target Values

For a system $S_a$, all observations of $S_a$ should have a direct influence on the predictions $\widehat{\mathit{Y}}$. This is ensured using a recurrent neural network $R$ that creates an estimation on the general reactions of $S_a$ based on all observations ${\left({\mathit{Y}}^{\mathrm{obs}}\to \mathit{X}\right)}_a$. In this paper, this is called behavior and is quantified in the behavioral vector $B_a$, which is estimated by calling the RNN $R$ as follows:

$${\widehat{B}}_a=R\left(\mathit{X},{\mathit{Y}}^{\mathrm{obs}}\right).$$

(2)

After that, a feed-forward DNN $F$ is called to generate an estimation of the target values:

$$\widehat{Y}=F\left(X,{\widehat{B}}_a\right).$$

(3)

To calculate estimations for different $X$, only the network $F$ is used, while ${\widehat{B}}_a$ stays the same until new observations ${\mathit{Y}}^{\mathrm{obs}}\to \mathit{X}$ are made. The whole process of predictions is shown in Figure 5.

3.3. Behavioral Vector

To define the behavioral vectors $B$ for various known systems, a DNN encoder–decoder is used. The encoder–decoder $ED$ is trained using predictions for the target value ${\widehat{\mathit{Y}}}^{\mathrm{LHC}}$ that is estimated using the DNN network $F$ and vectors ${\mathit{X}}^{\mathrm{LHC}}$ that are, respectively, derived from Latin hypercube sampling:

$${\widehat{Y}}^{\mathrm{LHC}}=F\left(X^{\mathrm{LHC}},{\widehat{B}}_a\right)$$

(4)

For the training of $ED$, the prediction for the target values ${\widehat{\mathit{Y}}}^{\mathrm{LHC}}$ are used as an input and output. $ED$ encodes the input to a CFV and decodes it afterwards. The CFV is defined as the behavioral vector $B$, because the CFV contains the necessary information to recover the prediction ${\widehat{\mathit{Y}}}^{\mathrm{LHC}}$. After training the DNN $ED$ is split in the encoder DNN $E$ and the decoder DNN $D$ to calculate $B$.

$$B=E\left({\widehat{Y}}^{\mathrm{LHC}}\right)$$

(5)

$${\widehat{Y}}^{\mathrm{LHC}}=D\left(B\right)$$

(6)

The complete usage of the encoder–decoder $ED$ is illustrated in Figure 6.

4. Method Transfer

The developed method is designed to ensure the system reliability in operating 3D printers used for mass production. In this case, disturbance variables, materials, and optimization targets are constantly changing, while in an industrial environment, high demands on quality are placed [24]. In the context of this work, print defects in additive manufacturing mean that the printing process must be canceled, mechanical properties are not met, dimensional accuracy could not be kept, or optical defects are present in the 3D-printed product [1]. Due to these print defects, the printed parts must be reprinted.

The influence of a single parameter on an outcome is called the total effect. The total effect consists of interaction effects and the main effect. The main effect is the influence of the parameter, which depends only on the parameter itself. Interaction effects are also dependent on other parameters. The order of an interaction effect indicates how many other parameters are involved.

In practice, there are guidelines (manufacturer´s instructions) to avoid print defects by changing printing parameters. One larger collection of print defects by Richter [25] names more than 200 total effects of 18 input parameters on 40 print defects while mechanical properties as print defects are not considered. All of these total effects are documented to be monotonic, and the general direction of the changing outcomes of the effects is claimed to be applicable to all FFF 3D printers, regardless of the printing material and printer design. According to Richter, some geometric features seem to have effects on print defects.

Scientific literature also names interaction effects [26] and non-monotonic effects [27] for 3D printers. The summarized results of those studies show that the avoidance of print defects could lead to opposing goals, such as surface roughness and dimensional accuracy, i.e., [26]. In consequence, the reliable operation of 3D printers is a complex task, as in the best case, all effects must be known in order to counteract print defects and to ensure high-quality parts.

As there are presumably more than 200 total effects on more than 40 print defects, an intelligent, automated process must be developed to adjust the print parameters just in time. This should enable the printing system to perform more efficiently concerning time and costs than is the case so far, and ultimately ensures the system reliability. The effects may be non-monotonic, so a simple linear regression is insufficient. Interaction effects make statistical investigation labor-intensive, as rapid screening methods are not sufficient, as they can only be used to determine total effects. The method developed and explained in the previous sections highly encourages knowledge transfer among systems, which is convenient given that the effects are comparable among 3D printers. By using an optimization algorithm, it is ensured that opposing goals could be weighted and individually adjusted to the printed part. The results of the method transfer to 3D-printing are shown in the following.

4.1. Application and Results

To apply the developed method on FFF 3D printers, the general system must be defined. Three-dimensional printers are considered individual systems, and with a substantial change of the 3D printer, a new system is formed. This can be a modification such as a changed print bed or print nozzle. Each system requires printing parameters, printer parameters, and disturbance variables $X$ to produce the outcomes $Y$. Print parameters are controllable parameters for the print, which can be optimized. Disturbance variables contain all parameters that can be measured or determined, such as ambient temperature or print time since the last maintenance, e.g., printer parameters are specific for the printer, such as, for example, the printer model or the nozzle diameter. Product parameters are, for example, geometric features of the printed product, such as the maximum overhang angle and the contact surface of the product on the print bed. The outcome contains all print defects and some other metrics, such as production time and used material for the final product. Overall, 75 printing parameters, 4 disturbance variables, 8 printer parameters, 9 product parameters, 40 print defects, and 4 print metrics are recorded for each observation of a print process.

The possible occurrence of certain printing defects related to geometry can be derived from the product parameters of the printed product. Combined with known customer requirements, the required outcome of a successful print and the optimization criterion can be created. This criterion is used for the optimization to find print parameters for a successful print and thus a reliable operation of the printer.

4.2. Use of Domain Knowledge (DK)

This paper addresses a well-known and crucial problem, i.e., the print bed adhesion [21], to show the method transfer exemplarily. If the print bed adhesion could be controlled, the printed part can be easily detached from the print bed after the print. This is crucial in order to enable an automation of the printing process and to make this technology more available for mass production.

Following the general initialization process explained in Section 2.4, DK must be defined based on scientific research on the subject. This is performed via various measurements and datasets of the print bed adhesion taken from scientific literature. Two publications suitable for this application are particularly noteworthy. The first publication is written by Kujawa [28], who investigated the influence of the first layer print parameters on the print bed adhesion. The second publication is written by Spoerk et al. [27], who optimized the print bed adhesion by varying print parameters. The problem with using literature data is that not all print parameters and disturbance variables are known for these measurements, so no complete observations (see Figure 7) can be created. In the next paragraphs it will be explained how DK can still be taught to neural networks.

In the first publication, by Kujawa [28], only total effects were measured in a one-factor-at-a-time design of experiments. Measurements were repeated, and usable data are given in the publication. To use the data in the developed method, synthetic observations are created from the data. This is achieved using linear interpolation between the measurements. See Figure 8a for a graphical representation.

In the second publication, by Spoerk et al. [27], linear main effects and interaction effects were calculated for three print parameters in a full-factorial design of experiment. In a second step, the printer bed temperature was singularly investigated from 20 °C to 80 °C in steps of 10 °Celsius. For the pretraining of the ANN used in the proposed method, these two designs of experiments are combined using superposition, as can be see in Figure 8b.

The authors of both publications have chosen parameters that are expected to have the biggest effects on the printer bed adhesion. Not-investigated effects, which are to be expected, should be equal to or smaller than the investigated effects. ANNs learn about important effects with DK, resulting in fewer experimental data needed until ANNs are successfully trained [29].

In order to train the first neuronal net $F^{\mathrm{phy}}$ using DK, the behavioral vector $B_i^{\mathrm{bin}}$ must be set, which is a binary representation of the system number. In this case, it is a vector of the length 1 with values 0 and 1. Synthetic data generation and initialization is carried out as described. Afterwards, FFF 3D printers are used and the observations should be documented in the format shown in Figure 5 in Section 4.1. The observations can then be used to update $F^{\mathrm{phy}}$ based on DK to $F$. Suggestions for print parameters can be generated using a simple gradient-based optimization algorithm with the predictions from $F$, as described in Section 3.1 and Figure 4.

4.3. Critical Discussion

The aim of the proposed method is to ensure system reliability in operating 3D printers. Thus, systematic errors are avoided by suggesting optimal parameters. A reliable operation is strongly dependent on the suggested parameters and, as a result, strongly dependent on the accuracy of the underlying prediction. With this in mind, the accuracy of the prediction can be used as an indicator for system reliability.

There are also other approaches to ensure the reliable operation of 3D printers. These approaches are the intuitive and the statistical approach that are discussed in the following. In an intuitive or popular approach, which is widely used in the non-scientific literature, the print parameters are only changed when print defects occur. Working print parameters are stored for different materials, conditions, and 3D printers. In this approach, the total effects are used only qualitatively, although most parameters have effects on multiple defects. The result is a trial-and-error analysis that can lead to long setup times due to various print defects, since different print defects can occur after each step [25].

With the newly developed method, setup times should be shortened, because if working as intended, all print defects should be predicted and avoided simultaneously. On the contrary, for the intuitive approach only the total effects are known qualitatively, which gives only an indication of how to adjust print parameters to avoid individual print defects. A reliable operation cannot be ensured, since the known global total effect could be locally incorrect. With the developed method, on the contrary, the print parameters can be adjusted before print defects occur.

In direct comparison of the proposed method with the widely used manual prevention of print defects, the proposed method has the potential to save setup time, to automatically avoid print defects, to ensure the reliability of the printing process, and thus improve the time and cost efficiency of the printing process.

Currently, one disadvantage of the newly developed method is certainly the initialization and updating process. However, since this initialization phase only needs to be performed once, the overall result could be a more efficient and reliable process that ensures the possibility of automating 3D printing in the future.

For example, a DNN can be combined with LHS to predict print defects using statistics. The statistical study of a 3D printer is lengthy but possible, even with more than 90 input parameters and more than 40 output parameters. In comparison, the proposed method uses previously published observations and further DK to predict print defects of a single 3D printer with fewer observations. In a statistical approach, a study is conducted at a particular time and under particular conditions. Any deviation from these conditions can lead to inaccurate predictions and thus a decrease in reliability. This can only be avoided by further investigation or by transferring knowledge from a single 3D printer to all other 3D printers and printing materials. Knowledge transfer could be an essential part of the proposed method and is achieved by creating a behavioral vector from the observations of one 3D printer. This vector is used to make predictions for all 3D printers using the same neural network. As conditions change, the behavior of the 3D printer changes and the behavioral vector could be adjusted by the neural network. Thus, compared to a statistical approach, the proposed method requires fewer observations and no manual statistical analysis is required to transfer knowledge. Another advantage is that no retraining is required to derive new predictions from new observations. As a result, the reliability of 3D printing is increased because real-time knowledge transfer allows real-time response to changing conditions. As it is summarized in

Table 1. Comparison among different approaches to specify parameters for the reliable operation of 3D printers.

	Intuitive Approach Using Qualitatively Known Total Effects	Statistical Approach	Proposed Method
Functionality	Total effects give multiple possibilities to eliminate print defects.	Predictions are based on previous observations to avoid print defects.	Predictions are based on observed and quantified behavior to avoid print defects.
Observations needed for setup	None, only DK is used.	Most, because a full sensitivity analysis is needed.	Less than the statistical approach, because DK is used.
Observations needed for reliable print	Eliminating print defects often leads to other print defects.	None.	Enough to quantify behavior.
Transfer of knowledge	Qualitatively known total effects are mostly universal.	Deviations in print process could make all observations obsolete.	Integrated through quantification of behavior.

.

Other approaches of mitigating print defects exist as well, such as closed-loop control [5] or ANN-based optimizing of process parameters [8,10]. These approaches have in common that they are supposed to eliminate certain or only a few defects at the same time. While these approaches might be better used to eliminate individual defects, it is unclear how these systems work together to eliminate all print defects. FFF 3D printers have a large number of parameters that influence numerous defects. The presented method aims to avoid all defects at the same time by only setting print parameters. Therefore, not all problems could be solved by the proposed method, such as mechanical problems, but it could be combined with other approaches as the 3D printers are not altered in any way.

4.4. Case Study

A case study is conducted to show the effectiveness of the proposed method. On four identical and unmodified Prusa i3 MK3s, 3D printers in a temperature- and humidity-controlled room without direct sunlight, 1273 print bed adhesion measurements are taken. Prior to printing, the print beds are cleaned using an alcohol-based solvent and a fresh paper towel every day, and each printer used one black PLA filament coil from Verbatim for all prints. For the measurements, a basic geometry is printed because our focus lies on print bed adhesion. The print bed is left to cool down to 35 °C after printing and the printed test cubes with an edge length of 10 mm are slowly pulled horizontally from the print bed using a pull arm with a worm gear driven by a stepper motor. The resulting force is recorded at 48 KHz and the maximum value is noted.

These measurements are part of an LHS experimental design, which includes 400 experiments; each comprises a minimum of three measurements, resulting in a total of 1273 print bed adhesion measurements. Within the design of experiments, 75 print parameters are varied and the print bed adhesion is recorded. Parameters which directly influence support structures, overhangs, or bridges are not chosen because of the focus on print bed adhesion. Temperature, relative humidity, prints since last print bed cleaning, and prints since last calibration are recorded as disturbance variables. The test data comprise 80 experiments with 210 measurements. The evaluation in the comparison is derived based on the prediction accuracy of the said test data with the loss-function ${ℒ}_{RMSE}$.

A comparison with the intuitive approach is not conducted, since it strongly depends on the person choosing print parameters and the prognosis of the printing defects is not calculated. Therefore, the proposed method is compared with a possible statistical approach from the last subsection. A similar approach is used in the statistical approach as in the proposed method for better comparison. A DNN is trained with the same data and nearly the same architecture as the DNN in the proposed method and with this comparison, the influence of the PIML approach in the proposed method can be determined due to the similarities of the two methods.

In the proposed method, the training of the encoder–decoder failed, arguably because of the low number of 3D printers in the case study, which were also very similar. As a result, the redefinition of the behavioral vector, as described in Section 3.3, was not carried out. In Figure 9, a comparison of RMSE in prognosis of the print bed adhesion is illustrated.

Around 3000 physical experiments as training data are created for pretraining for the proposed PIML approach, which are used 50 times for training. In addition to the pretraining with the physical experiments of the proposed PIML approach, both approaches used the real measurements 50 times for training. For every set of values in the figure, each approach was carried out and evaluated 10 times. Data and results can be found in the supplementary materials [30] linked below. High variance occurs in the evaluation of the statistical approach, when the number of measurements for training is low. The variance of the evaluated RMSE is mostly 50 to 100 times bigger in the statistical approach when compared to the proposed method. The variance as well as the prediction error is similar, with around 350 or more measurements available for training. The spike around 10 measurements in the proposed method is caused by some consecutive unexpectedly high measurements in the training data. In the case study, the proposed method has a lower RMSE for predicting unknown experiments while knowing only a limited amount of measurements. This indicates that transfer learning from domain knowledge is working, but with an increasing number of measurements, the effect disappears.

5. Conclusions

In this paper, a method for optimizing the system reliability of FFF 3D printers is presented. The developed method optimizes reliability by suggesting optimal print parameters using physics-informed machine learning. Print defects such as insufficient mechanical properties, dimensional accuracy, and aesthetics are avoided simultaneously by optimizing predictions from ANNs. Based on real-world experiments, a behavioral vector is determined for an individual 3D printer which is used to predict the system outcome of a 3D printer. Literature and first experiments on FFF 3D printers suggest that the total effects of the printing parameters on printing defects remain qualitatively the same for different materials and printers. This suggests that for accurate predictions, only a few observations are needed for each individual 3D printer when DK is used. In consequence, accurate predictions ensure the reliable operation of 3D printers by avoiding multiple print defects simultaneously.

For the initialization of the neural networks, DK is used. In the selected literature, effects of print parameters on print defects are investigated. The investigations are used to pretrain the neural network. This way, quantitative effects between print parameters and print defects are learned by the neural network without using real-world observations.

Different approaches for a reliable operation of multiple 3D printer are compared and discussed. The intuitive and popular approach is to use qualitative knowledge of effects to avoid print defects. This can cause the appearance of different print defects in the setup. Consequently, with slowly changing disturbance variables, print defects may occur. When the proposed method is compared to a general statistical approach, the main advantages of the new developed method are the integrated knowledge transfer among different printers and materials as well as less-needed observations by the use of DK. This is shown with the implementation of the proposed method in the form of a case study on the parameter print bed adhesion. Initial results indicate that the transferability of domain knowledge with the proposed method is given with a few experimental results, but with an increasing number of experimental results, this effect decreases.

Three-dimensional printers can behave in unforeseen ways, and to quantify such behavior, future research is investigating a synthetic data generation method using DK in neural networks. The presented approach is extended to all print defects. All significant parameters must be determined and investigated in terms of their influence on the final result. DK must be researched and models for PIML have to be derived for all print defects. It must be defined how those defects can be measured appropriately. Possible combinations with other conventional methods could be researched, such as reverse modelling of the print products and mitigating geometric inaccuracies in the product model. Further research on the effects of the parameters of the proposed approach on the prediction accuracy is needed and will be implemented. Investigations of the capabilities and limitations of the transfer of knowledge will be carried out.