Next Article in Journal
Real-Time Human Activity Recognition on Embedded Equipment: A Comparative Study
Previous Article in Journal
Mechanical Analysis of Semi-Rigid Base Asphalt Pavement under the Influence of Groundwater with the Spectral Element Method
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Deep Learning Regressors of Surface Properties from Atomic Force Microscopy Nanoindentations

by
Luís R. L. Pacheco
1,2,
João P. S. Ferreira
1 and
Marco P. L. Parente
1,2,*
1
DEMec—Department of Mechanical Engineering, Faculty of Engineering, University of Porto (FEUP), Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
2
INEGI—Institute of Science and Innovation in Mechanical and Industrial Engineering, Rua Dr. Roberto Frias 400, 4200-465 Porto, Portugal
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(6), 2376; https://doi.org/10.3390/app14062376
Submission received: 21 February 2024 / Revised: 6 March 2024 / Accepted: 8 March 2024 / Published: 12 March 2024

Abstract

:
Atomic force microscopy (AFM) is a powerful technique to study the nanomechanical properties of a wide range of materials at the piconewton level. AFM force–indentation curves can be fitted with appropriate contact models, enabling the determination of material properties for a given sample. However, the analysis of large datasets comprising thousands of curves using conventional methods presents a time-intensive challenge. As a result, there is an increasing interest in exploring alternative methodologies, such as integrating machine learning (ML) models to streamline and improve the efficiency of this process. In this work, two data-driven regressors were tuned to predict the Young’s modulus and adhesion energy from force–indentation curves of soft samples (Young’s modulus up to 10 kPa). Both models were trained exclusively on synthetic data derived from the contact theories developed by Hertz as well as Johnson, Kendall and Roberts (JKR). The PyTorch library was employed to build and train the models; then, the key hyperparameters were refined by implementing the optimization framework Optuna. The first model was successfully tested with synthetic and experimental curves from AFM nanoindentations, and the second presented promising results on the synthetic data. Our work suggests that experimental data may not be essential for training data-driven models to predict surface properties from AFM nanoindentations. By delivering accurate predictions in a computationally efficient way, our regressors validate the potential of a deep learning approach in exploring AFM nanoindentations and motivate further development of similar strategies to overcome current limitations in AFM postprocessing.

1. Introduction

The last few decades have brought significant advancements in the field of nanotechnology at various levels. Several areas of research have greatly benefited from these improvements, with the study of materials at their nanoscale being substantially leveraged. Atomic force microscopy (AFM) is one of the main techniques responsible for these studies [1]. Urging from the need for visualizing and manipulating materials at their lower scale, this method was established in the late 20th century, following the concept of scanning probe microscopy [2]. It is based on a small probe, usually consisting of a cantilever with a tip that can have several geometries attached to its end. From the interaction of the probe with the sampled surface, whether involving contact or not, the target properties can be inferred [3,4,5]. Despite being a well-established method, there are still some challenges associated with AFM, particularly those arising from the contact between probe and sample. It has been shown that uncontrolled forces in constant-height contact mode can be reduced by magnetic actuation [6], while difficulties in determining the shape of the tip can be overcome using blind tip reconstruction methods [7,8]. Correctly identifying the contact point also remains one of the main challenges in AFM, prompting the development of diverse methods to refine this process [9,10].
High-resolution and three-dimensional images of the surface topography at the nanoscale can be generated by measuring the topographic height through the probe-sample interaction [11]. The height measurements not only allow for the reproduction of such images but they are also the basis for obtaining the roughness and other surface characterization parameters [12]. Furthermore, due to its wide range of operating modes, one can obtain nanomechanical properties that contribute to understanding the behaviour of materials at increasingly smaller scales. This is the case of AFM nanoindentations, which consist in approaching the probe towards the sample and indenting it for a defined threshold, followed by withdrawing the probe until it detaches from the sample or reaches its original position.
Along this ramp cycle, the deflection of the probe cantilever is recorded as a function of the probe’s displacement, which can later be translated into force–indentation (F-I) curves. By fitting these curves with appropriate contact models, the target material properties are finally obtained. Furthermore, if there is knowledge about the location in the sample where each F-I curve was obtained, it is possible to create surface maps for each measured property [13,14,15].
Regarding approach curves, they play a crucial role in inferring the elastic modulus of the sample and can be fitted with the models developed by Hertz [16] or Sneddon [17]. On the other hand, retraction data offer essential insights on its adhesion properties [18,19,20], which are particularly relevant in low-stiffness materials, such as biological cells, where they play a major role in processes such as cell proliferation and migration, and sudden changes in these properties can lead to diseases such as cancer [21,22] or osteoporosis [23,24]. Since during the retraction stage, adhesion forces between both surfaces are predicted to have a higher impact in the experiments, an Hertzian-based analysis is no longer precise enough, so other models must be employed. Amongst them are JKR (Johnson–Kendall–Roberts) and DMT (Derjaguin–Muller–Toporov) [25,26], which represent two extreme cases in adhesive interaction: while DMT is valid for hard samples, large tip radius and low adhesion forces, JKR meets the specific features of this work, since it is applicable for soft samples, small tip radius and high adhesion forces [27].
Another field that has witnessed remarkable advancements in recent times is artificial intelligence (AI) and its subset of machine learning (ML), taking advantage of the exponential increase in computational power. Being able to adapt to different sorts of data allows ML to be applied across various fields, from medicine and engineering to finance or marketing. So, it only makes sense that it can also be used to boost techniques such as AFM, mainly in the postprocessing stage.
Although contact mechanics models have proven to be great tools for the study of surface properties from AFM nanoindentations, particularly in the domain of soft biological tissues, this traditional approach can be a time-consuming task. Hence, new strategies have been explored to overcome this limitation. Physics-informed ML models represent an alternative capable of taking advantage of contact mechanics theories to guide their development. By splitting AFM F-I curves in their two ramp cycle stages—approach and withdraw—and selecting an appropriate contact theory for each, it is expected that the synthetically generated data will have a fair resemblance of experimental curves. Another drawback associated with many contact models is their assumption of linear elastic behaviour. This limitation can also be addressed by employing ML models, which can easily capture material nonlinearity.
In recent years, several works have introduced different approaches that utilize ML methods to assist in diverse types of AFM analyses. A ML approach based on artificial neural networks (ANNs) is used in [28] to classify bladder cancer cells into different grades, based on cellular mechanical properties obtained with AFM. For an application more related to AFM intrinsic properties, Convolutional neural network (CNN) models have been developed to determine the tip sharpness directly from indentation images [29]. A different study [30] has presented a quasi-recurrent neural network to identify the coupling of vibrating modes in dynamic AFM (intermittent contact between probe and sample). At last, ML regression models were used to predict the elastic modulus based on nanoindentation curves, without having to fit them with a contact model, in the work presented in [31], where the ML models were trained with experimental F-I curves from AFM analyses. This last work shares some similarities with the current project, in the sense that they both focus on determining the sample’s stiffness without contact model fitting. Nevertheless, the work in [31] does not cover the development of deep learning strategies and it requires experimental data to train the models, unlike the current framework.
Despite the several ML applications in the AFM field, there is not yet a deep learning-based approach to determine surface properties related to the elastic modulus and to the sample’s adhesion, from both the approach and retraction curves, without requiring the model to be trained on actual data from AFM experiments. This study aims to address this gap by examining the effectiveness of a similar method. This is performed by training the models only on synthetically generated data and then analysing if they can accurately predict surface properties from AFM-studied samples.

2. Methods

2.1. Hertzian and JKR Contact Models

Foundations of contact mechanics can be traced back to the work of Hertz [16], where the contact between two perfectly elastic solids or surfaces was described, disregarding friction or adhesion. Considering two spherical bodies, the force obtained from such contact was derived as
F = 4 3 E * R 1 2 δ 3 2 ,
where R = ( 1 / R 1 + 1 / R 2 ) 1 accounts for the radii of curvature of both solids and δ is the contact displacement, corresponding to the indentation value in AFM studies. The effective modulus E * is defined with the elastic modulus (E) and Poisson ratio ( ν ) of each solid:
E * = 1 ν 1 2 E 1 + 1 ν 2 2 E 2 1 .
In the case of AFM nanoindentations with a spherical tip ( R 1 = R tip ), the sample can be often regarded as an elastic half-space ( R 2 = R sample ), so the radius of curvature is simplified to R = R tip . When the sample is a soft biological tissue, the tip may be treated as a rigid body, such that the effective modulus will only depend on the sample’s properties ( E 2 = E and ν 2 = ν ): E * = E / ( 1 ν ) 2 .
Many works [17,32,33,34,35] have been developed to achieve an accurate relation between applied force and indentation depth, supported on Hertzian principles, for a wide range of AFM tip shapes. Despite other accurate approaches for determining this relation, a closed-form solution will be implemented, based on applying a correction factor to Expression (1):
F = 4 3 E 1 ν 2 R 1 2 δ 3 2 Z ,
with Z, as suggested in [36], being approximated by
Z = 1 1 10 δ R 1 840 δ R 2 + 11 15120 δ R 3 + 1357 6652800 δ R 4 .
These relations will allow for the generation of curves for the approach stage of AFM nanoindentations, just by defining the tip radius and material properties for each sample. Regarding the withdraw stage, JKR contact theory translates the effect of adhesion energy γ (in units of energy per area) upon adhesion force and contact size, reaching the following expression for the normal force of contact:
F = R c 3 K R 6 π K γ R c 3 ,
in which K = 4 / 3 E * and R c stands for the contact radius, which is a function of indentation depth and normal force itself, so it does not provide an explicit formulation relating force and indentation. One way to tackle this problem is by approximating the contact radius as R c = R δ [37], so that a simpler closed-form solution is obtained:
F = K R 1 2 δ 3 2 U a K 1 2 R 3 4 δ 3 4 ,
where U a = 6 π γ . Accordingly, to obtain F-I withdraw curves, the tip radius must be defined, in addition to the material properties, comprising the Poisson ratio, Young’s modulus and adhesion energy.

2.2. AFM Experimental Data

Surface properties of a given sample can be extracted through the analysis of AFM nanoindentation data. In this work, the Young’s modulus and adhesion energy will be the main focus. To transform raw AFM data into force–indentation (F-I) curves, specific procedural steps must be followed. As depicted in Figure 1, while the probe approaches and withdraws from the sample, a photodetector records the voltage as a function of piezo position, z p (vertical displacement of the cantilever).
By knowing the system sensitivity, the voltage can be translated into cantilever displacement, z c . The tip-sample force is calculated with the cantilever stiffness, k, applying Hooke’s law:
F = k · z c .
The indentation depth, δ , relates to the cantilever deflection, z p position and the piezo position at contact point, z 0 , by
δ = ( z p z 0 ) z c
We used available experimental AFM F-I data from primary non-epithelial vaginal cells, isolated from vaginal tissue harvested below the cervix from nulliparous LOXL1 knockout and wild-type mice (refer to [38] for detailed methodology). For all approach curves, the corresponding Young’s modulus (apparent elastic modulus) had been obtained by Hertz contact model fitting. As for retraction curves, no surface properties were accessible from them, leading to the exclusive use of approach data for testing the developed machine learning models.
The accuracy of each fitting from the original approach curve dataset was evaluated using the R 2 metric ( R 2 = 1 i = 1 n ( f i f ^ i ) 2 / i = 1 n ( f i f ¯ ) 2 ) to compare the experimental force, f i , with the predicted force by the contact model, f ^ i , for each indentation value. Instances with an R 2 < 0.9 were removed from the analysis, due to their likelihood of exhibiting experimental anomalies or simply not being suitable to be fitted with Hertzian theory. Some of these instances presented negative forces for high indentation values, while others even presented an unphysical negative Young’s modulus resulting from the fitting procedure. Then, all the curves with a maximum indentation lower than 150 nm (the maximum indentation defined for our synthetic dataset) were dropped and force–indentation points were uniformly removed from each curve until they reached a final value of 50 points, thus enabling their application as inputs for the ML models. Following these restrictions, a total of 24 , 304 curves were obtained, with the apparent elastic modulus distribution being presented in Figure 2a, while examples of individual F-I curves are shown in Figure 2b.

2.3. Synthetic Nanoindentation Curves

Before starting the development of a machine learning model, a suitable synthetic dataset must be defined, representing the underlying patterns of a real dataset obtained from experimental AFM nanoindentations in soft biological samples, for both approach and retraction stages. Hertzian theory-based Equation (3) was the foundation to compute the force for the approach stage, while Equation (6) was used for the withdraw phase. It is thus important to define what the fundamental variables are that will affect the forces for each step ( F Hertz and F JKR ), so it can be written that
F Hertz = F Hertz ( δ , E , ν , R )
and
F JKR = F JKR ( δ , E , γ , ν , R ) .
Therefore, all these variables must be defined to reach the final curves. The Poisson ratio and the tip radius will be the same for all instances, while the other parameters are unique for a specific curve. Thus, each indentation vector will have material parameters assigned to it, and jointly with R and ν will allow tone to compute the force for all indentation values in the vector. Knowing the force and indentation for each point, the curves can be obtained.
Starting with the Poisson ratio, as the targets of this model are soft tissues, they can be approximated as incompressible due to their high water content [39], so a Poisson ratio of 0.5 was assigned. The tip radius value, presented in Table 1, was chosen according to the probe that was used to produce the experimental nanoindentations [38] that will be later used to test the developed models, and it is within a common range of values for spherical tip radii.
Regarding indentation depth vectors, their length and the maximum indentation value must be defined, so that the inputs to the ML models are consistent. Only the contact region ( δ > 0 ) was considered, since negative indentation values do not yield any information on surface properties according to the contact models chosen.
In order for the indentation vectors to resemble real data more closely and to have a greater impact on the ML models, they cannot all be composed of the same evenly spaced indentation values, so different instances are associated to different indentation values, always keeping the maximum indentation at 150 nm.
After specifying the Poisson ratio, tip radius and a set of displacement vectors, the only remaining parameters are the material properties E and γ , which must also be comprised in a typical range of values for soft tissues. This work focuses on studying a specific subset of soft tissues, with surface properties compatible with the available experimental data. However, if a broader range of materials was being analysed, standardizing the units of the physical quantities could potentially be helpful to improve the model’s behaviour. For the Young’s modulus, its values commonly fluctuate between the orders of magnitude of 10 1 to 10 1 kPa, reaching the hundreds of kPa in more specific scenarios [27,40,41,42]. Taking also into account the Young’s moduli of the samples in the experimental data available, a minimum E of 0.2 kPa and a maximum of 10 kPa were fixed, using a triangular distribution function with the mode as 1.8 kPa, so that the ML models could be trained more thoroughly on the most frequent range of values for the Young’s modulus of biological cells or hydrogels.
On the other hand, the same dataset does not contain any information about adhesion energy, since the available retraction curves are scarce and the corresponding samples did not necessarily exhibit strong adhesion properties. Hence, its range of values was searched in existing literature regarding biological samples [18,43,44,45]. The values found go from a few μ J/m2 to dozens of mJ/m2, making it difficult to establish a shorter interval. In addition, the samples in the available experimental dataset did not necessarily all have adhesive properties with high values, as the ones in the works mentioned. Thus, we sought to obtain a range of values that would somehow illustrate the relationship between the approach and retraction curves, considering the Young’s moduli range found in the dataset. Hence, this parameter was studied with an uniform distribution in a lower range of 1–3 μ J/m2. Figure 3 summarizes the relation between approach and retraction curves of the synthetic dataset.
At last, the dataset size must be carefully chosen. This will play a critical role in the development of our models, as insufficient data can lead to poor model performance and overfitting. In contrast, if the dataset is too extensive, its computational complexity increases, leading to much higher training times. In addition, there is also the chance of the model overfitting to noise in the data. In common ML applications, the number of instances in a dataset usually goes from a few thousand to hundreds of thousands, and it is no different when applied to AFM frameworks [31,46]. Thus, to ensure a robust initial model without excessively compromising computational costs, an initial dataset consisting of 40 , 000 curves was created.

2.4. Development of the Fully Connected Neural Networks

Given the absence of a definitive indication on whether analysing these curves demands a craftier model architecture, fully connected neural networks (FCNNs) were employed, with all the hyperparameters having undergone a thorough optimization process. Two deep learning models were developed: the first aiming to predict the Young’s modulus based on the approach stage of AFM F-I curves, and the second one taking the retraction curves as inputs and forecasting both the Young’s modulus and adhesion energy parameters. Hence, each model receives 100 inputs, corresponding to the indentation and force values for the 50 points in each curve. These models are referred to as the Hertz FCNN (one output—E) and JKR FCNN (two outputs—E and γ ), respectively, aligning with the contact theories employed in generating the respective training data. PyTorch 2.0 [47] was used to implement and run the models.
Despite the models being trained and evaluated based on loss functions, to obtain a better understanding of the scale of the loss values and their real meaning in the comparison of predicted ( y ^ i ) and actual ( y i ) surface properties, the mean absolute percentage error ε ¯ will be employed, with ε being the percentage error for each instance: ε ¯ = 1 n i = 1 n ε i = 100 % n i = 1 n y i y ^ i y i .
Baseline hyperparameter configurations were set for both Hertz and JKR FCNNs, and a subsequent hyperparameter tuning process was conducted using Optuna [48], a Python package that expedites this procedure by leveraging the performance history during the training process. Prior to this implementation, a brief study involved exploring diverse combinations of loss and activation functions. The evaluation encompassed potential loss functions as mean squared error (MSE), mean absolute error (MAE) and Huber loss, alongside an exploration of activation functions, including ReLU and leaky ReLU.
Further optimization required the use of Optuna, with the adopted workflow being schematised in Figure 4. Each hyperparameter analysis in this package is referred to as a “study”, composed of a predefined number of trials. Within each trial, it is first necessary to choose the range of hyperparameters under analysis and to set an objective function, which in this case is to minimize the validation loss. A pruning-based approach was implemented, so that unpromising trials would be discontinued, based on the median validation loss of the previous trials at a given epoch.
In addition to the models’ width and depth, the learning rate, number of epochs, optimizer and batch size were the hyperparameters tuned. A study on the batch size is represented in Figure 5a, clearly indicating that a batch size of 16 is a better fit for training the model, as the lower losses were achieved with that value, in addition to a lower median loss across all trials. Four optimization stages were executed for each model. After a preliminary model evaluation in which the search region for each hyperparameter was refined, we evaluated the overall architecture and defined the model’s width. Figure 5b illustrates a study focusing on the final layer of the Hertz FCNN, with 16 and 32 neurons being the most frequent values chosen in the provided search region. Despite the loss median being slightly higher for 16 neurons, using 32 neurons in this layer generated at least 4 trials with a lower loss than the other configuration, hence being more likely to reach a better local minimum. At last, the most suitable learning rate was determined.
From the complete synthetic dataset (N = 40,000), 15% of the curves were reserved for testing, while the remaining dataset was evenly divided into six folds for cross-validation, where each fold was consecutively used for validation, while the remaining were employed for training. All the splits, including the one for the synthetic test set, factored in the stratification of the target variables (surface properties E and γ ). Throughout the following section, the MAPE ( ε ¯ ) and the percentage error ( ε ) were commonly used as evaluation metrics.

3. Results and Discussion

The final hyperparameters and model architecture obtained through the optimization process are presented in Table 2. Regarding the evaluation metric of the models, despite being easy to understand what is the impact of a certain relative error in the values of both target properties, it is not that intuitive to realize what its consequences are in an AFM F-I approach curve. To bridge this gap, regarding the Hertz FCNN, graphs presented in Figure 6a,b depict a comparison between the actual synthetic curves and the curves that would be produced with the predicted stiffness for that instance, obtained for different models developed along this work, with relative errors of 2 % , 10 % and higher than 20 % . For the JKR FCNN, it is also important to infer if one of the material parameters has a higher influence than the other. Figure 6c,d show that for the same F-I curves with similar relative error values for the Young’s modulus and adhesion energy, the first parameter has a higher impact on the predicted curves; hence, if we would have to prioritize the improvement on the prediction of one property, it would be the Young’s modulus.

3.1. Hertz Regressor

For each of the six data splits in the cross-validation procedure for the Hertz FCNN, optimal models were selected based on validation error and subsequently evaluated on both synthetic and experimental test sets. Figure 7 illustrates the distribution of percentage error values for the best model in each split. As expected, due to the strong affinity between testing and training data, predictions exhibit notably higher accuracy in synthetic data, with an average percentage error of ε ¯ = 0.67 % . This outcome aligns with expectations, considering that synthetic data closely mirrors the instances on which the models were trained. Nevertheless, the results obtained on experimental test sets are also noteworthy, indicating the potential for predicting surface properties from nanoindentation curves obtained in real-world scenarios. The mean percentage error across all splits is ε ¯ = 6.14 % , emphasizing the capability of models trained and validated without the inclusion of real data inputs. This underscores the effectiveness of the proposed approach in predicting surface properties using only synthetic data during the model training and validation phases.
It is worth noting that the model exhibiting the lowest accuracy on synthetic data among the six evaluated models (model from Split 1: ε ¯ s y n = 1.42 % and ε ¯ e x p = 7.83 % ) also demonstrated the poorest performance on real curves. This observation suggests an alignment between the accuracy on synthetic and experimental data. However, this correlation is not consistently observed across all splits. For the remaining splits, where the MAPE for synthetic curves consistently remained below 1%, no clear relationship emerged. Despite achieving high accuracy on synthetic data, variations in the accuracy of experimental predictions were observed in both positive and negative directions. This means that while a model’s accuracy on synthetic data may offer some insights, it does not consistently guarantee a good performance on experimental data across all scenarios, as the complex nature of the experiments may introduce additional factors influencing predictive outcomes, such as AFM artifacts, often related to parameters as the probe shape and laser interference, or even the nonlinear nature of the sample at the indentation depth being studied.
To obtain a better insight into the predictions made by our models, Figure 8 provides a detailed description of the results obtained with the best model on experimental data (from Split 3, ε ¯ e x p = 5.17 % ). For synthetic data, the predicted Young’s modulus values closely align with their true labels across the entire domain studied (Figure 8a). This results in 93.5% of the curves in the synthetic test set being predicted with a percentage error below 2.5%, with 66.2% of the instances exhibiting an error lower than 1% (Figure 8c). These highly accurate predictions confirm that the simple FCNN architecture adopted for the ML model development is well-suited for the problem at hand. While not reaching the precision observed in synthetic data, experimental predictions remain closely aligned with the real values. There is a slight tendency to overestimate the elastic modulus for curves with E > 2 kPa (Figure 8b). Employing the chosen model, 85.9% of the 24,304 experimental nanoindentation curves were predicted with an error lower than 10% (Figure 8d).

3.2. JKR Regressor

Due to the absence of fitted experimental retraction curves, the evaluation of JKR FCNNs was confined to synthetic data. Figure 9 illustrates the error distribution for the best model in each of the six splits, focusing on surface properties E and γ . A notable observation is that predictions of Young’s modulus were consistently more accurate than those of adhesion energy across all splits. However, both parameters exhibited highly satisfactory results, achieving average errors of ε ¯ E = 0.95 % and ε ¯ γ = 1.68 % across all folds. The selected architecture and optimized hyperparameters contributed to a uniform performance across all splits and for both material parameters. The exception was Split 6, where error values were notably higher compared to the remaining splits, yet still within the range of acceptable predictions.
Once again, we delve into a detailed analysis of the predictions made by our best model, as elucidated in Figure 10. This time, the selected model strikes a balance between the predictive capabilities for the two surface properties. We specifically opted for the model from the fifth split, which exhibited the most favourable results for the stiffness variable ( ε ¯ E = 0.50 % ). Although this model ranks second in predicting adhesion energy ( ε ¯ γ = 1.37 % compared to ε ¯ γ = 1.19 % in the model from Split 3), the substantial difference in Young’s modulus prediction errors ( ε ¯ E = 0.96 % ) justifies the preference for the model from Split 5. The predictions of the selected model for both material parameters indicate that, for Young’s modulus, they closely align with the true values across the entire domain. In contrast, the accuracy for adhesion energy is comparatively lower, with the number of observations deviating from ideal predictions increasing with the rise in γ (see Figure 10a,b).
Taking as a reference a residual error of 2.5% in surface properties, which holds little significance in the comparison between real curves and those obtained with predicted outputs, we observe that 97.9% of synthetic test set curves had an elastic modulus prediction below that threshold. However, this value decreases to 86.5% for adhesion energy predictions (see Figure 10c,d).

3.3. Discussion

The results presented for the Hertz FCNN validate its application in the postprocessing of AFM nanoindentation data of soft tissues. These findings are particularly significant, as certain curves may exhibit irregular shapes or be unsuitable for fitting with Hertzian mechanics due to their nonlinear nature or potential influence from AFM artifacts. Moreover, the FCNN provides predictions for indentation ranges up to 150 nm, surpassing the typical threshold analysed in classic model fitting approaches. Given that the models were trained on instances with a shared maximum indentation depth, their performance on curves with smaller indentations will not match the presented results. Nonetheless, a similar approach could be applied to train models using curves with varying indentation ranges. This would necessitate a substantially larger dataset to establish a robust and precise framework.
Regarding the JKR regressor, not having any experimental data to verify the behaviour of these models in a real application is certainly a downsize, which leaves space to question if they would be able to generalize to experimental curves. Nonetheless, it has been shown that a deep learning approach is able to easily capture the shape of a force–indentation curve generated through JKR theory. By ensuring that these synthetic curves are good representations of nanoindentation retraction curves, this all indicates that this is a promising framework to study the adhesion energy of a wide range of samples, whose research is much more scarce than that of the elastic modulus.
Despite the significance of the results presented, there remains considerable potential for further exploration in employing ML models to the field of AFM. Throughout this work, synthetic instances consistently shared the same tip radius and Poisson ratio. Nevertheless, these parameters could also be included as output variables, enabling models to be applicable across a broader range of experiments. To accommodate these variables, new models would need to incorporate them into the list of instance labels alongside the surface properties. Another challenge in AFM involves precisely identifying contact and detachment points. While an ML-based approach could be promising for addressing these difficulties, an effective model training may require experimental data with the location of these points precisely identified.

4. Conclusions

Throughout this work, two FCNN models were designed from scratch, with the goal of predicting the Young’s modulus and adhesion energy from AFM nanoindentation curves. These models were trained exclusively from synthetic F-I curves, previously generated using Hertz and JKR contact theories. As a proof of concept, such F-I generation was made in the domain of our available experimental data. A fine hyperparameter optimization technique was then applied to select the best model.
To evaluate the models’ performance, cross-validation was conducted for the optimized Hertz and JKR FCNN architectures. For every split, we assessed the model with the minimum validation error on both synthetic and experimental (limited to the Hertz FCNN) test sets. Concerning the Hertz FCNN, designed to predict the Young’s modulus from AFM nanoindentation approach curves, we achieved a mean absolute percentage error (MAPE) of 0.67% for the synthetic set and 6.14% for the experimental set, considering all the splits. As for the JKR FCNN, its assessment relied exclusively on synthetic data due to the unavailability of experimental curves. The model was designed to predict both elastic modulus and adhesion energy from AFM nanoindentation retraction curves, and it effectively fulfilled its objective, yielding low mean absolute percentage error values for both surface properties: ε ¯ E = 0.95 % and ε ¯ γ = 1.68 % . However, its future validation on real data is still required, such as future adjustments on how to define the detachment point on retraction curves creation to increase their affinity with experimental curves.
This study showcases the potential of improving the postprocessing of AFM nanoindentation curves through the application of straightforward deep learning architectures, dismissing the need for experimental data in model training. Exploring similar methodologies could be relevant to other AFM-related problems, such as determining the location of the contact and detachment points or generalizing the current framework for a wider spectrum of tip radii.

Author Contributions

Conceptualization, L.R.L.P. and J.P.S.F.; methodology, L.R.L.P.; software, L.R.L.P. and J.P.S.F.; validation, L.R.L.P. and J.P.S.F.; formal analysis, L.R.L.P.; investigation, L.R.L.P.; resources, J.P.S.F. and M.P.L.P.; data curation, L.R.L.P. and J.P.S.F.; writing—original draft preparation, L.R.L.P.; writing—review and editing, L.R.L.P., J.P.S.F. and M.P.L.P.; visualization, L.R.L.P.; supervision, J.P.S.F. and M.P.L.P.; project administration, J.P.S.F. and M.P.L.P.; funding acquisition, J.P.S.F. and M.P.L.P. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by FCT/MCTES under the scope of the project PTDC/EMEAPL/1342/2020. http://doi.org/10.54499/PTDC/EME-APL/1342/2020.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data supporting the conclusions of this article will be made available by the authors on request.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Voigtländer, B. Atomic Force Microscopy, 2nd ed.; Springer: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
  2. Binnig, G.; Quate, C.F.; Gerber, C. Atomic Force Microscope. Phys. Rev. Lett. 1986, 56, 930–933. [Google Scholar] [CrossRef] [PubMed]
  3. Radmacher, M. Chapter 4—Measuring the Elastic Properties of Living Cells by the Atomic Force Microscope. In Atomic Force Microscopy in Cell Biology; Jena, B.P., Heinrich Hörber, J., Eds.; Methods in Cell Biology; Academic Press: Cambridge, MA, USA, 2002; Volume 68, pp. 67–90. [Google Scholar] [CrossRef]
  4. Rico, F.; Roca-Cusachs, P.; Gavara, N.; Farré, R.; Rotger, M.; Navajas, D. Probing mechanical properties of living cells by atomic force microscopy with blunted pyramidal cantilever tips. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. 2005, 72, 021914. [Google Scholar] [CrossRef] [PubMed]
  5. Mahaffy, R.; Park, S.; Gerde, E.; Käs, J.; Shih, C. Quantitative Analysis of the Viscoelastic Properties of Thin Regions of Fibroblasts Using Atomic Force Microscopy. Biophys. J. 2004, 86, 1777–1793. [Google Scholar] [CrossRef] [PubMed]
  6. Lavanya, S.; Jayanth, G. Control of interaction force in constant-height contact mode atomic force microscopy. Mechatronics 2022, 88, 102914. [Google Scholar] [CrossRef]
  7. Tian, F.; Qian, X.; Villarrubia, J. Blind estimation of general tip shape in AFM imaging. Ultramicroscopy 2008, 109, 44–53. [Google Scholar] [CrossRef] [PubMed]
  8. Flater, E.E.; Zacharakis-Jutz, G.E.; Dumba, B.G.; White, I.A.; Clifford, C.A. Towards easy and reliable AFM tip shape determination using blind tip reconstruction. Ultramicroscopy 2014, 146, 130–143. [Google Scholar] [CrossRef]
  9. Moseson, A.J.; Basu, S.; Barsoum, M.W. Determination of the effective zero point of contact for spherical nanoindentation. J. Mater. Res. 2008, 23, 204–209. [Google Scholar] [CrossRef]
  10. Yang, C.W.; Chen, C.H.; Ding, R.F.; Liao, H.S.; Hwang, I.S. Multiparametric characterization of heterogeneous soft materials using contact point detection-based atomic force microscopy. Appl. Surf. Sci. 2020, 522, 146423. [Google Scholar] [CrossRef]
  11. Peta, K.; Love, G.; Brown, C.A. Comparing repeatability and reproducibility of topographic measurement types directly using linear regression analyses of measured heights. Precis. Eng. 2024, 88, 192–203. [Google Scholar] [CrossRef]
  12. Mwema, F.M.; Oladijo, O.P.; Sathiaraj, T.S.; Akinlabi, E.T. Atomic force microscopy analysis of surface topography of pure thin aluminum films. Mater. Res. Express 2018, 5, 046416. [Google Scholar] [CrossRef]
  13. Picas, L.; Rico, F.; Deforet, M.; Scheuring, S. Structural and Mechanical Heterogeneity of the Erythrocyte Membrane Reveals Hallmarks of Membrane Stability. ACS Nano 2013, 7, 1054–1063. [Google Scholar] [CrossRef] [PubMed]
  14. Thomas, G.; Burnham, N.A.; Camesano, T.A.; Wen, Q. Measuring the Mechanical Properties of Living Cells Using Atomic Force Microscopy. JoVE (J. Vis. Exp.) 2013, 76, e50497. [Google Scholar] [CrossRef]
  15. Touhami, A.; Nysten, B.; Dufrêne, Y.F. Nanoscale mapping of the elasticity of microbial cells by atomic force microscopy. Langmuir 2003, 19, 4539–4543. [Google Scholar] [CrossRef]
  16. Hertz, H. Ueber die Berührung fester elastischer Körper. J. Reine Angew. Math. 1882, 1882, 156–171. [Google Scholar] [CrossRef]
  17. Sneddon, I.N. The relation between load and penetration in the axisymmetric boussinesq problem for a punch of arbitrary profile. Int. J. Eng. Sci. 1965, 3, 47–57. [Google Scholar] [CrossRef]
  18. Zhang, Y.; Zhu, X.; Chen, B. Adhesion force evolution of protein on the surfaces with varied hydration extent: Quantitative determination via atomic force microscopy. J. Colloid Interface Sci. 2022, 608, 255–264. [Google Scholar] [CrossRef] [PubMed]
  19. Kim, H.; Yamagishi, A.; Imaizumi, M.; Onomura, Y.; Nagasaki, A.; Miyagi, Y.; Okada, T.; Nakamura, C. Quantitative measurements of intercellular adhesion between a macrophage and cancer cells using a cup-attached AFM chip. Colloids Surf. B Biointerfaces 2017, 155, 366–372. [Google Scholar] [CrossRef]
  20. Cross, S.E.; Jin, Y.S.; Tondre, J.; Wong, R.; Rao, J.; Gimzewski, J.K. AFM-based analysis of human metastatic cancer cells. Nanotechnology 2008, 19, 384003. [Google Scholar] [CrossRef]
  21. Huang, S.; Ingber, D.E. The structural and mechanical complexity of cell-growth control. Nat. Cell Biol. 1999, 1, E131–E138. [Google Scholar] [CrossRef]
  22. Hirohashi, S.; Kanai, Y. Cell adhesion system and human cancer morphogenesis. Cancer Sci. 2003, 94, 575–581. [Google Scholar] [CrossRef]
  23. Perinpanayagam, H.; Zaharias, R.; Stanford, C.; Brand, R.; Keller, J.; Schneider, G. Early cell adhesion events differ between osteoporotic and non-osteoporotic osteoblasts. J. Orthop. Res. 2001, 19, 993–1000. [Google Scholar] [CrossRef]
  24. Cho, P.; Schneider, G.; Kellogg, B.; Zaharias, R.; Keller, J. Effect of Glucocorticoid-Induced Osteoporotic-Like Conditions on Osteoblast Cell Attachment to Implant Surface Microtopographies. Implant. Dent. 2006, 15, 377–385. [Google Scholar] [CrossRef]
  25. Johnson, K.L.; Kendall, K.; Roberts, A.D. Contact mechanics. Proc. R. Soc. Lond. Math. Phys. Sci. 1971, 324, 301–313. [Google Scholar]
  26. Derjaguin, B.; Muller, V.; Toporov, Y. Effect of contact deformations on the adhesion of particles. J. Colloid Interface Sci. 1975, 53, 314–326. [Google Scholar] [CrossRef]
  27. Efremov, Y.; Bagrov, D.; Kirpichnikov, M.; Shaitan, K. Application of the Johnson–Kendall–Roberts model in AFM-based mechanical measurements on cells and gel. Colloids Surf. B Biointerfaces 2015, 134, 131–139. [Google Scholar] [CrossRef] [PubMed]
  28. Zhu, X.; Qin, R.; Qu, K.; Wang, Z.; Zhao, X.; Xu, W. Atomic force microscopy–based assessment of multimechanical cellular properties for classification of graded bladder cancer cells and cancer early diagnosis using machine learning analysis. Acta Biomater. 2023, 158, 358–373. [Google Scholar] [CrossRef]
  29. Zaki, M.; Kasimuthumaniyan, S.; Sahoo, S.; Jayadeva; Gosvami, N.N.; Krishnan, N.M.A. Interpretable Machine Learning Approach for Identifying the Tip Sharpness in Atomic Force Microscopy. Scr. Mater. 2022, 221, 114965. [Google Scholar] [CrossRef]
  30. Belardinelli, P.; Chandrashekar, A.; Wiebe, R.; Alijani, F.; Lenci, S. Machine learning to probe modal interaction in dynamic atomic force microscopy. Mech. Syst. Signal Process. 2022, 179, 109312. [Google Scholar] [CrossRef]
  31. Nguyen, L.T.P.; Liu, B.H. Machine learning framework for determination of elastic modulus without contact model fitting. Int. J. Solids Struct. 2022, 256, 111976. [Google Scholar] [CrossRef]
  32. Briscoe, B.J.; Sebastian, K.S.; Adams, M.J. The effect of indenter geometry on the elastic response to indentation. J. Phys. D Appl. Phys. 1994, 27, 1156. [Google Scholar] [CrossRef]
  33. Kontomaris, S.V.; Malamou, A. A novel approximate method to calculate the force applied on an elastic half space by a rigid sphere. Eur. J. Phys. 2021, 42, 025010. [Google Scholar] [CrossRef]
  34. Oliver, W.; Pharr, G. Measurement of hardness and elastic modulus by instrumented indentation: Advances in understanding and refinements to methodology. J. Mater. Res. 2004, 19, 3–20. [Google Scholar] [CrossRef]
  35. Menčík, J. Uncertainties and errors in nanoindentation. Nanoindentation Mater. Sci. 2012, 54, 53–86. [Google Scholar]
  36. Müller, P.; Abuhattum, S.; Möllmert, S.; Ulbricht, E.; Taubenberger, A.V.; Guck, J. nanite: Using machine learning to assess the quality of atomic force microscopy-enabled nano-indentation data. BMC Bioinform. 2019, 20, 465. [Google Scholar] [CrossRef] [PubMed]
  37. Del Cid, L.I. A Discrete Element Methodology for the Analysis of Cohesive Granular Bulk Solid Materials. Ph.D. Thesis, Colorado School of Mines, Golden, CO, USA, 2015. [Google Scholar]
  38. Ferreira, J.; Kuang, M.; Parente, M.; Natal Jorge, R.; Wang, R.; Eppell, S.; Damaser, M. Altered mechanics of vaginal smooth muscle cells due to the lysyl oxidase-like1 knockout. Acta Biomater. 2020, 110, 175–187. [Google Scholar] [CrossRef]
  39. Dimitriadis, E.K.; Horkay, F.; Maresca, J.; Kachar, B.; Chadwick, R.S. Determination of Elastic Moduli of Thin Layers of Soft Material Using the Atomic Force Microscope. Biophys. J. 2002, 82, 2798–2810. [Google Scholar] [CrossRef]
  40. Nguyen, T.; Gu, Y. Investigation of Cell-Substrate Adhesion Properties of Living Chondrocyte by Measuring Adhesive Shear Force and Detachment Using AFM and Inverse FEA. Sci. Rep. 2016, 6, 38059. [Google Scholar] [CrossRef]
  41. Lekka, M.; Laidler, P.; Gil, D.; Lekki, J.; Stachura, Z.; Hrynkiewicz, A.Z. Elasticity of normal and cancerous human bladder cells studied by scanning force microscopy. Eur. Biophys. J. 1999, 28, 312–316. [Google Scholar] [CrossRef]
  42. Gandin, A.; Murugesan, Y.; Torresan, V.; Ulliana, L.; Citron, A.; Contessotto, P.; Battilana, G.; Panciera, T.; Ventre, M.; Netti, A.P.; et al. Simple yet effective methods to probe hydrogel stiffness for mechanobiology. Sci. Rep. 2021, 11, 22668. [Google Scholar] [CrossRef]
  43. Tyagi, G.; Ahmad, Z.; Pellegrino, L.; Torquato, L.M.; Robles, E.S.; Cabral, J.T. Effect of surface energy on the removal of supported triglyceride films by a flowing surfactant solution. Surf. Interfaces 2023, 39, 102992. [Google Scholar] [CrossRef]
  44. Moreira, J.; Simões, M.; Melo, L.; Mergulhão, F. Escherichia coli adhesion to surfaces—A thermodynamic assessment. Colloid Polym. Sci. 2015, 293, 177–185. [Google Scholar] [CrossRef]
  45. Zhu, X.; Siamantouras, E.; Liu, K.; Liu, X. Determination of work of adhesion of biological cell under AFM bead indentation. J. Mech. Behav. Biomed. Mater. 2016, 56, 77–86. [Google Scholar] [CrossRef] [PubMed]
  46. Waite, J.R.; Tan, S.Y.; Saha, H.; Sarkar, S.; Sarkar, A. Few-shot deep learning for AFM force curve characterization of single-molecule interactions. Patterns 2023, 4, 100672. [Google Scholar] [CrossRef] [PubMed]
  47. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Adv. Neural Inf. Process. Syst. 2019, 32. Available online: https://pytorch.org (accessed on 22 January 2024).
  48. Akiba, T.; Sano, S.; Yanase, T.; Ohta, T.; Koyama, M. Optuna: A Next-generation Hyperparameter Optimization Framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019. [Google Scholar]
Figure 1. Experimental setup to obtain AFM force–indentation curves. The dashed cantilever represents its original, undeformed configuration, z c is the cantilever deflection and δ is the indentation depth.
Figure 1. Experimental setup to obtain AFM force–indentation curves. The dashed cantilever represents its original, undeformed configuration, z c is the cantilever deflection and δ is the indentation depth.
Applsci 14 02376 g001
Figure 2. Experimental approach curves used to test the ML models. Only the contact region ( δ > 0 ) is relevant for the analysis. (a) Young’s modulus distribution, mean and standard deviation values. (b) Examples of experimental approach curves in the available AFM nanoindentation data.
Figure 2. Experimental approach curves used to test the ML models. Only the contact region ( δ > 0 ) is relevant for the analysis. (a) Young’s modulus distribution, mean and standard deviation values. (b) Examples of experimental approach curves in the available AFM nanoindentation data.
Applsci 14 02376 g002
Figure 3. Distribution of the synthetic approach and withdraw curves for the 40,000 instances in the synthetic dataset.
Figure 3. Distribution of the synthetic approach and withdraw curves for the 40,000 instances in the synthetic dataset.
Applsci 14 02376 g003
Figure 4. Optuna framework scheme with pruning, where N is the number of epochs.
Figure 4. Optuna framework scheme with pruning, where N is the number of epochs.
Applsci 14 02376 g004
Figure 5. Box plots regarding hyperparameter optimization in Optuna, where each point represents a different trial from the Hertz FCNN studies. (a) Loss obtained as a function of batch size. (b) Study that determined the width of the model’s third layer.
Figure 5. Box plots regarding hyperparameter optimization in Optuna, where each point represents a different trial from the Hertz FCNN studies. (a) Loss obtained as a function of batch size. (b) Study that determined the width of the model’s third layer.
Applsci 14 02376 g005
Figure 6. Predicted and actual retraction curves for different relative errors of E and γ . (a,b) Synthetic approach curves and the corresponding curves obtained by using Equation (3), for different Young’s moduli. (c,d) The same three synthetic retraction curves are represented in both graphs, showing the influence of the same relative errors for each surface property, separately.
Figure 6. Predicted and actual retraction curves for different relative errors of E and γ . (a,b) Synthetic approach curves and the corresponding curves obtained by using Equation (3), for different Young’s moduli. (c,d) The same three synthetic retraction curves are represented in both graphs, showing the influence of the same relative errors for each surface property, separately.
Applsci 14 02376 g006
Figure 7. Evaluation of the Hertz FCNN on both experimental (blue) and synthetic (orange) test sets. Boxplots depict the error distribution for Hertz FCNNs with the lowest validation error in each split, with the lines that extend above and below each box representing 1.5 times the interquartile range. Horizontal lines translate the average percentage error across the best models for all splits, ε ¯ e x p = 6.14 % and ε ¯ s y n = 0.67 % .
Figure 7. Evaluation of the Hertz FCNN on both experimental (blue) and synthetic (orange) test sets. Boxplots depict the error distribution for Hertz FCNNs with the lowest validation error in each split, with the lines that extend above and below each box representing 1.5 times the interquartile range. Horizontal lines translate the average percentage error across the best models for all splits, ε ¯ e x p = 6.14 % and ε ¯ s y n = 0.67 % .
Applsci 14 02376 g007
Figure 8. Detailed results for the Hertz FCNN model with lowest error in the experimental test set. (a,b) Comparison between the model’s predictions and the actual Young’s moduli from synthetic (left) and experimental (right) test data. (c,d) Histograms for the percentage error distribution in the synthetic (left) and experimental test sets, outlining the percentage of curves with percentage errors below defined thresholds.
Figure 8. Detailed results for the Hertz FCNN model with lowest error in the experimental test set. (a,b) Comparison between the model’s predictions and the actual Young’s moduli from synthetic (left) and experimental (right) test data. (c,d) Histograms for the percentage error distribution in the synthetic (left) and experimental test sets, outlining the percentage of curves with percentage errors below defined thresholds.
Applsci 14 02376 g008
Figure 9. Results for the best model in each split of the JKR FCNN, for both material parameters: E (dark orange) and γ (light orange). Lines extending above and below each boxplot represent 1.5 times the interquartile range. ε ¯ E = 0.95 % and ε ¯ γ = 1.68 % were the average percentage error (horizontal lines) values obtained across the 6 splits.
Figure 9. Results for the best model in each split of the JKR FCNN, for both material parameters: E (dark orange) and γ (light orange). Lines extending above and below each boxplot represent 1.5 times the interquartile range. ε ¯ E = 0.95 % and ε ¯ γ = 1.68 % were the average percentage error (horizontal lines) values obtained across the 6 splits.
Applsci 14 02376 g009
Figure 10. Comprehensive outcomes for the JKR FCNN model with the best compromise between performance on predicting the Young’s modulus (left) and adhesion energy (right). (a) Predicted E values are in close agreement with the true labels. (b) Accuracy for adhesion energy predictions is acceptable, but lower than Young’s modulus predictions. (c) Distribution of percentage errors for material parameter E. (d) Histogram with the percentage error distribution for adhesion energy.
Figure 10. Comprehensive outcomes for the JKR FCNN model with the best compromise between performance on predicting the Young’s modulus (left) and adhesion energy (right). (a) Predicted E values are in close agreement with the true labels. (b) Accuracy for adhesion energy predictions is acceptable, but lower than Young’s modulus predictions. (c) Distribution of percentage errors for material parameter E. (d) Histogram with the percentage error distribution for adhesion energy.
Applsci 14 02376 g010
Table 1. Initial parameter definition for generating synthetic data.
Table 1. Initial parameter definition for generating synthetic data.
ParameterValue
Poisson ratio ( ν ) 0.5
Tip radius ( R ) 1980 nm
Max. indentation depth ( δ m a x ) 150 nm
Table 2. Hyperparameters and architecture of the optimized FCNNs.
Table 2. Hyperparameters and architecture of the optimized FCNNs.
Hertz FCNNJKR FCNN
Optimizer AdamAdam
Activation Leaky ReLUReLU
Loss HuberMSE
Epochs 80110
Learning Rate 2.3 × 10 4 3.0 × 10 4
Batch Size 1616
Hidden Layers 33
1st HL 256128
2nd HL 25632
3rd HL 32128
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Pacheco, L.R.L.; Ferreira, J.P.S.; Parente, M.P.L. Deep Learning Regressors of Surface Properties from Atomic Force Microscopy Nanoindentations. Appl. Sci. 2024, 14, 2376. https://doi.org/10.3390/app14062376

AMA Style

Pacheco LRL, Ferreira JPS, Parente MPL. Deep Learning Regressors of Surface Properties from Atomic Force Microscopy Nanoindentations. Applied Sciences. 2024; 14(6):2376. https://doi.org/10.3390/app14062376

Chicago/Turabian Style

Pacheco, Luís R. L., João P. S. Ferreira, and Marco P. L. Parente. 2024. "Deep Learning Regressors of Surface Properties from Atomic Force Microscopy Nanoindentations" Applied Sciences 14, no. 6: 2376. https://doi.org/10.3390/app14062376

APA Style

Pacheco, L. R. L., Ferreira, J. P. S., & Parente, M. P. L. (2024). Deep Learning Regressors of Surface Properties from Atomic Force Microscopy Nanoindentations. Applied Sciences, 14(6), 2376. https://doi.org/10.3390/app14062376

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop