A Physics-Informed Assembly of Feed-Forward Neural Network Engines to Predict Inelasticity in Cross-Linked Polymers

Abstract

In solid mechanics, data-driven approaches are widely considered as the new paradigm that can overcome the classic problems of constitutive models such as limiting hypothesis, complexity, and accuracy. However, the implementation of machine-learned approaches in material modeling has been modest due to the high-dimensionality of the data space, the significant size of missing data, and limited convergence. This work proposes a framework to hire concepts from polymer science, statistical physics, and continuum mechanics to provide super-constrained machine-learning techniques of reduced-order to partly overcome the existing difficulties. Using a sequential order-reduction, we have simplified the 3D stress–strain tensor mapping problem into a limited number of super-constrained 1D mapping problems. Next, we introduce an assembly of multiple replicated neural network learning agents (L-agents) to systematically classify those mapping problems into a few categories, each of which were described by a distinct agent type. By capturing all loading modes through a simplified set of dispersed experimental data, the proposed hybrid assembly of L-agents provides a new generation of machine-learned approaches that simply outperform most constitutive laws in training speed, and accuracy even in complicated loading scenarios. Interestingly, the physics-based nature of the proposed model avoids the low interpretability of conventional machine-learned models.

1. Introduction

The wide range applications of cross-linked polymers in several industries such as automotive, structural, medical, to name but a few, have made them an attractive area of research. These materials have a 3D network configuration consisting of randomly oriented long molecular chains, which are cross-linked, spiraled and tangled among themselves or neighbors. They are typically classified into filled and unfilled categories. Fillers, in most cases, can reinforce polymers (see Figure 1). Regarding the various applications of these materials, modeling their mechanical behavior in a broad range of strains is of great importance. In quasi-static deformations, these materials show hyper-elastic behavior. This behavior is dominantly governed by changes in network entropy, where the chains reorient in response to the applied macroscopic deformations. Farhangi et al. investigated the effect of fiber reinforced polymer tubes filled with recycled materials [1,2]. Izadi et al. investigated the effect of nanoparticles on mechanical properties of polymers [3,4,5]. Accordingly, several studies have investigated this hyper-elastic behavior based on phenomenological or micromechanical approaches which use statistics of molecular chains networks (see reviews [6,7]). Shojaeifar et al. [8,9] developed a model for modeling of visco-hyperelasticity of materials. Phenomenological approaches are empirical, simple, and less interpretable; however, micromechanical approaches are highly interpretable but complex because they consider the readjustment of kinks, the rearrangement of convolutions, reorientation, and uncoiling of molecular chains. Meanwhile, the emergence of machine-learned (ML) models has attracted much attention as a way to address the mentioned challenges of the phenomenological and micromechanical approaches.

The exponential growth of computational power over the last decade has enabled the first-generation of ML models to be used in computational mechanics and polymer physics [10,11,12,13]. Current ML models were often developed based on “black box” approaches, which, besides low interpretability, require a large volume of training data to prescribe a particular behavior [14,15]. In solid mechanics, stress–strain tensors are only partially observable in lower-dimensions. Thus, obtaining data to feed the black-box ML model is exceptionally challenging. In general, one can classify current data-driven efforts in computational mechanics into three categories with lots of approaches placed between two categories (see review [16]).

- Model-free distance-minimization approaches were developed to circumvent the need for constitutive models by directly finding stress–strain pairs with the least distance to experimental data, which also satisfy compatibility and equilibrium constraints. This approach was initially set for nonlinear truss and linear elastic materials [17] and later was expanded to include hyper-elastic materials [18]. While being superior to other models by being statistically independent of any prior knowledge of the materials, the method has a few major limitations. It has an excessively high computational cost, strong sensitivity to data scattering, and in high-dimensional problems suffers from lack of data [19]. This approach is further amended by studies on the combination of data-driven identification and computational mechanics [20]. To reformulate the heuristic optimization approach adopted by [17], mixed-integer programming was used for its implementation [21]. Coelho and Breitkopf [22] in their book investigated the main approaches for constitutive modeling using optimization methods.

- Non-linear dimensionality reduction approaches seek to build a constitutive manifold from experimental data to describe an accurate approximation of the strain energy in different states of deformation. These approaches focus on describing the constitutive behavior through a set of shape functions, such as B-$spline$ [23], with constants derived through the LSQ error minimization [24,25,26] or an ML approach [27]. Mainly derived from the WYPiWYG model [28], this approach focuses on solving the system of linear equations which consist of coefficients of shape functions, rather than nonlinear fitting a predefined model. In elasticity, manifold learning is more efficient and more accurate than black-box ML models and it has already been generalized to cover damage [29]. In Matous’s study, a manifold-based reduced order model was proposed [30]. This model relies on non-linear dimensionality reduction and the connection of macroscopic loading parameters to reduced space using an artificial neural network (ANN). Fritzen et. al [31] proposed a data-driven homogenization method for hyper-elastic solids using the reduced order method. In their work, the surrogate model combines radial basis functions and piece-wise cubic polynomials. The main problem with these approaches is the large number of tests needed for validation and their dependency on the assumption of constitutive manifolds with a particular functional structure [26].

- Autonomous approaches incorporate ML models as surrogate functions to capture the high-dimensional and non-smooth micro-scale behavior of material constituents, which has been shown to be a successful approach in multi-scale analysis [32]. Several multi-scale methods of analysis have been proposed based on the implementation of micro-scale ML models into the reduced-order FE simulations of the macro-scale approach [12]. This coupling allows for the scalable utilization of ML surrogate models. However, the validity range of current ML models is extremely limited due to a number of reasons: (i) the unconstrained search space of optimization variables, (ii) neglecting underlying physics, (iii) difficulties in deriving parameter feasibility ranges, and (iv) lack of transition models to reduce the order of the problem. Recently, by implementing the reinforcement learning concept, a new class of ML meta-models has been successfully developed based on (non)cooperative games, where the model trains a pair of L-agents to emulate a specific performance through turn-based trial and error [33]. This paradigm employs ML techniques to capture the behavior and interaction of micro-structures as a surrogate model. In Stoffel’s study [34], they replace the viscoplastic material law in finite element simulation with a feed-forward neural network to make an intelligent element. Another study [35] was conducted to predict the tension response of rubber by a feed-forward neural network. They used strain values and filler percentages as inputs generated from a regression model and stress as output. Kaliske and Zopf [36] proposed an inelastic model-free approach represented by recurrent neural networks for uncured elastomers. For history-dependent functions, naturally, recurrent neural networks offer attractive alternatives, but require enormous amounts of training paths of standardized lengths, which are highly non-trivial. In 2019, Haghighat et al. [37] proposed a physics-informed neural network that solves any given law of physics described by non-linear partial differential equations. Another study [38] showed the performance of this model for linear elasticity, and Xu et al. [39] modeled viscoelastic materials using physic constrained learning. Recently, we developed a Bayesian surrogate constitutive model based on Bayesian regression and Gaussian process [40] to consider uncertainty of a model [41]. A recent study [12] proposed a data-driven constitutive model by predicting a non-linear constitutive law using a neural network surrogate model constructed using a learning phase on a set of RVE non-linear computations. An investigation was conducted to formulate a constitutive model for rate-dependent materials by neural network and its implementation in finite element analysis. The challenge of a sufficient dataset for training, however, still remains [42].

Here, a cooperative multi-agent system ${\mathsf{𝓑}}^{d_i}={\mathsf{𝓐}}_j^i,i\in \left\{1,n\right\},j\in \left\{1,m\right\}$ is proposed to describe different features in material behavior by using $n\times m$ different machine-learned agents (L-agents) which learn from experimental datasets. To reduce problem dimensionality, the 3D matrix is represented by m 1D directions, which allows researchers to replicate each L-agent m times to represent the 1D behavior of the material. The proposed model trains each agent to emulate a certain material behavior with the objective function being the error between the overall prediction of the system and the experimental data. Model fusion is used to integrate all L-agents back into a centralized system.

The main contributions of this work are to infuse knowledge of physics into the model through certain modeling constraints: namely, (1) by providing a new data-driven model based on physics behind a machine learning process for predicting non-linear mechanical behavior of cross-linked polymers, (2) the first data-driven model that captures inelastic behavior of cross-linked polymers such as Mullins effect and permanent set, (3) a new paradigm with the upgrade-ability of model from hyper-elastic to damage behavior roots from easy transformation from the integration of micro-mechanics to the machine learning process, (4) proposing a new model with better training speed and accuracy compared to several well-known models. There are two types of cross-linked polymers. One type shows hyper-elastic behavior; however, another type does not have hyper-elasticity at all. In this study, our focus is on cross-linked polymers which have hyper-elasticity such as rubbers and elastomers. This paper is organized as follows; in Section 2, the main concepts of non-linear behavior and deformation-induced damage in cross-linked polymers are introduced and described in detail. Section 3 explains the idea and formulation of the proposed model in detail. Model verification with experimental data on rubber inelasticity is discussed in Section 4. Section 5 provides some concluding remarks and outlines some perspectives. Finally, in the appendix section, we explain frame-independence, polyconvexity, and thermodynamic consistency.

2. Non-Linear Features in Cross-Linked Polymers

Hyper-elasticity in most materials (i.e., cross-linked polymers) is defined by the nonlinear elastic behavior in large deformations. Meanwhile, cross-linked polymers often exhibit inelastic features in their hyper-elastic behavior, e.g., damage after first elongation known as Mullins effect [43,44]. This phenomenon happens to both types of filled and non-filled cross-linked polymers. To provide a better understanding of micro-structural sources of such inelastic effects, Figure 2 shows some of the physical sources in amorphous polymeric systems such as chain breakage [45], chain disentanglement [46], molecules slipping [47], and rupture in cluster of fillers [48]. After primary loading, deformation-induced damage often leads to a residual strain known as permanent set. While the permanent set in unfilled rubber is negligible, it becomes prominent in most filled compounds. Figure 2e provides a schematic view on different inelastic features in the hyper-elastic behavior of a polymeric system.

To model the inelastic features in the behavior of cross-linked polymers, previous efforts were mostly focused on phenomenological and physics-based approaches. Physics-based approaches are often excessively complicated for real-time applications, and phenomenological models are not reliable outside design condition. Here, by coupling a physics-based approach to machine learned L-agents, we devise a knowledge-driven ML approach to address inelastic features in hyper-elastic behavior of cross-linked polymers. Therefore, to model the nonlinear behavior of cross-linked polymers, we select an appropriate neural network consisting of proper activation functions and the appropriate number of layers and neurons. In addition, to model damages, internal parameters of L-agents are designed based on the type of materials’ memory. In a material with full memory such as rubbery materials with damage, only the maximum status of history affects the next sequence. Using a micro-sphere as a directional model of polymer matrix guarantees modeling of inelastic features such as permanent set. These steps are explained in the next section in detail.

3. Physics-Based Reduction

To model the second order stress–strain fields required for characterization of hyper-elastic material, current approaches ranging from phenomenological to data-driven, face one major challenge, namely lack of data on 3D structures. There are no tools to measure stress field across a structure, and for strain we can only measure the strain field for relatively simple structures using digital image correlation (DIC) techniques [49].

Helmholtz free energy $\mathsf{\Psi }$ is a function of both deformation and temperature. Differentiating $\mathsf{\Psi }$ with respect to kinematic state variables at constant temperature yields internal forces (i.e., stresses) defined per unit mass. For the case in which $\mathsf{\Psi }$ is solely a function of deformation (i.e., isothermal processes), the Helmholtz free energy referred to as the strain energy function.

For hyper-elastic materials, strain energy is derived directly from the Clausius–Planck form of the second law of thermodynamics through different work conjugate pairs, such as two-point strain/stress tensors ($\mathbf{F}$/deformation gradient: $\mathbf{P}$/first-order Piola stress), material strain/stress tensors ($\mathbf{E}$/Lagrange strain: $\mathbf{S}$/second-order Piola stress), and spatial strain/stress tensors ($\mathbf{L}$/Hencky strain: $\tau$/Kirchhoff stress). Strain energy function must accompany conditions like normalization, growth conditions, isotropy, objectivity, and polyconvexity, which guarantee the uniqueness of the solution (Appendix A, Appendix B and Appendix C). In view of the lack of data on the stress fields, a proper modeling approach is expected to be able to only use the limited information obtained from the classical characterization tests on the collective sample behavior. In rare cases, a model can be provided by digital image correlation reconstruction of 2D strain fields, which shall be used for model validation but should not become necessary data for model fitting in view of the cost/complexity of the experiment. The challenge of significant missing data has been historically addressed by implementing knowledge of the material behavior in the model and to constrain the model in advance before having the data. Such a solution is not relevant in data-driven approaches due to the lack of infused knowledge of the material. Here, we propose addressing the challenge of significant missing data in high-dimensional data-driven approaches through a physics-driven order-reduction approach by infusing knowledge through implementation of the concepts of micro-sphere, network decomposition, continuum mechanics, and polymer physics. Accordingly, we developed a sufficiently constrained machine-learned model that can predict the material behavior solely based on the macro-scale collective behavior of the sample. Figure 3 demonstrates a schematic of the proposed model simplification idea.

3.1. Continuum Mechanics

We introduce the first and the most important constraint from continuum mechanics’ understanding on 3D mapping of second order stress/strain tensors. While $\mathbf{F}\to \mathbf{P}$ mapping generally needs a complicated fourth order tensor $\mathbb{C}=\frac{\mathbf{P}}{\mathbf{F}}$, in hyper-elastic materials, unlike hypo-elastic materials, the procedure can be simplified using an internal energy density function. Accordingly, we can use finite strain theory to simplify $\mathbf{F}\to \mathbf{P}$ mapping by introducing of the strain energy ${\mathsf{\Psi }}_m$ as the middle agent in mapping, where $\mathbf{F}\to {\mathsf{\Psi }}_m\to \mathbf{P}$. The strain energy is a non-negative scalar-valued function ${\mathsf{\Psi }}_m\left(\mathbf{F}\right)$ which can replace part of the process required to derive tensor-valued stress function $\mathbf{P}\left(\mathbf{F}\right)$. The increment of ${\mathsf{\Psi }}_m$ denotes the stress required to change the strain field, and thus ${\mathsf{\Psi }}_m$ can be described with respect to any stress–strain work conjugates such as (i) two-point tensors, (ii) material tensors, or (iii) spatial tensors, as shown below:

$$\mathbf{P}=\frac{\partial {\mathsf{\Psi }}_m}{\partial \mathbf{F}},\mathbf{S}=\frac{\partial {\mathsf{\Psi }}_m}{\partial \mathbf{E}},\tau =\frac{\partial {\mathsf{\Psi }}_m}{\partial \mathbf{L}}.$$

(1)

One particular advantage of using ${\mathsf{\Psi }}_m$ as a middle agent is that it ensures the material objectivity, and thermodynamic consistency on all the derived constitutive models (see Truesdell et al. [50]). Considering the physics of the problem, certain restrictions exist for strain energy which need to be enforced further in the data-driven model, namely

$$\begin{array}{cc}\hfill {\mathsf{\Psi }}_m\left(\mathbf{F}\right)\ge 0& \mathit{when}\mathbf{F}\ne 0\mathit{Increase}\mathit{energy}\mathit{by}\mathit{deforming},\hfill \\ \hfill {\mathsf{\Psi }}_m\left(\mathbf{F}\right)=0& \mathit{when}\mathbf{F}=\mathbf{I}\mathit{Normalization}\mathit{condition},\hfill \\ \hfill {\mathsf{\Psi }}_m\left(\mathbf{F}\right)\to \infty & \mathit{when}det\mathbf{F}\to \infty /0\mathit{Growth}\mathit{condition}.\hfill \end{array}$$

(2)

Further restrictions can be introduced by finite strain theory to ensure stability of ${\mathsf{\Psi }}_m$ in large deformations of certain materials. For hyper-elastic materials, ellipticity is a major concern which can be enforced by verifying the strain energy in the absence of traction forces in two arbitrary directions [51]. Verifying this condition is generally labor-intensive, so polyconvexity is introduced as a stronger condition that entails ellipticity. It is also simpler to verify [52], as discussed in the Appendix A, Appendix B and Appendix C. Therefore, the first constraint that we enforce in our model, is enforcing agents to derive ${\mathsf{\Psi }}_m\left(\mathbf{F}\right)$ such that it satisfies Equation (2) and polyconvexity condition.

3.2. Micro-Sphere

The second constraint is implemented based on polymer physics, in particular topology of a cross-linked amorphous network. Knowing amorphous systems are isotropic at the virgin state, polymer chains are considered to be uniformly distributed in all spatial directions. Such homogenized spatial arrangement of polymer chains allows us to use the micro-sphere concept to represent the 3D matrix as a homogeneous assembly of similar 1D elements that are distributed in different spatial directions over a micro-sphere (see Figure 3). This approach can transfer information from super-simplified 1D elements to generate complex 3D behavior of the matrix via homogenization over the unit-sphere. Furthermore, by discretizing the sphere into finite sections, the integration can be taken out numerically over $N_d$ integration directions ${\left[{\mathit{d}}_i\right]}_{i=1\dots N_d}$ with different weight factors ${\left[w_i\right]}_{i=1\dots n}$ [53]. Accordingly, strain energy of the matrix ${\mathsf{\Psi }}_m$ with respect to its elements can be written as:

$${\mathsf{\Psi }}_m=\frac{1}{4\pi }{\int }_S{\mathsf{\Psi }}_m^{\mathit{d}}d{S}^{\mathit{d}}\cong \sum _{i=1}^{N_d}{w}_i{{\mathsf{\Psi }}_m}^{{\mathit{d}}_{\mathit{i}}},\mathit{where}{{\mathsf{\Psi }}_m}^{{\mathit{d}}_{\mathit{i}}}={\mathsf{𝓑}}^{{\mathit{d}}_i}$$

(3)

where ${{\mathsf{\Psi }}_m}^{{\mathit{d}}_{\mathit{i}}}$ is the energy of sub-matrix element in direction ${\mathit{d}}_i$, which will be represented by one team of L-agent ${\mathsf{𝓑}}^{{\mathit{d}}_i}$, which represents an additive cooperation between multiple L-agents ${\mathsf{𝓐}}_{•}^i$. Equation (3) represents the integral $S(\theta ,\varphi )={\int }_0^{\theta }{\int }_0^{\varphi }sin\left(\theta \right)d\theta d\varphi$ over the unit-sphere with the unit vector $\mathit{r}=sin\left(\theta \right)cos\left(\varphi \right){\mathit{e}}_x+sin\left(\theta \right)sin\left(\varphi \right){\mathit{e}}_y+cos\left(\varphi \right){\mathit{e}}_z$ (see Figure 4). Assuming identical team in all directions in the virgin state, namely ${\mathsf{𝓑}}^{{\mathit{d}}_i}={\mathsf{𝓑}}^{{\mathit{d}}_j}$, initial isotropy is assured, although the material can quickly become anisotropic due to different loading on different directions. Moreover, since L-agents react to varying loading in each direction, the model can consider the onset of damage, deterioration, and propagation of cascading failure in materials with directional response.

3.3. Network Decomposition

The third constraint is derived from statistical mechanics, namely by infusing the concept of superposition, which allows us to predict complicated patterns by superposing simple patterns on top of each other. The concept, also known as the network decomposition concept in constitutive modeling [54], will be carried out by representing the energy of an element,${{\mathsf{\Psi }}_m}^{{\mathit{d}}_{\mathit{i}}}$ by superposing the energy of multiple sub-elements, ${{\mathsf{\Psi }}_m}^{{\mathit{d}}_{\mathit{i}}}={\sum }_{j=1}^{N_s}{{\mathsf{\Psi }}_j}^{{\mathit{d}}_{\mathit{i}}}$, where each sub-element is responsible for one simple inelastic feature. Representing each sub-element by one L-agent, we can calculate the energy of one element by a team of cooperative L-agents ${\mathsf{𝓑}}^{{\mathit{d}}_i}=\left[{\mathsf{𝓐}}_j^i\right]$, and then replicating this cooperative team in different directions to provide us with the energy of the matrix. To this end, by substituting Equation (3), we can directly derive the energy of the matrix with respect to sub-elements and the L-agents which represent them as given here

$$\begin{array}{cc}\hfill {\mathsf{\Psi }}_m& =\frac{1}{4\pi }{\int }_S{{\mathsf{\Psi }}_m}^{\mathit{d}}d{S}^{\mathit{d}}\cong \sum _{i=1}^{N_d}\sum _{j=1}^{N_s}{w}_i{{\psi }_j}^{{\mathit{d}}_{\mathit{i}}}\hfill \\ \hfill {\mathsf{\Psi }}_m& \approx \sum _{i=1}^{N_d}\sum _{j=1}^{N_s}{w}_i{\mathsf{𝓐}}_j^i\mathit{where}{{\mathsf{\Psi }}_m}^{{\mathit{d}}_{\mathit{i}}}=\sum _{j=1}^{N_s}{\mathsf{𝓐}}_j^i.\hfill \end{array}$$

(4)

where $N_s$ is the number of sub-elements considered for each element. Consequently, we derived super-simplified scalar-to-scalar mapping behavior for each element, which can be represented by a simplified 2-layer feed-forward neural network L-agent ${\mathsf{𝓐}}_j^i$. While training data are only available on collective behavior of the L-agents, the input parameters can be defined for each L-agent team individually. Each L-agent, ${\mathsf{𝓐}}_j^i:={\psi }_j^i({\mathbb{E}}^i,{\mathbb{M}}_j)$, will be trained based on a set of non-kinematic input ${\mathbb{E}}^i$ and internal ${\mathbb{M}}_j$ parameters, which depending on material memory (full or recent), can satisfy normalization, growth conditions, isotropy, objectivity, and polyconvexity.

The input vector is independent of the sub-element definition and should represent the problem setting, material or loading, e.g., stretch and time. Internal parameters are specifically hypothesized for the model to capture the evolution of damage and vary for each network. The behavior of all teams should be identical in the virgin state to represent initial isotropy, so one has ${\mathsf{𝓐}}_j^i={\mathsf{𝓐}}_j^k\forall i\ne k$. Accordingly, we only assume different ANN types for L-agents associated to different sub-elements. All replicated agents associated to one sub-element are the same despite being distributed in different teams to represent different directions. For the replicated agents, only the inputs are different depending on their direction (see Figure 3). Therefore, the energy of one sub-element can be written as

$${\psi }_j^{{\mathit{d}}_i}={\mathsf{𝓐}}_j^i={ANN}_j({\mathbf{W}}_{\mathbf{j}},{\mathbb{E}}^i,{\mathbb{M}}_j),$$

(5)

where ${\mathit{W}}_j$ is the weight vector associated to L-agent ${\mathsf{𝓐}}_j^{•}$, and $\mathbf{W}=\left[{\mathit{W}}_1\dots {\mathit{W}}_{N_s}\right]$ is the weight matrix representing assembly of all ${\mathit{W}}_j$. Consequently, based on Equations (1) and (4), the first Piola–Kirchhoff stress tensor $\mathbf{P}$ can be derived as:

$$\mathbf{P}=\frac{\partial {\mathsf{\Psi }}_m}{\partial \mathbf{F}}-p{\mathbf{F}}^{-T}=\sum _{i=1}^{N_d}\sum _{j=1}^{N_s}{w}_i\frac{\partial {\mathsf{𝓐}}_j^i}{\partial \mathbf{F}}-p{\mathbf{F}}^{-T},$$

(6)

where p denotes the Lagrange multiplier to guarantee incompressibility of the material. To train the model, a cost function should be derived to quantify prediction error against experimental observations on collective sample behavior, e.g., uni-axial tensile test provides the 1D dataset $\mathcal{S}=[\tilde{P},\tilde{\lambda }]$, with nominal stress $\tilde{P}$ and stretch $\tilde{\lambda }$ in the direction of principal stretch. Here, the error has been quantified using least-square method by writing:

$$E\left(\mathbf{W}\right)=\frac{1}{2}\sum _{n=1}{\left[{\mathit{g}}_1(\sum _{i=1}^{N_d}\sum _{j=1}^{N_s}{w}_i\frac{\partial {{\psi }_j}^{{\mathit{d}}_{\mathit{i}}}}{\partial \mathbf{F}}-p{\mathbf{F}}^{-T}){\mathit{g}}_1-P_n\right]}^2,$$

(7)

where $P_{(1,1)}:={\mathit{g}}_1\mathbf{P}{\mathit{g}}_1$ is the first component of the experimental macro-scale stress tensor $\mathbf{P}$ in loading direction ${\mathit{g}}_1$.

Neural network L-agents: Artificial neurons, also known as nodes, are the basic units in the neural system, which receive and transfer information to the other nodes through activation functions. Accordingly, the ability of “learning” of an ANN agent strongly depends on proper selection of the activation function for each node. Their purpose is to decide whether a neuron should be activated or not and introduce non-linearity into the output of a node. Therefore, they make the model to generalize or adapt to a variety of data and to differentiate between the output. The cost function is accounted for by estimating $\mathbf{W}$ using gradient descent (GD) algorithm, which gradually optimizes the initial guess towards the target values.

Material with full or recent memory: For history-dependent materials, parameters should be specifically chosen to represent material memory and then fed into the L-agents through internal parameters. However, different types of memory parameters may be required for describing different materials, e.g., for materials with recent memory such as visco-elastic materials, internal parameters should transfer information from each iteration to the next. In contrast, for materials with full memory such as elastomers, the internal parameters can be defined independent of the solution iterations as a damage-precursor of the external events, for example the maximum stretch in rubber material can be used as a damage precursor to show the history of maximum loading in each direction.

4. Implementation to Rubber Inelasticity

To show the performance of the proposed model, inelastic behavior of rubber has been studied. The number of teams and their associated agents can be chosen based on the trade-off between accuracy and computational cost. Here, we choose 21 teams, each with two agents, which is a relatively small number [53] (21 integration-points discussed in Appendix A, Appendix B and Appendix C). The inputs and internal parameters of L-agents are designed to capture the rubbers’ deformation with full memory through ${\lambda }_{j-max}$ parameters. To enable teams to predict different states of deformation, each team should be provided with the first and second invariants of deformation [55]. The condition is satisfied by providing input set ${\mathit{S}}_1^{d_i}=[{\lambda }_1^{{\mathit{d}}_{\mathit{i}}};{\lambda }_{1-max}^{{\mathit{d}}_{\mathit{i}}}]$ to L-agent 1 and ${\mathit{S}}_2^{d_i}=[{\lambda }_2^{{\mathit{d}}_{\mathit{i}}};{\lambda }_{2-max}^{{\mathit{d}}_{\mathit{i}}}]$ to L-agent 2

$${{\lambda }_1}^{{\mathit{d}}_{\mathit{i}}}=\sqrt{{\mathit{d}}_{\mathit{i}}\mathbf{C}{\mathit{d}}_{\mathit{i}}},{{\lambda }_2}^{{\mathit{d}}_{\mathit{i}}}=\sqrt{{\mathit{d}}_{\mathit{i}}{\mathbf{C}}^{-1}{\mathit{d}}_{\mathit{i}}},\mathbf{C}={\mathbf{F}}^T\mathbf{F}$$

(8)

where ${{\lambda }_1}^{{\mathit{d}}_{\mathit{i}}}$ and ${{\lambda }_2}^{{\mathit{d}}_{\mathit{i}}}$ are designed to lead to the first and second sub-elements and represent the $I_1$ and $I_2$, respectively. For the ANN structure of L-agents, we consider one input layer, one hidden layer with four neurons and three activation functions soft plus ($\psi (•)=ln(1+e^{•})$), sinusoid ($\psi (•)=sin(•)$) and hyperbolic tangent ($\psi (•)=tanh(•)$). In summary, we represented the rubber matrix by the cooperative game of 21 teams of 2 agents through ${\mathsf{𝓐}}_j^i,i\in \left\{1,21\right\},j\in \left\{1,2\right\}$. The final cost function after agent fusion is given by:

$$E({\mathbf{W}}_1,{\mathbf{W}}_2)=\frac{1}{2}\sum _{n=1}{[{\mathit{g}}_1(\sum _{i=1}^{21}\sum _{j=1}^2{w}_i\frac{\partial {\mathsf{𝓐}}_j^i}{\partial {{\lambda }_j}^{{\mathit{d}}_{\mathit{i}}}}\frac{\partial {{\lambda }_j}^{{\mathit{d}}_{\mathit{i}}}}{\partial \mathbf{F}}-p{\mathbf{F}}^{-T}){\mathit{g}}_1-P_n]}^2,$$

(9)

subjected to weights related to ${\lambda }_{1-max}$ and ${\lambda }_{2-max}$$⩽0$, and weights related to ${\lambda }_1$ and ${\lambda }_2$$⩾0$ to satisfy thermodynamic consistency and polyconvexity, respectively. Equations (10) and (11) show the derivation of each sub-element’s energy with respect to deformation gradient. Accordingly, Figure 3 shows the schematic concept of the derived model.

$$\sum _{i=1}^{21}{w}_i\frac{\partial {\mathsf{𝓐}}_1^i}{\partial {{\lambda }_1}^{{\mathit{d}}_{\mathit{i}}}}\frac{\partial {{\lambda }_1}^{{\mathit{d}}_{\mathit{i}}}}{\partial \mathbf{F}}=\sum _{i=1}^{21}{w}_i\frac{\partial {\mathsf{𝓐}}_1^i}{\partial {{\lambda }_1}^{{\mathit{d}}_{\mathit{i}}}}\frac{1}{{{\lambda }_1}^{{\mathit{d}}_{\mathit{i}}}}\mathbf{F}\left({\mathit{d}}_{\mathit{i}}\otimes {\mathit{d}}_{\mathit{i}}\right).$$

(10)

$$\sum _{i=1}^{21}{w}_i\frac{\partial {\mathsf{𝓐}}_2^i}{\partial {{\lambda }_2}^{{\mathit{d}}_{\mathit{i}}}}\frac{\partial {{\lambda }_2}^{{\mathit{d}}_{\mathit{i}}}}{\partial \mathbf{F}}=-\sum _{i=1}^{21}{w}_i\frac{\partial {\mathsf{𝓐}}_2^i}{\partial {{\lambda }_2}^{{\mathit{d}}_{\mathit{i}}}}\frac{1}{{{\lambda }_2}^{{\mathit{d}}_{\mathit{i}}}}{\mathbf{F}}^{-1}{\mathbf{F}}^{-T}{\mathbf{F}}^{-1}\left({\mathit{d}}_{\mathit{i}}\otimes {\mathit{d}}_{\mathit{i}}\right).$$

(11)

4.1. Minimizing Data Requirement for Training

4.1.1. Dataset Minimization

A critical step in the selection of the training dataset is to understand the role of the training points and assure their quality in the model predictions. Too little data may provide a false sense of confidence by preventing us from seeing the critical points, while low-quality data may provide faulty results which seems perfectly robust. For example, in the aforementioned model developed for rubber, we have introduced two L-agent types which represents two sub-elements using ${\mathit{S}}_1^{d_i}$ and ${\mathit{S}}_2^{d_i}$ input sets, respectively. In view of the definition of ${\mathit{S}}_2^{d_i}$, we know that it has a limited variation in uni-axial tensile loading which makes the contribution of the second L-agent almost limited in such loading. However, ${\mathit{S}}_2^{d_i}$ significantly varies in bi-axial loading, which makes the contribution of second L-agent quite considerable in this case. Therefore, training with uni-axial data cannot provide the quality information needed for confidently training both agents since second-agent cannot be fully engaged.

In essence, we cannot train agents with the scenarios that they are not participating in or have minimal contribution. Thus, the confidence in training of agents is directly correlated to the quality of the training data, and the contribution of agents in those scenarios. However, by defining the quality of data with respect to the input required by each agent, we can quantify the confidence interval in which an agent can be trained with high confidence with respect to the provided data.

The confidence interval of a system is equal to that of its agent with least confidence. The confidence of an agent can be calculated with respect to the deformation range used in each direction for training of that agent. Since the reliability of the predictions of each agent is related to its training, we can linearly correlate the agent’s reliability to their training range. As an example, in case of uni-axial tension where the sample is stretched until ${\chi }_x$, maximum first deformation state (axial) is ${\chi }_x$ which occurs in the loading direction, and minimum is $\frac{1}{\sqrt{{\chi }_x}}$, which occurs in the transverse directions. Similarly, the training domain for the second agent is $[\frac{1}{{\chi }_x},\sqrt{{\chi }_x}]$. In case of bi-axial tension, range of agents deformations are $[\frac{1}{{\chi }_{bi.}^2},{\chi }_{bi.}]$ and $[\frac{1}{{\chi }_{bi.}},{\chi }_{bi.}^2]$.

If we train the model based on uni-axial tensile data until ${\chi }_x$, the model can predict different states of deformation based the ranges that the model has been calibrated based on that. In order to ensure accurate prediction of the model, the prediction ranges should be in the range that agent is trained. Thus, in the bi-axial prediction case, the model limits to

$$\begin{array}{c}\hfill \mathit{Agent}\mathit{1}:\left[\frac{1}{{\chi }_{bi.}^2},{\chi }_{bi.}\right]\in \left[\frac{1}{\sqrt{{\chi }_x}},{\chi }_x\right]⟶{\chi }_{bi.}\le \sqrt[4]{{\chi }_x}\\ \hfill \mathit{Agent}\mathit{2}:\left[\frac{1}{{\chi }_{bi.}},{\chi }_{bi.}^2\right]\in \left[\frac{1}{{\chi }_x},\sqrt{{\chi }_x}\right]⟶{\chi }_{bi.}\le \sqrt[4]{{\chi }_x}.\end{array}$$

(12)

As it can be seen in Equation (12), these two ranges result into a same confidence interval for the agents. Accordingly, considering one of these agents confidence interval would be sufficient to calculate the network reliability. Likewise, these training/prediction domains can be calculated for different cases of training and predictions with different states of deformation, see

Table 1. Prediction domain for train until stretch $\chi$.

	Training	Uni. 1 Tensile	Bi. 2	Pure Shear	Uni. Comp. 3	Plane Strain Comp.
Prediction
Uni. Tensile		$\chi$	$\chi$	$\chi$	$\frac{1}{\sqrt{\chi }}$	$\frac{1}{\chi }$
Bi.		$\sqrt[4]{\chi }$	$\chi$	$\sqrt[4]{\chi }$	$\frac{1}{\sqrt{\chi }}$	$\frac{1}{\chi }$
Pure Shear		$\chi$	$\chi$	$\chi$	$\frac{1}{\sqrt{\chi }}$	$\frac{1}{\chi }$
Uni. Comp.		$\frac{1}{\sqrt{\chi }}$	$\frac{1}{{\chi }^2}$	$\frac{1}{\chi }$	$\chi$	$\chi$
Plane Strain Comp.		$\frac{1}{\sqrt{\chi }}$	$\frac{1}{{\chi }^2}$	$\frac{1}{\chi }$	$\chi$	$\chi$

1. Uniaxial; 2. Biaxial; 3. Compression.

. To show the confidence interval, we explore five different training datasets and their resulting agents. Using two sets of experiments for the training purpose can increase the predictability range of the model as each of the experiments can be activated in different ranges and agents. Note that the model may extrapolate and predict more than the confidence interval but that is not necessarily accurate.

• Training with uni-axial only vs. bi-axial only. Mars dataset which has three modes of pure shear, uni-axial, and bi-axial tensile tests have been used [56]. In first case, the model is trained by bi-axial data only until $\chi =1.65$ and validated against other modes (see Figure 5a). Confidence interval in uni-axial and pure shear predictions is also limited to $\chi =1.65$.
  In the second case, the model is trained by uni-axial data only until $\chi =2.18$ and validated against other modes (see Figure 5b). Confidence interval in shear will be limited to $\chi =2.18$ but in bi-axial will be dramatically reduced to $\chi =1.21$ due to the uncertainty in training L-agent 2.
• Training with uni-axial only over a long range. Here, we showed that we can improve the confidence interval of one agent not only by choosing the games in which it has high contribution, but also by increasing the length of the game in which one agent has small contribution. In essence, we can have a short game with high contribution, or long game with low contribution. In case of rubber, uni-axial tension is a game in which the 2nd L-agent has a low contribution. Therefore, here we show that for a sufficiently long game (uni-axial until $\chi =7.7$), we can increase the confidence interval for the second L-agent (bi-axial until $\chi =1.66$), see Figure 5c Treloar dataset which has three modes of pure shear, uni-axial, and bi-axial tensile tests that have been used [57].
• Training with uni-axial tension and compression. Here, we showed that we can improve the confidence interval by using multiple games to train the agents. So, here the model is trained by uni-axial tensile (till $\chi =3.7$) and compression data (till $\chi =0.4$). The confidence in training of the 1st L-agent is mainly defined by the uni-axial tensile test while that of the 2nd L-agent is formed by compression test. The predictions of the trained agents were validated against other modes (see Figure 5d), and as expected confidence interval in bi-axial until $\chi =1.58$ and pure shear predictions is also limited to $\chi =3.7$. Heuillet dataset with three modes of pure shear, uni-axial and bi-axial tensile tests have been used for training/validation [58].

4.1.2. Accuracy within Confidence Interval

The proposed engine shows exceptional accuracy within the confidence interval which is comparable to some of the most comprehensive and most expensive knowledge-based models. We have shown the predictions of the models against different sets of data provided by Uramaya [59] and Mars [56], where the models were trained using bi-axial tests only, see Figure 6. Bi-axial tests were chosen to provide the longest confidence interval for other modes (see Table 1).

To show the performance of the proposed model, we compared the relative error of our model in fitting and predicting Treloar’s dataset with the non-affine micro-sphere model [60], WYPiWYG model [23], and network averaging tube model [61]. Note that the error reported for the non-affine micro-sphere model and network averaging tube models is fitting error not prediction error, since they have used all three uni-axial, bi-axial, and pure shear at the same time in their published results.

Although, the proposed model and WYPiWYG model use uni-axial data for training and predicting other states of deformation. Thus, results show the excellent performance of our model; however, the proposed model is not complicated and as data-dependent as other physics-based models (see

Table 2. Relative error for several well-known models for Treloar’s dataset.

Model Type	AI	Phenomenological	Micro-Mechanical
	Proposed model	WYPiWYG model [23]	Non-affine micro-sphere model [60]	Network averaging tube model [61]
Error (%)	1.12	5.26	0.93	2.11
Training Set	Uni-axial	Uni-axial	Uni-axial + Pure shear + Bi-axial	Uni-axial + Pure shear + Bi-axial

).

The compression behavior of rubber-like material is another aspect that plays an essential role in industrial applications. We trained the model with the dataset of uni-axial compression experiments and predicted the behavior of plane strain compression. Figure 7 shows the performance of the proposed model for compression tests. The error in training and prediction of the proposed model for Arruda–Boyce data is $0.73\mathsf{\%}$, which compared to the non-affine micro-sphere model, which has $1.29\mathsf{\%}$ error, shows a significant performance of our less complicated model.

4.1.3. Damage Prediction and Deformation History

To further investigate the performance of the proposed model in material with full memory, we predicted the inelastic features in the behavior of filled elastomer, namely Mullins effect and permanent set. Figure 8 shows stress–stretch curves for this cross-linked polymer with experimental data of [59]. We used one set of bi-axial loading-unloading until $\chi =2.7$ for training and predicting inelastic effects in different states of deformation, e.g., uni-axial and pure shear at increasing stretch amplitudes which constitutes deformation histories.

4.1.4. Convergence Outside of the Confidence Interval

To investigate the convergence of the proposed model outside of the confidence interval, prediction of the inelastic behavior on stretch amplitudes larger than the confidence interval was illustrated on two different sets of experimental data on rubber, Itskov’s [63] and Zhong’s [64] dataset. While we strongly recommend the users to stay within the confidence interval, the model prediction accuracy outside of the confidence interval shows the relevance and reliability of the model in extreme cases which is mainly a result of the constraints induced by the knowledge infused into the model. Results indicated the trend and proposed model performance (Figure 9 and Figure 10). Here, we gradually reduce the confidence interval by using smaller ranges of training data to see the drop in quality of predictions. As expected, despite accuracy reduction, there is no significant change in the model predictions profile which is not usually the case for extrapolation methods. In Figure 9a, we trained the model with the largest amplitude. As we expected, the error in training and prediction is $4.6\mathsf{\%}$. As we reduce the amplitude of training in Figure 9, we see that the error has increased generally. There is an instability in the errors and overestimating in Figure 9b,c, which come from numerical simplification and choosing the same neural network structures and activation functions for different sets of data for training. The important point is that we want to show that the model is accurate for different modes of training. To ensure our result is general for different elastomers, we did the same training procedure for another dataset (Figure 10). The result shows the same results as the last dataset.

5. Conclusions

A physics-informed data-driven constitutive model for cross-linked polymers is developed by embedding neural networks into a multi-scale model. We propose a systematic approach to reduce the order of the constitutive mapping by leveraging existing knowledge of polymer science, continuum physics, and statistical mechanics. We use our model to predict the mechanical behavior of filled elastomers. The results indicate that our model can easily capture multiple inelastic effects in the behavior of the materials, is significantly less data-dependent, has lower dimensionality, and is interpretable. To illustrate the superior performance of knowledge-driven models developed by this approach, its predictions are bench-marked against several experimental datasets. We compare the stress responses from Treloar’s dataset in our model with several well-known models to show the accuracy and simplicity of our model. In summary, our model provides a hyper-elastic constitutive model which captures damage of polymer chains for cross-linked elastomers for quasi-static loading. In the future, the proposed model can be further extended to include the effect of the deformation rate. The modular platform nature of the proposed model allows the addition of such effects.