An Artificial Neural Network Based Solution Scheme for Periodic Computational Homogenization of Electrostatic Problems

Abstract

The present work addresses a solution algorithm for homogenization problems based on an artificial neural network (ANN) discretization. The core idea is the construction of trial functions through ANNs that fulfill a priori the periodic boundary conditions of the microscopic problem. A global potential serves as an objective function, which by construction of the trial function can be optimized without constraints. The aim of the new approach is to reduce the number of unknowns as ANNs are able to fit complicated functions with a relatively small number of internal parameters. We investigate the viability of the scheme on the basis of one-, two- and three-dimensional microstructure problems. Further, global and piecewise-defined approaches for constructing the trial function are discussed and compared to finite element (FE) and fast Fourier transform (FFT) based simulations.

1. Introduction

Artificial neural networks (ANNs) have attracted a lot of attention in the last few years due to their excellent universal approximation properties. Originally developed to model nervous activity in living brains [1,2], they nowadays grow popular in data-driven approaches. Tasks such as image and speech recognition [3,4] or the prediction of users’ behavior on social and commercial websites are characterized by a large amount of accessible data compared to a difficult analytic mathematical description. The use of ANNs or other machine learning algorithms such as anomaly detection [5] and support vector machines [6] is suited for such problems as it enables the fitting of even highly complex data with high accuracy. Recent trends in machine learning concern the physical constraining of data driven methods for even higher convergence rate and accuracy, as done in [7].

Due to the aforementioned advantages and improvements, machine learning algorithms gained entry into the field of continuum mechanics and material modeling as well. Successful implementations were performed for the prediction of material response based on the fitting of experimental data through ANNs [8,9,10]. Another interesting application is the reduction of microstructure data of a given material through pattern recognition in order to reduce computational demands (see, e.g., [11,12]).

In the present work, we employ ANNs to seek the solution of homogenization problems. Homogenization aims for the prediction of the macroscopic response of materials that have microstructures described on length scales far lower than the macroscopic dimension. In terms of analytical homogenization, Voigt [13] and Reuss [14] were the first to provide bounds of effective properties [15]. Hashin and Shtrikman [16] and Willis [17] improved the theory in terms of tighter bounds. Further estimates were developed using the self-consistent method [18,19] and the Mori–Tanaka method [20] afterwards.

In the case of a rather fine microstructures with complex topology and non-linear material behavior, those bounds are only able to make rough predictions on the effective properties. To describe microscopic fields and effective properties in a more detailed and accurate fashion, several computational approaches have been developed in the last decades. Two of the most commonly used discretization techniques are finite element (FE) methods [21,22] and fast Fourier transform (FFT) based solvers [23,24]. However, for materials with a microstructure that need fine discretization, the memory cost and solution time of the solvers increases vastly, making multiscale simulations uneconomical. Promising approaches to mitigate these problems are model order reduction methods (see, e.g., [25,26]).

In the present work, a memory efficient solution scheme based on the discretization through ANNs is presented. We therefore follow the idea of Lagaris et al. [27], who introduced a method for solving differential equations through ANN-based trial functions. The functions are constructed in a way that they a priori fulfill the given boundary conditions and the squared error residual of the differential equation is used as an objective that needs to be optimized with respect to the ANNs’ weights. The construction of the trial function might be a difficult task for complicated boundary value problems. However, in conventional homogenization problems, we usually deal with rather simple boundary geometries. In the present work, we consider square representative volume elements (RVE) under periodic boundary conditions, as described in Section 2. In Section 3.1, the concept of the ANN-based trial function according to Lagaris et al. [27] is explained. In contrast to Lagaris et al. [27], the optimization objective in our problem is not the squared error residual of a differential equation but emerges from a global energy potential. Section 3.2 and Section 3.3 give the ANNs’ structure used in the present work as well as the derivatives necessary to optimize the objective function. Finally, in Section 4, the robustness of the presented method is validated for one-, two- and three-dimensional problems and is compared to FE- and FFT-based computations. Further, we compare a global with a piecewise-defined approach for constructing the trial function. In the global approach, the solution is represented by only one global function using several ANNs and the topology of the microstructure must be learned by the neural networks itself. On the other hand, in a piecewise-defined approach, the solution is represented by many neural networks that are piecewise defined on different sub-domains of the RVE. A conclusion is provided in Section 5.

2. Homogenization Framework

In the present work, we consider first-order homogenization of electrostatic problems. The fundamental laws of electrostatics and the corresponding variational formulation are given. In terms of the homogenization framework, we assume separation of length scales between macro- and microscale in the sense that the length scale at which the material properties vary quickly, namely the microscale, is much smaller than the length scale at the same order of the body’s dimensions. A connection for the micro- and macrofields is given by the celebrated Hill–Mandel condition [28], which allows for the derivation of consistent boundary conditions for the microscopic boundary value problem.

2.1. Energy Formulation for Electrostatic Problems

We now recall the fundamental equations of electrostatics in the presence of matter [29]. The focus lies on a variational formulation as the energetic potential is later needed in the optimization principle. Assuming that there are neither free currents nor charges, the fundamental physics of electric fields in a body $\mathcal{B}$ are governed by the reduced Maxwell equations

$$\mathrm{curl} \mathit{E}=0\mathrm{and}\mathrm{div} \mathit{D}=0in\mathcal{B},$$

(1)

where E denotes the electric field and D the electric displacement. In vacuum, the electric field and displacement are connected through the vacuum permittivity ${\kappa }_0\approx 8.854·{10}^{-12}\frac{\mathrm{As}}{\mathrm{Vm}}$ as $\mathit{D}={\kappa }_0\mathit{E}$. In the presence of matter, the permittivity $\kappa ={\kappa }_0·{\kappa }_r$ must be adapted accordingly. To solve Equation (1), we choose the electric field as our primary variable and construct it as the negative gradient of some scalar electric potential $\varphi$ according to

$$\mathit{E}:=-\mathrm{grad}\varphi \Rightarrow \mathrm{curl}\mathit{E}\equiv 0,$$

(2)

and thus Equation (1)${}_1$ is automatically fulfilled. Next, we want to solve Equation (1)${}_2$ under some boundary conditions. We therefore introduce an energy potential

$$\mathsf{\Pi }\left(\varphi \right)={\int }_{\mathcal{B}}\mathsf{\Psi }\left(\mathit{E}\right)dV+{\int }_{{\partial \mathcal{B}}_q}q\varphi dA\mathrm{and}\varphi ={\varphi }^{*}\mathrm{on}{\partial \mathcal{B}}_{\varphi },$$

(3)

where $\mathsf{\Psi }\left(\mathit{E}\right)$ is a constitutive energy density, ${\partial \mathcal{B}}_q$ and ${\partial \mathcal{B}}_{\varphi }$ denote boundaries along with electric charges q and electric potential ${\varphi }^{*}$ as prescribed. From the latter potential, it can be shown that, for equilibrium, i.e. $\delta \mathsf{\Pi }=0$, Equation (1)${}_2$ is solved in a weak sense

$$\delta \mathsf{\Pi }\left(\varphi \right)=-{\int }_{\mathcal{B}}\left(\mathrm{div}\mathit{D}\right)\delta \varphi dV+{\int }_{{\partial \mathcal{B}}_q}(q+\mathit{D}·\mathit{N})\delta \varphi dA=0\mathrm{with}\mathit{D}=-\frac{\partial \mathsf{\Psi }}{\partial \mathit{E}},$$

(4)

for all $\delta \varphi$. Here, denotes the unit normal vector pointing outwards of ∂$ℬ$. By choosing $\mathsf{\Psi }=-1/2\kappa \mathit{E}·\mathit{E}$, the static Maxwell equations are recovered.

2.2. Microscopic Boundary Value Problem

In the present work, we consider homogenization problems governed by the existence of so-called representative volume elements (RVEs). They are chosen in a way that they are statistically representative for the overall microstructure of the material. Fields emerging on the microscale are driven by macroscopic fields, which are assumed to be constant in the RVE due to the separation of length scales. The separation of length scales leads to a degeneration of the RVEs to points on the macroscale. Scale transition rules can be derived from the Hill–Mandel condition [28] by postulating energy conservation between the microscopic RVE and a macroscopic observation in the form

$$\overline{\mathsf{\Pi }}=\underset{\mathit{E}}{\sup }\frac{1}{\left|\mathcal{B}\right|}\mathsf{\Pi }\left(\varphi \right),$$

(5)

where the macroscopic energy potential $\overline{\mathsf{\Pi }}$ is obtained at equilibrium of the microscopic energy potential $\mathsf{\Pi }\left(\varphi \right)$. According to Equation (3), the internal energy density appearing in the potential is now a function of the electric field. In line with the assumption of first-order homogenization and separation of length scales, the electric field vector

$$\mathit{E}=\overline{\mathit{E}}-\nabla \tilde{\varphi }$$

(6)

is decomposed into a macroscopic contribution

$$\overline{\mathit{E}}={\displaystyle \frac{1}{\left|\mathcal{B}\right|}}{\int }_{\mathcal{B}}\mathit{E}\mathrm{d}V,$$

(7)

which is constant on the microscale, and the gradient of the fluctuative scalar electric potential $\tilde{\varphi }$ that acts as primary variable. The system is closed by applying appropriate boundary conditions on the RVE. There are several boundary conditions that fulfill the Hill–Mandel condition (5). In the present work, we focus on periodic boundary conditions of the form

$$\tilde{\varphi }\left({\mathit{x}}^{+}\right)=\tilde{\varphi }\left({\mathit{x}}^{-}\right)\mathrm{and}\mathit{D}·\mathit{N}\left({\mathit{x}}^{+}\right)=-\mathit{D}·\mathit{N}\left({\mathit{x}}^{-}\right)\mathrm{on}\partial \mathcal{B},$$

(8)

where ${\mathit{x}}^{\pm }$ indicate opposite coordinates at the boundary of the RVE [30,31,32]. Note that we only need to prescribe one of the two boundary conditions given in the latter equation as the other one will emerge naturally in equilibrium.

3. Artificial Neural Network Based Solution Scheme

In this section, a solution procedure based on artificial neural networks (ANNs) for finding the equilibrium state of the potential in Equation (3) is presented. Core idea is the construction of trial functions, which consist of ANNs and multiplicative factors fulfilling the given set of boundary conditions in Equation (8). We then recapitulate the two main ANN structures used in the present work, namely the single layer perceptron net and the multilayer perceptron. Additionally, the derivatives of those nets with respect to its inputs as well as with respect to its weights are given as they are needed when using gradient descent methods.

3.1. Optimization Principle

Consider the previously introduced RVE occupying the space . Our goal is to find the microscopic equilibrium state of a given global potential for this body. Under prescribed periodic Dirichlet boundary conditions in Equation (8)${}_1$, the potential in Equation (3) takes the form

$$\mathsf{\Pi }\left(\varphi \right)={\int }_{\mathcal{B}}\mathsf{\Psi }(\mathit{x},\mathit{E})dV.$$

(9)

Having the physical problem covered, the question arises how to approximate the solution fields. While finite element approaches employ local shape functions and Fourier transform based methods use global trigonometric basis functions, in the present work, we want to investigate an approximation method based on artificial neural networks. Following the idea of Lagaris et al. [27], we construct a trial function ${\varphi }_t$ using arbitrary artificial neural networks $N_i(\mathit{x},{\mathit{p}}_i)$ as

$${\tilde{\varphi }}_t(\mathit{x},\mathit{p})=A_0\left(\mathit{x}\right)+A_1\left(\mathit{x}\right)N_1(\mathit{x},{\mathit{p}}_1)+A_2\left(\mathit{x}\right)N_2(\mathit{x},{\mathit{p}}_2)+...+A_n\left(\mathit{x}\right)N_n(\mathit{x},{\mathit{p}}_n),$$

(10)

where $A_i\left(\mathit{x}\right)$ are functions ensuring that the boundary conditions are fulfilled a priori. As a generic one-dimensional example, one could think of a scalar electric potential that should fulfill the boundary conditions $\tilde{\varphi }\left(0\right)=\tilde{\varphi }\left(1\right)=1$. In this case, we would have $A_0=1$ and $A_1=x(1-x)$, yielding the trial function according to Equation (10) as ${\tilde{\varphi }}_t(x,\mathit{p})=1+x(1-x)N_1(x,{\mathit{p}}_1)$. The corresponding electric field ${\mathit{E}}_t(\mathit{x},\mathit{p})$ in line with Equation (6) can be calculated analytically according to the neural network derivatives given in Section 3.2 and Section 3.3. Using the gradient field along with Equation (9) gives the global potential in terms of the neural network’s parameters as follows

$$\mathsf{\Pi }\left(\mathit{p}\right)={\int }_{\mathcal{B}}\mathsf{\Psi }(\mathit{x},{\mathit{E}}_t(\mathit{x},\mathit{p}))dV.$$

(11)

Finally, the objective function in our machine learning problem is obtained from the Hill–Mandel condition in Equation (5) in combination with the global potential in Equation (11) approximated by neural networks. It appears as

$$\begin{array}{|c|}\hline \overline{\mathsf{\Pi }}=\underset{\mathit{p}}{\sup }\frac{1}{\left|\mathcal{B}\right|}\mathsf{\Pi }\left(\mathit{p}\right),\\ \hline\end{array}$$

(12)

where the optimization is carried out with respect to the neural network’s parameters. By having the periodic boundary conditions fulfilled by construction, the optimization can be carried out without any constraints on the parameters p.

3.2. Single Layer Perceptron (SLP)

The single layer perceptron (SLP) is one of the most basic versions of artificial neural networks [33]. It is a reduced version of the general structure depicted in Figure 1. An SLP only consists of one hidden layer that connects the input and the output. Assuming an input vector x of dimension d, the response N of an SLP is calculated as

$$N(\mathit{x},\mathit{p})=\sum _{i=1}^H{v}_i\sigma \left(z_i\right)+\overline{b}\mathrm{with}{z}_i=\sum _{j=1}^d{w}_{ij}{x}_j+u_i,$$

(13)

where $v_i$, $w_{ij}$, $u_i$ and $\overline{b}$ are the weights and biases of the hidden unit and the output bias, respectively, and H is the overall number of neurons in the hidden layer. Those weights and biases are assembled in the neural network parameter vector p. Here, $\sigma$ denotes an activation function of a neuron. The activation functions may be chosen problem-dependent and can have a large impact on the training behavior of the artificial neural network. Figure 2 shows the three activation functions used in the present work, namely the logistic sigmoid, the hyperbolic tangent and the softplus function. Despite its popularity in machine learning tasks, we are not using the rectifier linear unit (ReLu) activation function in the present work. First tests using the ReLu function resulted in poor convergence rates. We suspect that this stems from errors in the numerical integration when using the ReLu function and its discontinuous derivative. The derivatives of the SLP net with respect to its input then appear as

$$N_{,j}=\frac{\partial N}{\partial x_j}=\sum _{i=1}^H{v}_i{w}_{ij}{\sigma }^{\prime }\left(z_i\right),$$

(14)

where ${\sigma }^{\prime }\left(z_i\right)$ denotes the derivative of the sigmoid function with respect to its argument. The spatial derivative can be perceived as an SLP with modified weights and activation function but it is now a gradient field. To use efficient gradient based solvers when optimizing the weights of the neural network, it is convenient to have the explicit derivatives of the ANNs with respect to the weights. These can be obtained as

$$\begin{array}{|c|}\hline \begin{array}{cccc}\hfill \frac{\partial N}{\partial u_i}& =v_i{\sigma }^{\prime }\left(z_i\right),\hfill & \hfill \frac{\partial N_{,j}}{\partial u_i}& =v_i{w}_{ij}{\sigma }^{\prime \prime }\left(z_i\right),\hfill \\ \hfill \frac{\partial N}{\partial w_{ij}}& =v_i{x}_j{\sigma }^{\prime }\left(z_i\right),\hfill & \hfill \frac{\partial N_{,j}}{\partial w_{im}}& =x_m{v}_i{w}_{ij}{\sigma }^{\prime \prime }\left(z_i\right)+v_i{\sigma }^{\prime }\left(z_i\right){\delta }_{jm},\hfill \\ \hfill \frac{\partial N}{\partial \overline{b}}& =1,\hfill & \hfill \frac{\partial N_{,j}}{\partial \overline{b}}& =0,\hfill \\ \hfill \frac{\partial N}{\partial v_i}& =\sigma \left(z_i\right),\hfill & \hfill \frac{\partial N_{,j}}{\partial v_i}& =w_{ij}{\sigma }^{\prime }\left(z_i\right).\hfill \end{array}\\ \hline\end{array}$$

(15)

Note that, in the derivatives above, the indices are not treated by Einstein’s summation convention. Indices appearing twice are rather multiplied pointwise in MATLAB [34] convention.

3.3. Multilayer Perceptron (MLP)

The multilayer perceptron (MLP) works similarly to the single layer perceptron. However, it is constructed by a higher number L of hidden layers. It can be shown that this deep structure enables a more general approximation property of the neural network and might lead to better training behavior [35]. In the present work, we focus on MLPs with only two hidden layers. The output is then computed as

$$N(\mathit{x},\mathit{p})=\sum _{k=1}^{H_2}{v}_k\sigma \left(r_k\right)+\overline{b},r_k=\sum _{i=1}^{H_1}{\theta }_{ki}\sigma \left(z_i\right)+c_k\mathrm{with}{z}_i=\sum _{j=1}^d{w}_{ij}{x}_j+u_i,$$

(16)

where we have now additional weights ${\theta }_{ki}$ and biases $c_k$ associated with the second hidden layer. The spatial derivative of the MLP appears as

$$N_{,j}=\frac{\partial N}{\partial x_j}=\sum _{k=1}^{H_2}\sum _{i=1}^{H_1}{v}_k{\sigma }^{\prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)w_{ij}.$$

(17)

The derivatives of the MLP with respect to the weights are computed as

$$\begin{array}{|c|}\hline \begin{array}{cccc}\hfill \frac{\partial N}{\partial u_i}& =\sum _{k=1}^{H_2}{v}_k{\sigma }^{\prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime }\left(z_i\right),\hfill & \hfill \frac{\partial N_{,j}}{\partial u_i}& =\sum _{k=1}^{H_2}[v_k{\sigma }^{\prime \prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)\sum _{i=1}^{H_1}\left[w_{ij}{\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)\right]\hfill \\ \hfill & & & \phantom{=}+v_k{\sigma }^{\prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime \prime }\left(z_i\right)w_{ij}\left)\right],\hfill \\ \hfill \frac{\partial N}{\partial w_{ij}}& =\sum _{k=1}^{H_2}{v}_k{\sigma }^{\prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)x_j,\hfill & \hfill \frac{\partial N_{,j}}{\partial w_{im}}& =\sum _{k=1}^{H_2}[v_k{\sigma }^{\prime \prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)x_m\sum _{i=1}^{H_1}\left[w_{ij}{\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)\right]\hfill \\ & & & \phantom{=}+v_k{\sigma }^{\prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime \prime }\left(z_i\right)w_{ij}{x}_m\hfill \\ & & & \phantom{=}+v_k{\sigma }^{\prime }\left(r_k\right){\theta }_{ki}{\sigma }^{\prime }\left(z_i\right){\delta }_{im}],\hfill \\ \hfill \frac{\partial N}{\partial c_k}& =v_k{\sigma }^{\prime }\left(r_k\right),\hfill & \hfill \frac{\partial N_{,j}}{\partial c_k}& =v_k{\sigma }^{\prime \prime }\left(r_k\right)\sum _{i=1}^{H_1}\left[w_{ij}{\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)\right],\hfill \\ \hfill \frac{\partial N}{\partial {\theta }_{ki}}& =v_k{\sigma }^{\prime }\left(r_k\right)\sigma \left(z_i\right),\hfill & \hfill \frac{\partial N_{,j}}{\partial {\theta }_{ki}}& =v_k{\sigma }^{\prime \prime }\left(r_k\right){\sigma }^{\prime }\left(z_i\right)\sum _{i=1}^{H_1}\left[w_{ij}{\theta }_{ki}\sigma \left(z_i\right)\right]\hfill \\ \hfill & & & \phantom{=}+v_k{\sigma }^{\prime }\left(r_k\right){\sigma }^{\prime }\left(z_i\right)w_{ij},\hfill \\ \hfill \frac{\partial N}{\partial \overline{b}}& =1,\hfill & \hfill \frac{\partial N_{,j}}{\partial \overline{b}}& =0,\hfill \\ \hfill \frac{\partial N}{\partial v_k}& =\sigma \left(r_k\right),\hfill & \hfill \frac{\partial N_{,j}}{\partial v_k}& ={\sigma }^{\prime }\left(r_k\right)\sum _{i=1}^{H_1}\left[w_{ij}{\theta }_{ki}{\sigma }^{\prime }\left(z_i\right)\right].\hfill \end{array}\\ \hline\end{array}$$

(18)

4. Numerical Examples

In this section, we test the robustness and reliability of the proposed method on a set of one-, two- and three-dimensional problems. The influence of global versus piecewise-defined constructions of the trial functions as well as the impact of the neuron count on the simulation results are explored. The one-dimensional example is implemented in MATLAB [34] while the two- and three-dimensional examples are carried out by a Fortran 77 code that utilizes the L-BFGS-B optimization algorithms [36,37]. For the sake of simplicity, all the following simulations are performed with normalized units.

4.1. One-Dimensional Example

We first consider a one-dimensional problem to demonstrate the features of the proposed method. The RVE is a simple two-phase laminate of unit length $l=1$ and unit cross section, which is loaded with a macroscopic electric field $\overline{E}=0.01$. The global potential takes the form

$$\mathsf{\Pi }\left(\varphi \right)={\int }_{\mathcal{B}}\mathsf{\Psi }(E,x)dx=-{\int }_{\mathcal{B}}\frac{1}{2}\kappa E^{2}dx,$$

(19)

where ${\kappa }_1=1$ for $x<0.5$ and ${\kappa }_2=2$ for $x>0.5$. Having the decomposition in Equation (6)

$$E=\overline{E}-\frac{\partial \tilde{\varphi }}{\partial x}$$

(20)

along with the boundary conditions $\tilde{\varphi }\left(0\right)=0$ and $\tilde{\varphi }\left(1\right)=0$, the analytical solution for the electric field reads

$$E=\left\{\begin{array}{cc}{\displaystyle \frac{2{\kappa }_2}{{\kappa }_1+{\kappa }_2}}\overline{E}\hfill & 0\le x<l/2\hfill \\ {\displaystyle \frac{2{\kappa }_1}{{\kappa }_1+{\kappa }_2}}\overline{E}\hfill & l/2<x\le l\hfill \end{array}\right.$$

(21)

and for the electric potential

$$\tilde{\varphi }=\left\{\begin{array}{cc}\overline{E}{\displaystyle \frac{{\kappa }_1-{\kappa }_2}{{\kappa }_1+{\kappa }_2}}x\hfill & 0\le x\le l/2\hfill \\ \overline{E}{\displaystyle \frac{{\kappa }_2-{\kappa }_1}{{\kappa }_1+{\kappa }_2}}(x-l)\hfill & l/2\le x\le l\hfill \end{array}\right..$$

(22)

4.1.1. Global Neural Net Approach on Equidistant Grid

To find the electric scalar potential that optimizes the energy potential in Equation (19), we construct a global trial function according to Equation (10) that automatically fulfills the boundary conditions given above as

$${\tilde{\varphi }}_t=A_o+A_1\left(x\right)N(x,\mathit{p})=x(1-x)N(x,\mathit{p}),$$

(23)

where N is a neural network that takes x as an input and has the weights and biases p. The derivatives in this case can be computed explicitly as

$$\frac{\partial {\tilde{\varphi }}_t}{\partial x}=(1-2x)N(x,\mathit{p})+x(1-x)\frac{\partial N(x,\mathit{p})}{\partial x}.$$

(24)

The derivatives then allow us to compute the global potential in terms of the neural network’s weights and biases as

$$\mathsf{\Pi }\left(\mathit{p}\right)=-{\int }_{\mathcal{B}}\frac{1}{2}\kappa {(\overline{E}-\frac{{\tilde{\varphi }}_t}{\partial x})}^{2}dx.$$

(25)

Finally, we need a numerical integration scheme to evaluate the integral. In the present work, we use quadrature points in terms of equidistant grid points $x_k$ with distance $\Delta x$ on the interval of $[0,1]$ as $\{\Delta x/2,3\Delta x/2,\dots ,1-\Delta x/2\}$, yielding the discrete objective

$$\begin{array}{|c|}\hline \overline{\mathsf{\Pi }}=\underset{\mathit{p}}{\sup }{\displaystyle \frac{1}{l}}{\displaystyle \sum _{0<x_k<1.0}}-\frac{1}{2}\kappa {(\overline{E}-\frac{{\tilde{\varphi }}_t}{\partial x})}^2\Delta x.\\ \hline\end{array}$$

(26)

The maximum of this objective function can be found by means of the gradient descent method. The gradients of the objective with respect to the weights p that are needed for such an iterative solver can be computed using Equation (15). We then have everything at hand to carry out the first numerical example. As for the ANN architecture, we use an SLP with $H=10$ neurons in the hidden layer. As for the activation function, the logistic sigmoid function $\sigma \left(z\right)=1/(1+e^{-z})$ is chosen, and the weights and biases p are randomly initalized with a uniform distribution between 0 and 1. Figure 3 shows the result of the numerical experiment (green) compared to the analytical results (black). One can see that the numerical scalar electric potential ${\tilde{\varphi }}_t$ is close to the analytical solution. However, having a look at the gradients in the form of E reveals the occurrence of oscillations around the discontinuity. The MATLAB [34] code that generates these results can be found in Appendix A. Note that the tolerances for the step size and the optimality as well as the maximum number of function evaluations and iterations is set different from the MATLAB [34] default values to obtain reasonable results. One might decrease the oscillations by having even higher iteration numbers. However, the jump in the solution at the vicinity of the material jump cannot be captured by the global smooth neural network function. The more neurons we use, the better accuracy we obtain, which leads to a trade-off between accuracy and speed.

4.1.2. Piecewise-defined Neural Net Approach on Equidistant Grid

To overcome the oscillations observed in the global approach, we next construct a trial function that is piecewise defined for the RVE’s different material regions

$${\tilde{\varphi }}_t=\left\{\begin{array}{cc}x{N}_1(x,{\mathit{p}}_1)\hfill & 0<x\le 0.5\hfill \\ (0.5-x)(1-x)N_2(x,{\mathit{p}}_2)+2(1-x){\tilde{\varphi }}_t(0.5)\hfill & 0.5<x\le 1.0\hfill \end{array}\right..$$

(27)

Its derivatives can be computed according to

$$\frac{\partial {\tilde{\varphi }}_t}{\partial x}=\left\{\begin{array}{cc}{N}_1(x,{\mathit{p}}_1)+x{\displaystyle \frac{\partial N_1(x,{\mathit{p}}_1)}{\partial x}}\hfill & 0<x<0.5\hfill \\ (2x-1.5)N_2(x,{\mathit{p}}_2)+(0.5-x)(1-x){\displaystyle \frac{\partial N_2(x,{\mathit{p}}_2)}{\partial x}}-2{\tilde{\varphi }}_t(0.5)\hfill & 0.5<x<1.0\hfill \end{array}\right..$$

(28)

The global potential in terms of the neural network’s weights and biases then appears as

$$\mathsf{\Pi }\left(\mathit{p}\right)={\int }_{\mathcal{B}}\frac{1}{2}\kappa {(\overline{E}-\frac{{\tilde{\varphi }}_t}{\partial x})}^{2}dx.$$

(29)

In analogy to the global approach, numerical integration is performed using equidistant grid points $x_k$ with distance $\Delta x$ as quadrature points in the interval $[0,1]$ to obtain the discrete objective

$$\begin{array}{|c|}\hline \overline{\mathsf{\Pi }}=\underset{\mathit{p}}{\sup }{\displaystyle \frac{1}{l}}\left({\displaystyle \sum _{0<x_k<0.5}}-\frac{1}{2}{\kappa }_1{(\overline{E}-\frac{{\tilde{\varphi }}_t}{\partial x})}^2-{\displaystyle \sum _{0.5<x_k<1.0}}\frac{1}{2}{\kappa }_2{(\overline{E}-\frac{{\tilde{\varphi }}_t}{\partial x})}^2\right)\Delta x.\\ \hline\end{array}$$

(30)

The gradient of the objective function with respect to the parameters p can again be obtained through the derivatives of Equation (15). However, the output and training behavior of artificial neural networks is dependent on the initialization of the weights and biases p. In a first run, we set the number of neurons in the two SLPs’ hidden layers to 10 and initialize the weights randomly between 0 and 1. As for the activation function, we choose the logistic sigmoid function $\sigma \left(z\right)=1/(1+e^{-z})$.

Figure 4 shows the numerical results in green compared to the analytical results in black. There is a distinct deviation between them. Having a look at the output layer of the SLP in Equation (13) and its derivative in Equation (14), one can see that the initial output for 10 neurons for an initialization of all weights between 0 and 1 is far from the exact solution. This difference becomes even larger for higher number of neurons. In contrast to the global approach, we use the default values of the unconstrained MATLAB [34] minimizer and, apparently, there are too few iterations.

Next, we want to have fewer iterations of the solver within the default values by using an adaptive way of initializing the weights. The key idea is to initialize the weights in a way that the net and its derivative output values are roughly in the range of the values we would expect with respect to the given macroscopic load. We therefore use a simple modification of the weight initialization given as

$${\mathit{p}}^{\left(0\right),*}\sim \frac{\overline{E}}{H}*\mathcal{U}(0,1),$$

(31)

where $\mathcal{U}(0,1)$ is a vector of random numbers uniformly distributed along 0 and 1. Please note that we use boldface calligraphic $𝒰$ to indicate the vector notation instead of univariate distribution. The results of the computation using the adaptive initialization method can be seen in Figure 5. The results are closer to the analytical solution and are independent of the number of hidden neurons used, making the overall method much more reliable. The MATLAB [34] code used for generating Figure 5 can be found in Appendix B.

4.2. Two-Dimensional Example

Next, we consider a two-dimensional microstructure with a circular inclusion, as shown in Figure 6. The radius of the inclusion is $r_0=0.178l$, corresponding to 10% volume fraction. The global energy potential per unit out-of-plane thickness is given as the integral of the internal energy density over the RVE’s domain

$$\mathsf{\Pi }={\int }_{\mathcal{B}}\mathsf{\Psi }\left(\mathit{E}\right)dA=-{\int }_{\mathcal{B}}\frac{1}{2}\kappa \mathit{E}·\mathit{E}dA.$$

(32)

In this example, the material parameter is set to $\kappa =1$ in the matrix and $\kappa =10$ in the inclusion. The square RVE of unit length $l=1$ is loaded with the constant macroscopic field ${\overline{E}}_1=1$ and ${\overline{E}}_2=0$ under the periodic Dirichlet boundary conditions in Equation (8)${}_1$. For reference, a simulation using the finite element method is performed for this optimization problem. The mesh is discretized by linear quadrilateral elements with four Gauss points. The optimized global potential is calculated to $|\overline{\mathsf{\Pi }}{|}^{\mathrm{FEM}}=0.588652$. Figure 6 shows the contour plot of the microscopic field $E_1$. The finite element results serve as a reference for the neural network based approaches in the following examples.

4.2.1. Global Neural Net Approach on Equidistant Grid

First, we want to build a global trial function, which is covering the whole RVE. As we want to implement periodic boundary conditions, a set of three neural networks is used to construct the trial function ${\varphi }_t$: $N_{x_1}\left(x_1\right)$ acting in the $x_1$-direction, $N_{x_2}\left(x_2\right)$ acting in the $x_2$-direction and $N_1\left(\mathit{x}\right)$ acting in both directions. The trial function then takes the form

$$\begin{array}{cc}\hfill {\tilde{\varphi }}_t(\mathit{x},\mathit{p})& =A_1(x_1,x_2)N_1(,{\mathit{p}}_1)+A_2\left(x_1\right)N_{x_1}(x_1,{\mathit{p}}_{x_1})+A_3\left(x_2\right)N_{x_2}(x_2,{\mathit{p}}_{x_2})\hfill \\ & =x_1(1-x_1)x_2(1-x_2)N_1(,{\mathit{p}}_1)\hfill \\ & \phantom{=}+x_1(1-x_1)N_{x_1}(x_1,{\mathit{p}}_{x_1})+x_2(1-x_2)N_{x_2}(x_2,{\mathit{p}}_{x_2}).\hfill \end{array}$$

(33)

Figure 7 shows a visualization of the functions ensuring the boundary conditions. Here, we use SLPs for the boundary networks and a two-layer MLP for the two-dimensional neural network. As for the activation function for the neurons, the hyperbolic tangent $\sigma \left(z\right)=\tan \mathrm{h}\left(z\right)$ is chosen. The negative gradient of the trial function can then be computed to obtain the trial field

$${\mathit{E}}_t=\overline{\mathit{E}}-\nabla \widetilde{{\varphi }_t},$$

(34)

where we drop dependencies on x and p in our notation. According to Equation (32), we arrive at the global potential as an objective

$$\overline{\mathsf{\Pi }}=\underset{\mathit{p}}{\sup }\frac{1}{\left|\mathcal{B}\right|}\mathsf{\Pi }\left(\mathit{p}\right)=\underset{\mathit{p}}{\sup }\frac{1}{\left|\mathcal{B}\right|}{\int }_{\mathcal{B}}-\frac{1}{2}\kappa {\left(\mathit{x}\right)}_t{\mathit{E}}_t·{\mathit{E}}_{t}dA.$$

(35)

The spatial gradient of the trial function $\nabla \widetilde{{\varphi }_t}$ as well as gradients of the global potential $\partial \mathsf{\Pi }\left(\mathit{p}\right)/\partial \mathit{p}$ used in optimization algorithms can be obtained analytically through the derivatives given in Section 3.2 and Section 3.3. We use equidistant grid points for establishing a regular mesh of elements and employ nine Gauss points per element as the quadrature points for numerical integration. The objective is optimized without any constraints on the parameters p, as we satisfy the boundary conditions a priori by the construction of ${\tilde{\varphi }}_t$.

The simulation is carried out for a different number of neurons and integration points as follows: (a) 51 × 51 elements, 15 neurons in each of the two hidden layers of the MLP and 5 neurons each in the layer of the boundary SLPs; (b) 51 × 51 elements, 10 neurons in each of the two hidden layers of the MLP and 5 neurons each in the layer of the boundary SLPs; and (c) 101 × 101 elements, 15 neurons in each of the two hidden layers of the MLP and 5 neurons each in the layer of the boundary SLPs. Additionally, all three set ups are run with a uniform initialization of the weights through

$${\mathit{p}}^{\left(0\right)}\sim \mathcal{U}(-1,1).$$

(36)

Figure 8 shows the contour plot of $E_{t1}$ of the neural network after 20,000 iterations. One can see that the neural network in Case (a) localizes in an unphysical state. This could be a problem related to overfitting. Case (b), with the same amount of integration points but lower neuron count, shows a qualitatively better result. The overall optimization seems to become more reliable as the integration is performed more accurately, as seen in Case (c). Quantitatively, the global potentials are: (a) $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.213522$; (b) $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.590266$; and (c) $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.589887$ compared to the FEM potential of $|\overline{\mathsf{\Pi }}{|}^{\mathrm{FEM}}\left(\mathit{E}\right)=0.588652$, taken from the simulation that creates Figure 6. In two dimensions and with complicated geometries at hand, it can be difficult to estimate the magnitude of the solution fields appearing a priori. A rather simple approach for initializing the weights as described in the one-dimensional case did not seem to significantly improve the method. Here, we test a second approach of initializing the weights according to a normal distribution

$${\mathit{p}}^{\left(0\right),*}\sim \mathcal{N}(\mu ,{\sigma }^2),$$

(37)

where we set the mean $\mu =0$ and the variance ${\sigma }^2=1$. Figure 8 shows the contour plot of $E_{t1}$ of the neural network after 20,000 iterations for the normal distribution. Case (a) now shows a more physical state. The energies are calculated as: (a) $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.590054$; (b) $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.592981$; and (c) $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.590544$. At this point, there is surely some potential left for improving weight initialization. This is subject to other fields of machine learning as well, especially in the context of training speed and vanishing gradient problems [38].

4.2.2. Piecewise-Defined Neural Net Approach

To further improve the training and approximation properties of the trial functions, we next construct it in a way that it captures the discontinuity of material properties in the microstructure a priori. This is possible due to the simple topology of the microstructure at hand. However, for more complicated microstructures, defining explicit expressions for the trial function might be difficult. We now need a set of four ANNs for the construction of ${\varphi }_t$: $N_{x_1}\left(x_1\right)$ acting in the $x_1$-direction, $N_{x_2}\left(x_2\right)$ acting in the $x_2$-direction, $N_1\left(\mathit{x}\right)$ acting in the matrix and $N_2\left(\mathit{x}\right)$ acting in the inclusion. The global trial function is defined piecewise in two sub-domains as follows

$$\widetilde{{\varphi }_t}=\left\{\begin{array}{cc}{x}_1(1-x_1)x_2(1-x_2)N_1(\mathit{x},{\mathit{p}}_1)+x_1(1-x_1)N_{x_1}(x_1,{\mathit{p}}_{x_1})\hfill & r\ge r_0\hfill \\ +x_2(1-x_2)N_{x_2}(x_2,{\mathit{p}}_{x_2})\hfill & \\ \widetilde{{\varphi }_t}(r_0,\phi )+(r_0-r)N_2(\mathit{r},{\mathit{p}}_1)\hfill & r<r_0\hfill \end{array}\right..$$

(38)

One can see that the above equations automatically fulfill periodicity at the boundaries as well as the transition condition at the phase interface of the circular inclusion. Here, we implement the trial function in Cartesian coordinates for the matrix and in polar coordinates for the inclusion. Thus, the neural network of the inclusion takes the coordinate vector $\mathit{r}=(r,\phi )$ in terms of the radius r and the angle $\phi$ as its input. Having the origin of our coordinate system in the bottom left corner, the transformation used here takes the form

$$x_1=0.5+r\cos \phi \mathrm{and}{x}_2=0.5+r\sin \phi .$$

(39)

With the coordinate transformation at hand, one can calculate the gradient in the matrix and inclusion as

$${\mathit{E}}_t=-\nabla (-\overline{\mathit{E}}·\mathit{x}+\widetilde{{\varphi }_t}),$$

(40)

where $\overline{\mathit{E}}$ is the applied macroscopic electric field. For a more detailed derivation, see Appendix C. Finally, the potential to optimize in this example takes the form

$$\overline{\mathsf{\Pi }}=\underset{\mathit{p}}{\sup }\frac{1}{\left|\mathcal{B}\right|}\mathsf{\Pi }\left(\mathit{p}\right)=\underset{\mathit{p}}{\sup }\frac{1}{\left|\mathcal{B}\right|}\left(-{\int }_{{\mathcal{B}}^{\mathrm{matr}}}\frac{1}{2}{\kappa }_1{\mathit{E}}_t·{\mathit{E}}_{t}dA-{\int }_{{\mathcal{B}}^{\mathrm{incl}}}\frac{1}{2}{\kappa }_2{\mathit{E}}_t·{\mathit{E}}_{t}dA\right).$$

(41)

As a numerical integration scheme, we use a simple one-point quadrature rule, where we have an equidistant grid of 3240 integration points in the matrix and 3600 integration points in the inclusion. The MLP $N_1(\mathit{x},{\mathit{p}}_1)$ has two layers with five neurons in each, the SLP $N_2(\mathit{x},{\mathit{p}}_2)$ has one layer with four neurons and the boundary SLPs $N_{x_1}(x_1,{\mathit{p}}_{x_1})$ and $N_{x_2}(x_2,{\mathit{p}}_{x_2})$ have seven neurons in the hidden layer. This sums up to a total of 116 degrees of freedom p. In the present example, we use the hyperbolic tangent $\sigma \left(z\right)=\tan \mathrm{h}\left(z\right)$ as the activation function of the neurons. The initial weights are randomly initialized with uniform distribution in the range of $-1$ to 1. As in the previous examples, the applied macroscopic load in Equation (40) is ${\overline{E}}_1=1.0$ and ${\overline{E}}_2=0.0$.

Figure 9 shows the result for $E_{t1}$ after 10,000 iterations for the local construction of the trial function. Compared with the previous simulations using only one global trial function, the fields have fewer oscillations. Additionally, the maximum energy after 10,000 iterations is $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.588414$, being lower than in the previous simulations and lower than the maximum energy computed with finite elements. Interestingly, in the first iterations, the learning rates are higher for the global schemes, whether the weights are initialized according to a normal distribution or a uniform distribution (see Figure 9).

4.3. Three-Dimensional Example

In the present example, we apply the method to a three-dimensional cubic RVE of unit length $l=1$ with a spherical inclusion of radius $r_0=0.178l$. We first construct a global trial function, for which we need a set of seven neural networks: $N_{x_1}\left(x_1\right)$ acting in the $x_1$-direction, $N_{x_2}\left(x_2\right)$ acting in the $x_2$-direction, $N_{x_3}\left(x_3\right)$ acting in the $x_3$-direction, $N_{x_{12}}(x_1,x_2)$ acting in the $x_1{x}_2$-plane, $N_{x_{13}}(x_1,x_3)$ acting in the $x_1{x}_3$-plane, $N_{x_{23}}(x_2,x_3)$ acting in the $x_1{x}_2$-plane and $N_1\left(\mathit{x}\right)$ acting in the RVE’s volume (see Figure 10). The trial function for the matrix then appears as

$$\begin{array}{cc}\hfill {\tilde{\varphi }}_t(\mathit{x},\mathit{p})=\phantom{+}& A_1{N}_1(\mathit{x},{\mathit{p}}_1)+A_2{N}_{x_{12}}(x_1,x_2,{\mathit{p}}_{x_{12}})+A_3{N}_{x_{13}}(x_1,x_3,{\mathit{p}}_{x_{13}})\hfill \\ & +A_4{N}_{x_{23}}(x_2,x_3,{\mathit{p}}_{x_{23}})+A_5{N}_{x_1}(x_1,{\mathit{p}}_{x_1})\hfill \\ & +A_6{N}_{x_2}(x_2,{\mathit{p}}_{x_2})+A_7{N}_{x_3}(x_3,{\mathit{p}}_{x_3}),\hfill \end{array}$$

(42)

where the functions $A_i$ take the form

$$\begin{array}{cccc}\hfill A_1& =x_1(1-x_1)x_2(1-x_2)x_3(1-x_3),\hfill & \hfill A_5& =x_1(1-x_1),\hfill \\ \hfill A_2& =x_1(1-x_1)x_2(1-x_2),\hfill & \hfill A_6& =x_2(1-x_2),\hfill \\ \hfill A_3& =x_1(1-x_1)x_3(1-x_3),\hfill & \hfill A_7& =x_3(1-x_3),\hfill \\ \hfill A_4& =x_2(1-x_2)x_3(1-x_3).\hfill \end{array}$$

(43)

By construction, the trial function fulfills the periodic boundary conditions. The negative gradient of the trial function can then be computed analytically according to Equation (40) (see also Appendix D for a more detailed derivation). With the gradient at hand, we are able optimize the global potential in Equation (32) with respect to the weights of the ANNs.

As seen in the previous sections, the use of one global trial function might lead to oscillations in the solution field. In line with Section 4.2.2, we additionally construct a piecewise-defined trial function for comparison

$${\widetilde{{\varphi }_t}}^{*}(\mathit{x},\mathit{p})=\left\{\begin{array}{cc}{A}_1{N}_1(\mathit{x},{\mathit{p}}_1)+A_2{N}_{x_{12}}(x_1,x_2,{\mathit{p}}_{x_{12}})+A_3{N}_{x_{13}}(x_1,x_3,{\mathit{p}}_{x_{13}})\hfill & r\ge r_0\hfill \\ +A_4{N}_{x_{23}}(x_2,x_3,{\mathit{p}}_{x_{23}})+A_5{N}_{x_1}(x_1,{\mathit{p}}_{x_1})+A_6{N}_{x_2}(x_2,{\mathit{p}}_{x_2})\hfill & \\ +A_7{N}_{x_3}(x_2,{\mathit{p}}_{x_3})\hfill & \\ \widetilde{{\varphi }_t}(r_0,\phi ,\theta )+A_8{N}_2(\mathit{r},{\mathit{p}}_2)\hfill & r<r_0.\hfill \end{array}\right.$$

(44)

where we add an eighth neural network for the inclusion and the function

$$A_8=r_0-r.$$

(45)

The transformation between the spherical coordinates $\mathit{r}=(r,\phi ,\theta )$ and the Cartesian coordinates $\mathit{x}=(x_1,x_2,x_3)$ is given as

$$x_1=0.5+r\sin \phi \cos \theta ,x_2=0.5+r\sin \phi \sin \theta ,x_3=r\cos \phi .$$

(46)

The gradient in Equation (40) can then be computed accordingly in Cartesian coordinates for the matrix and in spherical coordinates for the inclusion, as derived in Appendix D, allowing us to optimize the global potential

$${\overline{\mathsf{\Pi }}}^{*}=\underset{\mathit{p}}{\sup }\frac{1}{\left|\mathcal{B}\right|}{\mathsf{\Pi }}^{*}\left(\mathit{p}\right)=\underset{\mathit{p}}{\sup }\frac{1}{\left|\mathcal{B}\right|}\left(-{\int }_{{\mathcal{B}}^{\mathrm{matr}}}\frac{1}{2}{\kappa }_1{\mathit{E}}_{\mathrm{t}}^{*}·{\mathit{E}}_t^{*}dV-{\int }_{{\mathcal{B}}^{\mathrm{incl}}}\frac{1}{2}{\kappa }_2{\mathit{E}}_{\mathrm{t}}^{*}·{\mathit{E}}_t^{*}dV\right).$$

(47)

For our numerical experiment, the RVE depicted in Figure 10 is loaded with a macroscopic field of ${\overline{E}}_1=1.0$, ${\overline{E}}_2=0.0$ and ${\overline{E}}_3=0.0$. The material parameters are chosen as ${\kappa }_1=1$ and ${\kappa }_2=10$. For the global approach, the mesh consists of 43 × 43 × 43 equidistant integration points. The ANNs acting on the edges and surface boundaries are SLPs, having four neurons along each edge ANN and five neurons along each surface ANN. The ANN acting in the volume is a two-layer perceptron with eight neurons in each layer. This trial function totals 256 parameters to optimize in the ANNs. As for the activation function, we choose the softplus function, as it provides the best results in our examples. As for the piecewise-defined trial function in Equation (44), the ANNs are constructed in the same way. The additional ANN acting in the inclusion is an SLP with eight neurons and also has the softplus activation function. The mesh used in this case consists of 32,768 integration points in the matrix and 16,384 integration points in the inclusion. Additionally, an FFT-based simulation [23] using a conjugate gradient solver [24,39,40,41,42] is carried out for comparison, where the microstructure is discretized on a 43 × 43 × 43 grid.

Figure 11 shows the contour plot of $E_{t1}$ for the approach using the global and the piecewise-defined trial function after 20,000 iterations. In comparison to the FFT-based solution, both simulations produce qualitatively good results. The global approximation scheme captures the expected jump of $E_{t1}$ across the phase interface a little less distinctly compared with the piecewise-defined one. However, the piecewise-defined approximation is more difficult to construct and will be quite challenging to implement in the case of complicated microstructure geometries. Quantitatively, the optimized global potentials after 20,000 iterations are quite close to each other, with $|\overline{\mathsf{\Pi }}|\left(\mathit{p}\right)=0.529926$ for the global approach and $|\overline{\mathsf{\Pi }}{|}^{*}\left(\mathit{p}\right)=0.529692$ for the piecewise-defined approach, compared to a optimum energy $|\overline{\mathsf{\Pi }}{|}^{\mathrm{FFT}}=0.527590$ obtained by the FFT-based simulation.

5. Conclusions

We presented a solution scheme to periodic boundary value problems in homogenization based on the training of artificial neural networks. They were employed in tandem with the multiplicative factors in a way that the resulted trial function fulfilled the boundary conditions a priori and thus we arrived at the unconstrained optimization problem. The numerical examples showed that physically reasonable results can be obtained with rather low amounts of neurons, which allows for low memory demand. A construction of trial functions by defining them piecewise in separate sub-domains led to lower oscillations and a generally more stable training behavior compared with a global approach but was geometrically more challenging to construct. The scheme carried over to three dimensions quite well. We assume this to stem from the ratio of neurons compared with the number of integration points: In the considered example of a cube-shaped matrix with spherical inclusion, the solution could be approximated using a quite low neuron count while the number of integration points grew a lot. For future progress, the training speed of the neural network needs to be improved. More sophisticated ANN structures such as deep nets or recurrent nets might further improve the approximation and training behavior of the method, while methods such as dropout or regularization might assist to avoid problems of overfitting.