Learning Effective Good Variables from Physical Data

Abstract

We assume that a sufficiently large database is available, where a physical property of interest and a number of associated ruling primitive variables or observables are stored. We introduce and test two machine learning approaches to discover possible groups or combinations of primitive variables, regardless of data origin, being it numerical or experimental: the first approach is based on regression models, whereas the second on classification models. The variable group (here referred to as the new effective good variable) can be considered as successfully found when the physical property of interest is characterized by the following effective invariant behavior: in the first method, invariance of the group implies invariance of the property up to a given accuracy; in the other method, upon partition of the physical property values into two or more classes, invariance of the group implies invariance of the class. For the sake of illustration, the two methods are successfully applied to two popular empirical correlations describing the convective heat transfer phenomenon and to the Newton’s law of universal gravitation.

1. Introduction

Theoretical modeling and numerical simulations have become invaluable tools for the current scientific and technological advancement across various fields [1]. In essence, models make use of a mathematical description of the physical laws by establishing the relationships between physical variables. The latter variables have the key role of describing, in a complete and non-redundant manner, a system of interest. As such, their correct identification is never a trivial task, especially in systems with little prior knowledge [2,3,4].

Often, in order to effectively model complex systems, it is favorable to search for a more convenient description with a reduced number of effective variables [5,6]. In other words, the system behavior is not described by the directly accessible physical variables, but rather by some groups or combinations. In this respect, back in the late 19th century, the Buckingham theorem was introduced [7,8,9]. In the latter approach, based on dimensional analysis, it is possible to mix the primitive physical variables, thus creating fewer dimensionless numbers which are effectively relevant.

Over the years, sophisticated methodologies have also been developed based on machine learning, data mining and other data-driven approaches [10,11,12]. Some other approaches aim to unlock the discovery of symbolic expressions accurately matching data derived from an unknown function. This problem has been tackled with a number of methods [13,14], including sparse regression [15,16,17,18], genetic algorithms [19,20,21] and physics-inspired algorithms like AI Feynman, introduced by Udrescu and Tegmark [22]. Inspired by the latter, some authors of this work presented a multi-objective optimization procedure for reducing the set of composition-based material descriptors by optimally mixing them in power combination form. This resulted in improved classification performances, as demonstrated in a case study focused on superconductors [23]. Moreover, the same procedure was applied by Bonke et al. [24] to identify an effective reduced set of variables in a micelle-based photocatalytic system for solar fuels production. Specifically, the algorithm allowed for analytically mixing five physical primitive variables into two synthetic features for the optimal binary separation of the experimental samples according to their performances. The practical benefit of such an approach lies in its capability to replace an experimental sample achieving high performance with an alternative combination of its elemental components. This gives the possibility of reducing the most expensive ones, and re-balancing the others, at no cost on the overall performance. To our knowledge, developing a robust method for identifying symmetries in descriptors, and thereby determining analytically mixed features that govern a specific phenomenon, remains an unresolved issue [2,22]. Within this framework, the scientific question we aim to address is the same of interpretability algorithms like SHAP [25], i.e., identifying relevant features over a trained ML model. However, SHAP works in the original features space, even when aggregations of such features more effectively would explain the phenomenon of interest. Other recent works deal with methods for feature selection/removal by means of genetic algorithms (but not mixing) [26], for identification of the minimal intrinsic variables (but not analytically) [3]. Also, Udrescu and Tegmark [22] consider only a few possible symmetries in the data.

In this work, we present a general and automated methodology, suitable for regression tasks, able to identify groups and/or group sets of variables in the power form $x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}\cdots x_m^{{\alpha }_p}$. We show its effectiveness on popular thermo-fluid dynamics correlations (i.e., Dittus–Boelter and Gnielinski). Furthermore, we also demonstrate that the approach can be easily extended to a more general functional form, proving its ability on Newton’s law of universal gravitation. Also, we employ the optimal feature mixing procedure in ref. [23] for classification tasks, showing its successful application on the former two problems of this study. We refer to the resulting variable groups/sets as effective good variables, since either the examined property of interest or the class effectively depend on groups or sets of features rather than on individual features considered separately. Thus, such effective good variables comply with the definition adopted by Chen et al. [2], as a “complete and non-redundant description of the relevant system”. An overview of the two proposed methodologies is depicted in Figure 1.

The paper is organized in sections as follows. Section 2 presents the methodology for searching good variables in regression models and describes the implementation of classification models for optimal variable mixing towards class separation. This section is further subdivided into specific techniques, including the consideration of single and multiple invariant groups in power form, as well as a broader exploration into non-power forms. Section 3 presents numerical examples and discussions related to our study. We detail the procedure for generating the three datasets employed in this study, based on the three functional forms analyzed here (i.e., Dittus–Boelter correlation, Gnielinski correlation, and Newton’s law of universal gravitation). Here, we also provide a comprehensive evaluation of our approach. Finally, in Section 4, we draw conclusions providing an overview of the main contributions. We point out that, since this work is methodological, it deals only with numerically generated data. However, those methodologies are blessed by generality and, as such, are agnostic to the data origin.

2. Methods

2.1. Problem Statement

Within the context of analyzing complex physical phenomena, discovering effective combinations or groups of primitive variables that characterize a particular physical property of interest is highly desirable. Indeed, this deeper insight not only enhances understanding per se, but also offers practical benefits for optimization purposes, as already pointed out above. Herein, we introduce a methodology to detect invariant groups/sets of variables for regression tasks. In some cases, invariant groups may not be identifiable for regression tasks. Therefore, we also outline a method for identifying similar invariant groups for classification tasks, including the formulation of an analytical classifier.

2.2. Searching for Good Variables by Regression Models

2.2.1. Single Invariant Group in Power Form

Let $f\left(\mathbf{x}\right)=f(x_1,\dots ,x_n)$ denote a function depending on n physical variables $x_1,\dots ,x_n$. If $f\left(\mathbf{x}\right)$ is invariant with respect to a group of p variables in the form $(x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}\cdots x_m^{{\alpha }_p})$, ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_p\in \mathbb{R}$, when this group is kept at a constant value $\overline{c}$, regardless of the value of each component $x_i,x_j,\dots ,x_m$, $f\left(\mathbf{x}\right)$ does not change. Stated differently, the above invariance condition requires:

$${\alpha }_{1}ln\left(x_i\right)+{\alpha }_{2}ln\left(x_j\right)+\cdots +{\alpha }_{p}ln\left(x_m\right)=c$$

(1)

($c=ln\left(\overline{c}\right)$), which upon differentiation yields:

$${\alpha }_1\frac{\mathrm{d}{x}_i}{x_i}+{\alpha }_2\frac{\mathrm{d}{x}_j}{x_j}+\cdots +{\alpha }_p\frac{\mathrm{d}{x}_m}{x_m}=0.$$

(2)

We can thus construct the $(1\times p)$ matrix $\mathbf{B}$ at a generic point $\overline{{\mathbf{x}}_0}=(x_{i,0},x_{j,0},\cdots ,x_{m,0})$, as

$$\mathbf{B}=\left[\frac{{\alpha }_1}{x_{i,0}},\frac{{\alpha }_2}{x_{j,0}},\cdots ,\frac{{\alpha }_p}{x_{m,0}}\right].$$

(3)

Let $\mathbf{K}$ be a matrix whose columns form an ortho-normal basis in the null space (or kernel) of $\mathbf{B}$. Hence, the condition of invariance of $f\left(\mathbf{x}\right)$ with respect to the group $(x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}\cdots x_m^{{\alpha }_p})$ can be recast into an orthogonality condition in the configuration space between the normalized gradient of $f\left(\mathbf{x}\right)$ and each column of K, namely

$${\left(\nabla \tilde{f}\right)}_{\overline{{\mathbf{x}}_0}}·\mathbf{K}=0,$$

(4)

where

$$\nabla \tilde{f}=\frac{\nabla f}{\mathrm{norm}(\nabla f)}.$$

(5)

Coefficients ${\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_p$ can be conveniently normalized, thus yielding the following algebraic system:

$$\left\{\begin{array}{c}{\left(\nabla \tilde{f}\right)}_{\overline{{\mathbf{x}}_0}}·\mathbf{K}=0\hfill \\ {\alpha }_1^2+{\alpha }_2^2+\cdots +{\alpha }_p^2=1\hfill \end{array}\right..$$

(6)

If the non-linear system in Equation (6) is satisfied for the same exponents ($\overline{{\alpha }_1}$, $\overline{{\alpha }_2}$, …, $\overline{{\alpha }_p}$) over all the domains of the features ($x_i$, $x_j$, …, $x_m$), the group $x_i^{\overline{{\alpha }_1}}{x}_j^{\overline{{\alpha }_2}}\cdots x_m^{\overline{{\alpha }_p}}$ represents an intrinsic variable and $f\left(\mathbf{x}\right)$ is invariant with respect to that group.

2.2.2. Multiple Concurrent Invariant Groups in Power Form

Clearly, the above procedure can be extended to a function $f\left(\mathbf{x}\right)$ being invariant with respect to a higher number of feature groups, with each group even sharing some of the primitive physical variables. Without losing generality, and for the sake of simplicity, we limit this generalized description to sets consisting of two concurrent groups.

Let $f\left(\mathbf{x}\right)=f(x_1,\dots ,x_n)=f(x_1,\dots ,x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}\cdots x_m^{{\alpha }_p},x_j^{{\beta }_1}{x}_k^{{\beta }_2}\cdots x_r^{{\beta }_q},\dots ,x_n)$ denote a function where two groups share a generic primitive variable $x_j$. In this case, the above procedure applied to individual groups $x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}\cdots x_m^{{\alpha }_p}$ and $x_j^{{\beta }_1}{x}_k^{{\beta }_2}\cdots x_r^{{\beta }_q}$ independently is not suitable any longer and requires a generalization as discussed below.

To investigate the invariance with respect to both groups, we now construct the $(2\times l)$ matrix $\mathbf{B}$:

$$\mathbf{B}=\left[\begin{array}{cccccc}\frac{{\alpha }_1}{x_{i,0}}& \frac{{\alpha }_2}{x_{j,0}}& 0& \cdots & \frac{{\alpha }_p}{x_{m,0}}& 0\\ 0& \frac{{\beta }_1}{x_{j,0}}& \frac{{\beta }_2}{x_{k,0}}& \cdots & 0& \frac{{\beta }_q}{x_{r,0}}\end{array}\right],$$

(7)

with $max(p,q)\le l\le p+q$, $\mathbf{K}$ being a matrix whose columns represent an ortho-normal basis of the null space of $\mathbf{B}$.

As performed above, the condition of invariance requires that the normalized gradient of $f\left(\mathbf{x}\right)$ is orthogonal to each column of the kernel matrix K (see also Equation (4)).

Adding the normalization condition of the coefficients ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_p$ and ${\beta }_1,{\beta }_2,\dots ,{\beta }_q$, we obtain:

$$\left\{\begin{array}{c}{\left(\nabla \tilde{f}\right)}_{\overline{{\mathbf{x}}_0}}·\mathbf{K}=0\hfill \\ {\alpha }_1^2+{\alpha }_2^2+\cdots +{\alpha }_p^2=1\hfill \\ {\beta }_1^2+{\beta }_2^2+\cdots +{\beta }_q^2=1\hfill \end{array}\right.$$

(8)

with the partial derivatives being evaluated in $\overline{{\mathbf{x}}_0}$. If the non-linear system in Equation (8) is satisfied for the same exponents ($\overline{{\alpha }_1}$, …, $\overline{{\alpha }_p}$, $\overline{{\beta }_1}\dots ,\overline{{\beta }_q}$) in the entire feature domain ($x_i$, $x_j$, …, $x_r$), the two groups $x_i^{\overline{{\alpha }_1}}{x}_j^{\overline{{\alpha }_2}}\cdots x_m^{\overline{{\alpha }_p}}$ and $x_j^{\overline{{\beta }_1}}{x}_k^{\overline{{\beta }_2}}\cdots x_r^{\overline{{\beta }_q}}$ are intrinsic variables as the function $f\left(\mathbf{x}\right)$ is invariant with respect to both groups.

It is worth stressing that more general cases with three or more concurrent invariant groups will imply additional rows for the matrix $\mathbf{B}$. Furthermore, in general, the non-linear system (8) is not necessarily closed (see Section 2.2.4).

2.2.3. Further Generalization to Non Power Forms

The invariant group/set identification procedure, introduced for groups in the power form, can be generalized to other functional relationships. For the sake of illustration, in the following, we restrict to a single invariant group.

Let $f\left(\mathbf{x}\right)=f(x_1,\dots ,x_n)$ denote a function invariant with respect to a group involving p variables according to a generic functional dependence $g(x_i,x_j,\dots ,x_m)$, such that $f\left(\mathbf{x}\right)=f(x_1,\dots ,g(x_i,x_j,\dots ,x_m)\dots ,x_n)$. When such group is a constant $\overline{c}$, even varying its components $x_i,x_j,\dots ,x_m$ separately, $f\left(\mathbf{x}\right)$ does not change. This yields:

$$g(\underset{p}{\underbrace{x_i,x_j,\dots ,x_m}})=\overline{c}$$

(9)

which translates into:

$$\frac{\partial g}{\partial x_i}\mathrm{d}{x}_i+\frac{\partial g}{\partial x_j}\mathrm{d}{x}_j+\cdots +\frac{\partial g}{\partial x_m}\mathrm{d}{x}_m=0.$$

(10)

We can thus construct the $(1\times p)$ matrix $\mathbf{B}$ at a generic point $\overline{{\mathbf{x}}_0}=(x_{i,0},x_{j,0},\dots ,x_{m,0})$ as

$$\mathbf{B}=\left[\frac{\partial g}{\partial x_i},\frac{\partial g}{\partial x_j},\dots ,\frac{\partial g}{\partial x_m}\right].$$

(11)

Let $\mathbf{K}$ be a matrix whose columns represent an orthonormal basis of the null space of $\mathbf{B}$. Applying the invariance condition of Equation (4) leads to a non-linear system analogous to Equation (6). Notably, the procedure illustrated here can be further extended to sets of groups in any functional form, following a reasoning similar to that applied in Section 2.2.2.

2.2.4. Regression Model and Procedure Implementation

In this study, we assume that a database of physical data is available. Each sample in our database consists of n features $(x_1,\dots ,x_n)$ corresponding to a target quantity $f\left(\mathbf{x}\right)$. We further assume that the target quantity possibly depends on some invariant groups expressed for instance in the power form. We thus try to detect such invariance following the above procedure.

As already described, this requires the evaluation of the gradient $\nabla f\left(\mathbf{x}\right)$. To this end, $f\left(\mathbf{x}\right)$ can be conveniently approximated with a Deep Neural Network (DNN), allowing the computation of that gradient by means of automatic differentiation. The choice of a DNN (instead of a simpler approximator) is also beneficial to the method’s versatility, making it suitable for the generalized cases of non power forms. All of the DNNs of this study are trained and validated over the 85% of the corresponding databases—of which the 85% is used for the training and the remaining 15% for the validation—and tested over the remaining 15%. The DNN structure, as reported in Supplementary Note S2, is used for all the examples of this study. Upon network training and validation, automatic differentiation is utilized to calculate the gradient of the function at a specific point ${\mathbf{x}}_0$ within its domain. Here, the variables in the examined group are randomly chosen within their domains, while the values of the remaining variables are held constant at their averages in the original database.

The function’s gradient is computed and applied in Equations (6) and (8), depending on the specific case. Subsequently, the system is solved using a least-squares optimization method that utilizes the Trust Region Reflective [27] algorithm, which is suitable also for under-determined (non closed) systems that may arise during the process of identifying sets of groups.

This strategy is repeated with 20 different values of ${\mathbf{x}}_0$ and iterated multiple times, each time updating the initial guess to the mean value of the exponents $(\overline{{\alpha }_1},\overline{{\alpha }_2},\dots ,\overline{{\alpha }_p})$ averaged over the 20 solutions found in the previous iteration.

An overview of the methodology for identifying invariant groups/sets is depicted in Figure 2.

2.3. Searching for Good Variables by Classification Models

The methodology introduced above, based on regression models, is able to identify groups and/or group sets of physical variables upon which a generic function $f\left(\mathbf{x}\right)$ depends. If successful, this approach efficiently reduces the number of such primitive variables by optimizing their combination.

However, a similar goal can be achieved by means of an entirely different approach based on classification models, and it has been initially proposed by Trezza and Chiavazzo in [23]. The latter methodology, briefly reviewed in the following, enables the optimal mixing of primitive features by performing classification and subsequent optimal class separation of the available data samples.

Let $x_1,\dots ,x_n$ denote n features. Let $({\tilde{x}}_1,\dots ,{\tilde{x}}_n)$ be the corresponding dimensionless quantities

$${\tilde{x}}_i=\frac{x_i-x_{i,\min }}{x_{i,\max }-x_{i,\min }}+1$$

(12)

normalized by construction within the range $[1,2]$ to avoid singularities in the procedure below, where $x_{i,\min }$ and $x_{i,\max }$ represent the minimum and the maximum observed values for the ith feature over the training set, respectively. It is possible to synthetically create a set of $m\ll n$ mixed features $(y_1,\dots ,y_m)$ as

$$y_j=\prod _{i=1}^n{\tilde{x}}_i^{{\alpha }_{ij}}$$

(13)

with $\left\{{\alpha }_{ij}\right\}$ being an $(n\times m)$ matrix estimated by means of a multi-objective optimization algorithm, as described below. Finally, the new variables $y_j$ can be normalized within the interval $[0,1]$ according to

$${\tilde{y}}_j=\frac{y_j-y_{j,\min }}{y_{j,\max }-y_{j,\min }}.$$

(14)

The main idea is that the matrix ${\alpha }_{ij}$ shall be selected following an optimization criterion attempting the largest possible separation between two (or more) different classes partitioning the values of a physical property of interest. Clearly, this can be accomplished by maximizing a certain distance between the classes. However, a multi-objective optimization procedure could also be pursued.

For instance, in a binary classification, the matrix ${\alpha }_{ij}$ in Equation (13) can be chosen to lie on the Pareto front while simultaneously pursuing: (i) maximization of a carefully selected distance between the two classes; (ii) minimization of a norm of the covariance matrix of the first class distribution; and (iii) minimization of a norm of the covariance matrix of the second class distribution.

The main rationale behind the minimization of a norm of the covariance matrix for class distribution (along with a distance between classes) is the aim of possibly obtaining smooth distribution functions that can be analytically bet fitted. Following the latter idea, below, we will pursue multi-objective optimization for all the examined cases and provide some optimal solutions from Pareto fronts.

In this study, we adopt genetic algorithms for optimization whereas the Bhattacharyya distance between the histograms of the two equally binned classes [28,29] is evaluated to be maximized during the multi-objective optimization. However, as shown in ref. [23], other options for class distance may be considered, such as the Wasserstein distance [30] or the averaged number within a fixed radius of nearest neighbors of one class to each samples of the other class. Furthermore, herein we utilize variable power-form mixing outlined in Equation (13). Nonetheless, alternative grouping options (e.g., linear mixing as employed in ref. [23]) are also possible with the overall methodology remaining unchanged. Also, the procedure can be further generalized to a number of classes $>2$.

In this case, the genetic algorithm aims at simultaneously maximizing the pairwise distances between the classes [23]. For the remaining two objectives, in one-dimensional cases, a numerical estimate of the standard deviation of the binned data in the two classes is computed. In two-dimensional (or higher) cases, the determinant of the covariance matrix can be utilized. From the practical standpoint, considering a dataset of physical data samples characterized by features $x_1,\dots ,x_n$ and their corresponding response $f\left(\mathbf{x}\right)$, such a dataset is partitioned into two classes based on a carefully chosen threshold for $f\left(\mathbf{x}\right)$. After the aforementioned pre-processing steps, a genetic algorithm optimization is employed to identify the Pareto front. This optimization concurrently seeks to minimize the variances of the two classes (in one dimension) or the determinants of the covariances of the two classes (in two or more dimensions), while also maximizing the Bhattacharyya distance between the classes. A summary of this procedure is illustrated in Figure 3.

3. Numerical Examples and Discussion

3.1. Datasets Creation

We first create three datasets from physical models to be used to train and test Machine Learning (ML) tools, aiming to predict physical properties of interest. More specifically, the first two datasets are generated using popular thermo-fluid dynamic correlations (i.e., Dittus–Boelter and Gnielinski correlations) with the target quantity being the Nusselt number. Those datasets are created starting from the physical properties of 16 real liquids at ambient temperature and pressure [31], and evaluating the kinematic viscosity $\nu$ and the thermal diffusivity $\kappa$ of each liquid from tabulated data (see Supplementary Note S1). Each fluid accounts for 500 value combinations of the flow speed u, the hydraulic diameter D, and the friction factor f (the latter is only requested for the Gnielinski correlation). The values of these three variables are randomly chosen in the ranges $[0.1,1]$, $[0.01,0.1]$, and $[0.02,0.09]$, respectively. This leads to a total of 8000 samples for each dataset. The Nusselt number is thus calculated according to the corresponding equations. Finally, emulating what is typically experienced in experimental measurements, noise is added on top of the correct target values, sampling 8000 points ${\eta }_i$ from a Gaussian distribution, with mean $\mu =0$ and standard deviation $\sigma =0.1$; the target value of each entry ${\mathrm{Nu}}_i$ is then multiplied by $(1+{\eta }_i)$. The third dataset is created on the basis of Newton’s law of universal gravitation. It is constructed in a similar fashion, by generating random values within the range $[{10}^{16},{10}^{18}]$ for the masses $m_1$ and $m_2$, and within the ranges $[-{10}^{12},-{10}^{10}]$ and $[{10}^{10},{10}^{12}]$ for the spatial coordinates $x_1,y_1,z_1,x_2,y_2,z_2$. Here, the target value is the gravitational force $F_{\mathrm{g}}$, which is computed with some noise added following the same procedure as above.

3.2. Dittus–Boelter Equation

The Dittus–Boelter correlation for a fluid undergoing heating is expressed by the following equation:

$$\mathrm{Nu}=0.023{\mathrm{Re}}_D^{4/5}{\mathrm{Pr}}^{0.4}$$

(15)

where ${\mathrm{Re}}_D=\frac{ud}{\nu }$ represents the Reynolds number and $\mathrm{Pr}=\frac{\nu }{\kappa }$ denotes the Prandtl number. The equation can be rewritten by substituting the dimensionless quantities $Re$ and $Pr$, thus obtaining

$$\mathrm{Nu}=0.023{\left(\frac{ud}{\nu }\right)}^{4/5}{\left(\frac{\nu }{\kappa }\right)}^{0.4}=0.023\frac{u^{0.8}{d}^{0.8}}{{\nu }^{0.4}{\kappa }^{0.4}}.$$

(16)

It is easy to verify that the objective value is invariant with respect to all the possible combinations of input features, $u,d,\nu ,\kappa$, either couples or triplets.

3.2.1. Use of Regression Models

We attempt to recover the symmetries of the Nusselt number with respect to binary and ternary groups in the form $x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}$ and $x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}{x}_k^{{\alpha }_3}$, only relying on noised data and by adopting the methodology illustrated in Section 2.2. As outlined above, this requires the evaluation of the gradient of the noised Nusselt number with respect to the input features, namely $\nabla \overline{\mathrm{Nu}}(u,d,\nu ,\kappa )$, where for each sample i, ${\overline{\mathrm{Nu}}}_i={\mathrm{Nu}}_i(1+{\eta }_i)$ (see Section 2 for details). This can be conveniently computed by automatic differentiation over a DNN model approximating $\overline{\mathrm{Nu}}(u,d,\nu ,\kappa )$. As input features of the DNN, the four variables $u,d,\nu ,\kappa$ are used. The dataset is thus divided into three parts: (i) a training set, (ii) a validation set used to detect potential overfitting, and (iii) a testing set for the comprehensive evaluation of model performance. Figure 4a showcases the model predictions over the testing set, while in Figure 4b, the corresponding loss across epochs is depicted.

Notably, the model is highly predictive, with coefficient of determination $R^2=0.963$ over the testing set, with no evidence of overfitting observed. The model is thus fed to the optimization algorithm. For each of the ten expected invariant groups (six binary and four ternary), the algorithm estimates the normalized exponents ${\alpha }_1,{\alpha }_2,{\alpha }_3$ 20 times.

Table 1. Normalized exponents ${\alpha }_1,{\alpha }_2,{\alpha }_3$ for the Dittus–Boelter correlation together with their average estimates over 20 evaluations ${\mu }_{{\alpha }_1},{\mu }_{{\alpha }_2},{\mu }_{{\alpha }_3}$ and the corresponding standard deviations ${\sigma }_{{\alpha }_1},{\sigma }_{{\alpha }_2},{\sigma }_{{\alpha }_3}$. Found groups refer to low standard deviation, reliable groups refer to average far from 1 and 0.

Group	${\mathit{\alpha }}_1$	${\mathit{\alpha }}_2$	${\mathit{\alpha }}_3$	${\mathit{\mu }}_{{\mathit{\alpha }}_1}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_1}$	${\mathit{\mu }}_{{\mathit{\alpha }}_2}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_2}$	${\mathit{\mu }}_{{\mathit{\alpha }}_3}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_3}$	Found	Reliable
$(u,d)$	0.71	0.71	-	−0.71	0.05	−0.70	0.05	-	-	yes	yes
$(u,\nu )$	0.89	−0.45	-	0.91	0.04	−0.42	0.07	-	-	yes	yes
$(u,\kappa )$	0.89	−0.45	-	0.89	0.01	−0.45	0.03	-	-	yes	yes
$(d,\nu )$	0.89	−0.45	-	0.90	0.04	−0.42	0.11	-	-	yes	yes
$(d,\kappa )$	0.89	−0.45	-	0.87	0.02	−0.50	0.04	-	-	yes	yes
$(\nu ,\kappa )$	−0.71	−0.71	-	−0.67	0.10	−0.73	0.10	-	-	yes	yes
$(u,d,\nu )$	0.67	0.67	−0.33	0.68	0.03	0.65	0.04	−0.33	0.07	yes	yes
$(u,d,\kappa )$	0.67	0.67	−0.33	0.66	0.03	0.64	0.03	−0.37	0.03	yes	yes
$(u,\nu ,\kappa )$	0.82	−0.41	−0.41	0.87	0.04	−0.37	0.06	−0.34	0.10	yes	yes
$(d,\nu ,\kappa )$	0.82	−0.41	−0.41	0.83	0.06	−0.35	0.11	−0.41	0.05	yes	yes

shows such true normalized exponents ${\alpha }_1,{\alpha }_2,{\alpha }_3$, together with the means ${\mu }_{{\alpha }_1},{\mu }_{{\alpha }_2},{\mu }_{{\alpha }_3}$ and the standard deviations ${\sigma }_{{\alpha }_1},{\sigma }_{{\alpha }_2},{\sigma }_{{\alpha }_3}$ over those 20 evaluations. Our findings indicate that the method correctly identifies invariant groups in both couple and triplet forms. Furthermore, it demonstrates the ability to estimate normalized exponents ${\alpha }_1,{\alpha }_2,{\alpha }_3$ for individual primitive variables within each group, with relative percent errors not exceeding 11.0% for couples and 16.6% for triplets. Clearly, we notice that normalized exponents are obtained up to the sign: the exponents determined for the pair $(u,d)$ are negative instead of the expected positive values according to the Dittus–Boelter equation.

In Table 1, we label as found, all the groups with standard deviations ${\sigma }_{{\alpha }_1},{\sigma }_{{\alpha }_2},{\sigma }_{{\alpha }_3}\le 0.2$, since low values of $\sigma$ imply that the results for the 20 evaluations are consistent. We also label as reliable all the groups where none of the evaluated exponents approach 0 or 1, since such cases would correspond to trivial solutions. Remarkably, all six binary groups and all four ternary groups of input features are found to comply with the condition of invariance, and the results for all of them are reliable.

3.2.2. Use of Classification Models

In a second attempt to reduce the number of input variables, we investigate the possible existence of symmetries for classification, aiming at the construction of m mixed features in the form $y_j={\prod }_{i=1}^n{\tilde{x}}_i^{{\alpha }_{ij}}$, where $({\tilde{x}}_1,\cdots ,{\tilde{x}}_n)$ are the primitive variables properly normalized within the interval [1, 2] and $\left\{{\alpha }_{ij}\right\}$ denotes an $n\times m$ matrix optimally estimated, as detailed in Section 2.3. Such features are finally properly normalized as ${\tilde{y}}_j$ in the interval [0, 1]. In the case of the Dittus-Bolter, we set class 1 for samples with $\overline{\mathrm{Nu}}<395$ and class 2 for samples with $\overline{\mathrm{Nu}}\ge 395$. We thus create a single mixed feature ($m=1$) according to the above procedure. Figure 5a reports the Probability Density Function (PDF) binning of the training set data over the two classes (a higher value of the PDF means a higher number of items in the corresponding bin), against the normalized flow velocity $u_{\mathrm{norm}}$, while Figure 5b shows the same PDFs (i.e., class distributions) against the normalized mixed feature ${\tilde{y}}_1$. It is noteworthy to observe that when represented against the primitive feature, the two classes exhibit considerable overlap, whereas the same two classes appear well separated when plotted against the mixed feature. As a result, it is particularly convenient to attempt an analytical bet-fitting of the two distributions depicted in Figure 5b approximated by a Generalized Extreme Value (GEV) distribution, with density

$$g\left({\tilde{y}}_1\right)=\frac{1}{\sigma }{\left(1+\omega \frac{{\tilde{y}}_1-\mu }{\sigma }\right)}^{-\frac{\omega +1}{\omega }}exp\left(-{\left(1+\omega \frac{{\tilde{y}}_1-\mu }{\sigma }\right)}^{-1/\omega }\right).$$

(17)

The fitting is performed by means of the SciPy Python package [32]. The specific computed GEV distribution for samples with $\overline{\mathrm{Nu}}<395$ has factors $\mu =0.338$, $\sigma =0.137$, and $\omega =0.342$, whereas the GEV distribution for samples with $\overline{\mathrm{Nu}}\ge 395$ has factors $\mu =0.675$, $\sigma =0.077$, and $\omega =0.074$. Figure 5c shows the PDFs over the binned data of the testing set reported against the same mixed feature ${\tilde{y}}_1$, along with the GEV fittings computed on the training set. Notably, the classes are still well separated, with a good agreement between the GEV distributions and the testing set densities.

Furthermore, we make an attempt at the creation of two mixed normalized features ${\tilde{y}}_1,{\tilde{y}}_2$ ($m=2$) with the same classes. Specifically, Figure 6a shows the PDF two dimensional binning of the training set data over the two classes, against the normalized flow velocity $u_{\mathrm{norm}}$ and the hydraulic diameter $d_{\mathrm{norm}}$. Figure 6b shows the same PDFs against the normalized mixed features ${\tilde{y}}_1,{\tilde{y}}_2$ constructed according to Equation (13) by power combination of the four primitive variables and choosing the point of the Pareto front with the least overlapping of the two distributions. Similarly to the one dimensional case, the classes appear well separated when plotted against the mixed features, whereas there is a wide overlap when plotted against the primitive variables. Figure 6c shows the PDFs over the binned data of the testing set reported against the same mixed features. The two classes still appear well separated.

Finally, we aim at the construction of $m=1$ mixed feature for separating the data samples into three classes: class 0 for $\overline{\mathrm{Nu}}<197.5$, class 1 for $197.5\le \overline{\mathrm{Nu}}<395$, class 2 for $\overline{\mathrm{Nu}}\ge 395$. The multi-objective optimization is performed aiming at concurrently maximizing the pairwise distances between the classes. Figure 7 shows the separation in classes reported against the normalized flow velocity $u_{\mathrm{norm}}$ and the new mixed feature ${\tilde{y}}_1$: in line with previous cases, the substantial overlap observed when representing classes against the primitive feature is significantly reduced when using the optimized mixed feature. The bet-fitting GEV distributions for the three classes are computed according to Equation (17) over the training set samples. Specifically the GEV associated with Class 0 has factors $\mu =0.284$, $\sigma =0.103$, $\omega =0.494$, the GEV for Class 1 has factors $\mu =0.475$, $\sigma =0.067$, $\omega =0.193$, and the GEV for Class 2 has factors $\mu =0.684$, $\sigma =0.080$, $\omega =0.103$. When plotting the PDFs over the binned data of the testing set against the mixed feature, the classes appear well separated, and there is good agreement between the GEV distributions and the testing set densities.

3.3. Gnielinski Correlation

The Gnielinski correlation for turbulent flow in tubes is expressed by the equation

$$\mathrm{Nu}=\frac{(f/8)({\mathrm{Re}}_D-1000)\mathrm{Pr}}{1+12.7{(f/8)}^{1/2}({\mathrm{Pr}}^{2/3}-1)}$$

(18)

where ${\mathrm{Re}}_D=\frac{ud}{\nu }$ represents the Reynolds number, $\mathrm{Pr}=\frac{\nu }{\kappa }$ is the Prandtl number and f denotes the friction factor. The equation can be reformulated by substituting the dimensionless quantities $Re$ and $Pr$, resulting in:

$$\mathrm{Nu}=\frac{(f/8)\left(\frac{ud}{\nu }-1000\right)\left(\frac{\nu }{\kappa }\right)}{1+12.7{(f/8)}^{1/2}\left({\left(\frac{\nu }{\kappa }\right)}^{2/3}-1\right)}.$$

(19)

The Nusselt number is invariant only with respect to the combination of flow velocity and hydraulic diameter $(u,d)$ with real exponents $(1,1)$, normalized exponents $(0.707,0.707)$.

3.3.1. Use of Regression Models

Here, we first attempt the detection of possible symmetries of the noised Nusselt number $\overline{\mathrm{Nu}}$ adopting the methodology proposed in Section 2.2.2. We obtain access to the gradient $\nabla \overline{\mathrm{Nu}}(u,d,\nu ,\kappa ,f)$ by means of automatic differentiation over a DNN approximating $\overline{\mathrm{Nu}}(u,d,\nu ,\kappa ,f)$. Figure 8a shows the model predictions over the testing set, while in Figure 8b the corresponding loss across epochs is depicted. The model is highly predictive, with coefficient of determination $R^2=0.978$ over the testing set, and no evidence of overfitting is observed. The trained model is thus fed to the optimization algorithm, looking for binary groups in the form $x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}$. The average ${\mu }_{{\alpha }_1},{\mu }_{{\alpha }_2}$ estimates of the exponents ${\alpha }_1,{\alpha }_2$, together with their standard deviations ${\sigma }_{{\alpha }_1},{\sigma }_{{\alpha }_2}$, are reported in

Table 2. Normalized exponents ${\alpha }_1,{\alpha }_2,{\alpha }_3,{\beta }_1,{\beta }_2$ for the Gnielinski correlation, together with their average estimates over 20 evaluations ${\mu }_{{\alpha }_1}$, ${\mu }_{{\alpha }_2}$, ${\mu }_{{\alpha }_3}$, ${\mu }_{{\beta }_1}$, ${\mu }_{{\beta }_2}$ and the corresponding standard deviations ${\sigma }_{{\alpha }_1}$, ${\sigma }_{{\alpha }_2}$, ${\sigma }_{{\alpha }_3}$, ${\sigma }_{{\beta }_1}$, ${\sigma }_{{\beta }_2}$. Found groups/sets refer to low standard deviation, reliable groups refer to average far from 1 and 0.

Group/Set	${\mathit{\alpha }}_1$	${\mathit{\alpha }}_2$	${\mathit{\alpha }}_3$	${\mathit{\beta }}_1$	${\mathit{\beta }}_2$	${\mathit{\mu }}_{{\mathit{\alpha }}_1}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_1}$	${\mathit{\mu }}_{{\mathit{\alpha }}_2}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_2}$	${\mathit{\mu }}_{{\mathit{\alpha }}_3}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_3}$	${\mathit{\mu }}_{{\mathit{\beta }}_1}$	${\mathit{\sigma }}_{{\mathit{\beta }}_1}$	${\mathit{\mu }}_{{\mathit{\beta }}_2}$	${\mathit{\sigma }}_{{\mathit{\beta }}_2}$	Found	Reliable
$(u,d)$	0.707	0.707	-	-	-	−0.673	0.067	−0.734	0.056	-	-	-	-	-	-	yes	yes
$(u,\nu )$	-	-	-	-	-	0.801	0.022	−0.597	0.030	-	-	-	-	-	-	yes	yes
$(u,\kappa )$	-	-	-	-	-	0.949	0.010	−0.313	0.032	-	-	-	-	-	-	yes	no
$(u,f)$	-	-	-	-	-	−0.867	0.065	−0.468	0.161	-	-	-	-	-	-	yes	yes
$(d,\nu )$	-	-	-	-	-	0.807	0.042	−0.587	0.055	-	-	-	-	-	-	yes	yes
$(d,\kappa )$	-	-	-	-	-	0.948	0.015	−0.314	0.049	-	-	-	-	-	-	yes	no
$(d,f)$	-	-	-	-	-	0.868	0.068	0.479	0.111	-	-	-	-	-	-	yes	yes
$(\nu ,\kappa )$	-	-	-	-	-	−0.918	0.035	−0.389	0.076	-	-	-	-	-	-	yes	no
$(\nu ,f)$	-	-	-	-	-	0.756	0.052	−0.650	0.056	-	-	-	-	-	-	yes	yes
$(\kappa ,f)$	-	-	-	-	-	0.467	0.033	−0.883	0.017	-	-	-	-	-	-	yes	yes
$\left[\right(u,\nu ),(\nu ,\kappa \left)\right]$	0.707	−0.707	-	0.707	−0.707	0.730	0.047	−0.680	0.054	-	-	−0.656	0.011	0.755	0.010	yes	yes
$\left[\right(d,\nu ),(\nu ,\kappa \left)\right]$	0.707	−0.707	-	0.707	−0.707	0.695	0.028	−0.718	0.027	-	-	−0.687	0.008	0.727	0.007	yes	yes
$\left[\right(u,d),(u,\nu \left)\right]$	-	-	-	-	-	0.313	0.000	−0.950	0.000	-	-	−0.950	0.001	0.313	0.002	yes	no
$\left[\right(u,d),(d,f\left)\right]$	-	-	-	-	-	−0.576	0.056	−0.814	0.043	-	-	−0.593	0.017	0.805	0.013	yes	yes
$\left[\right(u,\nu ),(\nu ,f\left)\right]$	-	-	-	-	-	0.872	0.045	−0.482	0.074	-	-	−0.224	0.058	0.973	0.011	yes	no
$\left[\right(u,f),(f,d\left)\right]$	-	-	-	-	-	−0.665	0.052	−0.744	0.044	-	-	−0.503	0.036	0.863	0.020	yes	yes
$\left[\right(d,f),(f,\nu \left)\right]$	-	-	-	-	-	0.860	0.016	−0.510	0.026	-	-	−0.868	0.036	0.491	0.063	yes	yes
$\left[\right(u,d,\nu ),(\nu ,\kappa \left)\right]$	0.577	0.577	−0.577	0.707	−0.707	0.602	0.043	0.552	0.065	−0.570	0.053	0.698	0.011	−0.716	0.011	yes	yes
$\left[\right(u,d,f),(f,\nu \left)\right]$	-	-	-	-	-	0.580	0.002	0.660	0.014	−0.476	0.023	0.817	0.004	−0.576	0.006	yes	yes
$\left[\right(u,\kappa ,f),(\kappa ,\nu \left)\right]$	-	-	-	-	-	0.492	0.012	0.703	0.017	−0.512	0.026	0.625	0.012	−0.780	0.009	yes	yes

.

Specifically, the procedure correctly identifies the expected group $(u,d)$, whereas for most other groups, the results correspond to either the trivial solution (i.e., the absolute value of one of the exponents is close to 1 and the other is close to 0), or at least one of the evaluated values presents too high variance (i.e., ${\sigma }_{{\alpha }_i}>0.2$). Remarkably, for the remaining cases (namely $(u,\nu )$, $(u,f)$, $(d,\nu )$, $(d,f)$, $(\nu ,\kappa )$, $(\nu ,f)$), the procedure identifies groups complying with the invariance condition, albeit not explicitly expected in the Gnielinski correlation. We thus concluded that the suggested procedure has detected a local invariance and this can be numerically verified as follows (here we limit to groups of two variables only). First, the features not appearing in the group are kept constant. Second, a random sample i in the dataset is considered for the evaluation of the quantity $\tilde{c}=x_{1,i}^{{\alpha }_1}{x}_{2,i}^{{\alpha }_2}$. Third, a vector $\overline{{\mathbf{x}}_1}$ of evenly spaced points in the domain of the first feature is created. As such, the array corresponding to the second feature is obtained as $\overline{{\mathbf{x}}_2}={\left(\tilde{c}{\overline{{\mathbf{x}}_1}}^{-{\alpha }_1}\right)}^{1/{\alpha }_2}$. A new dataset is thus constructed, with the variables out of the group being constant and with the variables in the group being replaced by $\overline{{\mathbf{x}}_1}$, $\overline{{\mathbf{x}}_2}$. For all those samples, the response value $f\left(\mathbf{x}\right)$ is computed. The local invariance is demonstrated when $f\left(\mathbf{x}\right)$ is approximately constant over all the new constructed dataset. Further details can be found in Supplementary Note S3.

Furthermore, armed with the same DNN regression model, we try to detect invariance of the Nusselt number with respect to sets groups in the form $x_i^{{\alpha }_1}{x}_j^{{\alpha }_2}{x}_k^{{\alpha }_3}$ and $x_k^{{\beta }_1}{x}_l^{{\beta }_2}$ employing the procedure presented in Section 2.2.2. Table 2 also reports the results for selected sets of two feature couples, including sets appearing explicitly in the Gnielinski correlation—i.e., $\left[\right(u,\nu ),(\nu ,\kappa \left)\right]$ and $\left[\right(d,\nu ),(\nu ,\kappa \left)\right]$—and for selected sets of features comprising a triplet and a couple—e.g., $\left[\right(u,d,\nu ),(\nu ,\kappa \left)\right]$, which is extremely relevant as it comprises the Reynolds and the Prandtl numbers. For all of the mentioned sets, the procedure correctly identifies the exponents that make the groups compliant with the condition of invariance. Indeed, the normalized exponents obtained with the procedure for the sets $\left[\right(u,\nu ),(\nu ,\kappa \left)\right]$ and $\left[\right(d,\nu ),(\nu ,\kappa \left)\right]$ are $\left[\right(0.730,-0.680),(-0.656,0.755\left)\right]$ and $\left[\right(0.695,-0.718),(-0.687,0.727\left)\right]$, respectively, whereas the analytical solution is $\left[\right(0.707,-0.707),(0.707,-0.707\left)\right]$ for both the cases. The normalized exponents obtained for the set $\left[\right(u,d,\nu ),(\nu ,\kappa \left)\right]$ turn out to be $\left[\right(0.602,0.552,-0.570),(0.698,-0.716\left)\right]$, whereas the analytical solution corresponds to $\left[\right(0.577,0.577,-0.577),(0.707,-0.707\left)\right]$.

Table 2 also reports some identified sets that locally comply with the condition of invariance (see Supplementary Note S3 for more details), together with some randomly selected sets for which the procedure succeeds at finding the invariance property, but for which the results are close to the trivial solution, meaning that at least one of the evaluated exponents approaches either 0 or $\pm 1$, thus is not reliable. Following the same approach above, Table 2 labels groups with low standard deviations as found and all the groups in which none of the evaluated exponents approaches 0 or $\pm 1$ as reliable. Remarkably, the analytically evident couple $(u,d)$, sets of two couples $\left[\right(u,\nu ),(\nu ,\kappa \left)\right]$ and $\left[\right(d,\nu ),(\nu ,\kappa \left)\right]$, and set comprising a triplet and a couple $\left[\right(u,d,\nu ),(\nu ,\kappa \left)\right]$, are identified. Furthermore, eight additional couples are found that locally comply with the condition of invariance, with six of them being reliable, and only one couple is not found. In the case of the selected sets of two feature couples, five more sets of two couples locally comply with the condition of invariance, with only two of them not being reliable. Finally, the two additional sets comprising a triplet and a couple are reliably found to represent a local invariance.

3.3.2. Use of Classification Models

In a second attempt at reducing the number of input variables, we aim at the construction of mixed optimized features, such as power combinations of the normalized primitive variables for classification, according to Equation (13). In the case of the Gnielinski correlation, we set class 1 for samples with $\overline{\mathrm{Nu}}<500$ and class 2 for samples with $\overline{\mathrm{Nu}}\ge 500$. Once the optimization routine finds the Pareto front, the mixed features are created using the point of the Pareto front with the least overlapping of the two classes according to the Battacharyya distance. Figure 9a reports the PDF binning of the training set data over the two classes against the normalized flow velocity $u_{\mathrm{norm}}$, while Figure 9b shows the same PDFs against the normalized mixed feature ${\tilde{y}}_1$; still, when represented against the primitive variable, the two classes exhibit significant overlap, whereas they appear distinctly separated when plotted against the mixed feature. Two bet-fitting GEV distributions are computed for the two classes according to Equation (17) for samples in class 1 with factors $\mu =0.276$, $\sigma =0.112$, $\omega =0.223$; for samples in class 2 with factors $\mu =0.510$, $\sigma =0.091$, $\omega =0.000$. Figure 9c shows the PDFs over the binned data of the testing set reported against the same mixed feature ${\tilde{y}}_1$, along with the GEV fittings computed on the training set. Notably, the classes are still well separated, with a good agreement between the GEV distributions and the testing set densities. Furthermore, we attempt to create two mixed normalized features ${\tilde{y}}_1,{\tilde{y}}_2$ with the same classes. Figure 10a shows the PDF two dimensional binning of the training set data over the two classes against the normalized flow velocity $u_{\mathrm{norm}}$ and friction factor $f_{\mathrm{norm}}$. Figure 10b shows the same PDFs against the mixed features ${\tilde{y}}_1,{\tilde{y}}_2$ constructed according to Equation (13) by power combination of the five relevant features and choosing the point of the Pareto front with the least overlapping of the classes. Similarly to the one-dimensional case, the classes appear again well separated when plotted against the mixed features, whereas there is a wide overlap when plotted against the primitive variables. Figure 10c shows the PDFs over the binned data of the testing set, reported against the same mixed features ${\tilde{y}}_1,{\tilde{y}}_2$; the two classes still appear well separated. Finally, we aim at the construction of $m=1$ mixed feature for separating the data samples into three classes: class 0 for $\overline{\mathrm{Nu}}<400$, class 1 for $400\le \overline{\mathrm{Nu}}<900$, class 2 for $\overline{\mathrm{Nu}}\ge 900$. The multi-objective optimization is performed aiming at concurrently maximizing the pairwise distances between the classes. Figure 11 shows the separation in classes reported against the normalized flow velocity $u_{\mathrm{norm}}$ and the new mixed feature ${\tilde{y}}_1$: in line with previous cases, the considerable overlap observed when representing classes against the primitive feature is significantly reduced when using the optimized mixed feature. The bet-fitting GEV distributions for the three classes are computed according to Equation (17) over the training set samples; specifically, the GEV associated with Class 0 has factors $\mu =0.245$, $\sigma =0.083$, $\omega =0.346$, the GEV for Class 1 has factors $\mu =0.430$, $\sigma =0.053$, $\omega =0.143$, and the GEV for Class 2 has factors $\mu =0.599$, $\sigma =0.074$, $\omega =0.017$. When plotting the PDFs over the binned data of the testing set against the mixed feature, the classes appear well separated, and there is good agreement between the GEV distributions and the testing set densities.

3.4. Newton’s Law of Universal Gravitation

The module of the interacting force $F_{\mathrm{g}}$ for two objects of masses $m_1,m_2$ is expressed via Newton’s law of universal gravitation

$$F_{\mathrm{g}}=\frac{G{m}_1{m}_2}{{(x_1-x_2)}^2+{(y_1-y_2)}^2+{(z_1-z_2)}^2},$$

(20)

where G is the gravitational constant, $(x_1,y_1,z_1)$ and $(x_2,y_2,z_2)$ are the coordinates of the centers of their masses.

In this case, our aim is to identify the invariance of the noised $\overline{F_{\mathrm{g}}}$ with respect to groups $(x_1-x_2)$, $(y_1-y_2)$, $(z_1-z_2)$. To this end, we employ the procedure presented in Section 2.2.3 for identification of general group form, focusing specifically on groups with functional dependence ${\alpha }_1{x}_i+{\alpha }_2{x}_j$. As usual, we obtain access to the gradient $\nabla \overline{F_{\mathrm{g}}}(x_1,x_2,y_1,y_2,z_1,z_2,m_1,m_2)$ by means of automatic differentiation over a DNN approximating $\overline{F_{\mathrm{g}}}(x_1,x_2,y_1,y_2,z_1,z_2,m_1,m_2)$. Figure 12a shows the model predictions over the testing set, while in Figure 12b, the corresponding loss across epochs is depicted; specifically, the model is highly predictive, with coefficient of determination $R^2=0.962$ and no evidence of overfitting is observed. The trained model is thus fed to the optimization algorithm. The average ${\mu }_{{\alpha }_1},{\mu }_{{\alpha }_2}$ estimates of the coefficients ${\alpha }_1,{\alpha }_2$, together with their standard deviations ${\sigma }_{{\alpha }_1},{\sigma }_{{\alpha }_2}$, are reported in

Table 3. Normalized exponents ${\alpha }_1,{\alpha }_2$ for the Newton’s law of universal gravitation, together with their average estimates over 20 evaluations ${\mu }_{{\alpha }_1}$, ${\mu }_{{\alpha }_2}$, and the corresponding standard deviations ${\sigma }_{{\alpha }_1}$, ${\sigma }_{{\alpha }_2}$. Found groups/sets refer to low standard deviation, reliable groups refer to average far from 1 and 0.

Group	${\mathit{\alpha }}_1$	${\mathit{\alpha }}_2$	${\mathit{\mu }}_{{\mathit{\alpha }}_1}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_1}$	${\mathit{\mu }}_{{\mathit{\alpha }}_2}$	${\mathit{\sigma }}_{{\mathit{\alpha }}_2}$	Found	Reliable
$(x_1,x_2)$	0.707	−0.707	−0.679	0.111	0.721	0.083	yes	yes
$(y_1,y_2)$	0.707	−0.707	−0.711	0.023	0.702	0.023	yes	yes
$(z_1,z_2)$	0.707	−0.707	−0.683	0.030	0.722	0.044	yes	yes
$(m_1,m_2)$	-	-	0.105	0.556	0.759	0.322	no	-
$(m_1,x_1)$	-	-	0.000	0.000	1.000	0.000	no	-
$(x_1,y_1)$	-	-	−0.604	0.369	0.335	0.622	no	-

. Specifically, the procedure correctly identifies the groups $(x_1,x_2)$, $(y_1,y_2)$, $(z_1,z_2)$ with estimates ${\mu }_{{\alpha }_1},{\mu }_{{\alpha }_2}$ of $(-0.679,0.721)$, $(-0.711,0.702)$, $(-0.683,0.722)$, respectively, compared to the true normalized coefficients $(-0.707,0.707)$ for all cases. Table 3 flags groups with low standard deviations as found and all the groups in which none of the evaluated exponents approaches 0 or 1 as reliable. Specifically, other variables couples show high variance over the corresponding average estimates, meaning that no further invariant group is identified.

4. Conclusions and Final Remarks

In this study, we have implemented two innovative methodologies for searching optimal variables to describe physical data, making use of both regression and classification models applied to the data. Specifically, leveraged on well-suited datasets for machine learning, with the goal of predicting a property of interest.

In particular, the methodology introduced for regression tasks has the ambition to find groups of variables that are valid over all the parameter space spanned by the available data. However, due to various factors such as noise in the data, this method might not converge, even if the group exists. Conversely, the methodology introduced for classification tasks gives up on this ambition in favor of the greater robustness of an optimization framework, aiming to only find possible separations in the parameter space. This can be useful, for instance, to find optimal areas in the parameters space with the highest values of a properly defined objective function of interest, even if the group does not govern globally such objective function. Due to their different objectives, the two methods generally do not produce the same groups.

More precisely, the procedure based on the regression model introduced here enables the identification of invariant groups and/or sets of variable groups with respect to which the property of interest is invariant. We have demonstrated its effectiveness on noised data generated by three well known functional relationships: the Dittus–Boelter correlation, the Gnielinski correlation, and Newton’s law of universal gravitation. It is noteworthy that we did not apply any particular noise reduction technique to the three datasets; instead, our methods directly handle the noisy data as-is, demonstrating their robustness. Also, we acknowledge that direct measurement of the Nusselt number would not be feasible in real world scenarios, and that the calculation method of the examined properties of interest (i.e., Nusselt number and gravitational force) is already established. However, our selection of such examples is only aimed at showcasing the effectiveness of our methodologies in sufficiently simple cases. For the former two cases, the procedure has accurately detected groups/sets in power form, while for the latter case, a generalized algorithm successfully identified groups with a linear form. It is worth stressing that, in all the examples, our procedure did not end up with any false negative, i.e., whenever a group exists, it is correctly identified. Interestingly, the methodology is potentially applicable to any functional form, as illustrated in Section 2.2.3. Additionally, the effectiveness of identifying a group in a situation where that group is already known—like in this study—depends on a tolerance threshold set between the identified group and the actual one. In real world scenarios, the identified groups serve as candidates of mixed variables, whose validity has to be verified a posteriori.

The procedure based on classification of the physical property values, allows for the determination of an appropriate set of exponents to combine all the primitive variables in power form, thus constructing mixed features optimized for the classification task. Specifically, we have shown its effectiveness on the Dittus–Boelter and on the Gnielinski correlations. Indeed, the methodology effectively enables the separation of classes even with just one mixed feature, whereas a single primitive variable fails to achieve class separation. Furthermore, we also provide examples with one and two mixed variables, together with separation in two or three classes.

It is worth stressing that the methodologies presented in this study are blessed by generality and, as such, are not restricted to the selected case studies, and are agnostic in terms of data origin, being it numerical or experimental. Therefore, potential applications can be envisioned in various other fields in the future. In particular, we notice that the identification of effective good variables is not only interesting per se to possibly gain a deeper insight on a physical system, but can also be practically advantageous when designing experiments. Indeed, the methodology to detect groups/sets in regression facilitates efficient group/set-level adjustments rather than individually fine-tuning variables within the groups/sets themselves.

Moreover, the classification procedure can enable the reduction of the numerous original primitive variables to a minimal set of optimized variables concerning a specific physical property of interest. Notably, combinations of variables yielding identical mixed feature values exhibit similar performance in terms of the property being classified. As a result, it is possible to find alternative combinations of primitive variables without compromising the overall performance of a given system under study. From a more practical standpoint, the methodologies described here can help generalizing optimal system conditions, thus helping decreasing the most resource-intensive components while properly re-balancing the others. As an example, these general methodologies may thus hold the potential to save resources, such as costly reagents (e.g., Bonke et al. [24] for solar fuel production) or expensive materials as in perovskite solar cells optimization [33]. An additional advantage associated with the correct identification of a reduced set of ruling variables is their possible use for driving sequential learning or Bayesian optimization processes [34,35].

Finally, we believe that an interesting development of the presented methodologies to be pursued in the future, shall be in the direction of a possible handling of systems ruled not only by numerical parameters, but also categorical ones.