*Article* **Amalgamation of Geometry Structure, Meteorological and Thermophysical Parameters for Intelligent Prediction of Temperature Fields in 3D Scenes**

**Yuan Cao 1,2, Ligang Li 1,\*, Wei Ni 1, Bo Liu 1,2, Wenbo Zhou 1,2 and Qi Xiao 1,2**


**Abstract:** Temperature field calculation is an important step in infrared image simulation. However, the existing solutions, such as heat conduction modelling and pre-generated lookup tables based on temperature calculation tools, are difficult to meet the requirements of high-performance simulation of infrared images based on three-dimensional scenes under multi-environmental conditions in terms of accuracy, timeliness, and flexibility. In recent years, machine learning-based temperature field prediction methods have been proposed, but these methods only consider the influence of meteorological parameters on the temperature value, while not considering the geometric structure and the thermophysical parameters of the object, which results in the low accuracy. In this paper, a multivariate temperature field prediction network based on heterogeneous data (MTPHNet) is proposed. The network fuses geometry structure, meteorological, and thermophysical parameters to predict temperature. First, a Point Cloud Feature Extraction Module and Environmental Data Mapping Module are used to extract geometric information, thermophysical, and meteorological features. The extracted features are fused by the Data Fusion Module for temperature field prediction. Experiment results show that MTPHNet significantly improves the prediction accuracy of the temperature field. Compared with the v-Support Vector Regression and the combined back-propagation neural network, the mean absolute error and root mean square error of MTPHNet are reduced by at least 23.4% and 27.7%, respectively, while the R-square is increased by at least 5.85%. MTPHNet also achieves good results in multi-target and complex target temperature field prediction tasks. These results validate the effectiveness of the proposed method.

**Keywords:** three-dimensional scene; temperature field; intelligent prediction; network; geometry structure; meteorological parameters; thermophysical parameters

## **1. Introduction**

Infrared imaging technology has the characteristics of high penetration, strong antiinterference, good concealment, and high precision, which can significantly compensate for visible-light imaging technology's lack of night vision capability. With the rapid development of infrared imaging technologies, infrared imaging systems have been widely applied to military, industrial, and civilian applications [1]. To develop such systems, it is essential to obtain the appropriate system parameters in advance. This requires a large number of sample images under different lighting conditions for testing and evaluation. However, owing to the complex influences of region, scenery, time-of-day and meteorological conditions, obtaining a sufficient number of samples often requires extensive re-sources and labor. Under extreme conditions, it is impossible to obtain a sufficient number of test samples. To

**Citation:** Cao, Y.; Li, L.; Ni, W.; Liu, B.; Zhou, W.; Xiao, Q. Amalgamation of Geometry Structure, Meteorological and Thermophysical Parameters for Intelligent Prediction of Temperature Fields in 3D Scenes. *Sensors* **2022**, *22*, 2386. https://doi.org/10.3390/ s22062386

Academic Editors: Moulay A. Akhloufi and Mozhdeh Shahbazi

Received: 9 February 2022 Accepted: 18 March 2022 Published: 20 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

overcome this limitation, infrared simulation has been proposed. It obtains infrared images by simulating actual infrared imaging processes by traversing three-dimensional (3D) scene construction, temperature field and radiation calculations, atmospheric radiation transmission calculation, and imaging instrument simulation. Among these, the temperature field calculation is the most important.

Research on temperature field calculations has undergone a remarkable evolution. Initially, researchers used empirical or semi-empirical models to calculate a temperature field. For instance, Jacobs [2] used a one-dimensional thermal model to calculate the temperatures of simple geometries. Biesel and Rohlfing [3] obtained an object's surface temperature by setting a series of assumptions for the heat balance equation. Curtis and Rive-ra [4] established an empirical surface temperature model that comprehensively considers the influences of time, material type, meteorological conditions, and object orientation. Balfour and Bushlin [5] established a general expression of surface temperature with respect to the sun, sky, air temperature and wind speed. However, these models are labor- and resource-intensive. Moreover, they cannot adapt to changes in details, and their accuracy is low.

To meet the requirements of accuracy, first-principle models were used for temperature field calculations. This model is based on the principle of heat transfer. The heat balance equation is established by considering various factors that affect the temperature change of the object; the temperature value is calculated by numerical calculations. For instance, Gonda et al. [6] introduced the temperature prediction model, which uses a hot node network method to calculate the temperature field distribution on the surface of an object. Sheffer and Cathcart [7] developed a thermal calculation model using a first-principle model, which considers factors, such as solar and sky radiation, mass transfer process, fluid transmission, occlusion, and multiple reflections, and can more accurately obtain the temperature change of the object. Currently, several commercial temperature field calculation software programs, such as TAITherm (https://thermoanalytics.com/taitherm, accessed on 27 February 2022) [8], Fluent (https://www.ansys.com/zh-cn/products/ fluids/ansys-fluent, accessed on 27 February 2022), and Vega, which are based on firstprinciple models, have been developed. They realize high-precision target temperature field calculations by setting thermophysical and meteorological parameters. However, for calculating a temperature field to deter-mine a target, it is usually necessary to input several parameters, such as material, thickness, shape, atmospheric temperature, and wind speed and direction. This impedes calculations at different periods and under varying meteorological conditions, and over-whelms the current GPUs. Hence, it cannot support real-time infrared simulations.

Considering the first-principle models' calculation speed bottleneck, Hu et al. [9] proposed a scheme that uses the temperature field calculation method to generate the temperature data of a typical target scene under typical environmental conditions in advance and save it in a database lookup table. The temperature value is then obtained using database interpolation. A look-up table significantly increases the simulation speed, but it is limited in its ability to accommodate sampling resolution design and interpolation methods with numerous input meteorological parameters. Moreover, accuracy cannot be guaranteed if only the main input parameters are considered.

The temperature field calculation method proposed in this study is based on machine learning and is designed to meet real-time, high-precision and flexible infrared simulation requirements. It uses a data-driven approach to establish a mapping of parameters affecting the model's temperature field distribution to the model temperature, which essentially fits the heat-balance equation established by the first-principle model. Huang and Wu [10] proposed a similar method based on a combined back-propagation (BP) neural network to establish a relationship between model temperature and meteorological parameters. Huang et al. [11] screened meteorological parameters using the heat balance equation and used the ν-support vector regression (v-SVR) model [12] to fit the model temperature. This meets the real-time requirements of simulations. However, contemporary machine

learning models only consider the influence of meteorological parameters on temperature and ignore the influence of other factors, which affects accuracy.

This study provides a novel temperature field calculation method based on machine learning for high-precision real-time prediction of temperature field under the influence of multiple environmental variables in the real-time simulation of a 3D scene's infrared im-aging. It addresses the limitations of the contemporary models by comprehensively considering geometry structure, meteorological, and thermophysical parameters, which meets the requirements of real-time and accurate temperature field prediction. The main contributions of this study are as follows:


The experimental results validate the effectiveness of our proposed algorithm. The remainder of this article is organized as follows: Section 2 describes our analysis process and the proposed method in detail. In Section 3, the data formats, evaluation metrics, and training methods used for training are introduced. In Section 4, corresponding experiments are designed to verify the effectiveness of this method, and the experimental results are analyzed and discussed. Section 5 draws some conclusions about our method.

#### **2. Materials and Methods**

*2.1. Analysis of the Parameters That Affect the Temperature Field Distribution of the 3D Model in the Natural Environment*

#### 2.1.1. Calculation Principle

A series of heat transfer processes with different mechanisms occur between the surface of a 3D model in a natural environment and the atmospheric boundary layer. A 3D model comprises different materials; the methods and speeds of heat exchange between different materials and the external environment are different.

Figure 1 illustrates the energy interactions between an object and the external environment, which ultimately results in thermal equilibrium. For example, temperature differences between objects cause heat transfer (Figure 1A). Energy can also be transmitted directly to objects by solar radiation (Figure 1B). Atmospheric particles can also transfer energy to objects after absorbing external radiation (Figure 1C). Heat energy can be transferred from surrounding objects to the target object (Figure 1D). Fluid flow also contributes to energy transfer (Figure 1E). Lastly, energy transfer can be caused by the evaporation of water, water vapor condensation, and migration (Figure 1F).

Based on the law of the conservation of energy and the processes illustrated, the heat balance equation of an object's surface is as follows:

$$\left.k\_i \frac{\partial T}{\partial n}\right|\_i = a\_s E\_{\text{sum}} + a\_l E\_{\text{sky}} + \sum\_{j=1}^{M} Q\_{rj} - \epsilon \sigma T^4 \pm Q\_\epsilon \pm Q\_{\text{ec}} \tag{1}$$

where *ki ∂T ∂n*  is the heat conduction of the object, *asEsun* denotes the ability of an object to absorb solar radiation, *alEsky* denotes the ability of an object to absorb radiation from the sky, *M* ∑ *j*=1 *Qrj* denotes the radiative heat transfer from other objects around, *σT*<sup>4</sup> denotes the selfradiation of the object, *Qc* denotes convective heat transfer, and *Qec* denotes hidden heat.

**Figure 1. A** 3D model of processes of energy interactions to reach thermal equilibrium under natural conditions: (A) heat transferred by temperature differences between objects; (B) energy directly transmitted to objects by solar radiation; (C) energy transferred by particles to objects after absorbing external radiation; (D) heat radiation energy transferred from surrounding objects to the target object; (E) energy transferred by fluid flow; and (F) energy transferred by water evaporation, water vapor condensation and migration.

With Equation (1), when the boundary conditions at each moment are known, the temperature field distribution at each moment can be calculated. The calculation result at the current moment is also the boundary condition at the next moment. By analyzing the above-mentioned energy transfer process, we can filter the variables that play a key role in the calculation of the temperature field distribution.

By analyzing the above-mentioned energy transfer process, we can filter out the variables that play a key role in the calculation of the temperature field distribution:

(A) Heat conduction: Owing to the collision of numerous molecules and subatomic particles, energy flows from a high-temperature object to a low-temperature object. For a temperature change caused by heat conduction, the main influencing factors are the properties of the object itself, including thermal conductivity, thickness, shape, etc.

(B) Sun radiation: Objects absorb radiant energy from the sun, which is a form of radiant heat transfer. When the object is on a clear and cloudless level surface, the formula is as follows:

$$E\_{s0} = [1 - A(\mathcal{U}^\*, \beta)](0.349E\_0)sin\beta + \left(\frac{1 - \rho\_0}{1 - \rho\_0 \overline{\rho\_\mathcal{S}}}\right)(0.651E\_0)sin\beta \tag{2}$$

where *E*<sup>0</sup> denotes the solar radiation of the entire waveband, *A*(*U*∗, *β*) denotes the absorbable coefficient, which is a function of relative humidity, air temperature, and solar altitude, *β* denotes the solar elevation angle, *ρ<sup>g</sup>* denotes the reflectivity of the ground, and *ρ*<sup>0</sup> is the Rayleigh reflectivity of the atmosphere, which is a function of the solar elevation angle.

Considering the cloudy sky, Equation (2) is modified to obtain the following formula:

$$E\_{f\text{sum}} = E\_{s0} \cdot CF \tag{3}$$

where *CF* is a function related to cloud coverage.

Therefore, the main factors influencing the temperature changes caused by solar radiation are relative humidity, air temperature, solar altitude angle, and cloud coverage. The solar altitude angle is related to the longitude, latitude, time zone, and date. In this study, it is assumed that the temperature field is calculated in a fixed scene; hence, the longitude, latitude, and time zone are invariant. Therefore, the main influencing factors of temperature changes caused by solar radiation are relative humidity, air temperature, date, and cloud coverage.

(C) Sky radiation: Atmospheric particles, such as carbon dioxide and water vapor, are present in the atmosphere. These particles absorb external radiation; thus, they have a certain temperature. Therefore, sky radiation is essentially generated by the thermal radiation of atmospheric particles, and it affects objects on the ground. The formula for sky radiation is as follows:

$$E\_{sky} = \left(a + b\sqrt{\varepsilon}\right)\sigma T\_a^4 \tag{4}$$

where *Ta* is the sky temperature, which can be calculated from cloud coverage, atmospheric temperature, humidity, and altitude; a and b are related to the location and time of the measurement, and e is a function of relative humidity and atmospheric temperature.

Because altitude and location are constant in this study, the temperature changes caused by sky radiation are related to cloud coverage, atmospheric temperature, humidity, and time.

(D) Radiation from other objects: When the temperature of an object is higher than absolute zero, it spontaneously radiates energy. Therefore, when there are other objects around it, it is affected by their radiation. Hence, it is necessary to obtain the surrounding objects' temperature data.

(E) Convection heat transfer: Fluid flow further affects temperature changes. For ground objects, the main influencing factors of temperature changes caused by convective heat transfer are wind speed and direction, and air temperature.

(F) Latent heat is the energy transfer caused by the evaporation of water; and condensation and migration of water vapor. The object studied in this study does not involve heat exchange in this area.

2.1.2. Determination of the Parameters That Affect the Surface Temperature Field of the Object

Because this study focuses on the calculation of a 3D target's temperature field at a fixed altitude and location, the main meteorological parameters are date, atmospheric temperature, solar radiation, wind speed, relative humidity, cloud cover, and wind direction. The main thermophysical parameters are space coordinates, density, specific heat, conductivity, thickness, convection method, emissivity, absorptivity, and initial temperature.

#### *2.2. Design of 3D Target Temperature Field Prediction Model Based on Heterogeneous Data Fusion*

Predictive modelling of temperature fields based on machine learning is essentially a fitting of first-principle models of thermodynamics. According to the analysis in Section 2.1, this mainly includes three heat transfer processes: heat conduction, heat radiation, and heat convection.

In the first-principles model, the factors affecting the temperature of the model can be divided into two categories: the first category is meteorological parameters, which are time series data that record meteorological indicators at each moment, such as atmospheric temperature, wind speed, and direction, which characterizes the energy exchange between the object and the atmosphere, mainly reflects the heat radiation process and the heat convection process in the heat transfer process; the other is thermophysical parameters. If the object is regarded as composed of countless particles, then the thermophysical parameters can be regarded as a kind of point cloud data, which record the emissivity, thickness, and specific heat of each particle, which characterizes the energy exchange between points in the object and mainly reflects the heat conduction process in the heat transfer process. In addition, the spatial location distribution will cause occlusion and other phenomena and will also affect the exchange of energy. Therefore, the geometric structure information will also affect the distribution of the temperature field. Although it is not a thermophysical parameter, it corresponds to each point, so we classify it as a thermophysical parameter. These two types of data are heterogeneous and determine the temperature field distribution of the 3D model.

The existing temperature field prediction model based on machine learning only considers the influence of meteorological parameters on the temperature of the target model, while ignoring the influence of thermophysical parameters, which is equivalent to considering only the thermal convection and thermal radiation models in the first principle, while ignoring heat transfer. This results in poor prediction accuracy. We introduced a Transformer [13] to solve this problem.

The Transformer is a classic work by Google. It completely abandons the traditional neural network structure and uses an attention module [14] to process data. The use of self-attention to process data, which can effectively integrate is effective for integrating heterogeneous data.

This study comprehensively considers the thermophysical and external meteorological parameters that affect the temperature of a 3D target model. The proposed MTPHNet method improves the structure of the Transformer model using meteorological parameters as the input of the encoder, and thermophysical parameters as the input of the decoder. It uses the self-attention module to fuse the two parts of data to improve the generalization ability of the model. The structure of MTPHNet is shown in Figure 2.

**Figure 2.** Structure of the multivariate temperature field prediction network based on heterogeneous data (MTPHNet). MLP: Multilayer perceptron.

The use of MTPHNet to predict the model temperature field can be expressed by Equation (5):

$$\Psi\_{temp} = \psi\_t \left( \phi(Enc(\mathfrak{x}\_{obj}), Dec(\mathfrak{x}\_{env})) \right) \tag{5}$$

where *xobj* denotes the thermophysical parameters of the 3D model, such as space coordinates, thermal conductivity, and reflectivity; *xenv* denotes meteorological parameters, such as atmospheric temperature, wind speed, and direction; φ denotes the fusion process of

thermophysical parameters and meteorological parameters to obtain fusion features; and ψ*<sup>t</sup>* represents the regression prediction process, which calculates the temperature value to be predicted.

In this study, the 3D target model is represented as point cloud data. Each data point is considered an object in space and has its corresponding attribute information, such as material, thickness, and thermal conductivity. Therefore, *xobj* <sup>∈</sup> <sup>R</sup>*P*×*A*, where *<sup>P</sup>* denotes the number of points in the 3D target model, and *A* denotes the number of attributes corresponding to each data point. Meteorological parameters are time-related, and each moment corresponds to a set of meteorological data. Therefore, *xenv* <sup>∈</sup> <sup>R</sup>*T*×*E*, where *<sup>T</sup>* denotes the duration, which is obtained by sampling at a fixed step in a period, and *E* denotes the number of attributes used to describe the external environment.

It is evident that thermophysical and meteorological parameters are two sets of heterogeneous data with different dimensions. However, in the natural environment, meteorological parameters act on the thermophysical parameters of the 3D model. Simultaneously, the temperature of each data point is affected by the temperature of other points around it. Therefore, the thermophysical parameters of the 3D target model affect each other. These highly coupled heterogeneous data determine the temperature of the 3D target model; therefore, this complexity cannot be handled by general data fusion methods. Thus, we use the thermophysical parameters of each point as input to the Transformer encoder. A point corresponds to a token, and the interaction between the points of the 3D target model is simulated using the encoder's calculation. Subsequently, the meteorological parameters are used as input to the Transformer's decoder for feature mapping. Finally, the two parts of the features are fused, and the fused features are regressed to calculate the predicted value of the 3D target temperature field.

#### 2.2.1. Point-Cloud Feature Extraction Module (PCEM)

In the real environment, objects can be envisioned as a composition of countless particles, and different points have different materials and spatial positions. Different materials will often have very distinct emissivity, absorption, and scattering [15] properties, which can result in a variety of particle energy absorptions and releases. Different spatial positions will lead to phenomena, such as occlusion and shadows, resulting in uneven energy distributions. Therefore, the thermophysical parameters of the 3D object are crucial to the establishment of a temperature field. To improve the accuracy of temperature field prediction, we must extract the object's thermophysical parameters.

The thermophysical parameters include spatial coordinates, emissivity, and specific heat, which can be regarded as point cloud data with additional attributes. Temperature field prediction requires the calculation of the entire 3D target model, and each point interacts with all other points, implying that the thermophysical parameters of each are dot-produced with the thermophysical parameters of other points. The computational complexity of the original Transformer is proportional to the square of the length of the input sequence [16]; however, the number of 3D point cloud points is large, which is unsuitable for most hardware.

To solve this problem, we apply PointNet [17], a feature extraction layer for pointcloud data of the 3D target model. PointNet, proposed by Qi et al. (2017), can be directly used to process point cloud data. The model extracts features via feature mapping and maximum pooling of point cloud data and satisfactorily completes the classification task. However, because the model extracts features from single points and does not consider the relationship between points, its local feature extraction ability is weak [18]. Therefore, it is impossible to analyze complex scenes.

In this study, the 3D target model is assembled from different parts. First, we group the point clouds of the 3D target model according to the types of parts. The point cloud attributes of the same part are similar; however, the point cloud attributes of different parts are different. Subsequently, the point cloud data are organized according to the part category and each group of point cloud data are first sent to a self-attention module for calculation to obtain the relationship between points. The calculation results are sent to the PointNet for local feature extraction. The feature extraction process for the point cloud data of the 3D target model is shown in Figure 3.

Figure 3 indicates that we do not configure a self-attention module and PointNet, respectively, for the point cloud data of each group of parts to extract features. We rather refer to the convolution kernel of weight sharing in convolutional neural network [19] and use a unique self-attention module and PointNet which perform feature extraction on the point cloud data of different parts. Each group of parts is extracted as a local feature vector; the feature items extracted from all the parts are formed into a new sequence; and are then sent to a new self-attention module and PointNet to extract global features. This way, all features for the 3D target model are extracted.

#### 2.2.2. Environmental Data Feature Mapping Module (EMM)

Meteorological parameters directly affect the temperature of objects. Rain reduces the surface temperature of objects, the shielding effect of clouds weakens solar radiation, and wind accelerates the heat transfer between the air and the surface of the object [20].

The thermophysical parameters of the 3D target model are mapped into a fixed-size feature block after passing through the PCEM. The thermophysical and meteorological parameters of the 3D model are heterogeneous data from different sources. To achieve the integration of heterogeneous data, we introduced a multi-layer perceptron (MLP) module to map meteorological parameters to a high-dimensional space through feature mapping and map them to a fixed size to match the feature block of the thermophysical parameters. The EMM is illustrated in Figure 4.

**Figure 4.** Schematic of environmental data features mapping module.

#### 2.2.3. Data Fusion Module (DFM)

In the natural environment, meteorological and thermophysical parameters undergo complex physical interactions to determine the temperature field distribution of objects. In this study, we use a self-attention module to fuse the thermophysical and meteorological parameters. We use the feature block output by the encoder as the K and V of the self-attention module, and the feature block output by the decoder as the Q of the self-attention module. This process simulates the interaction between meteorological parameters and the 3D target model in the natural environment. A schematic of the integration of thermophysical and meteorological parameters is shown in Figure 5.

**Figure 5.** Schematic of the integration of internal parameter variables and external environmental parameter variables.

#### 2.2.4. Pseudocode

Based on the above analysis, the pseudocode of MTPHNet shown in Figure 2 is summarized, and the algorithm is given in Algorithm 1.


Algorithm 1 shows that *tmax* is the maximum duration of temperature field prediction, and *P* is the number of parts in the 3D model.

In line 2, the algorithm replaces the dimension representing temperature in *x<sup>t</sup> obj* with *<sup>Y</sup>t*−<sup>1</sup> *temp*. From lines 3 to 6, the algorithm extracts the local features of the 3D model using *attn*\_ *f eature* and *pointnet*\_ *f eature* for each part. In line 7, the algorithm uses *attn*\_ *f eature* and *pointnet*\_ *f eature* to extract the global features of the 3D model. In line 8, the algorithm uses *MLP* to make a feature map of *x<sup>t</sup> env*. Finally, the algorithm uses *attn*\_ *f eature* to fuse *xt dec*\_*in* and *<sup>x</sup><sup>t</sup> enc*\_*out*, and *Linear* to obtain the temperature value.

#### **3. Experimental Details and Data Exploitation**

*3.1. Experimental Environment and Index Design*

The experiment was conducted on an AMD Ryzen 7 CPU 5800H with 16 GB of RAM, NVIDIA GeForce RTX 3090 with 24 GB of memory, Python 3.7.2, and PyTorch 1.9.0 for network model training and testing.

To evaluate the effect of temperature field prediction, mean absolute error (*MAE*) [21], root mean square error (*RMSE*) [22], and *R*<sup>2</sup> [23] were selected as the evaluation criteria for the model quality. The calculation formulas are as follows:

$$MAE(y, \hat{y}) = \frac{1}{n} \sum\_{i=1}^{n} |y\_i - \hat{y}\_i| \tag{6}$$

$$RMSE(y, \hat{y}) = \frac{1}{n} \sum\_{i=1}^{n} \|y\_i - \hat{y}\_i\|\_2^2 \tag{7}$$

$$R^2(y, \hat{y}) = 1 - \frac{RSS}{TSS} = \frac{ESS + 2\sum\_{i=0}^{n} (y\_i - \hat{y}\_i)(\hat{y}\_i - \overline{y})}{TSS} \tag{8}$$

$$TSS = \sum\_{i=1}^{n} (y\_i - \overline{y})^2 \tag{9}$$

$$RSS = \sum\_{i=1}^{n} (y\_i - \hat{y}\_i)^2 \tag{10}$$

$$ESS = \sum\_{i=1}^{n} (\mathcal{Y}\_i - \overline{\mathcal{Y}})^2 \tag{11}$$

where *y* denotes the true value; *y*ˆ denotes the predicted value; *y* denotes the average value of the true value; *TSS* is the Total sum of squares, which defines the difference between *y* and *y*; *RSS* is the Residual sum of squares, which defines the difference between *y* and *y*ˆ; and *ESS* is the Explained sum of squares, which defines the difference between *y*ˆ and *y*.

Among the selected evaluation indicators, MAE and RMSE are used to measure the size of error between the predicted and real data; R-squared measures the quality of the fit.

#### *3.2. Dataset*

The training data used by existing temperature field prediction models based on machine learning methods were collected by instruments. These type of data are closer to reality. However, owing to the variability of natural environmental parameters and the in-stability of instruments, the data acquired by the instrument are noisy and costly.

We use our own temperature dataset constructed by ourselves, which includes the thermophysical parameters, meteorological parameters, and temperature field data of 3D objects.

#### 3.2.1. Dataset Format

We use the thermophysical and meteorological parameters of the dataset as input to MTPHNet and the corresponding temperature data as its output to train and optimize the model parameters.

The shape of the 3D target model has an impact on the temperature field formation. Under the same environmental conditions, different shapes will cause uneven heat distribution in the 3D target model, for instance, objects in shadow will be cooler than objects in direct sunlight. Therefore, the thermophysical parameters in the dataset first need to obtain the spatial position information of the 3D target model. We built several 3D models using 3D modeling software and exported them to OBJ file format. Because OBJ file uses the face

element data structure to build the 3D model and the proposed model uses the point cloud data structure, we processed the exported OBJ file and calculated the center coordinates of each face element to replace the face element. Figure 6 shows the constructed 3D model.

**Figure 6.** A 3D model and its corresponding point cloud data.

Figure 6 shows that each 3D object has several data points. In addition to spatial coordinates, each data point contains additional attribute information, such as material, thickness, and initial temperature. Table 1 shows the point cloud data format of the 3D target model during training.

**Table 1.** Point cloud data format for 3D targets.


In addition to the 3D point cloud data, meteorological parameter data are required. For this study, we collect meteorological parameter data for four seasons. Combined with the parameters that must be collected in the analysis above, we selected date, atmospheric temperature, solar radiation, wind speed and direction, relative humidity, and cloud coverage as environmental parameter variables. Figure 7 shows the meteorological parameters related to time, and the changing trends.

**Figure 7.** External meteorological parameters for temperature field prediction. Parameters from the left to right and top to bottom are atmospheric temperature, cloud cover, relative humidity, rainfall rate, solar radiation, wind speed, and direction.

According to the collected thermophysical and meteorological parameters, we use an internal temperature field calculation software to calculate the temperature field distribution of the 3D model data and add the calculation results to the dataset for training the model.

From Equation (5), *Ytemp* <sup>∈</sup> <sup>R</sup>*P*×*T*, which means *<sup>P</sup>* points, and each point has *<sup>T</sup>* temperature values.

#### 3.2.2. Teacher Forcing

As discussed before, the temperature of the 3D target is affected by the sun, atmosphere, and surrounding objects. It is evident that the temperature of the 3D target model at each moment is determined by the meteorological and thermophysical parameters at the current moment.

Among the features of the thermophysical parameters, one dimension of the feature represents the temperature of the point cloud data. Because the 3D model is represented by a point cloud, each point represents a distinct object. Therefore, this dimension represents the distribution of the temperature field at the current moment.

The temperature field distribution is obtained at the next moment by entering the data into MTPHNet to measure the difference between the two one time step.

Because unknown information cannot be used in the test, the calculated value is assigned to this dimension of the input data to calculate the temperature value at the next moment, after calculating the temperature value at the current moment. Figure 8 illustrates the process.

**Figure 8.** Temperature substitution process. The temperature value calculated by the model replaces a certain dimension of the input to simulate the temperature change of all objects in the temperature field at each moment.

During the training, the temperature value at any time is known. Therefore, there is no need to use the temperature value calculated by the model to replace the value of the dimension, which allows parallel calculations during the training.

#### **4. Results and Discussion**

#### *4.1. Performance of the MTPHNet*

To demonstrate that the MTPHNet successfully integrates an object's thermophysical and meteorological parameters can further improve the prediction of the temperature field, we used temperature field data, thermophysical parameters, and meteorological parameters as training data and compared the performance of MTPHNet with those of v-SVR and a combined BP neural network (CBPNN) model.

When training MPTHNet, the hyperparameters needed by the model included batch size, epoch, number of multi-heads, and initial learning rate. The batch size affects the degree of optimization and model speed. The size of the epoch affects the fitting effect of the model. Tuning the number of multi-heads helps the network capture richer features. The initial learning rate determines if and when the objective function converges to a

local minimum. To obtain better hyperparameter values, we used Microsoft's automatic parameter tuning tool, NNI, for hyperparameter selection, which runs the code in a loop to obtain the optimal hyperparameter values. The results of the operation are shown in Figure 9:

**Figure 9.** Results of NNI.

As can be seen in Figure 9, the batch size was set to 16; the number of epochs was 100, the number of multi-heads was 6 and the initial learning rate was 0.0003. As it is based on the Transformer structure, the MTPHNet model is large and needs a significant amount of memory. Considering computational efficiency and fitting accuracy, we selected the Huber loss function and the Adam optimizer for optimization. Dropout was used for overfitting mitigation, and the deletion ratio, *p*, was 0.05.

Because the thermophysical parameters of the 3D target were considered, the MTPH-Net trained different 3D models with the same number of point clouds. However, v-SVR and CBPNN only consider the impact of meteorological parameters on the temperature of the 3D target and cannot simultaneously predict the temperature field of different 3D target models. For referential significance, all three models were trained with the same training set, which includes the temperature field distribution data of a single 3D model. MTPHNet was better than the other prediction models after testing on the test set. Table 2 presents the generalization performance of the models.


**Table 2.** Comparison of generalization performance of MTPHNet, v-SVR, and CBPNN.

As shown in Table 2, the MTPHNet prediction error, was significantly lower than that of the existing temperature field prediction models. Compared with the CBPNN model, its MAE and RMSE decreased by 23.4% and 27.7%, respectively, whereas the R-squared increased by 5.85%. Figure 10 shows the prediction effects of the models.

Because the 3D model was composed of patches, in the experiment, we extracted several patches by generating random numbers to show the effect of temperature field prediction. We selected patches 96,231 and 423 for presentation. The experiments demonstrated that, although the existing temperature field prediction methods fit the temperature field on the change trend, their accuracies were insufficient. Therefore, it is necessary to combine the energy interaction mode of the 3D object in the natural environment and its meteorological and thermophysical parameters to further improve prediction accuracy.

**Figure 10.** Generalization performance renderings. Prediction effect diagrams of (**a**) v-SVR; (**b**) CBPNN; and (**c**) MTPHNet.

#### *4.2. Advantages of MTPHNet*

#### 4.2.1. Multi-Object Temperature Field Prediction

The results indicate that MTPHNet has a better fitting ability than the existing temperature field prediction models. Because MTPHNet comprehensively considers the various energy exchanges between the object and the environment and combines the thermophysical parameters of the object for training, it simultaneously trains and predicts the temperature field for different 3D targets. Existing temperature field prediction models cannot achieve this.

In this study, we summarized the 3D target temperature field data shown in Figure 5 and imported them into MTPHNet for training and verification. Table 3 presents the fitting performances.

As shown in Table 3, when the MTPHNet model was used to predict the temperature field of multiple objects, the values of its various indicators were satisfactory. The experimental results demonstrate that the thermophysical parameters of the 3D target model are significant for temperature field prediction. Figure 11 shows the effects of the multiobject temperature field prediction. Here, three materials were used for the temperature field calculation. For a convenient comparison, we selected patch 1 for presentation. The same patch shows the effect of different materials on temperature and the adaptability of MTPHNet to different temperature changes.


**Table 3.** MTPHNet's generalization performance for multi-object temperature field prediction.

(**b**)

**Figure 11.** Multi-object temperature field prediction effects. Fitting of different materials of (**a**) box; (**b**) cylinder; and (**c**) sphere.

4.2.2. Prediction of Temperature Field of Complex Objects

When predicting multiple objects, this study assumed that each object had only one part; thus, the attribute data of different points are the same. In reality, however, a complex 3D object is composed of different materials, and the energy exchange between them is more complicated than that of a single material. Therefore, we chose a complex model for training and prediction. Figure 12 shows the geometry of the model.

**Figure 12.** Complex house model with 5660 patches and 30 parts: (**a**) geometric structure; (**b**) temperature field distribution at a given moment.

Table 4 demonstrate that the model has a good generalization performance for the temperature field prediction of complex models, which further reflects the superiority of MTPHNet. Figure 13 shows the prediction effect of the temperature field of complex objects. We randomly selected patches 1032 and 4000 for presentation.

**Table 4.** MTPHNet's generalization performance for temperature field prediction of complex objects.

**Figure 13.** Prediction of the temperature field of complex objects.

#### *4.3. Ablation Analysis*

To verify the effectiveness of our proposed network, we conducted three ablation experiments to verify the performance of the main design components: environmental data feature mapping module (EMM), point cloud feature extraction module (PCEM), and data fusion module (DFM). The proposed MTPHNet is given as MTPHNet-A, and its variants for ablation are MTPHNet-B, MTPHNet-C, and MTPHNet-D. All variants were trained and validated using the same procedure described in Section 4.1. Each ablation experiment was performed three times and the results were averaged, and shown in the Table 5 and Figure 14.

**Table 5.** Quantitative evaluation metrics of MTPHNet and its variants. All models follow the same procedure and training environment as described in Section 4.1 and are evaluated on the same test set. The best results are shown in bold.


(**c**)

**Figure 14.** Multivariate temperature field prediction network based on heterogeneous data (MTPH-Net) variants for ablation experiments: (**a**) MTPHNet-B removes the environmental data feature mapping module (EMM) to study the effect of meteorological parameters on temperature field prediction; (**b**) MTPHNet-C removes the point cloud feature extraction module (PCEM) to study the effect of thermophysical parameters on temperature field prediction; and (**c**) MTPHNet-D re-places the data fusion module (DFM) with an additive fusion method to study the effect of data fusion on temperature field prediction.
