*5.1. Test Object*

The experiments were carried out on a particular object, designed so that the two aforementioned extrinsic calibration methods were both equally suitable and easy comparable. In addition, the particular design of the object shown in Figure 8 allowed us to evaluate the effectiveness of the ray-casting technique on a geometry with concavities. It has an internal conic hole and three different radial parts (shifted by angles of 120 degrees), characterized by many edges.

**Figure 8.** (**a**) 3D printed object; (**b**) Its CAD model.

The object was designed with CAD, 3D-printed with acrylonitrile butadiene styrene (ABS) material and then scanned with the Konica Minolta Vivid 9i laser scanner. The point cloud could be obtained directly from the CAD model, but the object was scanned to take into account the acquisition bias and test the effectiveness of the method on laser scanner data.

Here we present a first set of simulations, carried out to assess the effectiveness of the automatic extrinsic calibration method. This was made by comparing the extrinsic parameter values obtained by means of this automatic procedure with the ones resulting from the method based on the manual selection of homologous points. Because of the specially designed geometry features of the printed object, it was possible to choose for the first method a set of about 20–30 homologous points for each analyzed thermogram, with a mean reprojection error (MRE) of about 1–2 pixels. In this way, the proper reliability of the first method parameters was assured, and so it was possible to evaluate the ones obtained with the second method. Equation (9) shows the expression for computing the MRE. The units of MRE are pixels.

$$MRE = \frac{\sum\_{i=1}^{n} \|P\_i - Q\_i\|}{n} \tag{9}$$

In Equation (9), *n* is the number of homologous points selected, *Pi* the 2D coordinates of the *i-th* point selected on the thermogram, *Qi* the 2D coordinates of the reprojection of the *i-th* homologous 3D point.

For each thermogram, the convergence from different initial positions was analyzed, within a range of ±25 degrees for the Euler angles and ±20 mm for the translation vector from the parameters obtained with the first method.

In Table 1, some of the results of the first calibration method are shown for four different thermograms, whereas Table 2 gives the differences (considering the same thermograms) between the extrinsic parameters obtained with the second method and with the first one respectively.


**Table 1.** Extrinsic parameters for four different thermograms, in the case of the method based on the manual selection of the homologous points (first method).

**Table 2.** Differences between the extrinsic parameters obtained with the automatic method (second method) and the ones obtained with the first method.


Figure 9 shows a set of 2D points and the reprojection of 3D homologous points, once the extrinsic parameters were computed.

**Figure 9.** Grayscale thermogram with highlighted the set of 2D points (purple circles) and the reprojection of 3D homologous points (yellow cross) once the extrinsic parameters were computed, in the case of thermogram B of Table 1.

With reference to Table 2, thermograms A and B were obtained starting from a first guess of the parameters with the following differences with respect to the ones of Table 1: Δ*t*1 = 20 mm, Δ*t*2 = −20 mm, Δ*t*3 = 20 mm, Δα = <sup>−</sup>20◦, Δβ = 0◦, Δγ = 20◦. For thermograms C and D, the initial differences were set as follows: Δ*t*1= −10 mm, Δ*t*2= 20 mm, Δ*t*3= 10 mm, Δα = <sup>−</sup>20◦, Δβ = 15◦, Δγ = 15◦.

As can be seen in Table 2, the final differences are relatively low, except for the Δ*t*3, which is of several millimeters. However, this can be acceptable, considering that the *t*3 parameter is greater by an order of magnitude compared to the parameters *t*1 and *t*2 (relative error of about 2%).

In Figure 10, the initial (a) and final (b) projection in the case of automatic extrinsic calibration of the thermogram B are shown. This type of visualization allows for a qualitative consideration on the effectiveness of the method. As can be easily seen, in the case (b), the 3D projected points (in red) fill the thermogram contour (yellow line) well.

**Figure 10.** Case of automatic calibration of thermogram B: (**a**) Initial projection of 3D points, using a set of initial extrinsic parameters; (**b**) Projection utilizing the parameters for which the objective function of Equation (1) presents a global minimum.

Figure 11 shows a representation of the matrix *M*1 (see Section 3, Figures 3 and 4) converted for visual purposes into a color image.

=

**Figure 11.** Visual representation of the matrix *M*1 created from the yellow contour of Figure 10. In the scale, *y* refers to the function graphed in Figure 4, which is here multiplied by a factor of 10<sup>4</sup> for the sake of clarity.

=

The second set of experiments was aimed at evaluating the general reliability of the technique, by qualitatively assessing the final results obtained with the integration of different thermograms.

In Figure 12, the result in the case of the integration of thermogram B is shown, whereas Figure 13 shows an example of the integration of multiple thermograms (17 in total) acquired from different poses. The shots were taken while the object was being heated with a drier, fixed so that the heat flux came in contact with one side in particular (the one with higher temperature, colored yellow).

**Figure 12.** Two different views of the result of the integration of thermogram B on the point cloud.

**Figure 13.** Four different views of the result of the integration of seventeen thermograms.

Figure 14 shows how the union of several thermograms (acquired from different poses) by the method explained in Section 4.5 can compensate for the systematic error in the temperature caused by the dependence of the emissivity on the viewing angle. In Figure 14c, the temperatures of a single thermogram and of the union of several thermograms (the temperature superimpositions vary from 4 to 8 depending on the point) are compared along a key zone, shown in Figure 14a,b, in which the normal vectors of the surface have a significant variation, which implies a significant variation on the viewing angles (referring to the case of the single thermogram, since after the union the concept of viewing angle loses meaning). In this zone, the temperature of the single thermogram is appreciably lower than the temperature of the union (with a maximum difference of roughly 1 ◦C). This behavior can be explained by correlating the difference Δ*T* of the temperatures (Δ*T* = *TUNION* − *TSINGLE*) with the variation of the viewing angle, as shown in Figure 14d. The camera view during the acquisition of the single thermogram can be approximatively assumed to be the view of Figure 14a, which explains the angle of about 40 degrees for *Y* in the range 5–10 and 16–25 mm (the camera was tilted downwards with respect to the normal vectors in these points by about 40 degrees). For non-conductor materials, emissivity is nearly constant from 0 to 40–45 degrees, whereas at larger angles it has a significant drop [1] (pp. 39–40). This justifies the fact that the temperature in the zone of high viewing angles is underestimated in the case of the single thermogram. This error can be effectively compensated by the adopted method, as long as, for the same points, temperatures with lower viewing angles associated are available.

**Figure 14.** Effect of the temperature correction achieved by the union of different thermograms. (**a**) Zone in which the temperatures are examined, with the vectors normal to the surface in these points colored red; temperatures are shown with a "winter" colormap; (**b**) Close-up of the normal vectors, from a top view; (**c**) Comparison between the temperature of a single thermogram and the one resulting from the union of several thermograms; (**d**) Temperature difference (*T* resulting from the union minus *T* of the single thermogram) and viewing angle for each point examined.

#### *5.2. Statue "Madonna with the Child"*

After testing the method on the object previously described, in this section, an application in the cultural heritage field is considered. The integration was carried out on the statue Madonna with the Child, by the Florentine sculptor Agostino di Duccio (1418–1481), with an eye to monitoring its condition and gather additional information about its state. The point cloud of the statue was already available from previous works. Figure 15 shows the statue and the laser scanner in the Bishop's palace in Forlì. For the sake of brevity here, we present only some of the most significant results.

**Figure 15.** Statue Madonna with the Child, with the laser scanner in position.

In Figure 16, three thermograms of the statue acquired from different poses are shown.

**Figure 16.** Three thermograms of the statue, acquired from different poses.

Figure 17 shows several outcomes of the integration. There is a little spot on the top of the head to which no temperature is assigned, and to which a specific colour not belonging to the colormap is assigned. The survey shows that there is one side of the statue that is slightly hotter, whereas the base and the head present lower temperatures. It is probable that the particular lighting system and the statue arrangemen<sup>t</sup> near a window with a non-insulated frame give rise to these changes, which is not favourable for optimal preservation. Further investigations are needed to better clarify the cause.

**Figure 17.** Results of the integration of twelve thermograms. (**a**) Three different views coloured with the colormap "hot". (**b**) Frontal view coloured with the colormap "jet".
