Towards the Use of 3D Thermal Models in Constructions

Abstract

The use of point clouds in architecture and civil engineering has, to date, been limited almost exclusively to functional geometric features. Nevertheless, hardly any works have attempted to process and explore 3D thermal models for buildings. This paper presents a method for the visualisation and exploration of 3D thermal models (3D-T) of building interiors. A 3D-T model consists of a thermal point cloud, which has been generated with a 3D thermal-scanner platform. Given a 3D-T of a building at a specific time, the user can visualise and navigate through different room models and each room can, in turn, be segmented into its architectonic components (walls, ceilings and floors), from which thermal orthoimages can be generated. When the building is sensed at different times, a 3D temporal-thermal (3D-TT) model is integrated. The temporal-thermal evolution of these structural components, along with selected zones of them, can then be analysed by performing a new type of thermal characterisation. This method has successfully been tested using real building-related data.

1. Introduction

The digitalisation of buildings with 3D data-generation devices and the use of the resulting point clouds is currently a very widespread practice in the field of Architecture, Engineering and Construction (AEC). However, its application has been limited almost exclusively to the evaluation of the geometric characteristics of the building [1,2,3].

Although the geometric information is of great importance as regards assessing the state of a building, other kinds of evaluations are also necessary. One of these is thermal evaluation, which provides data on the energy performance of a building and identifies zones with special thermal characteristics [4,5].

The thermal evaluation of a building is often carried out using thermal cameras. These cameras generate 2D images containing information on the temperatures of different zones of the building. The thermal characteristics of the different architectural components can also be explored. However, thermal cameras have a reduced field of vision, which complicates the task of correctly positioning the images in order to create a whole model of the building. [6]

Most thermal camera manufacturers provide specific tools for the analysis of these images, as occurs with, for example, the Fluke (“Fluke SmartView”) [7], the FLIR (“Advanced Thermal Analysis and Reporting Software”) [8] and the “Thermal Analysis and Reporting” [9]) software applications. With regard to the visualisation and evaluation of point clouds, most manufacturers of 3D scanners also provide their own tools, such as Riegl (“Riscan Pro 2.0” [10]), Faro (“Faro Scene” [11]), and Leica (“Leica Cyclone” [12]). Apart from these specific applications, there are also multiple tools that are not strictly focused on the evaluation of buildings, such as Meshlab [13] and CloudCompare [14], which have a wide range of functionalities. These functionalities may be of little use to AEC professionals since they are not adapted to specific civil engineering nor architectural applications. Meshlab focuses on processing and editing 3D triangular meshes and provides tools for editing, cleaning, healing, inspecting and rendering meshes. It is mostly used for free-shape object modelling. CloudCompare provides basic tools for editing and rendering 3D points clouds but does not generate 3D geometric models of architectural components from segmented points.

Over recent years, new tools have appeared on the market that are focused on the analysis of point clouds within the field of AEC (“Nubigon” [15], “PointCab 3DPro” [16], “ReCap Pro” [17]), and some of the most popular software tools among architects and civil engineers have incorporated new functionalities in order to process these types of data (“Autocad” [18], “IFC Builder” [19], “Revit” [20]).

As mentioned earlier, various applications can currently be found on the market within the field of AEC for the exploration of 2D thermal images and point clouds of buildings, but there are no applications that specifically process and explore 3D thermal models of buildings. Some related tools, such as Recap [17] and CloudCompare [14], provide the option of point cloud imports with intensity values (in this case, corresponding to the temperature), making use of colour schemes to display the point clouds. Only a visual thermal exploration of the model can be completed, because those applications include functional tools for neither thermal evaluation nor the exploration of a building.

Some new studies have emerged that provide the means to simultaneously evaluate both thermal information and 3D information. For example, Nüchter et al. developed The 3D Toolkit [21], which provides algorithms and methods with which to process 3D point clouds, including a viewer for the inspection of the models, with information on reflectance, colour, and thermal information. The option of switching between one mode of representation and another helps to detect and to locate sources of energy expenditure in the RGB model. Moghadam et al. presented the Spectra application [22], which contains a set of tools for multispectral fusion and visualisation. This software, despite not specialising in the inspection of building models, permits the simultaneous representation of the thermal spectra and the visual spectra on a single 3D mesh, which, among other options, can highlight extreme temperature zones. In [23], a tool was presented for the ex-situ inspection of building interiors, acquired through the use of the IMAS digitalisation platform. The user can inspect the interior of the building by means of immersive virtual navigation through the point clouds, where the following information is shown: the point at which the user is located at any one time, the RGB and the thermal images taken from that point, and all luminance, temperature, and humidity values measured at that point. In another study [24], a system was proposed for the qualitative and quantitative viewing of thermal information and the evaluation of the conditions required for thermal comfort at different interior locations of a building. The Rhinoceros tool [25] was used with the Grasshopper plug-in [26]. The system permits the user to view changes in thermal information in the Building Information Model (BIM) over time. In [27], a method was presented for the automatic analysis of deviations between the real and the simulated energy states of a building and their visualisation. The EPARV2.0 visualisation tool generates the initial energy model of the building through the use of a set of digital and thermal images, and the simulated model is calculated by analysing Computational Fluid Dynamics (CFD). These models are superimposed onto a common virtual 3D environment, on which the areas identified as potentially problematic are shown.

In this paper, we present a development that can be used to evaluate 3D thermal models of buildings, which provides a series of functionalities designed to facilitate the evaluation of those models. The present article is organised as follows. Section 2 presents an overview of the acquisition of the 3D-T models, while Section 3 shows an exploration and visualisation of 3D-T models. Section 4 deals with the analysis of 3D-TT models, and the applicability of our system is shown in a case study in Section 5. Finally, our conclusions are presented in Section 6.

2. Obtaining 3D Thermal Models of Buildings

The principal objective of the proposed method is to monitor 3D thermal models of building interiors. It provides Architecture, Engineering and Construction (AEC) professionals with the opportunity to visualise and explore the models without the need to be in situ in the building. This exploration will be of great use as regards detecting zones with thermal features, such as thermal bridges and dampness, while simultaneously positioning them with great precision within the building.

The acquisition of a 3D point cloud with thermal information must first be carried out, after which several algorithms will process this huge amount of information and provide users with valuable functionalities and results. The 3D thermal data-acquisition system is composed of a 3D RieglVz 400 scanner (RIEGL Laser Measurement Systems, Horn, Austria), an RGB Nikon D90 camera and a FLIR AX5 thermal camera, as shown in Figure 1a.

During the acquisition stage, the whole system rotates twice. First, the scanner captures the 3D coordinates of the visible scene in a 360° turn. Afterwards, the thermal cameras shot 12 overlapped photos in a second 360° turn. Points and temperatures are eventually matched, yielding a 360-thermal point cloud of the visible scene. More information on this technique can be found in [28].

The T-point cloud of a room is then obtained by aligning a series of consecutive T-scans from different positions of the scanner. When a new scan is taken, the thermal point cloud is first preprocessed and then aligned into a common reference coordinate system. With the help of an inclinometer the point cloud is aligned in such a way that the floor points lie on horizontal planes and the wall points lie on vertical ones. The registration of the current point cloud with the previous scan is primarily carried out by using the odometry of our scanning platform MoPAD [29], and is later refined by applying the well-known ICP (Iterative Closest Point) technique. The accumulated point cloud of the scene is eventually updated. More information on this stage can be found in [30]

Figure 1b shows an example of a 3D thermal point cloud composed of three T-scans. The colour palette assigned to the points helps the user to understand the current scene in a 3D environment from a thermic point of view.

The T-scanning of a building follows the strategy shown in Figure 2. This is a sequential process in which the 3D-T models of different adjacent rooms are integrated into an accumulated model. In practice, we use MoPAD, which is able to autonomously navigate and collect 3D thermal data in the indoor environments of buildings, as shown in Figure 3a. A 3D thermal model of a floor plan of a building obtained with MoPAD is shown in Figure 3b,c.

3. Exploring 3D Thermal Models

The complete T-point cloud of a building can be explored by studying separate rooms, which can also be segmented into their different Architectonic Components (AC). We have established a chain of exploring functionalities that ranges from the generation and visualisation of a building floor plan to the exportation of orthoimages of the architectonic components of which it is composed. Figure 4 shows a schematic diagram of the different functions and steps.

The thermal model of the building (such as that shown in Figure 3c) is initially a disorganised point cloud, consisting of millions of 3D coordinates with their respective temperature values. A 2D image of the floor plan of the building provides a general view of the structure of the building, in which the user can move around and select the room/s to be explored. This automatically activates the room exploration functionality. The observer can then move to different positions in the interior of the room, rotate the camera or zoom the view. This is very useful as regards approaching specific zones of the architectonic components and analysing temperature details in them.

Our system additionally provides other options, such as setting the temperature range or defining a specific colour code for a better visualisation. Apart from the visual exploration of the model, the system can yield quantitative thermal information on rectangular zones, which are selected beforehand, and provide the average, minimum and maximum temperature values.

As mentioned above, the thermal model can be segmented into its architectonic components, which are typically walls, ground, ceiling and columns. A RANSAC based algorithm [30] yields point cloud clusters belonging to different architectural components of the building. The input of the algorithm here is the unorganised point cloud of the scene together with certain parameters, such as the maximum permitted distance on the plane, the number of iterations and the stop threshold. These parameters greatly affect the segmentation results and must first be tested and then fitted. The raw 3D point cloud is eventually visualised as a set of segments in different colours, which are also freely hidden or displayed. Since segmentation failures might occur, the input parameters can be refitted in order to obtain better segmentation results.

Note that the information captured by our system corresponds to 3D thermal points in a 3D context, which is an unusual format when dealing with thermal data. A thermal reconstruction of a building using exclusively thermographic cameras is unpractical. Such a technique would need hundreds of photos to generate a 3D thermal model of the scene, as it happens with a conventional RGB-photogrammetric technique. Thus, performing a thermal evaluation of a building with such an enormous number of images, each corresponding to small areas of the building, is a tedious and impractical task. Additionally, the precise positioning of each image in the 3D geometric model of the building (if it is available) is also very difficult.

With a view of resolving the aforementioned problem, our system first obtains the whole 3D point cloud of the building, then segments it into a set of architectural components and, finally, calculates the corresponding thermal orthoimages. This last stage is carried out by projecting each segmented AC point cloud onto its corresponding fitted plane and discretising the projected points. We then generate a set of 2D images and fill the gaps in each one by using the Navier–Stokes filling algorithm [31]. The reason for these gaps comes from the fact that the number of points collected from the scanner with an assigned temperature is less than the resolution of the 2D orthoimage. In other words, not all pixels of the orthoimage have an assigned temperature. If the resolution is very high, there will be some pixels without assigned temperature. On the contrary, if it is very low, all pixels will have an assigned temperature (even several temperatures might fall on the same pixel and, in this case, the average temperature would be assigned). In our case, the resolution of the orthoimage is one pixel per centimetre.

4. Temporal Models

This is a new way in which to manage the thermal information of a building. If several T-scan sessions are repeated at regular time intervals, a multi-temporal T-point cloud of the building is obtained. This is what was denominated as the 3D-TT model in the abstract. Our system has been designed to use this data structure in order to analyse the temporal evolution of each room, along with its corresponding ACs. The analysis is carried out by means of a visual exploration of the orthoimages of the ACs and several graphics of the mean temperatures of selected zones. Figure 5 shows a schematic diagram of the different steps required to deal with and compare 3D-TT models.

The original input of this process is, therefore, a set of 3D thermal models of the same scene but acquired at different times. These models are subsequently segmented into a set of 2D orthoimages corresponding to different ACs. It is, therefore, possible to attain a visual comparison of the temperature at a specific time, along with its evolution over time, for each AC.

The system additionally allows the selection and labelling of regions within an AC, denominated as regions of interest (RoI) in Figure 5. This is the thermal characterisation stage in Figure 5, in which the thermal evolution over time of a selected zone is evaluated by employing a series of basic statistical indicators, such as the average, the minimum and the maximum temperatures. This basic statistical analysis can be exported to a .csv file, which facilitates comparisons with data gathered by other sensors and data estimated using common energy simulation software.

Data acquisition and data processing stages have been programmed by the authors of this paper using free software libraries (with the exception of Matlab). Specifically, the data acquisition process has been developed under ROS framework and using C/C++ and Matlab. For data processing and monitoring, we have used C++, OpenCV and PCL libraries.

5. Case Study

In this section, we show the exploration and analysis results of a 3D thermal model obtained with the aforementioned MoPAD platform. The model is composed of 122 million points and corresponds to a 16-room building.

Figure 6a shows the 2D floor plan (plan view) of the building, which was generated by projecting the raw 3D thermal point cloud. This thermal point cloud model is composed of 165 million points that covers 17 rooms and a large central nave of 7.4 m height at the west side of the building. The scenario is 40.70 m × 28.45 m in size with average height of 3.95 m. The complete thermal digitalisation required 87 thermal scans from 27 scanner positions.

Several views of different selected rooms are presented in Figure 6b–d. These images clearly show hot and cold areas of the thermal point cloud viewed. The possibility of personalising the properties of the scale and the colour-code map helps the users to visualise the temperature variations in the scene. The mean temperature of the selected zones is always available.

Figure 7a illustrates the 3D point cloud model of a room that is segmented into different architectonic components, with the points belonging to each one in a different colour. Figure 7b presents the thermal point cloud of some of the segmented architectonic components in separate windows. Figure 7c shows an example of the original thermal orthoimage of a wall and the final result after filling the thermal gaps.

In order to carry out the temporal thermal analysis, 48 3D thermal models of the same room were taken every 30 min over 24 h. The scenario is an inner courtyard illuminated by a high and large cover glass, which provides a natural source of heat. Figure 8 shows a panel with 12 representative orthoimages of a particular AC at times in which the temperature variation is evident.

In a second step, a detailed study on this architectural component is performed by selecting and labelling certain zones of interest. Figure 9b presents the temporal evolution of the temperature in four selected zones. We have chosen four regions belonging to: (1) a picture on the wall, (2) the left-hand side of the wall, (3) the door on the right-hand side of the wall, and (4) the alarm device see Figure 9a.

It is clear that the temperatures of regions (1) and (2) follow similar evolutions during the day, slighting decreasing from 16:30 onwards. The temperature of region (4) is by far the highest, reaching 24 °C, and is maintained throughout almost the entire day. On the contrary, region (3), which corresponds to the door, undergoes the highest variation, rising to 19.6 °C at 16:30 and falling to 15.2 °C at 08:30. We assume that the door communicates with some sort of storeroom with deficient thermal insulation.

In order to compare the evolution of the temperature of regions (1) to (4) with respect to external and internal (ambient) temperatures, we include the respective graphics in Figure 9c. Note that, although the external temperature varies significantly from one day to the next, the internal temperature remains quite constant for 24 h, decreasing from 18 to 17 °C. Note also that regions (1), (2) and (4) follow the same decreasing pattern, which demonstrates the good thermal insulation of the courtyard. Nevertheless, the surface of the door (region (3)) follows the external temperature pattern and is, therefore, considered as the main thermal leakage area of that wall.

This is just one example of how the use of 3D temporal thermal models can help engineers to understand the thermal behaviour of a building and locate different regions with different thermal inertias and transmittances. This process can be carried out for any part of the 3D thermal model, and is a new way in which to explore and analyse the thermal efficiency of buildings.

One of the aspects to be considered when dealing with thermal data regards the ground truth. Obviously, the ground truth in this kind of experiment is not available because there is not a system able to register the true temperature of such amount of points on walls, ceilings and floors. Therefore, in order to yield an approximate error measure, we conducted a set of local measurements in walls by using S-TMB onset temperature sensors with accuracy of 0.2 °C and compared temperatures at the corresponding positions. The mean error reported in this experiment was below 0.7 °C.

Finally,

Table 1. Times of the acquisition and data processing stages.

Process	Time
360-scan (Point Cloud)	47 s
360-thermal panoramic photo (12 2D thermal images)	59 s
Generation of a 360-thermal point cloud (matching temperatures to 3D points)	1 s
Segmentation of an accumulated point cloud	250 s
Orthoimage generation process	2 s
Orthoimage refining process	10 s
Temperature evolution of a single AC
-  Acquisition and data processing	110 s
-  Interval time between samples	30 min
-  Total time of the session	24 h

provides interesting information of average times for different parts of the whole process. As regards the acquisition phase, the time of a single 360-thermal scan is separated into the time spent in the collection of 3D coordinates and the time devoted to obtaining thermal photos. As can be seen, each of these takes less than a minute in time. The point-temperature matching step lasts around one second. Times assigned to the segmentation and recognition processes greatly depend on the amount of points collected. For a point cloud of ten million points we have estimated an average time of around 250 s. Orthoimage generation and refining can take 10 to 12 s depending on the resolution of the image. Finally, times for experiments of temperature evolution are included in the last rows. Taking a single architectural component (e.g., a wall), the time is greatly reduced because the 3D coordinates are only collected in the first scan. Thus, in the following samples, the scanner does not take 3D data and only the set of 12 thermal photos are captured, which are afterwards matched to the already existing points. For sessions of 24 h, 48 samples at time intervals of 30 min are taken.

6. Discussion

This article presents the work developed in our lab in order to obtain and explore 3D thermal models of buildings and to perform analysis of the thermal evolution over time

In the last ten years, several research groups have worked on the creation of the so-called geometric BIM models, including ours. However, there are very few groups in the world that obtain thermal BIM models, which include the dimension “temperature”. Almost all of these groups generate only unstructured and partial point cloud models, with colour information and temperature.

The applicability of the methodology proposed in this article is principally intended for architects and civil engineers. Among other applications we firstly point out the visualisation and detection of zones with special thermal features in a 3D environment and the extraction of thermal images of architectural components separately. Secondly, a new application is the study of the evolution over time of the temperature of different architectural components and the subsequent comparison. This analysis leads the engineers to better understand the energetic behaviour of the building and the energy-modelling process.

As mentioned in Section 2, our hardware component is a mobile scanning platform that is able to collect 3D thermal data of interior of buildings without the human intervention. All this huge information is later organised and prepared in order to be easily explored and analysed. Several visualisation options have been programmed within our system by using low level programming languages and free software tools (with the exception of MATLAB). The objective here is to explore, at stated times, whatever zone of the building in a 3D framework. On the other hand, the study in time of thermal point clouds has entailed a great increase of data and new complex processing algorithms. With the help of these algorithms, the construction professionals have identified specific zones with different thermal inertia.

The current version of MoPAD has several limitations and disadvantages. The first limitation concerns the energetic autonomy of the system. The MoPAD’s battery provides power supply to the robot, the sensors and the two on-board computers. Owing to the amount of data collected and the hard processing, the whole system can operate only for several hours. This implies to stop the scanning process every four hours in order to recharge MoPAD batteries. In the case of a large building this might break the scanning process in several sessions.

The second limitation regards the applicability outdoors. So far, our system is only applicable to building interiors. The navigation procedure and the data processing (including registration and segmentation of point clouds) is only designed for indoors of buildings. Of course, outdoors could be scanned using MoPAD, but most of the acquisition algorithms should be adapted or totally reprogrammed.

The third disadvantage is that the 3D data processing is focussed on dealing with plane architectural components. Therefore, data corresponding to curve and irregular ACs are neither segmented not recognised, and the final 3D thermal model in these regions would be wrong.

Finally, it is worth noting that, during the data acquisition stage, neither people nor moving objects are permitted. Note that the inclusion of people would even entail a temperature noise source in the scene.

7. Conclusions

The use of 3D data-acquisition devices to generate point clouds with geometric information on buildings has become a common practice in the field of architecture and civil engineering. Some researchers and companies provide developments with effective and practical functionalities that can be used to evaluate this geometric information. However, few specific works that explore and analyse 3D thermal models can be found to date.

This article presents a representative case study in which a thermal point cloud of the floor plan of a building has been generated and analysed. The results obtained demonstrate the potential utility of our system and its applicability to real scenarios. Nevertheless, this work needs to be improved and extended in the near future. Among other improvements, we aim to generate efficient algorithms that will segment an AC into its representative thermal zones. This could be solved by studying the temporal evolution of selected thermal regions. Furthermore, we feel encouraged to extend our research to multi-story buildings with new practical functionalities.