Towards Airborne Thermography via Low-Cost Thermopile Infrared Sensors

Abstract

This paper presents a novel tool capable of collecting thermal signatures inside a building by using low-cost IR temperature sensors mounted on-board an aerial platform. The proposed system aims to facilitate the detection of heat loss inside buildings, which is a key aspect for improving energy efficiency in large commercial or industrial buildings. Current detection systems usually require manual labor as well as the use of expensive instrumentation. The proposed system on the other hand, relies on the use of a small unmanned aerial vehicle carrying low-cost thermopile IR sensors. Moreover, the system delivers a fast temperature sensing scheme and it provides coverage to inaccessible areas, thus overcoming the limitations of current mobile platforms which use ground robots. Different experiments were carried out in order to assess the behavior of the sensors as well as to validate the full system. Moreover, the hypothesis that thermopile IR sensors can be used to track temperature signature on-the-fly is validated experimentally with the use of the proposed system over different targets.

1. Introduction

Environmental protection and conservation has been a matter of high concern for many public and private entities in the past decades, as a response of these concerns, many government agencies have developed and implemented action plans so as to overcome or correctly manage different environmental challenges [1]. Those plans are oriented, among many other aspects, to improve the lifetime of energetic resources, also to reduce misspent energy, and to increase the quality of pollution control techniques. Moreover, one of the most important, environment-oriented, challenges nowadays is the improvement of energy efficiency in buildings and other infrastructures [2]. These challenge of reducing of misspent energy in buildings is the motivation behind the development of the system presented in this paper, which intends to develop an autonomous remote sensing platform capable of monitoring and mapping heat loses in large buildings.

In the European Union, buildings account for 40% of energy consumption and 36% of CO₂ emissions, in order to wade into this issue, in June of 2018, the Council of the European Union published the amending Energy Performance of Buildings Directive (2018/844/EU), this directive requires state members to define a strategy to improve Europe’s energy saving and de-carbonization by 2050 [3]. One of the strategies that can help to achieve the goals of the aforementioned directive, is the detection of energy losses in large non-residential infrastructures. Furthermore, with this objective in mind, the continuous inspection and monitoring systems become essential steps towards the setting up of autonomous energy control systems that can improve the energetic efficiency in those type of buildings.

In many industrial and commercial buildings, the continuous heating and cooling airflow cycles constitute a critical source of energy waste [4]. In order to overcome this problem, different energy monitoring methodologies have been applied, both outside and inside buildings. One of the main techniques used is the creation of thermal maps from specific sites of the building. These maps can be used by experts to perform post hoc analysis and to design energy efficiency strategies. The system presented in this paper proposes a novel approach to these tasks, by developing an aerial autonomous remote sensing platform, which is capable of monitoring and mapping heat loses in large buildings.

Remote sensing practices have improved significantly with the adoption of Unmanned Aerial Vehicle (UAV) equipped with color and false color cameras [5,6,7]. Nevertheless, thermal imaging or thermography, with these small UAV is only emerging and there is room for improvements, thus making the proposed system a relevant development.

The most common approach for monitoring the energy of a building, is taking images from outside the building using a UAV equipped with a high-resolution thermal camera. Several views can be used to get temperature profiles [8]. Another example is found in the work of Aguilera et al. that proposes a system capable of generating thermographic 3D models as well as orthoimages in order to assess energy efficiency in buildings facades [9].

Regarding indoor energy-aware applications, most of them rely on static sensors monitoring [10], manual measuring [11], or mobile sensor platforms [12]. Finally, when it comes to continuous monitoring or exploring the building, an autonomous robot with the appropriate sensors can be a suitable solution [13]. A mobile sensor platform denoted as Irma3D, which carries a thermal camera and a laser scanner is used for building indoor 3D heat distribution and flow models [12]. Mobile platforms have been applied to more specific environments as well, for example, a robot named THOMAS was used for temperature monitoring in data centers [14]. The robot is made up with inexpensive chassis and carries a conventional digital camera and a thermocouple interface.

In this paper, a novel approach to the continuous monitoring is presented, it is based on the use of an Aerial Remote Sensing System (ARSS) equipped with several low-cost Infra-Red (IR) temperature sensors. It was developed in order to advance in the supervision of energy efficiency with autonomous remote sensing platforms. This work explores an scenario where high resolution thermography is not a requirement. Furthermore, this prototype was designed to aid critical energy-aware buildings, such as a data center. This type of infrastructure consumes much more energy than standard office workspaces and there is usually a large margin for energy efficiency improvement with adequate temperature monitoring [15].

A comparative between the existing thermography assessment approaches is given in

Table 1. Comparison of different cost/effort per thermography approach found in literature. The crosses used for qualifying the costs can be read as: (+) less costly, (++) costly, and (+++) very costly.

			Automated
		Manual	Fix Cameras	UGV	UAV
Cost/Effort	Human effort	+++	+	++	++
	Time	+++	+	++	++
	ROI	+++	+++	+	++
	Area coverage	+++	+++	++	+
	Accuracy	+	+	+++	++
	Spatial resolution	+	+	+++	++
	Sensors	+++	+++	+	+

. This qualitative analysis focus on the human effort, completion time, return of investment (ROI), area coverage by the sensing systems, accuracy of the sensors, spatial resolution achieved, and price of the instrumentation. The detailed ROI analysis of manual and automated methods is provided in Appendix A.

The main contributions of this work are: (1) Design and experimentally validate a novel Software-Hardware (SW/HW) architecture for measuring and processing temperature data; (2) to prove the hypothesis that the MLX90614 infrared sensors can be placed in an aerial platform for measuring temperature in a real-time flight; and (3) to investigate if high changes of temperature could be detected on the fly using the combination of airborne and IR temperature sensors.

2. Proposed Approach

This section describes the proposed approach, it starts with a brief description of the overall approach, then the UAV and sensors used for the system are detailed, and finally, the software developed is explained.

2.1. Overall Approach

As aforementioned, the proposed system is based on the use of an ARSS equipped with several IR thermal sensors. This approach allows to track temperature signatures inside buildings because of its high-maneuverability, as well as its ability to perform vertical take-off and land in small areas.

The system is complemented with a specifically designed software, which allows to control and collect the measurements of the sensors, perform filtering and also to combine this information with the other systems running on the main computer of the ARSS.

2.2. Aerial Platform

The aerial platform selected is the Ascending Technologies quad-rotor Pelican, which is suitable for indoor environments. This UAV is characterized by its robustness among the small quad-rotor systems. Its minimum take-off weight is 630 $\mathrm{g}$ and its maximum payload is 500 $\mathrm{g}$. It can fly up to 20 $\min$ depending on the payload. The Pelican is also easily configurable due to a central main frame made of balsam wood where several carbon plates can be placed, thus facilitating the incorporation of additional devices and sensors.

The Pelican is equipped with a $2.1$ $\mathrm{G}$$\mathrm{Hz}$ quad-core processor, 4 $\mathrm{GB}$ of RAM, and an autopilot board. The autopilot contains 2 ARM7 micro-controllers, an Inertial Measurement Unit (IMU), and a pressure sensor. An additional Arduino Nano was used for the low-level sensors interface, i.e., temperature and ultrasonic distance sensor used to measure the quad-rotor height above the ground. Finally, the indoor coordinates can be delivered by the system proposed by Dryanovski et al. [16]. The complete ARSS payload, including some additional power and control boards, is presented in

Table 2. ARSS payload discriminated.

Device	Weight	Quantity
Atomboard and control board	220 $\mathrm{g}$	1
Yellow propeller protection	29 $\mathrm{g}$	2
Arduino Nano	6 $\mathrm{g}$	1
Arduino support	12 $\mathrm{g}$	1
I2C and power connection board	9 g	1
MLX sensor board	12 $\mathrm{g}$	5
MaxSonar sensor	4 $\mathrm{g}$	1
Total	292 $\mathrm{g}$	12

.

2.3. Thermal Sensors

The temperature sensors from the ARSS were selected based on three requisites: the sensors must be able to measure temperature remotely, have a low weight, be economically affordable, and provide an increased accuracy. The temperature sensors that were found to sum up all those characteristics are the IR thermopile sensors, e.g., the models MLX90614. The IR sensor measures the surface temperature of an object when pointing towards it via its thermal radiation. This phenomenon is explained by the fact that all the objects with a $T>0$ K emit a continuously thermal radiation [17]. The radiation emitted by the object is detected by the IR sensor that converts it in voltage, and later in temperature. This relationship is known by the Stefan–Boltzmann power radiated law and is given by:

$$P=A\epsilon \sigma T^4,$$

where A is the area of the surface radiated, $\epsilon$ is the emissivity value that ranges from 0.0–1.0, $\sigma$ is the Stefan–Boltzmann constant $(5.670367\times {10}^{-8}\mathrm{Wm}{}^{-2}{\mathrm{K}}^{-4})$, and T is the object temperature. The two main parameters of IR sensors are: Field of View (FoV) and Distance from the Object (DO). The sensors field-of-view should be enough to cover the whole object surface at an affordable distance-from-the-object, because the temperature readings are an average of the temperature covered by the sensor. The emissivities of the surfaces should be provided to the data processing algorithm in order to obtain more accurate measurements. In this work, we have considered a constant value for emissivity of 0.95, since most of the used materials present closer values. However, if the system is deployed in a specific scenario with a defined purpose, these emissivity values should be adjusted taking into account the target materials.

The MLX90614 is a non-contact IR temperature sensor with a thermopile chip made by an array of thermal elements. The sensor is hermetically sealed into a TO-39 metal housing. The MLX90614 is factory calibrated for an object emissivity of 1. This parameter can be customized without any further calibration and black body. Moreover, the sensor provides two temperature readings: object and room (ambient). It can measure a wide range of temperatures with resolution up to $0.02$ ${}^{\circ }\mathrm{C}$. The maximum—object and room—accuracy that can be achieved is $0.5$ ${}^{\circ }\mathrm{C}$ from temperatures measurements ranging from 0.0${}^{\circ }\mathrm{C}$ to 50${}^{\circ }\mathrm{C}$. Finally, an integrated SMBus provides a fast interface for building up a temperature sensor network [18].

Figure 1 shows a representation of the temperature sensing arrangement on-board the UAV. As can be observed, two different model thermal sensors (MLX90614) are used, the offer a FoV of 90° and 10°, respectively. The system must be able to measure data in all directions. Therefore, it is desirable to have the sensors set up on the edges of the robot as far as possible from each other, symmetrical (opposed arms of the quad-rotor) and pointing upwards and downwards.

The following configuration has been selected: one 90° FoV sensor on the top of the tower of the Pelican to measure the temperature upwards (called 13 and 14) and two 10° FoV sensors oriented to the ground—one on the left arm and one on the right arm (named 7, 8, 10 and 11). These sensors enable the monitoring of surfaces that are under the ARSS. On the sides there are two 90° FoV sensors (named 1, 2, 4 and 5). The sensors are placed on the plastic protections from the arms to measure the temperature on the ARSS sides (see Figure 2).

2.4. Software

Finally, all the software necessary to manage the ARSS middleware is open source and ROS-based [19]. A node has been implemented to address the graphical representation of the temperature values over the robot position on the workspace. This is the node that is responsible for creating a map of temperatures. One of its functions is to establish a range for each RGB color temperature linearly. Thus, the node receives a temperature and a position given by the robot and issues a color and a position to be represented. The diagram that shows the nodes and topics that address the temperature readings and mapping is shown in Figure 3.

3. Experiments and Results

In order to test the SW/HW system architecture described in Section 2, several tests were performed inside a building, during this experiments the UAV was manually controlled by an experienced pilot. The main objective of the tests is to validate the real-time data acquisition of both the omni-directional temperature and the flying height. Moreover, the test will also allow us to determine that there is not any significant electromechanical noise (i.e., discrete high/low temperature peaks) in the readings.

3.1. System Behavior Assessment

In the first experiment the relationship between the Sensor Coverage (SC) and the Distance from the Object (DO) was analyzed. The experiments were carried out by using a surface with 1 ${\mathrm{m}}^2$ that was heated up to 37 ${}^{\circ }\mathrm{C}$ (as measured with an additional hand-held sensor). The DO ranged from 50$\mathrm{cm}$ to 400$\mathrm{cm}$.

As aforementioned, one measurement value is given by the average of all the internal temperature points from the sensor. Therefore, the object or surface temperature measurement will be more accurate if the Object Length (OL) is larger than the Sensor Coverage at a determinateDistance from the Object. If the OL is smaller than the SC, then the internal sensor’s average readings will also include environment temperature measurements, which are usually lower. This produces a measurement that does not exactly corresponds to the actual temperature of the object that is being measured. Therefore, the DO must becomes one very important aspect to be taken into account when inspecting a building. The relationship between SC and DO is shown in the upper left image from Figure 4. For instance, the SC will be less than 1 $\mathrm{m}$ when the DO with the 90° sensor is less than 40 $\mathrm{cm}$, and the DO with the 10° sensor is less than 400 $\mathrm{cm}$, orange circles on the left image represent environmental temperature data that can affect the accuracy of the readings.

When measuring with 10° sensors the temperature readings will be less affected if the DO increases. This behavior is shown in the Figure 5, which depicts the temperature versus DO relationship. It can also be noticed that the temperature drops linearly with the 10° sensor, and exponentially with the 90° sensor. Consequently, the object temperature error will increase faster with the 90° sensor than with the 10° when the ARSS moves away from the object.

From the results of is experiment, it can be observed that the 10° sensor provides more accurate measurements when measuring smaller surfaces. However, if the target object has a large surface, such as a roofs or a wall, the 90° will perform good as well.

3.2. Full Platform Validation

The second set of experiments are oriented ot validate the overall performance of the proposed system. To achieve this, the platform was validated in indoor scenarios by measuring the temperature values that are above, below and in front the platform during a real-time flight. Two different tests were carried out, during the first one, the UAV was maneuvered close to some pedestrians, whereas for the second one, the platform flew over a warm electrical device. In both cases, the temperature profiles were obtained for every sensor on-board the platform, the positioning and labeling of each sensor is described in Section 2.3.

Figure 6 shows the measurements as the platform takes off and moves near some pedestrians. It can be observed how room temperatures start to increase during take-off (sensors 2, 5, 8, 11 and 14). A dashed rectangular area shows the time lapse when the UAV flew up to 2 $\mathrm{m}$ above the ground. This thermodynamic behavior is well known and can be recorded inside any building. Moreover, during this test, the bottom image shows several temperature peaks (i.e., data[4] in Figure 6 legend). Those measurements correspond to a side sensor pointing towards pedestrians located nearby, at distances up to 3 $\mathrm{m}$.

The second test involved some more advanced maneuvers. In this case, the UAV flew over a mobile robot with several working equipment, this robot has therefore a warm cover that was used as the main object to be measured. The measurements obtained during this tests are presented in Figure 7. It can be observed that as when the UAV hovered over the platform the sensors pointing down (7 and 10) indicate that an object with a high temperature was found below the platform. The images acquired by the on-board camera in the time frame 14.3–34.9 s are shown in Figure 8.

In order to have a reference value, the temperature inside the ground robot, as measured by its internal sensors was obtained, these sensor give a temperature of $40.9$ ${}^{\circ }\mathrm{C}$ when the robot was stopped and $41.6$ ${}^{\circ }\mathrm{C}$ when it was moving. The measurements obtained by the proposed system are presented in Figure 9. They show the measurements taken with the robot stopped and moving, as well as the previously mentioned reference values. The robot’s case was the surface area measured, as it evacuates the heat generated by the electromechanical components of the robot (i.e., computer, sensors and motors).

The heat provided by some of the additional components of the robot, is coherent with the fact that there are some of the temperature measurements detected by the drone while flying over the ground are higher that the average or reference value measured by the robot’s computer. The exact values that it should measure cannot be determined without accurate models of the heat sources and transmitting materials in the robot. However, the collected data can explain the measurements obtained by sensors 7 and 10 during the flight, since the robot surface has a temperature between the ones inside the robot (40${}^{\circ }\mathrm{C}$ to 42${}^{\circ }\mathrm{C}$) and around it (20${}^{\circ }\mathrm{C}$ to 22${}^{\circ }\mathrm{C}$).

4. Discussion and Conclusions

A novel temperature tracking concept oriented to energy efficiency management inside buildings is proposed in this paper. This tool is suitable to identify heat losses in industrial buildings, providing a key information to optimize the cooling management. This system is especially useful in large data centers with dozens of servers. The persistent temperature monitoring in those places, together with the adequate actions to improve the energy efficiency, will help to reduce the costs in millions of euros and contribute to a more sustainable planet.

The system proposed improves current state-of-the-art in mobile temperature tracking platforms, i.e., the thermal mapper [12]. This is possible because the sensing technology employed in ARSS improves the thermal mapper sensor response in 50%. Furthermore, the system proposed can achieve a maximum navigation speed around eight times higher than the speed from the mobile platform used in [12] (16 $\frac{\mathrm{m}}{\mathrm{s}}$ versus $2.2$ $\frac{\mathrm{m}}{\mathrm{s}}$). Additionally, this platform was developed to be cost/effort-effective providing many advantages in relation to conventional, and other merging thermography alternatives (see Table 1).

It was verified that the SW/HW decisions taken were appropriated for detecting temperature patches in real-time and without any random fluctuation in the readings. This conclusion is supported through a series of experiments (please refer to Section 3.2) conducted in an indoor setup.

The most important feature to validate was the viability of using the MLX90614 sensors for real-time temperature data acquisition from a flying UAV. It was additionally shown that a good relationship between distance-to-target and temperature readings can be achieved with the 10° FoV sensor. This feature might be important in applications where a secure flying distance is needed.

The ROI analysis provided in Appendix A shows that the initial investment of the proposed approach is less than that using fixed thermal cameras. It can be noticed that, over the years, there is an insurance factor that had not been considered in previous cases [20]. In any case, a company adopting these technologies is covered by some sort of insurance, which means that the small value that accumulates over the years could be discard and summed up to the company insurance. Even though, it is important to take into account that the figures indicated in [20] are not precise regarding which cameras are used and how many. This could be crucial when dealing with large areas to be monitored. Therefore, initial investment can increase considerably in these cases. Finally, cameras also have a life-time and maintenance that is not specified.

This work proved that this combination of technologies is viable and that it can be further extended to other applications such as motion detection.