*Article* **Monitoring Time-Non-Stable Surfaces Using Mobile NIR DLP Spectroscopy**

**Marek G ˛asiorowski 1, Piotr Szymak 2,\*, Aleksy Patryn <sup>1</sup> and Krzysztof Naus <sup>2</sup>**


**Abstract:** In recent years, Near Infrared (NIR) spectroscopy has increased in popularity and usage for different purposes, including the detection of particular substances, evaluation of food quality, etc. Usually, mobile handheld NIR spectroscopy devices are used on the surfaces of different materials, very often organic ones. The features of these materials change as they age, leading to changes in their spectra. The ageing process often occurs only slowly, i.e., corresponding reflection spectra can be analyzed each hour or at an even longer interval. This paper undertakes the problem of analyzing surfaces of non-stable, rapidly changing materials such as waxes or adhesive materials. To obtain their characteristic spectra, NIR spectroscopy using a Digital Light Projection (DLP) spectrometer was used. Based on earlier experiences and the current state of the art, Artificial Neural Networks (ANNs) were used to process spectral sequences to proceed with an enormous value of spectra gathered during measurements.

**Keywords:** NIR DLP spectroscopy; reflectance time-non-stable spectra; artificial neural network

#### **1. Introduction**

Nowadays, Near Infrared (NIR) spectroscopy has become very popular. Modern computers with more and more computing power and appropriate data analysis software make it possible to obtain a powerful tool that enables efficient processing of thousands of data points to obtain useful information about a tested sample. NIR spectroscopy has many significant advantages: the non-destructive nature of the test, no need to prepare a sample, simplicity and speed of measurements, and low cost. Some disadvantages bring limitations: the useful information is not directly available, and the sample may be described by thousands of variables. It often turns out that the spectra obtained are not repeatable [1].

Today, there are several examples of using NIR spectroscopy being used for various purposes, including the evaluation of food quality. One of them is to use a portable NIR spectrometer working in the range of 1396–2396 [nm] to collect the spectra of breast milk samples for quality evaluation [2], which is an essential matter for newborn children. The authors use different chemometrics to calculate and then develop 18 calibration models with and without using derivatives and the standard normal variate. Once the calibration models were developed, the best treatments were selected according to the correlation coefficients and prediction errors. The other example of using NIR to estimate food quality is included in [3]. The authors examined Visible (VS) and NIR spectroscopy usage to monitor grape composition within a vineyard to facilitate the decision-making process with regards to grape quality sorting and harvest scheduling. Measurements of grape clusters were acquired in the field using a VS/NIR spectrometer, operating in the 570–990 [nm] spectral range, from a motorised platform moving at 5 km/h. To analyse the obtained spectra, they used classical methods, e.g., a correlation function. In [4], the authors examined a novel prototype NIR instrument designed to measure dry matter content in single potatoes. The instrument is based on interaction measurements to measure deeper into the potatoes. It

**Citation:** G ˛asiorowski, M.; Szymak, P.; Patryn, A.; Naus, K. Monitoring Time-Non-Stable Surfaces Using Mobile NIR DLP Spectroscopy. *Electronics* **2022**, *11*, 1945. https:// doi.org/10.3390/electronics11131945

Academic Editor: Hamid Reza Karimi

Received: 25 May 2022 Accepted: 18 June 2022 Published: 22 June 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

measures rapidly, up to 50 sizes per second. The device also enables several distances to be recorded for each measurement. The instrument was calibrated based on three different potato varieties, and the calibration measurements were done in a process plant, making the calibration model suitable for production line use.

The other type of NIR spectroscopy usage is connected with the need to detect undesired substances. One of the examples is non-destructive detection of tomato pesticide residues using VS/NIR spectroscopy and prediction models such as ANNs [5]. The authors used VS/NIR spectral data from 180 samples of non-pesticide tomatoes (used as a control treatment) and samples impregnated with a pesticide with a concentration of 2/1000 [L], recorded by a spectroradiometer working in the range 350–1100 [nm], to train and then verify ANNs. The other example is included in [6]. The authors used NIR spectroscopy and characteristic variables selection methods to develop a quick way of determining cellulose, hemicellulose, and lignin contents in Sargassum horneri, i.e., the species of brown macroalgae that is common along the coast of Japan and Korea. Calibration models for cellulose, hemicellulose, and lignin in Sargassum horneri were established using partial least square regression methods with full variables. The last example showed the detection of measuring vitamin C and ellagic acid in wild-harvested Kakadu plum fruit samples [7]. The results of this study demonstrated the ability to predict vitamin C and ellagic acid in whole and pureed Kakadu plum fruit samples using a handheld NIR spectrophotometer. In the next paper [8], the NIR spectroscopy method was developed to analyze the oil and moisture contents of the plant Camellia gauchowensis Chang and C. semiserrata Chi seeds kernels. The authors used principal component analysis (PCA) and partial least squares (PLS) regression methods for calibration and validation. Finally, they obtained correlation coefficients of 0.98 and 0.95 for oil, and 0.92 and 0.89 for moisture, respectively, for calibration and validation.

More often, NIR spectroscopy is used with other methods to obtain the expected solution. One example is presented in [9]. The authors used NIR spectroscopy coupled with chemometric tools and obtained a fast and low-cost alternative solution for evaluating wood properties and quality categories. The obtained results of research showed that NIR spectroscopic data combined with powerful multivariate statistic tools and artificial intelligence solutions provided a fast and reliable tool, helpful in the decision-making process. The other opportunities are connected with NIR-absorbing organic semiconductors, especially for organic photovoltaic cells (OPV) [10]. OPV has increased its popularity in the field of renewable energy due to its lightweight, flexibility, and relatively low cost. To find new OPV materials, experiments with different types of NIR materials as active layers were conducted. Good results have also been achieved using NIR spectroscopy in conjunction with machine learning methods [11]. In this study, the authors used VIS/NIR spectroscopy for the effective discrimination of genetically modified (GM) and non-GM Brassica napus, B. rapa, and F1 hybrids (B. rapa X GM B. napus). As a classification method, the convolutional ANNs were used with success. More references to the advances in NIR spectroscopy and related computational methods can be found in [12,13].

Progress in the development of spectroscopy has led to the creation of a class of miniaturized spectrometers, including the VS and NIR [14,15]. The miniaturization of devices and systems is related to the tendency to perform non-laboratory measurements, including the adaptation of sizes to the in-line version [16,17]. An essential advantage of using smaller instruments is the potential possibility of implementing distributed measurement schemes and approaches to "remote" monitoring of environmental and "field" measurements.

As can be seen over a dozen different examples, the use of NIR spectroscopy, often supported by other classical methods and artificial intelligence methods, has been analyzed. This proves the great potential of NIR spectroscopy supported by other methods for data analysis. The factor connecting the above examples of NIR spectroscopy applications are the relatively slow processes of change taking place in the tested materials and substances, usually demanding monitoring every hour or at even longer intervals. Therefore, in this article the problem of using NIR spectroscopy supported by ANNs for the analysis of

rapidly changing processes of the transition from liquid to solid is discussed. The authors of this article did not find any other article dealing with this subject using the commonly used NIR spectroscopy. Waxes and adhesives commonly used in everyday life were selected as test objects. As shown in [18], features of the materials change in different environment conditions, e.g., ambient temperature. In this paper, the method of analyzing optical reflection spectra of objects whose properties change quickly over time using the Digital Light Projection (DLP) measurement technique was proposed. The surfaces of several different materials were selected as test objects, the optical properties of which can vary even within a few seconds and which can be first approximated as prototypes of materials used in electron technology. At the current stage, these test objects were: quick-drying glue, two-component epoxy glue, and natural beeswax. Spectroscopic measurements were carried out using the DLP NIRSCAN Nano EVM Spectrometer by Texas Instruments. To process the obtained spectra, ANNs were used. The earlier results of the research with the ANNs are included in [19,20].

The test measurements were carried out to estimate the possibilities and effectiveness of further mobile spectroscopic measurements of the surface of materials, emphasising mobility and the possibility of carrying out measurements in various conditions, especially apart from stationary laboratories presented.

A method of measuring reflectance with the spectrometer window positioned practically in direct contact with the tested surface was chosen. In-line measurement results were saved in .dat or .xlms files for processing with Matlab. The recording time of one reflectance spectrum in the 900–1700 nm range was 2.67 s, while the spectrum was recorded six times and averaged.

The kit version of DLP NIRSCAN NANO and the primary measurement schemes were used in the same way as included in [21,22]. The modernisation of the device allowed it to work more efficiently after using its own housing design. The test object in the configuration is typically placed on the windows at the top of the monochromator.

In the next section, the applied measurement method and stand were described. Then, the results of measurements were presented. At the end, the discussion on received results and final conclusions were included.

#### **2. Methods**

The spectra were recorded using DLP NIRSCAN NANO, i.e., a small-size Texas Instrument spectrometer operating in the light wavelength range from 900 to 1700 nm with optical resolution equal to 10 nm [23]. The measurements were carried out several times at a given measuring point, depending on the selected option. The dimensions of the device, equal to 58 × 62 × 36 mm allowed us to perform mobile studies of reflectance spectra. The device can work in the reflective mode. In Figure 1, the general view and data flow in the measurement stand are visualised. The hardware part of the stand consists of the DLP NIR EVM spectrometer connected with a PC using USB. The spectrometer produces light of a specific wavelength illuminating the examined sample and then measures the intensity of light that passes through the sample. The software part of the stand includes the DLP NIRscan Nano GUI (used to control the device, simple visualisation of measurements, collecting and saving of data in the appropriate format) and the Matlab program (used for data processing employing ANNs). The spectra obtained from the first software can be displayed and saved in two formats: CSV and DAT. Matlab accepts both file formats.

**Figure 1.** The measurement stand based on the DLP NIR EVM spectrometer: (**A**)—a scheme of the data flow, (**B**)—a general view of the stand.

#### **3. Results**

The research results were divided into two groups, depending on the type of examined materials:


Both types of materials changing from liquid to solid (solidification time) at different rates. Therefore, the research was conducted with different time scales to observe the essential parts of the processes.

The measurements results of both material groups are presented in the two following subsections. In the next subsection, the results of research using Artificial Neural Networks (ANNs) are included.

#### *3.1. Wax Materials*

Paraffin and beeswax were selected as representatives of the first group of materials. These materials are characterised by a quick transition from liquid to solid phase of approx. 30 [s]. Therefore, the time interval between consecutive measurements equal to 2.67 [s] was chosen. In Figure 2, the obtained results of seizing reflectance *R* for paraffin and natural beeswax for subsequent time steps and different wavelengths of light are illustrated. The reflectance is given without a physical unit. If *R* achieves 1, it means that all the light was reflected. At first glance, the transition of the paraffin is similar to beeswax. However, as it

turned out, the spectra of these two substances differed significantly in reflectance levels during the change of the state of aggregation. Still, the characteristics are very similar. Both materials have characteristic peaks around 1200 [nm] and 1400 [nm]. Some differences between the materials may be because due to their different colours, i.e., the tested beeswax is yellow while the paraffin is white. The total observation time ranged from 0 to 28.6 s, during which time the materials changed from liquid to solid. The value *t*<sup>1</sup> is equal to 0 [s], and *t*<sup>11</sup> is equal to 28.6 [s].

$$\bf{(A)}$$

**Figure 2.** Reflectance spectra of two wax materials: (**A**)—paraffin, (**B**)—beeswax.

#### *3.2. Adhesive Materials*

The next group of tested materials were adhesives. General-purpose adhesives available on the commercial market and used in households. The two following kinds of such adhesives were selected and then investigated using DLP NIR spectrometry:


Due to the longer time of transition from liquid to solid phase of the second group of materials, a longer time interval between successive measurements was selected, i.e., 60 [s]. All the measurements took 10 [min]. In Figure 3, the results of changes in reflection spectra over time were illustrated, i.e., over the course of the solidification process. The changes during the superglue test turned out to be not very expressive. Even after hardening, the glue was characterised by high transparency, and no apparent changes over time could be observed. Significant differences were observed only at the edge of the measurement ranges. A considerable fluctuation can be seen in the long-wave region. It may be due to the operating range of the device. A more extensive measuring range would allow a more precise material analysis.

Considering the reflectance spectrum of epoxy glue (Figure 3), similar to wax materials, characteristic peaks around 1200 [nm] and 1400 [nm] can be observed. After a detailed comparison, it should be stated that the characteristics peaks are more distant from each other than for the wax materials, i.e., the first peak can be seen below a wavelength of 1200 [nm] and the second at wavelengths higher than 1400 [nm].

**Figure 3.** Reflectance spectra of two adhesive materials: (**A**)—superglue, (**B**)—epoxy glue.

Moreover, it is tough to differentiate the lines corresponding to the successive time steps. The lines are overlapped, and their order changes for various wavelength ranges, e.g., the range from 900 [nm] to 1430 [nm] and the range from 1430 [nm] to 1700 [nm]. It is possible that analysis of the adhesive materials requires a spectrometer offering a broader wavelength.

The obtained in the following time steps reflectance spectra of selected waxes and adhesives showed that the spectra course and their change over time are unique for each material. It can be used for determining relationships between specific spectra and the progress of the solidification process of the particular material. However, the measurement of adhesives needs to use a spectrometer offering a wider wavelength than used in research. It results from the observations that it is tough to differentiate the lines corresponding to the successive time steps. Moreover, the lines are overlapped and their order changes for various wavelength ranges.

#### *3.3. Different Variants of ANNs*

The obtained results (reflection spectra of individual materials) were uploaded to the Matlab program (Figure 4) to train the neural networks based on the obtained series of measurements, which can be used to identify the solidification stage (stage) of the material observed. However, in the first place, it was necessary to check the efficiency of the available learning methods and the correct minimum number of hidden neurons *nh* to minimise the Mean Square Error (MSE). This is essential because the level of this error tells us about the degree of network training (the ability to recognise objects with a relatively low time error).

**Figure 4.** Matlab toolbox for ANN training and verification.

Different training methods for ANNs available in Matlab were also compared to find the method which reduces the MSE error to a minimum with the lowest possible number of hidden neurons. For example, the analysis for a network containing 200 neurons using a typical PC (4 GB operating memory, 3.30 GHz processor) takes about 2 h. During the tests, eleven methods of learning the network were tested based on up to 80 measurement spectra for each of the selected test objects.

The general structure of ANNs is illustrated in Figure 5. ANNs consists of three inputs, a hidden layer including the specific number of neurons, and one output layer. As can be seen, the number of neurons in the hidden layer was changed from 10 to 200 at step 10 of the research.

**Figure 5.** General structure of ANNs.

#### **4. Discussion**

In Figure 6, the ANNs verification results have been presented. The ANNs were trained by different learning methods on two measurement series for beeswax and paraffin materials. For the appropriate methods, the network in the first case achieves a satisfactory level of MSE with about 40 hidden neurons, which directly translates into a short analysis time. However, when analysing paraffin, the course of the MSE change is quite different, despite their being similar materials. A satisfactorily low error rate is obtained with about 20 hidden neurons. The mathematical description of the learning methods, using abbreviated names from Matlab, is included in [24]. In Table 1, the abbreviated names are presented with the corresponding full name of the methods.

**Figure 6.** MSE for ANN's verification for different learning methods and number on neurons in the hidden layer for wax materials: (**A**)—beeswax, (**B**)—paraffin.


**Table 1.** The abbreviated and full names of the learning methods for ANNs.

The trangdm method (Gradient descent with momentum backpropagation) achieved a lower error rate with about 50 hidden neurons (Table 2). Nevertheless, the course of MSE changes as a function of the number of neurons is chaotic. The network learning method that achieves a significantly lower MSE rate for the value of around 110 hidden neurons is traingdx (gradient descent with momentum and adaptive learning rate backpropagation) method. An essential piece of information resulting from comparing these two allegedly similar materials is that, as can be observed, increasing the number of hidden neurons does not significantly improve the MSE value. It only causes a significant extension of the analysis time. In addition, it turns out that the bottom learning method optimal for a given material (measurement series of reflection spectra) will not be suitable for another measurement series of a different material, even for materials supposedly similar to each other.

**Table 2.** The obtained minimum MSE and corresponding number of neurons *nh* in the hidden layer for the selected learning methods and the wax materials.


The idea for the further development of the research is presented in Figure 7. The use of the statistical Principal Component Analysis (PCA) method is also considered, which would allow for a reduction in the statistical data set and speed up the analysis time. The idea behind the algorithm is to monitor and save the most significant variances in the data set. Elements with low or zero variance of the set are ignored [25]. However, as already mentioned, for a given measurement series of a specific material, it will be necessary to

select a particular method and determine the optimal number of hidden neurons to shorten the analysis time as much as possible.

**Figure 7.** Conception of future development of software in Matlab.

#### **5. Conclusions**

Measurements of optical properties of materials whose aggregate state is unstable can be effectively carried out using the DLP method using DPL NIR Scan equipment. To increase the certainty of analysis of rapidly changing spectra, the method of spectra analysis using neural networks was tested. Four materials were tested in the study: epoxy, cyanoacrylate glue, paraffin, and beeswax. The first two materials are adhesives, while the second two are waxes. Based on the results of the research, the following conclusions can be stated:


Thanks to the DLP NIRSCAN Nano device, it is possible to monitor the reflection spectra and thus the changes taking place in various types of materials. Analysing the spectra makes it possible to observe the course and dynamics of changes, which may be helpful information during technological processes where the given materials are used. The analysis of the spectra of objects with variable optical parameters using ANNs tells us that the described learning methods implemented in Matlab can learn entire series of measurements without significant problems and then recognise specific measures while maintaining a relatively small error (the described learning methods used about 80% of the measurement series for learning and about 20% of the series were used for network verification and MSE determination). During the analysis, it turned out that there is no universal method of training the networks, even for similar materials. The method and the

optimal number of neurons should be matched to a given material, which in the future may allow for the recognition of materials or determining their condition. The next planned stage of the study is the modification of the program. After reading the given measuring series of the spectra of a given object, it will be possible to determine the time and stage of setting of a random measured adhesive sample. Thanks to this solution, it will be possible to determine the time (solidification stage) with a small error in any technological processes where wax is used. Such information can be beneficial in solidifying glues, resins, and other materials.

**Author Contributions:** Conceptualization, M.G. and A.P.; methodology, M.G. and A.P.; software, P.S. and K.N.; validation, M.G. and P.S.; formal analysis, A.P.; investigation, M.G.; resources, M.G.; data curation, M.G.; writing—original draft preparation, M.G. and P.S.; writing—review and editing, P.S.; visualization, M.G. and P.S.; supervision, A.P.; project administration, A.P.; funding acquisition, P.S. and K.N. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** Authors thank Leszek Bychto for useful discussion and consultations.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Abbreviations**

The following abbreviations are used in this manuscript:

ANN Artificial Neural Network CSV Comma-Separated Values DAT generic DATa file DLP Digital Light Projection DMD Digital Micromirror Device EVM Evaluation Module MSE Mean Square Error NIR Near Infrared Spectroscopy PC Personal Computer USB Universal Serial Bus VS Visible

#### **References**


**Tomasz Buratowski 1, Jerzy Garus 2, Mariusz Giergiel 1,\* and Andrii Kudriashov <sup>1</sup>**


**Abstract:** Simultaneous localization and mapping (SLAM) is a dual process responsible for the ability of a robotic vehicle to build a map of its surroundings and estimate its position on that map. This paper presents the novel concept of creating a 3D map based on the adaptive Monte-Carlo location (AMCL) and the extended Kalman filter (EKF). This approach is intended for inspection or rescue operations in a closed or isolated area where there is a risk to humans. The proposed solution uses particle filters together with data from on-board sensors to estimate the local position of the robot. Its global position is determined through the Rao–Blackwellized technique. The developed system was implemented on a wheeled mobile robot equipped with a sensing system consisting of a laser scanner (LIDAR) and an inertial measurement unit (IMU), and was tested in the real conditions of an underground mine. One of the contributions of this work is to propose a low-complexity and low-cost solution to real-time 3D-map creation. The conducted experimental trials confirmed that the performance of the three-dimensional mapping was characterized by high accuracy and usefulness for recognition and inspection tasks in an unknown industrial environment.

**Keywords:** SLAM; 3D mapping; mobile robot; underground inspection

#### **1. Introduction**

Mobile robotics have seen an increase in interest in the last few decades, especially due to their ability to be deployed in a hazardous environment without endangering humans. Currently, it is common to use mobile robotic vehicles to accomplish missions such as environmental recognition, the inspection of urbanized and industrial terrains, and search and rescue operations. In the military field, they are employed in tasks such as reconnaissance missions, surveillance, intelligence gathering, hazardous-site exploration, and more. The robot can explore an area of interest, teleoperated from a remote area or autonomously. Knowing the obstacles in the vicinity of the robot is essential for the successful completion of a mission in a working area, avoiding any collisions that could make the robot unusable.

If the map is given a priori, the robot can self-localize by matching some features of the environment observed at the given moment to features of the same type existing in the known map. A feature suitable for self-localization is a static, salient object (or part of an object) in the environment, which can be described with respect to some other co-ordinate frame. On the other hand, the predefined map can be perceptually incompatible, i.e., it may not properly reflect all features in the environment perceived with the given sensing modality. Therefore, the robot should be able to build its own model of the environment.

In recent decades, many designs for mobile robotic vehicles created to explore a partially or fully unknown environment have been developed and demonstrated in both academia and industry [1–7]. The simultaneous localization and mapping (SLAM), currently one of the main research topics in robotics, appears to be the attractive solution to

**Citation:** Buratowski, T.; Garus, J.; Giergiel, M.; Kudriashov, A. Real-Time 3D Mapping in Isolated Industrial Terrain with Use of Mobile Robotic Vehicle. *Electronics* **2022**, *11*, 2086. https://doi.org/10.3390/ electronics11132086

Academic Editor: Hamid Reza Karimi

Received: 5 June 2022 Accepted: 29 June 2022 Published: 3 July 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

this problem [8–16]. The objective of SLAM is to concurrently build a map of investigated environment and allow the moving vehicle to localize itself within this environment using its on-board sensors.

At present, the most widely used SLAM frameworks are based on laser sensors, because these kinds of sensors can detect the distances of the obstacles around the robot with high accuracy. However, 2D-mapping methods based on laser scanners still have some problems. The obstacles can only be detected when they are in the same height with the sensors. Therefore, 3D-mapping methods based on multiple-laser sensors or 3D-distance sensors have been developed because these methods can reflect the shapes of an area and objects in the environment and allow the robot to plan its path with collision avoidance.

This paper focuses on a map creation for the recognition and inspection purposes of an isolated industrial environment carried out by a mobile wheeled robotic vehicle. The map is generated based on data sets obtained from a laser scanner (LIDAR) and an inertial measurement unit (IMU) installed on its board. We propose a robust real-time mapping concept based on probabilistic SLAM formulation, including both extended Kalman filters and particle filters, to build a 3D map of the surrounding area. The effectiveness of the developed solution has been confirmed by mapping experiments in the underground environment of the mine for different operational conditions.

The remainder of this paper is organized as follows. In Section 2, a brief description of the SLAM problem is provided. Details of the implementation of the proposed approach are presented in Section 3. Section 4 provides the results of the experimental studies carried out in an experimental underground mine. Finally, the conclusions and future works are summarized in Section 5.

#### **2. SLAM Mapping Overview**

Mapping approaches are generally divided into a 2D and 3D representation of the environment. Each type can be used for different tasks, e.g., navigation or workspace-state representation purposes. A planar map is good enough for navigation tasks. However, from the inspection point of view, a three-dimensional picture of the surroundings is definitely more useful, but it impacts on the calculation loading of the SLAM process and requires more advanced sensory systems.

There are many ways to describe the SLAM problem represented in the literature [17–24]. Currently, the most common way is to define SLAM as a probability density function, which can be described by the following generic form [25–27]:

$$p(X\_{0:t\prime} \, m | Z\_{1:t\prime} \, \mathcal{U}\_{1:t}) \tag{1}$$

where:

*X*0:t = {*X*0, *X*1, ..., *Xt*}—a sequence of the mobile vehicle's poses (passed path) in a sampling time interval <0, *t*>;

*U*1:t = {*U*1*U*<sup>2</sup> ... *Ut*}—a sequence of control signals;

*Z*1:t = {*Z*1*Z*<sup>2</sup> ... *Zt*}—a sequence of relative observations.

The observations *Z*1:*<sup>t</sup>* are made by sensors, which measure the distance to the closest objects in all directions from the poses *X*1:*<sup>t</sup>* at the same time instant.

Such a form of expression (1), where wanted values are reconstructed for all previous states, is called "Full SLAM". The opposite approach, where only a recent position is estimated, called "Online SLAM", can be calculated by recursive integration [23]. It is possible to apply Bayes and Markov rules to (1) and define the probability density function as a recursive process of predictions and corrections of the robot's localization in the map m, which depends on motion (kinematic constrains and controls) and observation models [19]:

$$p(X\_{0:t}, m | Z\_{1:t}, \mathsf{U}\_{1:t}) = a \cdot p(Z\_t | X\_{0:t}, m, Z\_{1:t-1}, \mathsf{U}\_{1:t} \cdot p(X\_{0:t}, m | Z\_{1:t-1}, \mathsf{U}\_{1:t}) \tag{2}$$

where *α* is a normalization constant coefficient.

The recursive SLAM definition (2) is very convenient, since estimators such as the extended Kalman filter (EKF) [12,28] and particle filters [29–31] began to be applied to solve the estimation problem. The Kalman filter has the ability to use data from different sources and then efficiently fuse them into precise estimated values. It is one of its more valuable features. On the other hand, the biggest disadvantage of the EKF is its dependency on previous states that might cause huge average error decreases in the case of an unexpected measurement disaster. The use of EKF for the SLAM problem was first proposed in the work in reference [32], after which, many researchers further improved this method [12,33,34].

The particle filter [18,35,36] is a recursive filter using a Monte-Carlo algorithm to estimate the pose and movement path of the moving object. They are good and fast in global localization but are limited in their use of data from only one domain. Their wide usage for SLAM came from the brilliant idea of the Rao–Blackwellization method, which was introduced into regular map-building algorithms [37]. Although the particle filter algorithm can be used as an effective method to solve the SLAM problem [9,13,18,21,38], the main problem is that a large number of samples are required to approximate the posterior probability density of the system. The more complex the environment that the robot meets, the more samples that are needed to describe the posterior probability, and the higher the complexity of the algorithm.

Regarding the map construction of SLAM, there are three commonly used maps [39], i.e., feature-based maps, topological maps, and grid maps. The last one, also called an occupancy map, is the most common way for robots to describe the model of the environment. It divides the workspace into a series of grids, where each cell in the grid corresponds to binary random variables that describe the occupancy probability [8,20,40–42]. Hence, the map *m* is given by a product occupancy probability of each cell mi, which is associated with a certain point in three-dimensional Cartesian space [23,34]:

$$p(m) = \prod\_i p\left(m\_{x\_i\mathcal{Y}\_i\sigma\_i}\right)$$

From the SLAM perspective, Equation (3) can be formulated as a probabilistic problem of the recursive pose estimation from given observations. There are many different SLAM techniques based on occupancy grids for map building in 2D or 3D space [10,22]. One of possible representations of the 3D map as an occupancy grid is octrees, which instead of the 2D quadtrees representation presented directly in references [8,42], uses an octree hierarchical data structure, where each volume-named node has eight child connections with inner nodes [25]. Currently, in the scientific literature, a rich number of occupancy grids-based SLAM techniques can be found [18,20,26,43]. Nevertheless, in any case, the best SLAM algorithm for a particular environment depends on hardware restrictions, the size of the map to be built, and the optimization criterion of the processing time.

The algorithms depend heavily on the sensors with which the robot perceives its surrounding. The selection and installation of sensors determine the specific form of observation results, and also affect the difficulty of SLAM problems. In case of mobile robots performing in indoor environments, exploration cameras or LIDARs are mostly used. Some approaches use a fusion of laser and visual data [34]. As mentioned before, LIDARs provide data that are more accurate, robust, less noisy, and sensitive to changes in the lighting. For these reasons, it is a very reliable data acquisition mechanism for industrial site mapping [44]. However, due to the limited number of dimensions, the laser data are not sufficient to estimate the robot's pose on uneven ground with six degrees of freedom (DOF) [45]. Therefore, most algorithms need auxiliary data from other sensors, such as an inertial measurement unit (IMU) or odometer. In recent years, along with the rapid development of artificial intelligence (AI), learning methods are also used to solve the SLAM problem [46–48].

#### **3. Hybrid Mapping Concept for Inspection Purposes**

The map-building approach applied in this paper has its basis in a robot's pose estimation, which is presented in references [42,49], called the AMCL-EKF, which connects two separate paradigms—Kalman and particle filters. The pose is estimated by the extended Kalman filter by using measurements data from LIDAR and IMU sensors and adaptive Monte-Carlo localization (AMCL).

In the case of a planar motion, the local pose is described by a state vector *X*(2) consisting of three components:

$$\mathbf{X}^{(2)} = \begin{bmatrix} \mathbf{x} & \mathbf{y} & \boldsymbol{\Psi} \end{bmatrix}^T \tag{3}$$

where *x*, *y* are Cartesian positions and *ψ* is a yaw angle.

However, from the inspection point of view, tridimensional maps showing environment features in their natural scale are more preferred. Due to the fact that 3D mapping requires information about the global location of the robotic vehicle with six degrees of freedom (DOF), it is necessary to recreate missing coordinates. It may be performed by taking the needed coordinates from the local 3D pose that the EKF performs [49,50] (see Figure 1). Then, the state vector *X*(3) can be written in the following form:

$$\mathbf{X}^{(3)} = \begin{Bmatrix} \mathbf{X}^{(2)} & z & \boldsymbol{\varrho} & \boldsymbol{\theta} \end{Bmatrix}^T \tag{4}$$

where *z* is the Cartesian position and *ϕ*, *θ* are roll and pitch angles. The global localization can be solved by a Rao–Blackwellized 2D SLAM algorithm [18].

**Figure 1.** Schematic diagram of local 3D pose estimation by AMCL-EKF algorithm.

As far as *X*(3) is delivered, it allows for the creation of a three-dimensional map. In this way, the 2D SLAM algorithm is used continuously to efficiently build the 2D map, and at the same time the 3D SLAM algorithm (the delivered precise *X*(3) and recent cloud of points data allows to create the 3D map (see Figure 2)). This task can be performed both online as a separate task or offline postprocess based on previously collected sensor data.

The efficiency and accuracy of the developed method, called "hybrid 3D SLAM", for locating the robot in unknown terrains, have been confirmed in several simulation experiments presented in our previous works [19,50,51], where the 3D-mapping algorithm was implemented in accordance with the Robot Operation System framework, and was verified with using the V-REP simulation environment. The carried-out investigations affirmed the legitimacy of the proposed approach to 3D-map building for inspection tasks.

**Figure 2.** Block diagram of hybrid 3D-map-building bases on 2D SLAM and reduced Octomap algorithms.

#### **4. Environment Description and Experimental Results**

#### *4.1. Robotic Vehicle and Environment*

The SLAM strategy presented in the previous section was implemented in the Dr Robot Jaguar 4 × 4 mobile vehicle constructed by the Canadian company Dr Robot Inc. (see Figure 3). The robot is equipped with the LIDAR (Velodyne VPL-16) and the IMU (Pololu MinIMU-9) sensors.

**Figure 3.** Mobile robotic vehicle used for tests.

The robot is a four-wheeled robotic platform designed for tough indoor and outdoor operation, and has the following operational parameters: mass 20 kg, max speed 11 km/h, and a ground clearance 88 mm. The Velodyne VLP-16 LIDAR uses an array of 16 infra-red lasers paired to detect and measure distances to objects. The array provides 300,000 data points per second in real-time. The IMU sensor Pololu MinIMU-9 is a compact board that combines a 3-axis gyroscope, a 3-axis accelerometer, and a 3-axis magnetometer for measuring robot data parameters.

Investigations were conducted in a real industrial environment—the experimental underground mine being a part of the AGH University of Science and Technology infrastructure. It was assumed that the robotic vehicle had to inspect the site in two operational cases, i.e., before and after the mining disaster. This task required the robot to move on

both smooth and rough surfaces that were sometimes blocked by degraded infrastructure. Figure 4 shows two fragments of real operational scenarios.

**Figure 4.** Underground inspection scenarios: (**a**) smooth terrain before accident; (**b**) rough terrain after accident.

#### *4.2. Experiments*

Carried out trials were divided into two stages, i.e., creating the map of the mine area for normal working conditions and after shutdown due to an accident. In accordance with the methodology described in Section 3, the 2D maps were first built using the AMCL-EKF technique, and then the 3D maps were estimated by applying the Rao–Blackwellized SLAM approach.

The results of the map creation for the terrain before accident are presented in Figure 5. Both 2D and 3D maps precisely recreate the real state of the mine's corridors with all observable main features. The 2D map (Figure 5a) clearly shows free spaces and therefore can be fully used for navigation tasks, whereas the 3D map (Figure 5b) contains all features of the passed corridors and therefore is much more useful for a precise and safe inspection of the underground environment. It is especially important when some parts of the terrain have become inaccessible.

**Figure 5.** *Cont*.

**Figure 5.** Maps obtained before accident: (**a**) created with the use of 2D SLAM technique; (**b**) created with the use of proposed hybrid 3D SLAM approach.

Such a situation is presented in Figure 6, where a photographic image of the corridor and destroyed infrastructure is shown. Due to the disaster, the main road was partially blocked, and the terrain became rough.

**Figure 6.** Photographic image of the mine corridor after accident.

The results of the map creation are depicted in Figure 7. The depicted maps show the examined area from the same perspective in which it was captured by the camera. The comparison of the scenes from the photo and the generated maps shows a high compliance of the presented mine's environment. The general differences between 2D and 3D maps, shown in Figure 7a,b, respectively, are similar to those described for the previous case. It can be seen that the 3D map is accurate enough to assess the real state of the environment.

The performed investigations showed a good compliance of the obtained 2D map with the geodetic map of the laboratory ground mine. The final product, which is a 3D-space map, properly reflects the current of the working environment.

(**b**)

**Figure 7.** Maps obtained during underground inspection in the mine terrain after accident: (**a**) created with the use of 2D SLAM technique; (**b**) created with the use of proposed hybrid 3D SLAM approach.

However, its quality and practical usability is limited by specific problems related to the limitations connected with real working conditions. Although the AMCL provides a global location based on reference points, the quality of the pose estimation process strongly depends on errors concerned with the robot's slippage or skidding. A problem with LIDAR can be long spaces with a limited number of potential reference points, such as a tunnel with single surrounding features.

#### **5. Conclusions**

The method for the concurrent construction of a 3D map of the industrial terrain realized with use of a mobile wheeled robot is presented and discussed in the paper. Since the three-dimensional map shows the complete picture of the explored terrain, it is more convenient for use in environmental inspection tasks.

The main contribution of this work is to develop a real-time method for creating a 3D map in both structured and unstructured industrial environments. We propose to solve this problem by using LIDAR to preprocess the scanning and IMU to generate odometries.

After mapping the environment, we created a grid map based on the 2D SLAM, the AMCL, and the EKF. The robot's global position was determined by means of the Rao–Blackwellized technique. This approach significantly reduced computational costs in comparison with the Full 3D SLAM.

The efficiency, accuracy, and robustness of the developed hybrid 3D SLAM approach for 3D-map creation was validated by a number of real experiments conducted in the different scenarios of the underground mine. All obtained results confirmed the correctness of the developed solution for the creation of the 3D map in the indoor environment. The findings also demonstrated that the worked-out algorithm can be run in conditions similar to real ones by using a mobile robotic vehicle equipped with modest hardware.

Further works will be devoted to examining the applicability of the developed solution to create a 3D map in other types of real environments, such as underwater or aerial.

**Author Contributions:** Conceptualization, T.B., J.G., M.G. and A.K.; methodology, M.G.; software, A.K.; validation, T.B. and A.K.; formal analysis, T.B., J.G., M.G. and A.K.; investigation, T.B., M.G. and A.K.; resources, M.G. and A.K.; data curation T.B. and A.K.; writing—original draft preparation, J.G. and M.G.; writing—review and editing, T.B., J.G., M.G. and A.K.; visualization, A.K.; supervision, J.G. and M.G.; project administration, M.G. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Data Availability Statement:** All necessary data are given in this article.

**Acknowledgments:** The authors express their gratitude to the AGH UST Center of Energy for enabling the use of scientific and laboratory infrastructure for experiments.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

