Sound Localization Framework for Construction Site Monitoring

Abstract

This study developed a construction sound localization framework (CSLF) using the time delay of the arrival technique and the generalized cross-correlation phase transform. Based on the spatial characteristics of construction sites, this study identified sound-source locations at construction sites to overcome the limitations of existing sensor-based localization methods. Verification tests were performed on the CSLF in both interior and exterior environments. The interior tests showed that the average error was less than 0.5 m for each axis; which is 0.26, 0.27, and 0.36 m for the x-, y-, and z-axes, respectively. The exterior tests conducted at an actual construction site showed average x-, y-, and z-axis errors of 0.19, 0.21, and 0.24 m, respectively. These results show that the CSLF can effectively identify sound locations at construction sites within an arm’s length. Furthermore, CSLF is applicable on the sound data categorization-related research. The system can categorize an accidental or abnormal sound and locate the sound source for safety management at the construction site. It can also be utilized to monitor construction productivity by categorizing and locating the machinery-related sound sources.

1. Introduction

The distances, topographies, and obstacles at construction sites cause restrictions in management and supervision, which could result in minor and even fatal accidents. Therefore, safety blind spots at construction sites have motivated researchers to investigate safety-monitoring technologies. In addition, securing real-time monitoring and decision-making support technology in large-scale construction sites is crucial for achieving proactive and effective construction and process management.

Various innovative technologies and measures are being applied at construction sites, considering that location information collection is essential for real-time monitoring. Various sensor-based technologies, such as radio frequency identification (RFID), Bluetooth, and inertial measurement units (IMUs) already exist. RFID technology can identify location information by reading tags associated with resources, such as for labor access control [1], material logistics [2], and equipment control [3]. Bluetooth technology uses a beacon device to identify location information based on the strength of the signal. For example, beacon devices are attached to heavy equipment at construction sites to monitor the location and condition of the equipment to increase the safety and work efficiency of workers at construction sites [4]. Finally, IMU sensors are attached to objects such as construction equipment to indicate movement and movement paths with gyroscopes to prevent collisions and improve their mobility [5]. While the above-mentioned sensor-based systems are used to collect the location information of major materials that need to be managed, they can also analyze the status and activity data of specific equipment and construction workers. However, they cannot monitor the work conditions or safety.

Unlike sensor- and tag-based monitoring systems, construction sites are also monitored using signals from tactile data (e.g., temperature and material characteristics), olfactory data, and sound data generated during equipment operation and construction work on the site, as well as visual data of recorded photos and videos. Among them, olfactory and tactile data, which can check for harmful gasses or temperatures, are used for monitoring the overall atmospheric and spatial information of the space instead of determining the situation during actual work. For example, these data can identify harmful gas and oxygen concentrations underground in real time [6].

Visual data are most widely used along with a location estimation system as it is the most intuitive means of determining the worker’s risk situation, construction progress, safety, and the environment via closed-circuit television (CCTV) at construction sites [7,8]. However, visual data have limitations at construction sites [9], such as the installation location and scope of the camera, speed of specification and data transmission, accuracy of the expected location information, and influence of external environmental factors.

This study focuses on sound data identification at construction sites to potentially improve risk management, either independently or combined with other technologies at construction sites (as mentioned above), even though sound-data-based position estimation systems that analyze the acoustic signals collected by microphone arrays have been actively used in other fields, such as automobiles, maritime navigation, robots, and military applications. As sound data are also considered management targets rather than noise at construction sites, studies on data pattern analysis have recently been conducted to classify equipment sounds. While most previous studies on sound data in the construction sector are focused on sound pattern analysis, studies on estimating sound-source locations are insufficient. Research on estimating sound-source locations at construction sites is essential for applying monitoring systems based on sound data. Therefore, in this study, a framework for estimating sound-source locations at construction sites was developed and was applied to an actual site to derive test results.

This study proposes a method that uses sound data to overcome the limitations of sensor- and visual-data-based technologies. Sound data complements the blind spots of a camera, and unlike high-resolution image data, the size of the data is only one-thousandth of the video data, thus, enabling the real-time data transmission at construction sites, even at low transmission speeds. Therefore, our aim is to establish a framework that serves as the basis of a system that can react to disasters proactively by identifying unstable conditions and behaviors.

The objective of this study is to develop a sound localization framework for an effective monitoring method that can be applied for labor and safety management at construction sites. We believe this system can contribute to the identification of abnormal audio signals as well as the work status, progress, and machine productivity.

2. Research Methods

The development of the sound-based localization framework suitable for construction sites involved the following steps (Figure 1): First, we examined the existing literature and case studies, and identified the limitations and features of sensor-based monitoring methods currently used at construction sites. We also analyzed sound-based localization methods used in several industrial sectors.

Second, we analyzed the spatial characteristics of construction sites to determine the factors that should be considered for developing a monitoring system. These factors were then used to perform a comparative analysis of sound-based localization approaches for onsite use and to develop an integrated localization framework.

Furthermore, pilot tests were performed in a laboratory to verify the developed framework using various microphone arrangements in 2D and 3D environments. The tests were performed at an actual construction site to confirm the usability of the framework.

Finally, we analyzed the experimental results and evaluated the limitations identified during the framework development process, and identified opportunities for future system development and research directions.

3. Literature Review

3.1. Monitoring Approaches Suitable for Construction Sites

3.1.1. Active Localization Approaches

RFID is one of the most widely used localization methods. RFID-based localization systems are highly accessible as they are easy to process RFID data at low costs [10]. Furthermore, they are used in closed environments for material status updates and supply network management [2,11]. However, RFID tags are more suitable for updating data and have limited use at construction sites for localization [12,13].

Bluetooth technology involves low-bandwidth wireless frequency signals used for communication between devices. Beacon-based localization systems, where a beacon is a mobile tag that transmits Bluetooth signals, estimate locations by measuring distances based on the signal strength recorded by the receiver [14]. While these systems have a wide application range and are easy to install, tags can be easily lost or damaged at construction sites, while systems can face problems such as weak Bluetooth signals and repetitive receiver installations [15].

Furthermore, there are localization methods that use GPS and IMU sensors. GPS-based localization methods calculate the location of mobile devices by using data connections with satellites or base stations. GPS provides state-of-the-art localization when used outdoors, especially when utilizing unencrypted military satellite signals. However, GPS is difficult to use indoors [16]. IMU sensors measure the movement of an internal gyroscope to calculate the movement path of an object and measure the distances. Furthermore, they can operate without an external connection. However, the movement paths are calculated relative to the starting point, and acceleration errors quickly accumulate into larger position errors if no additional correction is applied [17].

Despite the high accuracy and easy accessibility of sensor-based localization methods, they have several limitations when applied to construction sites. Sensors attached to materials and equipment can be lost or damaged easily during construction. Furthermore, considering that several tags are used at a construction site, the number of added receivers is significantly large, which increases costs and causes management difficulties.

3.1.2. Passive Localization Approaches

Currently, passive localization approaches that use vision data are widely used on construction sites. Vision-based monitoring algorithms have been developed to classify factors such as workers, equipment, and damaged parts. The locations of objects and actions are determined through the vector analysis of the reference points of the objects [18]. Topak et al. developed systems that use stereo cameras installed at construction sites to measure the changes in worker locations to perform localization. Konstantinou and Brilakis demonstrated systems that use cameras in indoor environments to accurately monitor worker locations and work status [19].

Despite substantial research and development efforts, there are several limitations regarding the use of vision data-based localization systems on construction sites, such as the positioning and monitoring range of cameras. When a large area needs to be monitored at obstruction-filled sites, several cameras need to be installed. Additionally, the quality of the vision data depends on the camera performance and resolution. Even when high-resolution cameras are used, their performance can be easily degraded by unexpected environmental factors at the construction site, such as dust and foreign substances. In addition, it can be used as a supplementary measure to solve problems regarding continuously monitoring the work status due to the privacy problem.

3.2. Sound-Based Localization and Applications

Localization methods that use sound data are widely used in several industrial sectors. The automotive sector uses localization methods to analyze and estimate the location of interior and exterior sounds to actively respond to changes in vehicle interiors, engines, and exteriors to manage car maintenance [20,21].

The military sector uses localization methods to analyze sounds recorded in open spaces. Researchers have developed algorithms for the armed forces to analyze shooting sounds to verify the type of ammunition and estimate the positions of the shooters by measuring the arrival time and angle [22].

Sound-based localization systems have been developed for indoor service robots, considering robots that operate indoors must accurately calculate their targets and destinations to provide services. Therefore, sound-based localization systems have been developed to supplement vision-based localization systems and improve robot navigation performance [23].

In construction, sound-based localization systems are mostly used for the nondestructive inspection of structures. The status of bridge structures can be monitored by performing localization on degraded or damaged wires using sound data generated by pipes at gas facilities to locate leaks and monitor the status of such facilities [6,24,25]. However, because they involve the analysis of a single structural component, these acoustic-emission-based monitoring methods are not suitable for monitoring workspaces at construction sites.

Additionally, algorithms are being developed to classify equipment sounds to monitor the status of civil engineering sites [26,27,28]. Equipment sounds are no longer considered noise that should block sound data in the construction field, but instead are necessary data for management. Therefore, this study aimed to develop a system that uses sound data to monitor the work status or detect abnormal signals related to accidents that could occur at construction sites. Compared to sensor-based monitoring systems, sound-based site monitoring systems are easy to install and recover and can monitor a wider area compared to camera-based monitoring systems, while providing 24 h monitoring. Their usefulness at construction worksites could be very high and could enable more active and trouble-free site supervision by identifying the locations of sound sources in real time.

4. Construction Sound Localization Framework (CSLF)

New workspaces are constantly created at construction sites, and thus, are necessary to cope with various situations. Therefore, Internet of Things (“IoT”) devices, which are easy to install and dismantle without needing a separate power supply, are required at construction sites. Moreover, due to high damage risks in the field, devices with low unit costs will be suitable in the consideration of economic efficiency. Under these conditions at construction sites, a system was designed considering usability and economic efficiency in this study, and a more efficient method was presented through preliminary and field tests.

When developing a framework to estimate the location of a sound source, it is necessary to analyze the characteristics of the space. Existing studies monitored the characteristics of sound sources while the research method and scope were determined based on whether the space was linear, indoor, or external. Therefore, developing a CSLF system to estimate the location of sound sources generated within a construction site should be preceded by a spatial characteristics analysis of the construction site. Furthermore, the characteristics of the device setting and layout (e.g., position, distance, direction, and number of microphones) should be considered for accurate location estimation.

4.1. Analysis of Spatial Characteristics of Construction Sites

Building spaces can be classified into 65 types depending on the construction type, opening type, functions, environment, and structures (MOILT 2002). These spaces can be further reorganized into vertical exterior, horizontal exterior, open interior, and closed interior, depending on the location of the work performed at the construction site (Figure 2).

Work is performed in several interior and exterior areas at construction sites. In exterior areas, work is performed in horizontal exterior areas, such as a building lot or at the top of a building structure. Furthermore, work is performed in vertical exterior areas while climbing a building structure. Therefore, it is necessary to design an algorithm that can perform localization in both 2D and 3D.

Open interior areas include large open spaces such as main lobbies, pilotis, and open halls. Although sounds in open interior areas can spread through air, reverberations can also reflect off the structure. Closed interior areas include classrooms and office rooms, where sound reflections and reverberations occur easily. These sounds can affect the localization results considerably. Therefore, the algorithm must be robust against reverberations and sound reflections to localize sound sources at a construction site.

Various situations arise at construction sites, with new spaces continually created as the workspaces, work types, and personnel, and the work timings change until the work is completed. Owing to the variable and fluid nature of construction sites, moving and installing monitoring equipment at a construction site must be easy to maximize system effectiveness. Therefore, it is necessary to configure an efficient system with less computing power.

4.2. Key Components of the Sound-Based Localization Approach

4.2.1. Sound-Based Localization Approach

Sounds are waveform signals that propagate through the vibration of a medium, and there are various approaches for signal-based localization. Considering these approaches, we compared the pros and cons of existing localization methods and selected a sound-based localization technique suitable for construction sites.

The time of arrival (TOA) localization technique uses a triangulation method to measure the absolute transmission time from the sound source to the sound receiver to determine the distance measured by three or more sound receivers [29]. Considering that it is necessary to synchronize the monitored target with the receiver, the target must be clear, and separate tags must be attached to the target. However, the goal of this study is to localize and identify multiple sound sources that naturally occur at a construction site. Therefore, the TOA technique, which requires the monitored target to be clear, is not suitable.

The angle of arrival (AOA) localization technique calculates the location of the sound signal using the AOA calculated by each receiver [30]. While the AOA method can perform sound localization using only two sound receivers, the location of the sound is presented only as an AOA for each receiver, and calculations are performed using the triangulation method. Therefore, small differences in the estimated angle can significantly affect the accuracy. If the distance between the sound source and the sound receiver is too large or small, localization errors can become significantly large. Considering that construction sites require the monitoring of large spaces and sounds to occur in unexpected places, the AOA technique is not suitable for construction sites.

The time delay of arrival (TDOA) localization technique, a beamforming monitoring method, performs sound localization based on the differences in the arrival times between sounds received by multiple receivers [31]. This technique is suitable for interior and exterior environments as it does not measure the absolute transmission time of the sound, but only measures the difference in the receiving times at each sound receiver. Therefore, there is no need for synchronization with the target, and the technique is easy to implement. Considering that unspecified noises occur from unexpected places at construction sites, the TDOA localization method is considered to be the most effective for this study.

Moreover, the TDOA localization method [20,31,32,33] can calculate the relative distance from the sound-source location to the receiver if the recorded sound data are captured by microphones (Sounds 1, 2, and 3) based on the coordinates of the installed receivers (M1, M2, and M3). The differences in the calculated arrival times between the microphones are used to iteratively calculate the approximate location of the recorded sound.

4.2.2. Time Delay Calculation Approach

To perform sound localization using the TDOA localization technique, the arrival time delay of each receiver is calculated by comparing the recorded sound spectrum of a certain receiver to that of a different receiver [30]. In TDOA, the signal generated by the sound source reaches each microphone with different phase information, and this phase information is converted to time information to figure out the TDOA. Generalized Cross Correlation-Phase Transform (GCC-PHAT) in the frequency domain is a method of figuring out the TDOA. GCC-PHAT whitens the incoming signal to remove the frequency’s sound pressure and uses the phase delay to estimate the position. GCC-PHAT is mainly used in specific environments for reducing the echo and lowering the noises [20,31,32,33].

The GCC-PHAT-based TDOA uses a correlation in a mathematical theoretical approach. The correlation indicates the similarity between the signals and the correlation function is divided into auto-correlation function and cross-correlation function. The auto-correlation function shows the correlation with the time-shifted or frequency-shifted version of itself. On the other hand, the cross-correlation function shows the similarities between the different signals [20,31,32,33]. The cross-correlation in Equation (1) is obtained by setting the signal from the sound source to the first microphone as $x_1\left(t\right)$, and the signal from the sound source to the second microphone as $x_2\left(t\right)$.

$$R_{x_1{x}_2}\left(\tau \right)={{\displaystyle \iint }}_{\infty }^{\infty }{x}_1\left(t\right)x_2\left(t-\tau \right)dt$$

(1)

In $R_{x_1{x}_2}\left(\tau \right)$ calculated from Equation (1), the $\tau$–value showing the biggest value is the TDOA between the microphones. Thus, the TODA is expressed as Equation (2).

$$\widehat{\tau }=argmax{R}_{x_1{x}_2}\left(\tau \right)$$

(2)

While the cross-correlation function can be used for the calculation in the time scale, this slows down the processing speed because the cross-correlation’s calculation amount exponentially increases depending on the number of samples. As a result, it is hard to apply the cross-correlation function on occasions that require real-time processing. Thus, the GCC method with a calculation amount that is relatively less is adopted. Equation (3) shows the Fourier Transform (FT) of Equation (1).

$$R_{x_1{x}_2}\left(\tau \right)=\frac{1}{2\pi }{{\displaystyle \iint }}_{\infty }^{\infty }W\left(\omega \right)X_1\left(\omega \right)X_2{}^{\ast }\left(\omega \right)e^{iw\tau }d\omega$$

(3)

When the frequency weighting function is $W\left(\omega \right)=1$, the according formula is set to GCC, and this weighting function is adjusted appropriately to reduce the noise and echo. One of the weighting functions that is mainly used is the PHAT-based weighting function which whitens and removes the sound pressure level in the frequency domain.

$$W\left(\omega \right)=\frac{1}{\left|X_1\left(\omega \right)X_2{}^{\ast }\left(\omega \right)\right|}$$

(4)

When the weighting function is given as above, the according formula is set to GCC-PHAT. The delayed time can be figured out like Equation (2) by using the $\tau$-value for the maximum value of $R_{x_1{x}_2}\left(\tau \right)$ in calculating the delayed time from the sound source to the two microphones [20,31,32,33].

Furthermore, generalized cross-correlation analysis minimizes the effect of noise in the sound data, considering that it is simpler and requires fewer computing resources compared to conventional cross-correlation methods. Therefore, it is suitable for use on mobile phones and IoT devices.

4.2.3. Location Estimation Approach

The location estimation approach is an iterative process that uses the coordinates and relative distances of microphones in an array to calculate the approximate locations of detected sound sources.

The brute force method is the simplest form of the location estimation approach, with all variables entered manually to calculate the location of the sound source. Furthermore, there is a method that uses both the TDOA and brute force methods to estimate the location of sound sources. However, this method requires significant preprocessing and computing capacity, considering that information must be generated on every possible variable. Therefore, this method is only suitable for permanently installed systems at specific locations.

Reichenbach et al. and Tian consider sound-source reverberations in line-of-sight and non-line-of-sight environments [34,35] in terms of linear and nonlinear least-square calculation methods (LSMs). In a line-of-sight environment, the sound source is within the direct view of the sound receiver, whereas in a non-line-of-sight environment, the path that the sound signal travels to reach the sound receiver is blocked by certain objects. However, because these methods must consider background noise discrimination and reverberations, high-order equations need to be solved, requiring a high computation capacity and computing power.

This study selected a calculation method based on the linear LSM which does not consider reverberations and reflected noise, and hence, has a low computational capacity. LSM is an iterative formula that determines the approximate solution of an equation to find the squares of the error between the actual and calculated location values based on the initially expected location value. Then, it determines the location estimation value that minimizes the sum of the squares. There are three factors that must be defined beforehand: the initial estimation value, the maximum number of repetitions, and the allowable error value. LSM calculates the first estimated location $S_n$ based on the predefined initial estimated coordinates $S_0$. If the squared difference between $S_n$ and $S_0$ is greater than the predefined allowable error, the LSM will calculate the new estimated location based on Ln (previously estimated location), i.e., $S_{n+1}$. This process is repeated until the squared difference between the current and previously estimated value is within the error limit or until the predefined maximum repetition count is achieved (Figure 3).

4.3. Construction Sound Localization Framework (CSLF)

Herein, we discuss the overall process involved in the CSLF. The TDOA localization technique is used to measure the relative coordinates of the microphones, whereas the GCC-PHAT is used to calculate the relative arrival time delay of the standard microphone and other microphones. LSM is used to estimate the location of the sound signal by calculating the approximation repeatedly based on the data produced by TDOA and GCC-PHAT (Figure 3).

Owing to the nature of the construction site, there are restrictions on wireless communications and a high risk of physical damage. Therefore, we determined that the hardware and components should be inexpensive and appropriate for use on actual construction sites. Therefore, we employed a least-squares method for the estimated position error correction method, considering it does not require high computational power [36]. The algorithm enables the easy computation of approximate locations in the relevant space by analyzing the spatial characteristics of the construction site.

The test system for verifying the framework was developed as follows: RaspberryPi3b+ was used as the main development board. A wireless router, network connection hub, and an omnidirectional microphone (miniDSPssi-1) were installed for recording in all directions. Four client Pi and one server Pi were created. The server Pi transmitted commands to the client Pi and analyzed the data gathered from the client Pi to calculate the sound location values.

The specific framework comprises six parts: config.python, thread.python. client.python, record.python, server. python, and analyze.python, as shown in Figure 4. config.python stores constant values required for additional estimate calculations, including the IP address of the server Pi used in the array thread connection, sampling information for record.python, task options, iteration and error limits, and the relative coordinates of the microphones for analyze.python. client.python transmits the connected socket data to the server Pi when the client Pi is connected to the system. If a request is received, the recorded sound data are transmitted along with config.python.

The Pyaudio function is used by record.python to record ambient audio in an array format. server.python performs thread connections and synchronizes the local time of the client Pi. The data received from the client Pi are analyzed, and the estimates are calculated. The analyze.python method calculates the arrival time difference based on the data recorded by the client Pi using the GCC PHAT method. Then, LSM is used to return the estimated sound-source location values.

5. Experiments

Before verifying the system’s performance on an actual construction site, preliminary tests were conducted in a restricted experimental environment by using several microphones to examine the relationship between the number of microphones included in the array and the estimation accuracy. Furthermore, tests were performed with varying microphone arrangements to determine the most suitable arrangement for obtaining optimal performances in both 2D and 3D spaces. In this study, we conducted experiments with three or more microphones as the two-microphone arrangement exhibited a significantly larger error range in location estimation and was limited in determining a particular position in a 3D space [37].

5.1. Microphone Array Shape for 2D Localization

Two array shapes—a triangular array with three microphones and a square array with four microphones—were tested for estimating sound-source locations in a 2D space [32,38]. In the square array, the data transmission increased when a microphone was added. However, the size of the dead zone, which refers to the space not included in the internal space of the microphone array, decreased, enabling the system to monitor a larger area.

The triangular array was designed as an equilateral triangle to minimize errors that could occur because of the differences in the distance between microphones. The microphones were installed at points (0, 0), (2.70, 0.00), and (1.35, 2.624), as shown in Figure 5. Sound-source locations were estimated 20 times each at randomly selected points within the internal space of the microphone array.

In the square array, microphones were installed at points (0, 0), (0, 4.2), (5.4, 4.2), and (5.4, 0), which corresponded to the maximum size of the laboratory environment, as shown in Figure 6. The tests were performed within the internal space of the array.

Considering that the developed framework presents its results as x, y coordinate values, the validity of the system was verified depending on how close the estimates were to the sound source by using the average estimation error in the x and y directions and the distribution chart. In the triangular array, the average error was 0.10 m and 0.33 m in the x and y directions, respectively (

Table 1. 2D localization results with a triangular array.

Source	Test Run	Avg δx (m)	Avg δy (m)
(0.9, 0.9)	20	0.06	0.26
(2.25, 1.35)	20	0.16	0.28
(1.35, 0.00)	20	0.08	0.45
Average		0.10	0.33

). Although there was a larger error in the y-direction, a similar distribution point cloud was formed.

However, in the square array, the average error value was 0.56 m and 0.34 m in the x and y directions, respectively, and the distribution of the estimation results was wider (

Table 2. 2D localization results with a rectangular array.

Source	x (m)	y (m)	δx (m)	δy (m)	δD (m)
Test	1.2	3.0
1	1.08	3.26	0.12	0.26	0.20
2	0.99	3.38	0.21	0.38	0.29
3	0.96	3.41	0.24	0.41	0.31
4	1.95	3.30	0.75	0.30	0.60
5	1.98	2.91	0.78	0.09	0.29
6	1.87	2.85	0.67	0.15	0.17
7	1.42	3.08	0.22	0.08	0.16
8	−0.22	3.24	1.42	0.24	0.02
9	1.55	3.38	0.35	0.38	0.49
10	1.47	3.68	0.27	0.68	0.73
11	0.84	2.20	0.36	0.80	0.87
12	−0.14	3.33	1.34	0.33	0.10
Average	1.15	3.17	0.56	0.34	0.35

). Compared to the square array results, the triangular array results had a more clustered distribution [37]. Therefore, the triangular array, which has uniform distances between the microphones, is more suitable for localization.

5.2. Microphone Array Shape for 3D Localization

At least four microphones were needed to localize sound sources in a 3D space. The triangular microphone array for the pilot localization test in 2D was used in two forms for 3D localization tests: a right-angled pyramidal array and a tetrahedral array [39]. The right-angled pyramidal array is the most common array configuration for localizing 3D sound sources (Figure 7) [40]. However, considering that the distance between the microphones is not uniform, the points are mainly in the space inside the pyramid, while errors occur in the space outside the array [40].

A tetrahedral array refers to a pyramidal volume surrounded by four triangular faces, where each triangular face of the tetrahedron is the same (Figure 8). The tetrahedral array overcomes the monitoring directionality shortcomings of the right-angled pyramidal array, as all directions can be monitored from the center point of the base triangle.

In the right-angled pyramidal array tests, microphone 1 was used as the relative origin point, whereas microphones 2, 3, and 4 were placed at positions (0.00, 0.00, 0.00), (6.3, 0.00, 0.00), (0.00, 11.25, 0.00), and (−0.60, 0.00, 4.00), respectively (

Table 3. Preliminary test results with the right-angled pyramidal arrangement.

	x (m)	y (m)	z (m)	δx (m)	δy (m)	δz (m)	δD (m)
Source	4.05	7.65	1.15
1	3.39	8.33	1.64	0.66	0.68	0.49	0.41
2	3.66	8.42	0.24	0.39	0.77	0.91	0.45
3	3.67	8.36	0.00	0.38	0.71	1.15	0.40
4	4.15	7.96	1.59	0.10	0.31	0.44	0.39
5	3.94	7.89	1.32	0.11	0.24	0.17	0.18
6	3.66	7.90	1.64	0.39	0.25	0.49	0.13
7	4.21	8.10	1.25	0.16	0.45	0.10	0.48
8	3.85	7.92	1.09	0.20	0.27	0.06	0.14
9	4.07	8.15	1.17	0.02	0.50	0.02	0.45
10	3.55	6.76	2.29	0.50	0.89	1.14	0.75
Average				0.29	0.51	0.50	0.38

). Then, certain recorded sounds were played at (4.05, 7.65, 1.15).

In the tetrahedral array tests, microphone 1 was used as the relative origin point (0.00, 0.00, 0.00), owing to the limited height of the laboratory, and microphones 2, 3, and 4 were placed at positions (5.40, 6.30, 0.00), (5.40, 6.30, 0.00), and (5.40, 3.125, 3.00), respectively (

Table 4. Preliminary test results with the tetrahedral arrangement.

	x (m)	y (m)	z (m)	δx (m)	δy (m)	δz (m)	δD (m)
Source	2.25	4.50	1.40
1	2.58	4.48	1.20	0.33	0.02	0.20	0.09
2	2.70	4.40	1.34	0.45	0.10	0.06	0.11
3	2.34	4.22	1.93	0.09	0.28	0.53	0.03
4	2.80	4.35	1.02	0.55	0.15	0.38	0.05
5	2.47	4.01	0.76	0.22	0.49	0.64	0.45
6	2.21	5.13	1.69	0.04	0.63	0.29	0.61
7	3.12	4.47	1.20	0.87	0.03	0.20	0.36
8	2.57	4.36	1.05	0.32	0.14	0.35	0.05
9	2.09	4.15	1.27	0.16	0.35	0.13	0.40
10	2.05	4.06	1.56	0.20	0.44	0.16	0.41
Average				0.32	0.26	0.29	0.26

).

The arrangement of each microphone was tested 10 times using sample sounds from a specific test point (x, y, z). During each test, the system performed an approximate localization of the recorded sound. The differences in the x, y, and z distances and the absolute distance from the sound source were calculated. The experimental results obtained using the tetrahedral array showed the average x-, y-, and z-axis errors of 0.32, 0.26, and 0.29 m, respectively, and 0.26 m in the absolute distance. This arrangement showed a relatively high accuracy compared to the right-angled pyramidal array.

6. System Validation on a Real Site

6.1. Interior Environment Tests

Interior environment tests were performed at Sungkyunkwan University in a warehouse made of brick walls and a steel roof with dimensions of 30 × 12 × 6 m (l × w × h). The collision sounds and noises of construction material were recorded and played on a Bluetooth speaker to create an environment similar to an actual site (Figure 9). We aimed to determine the location of the impact sounds that registered over a certain decibel value instead of simply classifying and assessing the characteristics of the sound sources. Therefore, it was possible to capture the noise forms of the construction sites in waveforms by situating a Bluetooth speaker in an enclosed space, and there was no difficulty in verifying the effectiveness of the proposed algorithm. There was an actual pre-test case where the experiment involved a microphone error of 0.5 m. This may depend on the location, but such an error results in a relatively higher error range and even system errors in some cases. In the sound signal, if the parameter, such as actual microphone location, slightly changes after measuring the distances between the microphones repeatedly, the measured value may turn out to be relatively higher. Thus, the experiment was conducted repeatedly without changing the parameters to verify the framework in the given experiment environment for the actual algorithm’s accuracy and system’s safety.

The first interior environment test focused on the borderline points that were relatively vulnerable compared to the sound-source estimations within the array space. The angles formed by the microphones are arranged on the floor level. The origin point was 120°, and the angles between the pairs of microphones were divided by 20° to create 15 test points (yellow color points in Figure 10). The test point coordinates are shown in Figure 10. A total of 1141 tests were performed (over 75 tests at each point).

The second interior environment test verified the maximum monitoring range of the proposed system. Considering that construction sites have a large range and are complex, it is necessary for the system to be accurate, monitor a large area, and immediately report its findings to make on-site installation easy. Therefore, in the second test, the test points were moved outside the monitoring range within the array to test the maximum detection range of the proposed system (Figure 11).

6.2. Exterior Environment Tests

Exterior environment tests were performed at a reinforced concrete apartment construction site. With the foundation placement completed, a tower crane installation point was set as the center, and microphones were installed.

In the exterior tests, the maximum test radius was set at 10 m—the same as that used in the interior environment tests (Figure 12). The devices were connected to each other through LAN cables, and the maximum test radius was set as the maximum length of the LAN cable (i.e., 10 m). If the length of the LAN cable in the exterior environment tests is changed, the data transmission speed will vary, thereby affecting the localization values. Therefore, the length was limited to 10 m.

7. Results and Discussion

To verify the CSLF, we determined the ratio of the error values over the course of the test repetitions (Figure 13). The average recall rate in these tests was 83.2% (

Table 5. Results of the interior construction site validation test 1.

Sound-Source Point	Test Repetitions	Recall (%)	X Error Mean (m)	Y Error Mean (m)	Z Error Mean (m)	Point-Cloud Distribution Rate
						Less than 10% (%)	Less than 5% (%)
1	84	88	0.24	0.42	0.32	69.05	45.24
2	79	94	0.09	0.40	0.35	75.95	44.30
3	72	86	0.09	0.44	0.33	70.83	41.67
4	109	89	0.13	0.43	0.39	77.06	47.71
5	64	78	0.19	0.41	0.40	71.88	46.88
6	80	92	0.35	0.11	0.31	83.33	50.00
7	76	63	0.32	0.14	0.35	83.78	48.65
8	72	68	0.35	0.18	0.33	76.39	43.06
9	85	70	0.30	0.22	0.41	87.95	51.81
10	60	85	0.23	0.27	0.38	78.33	58.33
11	73	84	0.35	0.16	0.33	82.19	50.68
12	79	85	0.34	0.11	0.38	82.28	46.84
13	67	95	0.35	0.19	0.35	79.10	47.76
14	79	90	0.34	0.25	0.34	82.28	46.84
15	62	81	0.23	0.28	0.36	80.65	59.68
Average for 1141 repetitions		83.20	0.26	0.27	0.36	69.05	45.24

). “Recall” refers to the proportion of the received experimental values in the entire test results. The main cause for not receiving the result values was the excess load owing to Pi power supply problems and reduced data transmission speeds caused by the wired connections.

The validity verification test 1 results showed that the average differences along the x-, y-, and z-axes were the average differences between the estimated location coordinates and the actual sound-source locations (Table 5). The x-, y-, and z-axes differences were 0.09–0.35, 0.11–0.44, and 0.32–0.41 m with averages of 0.26, 0.27, and 0.36, respectively. The average difference along all axes was less than 0.50 m. Considering the task ranges of construction site workers and the size of the space, this can be considered as an allowable error since it is within the range of an arm of a human being, and hence, people can easily locate the sound source.

Additionally, we analyzed the level of distribution of the estimated coordinates from the actual location of the sound source to examine the precision of the test results. A reference distance was used, and the distribution levels for 10% and 5% of the reference distance were calculated (Figure 14). Within 5% and 10% of the reference distance, the distribution was 41.67–59.68% and 69.05–87.95%, with an average of 45.24% and 69.05, respectively (Table 5).

In validation test 2, the result recall rate dropped as one moved farther away from the circle, and the error was greater along the y-axis than on the x-axis.

When the tests were conducted from points 1 to 3, the recall rate was low, and the sound-source locations could not be estimated accurately (

Table 6. Results of the interior validation test 2.

Sound-Source Point	Test Repetitions	X Error Mean (m)	Y Error Mean (m)	Z Error Mean (m)	Average δD (m)
Point 1	18	0.39	1.35	0.39	1.27
Point 2	12	0.37	1.09	0.56	1.08
Point 3	6	0.80	2.82	1.17	2.48

). However, the estimated coordinate values formed a detection boundary at (7.00, −4.50, 1.35), as shown in Figure 15. The distance from the array center to the detection boundary was 8.17 m, 1.38 times the radius (5.88 m) of the monitoring circle created by the microphone array. This test was intended to verify the monitoring range, with the experimental radius set to 5.88 m for indoor space sizes and a stable driving system. The detection radius may be wider depending on the location being estimated (even within a 10 m radius in outdoor environments), and a certain space could be estimated outside the detection radius. Therefore, this technique can also be used to consider installation intervals at construction sites.

Verification test 3 provided the test results for the actual construction site (

Table 7. Results of the exterior validation test 3.

Sound-Source Point	Test Repetitions	X Error Mean (m)	Y Error Mean (m)	Z Error Mean (m)
Point 1	75	0.24	0.25	0.20
Point 2	75	0.15	0.21	0.25
Point 3	75	0.19	0.19	0.27
Average for 225 repetitions		0.19	0.21	0.24

). Compared to the interior environment test, which had a monitoring radius of approximately 5.8 m, the task radius was expanded to 10 m. Nonetheless, the average difference for each radius was lower than that in the interior test, thereby indicating that the effect of reverberation and reflected sounds was smaller in the exterior environment. As a result, the exterior validation test showed more accurate localization values. Additionally, these results were obtained with the consideration that the framework performs localization using the arrival delay time, which is a very small value, and the difference between arrival delay times becomes clearer as the distance between microphones increases.

According to the overall test results, approximately 16.8% of the data were not received (Table 5), with the majority of errors related to hardware overload. Over a few hundred instances of experimentation resulted in errors in the receiving data owing to overload and unstable power supply issues with the hardware equipment. Upgrading the hardware specifications and supporting a stable power supply can presumably solve this issue. Additionally, the framework in this study comprised hardwired connections; therefore, the longer the LAN and hardwired lengths are when calculating the dt values (arrival time delays), the more likely a data delay will occur, possibly resulting in errors in the estimated sound-source location values. Therefore, wireless connectivity and data transmission can significantly reduce the estimated error values of such sound-source locations, while increasing the installation radius can maximize the on-site utilization.

8. Conclusions

This study proposed a TDOA technology-based sound localization framework called CSLF by analyzing the spatial characteristics of construction sites. CSLF calculated the arrival delay time using GCC-PHAT and repeatedly calculated the expected coordinate value of the sound source using LSM to estimate the location of sounds. Various tests were conducted in laboratories and construction sites using a development kit (Raspberry Pi) to verify the proposed framework.

In the 2D array tests, the triangular array showed better estimation results compared to the square array. Accordingly, a tetrahedral array with the minimum number of microphones (four) was used in the 3D tests. The interior effectiveness of the verification test results showed that the average error was within 0.5 m along each axis, with x-, y-, and z-axes errors of 0.26, 0.27, and 0.36 m, respectively. In the exterior tests conducted at an actual construction site, the framework showed average x-, y-, and z-axes errors of 0.19, 0.21, and 0.24 m, respectively. Compared to the outdoor environment, the indoor environment exhibited a relatively large error range owing to the influence of reverberation and the reflected sound. However, considering that both the indoor and outdoor environments demonstrated an error range of less than 0.5 m and a minimum error range of 0.36 m, the cause or source of sound generation can be easily identified at construction sites.

Previous studies that used sound data at construction sites mostly focused on classifying activities through sound pattern analysis under various scenarios. However, in this study, a sound-source localization framework was developed rather than sound data pattern analysis to contribute to developing a system that can be applied in connection with previously researched sound-source classification algorithms and classification algorithms to be researched in the future.

This framework can perform localization and site monitoring using noises (sound data) at construction sites that need immediate identification, thereby allowing site managers to perform management and supervision activities more proactively. In addition, it can be used as a supplementary measure to solve problems with environmental factors (lighting, dust, snow, and fog) and privacy, thus, addressing the limitations of existing vision data. The CSLF developed in this study will be capable of identifying the location of abnormal events with sound data at construction sites and leveraging this information to acquire rapid responses from people (workers and managers) or mechanical equipment (CCTV and robots). Additionally, the CSLF can also identify work conditions and check construction productivity through various types of sound analysis at construction sites. We believe this can contribute to the construction management field significantly.

As data transmission is made through hardwired connections in the current version of the CSLF, the detection radius is determined by the length of the LAN line. In the future, wireless data transfer-enabled frameworks will be developed by incorporating the proposed framework into IoT to expand the detection range. Additionally, more accurate location estimates would be possible by reducing errors resulting from data delays. Once a cloud computing environment is integrated with IoT devices, it can be installed into any space at the construction site, thereby allowing the monitoring of the situation inside the site in real time. In addition, the CSLF could be integrated with algorithms that use machine learning to understand sound types and determine abnormal events (sounds). For sheathing work, as an example, it will be possible to prevent large accidents, such as the collapse of sheathing and ground, by analyzing the sound of falling building materials or leaking soil at night and judging it as a sign of collapse. Such integrations would be very effective in understanding and classifying captured sound types based on the sound localization framework, thus, enabling the real-time monitoring of the work status at construction sites and abnormal audio events.