Automated Photogrammetric Tool for Landslide Recognition and Volume Calculation Using Time-Lapse Imagery

Abstract

Digital photogrammetry has attracted widespread attention in the field of geotechnical and geological surveys due to its low-cost, ease of use, and contactless mode. In this work, with the purpose of studying the progressive block surficial detachments of a landslide, we developed a monitoring system based on fixed multi-view time-lapse cameras. Thanks to a newly developed photogrammetric algorithm based on the comparison of photo sequences through a structural similarity metric and the computation of the disparity map of two convergent views, we can quickly detect the occurrence of collapse events, determine their location, and calculate the collapse volume. With the field data obtained at the Perarolo landslide site (Belluno Province, Italy), we conducted preliminary tests of the effectiveness of the algorithm and its accuracy in the volume calculation. The method of quickly and automatically obtaining the collapse information proposed in this paper can extend the potential of landslide monitoring systems based on videos or photo sequence and it will be of great significance for further research on the link between the frequency of collapse events and the driving factors.

1. Introduction

In recent decades, digital photogrammetry has emerged as a powerful and cost-effective method for the 3D reconstruction of objects and surfaces across various scientific and engineering disciplines. Its advantages, including ease of use and the ability to mount cameras on drones, have led to its expanding application in engineering geology. This technology offers an affordable and efficient survey and monitoring solution compared to other terrestrial approaches like laser scanning and radar interferometry. Most photogrammetric applications in this field focus on the morphological 3D reconstruction of slope surfaces and rock masses [1,2,3]. More recently, its use has been extended to landslide displacement monitoring [4,5,6], but, to the authors’ knowledge, there are still few applications specifically focused on sudden collapse events. In the mining sector, technologies using calibrated fixed camera systems to monitor rock faces and quantify detached block volumes through point cloud comparison have been successfully tested [7]. Despite their success, these systems are not yet fully automatable. They typically rely on volume difference calculations and are prone to false positives when determining actual block detachment. Additionally, they may not be suitable for more complex surfaces such as landslides, where ongoing movement, vegetation growth, and weather events can significantly alter the image color contents. The occurrence of sudden collapses and local surface sliding on nearly vertical slopes may represent the precursor signals of landslide reactivation but is rarely taken into account due to the difficulty of detecting and quantifying these events with currently available technologies. These events are crucial for understanding landslide behavior, assessing stability conditions, and forecasting potential evolution.

Conventional techniques, such as topographic surveys, generally fail to capture these events due to excessive deformations associated with collapses and the disruption of optical targets. Moreover, these methods often provide sparse spatial information, limited to the locations of optical prisms. Radar interferometry, while effective for detecting gradual movements, is inadequate for capturing large sudden displacements due to loss of signal coherence. Similarly, laser scanner surveys can estimate missing volumes but face challenges in long-term monitoring. In this context, time-lapse close-range photogrammetry offers a viable alternative. It allows for the prolonged observation of landslides, automatically obtaining detailed optical and spatial information from images. By comparing and analyzing image sequences from multiple cameras through appropriate image processing algorithms, local slip events and collapsed areas can be quickly identified.

The volume of collapsed material can be estimated based on the relative positions of the cameras. These data can then be used to hypothesize correlations between detachment events and driving factors such as rainfall and piezometric levels. Given the importance of automatic detection and susceptibility mapping for understanding landslide characteristics and risk assessment, numerous intelligent methods based on image analysis have been proposed [8,9,10]. Lei et al. [11] proposed an end-to-end change detection algorithm using a symmetric, fully convolutional network. Lv et al. [12] introduced a landslide detection method based on multiscale segmentation and the object-based majority voting of images. Convolutional neural network-based methods have been widely adopted in this field [10,13,14]. Additionally, Lu et al. [15] presented an object-oriented change detection method for rapid landslide mapping. While these algorithms can detect landslides over large areas using aerial or satellite images, they typically operate at low temporal resolution and often fail to detect detachments on nearly vertical slopes. In contrast, our approach focuses on terrestrial photo sequences at high spatial and medium-high temporal resolution. Terrestrial photos offer the advantage of adjustable spatial resolution (e.g., by using different lenses or changing the camera distance) and temporal resolution.

Among image comparison algorithms suitable for multitemporal sequences, simple pixel-based methods (e.g., RMSE, PSNR) are effective in controlled environments [16]. However, these methods perform poorly with natural slopes, where factors such as local erosion and variations in lighting and environmental conditions can significantly impact the images. Similarly, feature-based algorithms (e.g., SIFT [17], SURF [18,19]), which are commonly used for convergent image matching and the 3D reconstruction of landslide slopes, are also less effective for fixed-frame time-lapse imagery. While deep learning algorithms have the potential to be more robust, they are not applicable in this case due to the absence of a sufficiently large dataset of supporting images.

In this paper, we introduce a novel algorithm that automatically analyzes and processes sequences of terrestrial images from multiple cameras by searching for areas where differences are evident in terms of luminance, contrast, and texture. Utilizing the structural similarity index measure (SSIM) algorithm [20] and a series of convolution filters, our program identifies collapse events and determines the collapsed areas. The stereo configuration of the cameras enables the calculation of the 3D shape and volume of the detached mass [7,21,22]. This stereo-camera system has been prototyped and tested in the field at the Perarolo landslide site (Belluno Province, Italy), where spontaneous slope collapses frequently occur. The system, comprising three cameras and a large spotlight, was installed on the opposite side of the valley. Nearly a year of daily and nightly photo sequences were analyzed. Additionally, three large-scale slope stabilization operations using blasting and excavators at the same site were employed to evaluate the effectiveness of the collapse detection method and the accuracy of volume estimation, which was compared to laser scanner surveys.

2. Methods

2.1. Architecture of the Algorithm

Our program is designed to work with fixed multi-view cameras to perform image acquisition, collapse detection, and the calculation of collapsed areas and volumes in a fully automatic and continuous manner, without any third-party intervention or manual operation. As shown in Figure 1, the process consists of two main phases, the collapse recognition on the image plane through image comparison at different time steps, and the identification and calculation of the collapsed area and volume. Before initiating the collapse recognition phase, wooded and grassy areas, if present, must be excluded from the analysis. The color and texture of these areas can change continuously due to wind and seasonal vegetation growth, hindering accurate comparison between successive images. This operation can be performed using a deep learning mask [23]. It should be noted that this limitation naturally restricts the method’s applicability in fully vegetated areas.

The second pre-processing step involves filtering out areas covered by shadows, which could introduce collapse artifacts, particularly due to changes in the sun position over time [24]. A detailed analysis of this aspect will be provided in Section 2.3.

The core of the algorithm uses the structural similarity index measure (SSIM) to compare pairs of images in sequence, generating a structural similarity map at each time interval [20,25]. To reduce noise caused by natural small variations in color intensity, brightness, and contrast, image convolution filters were applied to the similarity map. We employed a cascade of Gaussian and median convolution filters with different patch sizes for this purpose [26]. Based on the statistics of pixel values in the final structural similarity map, potential collapse areas can be identified, and a unique structural similarity index is obtained. This index is used to determine whether a collapse occurred between the images taken at times t(i) and t(i + 1). When the structural similarity index exceeds a certain threshold, the program signals the presence of a collapse and automatically initiates the calculation of the collapsed volume using images from a second camera. This second phase of the process involves comparing two point-clouds generated from the 3D reconstruction of the slope before and after the collapse. The details of this phase will be discussed in Section 2.4.

2.2. Structural Similarity Algorithm Approach

Among image comparison algorithms, the simplest and most reliable ones for this work seem to be those that compare, even locally (i.e., pixel by pixel or patch by patch), some of the key image components such as brightness, contrast, and texture. More advanced algorithms, such as those based on feature matching and commonly used to account for possible rotations or other spatial transformations of objects or cameras (e.g., SIFT [17], SURF [18,19]), are in fact to be considered overly complex with fixed-frame images such as those in this paper. Other semantic-type algorithms that exploit convolutional neural networks are also to be considered over-refined and prone to over-fitting when changes are expected in only small portions of the images [27]. Indeed, note how the latter require a sufficiently large training set of images, whereas in this case, collapse events are generally few and site-dependent (i.e., the training set of images of one landslide cannot be easily used to refine the model of another type of landslide).

The structural similarity index measure (SSIM) algorithm is commonly used for assessing image quality and evaluating the performance of image compression algorithms and systems [28,29]. It measures the level of degradation of the image relative to a reference one, and although it is one of the most widely used metrics for defining loss functions during the training of (convolutional) neural networks [30,31], to the authors’ knowledge, it has never been used for this type of application.

In this study, we propose using SSIM as a local measure of structural changes to detect collapses in a sequence of images taken from a fixed camera oriented towards the slope. The SSIM calculates the structural similarity index (SSI) for each pixel in the image and generates a structural similarity map (SSM) by comparing a target photo with a reference photo [25,32]. The SSI is derived from the variation in the following three components between the two images or portions of images x and y: the luminance term, the contrast term, and the structural term, according to the following metric:

$$\mathrm{S}\mathrm{S}\mathrm{I}(\mathrm{x},\mathrm{y})={\left[\mathrm{l}\right(\mathrm{x},\mathrm{y}\left)\right]}^{\mathsf{\alpha }}·{\left[\mathrm{c}\right(\mathrm{x},\mathrm{y}\left)\right]}^{\mathsf{\beta }}·{\left[\mathrm{s}\right(\mathrm{x},\mathrm{y}\left)\right]}^{\mathsf{\gamma }}$$

(1)

where

$$\mathrm{l}(\mathrm{x},\mathrm{y})={\displaystyle \frac{2{\mathsf{\mu }}_{\mathrm{x}}{\mathsf{\mu }}_{\mathrm{y}}+{\mathrm{C}}_1}{{\mathsf{\mu }}_{\mathrm{x}}^2+{\mathsf{\mu }}_{\mathrm{y}}^2+{\mathrm{C}}_1}}$$

(2)

$$\mathrm{c}(\mathrm{x},\mathrm{y})={\displaystyle \frac{2{\mathsf{\sigma }}_{\mathrm{x}}{\mathsf{\sigma }}_{\mathrm{y}}+{\mathrm{C}}_2}{{\mathsf{\sigma }}_{\mathrm{x}}^2+{\mathsf{\sigma }}_{\mathrm{y}}^2+{\mathrm{C}}_2}}$$

(3)

$$\mathrm{s}(\mathrm{x},\mathrm{y})={\displaystyle \frac{{\mathsf{\sigma }}_{\mathrm{x}\mathrm{y}}+{\mathrm{C}}_3}{{\mathsf{\sigma }}_{\mathrm{x}}{\mathsf{\sigma }}_{\mathrm{y}}+{\mathrm{C}}_3}}$$

(4)

and ${\mathsf{\mu }}_{\mathrm{x}},{\mathsf{\mu }}_{\mathrm{y}}$, ${\mathsf{\sigma }}_{\mathrm{x}}{,\mathsf{\sigma }}_{\mathrm{y}}$ and ${\mathsf{\sigma }}_{\mathrm{x}\mathrm{y}}$ are the local means, standard deviations, and cross-covariances for images x and y. The parameters α, β, and γ are the exponents for the luminance, contrast, and structural terms, respectively. Their value is usually set to one [33,34], and was kept as such in this work to give equal weight to the three visual perception components. C₁, C₂, and C₃ are the small regularization constants for luminance, contrast, and texture and are used to avoid instability in image regions where the local mean or standard deviation is close to zero (i.e., denominator close to zero). In this case, their value was set to 1 × 10⁻⁴.

The structural similarity map (SSM) is a grayscale map (values ranging from 0 to 1) with the same dimensions as the target image in SSIM. It consists of the SSI values for each pixel in the target image. Figure 2 illustrates an example of an SSM applied to two consecutive images of a landslide taken at different time steps, where the darker pixels indicate the smallest similarity values, representing areas with a higher probability of erosion or collapse, while the lighter pixels indicate areas with high SSI values, signifying minimal structural and color changes. At first glance, it is evident that the vegetated parts are classified as dark areas, which could lead to erroneous collapse detection. Additionally, due to the complex and changing environmental conditions of the landslide scene, factors such as brightness variation and shadows—primarily caused by clouds, fog, and sun position—can significantly affect the structural and chromatic information of the image. This can result in artifacts in the assessment of collapse areas. Therefore, it was deemed necessary to investigate the impact of these variables on the calculation of the SSI in detail, using sets of images where no collapse events occurred.

2.2.1. Effect of Shadows

Shadow is well known as one of the main sources of error in the analysis of time-lapse aerial and satellite images, and its detection remains a challenging task [24,33,34,35]. The following two types of shadows may affect the images: cast shadows, produced by sunlight being blocked by asperities of the slope, which are regular and have a fairly constant periodicity, and shadows produced by clouds, which are generally less sharp and unpredictable. Essentially, there are property-based and model-based methods for identifying and extracting the shadows cast on the ground from digital images [34,35]. The first category of methods uses information about colors and textures in the images for shadow segmentation and can be applied to most shadows. The second category uses external models based on timing, latitude, longitude, and digital elevation models to estimate the sun’s position and forecast shadow casting. The latter approach is suitable for analyzing zenithal images and is particularly effective if elevation information can be integrated [36]. In our application, we will employ the first type of shadow detection methods.

To evaluate the influence of shadows cast by the sun on the SSI, we selected a study case involving five sunny days from 27 July 2020 to 31 July 2020, choosing consecutive hourly images taken from 11:00 a.m. to 5:00 p.m. For these five sets of images, we computed the SSI using the 3:00 p.m. shot as the reference image. In each group of images, the shadow cast on the slope gradually decreases from 11:00 a.m. to 3:00 p.m., with the images taken at 4:00 p.m. and 5:00 p.m. having almost no shadows, as shown in Figure 3.

As shown in Figure 4, the SSI is very high and changes little when the reference image is compared with images with less shadow cast (Δt = −1, +1, +2 h), while it significantly decreases and changes greatly when the reference image is compared with images with strong shadows. We also found in Figure 4 that as the shadow in the images gradually decreases (Δt = −4, −3, −2, −1 h), the difference between the SSIM results with and without the shadow filter decreases significantly. Moreover, the results of the five groups of tests maintained the same trend. The overall trend and the standard deviation shown in Figure 4 suggest that shadows have a significant impact on SSIM.

2.2.2. Effect of Illuminance

To evaluate the effect of illuminance on the structural similarity metric, we conducted five groups of image sequence comparison tests using photos with different illuminance levels. Vegetation and shadow filters were applied in these tests. Each group of tests comprised consecutive night images and day images taken within the same time interval. From 23 February 2021 to 28 February 2021, we selected four night-images (taken at 11:00 p.m., 1:00 a.m., 3:00 a.m., 5:00 a.m.) and four day-images (taken at 11:00 a.m., 1:00 p.m., 3:00 p.m., 5:00 p.m.) as test images to compare with the reference image (taken at 8:00 a.m.) in each group. Figure 5 shows an example set of images used in one of these tests. The SSI between the target image and the reference image is illustrated in Figure 6a. It can be observed that the SSI changes little when the reference image is compared with the night images (t = 11:00 p.m., 1:00 a.m., 03:00 a.m., 05:00 a.m.). This is due to the artificial illuminance provided by the spotlight, which ensures a constant level of brightness. Shadows cast are fixed and limited within the region of interest, since the illuminator is close to the cameras and directed almost along the same line-of-sight. On the other hand, the varying illuminance of the day images (t = 11:00 a.m., 13:00 a.m., 15:00 a.m., 17:00 a.m.) has a significant effect on SSI due to the substantial changes in sunlight intensity and direction, as discussed earlier. Moreover, the results of the five groups of tests consistently showed the same trend. It is worth mentioning that the reference images of Group 5 have different illuminance levels compared to the reference images of the other groups (see Figure 6b), resulting in a significantly lower SSI for Group 5 compared to the others.

2.3. Image Filters

Several filters are used in our program to enhance the accuracy of the collapse detection phase from images. These filters can be broadly categorized into the following two groups: image pre-processing filters and noise filters applied directly to the structural similarity map (SSM). The image pre-processing filters include a vegetation filter and a shadow filter. The vegetation filter removes areas covered by vegetation, which were visually identified and sampled from a set of pre-learning images. These sampled areas were used to create a deep learning mask [37]. Similarly, the shadow filter removes areas covered by shadows, which were also visually identified and sampled from a set of pre-learning images. These sampled areas were used to create a color mask. These filters are applied before comparing two images to prevent false-positive identification of collapsed areas due to movement of vegetation and changes in shadows.

The filters applied to the SSM include various image convolution filters, which smooth the noise generated by SSIM to avoid detection of small spots with low SSI and to consolidate detected collapsed areas. In this case, we used multiple median filters with kernel sizes of 23 × 23 and 7 × 7 pixels to preserve edges of the SSM [38,39,40], threshold filters [41], and Gaussian filters with a 19 × 19 kernel and standard deviation of 3 [42,43] for final smoothing. Figure 7 illustrates the specific roles of these filters in image comparison by showing the evolution of the SSM at each step.

2.4. Calculation of Collapsed Volumes

2.4.1. The 3D Surface Reconstruction and Disparity Calculation

Once a collapse is detected, the program automatically calculates the collapsed volume. The first step in this phase involves the 3D reconstruction of the slope surface before and after the collapse using fixed calibrated cameras that enable a stereoscopic view of the scene [44,45,46]. Each camera must be calibrated with a series of shots taken from different angles to calculate the intrinsic parameters, including the focal length distortion parameters as well as the extrinsic parameters, which include the position and orientation of each image [47]. The extrinsic parameters require the system to be geolocalized. This is achieved using fixed points and colored balls specially positioned on the landslide, whose exact positions are determined through a GPS survey [48]. The final image used for calibration is taken from the camera’s operational position to obtain its accurate pose.

For the reconstruction of the scene based on only two images, the principles of stereophotogrammetry were used. Given the large area to be monitored and its distance from the cameras, a larger baseline with two slightly converging lines of sight was chosen. Using the intrinsic and extrinsic parameters of the two cameras, the stereo parameters can be derived [49,50]. This includes the intrinsic parameters of the individual cameras and a roto-translation matrix that allows conversion from the reference system of the left camera to the reference system of the right camera. This matrix provides the relative position between the two cameras.

The second step is to obtain the undistorted and rectified images from the left and right cameras using the stereo camera parameters and considering an epipolar geometry [51,52,53]. To ensure the stereo images are well-aligned and the 3D reconstruction is as accurate as possible, it is necessary to correct for any image oscillations. This correction is achieved by aligning fixed points in the scene using the calibration image as a reference.

The third and final step involves calculating the disparity value of each pixel in the rectified image to obtain the disparity map, which contains the depth information of each pixel in space. Using the disparity map and stereo camera parameters [54,55,56], the 3D position of each point is calculated, resulting in the 3D point cloud.

2.4.2. Point Cloud Comparison and Volume Calculation

After the 3D reconstruction, the next step is to combine the information of the collapse area obtained with SSIM and the point cloud of the entire landslide. As described in the workflow of Figure 8, the first step involves using a median filter with a 10-by-10 kernel to remove the noise generated in the 3D reconstruction stage. To determine the collapse volume, we compute the difference between the point clouds before and after the collapse and apply a filter with a 0.5 m threshold. Subsequently, a median filter with a 21-by-21 kernel is used to remove any remaining noise. Finally, we search for continuous areas in the binary map and use the largest area as the approximate collapsed area.

Next, we dilate the boundary of the approximate collapsed area and fill the invalid points within the collapsed area on the depth information map (disparity map or Z-axis coordinate map), as illustrated in Figure 9 and Figure 10. We then compare the point clouds again to determine the exact collapsed area and its pixel coordinates. This step is crucial because invalid points in the disparity map can cause some real collapsed areas to go unrecognized; hence, identifying the lost real collapsed areas near the boundaries is essential. Figure 10 shows an example from the Perarolo landslide site.

Based on the pixel coordinates of the collapsed area on the rectified image, we can identify the points on the surface of the collapsed area. As shown in Figure 11, by using all the point clouds on the surface of the collapsed area before and after the collapse, we can create the alpha shape representing the collapsed bodies and easily estimate the collapsed volumes [57,58,59]. This approach also allows us to visualize the spatial shape of the collapsed bodies and calculate their 3D coordinates.

3. Case Study and Results

3.1. Perarolo Landslide Site

The Perarolo landslide, also known as the Sant’Andrea landslide, is located in the Cadore area of the Belluno province in northeastern Italy. This area is one of the most geohazard-prone regions in the southern Alpine mountains. The coordinates of the landslide are 46°23′44″N and 12°21′23″E close to the Alpi Carniche. The landslide is part of an older landslide on the southern flank of Mount Zucco, covering approximately 72,000 square meters and extending from an elevation of 490 to 580 m above sea level.

As shown in Figure 12, the active portion of the landslide is near the left bank of the Boite River. A village is on the right bank along the river, with its central area located downstream relative to the landslide. Additionally, about 100 m southeast from the bottom of the landslide, there is a road (SP42) that ascends the gentle slope on the southeast side of the landslide. This road features a bend adjacent to the northeast corner of the upper part of the landslide. In the 1980s, a small railway line ran along the upper section of the landslide. However, when landslide activity was detected, the railway was relocated to a tunnel inside the mountain. Furthermore, the Piave River flows southwest on the east side of the landslide body, intersecting with the Boite River approximately 150 m southeast of the landslide’s base. Behind the village, there is a hillside with a height comparable to that of the landslide, where the cameras used for visual monitoring in this study were installed. As shown in Figure 13, the camera system consists of three Canon EOS 1300D cameras (Manufacturer: Canon Inc., Taiwan. Italian distributor: Canon Italia S.p.A., Milan, Italy), each mounted in a covered iron box and equipped with remote connection and a solar charging panel. Moreover, a searchlight is also present for night photo acquisition. Near the photographic monitoring system, a terrestrial interferometric radar was later installed to measure slope displacements along the line of sight. To further enhance redundancy and, consequently, the robustness of the monitoring system, a topographic system was also present on-site, continuously collecting position measurements of reflectors specifically installed on the landslide body. Over the years, the landslide has exhibited intermittent activity, with significant accelerations that have sometimes caused collapses of the uppermost material, typically followed by periods of temporary return to an apparent stable condition [60,61,62,63].

3.2. Image-Based Collapse Detection at the Perarolo Landslide Site

To confirm the capability of our photogrammetric tool based on SSIM to detect collapses, we conducted a test using an image sequence acquired at the Perarolo landslide site from July 2020 to July 2021. One image was taken per day, except during the winter months when the slope was partially or totally covered by snow, and interferometric data and total station data were unavailable. The trend in SSI between the images on day i and i + 1 is shown in Figure 14, where sudden changes above the threshold indicate the occurrence of a collapse, in four different periods (from 8 July 2020 to 30 September 2020; from 1 October 2020 to 1 December 2020; from 15 February 2021 to 30 April 2021; from 30 April 2021 to 19 July 2021.). We defined that when the SSI is lower than 0.9998, a collapse has occurred between the two images. This threshold is calculated based on the minimum area of detached mass that we aim to detect, depending on the ground pixel size. Considering the camera resolution, the distance from the slope surface, and the mean angle between the line of sight and surface normal, an SSI lower than 0.9998 corresponds to a collapsed area larger than about 7 m². As shown in Figure 14, the program detected 13 distinct collapse events during the monitoring period. These events are consistent with on-site observations and the sudden acceleration of displacements obtained from ground-based radar interferometry and the topographic system. It should be noted, however, that these latter monitoring systems, although they can effectively measure the phases preceding the detachment, are generally unable to capture large displacements due to the loss of spatio-temporal coherence of the signal [64] or the excessive instability of the optical targets.

The first nine events correspond to small natural collapses, as shown in Figure 15b. The comparison between Figure 15a and b allows the identification of changes in the slope’s condition before and after the event of 28 February 2021. Figure 15c–e illustrate other significant collapses that occurred in the Perarolo landslide area during June–July 2021. As mentioned earlier, a major natural collapse happened on 9 June 2021; the algorithm detected this extreme condition, assigning an SSI of 0.9135 (Figure 15b). Subsequently, on 25 June 2021, an artificial collapse was induced by explosives to restore the safety condition of the slope (SSI = 0.9449, Figure 15c). Finally, in July, anthropic activities were carried out, where a bulldozer reshaped the toe of the landslide. These operations were also correctly detected by the algorithm, with SSIs of 0.9936 and 0.9941 for July 13 and 14, respectively (Figure 15d,e).

For all the cases, the program identified the location of the collapsed area on the original images, as shown in Figure 16. We can assess its accuracy by comparing the original images obtained before and after the events. Since image availability may not be ensured on some days due to bad weather conditions or other technical reasons, we tested the accuracy of our method by comparing pairs of images taken at different time intervals and computing the SSI, essentially estimating a temporal correlation function. For this purpose, we used a daily sequence of 25 images before a collapse event and 25 images after the collapse, from 6 August 2020 to 24 September 2020, to calculate the SSI between images on day i and i + Δt. As shown in Figure 16, among the possible pairs of images, they can be categorized according to the Δt, and grouped into the following two categories: pairs of images crossing the collapse event (shown in red in Figure 17 and called “with collapse”), and pairs not crossing the event (shown in black and called “without collapse”).

With boxplots using whiskers at 1.5 times the interquartile range, we found that the median of the “with collapse” group fluctuates little with the change in Δt, and there are very few outliers, as shown in Figure 18a. It was found that almost all the SSI values in the “with collapse” group were lower than the threshold value, indicating that the lack of images for several days after a collapse does not affect its detection. On the other hand, in the boxplot of image pairs “without collapse” (Figure 18b), we can see that there are cases where the first quartile (Q1) is below the threshold when Δt ≥ 14 days. When Δt = 20 days, the median is also below the threshold. This means that the time interval between the two consecutive images should be less than 14 days to ensure a 75% probability of detection. When the interval is 20 days, this probability drops to 50%.

3.3. Volume Calculation in Perarolo Landslide Site

Our photogrammetric tool automatically calculates the collapse volume for each detected event.

Table 1. Volume of some collapse events detected after 9 April 2021 calculated by our photogrammetric tool.

Date	Type of Collapse	Volume (m3)	SSI
09/04/2021	Natural	353.2	0.9971
09/06/2021	Natural	8558.7	0.9135
25/06/2021	Explosive	2739.6	0.9449
13/07/2021	Anthropic activity	217.9	0.9936
14/07/2021	Anthropic activity	457.9	0.9941

illustrates the volumes of collapse events occurring after 9 April 2021, calculated using our photogrammetric tool. To assess the accuracy of this approach, we conducted a couple of laser scanner surveys before and after each of the collapses on 9 June 2021 and 25 June 2021. From the relative point clouds, we measured the collapse volumes using CloudCompare software (version 2.11.0, https://www.cloudcompare.org) (see Figure 19), evaluating the distance in vertical direction between the slope surfaces before and after each event. We compared them with the results of our photogrammetric approach, obtaining that the relative errors were found to be 6.6% and 3.1%, respectively.

4. Discussion

The results of the case study at the landslide site demonstrate the capability of our photogrammetric tool in rapid collapse detection and volume calculation. This small-scale time-lapse photogrammetric tool allows for the quick collection of collapse information for landslides that continue to collapse, which is crucial for understanding landslide characteristics, back-analysis of collapse causes, and further research toward developing a collapse prediction tool. The fully automatic, non-contact, and low-cost features of this technology make it easily acceptable and suitable for widespread adoption.

However, the results from the tests over multiple days in Section 3.2 have shown that using the current threshold of 0.9998, comparing two images with an interval of more than 14 days may cause the program to fail. It is important to understand the maximum allowable interval between two consecutive images for reliable program operation, especially when natural environmental factors such as fog can prevent image acquisition. In this case, we found that the probability of program failure is 50% when the interval is 20 days. For other probabilities (75% or 90%), we plan to conduct tests with longer image sequences that include a collapse event in the middle, as soon as suitable data become available in the future. To further improve the accuracy of collapse detection, it is necessary to study the SSIM specifically for geotechnical materials and utilize machine learning algorithms to identify the most suitable image sequences. These efforts are currently ongoing.

5. Conclusions

In this paper, a new automatic photogrammetric tool is presented, designed for rapid collapse detection and calculation of the collapsed volume of a landslide slope. The identification of the collapsed areas is fully automated and achieved through the structural similarity comparison of consecutive image sequences. We investigated the impact of natural environmental factors at the landslide site on the classic image structural similarity algorithm (SSIM) and developed multiple filters within the tool to eliminate noise and optimize detection accuracy. After that, the algorithm for automatic calculation of collapsed volume relies on 3D reconstruction and point cloud comparison using several previously calibrated fixed cameras.

By analyzing image sequences taken over one year at the Perarolo landslide site, along with laser scanner surveys, we validated the tool’s effectiveness under the following various conditions: day and night images; images captured at different time intervals; and the effect of vegetation and shadows. This on-site application allowed some of the potential and limitations to be identified concerning this method:

• Full Automation. The collapse event detection algorithm based on the structural similarity metric, once calibrated, allows for reducing the need for human interaction and mitigating false positives that could arise from merely comparing 3D surfaces. This also makes it possible to potentially perform real-time detection by comparing images at very short time intervals.
• Spatial accuracy and precision. The proposed algorithm has the advantage of processing the entire image area while excluding shaded and vegetated areas. The minimum identifiable collapsed volume has been estimated for our test site but in most cases, it is not easily controllable and is certainly higher than what can be obtained by comparing laser scanner point clouds. It depends on a multitude of factors related to the collapse detection process and the 3D reconstruction process. Among the most important are factors related to the instantaneous field of view (i.e., camera resolution, sensor size, focal length), factors related to the geometry of the subject being framed and the 3D reconstruction process (i.e., distance from the camera, angle between the local plane normal and the line of sight, baseline between cameras), and factors related to the image analysis process (e.g., the type and size of the filters’ kernel applied). Further details on the quantification of errors introduced by this method can be found in [6,65]. Further insights may result from the extensive use of this technique on different landslide surfaces.
• Low-Cost and Long-Term. Digital cameras have significantly lower costs compared to laser scanner systems or interferometry. Moreover, their maintenance or replacement is easier, making the system potentially suitable for long-term monitoring.

In general, the optimal operating conditions for this method are those where the cameras can be positioned frontally to the area under investigation, sufficiently close to it to maximize resolution, with a line of sight almost normal to the slope surface. The best landslide surface is also a bare, non-vegetated surface with irregular texture (i.e., not a monochromatic surface), not exposed to direct sunlight (to minimize shaded areas, at least during certain hours of the day), and not too susceptible to shallow erosion (to avoid false positives). Heavy rain, snow, clouds, and fog can naturally compromise the method’s applicability, although a preliminary selection of the most suitable images could be made using deep learning algorithms.

Further analysis is needed to refine this tool and gain more precise insights into the dynamics of the Perarolo landslide, as well as other landslide sites monitored with camera-based systems. Specifically, the ongoing work includes studying the SSIM specific to geotechnical materials and applying machine learning algorithms to enhance the selection of the most suitable image sequences [63]. For the further and improved development of this methodology, it will be particularly strategic to evaluate its performance across different types of landslides and varying environmental conditions. For instance, rainfall that frequently precedes collapse events can significantly alter the color and texture of landslide surfaces, depending on the material type, potentially leading to false positives during the collapse detection phase. Specific filters and algorithms will need to be developed to address the various scenarios that may arise, potentially utilizing artificial intelligence for both identifying the scenarios and selecting the appropriate resolution strategies.