**4. Results**

This section outlines the clustering results of the submarine landslide susceptibility using k-means, spectral, and hierarchical models after parameter validation and selection. The study area was divided into four areas of different submarine landslide susceptibilities without real geological labels. To define the final labels, all the geological parameters should be considered. As mentioned in Section 3.2, the hydrodynamic force to the seabed increases first with the deepening of water depth and then decreases. Moreover, the sediment type changes from silt to clay, which is difficult to be influenced, when the water depth is deeper than 15 m. Therefore, the submarine landslide susceptibility labeling principle of this area is that: the least serious part is the clay region; the second least serious part is the shallow water region; the most serious part is around the 10 m bathymetric contour; and the second most serious part is beside or around it.

As shown in Figure 5, the study area was divided into four parts of submarine landslide susceptibility using the k-means model. The very high-susceptibility part is located at a water depth of 5–11 m, and the edge is a discrete distribution. The highsusceptibility part is distributed at a water depth of 11–13 m. The low-susceptibility part is situated at a water depth of less than 5 m. The very low-susceptibility part is located at a water depth deeper than 13 m.

**Figure 5.** Distribution of submarine landslide susceptibility using k-means. The different colors represent different submarine landslide susceptibilities.

It can be seen from Figure 5 that the study area was divided into four parts of submarine landslide susceptibility using the spectral clustering model. The very highsusceptibility part is located at a water depth of 5–12 m. The high-susceptibility part is distributed at a water depth of 12–15 m. The low-susceptibility part is situated at a water depth of less than 5 m. The very low-susceptibility part is located at a water depth deeper than 15 m. Compared with the results obtained by k-means, the distribution of submarine landslide susceptibility using spectral clustering is more continuous than that using the k-means algorithm. Furthermore, the very high-susceptibility part is wider and the very low part is narrower in Figure 6 than in Figure 5.

**Figure 6.** Distribution of submarine landslide susceptibility using spectral clustering.

As shown in Figure 7, the study area was divided into four parts of submarine landslide susceptibility using the hierarchical clustering model. The very high-susceptibility part is located at a water depth of 5–8 m. The high-susceptibility part is distributed at a water depth of 8–13 m. The low-susceptibility part is situated at a water depth of less than 5 m, and the very low-susceptibility part is located at a water depth deeper than 13 m. By comparing the results of the three algorithms, we can find that the very high-susceptibility part obtained by using the hierarchical clustering model is much less than other two of the other methods.

Generally, the low- and very low-susceptibility parts of these three methods are very close. The main differences are in the size and distribution of very high and high areas. The three unsupervised machine learning methods obtained two main common parts: one is a low-hazard region at shallow depths of 5 m; the other is a very low-hazard region at depths of 13 m. The reason for this phenomenon is that the hydrodynamic conditions are relatively weak in these two parts of the area, and the influence of various geological-impact parameters on submarine landslides is also small; thus, the results obtained by different algorithms are more consistent.

**Figure 7.** Distribution of submarine landslide susceptibility using hierarchical clustering.

#### **5. Discussion**

#### *5.1. Model Performance Comparison*

To test the performance of cluster results, both internal validation measures and external validation measures were used. For internal validation measures, we validated the accuracy of three different submarine landslide susceptibility models by using the Calinski–Harabasz index, silhouette index, and Davies–Bouldin index. The three indexes use different algorithms to examine the merits of the classification results from a mathematical perspective. As shown in Figure 8, the k-means model performed best under the evaluation of the Calinski–Harabasz index, whereas the spectral clustering model performed best when evaluating the silhouette index and Davies–Bouldin index. Therefore, spectral clustering has the best performance compared to k-means and hierarchical clustering in the internal validation measures.

In general, external causes of submarine landslides include earthquakes, gas hydration, wave action, volcanic activity, tsunamis, etc. There are no severe geological phenomena such as earthquakes, tsunamis, and volcanoes in the study area. The main external influence factor is the liquefaction of seabed soil caused by waves. Therefore, for the external validation measure, the three distributions of submarine landslide susceptibility were compared with the distribution of liquefaction depth (Figure 9). As shown in Figure 9, the area with the deepest liquefaction depth (ellipse A) is distributed at a water depth of 6–12 m. The area of ellipse A agrees well with the submarine landslide susceptibility results using k-means and spectral clustering, whereas the result obtained using hierarchical clustering is quite different from ellipse A. The area with a deep liquefaction depth (ellipse B) is distributed at a water depth of 12–15 m. It is in good agreement with the result obtained using hierarchical clustering, and the areas obtained by k-means and hierarchical clustering are different from the area of ellipse B. The area with a small, shallow liquefaction depth (ellipse C) is distributed at a water depth of less than 5 m, which agrees well with all the clustering results. The area with a very shallow liquefaction depth (ellipse D) is distributed at a water depth deeper than 15 m. It is in good agreement with the result obtained using

hierarchical clustering, and the areas obtained by k-means and hierarchical clustering are larger than the area of ellipse D. As a result, the submarine landslide susceptibility results using spectral clustering performed better than those obtained using k-means and hierarchical clustering in the external validation.

**Figure 8.** Comparison of clustering results validated with different internal validation measures. (**a**) Performance of different models under Calinski–Harabasz validation methods. (**b**) Performance of different models under silhouette validation methods. (**c**) Performance of different models under Davies–Bouldin validation methods. The higher the Calinski–Harabasz index and silhouette index are, the more accurate the clustering result will be. The lower the Davies–Bouldin index is, the more accurate the clustering result will be.

**Figure 9.** Distribution of liquefaction depth in the Yellow River Estuary. The area of ellipses A, B, C, and D represents the very deep, deep, shallow, and very shallow parts of liquefaction depth, respectively.

In conclusion, all three models used in this study are capable of producing correct clustering results, with the spectral clustering model being the most precise when grouping the undersea landslide susceptibility. Although the best algorithm can be obtained by internal verification methods, the scores of different algorithms are not different enough to represent the difference. To obtain the best results, it is still necessary to combine external verification methods at the same time.

#### *5.2. Comparison of Model Results with Other Studies*

In order to verify the accuracy of the model calculations in this paper, comparisons with the results of other studies are still needed. Therefore, we selected the results of a geophysical survey and the results of traditional GIS-based landslide analysis methods to compare and analyze with the conclusions of this paper.

In the 1980s, a comprehensive, integrated geophysical survey was conducted in the Yellow River Estuary waters [19]. The results of the survey showed that a large number of microslides on the leading edge of the delta existed at a water depth between 4 and 12 m in the study area. It can be seen from Figure 5 that the classification of landslides as hazard results using the spectral clustering algorithm shows that the areas with very high hazards are concentrated at a water depth range of 5–12 m. The simulation results are highly consistent with the actual survey results, indicating the accuracy and reliability of the algorithm in this paper.

As for the conventional approach used to analyze submarine landslide hazards, the GIS-based analyzing method was also used to study the stability of submarine landslides in the Yellow River Estuary [27]. Results show that the most prone landslide areas in the study area are located at a water depth between 8 and 13 m, and the more prone landslide areas are located at a water depth between 5 and 15 m, with an average water depth of 10 m. The range of landslide-hazard areas derived from GIS results is consistent with the overall distribution range and trend compared with those derived in this paper, and the range of landslide hazards is slightly smaller. The model calculation results in this paper are closer to the results of the actual geophysical survey compared with the GIS results.

Therefore, it is clear that the unsupervised machine learning method used in this paper has high reliability and stability after being compared with the traditional methods of submarine-landslide-hazard analysis and the actual geophysical survey results in the field.

Although just one research area was used for the study of submarine landslide hazards, the general geological formation conditions and triggering mechanisms of submarine landslides in all study regions are comparable, despite the differences in causes and triggering variables. As a result, this paper's research technique and research hypothesis can serve as a guide for the global study of submarine landslide hazards.

#### *5.3. Importance of Landslide-Influencing Factors*

To figure out the significance of influencing elements, each factor was eliminated and the results were recalculated. New cluster results with different factors were compared with the original clustering results using the Calinski–Harabasz index, silhouette index, and Davies–Bouldin index. Nine clustering results correspond to the missing influence factors and the evaluation scores were normalized. The higher the normalized ratio is, the less important this factor is. The order of importance in this study does not represent the absolute order of the corresponding impact factors, but only represents the relative order in the study area.

Test results can be seen in Figure 10, where the CH represents the normalized ratio of the Calinski–Harabasz index and the SI means the normalized ratio of the silhouette index. The DAV is obtained by 2 minus the normalized ratio of the Davies–Bouldin index so that the trend is the same as the other two indexes. As seen in Figure 10, the model without liquefaction obtained the lowest score, which means liquefaction is the most important factor that influences landslides in the study area. Models that exclude water depth, wave height, and soil strength obtained higher scores than those that exclude liquefaction, which means they are the second most important factors. These three elements may affect whether liquefaction happens, but none of them can predict it on their own. Consequently, these components are less significant than liquefaction, but more significant than other factors. As for sediment type, erosion, and maximum current velocity of the bottom, they are less important than the factors mentioned before. There is a great correlation between sediment types and soil strength. However, the physical and mechanical properties of the same sediment may be different because of the different depositional states and times. Therefore, soil strength has a greater impact on landslides. Erosion and maximum current velocity of the bottom have certain influences on the stability and strength of sediments, but are not decisive; therefore, the importance degree is relatively low. The least important factors are slope angle and human engineering activities in this study area. Slope angle has an important influence on the landslide according to previous studies [28,29], but the difference in slope angle in this study area is not large enough, and thus, the influence degree is low. As described in Section 3.2, the biggest slope angles are >1/500 radian, which is about equal to 0.11◦. However, Hance [30] counted 399 cases of submarine landslide slope angles and found that the most frequent value was 3~4◦ and the average value was 5.8◦. These data are far larger than the slope value in our study area, and the average value is 50 times the maximum value in the region; thus, the slope angle is negligible and is very unimportant in the landslide evaluation of the study area. As for human engineering activities, some engineering activities, such as hydrate exploitation [31,32], can play an important role in the formation of submarine landslides, but the engineering activities in the study area are mainly offshore platforms and submarine pipelines, which are widely distributed, and therefore, their importance is very low and can be ignored.

**Figure 10.** The normalized score of clustering results changed with different influence parameters. The normalized results were obtained by dividing the new scores of eight parameters by the scores of nine parameters.

In conclusion, various influencing variables have varying degrees of impact influence on submarine landslide risk assessment, and the significance relies on the degree of correlation between the landslide and its distribution in the study area. The order of importance and degree of effect of variables acquired in this research only reflect one study area; the techniques described in Section 5.2 must be employed to conduct particular analyses in other study areas.

#### **6. Conclusions**

In this paper, a submarine landslide susceptibility assessment was carried out using unsupervised machine learning models in the Yellow River Estuary, China. Nine influential factors were selected to analyze the susceptibility of submarine landslides based on terrain data and remote sensing images. We used different unsupervised machine learning models to classify landslide risk and discussed the accuracy of the model and the importance of a single factor. The main conclusions are as follows:

(1) Unsupervised machine learning models can be used to study and assess submarine landslide susceptibility and provide high accuracy.


Due to the complexity of the elements impacting the submarine-landslide-hazard triggers and the difficulties of monitoring submarine landslide sites, it is challenging to determine the precise location and hazard information of each landslide in the research region. Currently, unsupervised machine learning can only be conducted using limited data for semiquantitative description. For a more in-depth examination of submarine landslide hazards, accurate training data is necessary. In the future, we must, therefore, place a greater emphasis on the collection of field data for undersea landslide identification and monitoring.

**Author Contributions:** Conceptualization, Y.S. (Yongfu Sun); methodology, X.D.; program, X.D.; validation, Y.S. (Yupeng Song) and Z.X.; resources, Z.S.; data curation, Z.S.; writing—original draft preparation, X.D.; writing—review and editing, X.D.; visualization, X.D.; supervision, Y.S. (Yupeng Song); project administration, X.D. and Y.S. (Yongfu Sun); funding acquisition, X.D. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Foundation item: The National Natural Science Foundation of China under contract NO. 42102326; the Basic Scientific Fund for National Public Research Institutes of China under contract NO. GY0222Q05; and The Shandong Provincial Natural Science Foundation, China under contract NO. ZR2020QD073.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data that support the findings of this study are available from the corresponding author upon reasonable request.

**Conflicts of Interest:** The authors declare no conflict of interest.
