Cultural Heritage Risk Assessment Based on Explainable Machine Learning Models: A Case Study of the Ancient Tea Horse Road in China
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe document presents a relevant topic on the vulnerability of cultural heritage considering natural disasters and anthropogenic factors. They use a machine learning model that allows obtain the vulnerability level. The following images should be improved: Figure 2, larger font Figure 3 and Figure 4, larger font Figure 5 larger font Figure 6,7,8, 9,10 and 11 larger font General review of English writing In the discussion it would be important to contrast the results with other studies or existing methodological frameworks, including references that provide support. The conclusions could include some recommendations on actions or strategies that allow minimizing and/or managing risk, to reinforce what was mentioned in point (2) of the discussion. The great difference in scales in the data generates some skewing... how was this addressed? (Information from Table 1). Was an uncertainty analysis carried out? If so, it may be included.
Comments on the Quality of English LanguageGeneral review of English writing
Author Response
Comments 1: The following images should be improved: Figure 2, larger font Figure 3 and Figure 4, larger font Figure 5 larger font Figure 6,7,8, 9,10 and 11 larger font.
Response 1: Thank you very much for your careful review of our paper and for your valuable suggestions for revision. Your feedback is important to improving the quality of your paper, and we value it. We've carefully adjusted the font sizes of Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, and Figure 11 to improve readability and hopefully better meet your expectations. Thank you again for your hard work and professional guidance. We will be more than happy to continue to refine the paper if there are any further revisions. Looking forward to your valuable feedback!
[See the 218 on Page 7、the 346、357 on Page 10、the 370、371、372 on Page 11、the 374 on Page 12.]
Comments 2: General review of English writing In the discussion it would be important to contrast the results with other studies or existing methodological frameworks, including references that provide support.
Response 2: Thank you for your affirmation of the research value of this topic and your valuable comments. In response to your point that it was important to compare the results with other studies or existing methodological frameworks, including supporting references, in the discussion, we undertook serious reflection and in-depth revisions. Here's what happened: We have strengthened our discussion of the vulnerability analysis of cultural heritage, particularly in the face of natural disasters (earthquakes, landslides, collapses, mudslides), and cited research to support our conclusions. ...... In terms of policy recommendations, we have strengthened the importance of public participation and environmental governance, and added more in-depth comparative analysis. Thank you again for your time and professional advice, and look forward to your further guidance!
[See the 739-820 on Page 27、28、29.]
Discussion
“（1）This study reveals the vulnerability of cultural heritage along the Ancient Tea Horse Road to the impact of natural disasters, especially earthquakes, landslides, avalanches and mudslides. The complex geological structure, high altitude and frequent seismic activity in the study area have made the cultural heritage face a high disaster risk for a long time. This conclusion is consistent with existing research that cultural heritage is more vulnerable to natural disasters in areas with intense geological activity, steep terrain slopes and complex geological conditions(Tarragüel, Krol et al. 2012, Frodella, Rosi et al. 2022, Zhu, Chen et al. 2023)。In addition to natural factors, human activities are also altering the original ecological environment and increasing the risk exposure of cultural heritage. Large-scale infrastructure construction has destroyed natural barriers, resulting in a significant increase in the frequency of landslides and mudslides(Luo, Zhang et al. 2024)，This conclusion is highly consistent with the results of previous studies(Li, Wang et al. 2020, Chiarelli, D'Odorico et al. 2021)。Therefore, rational planning of the relationship between human development activities and cultural heritage protection and formulating scientific land use policies are important contents of cultural heritage protection.
This study further analyzes the impact of settlement spatial structure (e.g., settlement boundary shape index, building density) and environmental factors (e.g., slope, distance from major rivers) on cultural heritage risk. The results show that the spatial pattern and environmental characteristics of settlements have a significant impact on the risk level, especially in earthquake- and landslide-prone areas. For example, due to soil instability and serious soil erosion in steep slope areas, the probability of landslide disasters has increased significantly, which in turn increases the vulnerability of cultural heritage. This finding is consistent with the conclusion that slope is regarded as a key factor of ecological risk in previous studies(Cellek 2022)。In addition, high-density buildings and complex boundary settlements may make evacuation more difficult after a disaster, thus increasing the risk of loss of cultural heritage. This is especially true in high-density settlement areas such as the Hengduan Mountains. Settlements close to major rivers are also at high risk to their cultural heritage due to soil erosion and flooding. In terms of risk assessment, the LightGBM model was used to improve the accuracy and stability of risk prediction. Compared with the traditional analytic hierarchy process (AHP), the LightGBM model can automatically process high-dimensional variables, reduce the influence of human subjective factors on the evaluation results, and show strong prediction ability in complex environments. Its efficient training speed and ability to adapt to large-scale datasets make it outstanding in capturing nonlinear relationships. Combined with SHAP value analysis, the contribution of different influencing factors to cultural heritage risk can be more clearly quantified, so as to improve the interpretability of the model(Yang, Yang et al. 2024)。Its core strength lies in its ability to provide both global and local explanations, which not only reveal the contribution of each feature to the overall model prediction, but also analyze how specific environmental factors affect the prediction results for a single sample. In addition, SHAP is suitable for Gradient Boosting Decision Tree (GBDT) models, which can accurately measure the contribution of input features to the prediction results, making model decisions more transparent. Therefore, this study uses SHAP to conduct feature importance analysis, identify key influencing factors, and visually demonstrate the specific contribution of different environmental factors to individual disaster susceptibility prediction.
Although some progress has been made in this study, there are still some limitations. First, the effectiveness of the LightGBM model depends on data quality and completeness, especially the lack of historical disaster data may affect the accuracy of risk assessment. Second, although LightGBM is capable of addressing most risk assessment scenarios, it may still be necessary to combine other modeling approaches to optimize assessment accuracy in more complex environments. Future research may consider introducing multi-source data, such as remote sensing monitoring data and field survey data, to improve the generalization ability of the model and further explore the long-term impact mechanism of different risk factors on cultural heritage security. Cultural heritage is not only about physical structures, but also about the natural environment in which it is located and the culture of the community. This study focuses on the impact of natural disasters, infrastructure and settlement characteristics on cultural heritage, with less emphasis on social, spiritual and community values. However, the vulnerability of cultural heritage is not only influenced by geological and environmental factors, but also by socio-economic development, community awareness and cultural heritage. For example, community attitudes towards the preservation of cultural heritage, visitor activities, religious beliefs and cultural identities may all affect the long-term preservation of heritage. Future research can combine information such as social surveys, cultural identity assessments, and community participation to further improve the heritage vulnerability assessment system, make the assessment results more comprehensive, and provide more scientific support for the protection and management of cultural heritage.
（2）This study highlights the need for systematic protection measures to protect cultural heritage. Effective risk management relies on strict regulations, a sound early warning system and professional management. All localities should strengthen the implementation of laws and regulations, optimize monitoring and early warning, and regularly maintain protection facilities, especially in high-risk areas such as earthquakes and landslides. In addition, unnatural factors such as deterioration in environmental quality, agricultural pollution and overtourism also exacerbate cultural heritage risks(Yang, You et al. 2023)。Raise public awareness of disaster preparedness to enhance community participation and governance effectiveness. In the future, we should rely on remote sensing monitoring, big data and other technologies to optimize risk assessment, promote collaborative governance between the government, scientific research and society, improve the adaptability of cultural heritage, and provide reference for high-risk areas.”
Comments 3: The conclusions could include some recommendations on actions or strategies that allow minimizing and/or managing risk, to reinforce what was mentioned in point (2) of the discussion.
Response 3: Thank you very much for your valuable advice. It was very helpful to respond to your question about the lack of specific actions and strategies to reduce and manage risks to cultural heritage in the conclusions; We have revised this section to supplement the corresponding measures. These suggestions further reinforce the core points in discussion section (2).
[See the 846-859 on Page 29、30.]
“On this basis, in order to more effectively reduce the risks faced by cultural heritage, it is recommended to start from multiple aspects. First of all, in the process of urban development and transportation construction, rational planning should be made to reduce direct interference with cultural heritage, and the implementation of protection policies should be strengthened, so as to achieve the parallel protection and development. Second, a geological hazard monitoring system can be established to improve risk early warning capabilities, and appropriate restoration technologies, such as 3D scanning and digital archiving, can be used to ensure the long-term preservation of cultural heritage information. In view of the potential impact of tourism, measures can be taken to limit the flow of tourists and guide tourists to participate in low-impact ecotourism to reduce the pressure of overexploitation. In addition, governments, communities and relevant institutions need to work together to develop more operational strategies for the protection of cultural heritage to ensure its sustainable development.”
Comments 4: The great difference in scales in the data generates some skewing... how was this addressed? (Information from Table 1). Was an uncertainty analysis carried out? If so, it may be included.
Response 4: Thank you very much for your valuable comments. We are fully aware that data scale differences can lead to skew, thereforeIn this study, the data have been normalized and standardized when calculating the index weights using LightGBM to eliminate possible bias from scale differences. In addition, in order to evaluate the robustness of the model and reduce the uncertainty caused by data partitioning, we adopted the K-Fold Cross-Validation method with the root mean square error (RMSE) as the evaluation metric. The cross-validation results show that the smaller the RMSE value, the smaller the error of the model (Table 2), indicating that the index weight calculation has high reliability. Although uncertainty analysis is not specifically performed in this study, cross-validation has reduced the bias caused by different data sets to a certain extent. We look forward to your further guidance and feedback.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper aims to present a quantitative approach and a methodological framework for cultural heritage risk assessment. It introduces an innovative protocol using machine learning processes, the LightGBM model and SHAP method, integrated with GIS technology to analyze risk factors affecting cultural heritage. The research experimentation took place along the Ancient Tea Horse Road in Sichuan, China. The study clearly identifies seven major risk factors such as landslides, collapses, debris flows, earthquakes, soil erosion, urban road networks, and cultural heritage vulnerability, providing a multi-risk framework that incorporates both natural and anthropogenic threats strengthening the practical applications of the research.
The research objective is significant and relevant to the field in the light of climate change issues and anthropic pressure on our cultural heritage assets. Practitioners, researchers and policymakers working on related topics may be interested. Thus, it is relevant to an international audience. It is related to the scope of the Journal. To my understanding and knowledge, there is no current commercial interest in the paper.
The authors present an understanding of relevant literature about the research aims especially considering the different multi-risk assessment processes currently adopted. The introduction (1.Introduction) is supported by up-to-date and appropriate sources related to the research field and aims. The following chapter could be implemented and strengthened with more references: 2.3.1 LightGBM. Implementing the references can allow other researchers to trace back the theoretical framework of this specific analysis methodology.
The article presents a robust methodology section that sounds comprehensive, well-structured, and scientifically rigorous. The clarity of this section allows higher replicability in different contexts. Nevertheless, an innovative approach to the multi-risk analysis such as machine learning is presented combining machine learning predictions with GIS-based spatial analysis. The authors provide in-depth detail about data collection, data processing, results review, and results discussion. The study utilizes high quality data-set and all the processing techniques are well explained.
The results are reliable due to the methodology adopted. They are conveyed clearly and concisely. The data visualization protocol adopted helps to have a clear understanding of the results. Moreover, some improvements can be made (but I totally understand that synthesis is needed too): 1- LightGBM implementation details such as training parameters and feature selection; 2 - clarify the use of SHAP over other explainability techniques.
More details on model validation, including cross-validation methods, alternative models, and dataset splits could be introduced.
Finally, GIS spatial analysis techniques such as risk weighting and validation approaches could be clarified more.
The title of the paper is appropriate and reflects the article's content. The abstract provides a well-structured overview of the objectives, and the methodological framework adopted and it emphasizes the outcomes of the paper making its impact more explicit.
The data sources are referenced clearly. There is clarity of purpose, structure, and expression. The narrative and detailed argumentation are adequate. The steps of the methodology are conceived clearly. The findings are given and they can be noticed and perceived easily. The quality of the illustrations is adequate. Terms and contexts are explained clearly for an international audience.
Author Response
Comments 1: 2.3.1 LightGBM. Implementing the references can allow other researchers to trace back the theoretical framework of this specific analysis methodology.
Response 1: Thank you very much for your careful review and valuable suggestions. We fully recognize your interest in the traceability of the theoretical framework and have added references to the revised draft to more clearly clarify the rationale for the LightGBM approach in this study. These documents are helpful to further consolidate the theoretical support of research methods, and also facilitate the reference and traceability of other researchers. If there are further suggestions, we are more than happy to continue to refine and improve. Thank you again for your valuable comments!
[See the 232-233 on Page 7.]
（1）Ke G, Meng Q, Finley T, et al. Lightgbm: A highly efficient gradient boosting decision tree[J]. Advances in neural information processing systems, 2017, 30.
（2）Meng Q, Ke G, Wang T, et al. A communication-efficient parallel algorithm for decision tree[J]. Advances in Neural Information Processing Systems, 2016, 29.
Comments 2: LightGBM implementation details such as training parameters and feature selection; 2 - clarify the use of SHAP over other explainability techniques.
Response 2: Thank you very much for your valuable comments. For the implementation details of LightGBM, we carefully selected a series of hyperparameters in model training to improve the stability and predictive ability of the model. ...... In addition, SHAP is particularly suitable for Gradient Boosting Decision Tree (GBDT) models, which can accurately measure the contribution of input features to the prediction results, making model decisions more transparent. Therefore, in this study, we used SHAP to conduct feature importance analysis, identify key influencing factors, and visually demonstrate the specific contribution of different environmental factors to individual disaster susceptibility prediction. Thank you again for your careful review and valuable suggestions, and we hope that these additional notes will further improve the clarity and explainability of the study. If there is further feedback, we are more than happy to continue to optimize and improve.
[See the 197-203 on Page 6.See the 776-785 on Page 28]
“In order to improve the stability and prediction ability of the model, parameters such as learning rate of 0.05, maximum depth of 7, number of leaf nodes of 31, and feature sampling rate of 0.7 were finally selected to prevent overfitting and merging and enhance the generalization ability of the model. At the same time, LightGBM is used to calculate the importance of all features, and the features with low contribution to the model are eliminated to reduce the interference of redundant information and improve the generalization ability of the model.
Its core strength lies in its ability to provide both global and local explanations, which not only reveal the contribution of each feature to the overall model prediction, but also analyze how specific environmental factors affect the prediction results for a single sample. In addition, SHAP is suitable for Gradient Boosting Decision Tree (GBDT) models, which can accurately measure the contribution of input features to the prediction results, making model decisions more transparent. Therefore, this study uses SHAP to conduct feature importance analysis, identify key influencing factors, and visually demonstrate the specific contribution of different environmental factors to individual disaster susceptibility prediction.”
Comments 3: More details on model validation, including cross-validation methods, alternative models, and dataset splits could be introduced.
Response 3: Thank you very much for your valuable comments. In this study, the data have been normalized and standardized when calculating the index weights using LightGBM to eliminate possible bias from scale differences. In addition, in order to evaluate the robustness of the model and reduce the uncertainty caused by data partitioning, we adopted the K-Fold Cross-Validation method with the root mean square error (RMSE) as the evaluation metric. The cross-validation results show that the smaller the RMSE value, the smaller the error of the model (Table 2), indicating that the index weight calculation has high reliability.
With regard to model selection, LightGBM has been compared in detail with other possible modeling approaches in the discussion section. At the same time, in order to enhance the interpretability of the model, this study combined with SHAP to perform feature importance analysis to ensure that the results are transparent and interpretable. Therefore, based on the comprehensive consideration of model performance and interpretability, LightGBM was finally selected as the research method.
[See the 768-776 on Page 28.]
“In terms of risk assessment, the LightGBM model was used to improve the accuracy and stability of risk prediction. Compared with the traditional analytic hierarchy process (AHP), the LightGBM model can automatically process high-dimensional variables, reduce the influence of human subjective factors on the evaluation results, and show strong prediction ability in complex environments. Its efficient training speed and ability to adapt to large-scale datasets make it outstanding in capturing nonlinear relationships. Combined with SHAP value analysis, the contribution of different influencing factors to cultural heritage risk can be more clearly quantified, so as to improve the interpretability of the model.”
In terms of dataset splitting, we use an 80% training set and a 20% test set to strike a balance between model generalization ability and test data representativeness, and ensure consistent distribution of training and test data through multiple random partitions.Thanks again to the reviewers for their meticulous review, and we hope that these additions will improve the completeness and explainability of the study. If you have any further suggestions, we will be happy to continue to improve!
Comments 4: Finally, GIS spatial analysis techniques such as risk weighting and validation approaches could be clarified more.
Response 4: Thank you for your careful review of this article. In this study, a weighted overlay analysis was used to evaluate the comprehensive risk of cultural heritage along the Ancient Tea Horse Road in Sichuan. ...... In order to verify the reliability of the risk assessment results, this study combined field research to verify the results. We conducted field surveys and site surveys within the study area and collected information on the damage to cultural heritage.
[See the 203-217 on Page 6.]
“In this study, a weighted overlay analysis was used to evaluate the comprehensive risk of cultural heritage along the Ancient Tea Horse Road in Sichuan. The weights of each risk factor are determined by the feature importance calculated by LightGBM, and after the weights are normalized, they are multiplied and accumulated with the normalized risk factors to calculate the comprehensive risk index for each grid. Then, ArcGIS Spatial Analyst was used to perform a weighted overlay calculation, and the natural breakpoint classification method was used to classify the risk index into five levels: very low, low, medium, high, and very high, with higher risk values indicating greater threats to cultural heritage in the region. In order to verify the reliability of the risk assessment results, the results were verified by field investigation. Field surveys and field surveys were carried out within the study area, and information on the destruction of cultural heritage was collected. Finally, the comprehensive risk analysis map clearly shows the spatial risk distribution of cultural heritage in the study area, which provides a basis for the formulation of risk management and protection measures.”
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThe paper presents an important discussion on heritage risk assessment and mitigation, with a focus on China. The topic is highly relevant, and the research offers valuable insights into the challenges faced in preserving cultural heritage. However, certain areas would benefit from further clarification and expansion.
117. First, the paper would be strengthened by a clearer explanation of the current framework for assessing and mitigating risks in China. This should include relevant cultural heritage laws and policies to provide readers with the necessary context for understanding the regulatory landscape.
160. Additionally, the criteria used for selecting the sites analyzed in the study should be explicitly stated to clarify the methodology and ensure transparency in the research approach.
581. Another key aspect to consider is the broader definition of heritage. Heritage extends beyond physical structures to include the natural landscape and, crucially, the communities that give sites their historical and cultural significance. It would be beneficial to address how the proposed model incorporates social, spiritual, and community values when assessing heritage fragility. A more holistic approach would enhance the paper’s contribution to the field.
Author Response
Comments 1: First, the paper would be strengthened by a clearer explanation of the current framework for assessing and mitigating risks in China. This should include relevant cultural heritage laws and policies to provide readers with the necessary context for understanding the regulatory landscape.
Response 1: Thank you for your careful review of this article. Your valuable suggestions provide important guidance for refining the framework and content of this study, making the paper more rigorous and clear. We take your comments very seriously and have made corresponding changes in the revised draft to enhance the completeness and readability of the research.
[See the 115-127 on Page 3.]
“At the academic level, this study expands the methods and content of risk assessment of China's cultural heritage, and proposes a new assessment framework and perspective. At the same time, the basic framework of cultural heritage risk management is analyzed in combination with the current laws and policies on cultural heritage protection, such as the relevant regulations of the State Administration of Cultural Heritage. These policies provide guidance on risk assessment, disaster preparedness and emergency management, but there is still room for improvement in spatialized risk assessment and multifactorial integrated analysis. This study uses a comprehensive risk assessment approach, combined with machine learning technology and GIS, to provide a more refined risk quantification method, provide a scientific basis for the government and relevant departments to formulate protection measures, and facilitate the precise protection and sustainable management of cultural heritage.”
Comments 2: Additionally, the criteria used for selecting the sites analyzed in the study should be explicitly stated to clarify the methodology and ensure transparency in the research approach.
Response 2: Thank you very much for your valuable comments on this article. Your suggestions are of great value in improving the transparency of the research and the clarity of the methodology.
[See the 169-176 on Page 4.See the 188 on Page 5.]
“In this paper, the cultural heritage along the Ancient Tea Horse Road was selected as the research object, and the selection criteria were based on the three categories of traditional Chinese villages, national cultural relics protection units, and national historical and cultural towns, as well as relevant data from official data platforms, so as to ensure the representativeness of the research and the authority of the data. The spatial distribution, geographical features and risk profile of these cultural heritages were analysed in detail using high-resolution remote sensing imagery and geographic information system (GIS) technology.”
Comments 3: Another key aspect to consider is the broader definition of heritage. Heritage extends beyond physical structures to include the natural landscape and, crucially, the communities that give sites their historical and cultural significance. It would be beneficial to address how the proposed model incorporates social, spiritual, and community values when assessing heritage fragility. A more holistic approach would enhance the paper’s contribution to the field.
Response 3: Thanks to the reviewers for their valuable suggestions. We recognize that the fragility of cultural heritage is not only affected by natural disasters, infrastructure and settlement characteristics, but is also closely related to society and spirituality. Based on the suggestions of the reviewers, we have added relevant content in the discussion section. Thanks again to the reviewers for their meticulous review and valuable suggestions, we hope that these additions will improve the integrity of the paper. If you have any further suggestions, we will be happy to continue to improve!
[See the 796-807 on Page 28、29.]
“Cultural heritage is not only about physical structures, but also about the natural environment in which it is located and the culture of the community. This study focuses on the impact of natural disasters, infrastructure, and settlement characteristics on cultural heritage, with little social and spiritual implications. However, the vulnerability of cultural heritage is not only influenced by geological and environmental factors, but also by socio-economic development and cultural heritage. For example, community attitudes towards the preservation of cultural heritage, visitor activities, religious beliefs and cultural identities may all affect the long-term preservation of heritage. Future research can combine information such as social surveys, cultural identity assessments, and community participation to further improve the heritage vulnerability assessment system, make the assessment results more comprehensive, and provide more scientific support for the protection and management of cultural heritage.”
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
Comments and Suggestions for Authorsno further comments
