Supervised Machine Learning Algorithms to Predict Provenance of Archaeological Pottery Fragments

Round 1

Reviewer 1 Report

Anna Anglisano et al. present a archaeologica l classification problem based on supervised machine learning. The authors demonstrated authenticate and distinguish similar local modern productions at  the five sets of pottery of unknown provenience but suspected to be produced locally in Catalonia. The work is interesting and inspiring to the field of sustainable archaeo-logical practices. 

The paper's organization is good. The text in English is moderate. The figures are well presented and constitute the strength of the paper. 

I therefore recmmend this paper to be publicated. And it is better if the authors consider the following mentioned remarks and further improve the manuscript before submitting the final version. 

Specific comments:

1. Please state the reason for choosing the particular 6 modeling techniques, and add substantially to the existing literature.

 2. Analysis primarily discusses the accuracy score of the models. Why was F1 score (A common index for evaluating machine learning models) not discussed ? Accuracy is not the best indicator of performance. Please include analysis of the F1 score in particular. Also talk about Recall and why looking at recall is necessary as well for false positive identification.

 3.Discuss how feasible is this model due to relevant data availability. Are balanced number of samples readily available in all reference classes? If not, what is the scope of this work?

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper entitled “Supervised machine learning algorithms to predict provenance of archaeological pottery fragments” is addressed to explore the performance of supervised machine learning classification methods applied to provenance archaeological pottery.

There are several issues which should be considered by the authors during paper revision.

[Abstract].

The abstract is not well-written and badly organized. First and foremost, after reading your abstract I have not obtained the summary of the paper. Instead there is a lot of irrelevant information such as:

·        “The methodology is based on a protocol that had already proven to be successful to authenticate and distinguish similar local modern productions”; moreover, what do you mean by saying “local modern productions”?

·        “Two practical Rmarkdown files developed in R programming language including sets of data processing elements connected in series are made freely available”; is it a contribution of your study? In my opinion, these files do not contribute to the body of knowledge.

Please take a look on this web page: https://www.nlm.nih.gov/bsd/policy/structured_abstracts.html#:~:text=What%20are%20structured%20abstracts%3F,comprehension%20(see%20Figure%201).

Now considering the guidelines given above, please rewrite the abstract.

[Introduction]

·        Please rewrite the sentence: “the model was used to infer the provenance of several pottery samples retrieved from archaeological sites from Catalonia and it could potentially be applied to the hundreds of archaeological sites reported in this region”. Please stop using such colloquial statements such as “hundreds”.

·        I don’t understand this sentence: “Ultimately, the aim is to help (…)”, the aim of what?

 Moreover, what is the goal of your paper? Please split the last paragraph and distinguish:

a)      the primary goal of your study (performance evaluation),

b)     the secondary goal (algorithms implementation).

[Materials and Methods]

Please provide the references to the Tables and Figures in the brackets. For instance, one can read “To obtain the geochemical composition, exactly like it was done with other reference samples, the samples”, which samples are you referring to?

Please apply the list, making the relatively long paragraphs more readable. For instance, taking into account this sentence: “They belong to six traditional pottery production centers relatively close to Barcelona (Figure 1): Esparreguera (~35 km from it), Breda (~50 km from it), Sant Julià de Vilatorta (~60 km from it), Quart (~80 km from it), Verdú (~90 km from it) and La Bisbal d’Empordà (~100 km from it).”, for sure you could use the numbered list.

In case of Figure 1, the text of the legend is hardly readable, considering the points from 1 to 8.

Your samples are not described in the way which let me understand what exact attributes and values are collected. This is a major obstacle in reviewing your paper. In my opinion, at this stage, you simply provide the meta-data of the collected and processed datasets. Undeniably, from my point of view as a reviewer, it is not enough to perform a comprehensive review of your paper. Therefore, I have finished my review on subsection 2.2. To sump, I opt for a major revision in case of the current version of your paper.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear Authors;

It is quite a good topic. However, the novelty brought by the research needs to be clearly demonstrated by more extensive literature research. 

For example, these are related to your topic;

-Automatic classification of archaeological pottery sherds

-Classification of engraved pottery sherds mixing deep-learning features by compact bilinear pooling

Please show your literature review in a table for more practically check by readers. For example, the common aspects of the studies you reviewed, their deficiencies, their methods, and the novelty they brought.

It will be more understandable if you show the research method part step by step with a diagram.

Accuracy and other diagnostic values for your prediction models should be shown in a table. Because you are sharing your data, these values will be a benchmark and can be developed for future studies.

Please try to use deep learning methods; outstanding results may come out.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The revised manuscript has been in well-form to be published. No further revisions are required.

Author Response

Thank you for your valuable remarks and comments, the paper has greatly improved thanks to them.

Reviewer 2 Report

Dear Authors,

Thank you for addressing my concerns.

Taking into account your responses, and the content of your paper (especially Section no 4), in my humble opinion, the Sustainability journal is not devoted for such type of study. There are other journals which publish similar vein of research.

 

Author Response

Comment: Thank you for addressing my concerns.

Reply: Thank you for your comments and remarks, the paper has greatly improved thanks to them

----

Comment: Taking into account your responses, and the content of your paper (especially Section no 4), in my humble opinion, the Sustainability journal is not devoted for such type of study. There are other journals which publish similar vein of research.

Reply: According to the other reviewer’s opinion the paper is suitable for Sustainability. In any case and according to the suggestion by the editor we have added an additional subsection within the discussion section (4.3 Contribution to sustainable archaeology) with a solid justification of how this study can contribute to sustainable archaeology including reflections to explain why our approach would promote sustainability in artefact studies, and what the future perspectives of this approach might be.

Round 3

Reviewer 2 Report

Dear Authors,

We don't agree on what a research paper is. Therefore, further discussion is simply irrelevant and unnecessary. 

As I mentioned before, there are other journals which are focused on the development of the software solutions. In my opinion, this journal is not focused on such contributions. On the other hand, I think that the elaborated solution is valuable and will find the followers.