Exploring the Effectiveness of Shallow and L2 Learner-Suitable Textual Features for Supervised and Unsupervised Sentence-Based Readability Assessment
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe subject of the paper can be interesting, as long as the authors provide possible applications of readability assessment at the sentence level.
One ambiguous aspect is the use of L2-Learner term. At the beginning of the paper, the point seems to be on the L2 learners, and providing tailored information at an appropriate complexity level is the core of the paper (lines 19-60).
Next, L2 -Learner term appears as a category of textual features used to predict readability level for some particular textual information (line 69). Even the title suggests that use of the term.
Basically, the subject of the paper consists in a binary classification problem, and the obtained result around 60% correctly classified instances is not a good one.
Lines 145 – 148 All seven references provided here (from 10 to 16) are not related to the subject of the paper; the only purpose of adding the entire sentence is to obtain an increased number of references. And the sentence ”The interested reader may consult [14], [15], and [16] for some recent results” is simply surprising – the idea of literature overview is to extract and present to the readers how the proposed paper fits in the research area, not to send them to read other papers if they are interested in learning more about a tangential domain.
Line 446 – Is the sentence ”It is clear that the complex sentences introduce difficult words with many more syllables” really considered an experimental result of the paper?
Lines 515-516 – The information provided in the sentence ”Random Forest algorithm returns the best results when we increase the number of trees and reduce the bag size significantly” is well known, why do the authors consider it a result?
Line 542-546 - When dividing data set in 95/5 ratio, it is possible to have no sufficient data to test the performance of the method, thus the results can be unreliable.
Line 664 – ” .... and comparing to other sources in the literature” - there is no comparison provided in the paper.
The authors want to impress the readers, using semi-technical phrases. An example is in the lines 79-80 - ” By immersing ourselves in the extensive body of existing literature…”. In my opinion, the paper must be presented in a more concise manner. It can be reduced even by 50% without affecting the subject and the results.
Comments on the Quality of English LanguageSmall editing mistakes were found.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsIn this paper, the authors analyzed the performance of novel shallow feature sets on a specific Wikipedia dataset and featured new results using a variety of optimally set-up classifiers. The topic is interesting and within the scope of the journal. However, the organization of this paper is not clear enough. The following concerns should be considered to improve the manuscript.
1. The main contribution is confused. The classification algorithm should be the critical innovation instead of describing the feature groups.
2. Literature reviews should analyze what is the weaknesses of the related works. The authors only list some papers without further analysis, which makes the motivation confused.
3. The format of the references is bad. It can not be inferred whether these are the latest researches.
4. Section 3 and Section 4 are both confused. The authors should clearly illustrate the methodology and classification process in Section 3, and demonstrate the experiment results in Section 4.
5. The methodology should be illustrated with figures. The current form is not clear enough.
6. The experiment results are extensively long and confused. The authors should emphasize the most important part and show its advantages in comparative results.
7. The language can be improved.
Comments on the Quality of English LanguageThe language can be improved.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsReview on paper “Exploring the Effectiveness of Shallow and L2-Learner-Suitable Textual Features for Supervised and Unsupervised Sentence-based Readability Assessment”
The article provides a comparative analysis of various models, algorithms and sets of functions in order to optimize the setting and assess readability based on sentence complexity. The analysis is based on data from sentences obtained from Wikipedia, a widely accessible online encyclopedia intended for a general audience. The authors took a deep look at methods to improve readability and developed improved models and new functions for future applications with low processing time.
I did not find any significant flaws in the text of the work, but I have some comments on the text.
1. Given the rather large volume of the article, it would be desirable to at least briefly describe the research plan at the beginning of the research sections, for example, ...In section 3, we describe A and B...
2. Figure 1 and figure 2, Figure 7, Figure 8 has a pretty small text to read. Maybe it make sense to make font bigger of all figures
3. Figure 1 too long description. It is good that it is described, but better to move description to text and make reference to figure, this will connect data to whole context of article.
4. The conclusion is quite "philosophical". More specifics are desirable.
The overall impression of the work and the results obtained: a high-quality article, with a professional review of the literature, clear relevance, set goals, expressed purpose, well-developed all steps. Thus, the main drawback of the work is the inadequate quality of graphic materials.
Conclusion: in general, after making corrections, the article can be accepted for publication.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsThe authors' primary objective is to classify sentences as simple or complex. In that case, it raises questions about why they utilized seven algorithms, some of which are unsuitable for clustering or classification, such as Simple Logistic. The authors should justify the chosen parameters, the number of iterations, and the achieved precision for each algorithm. Furthermore, the precision levels achieved are below 60%, indicating a 40% error rate. It is recommended that the study be reconsidered in terms of experimental design to achieve better precision in the binary classification of sentences.
It is important for the authors to make the dataset available in a repository like GitHub to facilitate reproducibility and further research. Merely providing a generic and incomplete reference (36) is not sufficient in this case.
Comments on the Quality of English LanguageModerate editing of English language required.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors addressed the majority of the comments. However, some issues still remain.
The last comment was not addressed – the paper still has semi-technical phrases. The phrase between lines 98 and 101 is a perfect example – “Section 4 is arguably the most important section of this paper because it contains a plethora of experimental tests, the results obtained from these tests, and illuminating comments explaining their significance.”
Another example is in line 357 – “In our venture to perform sentence leveled binary readability classification”. The provided examples are not singulars.
Also, despite the fact that the authors replied that they have reduced the size of the paper, the revised version has 25 pages, with 2 more compared to the first version.
Comments on the Quality of English LanguageEliminate semi-technical phrases.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper can be accepted.
Author Response
We are grateful to the reviewer for these kind comments.
Reviewer 4 Report
Comments and Suggestions for AuthorsI have observed that in their work, the authors have utilized classification and clustering algorithms without clearly distinguishing between them. They have also utilized algorithms implemented in Weka. While this is a good starting point, the authors need to improve in comparing the performance of these algorithms, particularly in showcasing the results of both the supervised and unsupervised learning approaches. Additionally, the evaluation of performance needs more metrics. Another limitation is that Weka does not support big data. Therefore, it is recommended that the authors reorganize their work and compare either clustering or classification algorithms, providing specific metrics for evaluation.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
