A Review of the Evaluation System for Curriculum Learning

Round 1

Reviewer 1 Report

Machine learning from “Basic Theory” to “Machine learning concepts similar to curriculum learning” has been comprehensively reviewed and exploited in the presented work. Five methods similar to curriculum learning in the field of machine learning are summarized well.

However, my concerns/suggestions are:

1. The author should present a case study (solution of some classification/regression/clustering problem) at least of Machine Learning to validate its significance.

2. In section 3.2, Table 1 includes Reinforced Learning and Direct Calculation in the non-heuristic difficulty measures; however, it has not been mentioned in the buildup paragraph to it. Furthermore, these are not explained in the subsections of 3.2. The rows of the table could also be arranged according to the subsections of 3.2.

3. In section 6, Discussion, the appropriate selection mechanism is discussed for practical applications. This information should also be given in a tabular form.

4. Figure 10 mentions eight fragment scheduling strategies, whereas related paragraphs discuss only four strategies. Some other strategies have also been discussed in brief, but their fragments have not been shown in any of the figures.

5. Some grammatical/formatting issues are:

·         please use either non-convex or nonconvex throughout the paper.

·         line 92 and line 94 may be rephrased to be more grammatically in line with the premise of line 89.

·         line no. 1061 to 1068 have formatting/flow issues.

·         MDPI format has not been entirely followed.

·         Most of the citations are from conference papers.

·         Line 127 in section 2.1 is not in line with line 126; similarly, there is a repetition of a few words in section 2.3. Sub-blocks in Figure 3 are not clear.

6. Further check if the paper falls within the scope of the electronics journal.

 

Good luck!

Author Response

Please see the attachment.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

This review paper focuses on the summary of the curriculum learning method, which is divided into three modules of difficulty evaluator, training scheduler, and loss evaluator to constitute the evaluation system of curriculum learning. It first summarizes the history of the proposal and development of curriculum learning, dividing it into four stages, and then briefly outlines three types of innovative directions for the application of curricula learning at three levels: task, data, and model. The main body of the paper compares and analyzes the three major evaluators of curriculum learning, and discusses how to choose the appropriate combination of evaluators for different domain tasks. Finally, the

differences in terminology mentioned by different authors are discussed, and approaches similar to curriculum learning in the field of machine learning are compared and analyzed.

For this paper, I have some comments as

1, the presence of unexplained method abbreviations in the convergence phase in section 2.1 (TATD-HER...) , inappropriate brackets. CILCIA)

2, The abbreviations in the lower left corner of Figure 2 need to be explained in the text or in footnotes.

3, the parameter k of the linear function in section 4.1.1 does not appear in Eq.6

4, some parameters in Eq. 12 are not given explanations (r,w,...)

5, In the machine learning concept similar to curriculum learning, the comparison and analysis of the interval repetition method and curriculum learning need to be given.

 

Author Response

Please see the attachment.

 

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have addressed most of the comments comprehensively.

One concern still needs to be addressed.

The case study is usually presented at the paper’s end (before the conclusion) once all about “Curriculum Learning” has been discussed, not in the Introduction Section. The authors are suggested to add a separate heading/Section of the Case Study and present the case study while providing performance parameters like Confusion Matrix etc.

 I believe this will add value to the paper.

 Good luck!

 

Author Response

Please see the attachment.

 

 

Author Response File: Author Response.pdf