Visualization and Data Analysis of Multi-Factors for the Scientific Research Training of Graduate Students

Round 1

Reviewer 1 Report

The paper describes a study where training of graduate students is evaluated. First, a questionnaire is designed and given to the participants. Then, analytical methods are applied to detect the most important factors. Different methods are being applied and compared against each other. The methods applied are generally suitable for the given case study. No new methods have been developed. The title of the paper is misleading. Thi sis not a visual analytics paper. The visualizations are only used to report the results of the data analytics methods. The visualizations are not used in an interactive analysis. The visualization designs are questionable and not discussed. Why is a radial layout used to create a coxcomb plot in Figure 4?  Why are bubble plots used in Figure 7? A bar plot as in Figure 3 would allow for a better comparison of the quantities in both cases. 

Author Response

Point 1: The title of the paper is misleading. This is not a visual analytics paper. The visualizations are only used to report the results of the data analytics methods. The visualizations are not used in an interactive analysis.

Response 1: Many thanks to the reviewer for your valuable comments and suggestions.  After much thought and discussion among the authors, we have finally decided to revise the title of the paper to “Visualization and Data Analysis of Multi-Factors for the Scientific Research Training of Graduate Students”.                 Also, we reorganize the overall structure of the paper. We have split and integrated Section 4 and Section 5 of the original manuscript, combining the introduction of the algorithm and visualization into Section 4 (Data Mining Methods for Graduates Cultivation) and the final results of feature selection, classification and prediction into Section 5 (Evaluation). We have also made suggestions for the three groups of graduate students, supervisors and university administrators in section 5.3 in response to the results of the analysis. The specific changes can be seen in the revised version.

Point 2: The visualization designs are questionable and not discussed. Why is a radial layout used to create a coxcomb plot in Figure 4?  Why are bubble plots used in Figure 7? A bar plot as in Figure 3 would allow for a better comparison of the quantities in both cases.

Response 2:  Many thanks to the reviewer for your valuable comments and suggestions.   As previously stated, visualization is used as an aid in the data mining analysis process to help readers understand the results of data analysis more visually and clearly. 

     There was a lack of discussion on the design rationale of the visualization view. Thus, according to your suggestion,  we have added the discussions of visual design in the revised edition, with the following changes in Section 4.1:

     In the Factor Comparison View, the factors in each module are presented in the form of a bubble diagram, where the outermost bubble represents each module, the middle the factors in this module and the innermost the options within this factor. The bubble plots enable a visual comparison of the individual factors and the percentage of options within them. The size of the bubbles is …

      Meanwhile, the "Feature Method Selection" button provides the user with the ability to combine features for different feature selection algorithms. The final combined ranking result is shown in Ranking View via a Nightingale rose chart, where the size of each sector indicates the level of importance after the combined ranking and the importance level is mapped by colour. …

    In Figure 2(f), the AUC scores for each classification method with different feature selection algorithms and original factors are shown in a radar plot. For ease of observation, we set the inner boundary of the radar plots to 0.5 and the outer boundary to 0.7 as a way of expanding the differences in the AUC values of the methods for comparison. …

Reviewer 2 Report

Thanks for the opportunity to review the work. The manuscript has an innovative contribution to the graduate education training process. Based on the survey data and model results by using data mining techniques to predict students’ academic research capabilities. The authors designed a series of visual interfaces to assist administrators to improve the quality of education training. However, the overall structure of the paper needs to be better organized and well thought out. The current structure and organization of the paper are not commonly seen as scientific papers. Also, the survey/questionnaire is the main data collection tool, all the rest heavily rely on the tool, so more details are needed for the data collection process and the psychometric properties of the questionnaire (reliability/validity) etc.

In order to better deliver the research process and findings, some of the language expressions need to be edited by an editing service or native speakers. For example:

Page 5 line 195:  I am not clear how the data was collected, such as, frequency of the data collection (one time or multiple time), how to distribute/recruit such a large number of graduate students (informed consent/IRB), what the measurement tool/questions look like etc.?

Page 5 line 199-202: what you have done with the pre-process of the data was not clear to me. You might need to rephrase it in a different way to let readers easy to understand.

Table 1: change the word “Rate” to “%”; where (what school or major) those graduate students come from?

I would suggest using “descriptive results of …” instead of “statistical results” for the title for both Table 1 and Table 2.

Table 2:“number of published paper”à “…papers”, be sure to check through the grammar

Regarding the measure of “Number of published papers”, how do you define that, time frame (per year?), does that take account of conference papers, order of authorship, and published in what kinds of journals?

Author Response

Point 1: However, the overall structure of the paper needs to be better organized and well thought out. The current structure and organization of the paper are not commonly seen as scientific papers.

Response 1:  Many thanks for the reviewer’s valuable comments and suggestions.

     After much thought and discussion among the authors, we have finally decided to reorganize the overall structure of the paper. We have split and integrated Section 4 and Section 5 of the original manuscript, combining the introduction of the algorithm and visualization into Section 4 (Data Mining Methods for Graduates Cultivation) and the final results of feature selection, classification and prediction into Section 5 (Evaluation). We have also made suggestions for the three groups of graduate students, supervisors and university administrators in section 5.3 in response to the results of the analysis.

    We believe that the reorganized structure of the paper is a good composite of the standards of a scientific paper. You can see the details of the changes in the revised version.

Point 2: Also, the survey/questionnaire is the main data collection tool, all the rest heavily rely on the tool, so more details are needed for the data collection process and the psychometric properties of the questionnaire (reliability/validity) etc. In order to better deliver the research process and findings, some of the language expressions need to be edited by an editing service or native speakers. For example: Page 5 line 195:  I am not clear how the data was collected, such as, frequency of the data collection (one time or multiple time), how to distribute/recruit such a large number of graduate students (informed consent/IRB), what the measurement tool/questions look like etc.? Page 5 line 199-202: what you have done with the pre-process of the data was not clear to me. You might need to rephrase it in a different way to let readers easy to understand. 

Response 2:  Many thanks for the reviewer’s kind reminders.

     The errors in the paper above have been revised. Furthermore, we have performed a thorough proof-reading again on the original manuscript and the revised manuscript has improved a lot.

    In addition, in response to your questions about the whole process of the questionnaire, including design, distribution, collection and statistics, we would like to provide you with the following response, which has been added in the revised version in Section 3.1:  This questionnaire was designed by professionals from the graduate school of a university in Zhejiang Province, China, and its design was reasonable. It was distributed to all grades of graduate students (majors not counted) enrolled in the university at that time, and 677 questionnaires were collected in a timely and effective manner.

     In addition, by pre-processing the collected questionnaire data as described in section 3.1 of the manuscript, we finally obtained the statistics in Table 1 and Table 2. The specific questions in the questionnaire were abstracted into influencing “Factors” and presented in tables with “Values” to express their options.

Point 3:  Table 1: change the word “Rate” to “%”; where (what school or major) those graduate students come from? I would suggest using “descriptive results of …” instead of “statistical results” for the title for both Table 1 and Table 2.

Response 3: Many thanks for the reviewer’s kind reminders.

     We have changed the word “Rate” to “PCT”, which represents “percentage” in Table 1 and Table 2. Also we changed the titles of Table 1 and 2 as the review suggested.

Point 4:  Table 2:“number of published paper”à “…papers”, be sure to check through the grammar. Regarding the measure of “Number of published papers”, how do you define that, time frame (per year?), does that take account of conference papers, order of authorship, and published in what kinds of journals?.

Response 4: Many thanks for the reviewer’s kind reminders.

For the definition of the number of published papers, we only consider the papers that graduates have participated in and published during their studies (including conference papers), without taking into account the order of authorship and published in what kinds of journals.