The Role of General and Study-Related Intraindividual Factors on Academic Learning Outcomes under COVID-19: A Cross-Sectional and Longitudinal Analysis

Round 1

Reviewer 1 Report

Thank you for the opportunity to review the manuscript.

The work submitted for review examines a topic of great relevance in the field of psychology. The topic is very important and the conceptual analysis made in the text is quite deep. The literature consulted is quite current but the sample isn´t quite large (which is a limitation for your work.). I would like to thank the efforts by the authors of the manuscript and congratulate them on the work. Overall, the writing is clear, the goals are well described, the introduction should explain the objectives of the study based on the review of the previous literature and the conclusions are properly made and presented. I consider that the constructs proposed in the abstract of the work are quite well explained. Therefore, the manuscript brings significant knowledge of the scientific literature so and still covers existing gaps in the field. On a formal level, the manuscript complies with the requirements of the Journal. The work is ambitious and the results confirm the most of the hypotheses and the relevance and potential of the work is therefore recognized, but this Reviewer considers that several changes are needed to the manuscript is publishable. In this sense, it should better explain the novelty and relevance of the work taking into account the previous empirical evidence and should better describe the practical implications. The process for selecting participants and the procedure should be better described. It should better describe the measuring instruments, providing the psychometric properties for each one. On the other hand,. It should describe the discussion and conclusions of the work better and, above all, The text must be adapted to the rules of the Journal (the citation of authors is not allowed throughout the document and the references do not comply with the required style). The format of Table 1 should be rethought, it takes up too much space in the text and the data is not clearly visible. Finally, I wish the Authors the best in continuing this line of research.

Best wishes for Authors.

Author Response

Thank you for taking the time to review our manuscript and provide such useful comments to improve its quality, as well as for the appreciation you showed for it. Following your suggestions, we carefully reviewed our manuscript, as detailed later in our point-by-point answers.

Comment: It should better explain the novelty and relevance of the work taking into account the previous empirical evidence and should better describe the practical implications.

Reply: Thank you for this comment. We now better highlight the novelty and relevance of our work both in the Introduction (see pp. 2-5) and Discussion (see pp. 12-15) sections, as well as expand on the practical implications (see pp. 2, 12, 15).

Comment: The process for selecting participants and the procedure should be better described.

Reply: Following this suggestion, we expanded on the participants’ recruitment and selection process and the study procedure (see pp. 6 and 9).

Comment: It should better describe the measuring instruments, providing the psychometric properties for each one.

Reply: Thank you for noticing this. We now better describe the measures considered providing their psychometric properties (see pp. 7-9).

Comment: It should describe the discussion and conclusions of the work better.

Reply: Thanks for bringing this to our attention. We have now articulated the discussion and conclusions more thoroughly (see pp. 12-15).

Comment: The text must be adapted to the rules of the Journal (the citation of authors is not allowed throughout the document and the references do not comply with the required style). The format of Table 1 should be rethought, it takes up too much space in the text and the data is not clearly visible.

Reply: Thank you very much for noticing this. We have now revised the reference style so as to comply with the Journal’s rules. Table S1 was wrongly put within the text but was supposed to be placed in the Supplementary materials because it takes up quite some space. We now corrected the Tables’ position in the text and hope they provide clearly visible information on the data.

Reviewer 2 Report

The manuscript is designed to analyze the role of university students’ intraindividual factors on learning outcomes under COVID-19. The authors aim to evaluate the relationship between intraindividual factors and learning outcomes as well as how these factors affect the outcomes over time. The topic developed in the manuscript is relevant and provides valuable evidence-based knowledge of how university students are coping with the challenges they face in distance learning. In my opinion, a major advantage of the study is the authors’ attempt to integrate cross-sectional and longitudinal data.

            I do have some specific comments on various aspects of this manuscript as outlined below.

1. In the introduction, the authors discuss in detail the relationship between selected intraindividual factors and learning outcomes. On the other hand, the distinction between factors and outcomes should be further clarified and highlighted. For example, self-regulated learning is classified as both an intraindividual factor and a learning outcome (e.g., lines 8–10). In addition, is it appropriate to call as ‘academic learning outcomes’ those variables that are measured by Depression, Anxiety, and Stress Scales-21?
2. There is a need to discuss whether/to what extent the results of this study are specific to a pandemic situation. That is, does the study really focus on COVID-19 (as indicated in the title), or on peculiarities of distance learning? It has recently become fashionable to use the term COVID-19, so it is important to ensure that it deserves to be added between the key terms in this study. In addition, it would be important to know whether online/distance learning is a new experience for study participants. Has such information been collected?
3. The theoretical background of the study and its results would be easier to understand for readers if some visualizations were used (for example, to illustrate a theoretical and/or empirical model as well as a research scheme). There are many variables, many scales are used to measure them, and visualizations would be really helpful.
4. To ensure consistency and integrity of results, I suggest using the same standard for their analysis (description), that is, choosing the name of the variable rather than the scale, for example, soft skills, distress level, intolerance of uncertainty instead of soft skills, DASS-21, IUS-R (e.g., Tables 2 and 3).
5. When describing the sample, it would be important to indicate how many students participated in all the waves of the study.
6. References to some tables and supplementary material appear to be inaccurate (e.g., lines 272, 273, 642).

Author Response

We thank you for taking the time to review our work and for providing us with insightful comments to improve it, as well as appreciating it. Following your suggestions, we carefully reviewed our manuscript, as detailed later in our point-by-point answers.

            I do have some specific comments on various aspects of this manuscript as outlined below.

1. In the introduction, the authors discuss in detail the relationship between selected intraindividual factors and learning outcomes. On the other hand, the distinction between factors and outcomes should be further clarified and highlighted. For example, self-regulated learning is classified as both an intraindividual factor and a learning outcome (e.g., lines 8–10). In addition, is it appropriate to call as ‘academic learning outcomes’ those variables that are measured by Depression, Anxiety, and Stress Scales-21?

Reply: Thank you for this comment. We now further clarify the distinction between intraindividual factors and learning outcomes, as well as better explain why general distress (as measured by DASS-21) was considered as an academic learning outcome in this context (see pp. 2-5).

1. There is a need to discuss whether/to what extent the results of this study are specific to a pandemic situation. That is, does the study really focus on COVID-19 (as indicated in the title), or on peculiarities of distance learning? It has recently become fashionable to use the term COVID-19, so it is important to ensure that it deserves to be added between the key terms in this study. In addition, it would be important to know whether online/distance learning is a new experience for study participants. Has such information been collected?

Reply: Thank you for pointing this out. We have now better explained the peculiarities of the pandemic situation compared to distance learning per se, so to make it more apparent to the reader what can be considered specific to the pandemic situation and what could instead be generalized to other non-pandemic contexts (see pp. 2-5 of the Introduction as well as pp. 12-15 of the Discussion). As for your question, we did not collect the information on students' previous experience with distance learning. Nevertheless, all our participants were enrolled in courses in which distance learning was not provided prior to the pandemic.

1. The theoretical background of the study and its results would be easier to understand for readers if some visualizations were used (for example, to illustrate a theoretical and/or empirical model as well as a research scheme). There are many variables, many scales are used to measure them, and visualizations would be really helpful.

Reply: Following this suggestion, we have now added a visual representation of the theoretical background applied to our study (see pp- 1-2).

1. To ensure consistency and integrity of results, I suggest using the same standard for their analysis (description), that is, choosing the name of the variable rather than the scale, for example, soft skills, distress level, intolerance of uncertainty instead of soft skills, DASS-21, IUS-R (e.g., Tables 2 and 3).

Reply: Thank you for this suggestion. For clarity’s sake, we have now made the variables’ names more consistent throughout the manuscript.

1. When describing the sample, it would be important to indicate how many students participated in all the waves of the study.

Reply: Following this comment, we now specified that in the Participants section (see p. 7).

1. References to some tables and supplementary material appear to be inaccurate (e.g., lines 272, 273, 642).

Reply: Thank you for noticing this. We have amended those inaccuracies.

Reviewer 3 Report

First I would like to thank having the opportunity to review this research study. It deals with an interesting topic and performs a deep literature review on the subject. Nevertheless, although this research has positive features, its design may undermine the obtained results. First, the sample has been taken from 331 university students that have voluntarily participated. The main issue with this is that students may be self-selected into answering, i.e., those students who answer the questionnaire may be a particular kind of student, with particular characteristics. Therefore, the obtained results may reflect the particular case of that self-selected sample, so they would not be extrapolated to other bigger sample or to the population. This, joined to the methodology employed, would mean that the obtained results are mostly correlational and not causal.

Author Response

Thank you for taking time to review our study and for providing interesting comments on how improve it, as well as appreciating it. Following these comments, we now better highlighted the study’s limitations (e.g., the possibility of self-selection bias; see p. X) and the need to avoid making causal interpretations (see p. 14), as well as reviewed the entire manuscript to make sure there are is no causality implied in the language used.

Round 2

Reviewer 1 Report

Overall, the writing is clear, the goals are well described, well-considered introduction and the results properly made and presented. Therefore, the manuscript brings significant knowledge of the scientific literature so and still covers existing gaps in the field of Education. Therefore, my assessment is positive for the publication of this work.

Firstly, I would like to thank the efforts by the authors of the manuscript and congratulate them on the work. I recognize that they have considered almost all considerations of the Reviewers. Clearly, all the comments from Reviewers have contributed to a better quality of the manuscript. I have checked in the revised manuscript are corrected the most of errors found by the reviewers, both formally and content.

Secondly, I have verified that the information is presented in a clear and organized way in subtitles. It assumes good work with great potential.

Thirdly, in the Discussion section appears practical and educational implications and future directions correctly described. I have found the manuscript show a paragraph of study limitations. However, the conclusions section is somewhat concise. It is precisely a section to highlight the main practical implications of the study.

Fourthly, I have found that all the references are correctly written. The references are quite current, and all references comply with the Journal style.

Fifthly, I have verified that the format of the figures complies with the regulations of the Journal.

Finally, considering the changes made to the manuscript by the authors and the new suggestion, I consider that the manuscript can continue with the review process, taking into account the opinion and suggestions of other Reviewers.

Best wishes for Authors.

Author Response

Thank you for appreciating the changes made during revision and for providing us with additional suggestions. Following them, we expanded the Conclusions to make them more thorough.

Reviewer 3 Report

The authors have acknowledged the limitations of the paper in their revision. Nevertheless, these limitations make the contribution of the paper very low, as previously indicated in my report: students may be self-selected into answering, so those students who answered the questionnaire may be a particular kind of student, with particular characteristics. This means that the results come from a self-selected sample, so results would not be extrapolated to other bigger sample or to the population. This, joined to the methodology employed, would mean that the obtained results are mostly correlational and not causal. Therefore, resampling in a random way would be an appropiate way of improving the research.

Author Response

Thank you for providing us with this insightful comment. After some discussion, we agreed that carrying out additional sampling may not be a feasible solution in our case, as the pandemic conditions have changed and students are now under different learning conditions (i.e., some resumed traditional learning, while others are still learning online or alternating both). Nevertheless, we agree with you that the possibility of self-selection bias would undermine the contribution of our work. Therefore, we addressed this concern more deeply in the Discussion. As you will see, we report that our results on the relationships between intraindividual variables and learning outcomes seem in line with similar studies conducted before and during the pandemic using both convenience and random sampling, which may indicate the possibility to generalize our results to other samples.