What Does the Data Say about Effective University Online Internships? The Universitat Politècnica de València Experience Using MOOC during COVID-19 Lockdown

Round 1

Reviewer 1 Report

The paper addresses a relevant and interesting application for the use of MOOCs in combination with formal education: the use of MOOCs to substitute company internships.

Despite the relevance, I have some suggestions/concerns about the paper:

Major points

Introduction: The authors introduce the topic and immediately start by describing the history of MOOCs. I think this should came later, as first it is important to understand what is the paper is about, why the topic is relevant, which is the gap that is addressed and what is the objective of the paper. Connected to this, the research questions should be anticipated in the introduction, as they are not related to the "materials and methods".

Moreover, the authors refer to Learning Analytics as the reference domain for their analysis. If it the case, the concept is broad as barely defined (only mentioned in lines 89-90). It would be important also to discuss why the data collected are considered Learning Analytics, as the data are basically baed on a survey design (is this learning analytics?).

Matherials and Methods: When stating the research questions would be important to connect the way in which that RQ is addresses empirically. In detail, for the first research question the authors use the proportion of "passing" students. For the second research questions, they measure the level of satisfaction. I do not believe that the level of satisfaction is a good measure of output of the effectiveness of such a program. Indeed, the perception of equivalence between MOOC and internship (reported to be 64.5% in line 314) would be a better measure. 

Results: It is not clear how students could choose the MOOCs that could have been considered as equivalent to part of an internship. Moreover, how relevant is the role of academic commission and what were the activities carried out? This should be anticipated (in the research context).

Conclusions and discussion: I do not think that the repetition of the research questions here brings value. Moreover, the academic and the practitioner contribution of the paper must be reinforced. What does this paper adds? It is an interesting application of a novel way to use MOOCs, but the contribution must be strssed in the last paragraph. moreover, how can the findings be applied to an "out-of-emergency" context? This is menetioned in line 342-348, but should be reinforced. 

Minor points:

Please be consistent in the use of Learning Analytics - learning analytics - LA

Please do not repeat exactly the same santence in the abstract and conclusions.

Authors state that the MOOC movement accelerated in 2018 (line 35). Why?

Line 77 refers to low completion rates of MOOCs for migrants. the completion rate is actually very low for all MOOCs, so this could be better discussed.

Good luck with your revision!

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

1. Introduction

"CoVid" in the title and Introduction is inconsistent with "COVID" elesewhere in the paper; I suggest you standardise on the latter.

On line 30 I suggest you remove "good", as it prejudges; perhaps replace with "possible".

This section contains a detailed history of MOOCs but minimal critical analysis of the substantial literature on their effectiveness: in particular, their poor completion and student retention rates (briefly mentioned in passing on line 60-61). Academic writing should be even-handed and fully informed in order to avoid potential bias in its selection of evidence. The paper therefore cannot be accepted for publication unless this significant omission is rectified.

There are four references to Learning Analytics but no indication of what this term means. As data from Learning Analytics is very important in this paper, it should be explained, upon first use of the term (line 74), by reference to cited literature.

2. Materials and Methods

RQ2 is straightforwardly addressed by data collected in the satisfaction survey. RQ1 is more problematic, however, and it is difficult to see how data collected through learning analytics can address the issue of how MOOCs can "cover the knowledge acquired in a company internship" (line 103). This weakness of the study must be acknowledged in the Conclusion section.

The statement (line 159) that "All data from the applications of the students that wanted to change the remaining academic credits of their internships were recorded" has not been explained. What was the sampling strategy rationale for selecting this subset?

3. Results

The results have been competently analysed and are clearly presented.

4. Conclusions and Discussion

In order to establish the validity of the study, the Conclusions and Discussion section should include some comparison with the large body of literature on students' experience of the effectiveness of MOOCs. However, much of the older literature does not go beyond easily-obtained quantitative data, so it is disappointing that the present study has not attempted to gather more fine-grained qualitative data from staff as well as students on the perceived strengths and weaknesses of MOOCs as a substitute for conventional internship. This approach would help address RQ1, which is problematic. Another significant limitation of the present study is that the respondents had no direct experience of conventional internship against which to judge their experience with MOOCs. Perhaps this research focus could shape future studies at UPV.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Thank you for your revision. 

However, I do not see some of the revisions mentioned by the authors. For example, the change to the section 2.1 mentioned in the authors' reply is not in text. 

In addition, despite I see the effort of generalizing the findings of the research in the conclusions, the introduction section is still stricly related to the emergency context. They state: The paper aims to study an emergency case in which face-to-face education was not an option and an online education option had to be adopted, to see if some findings can be applied to other situations in which similar limitations are found. However, I do believe that the findings of the study may be useful independently from a lockdown, as a way to reinforce "traditional" internships, for example by introducing mandatory pre-courses on basic skills for internships (like MOOCs on Excel or so). I think this may be expanded in the paper.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf