Take-Home Exams in Higher Education: A Systematic Review

Round 1

Reviewer 1 Report

An interesting article that helps to synthesise the research on Take-Home Exams (THEs) in Higher Education. My overall comment is that the content of this manuscript would benefit from being rewritten in places to improve clarity. Specific comments and suggestions are provided below:

Clarify that this systematic review is a qualitative review as opposed to a quantitative review, i.e. the process of reviewing the papers is based on identifying the prevalence of themes. Related to this, explain how the themes were chosen and coded from the papers so that the study could be replicated.

Introduction: Row 19-32, interesting background on the role of ICEs and THEs - would be useful to unpack some of the research cited. For example, explain why ICEs are not suitable for assessing performance on higher-order levels of Bloom's taxonomy. 

Specify which subjects were included in the review. This would give the reader a better sense of the focus of the review. In addition, which students in HE were the focus on this review (e.g. only undergraduates or undergraduates and postgraduates etc.).

Research question 3 is concerned with whether THEs are only appropriate for certain levels of Bloom's taxonomy, but the statement which follows is that no research has addressed this issue. Therefore, should this be kept in as a research question as the research does not address this question directly.  

The Discussion section is quite long and would benefit from sub-headings to navigate readers to the key themes. Also, unclear how the recommendations follow from results of the systematic review.  

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Manuscript ID            : Education-609901

Type of manuscript    : Review

Title                            : Take-Home Exams in Higher Education: A Systematic Review

 

Comment

The paper conducted a systematic review on publications on take-home exams (THE) in higher education that have been applied only for some soft disciplines. The main objective of this paper was to peruse if the THE can be considered as an alternative to the in-class exams. Using the searching method from six academic databases, the paper filtered 168 THE-related publications but only 35 publications answered most of the paper’s research questions. The findings indicated that take-home exams may be the preferred choice of assessment method on the higher taxonomy levels rather than the lower ones because they promote higher-order thinking skills and allow time for reflection. Some recommendations are made such as applying THE to “hard” disciplines like STEM and THE should be used in a proper manner to both promote and assess the highest taxonomy levels and foster higher-order cognitive skills

This paper is interesting to discuss a sensitive matter like take-home exams in tertiary education environment. The paper has been highly evaluated by searching and reviewing quite large number of publications from common journal databases to address research questions. The paper can be potentially published if the following points should be perused in a relevant manner.

In the introduction section, more comprehensive information is needed to discuss about the practical context of take-home exams in universities. What is motivation for your paper? What is difference between in-class exams and take-home exams, and between take-home exam and assignments? A comparison between two types and even with no exam at all are necessary to support your argument on why you want to conduct research on THE. Your research questions need to be backed up by the literature review. Why you can come up with proposed research questions. Once this can be done, your paper quality would be much improved. You will need to explain Bloom’s taxonomy scale in detail. Why universities must use this scale for their curricula. This is also similar to higher-order cognitive skills (HOCS. These concepts need to be clarified for readers who are not familiar with learning and teaching discipline. In the methodology section, you should mention about scope of publications. That is, what countries did papers come from and whether you take care of tertiary education types (universities vs colleges) or not. Note that learning and teaching methods are different for each nation. In other words, “one size fits all” is not applicable in tertiary education for all nations. Good to see that you summarised advantages and disadvantages of THE in Tables 3 & 4 to answer research questions 1 & 2. However, rather than just listing key points from different publications, you should discuss more about the method of analysis used in these publications to draw conclusions about advantages/disadvantages of THE and if these findings have been validated. This is important to reliability of your findings and provides strong evidence for your recommendation to further studies. The same structure should be applied to other research questions. You should combine sections of conclusions, recommendation and suggested research and authors’ comments into one section, say Conclusions and recommendations, make it shaper and avoid overlap in your ideas. General questions without answer should be avoided. Instead, you should convert them into statements. For example, line 306-307, page 1 or line 374, p. 14. Please check throughout the paper. Check citation in text and the list of references to ensure that all is consistent. For example, at line 279, p. 12, “Marsh ….”, the year, page number should be added after the author’s surname. Editing the whole paper (English grammar, punctuation and sentences) is highly recommended.

 

Thank you and good luck with your publication.  

 

 

 

 

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

MS #: education-609901

Title: take-home exams in higher education: A systematic review

The current research provides a review of the effects of take-home exams (THEs). The review focused on advantages/disadvantages, how to mitigate risks, study habits leading up to exams, and levels of Bloom’s taxonomy/higher-order cognitive skills of students. In general, the authors conclude that opinions about advantages, disadvantages, and risks are consistent across the literature, and that more work needs to be done to further understand memorial consequences of THEs.

In general, I find some statements throughout the manuscript to be a bit too bold. One notable example is on p. 13, “This review has shown that THEs should probably not be used on the lowest levels of Bloom’s taxonomy scale.”  I’d argue that the current review didn’t actually provide evidence that THEs are worse for lower versus higher levels of the taxonomy—indeed, early in the discussion the authors note evidence for retention following THEs versus ICEs is limited. A preference or belief that THEs are better for higher levels of Bloom’s taxonomy and/or that THEs promote higher-order cognitive skills is not the same as empirical evidence; I think this distinction unclear at various points throughout the review. I would encourage the authors to revisit the manuscript and soften their language where appropriate.

Parts of the results section seem a bit brief in that very little prose is provided. The authors should consider adding a bit more discussion so that the reader isn’t required to do the heavy lifting of extracting relevant/important information from all of the tables.  

The authors may want to consider using fewer acronyms throughout the review. Although using acronyms as a shorthand can be helpful at times, when too many are used, it can increase cognitive load on the reader. Early on in the paper, I found myself having to reread sections to remind myself what all the acronyms stood for (i.e., THE, OBE, CBE, ICE, HOCs).

Overall, I was left feeling a bit underwhelmed by conclusions of the current work. However, the lack of empirical evidence concerning certain aspects of THEs is no fault of the authors, and I think the authors provide a sufficient overview of the current state of the literature. Pending that my comments above are addressed, I would be favorable toward the review being published. I hope the comments are helpful to the authors.

Author Response

Please see attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

Dear Authors,

Thanks for your sterling efforts to revise your paper in a short time.

You have addressed almost questions raised. That's great.

Just a minor point that you should addressed is a critical analysis of Table 3. You should clarify about the Validation: NO, YES, Hypothesis testing. What is hypothesis testing? case study and reflection?. Brief explanation should be good to make your paper better.

Please check carefully the grammar, sentences, and wording.

Good luck with your publication.

 

Author Response

You are right. I have added explanations to the concepts introduced in table 3 and explained how I have classified the works as “Validated”. See rows 183-195.