Exploring Correlates of Student Preferences for Virtual or In-Class Learning among Neurodiverse Adolescents Using a Single-Case Design Methodology

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript relates the preference for the types of teaching (in-person, virtual and mixed), personality and commitment to learning of a sample from a school with a neurodiverse population. To do this, data is obtained through an adhoc survey on teaching preferences, a personality questionnaire and another on commitment to learning. The authors propose five hypotheses and present the results of the descriptive statistics, some contrasts of means and parametric correlations. The main conclusions about the relationship between the variables are presented.

 

Limitations

-  Introduction

-  Results

-  Very limited and unbalanced sample

 

Strengths

- Conclusions

 

Aspects to improve:

Qualification

1. It is too long, and should be shortened, in fact it includes the three variables of the study. The title should be about preferences in the types of teaching

Keywords:

2. Include personality instead of the big five,

3. eliminate covid19,

4. eliminate educational technology,

5. student learning preference is ambiguous and not very clarifying, it should include virtual and in-person teaching

 

Theoric introduction

6. citations (and references) do not conform to the journal's standards (Chicago system) instead of APA that the authors use

7. The authors frequently use the first person plural, when the correct thing is to use the impersonal form (We recruited, we discuss...)

8. paragraph 1 (lines 22-31) the idea of covid 19 is not central to the manuscript, but rather the cause that provokes the teaching modalities (main idea of the paragraph), but it is repeated three times

9. Paragraph 2 (33-45) seems to relate the type of teaching to personality, but the description of any other research is not included, although it is cited. This paragraph includes many unrelated ideas and the purpose is not clear.

10. Paragraph 3 (46-65) continues with the same idea, but includes academic performance, which is neither objective nor variable in the study, this idea should be eliminated

11. Paragraph 4 addresses engagement in learning and the same applies, as the information on lines 74-75 should be deleted.

12. Under the heading “statement of the problem” information on the variables is included, which could have been included in the previous paragraphs and this section omitted.

13. The research cited in each section must be described, only some conclusions of the studies are reported without delving into these researches.

14. Line 96 includes a prediction by the authors as a hypothesis, which should be included in the hypothesis section, and not in that paragraph.

15. Although the study focuses on personality type, each of them is not explained in depth.

16. Also includes non-pertinent information in the paragraph (line 58-62), since that information is included in the method

The current study

17. The objective and hypotheses must be clarified. The neurodiverse conditions of the students is a characteristic of the sample, and must appear that way in the objective

18. It could be: “The objectives of this study were to explore the relationships between their perceptions of varied learning models (in-person, online), personality traits, and their degree of learning engagement in neurodiverse students.”

19. The hypotheses are not ordered or clearly related to the objective of the study, 1 and 4 are related to learning commitment, and 2 and 3 to the type of teaching. Hypothesis 3 is not understood, since it relates the characteristics of the sample (the presence of a disorder) with the preference for a type of teaching.

Method

Participants

20. The participants are described, although the sociodemographic characteristics are not reported: sociocultural and economic level of the families

21. A relevant limitation is that the source is self-reported about his own diagnoses, which must be resolved somehow. For example, requesting reports from students who finally answered the questionnaire, and comparing the percentages of self-reported and objective diagnoses. It is not clear if there are students without ADHD or ASD, or if all students had some type of disability.

This variable is used as independent when it is not certain that these data are real.

Measurements

22. The learning preferences variable is ambiguous and must be specified. The questionnaire used should be better described, providing information on validity and reliability. The use of the closed question (lin 194) is not understood if it has already been asked before in liker type.

23. Acronyms should be stated before use, it appears that E stands for Engagement. It must be stated explicitly. The same happens with D.

24. It is not indicated that it is Conbrach's Alpha and it must be explained in the same way in all variables (eliminate zero before the point), the appropriate thing would be in parentheses and with the symbol, for example: (a = .78)

Data analysis

25. This section is appreciated, but this order is not followed in the development of the results

26. It must be indicated which statistical tests have been followed to respond to each objective/hypothesis.

Result

27. This section must be rewritten and given clarity, according to the data analysis section

Response rates

28. The response rates section must appear in the description of the sample, since it is limited to the subjects who have responded.

Descriptive statistics

29. In the “descriptive statistics” section, percentages of choice are presented, which does not add information to the Likert question that appears below and in table 2

30. Personality and learning engagement statistics are not commented on in the paper and do not seem to be relevant.

31. The means of both variables could have been presented according to the students who prefer virtual/in-person teaching, and it would have provided relevant information.

32. The means must be compared using some contrast statistic, even if it is a non-parametric test (kruskal Wallis)

33. Table 3 does not understand its elaboration and meaning

34. It seems that line 249 indicates table 2, when it refers to table 3.

Intercorrelations amogs the variables

35. Under this heading, the authors do not provide this information but rather contrast the hypotheses

36. In this section you must comment on the correlations of each pair of variables, and then, in the conclusions, respond to whether or not the hypotheses are met.

37. In each hypothesis they mix the contrasts of means and correlations

Discussion

38. This section should comment on whether or not the hypotheses are accepted, and the relationship with previous research.

 

References

39. It does not comply with the journal´s standards

Comments on the Quality of English Language

it´s ok

Author Response

 

Thank you for your prompt and instructive review.  Collectively, we were impressed by the reviewers’ speed and thoughtful comments.   We agree with reviewer concerns about our restricted sample size, and we have removed several analyses and added a non-parametric analysis.  We have highlighted the explorative nature of this work and have briefly emphasized the importance of our neurodiverse student participants with their extensive pre-pandemic exposure to computer-/internet-based learning.  Of note, our focus on this one small school stems directly from its exceptional investment in empirically based internal decision-making and planned data sharing.

 

You will find this revision to be an appreciable change from the first draft.  The writing is now in in third person impersonal (rather than first person), we attempted to place references in Chicago style, we focused directly on relevant variables and embellished the pertinent literature review where prior literature existed.  Point-by-point responses to all three reviewers follow:

 

Aspects to Improve

 

• The paper is now shorter; and its title is narrowed at the reviewer’s suggestion.

Key Words

• “personality traits” has been substituted for “Big 5,” “COVID-19” and “educational technology” was deleted, and both  “virtual versus in-class learning,” and “neurodiversity”  were added.
• We converted all reference citations to the Chicago system (away from APA).
• Language style is now third person impersonal vs. first person plural.
• COVID-19 is de-emphasized in paragraph 1.
• In paragraphs 2 & 3, the research question is addressed more directly and excessive information has been deleted.
• The “statement of the problem” section has been deleted and reorganized into earlier paragraphs. Sub-headings address the possible correlates of learning mode preferences: personality traits and learning engagement.
• In the section on personality traits and learning preferences, we believe it is important to retain prior literature on personality traits and academic achievement, because there is no prior literature relating personality correlates with learning mode preferences and we think academic achievement may be indirectly relevant.
• We added additional citations related to personality assessment with the Five Factor Model.

 

The Current Study

 

• At the reviewer’s suggestion, we limited an introductory discussion of our hypotheses to the “Current Study” section, and we simplified our hypotheses to just three clearly ordered hypotheses (and a description of their exploratory nature). We agree that neurodiversity and exposure to computer-/internet-based learning are sample characteristics, not measured variables. Hence, we removed any hypothesis referring to them.   Descriptive information about the sample has been moved to the Participants section. We deleted any attempt to evaluate whether a clinical disorder is related to learning preferences, agreeing that this analysis was flawed a self-reported diagnosis and too few students with this disorder to analyze this subgroup. 

 

Method

 

• Unfortunately, we have no further sociodemographic information about the participants’ economic levels to report.
• We now acknowledge, in the Limitations section, that self-reported diagnoses give only an approximation of the incidence of these diagnoses in this sample. While there were students who self-reported neither ADHD or ASD, we neglected to specify other neurodevelopmental problems (e.g., learning disability (LD), developmental coordination disorder (DCD) m Tourette’s disorder, etc. and cannot give the number in these categories. The Head of School has reported that all participants can be characterized by some neurodevelopmental difficulty, and we are confident in our wholistic characterization of this sample as “neurodiverse.”

 

Measurements

 

• Our questionnaire on learning preferences was created specifically for this study in the absence of any similar tools in the educational literature. The formats of the questions we used are not well suited for reliability analysis with the Cronbach alpha, and there are no comparable tools for determining criterion validity.  We added Cronbach symbols for the Five Factor Model personality assessment instrument the learning engagement instrument we used in this study.
• E v D are other authors’ specific acronym for the learning engagement questionnaire we used; those letters are now more clearly explained in that context when this test is introduced.

 

 

Data Analysis

 

• At the reviewers’ suggestion, we consistently ordered our presentations of analyses in both Data Analysis and Results sections to match our revised hypotheses in the Introduction, and we follow the same succinct order in the Discussion.
• There are now clear links in the Data Analysis section between statistical tests and objective hypotheses.

 

Results

 

• This section has been rewritten to match the order of hypotheses in the Introduction and to synchronize with data analysis descriptions.

 

Response Rates

 

• Data regarding participant response rates were moved from the Results to the Participants section as requested.

 

Descriptive statistics

 

• At the reviewer’s recommendation, we removed the descriptive statistics table for students’ responses to the three questionnaires (learning preferences, Five Factor Model of personality, and learning engagement). We briefly presented only the data for student learning preferences in the text instead.
• At the reviewer’s suggestion we ran non-parametric ( Kruskal Wallis) tests to identify any significant differences between the students’ scores on both the FFM-APQ and the student engagement measure and found that the only significant relationship was between choosing to learn in-person/in-class and showing higher emotional engagement.
• To better explain what is now Table 2 (Rank Ordered Preferences for Modes of Learning), we now describe that this table presents results of a question students answered, asking them to rank order 6 possible iterations/combinations of virtual and in-class learning modes that have evolved from the COVID-19 pandemic. We developed a means of summarizing the entire group of participants by weighting the frequency of high and low ranks given to each mode iteration, and these are listed in Table 3 most popular to least popular choice.  Table 3 makes clear that students most preferred having “all students always in-class,” showing that even when given other options (and not forced to choose between virtual and in-class) students most preferred in-class.  Hopefully, this Table is now better described, and the typo that inadvertently named Table 3 as Table 2 has been corrected.

 

Intercorrelations among the variables

 

• There is now a paragraph describing the intercorrelations table under the intercorrelations heading. We then separately discuss the study’s three hypotheses more clearly than before, with no mention of mean scores in this context.

 

Discussion

 

• We have re-written the Discussion section to give a more pointed summary of tests we ran and results as they related to our hypotheses.

Reviewer 2 Report

Comments and Suggestions for Authors

-The sample of 27 students is very small and insufficient to conduct descriptive research.

- Detailing the characteristics of the sample is unnecessary, as long as it is not used in the results of the study (measuring and interpreting hypotheses).

- The study’s methodology is the descriptive, correlational method, according to what was stipulated in the study’s hypotheses, and it was not limited to a survey research design only.

- Participation in learning: Cronbach’s alpha value for the behavioral detachment subscale (0.50) is very weak.

- Honesty coefficients were not calculated for the study questionnaires.

- The moderate distribution of the data was not calculated, on the basis that the sample is small (less than 30 individuals).

- On what basis was parametric statistics (Pearson, t-test) applied?

- It is necessary that the tables be attached to the presentation and analysis of the results, and not placed as appendices at the end of the manuscript.

- The result of the study, which found that a preference for virtual learning was linked to an increase in social anxiety, and that it was linked to a diagnosis of autism disorder, is not appropriate.

- The researchers’ declaration of the restrictions is sufficient to disqualify the manuscript for publication.

Author Response

 

Thank you for your prompt and instructive review.  Collectively, we were impressed by the reviewers’ speed and thoughtful comments.   We agree with reviewer concerns about our restricted sample size, and we have removed several analyses and added a non-parametric analysis.  We have highlighted the explorative nature of this work and have briefly emphasized the importance of our neurodiverse student participants with their extensive pre-pandemic exposure to computer-/internet-based learning.  Of note, our focus on this one small school stems directly from its exceptional investment in empirically based internal decision-making and planned data sharing.

 

You will find this revision to be an appreciable change from the first draft.  The writing is now in in third person impersonal (rather than first person), we attempted to place references in Chicago style, we focused directly on relevant variables and embellished the pertinent literature review where prior literature existed.  Point-by-point responses follow:

 

• While we agree that our sample is small, we believe that strong trends regarding student learning preferences and statistically significant findings within these novel data and our unique sample warrant publication. We have now emphasized the explorative nature of and limited analyses to correlations and non-parametric significance testing. 
• Details regarding our sample characteristics are relevant to unexpected findings of a strong in class learning preference even among students were highly skilled with online learning in the pre-pandemic period.
• We agree with the reviewer that the Cronbach’s alpha value for the Behavioral Detachment subscale (a = .50) is very weak; accordingly, we decided not to report or interpret any correlations with this measure.
• We assume the reviewer’s reference to Honesty Cronbach alpha coefficients relates to some researcher’s arguments for adding Honesty as a sixth personality trait in the Five Factor Model for the Big Five personality traits. Our study was based on the Five Factor Model without Honesty, and the adolescent questionnaire we used (FFM-AQT) to correlate the Big Five personality traits with other variables of interest in this study did not include Honesty.  Accordingly, we have no Honesty coefficient to report.
• Tables are now shown within the manuscript, rather than as appendices.
• We agree with the reviewer that our data to not meet an assumption of normality in the data distribution; we eliminated an earlier t-test report for differences between students who preferred in-class or virtual learning modes.
• We have reframed these data as exploratory and explained their value, despite their restrictions.

Reviewer 3 Report

Comments and Suggestions for Authors

The authors raised an interesting topic regarding the relationship between personality traits, school involvement, and the preferred form of classes (online vs. in the classroom) about a vital population of neurodiverse adolescents. Unfortunately, a few issues related to the sections: theoretical background, statistical analysis, and conclusions temper enthusiasm and trust in the results presented. 

The authors could provide a more precise theoretical background by defining the main theories they use in their research. That would help readers better understand the concepts and variables being studied. For example, concerning the Five-Factor Model Adolescent Personality Questionnaire (FFM-APQ) tool used, the authors do not explain how personality is understood by Rogers and Glendon (2018).

The description of the results overlaps too much with the information presented in the tables (duplication of information). Moreover, the t-test analysis raises particular concerns: (1) there is no mention of the verification of assumptions, and (2) comparing a group of 23 with only 3 students, even to identify a trend, appears to be an abuse of the statistical method. Also, the selection of the sample and its size do not allow the obtained results to be treated as valid and reliable.

After carefully considering methodological and statistical concerns, the discussion, limitations, and conclusions are not critical enough.

Perhaps a worthwhile solution would be to expand the research group to include students from other schools with a similar level of preparation for conducting online classes.

References to anecdotal reports appear far too often throughout the text.

Therefore, the article still needs to be thought over and refined. 

Comments on the Quality of English Language

The article requires proofreading.

Author Response

 

Thank you for your prompt and instructive review.  Collectively, we were impressed by the reviewers’ speed and thoughtful comments.   We agree with reviewer concerns about our restricted sample size, and we have removed several analyses and added a non-parametric analysis.  We have highlighted the explorative nature of this work and have briefly emphasized the importance of our neurodiverse student participants with their extensive pre-pandemic exposure to computer-/internet-based learning.  Of note, our focus on this one small school stems directly from its exceptional investment in empirically based internal decision-making and planned data sharing.

 

You will find this revision to be an appreciable change from the first draft.  The writing is now in in third person impersonal (rather than first person), we attempted to place references in Chicago style, we focused directly on relevant variables and embellished the pertinent literature review where prior literature existed.  

• We have provided a fuller description of the Five Factor Model of personality that is the basis for Rodgers and Glendon’s adolescent test for the Big Five personality traits.
• The paper has been rewritten and reorganized to better address three main hypotheses, and reframe the data as exploratory but useful for their strong trends in a small, rare, and valuable adolescent sample of neurodiverse students. We removed group comparison testing of participants with and without Autism Spectrum Disorder (ASD).
• We have removed references to anecdotal reports.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Revision 2.

The authors' efforts to incorporate the reviewers' suggestions are appreciated, increasing the scientific quality of their writing.

There are still some questions:

- Errata or texts from the previous version appear (line 201)

- Neither the citations nor the references conform to the Chicago format

- There are comments from the authors in the final text about this, which must be resolved

Author Response

The authors' efforts to incorporate the reviewers' suggestions are appreciated, increasing the scientific quality of their writing.

We appreciate reviewer one’s feedback on the improvements to the manuscript.

There are still some questions:

- Errata or texts from the previous version appear (line 201)

We have made every effort to remove errata or misplaced texts in the current version.

- Neither the citations nor the references conform to the Chicago format

In-text citations and references have been checked for Chicago format and adjusted accordingly

- There are comments from the authors in the final text about this, which must be resolved

Thank you for pointing this out. These comments have been resolved.

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

- The researchers did not provide sufficient justification for the sample of 27 students, which is too small and insufficient to conduct descriptive research.

- There is confusion in the study methodology between the exploratory study and the correlational study.

- Honesty coefficients were not calculated for the study questionnaires.

- The researchers mentioned that non-parametric statistics were adopted by the Croxall-Wallis test, but it did not appear in the study results (tables).

- The Pearson correlation coefficient belongs to parametric statistics, and is not suitable for application in the statistical treatment of the study (Table 4).

- Finally, I have reservations about accepting the manuscript for publication.

Author Response

- The researchers did not provide sufficient justification for the sample of 27 students, which is too small and insufficient to conduct descriptive research.

 

We understand the concern about small sample sizes. However, we still believe this manuscript has the ability to contribute to the literature. Therefore, we have added information to the Participants section about the fact that we actually got participation from 67.5% of our Upper School students. In addition, with the guidance of the Editor, we have reframed this paper as a case study.

 

- There is confusion in the study methodology between the exploratory study and the correlational study.

 

A study being exploratory does not preclude from using a correlational framework. However, this study has been reframed as a case design exploring the correlations among the variables of interest.  

 

- Honesty coefficients were not calculated for the study questionnaires.

 

None of the members of our research team had ever heard of Honesty Coefficients, so we reached out to the editor for guidance. They suggested that what was meant here was reliability coefficients. Cronbach’s Alpha values for the study questionnaires are included in the Method section where each questionnaire is described.

 

- The researchers mentioned that non-parametric statistics were adopted by the Croxall-Wallis test, but it did not appear in the study results (tables).

 

Based on the advice of Reviewer 3, we have eliminated Kruskal-Wallis Tests, which resolves this issue.

 

- The Pearson correlation coefficient belongs to parametric statistics, and is not suitable for application in the statistical treatment of the study (Table 4).

 

Thank you for this recommendation. We have recalculated our correlations using the non-parametric Spearman correlations and included the new results in Table 4.

 

- Finally, I have reservations about accepting the manuscript for publication.

 

We understand your reservations but still believe our manuscript has merit, particularly in light of our unique sample. The fact that we found significant results even with such a small sample size illustrates the power of these relationships and merits publication.

 

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

I appreciate the changes made. However, I believe that a single study on a sample of 27 students, in which the authors continue to compare groups of 23, three and one (!), instead of giving up that idea, is not sufficient even at the exploratory level.

Comments on the Quality of English Language

I have no comments. 

Author Response

I appreciate the changes made. However, I believe that a single study on a sample of 27 students, in which the authors continue to compare groups of 23, three and one (!), instead of giving up that idea, is not sufficient even at the exploratory level.

 

We respectfully disagree that our sample is insufficient for exploratory analyses. We have changed our analyses to non-parametric statistics to acknowledge our unique sample. In addition, we have removed any group comparisons due to the vastly unequal group sizes. However, we still strongly believe that examining these issues in a sample of neurodivergent students, regardless of sample size, merits publication. With the guidance of the Editor, we have now reframed the study as a case design.

Author Response File: Author Response.pdf