Exploring the Effects of Early Extramural English Exposure on the Vocabulary Size of University Students

Round 1

Reviewer 1 Report

The authors present an exploratory study within the general domain of out-of-school L2 English learning. The study aims to contribute to establishing the relevance of an early starting age of engagement with extramural English for later-in-life L2 English proficiency. In the absence of longitudinal data, the authors have asked Norwegian university student research participants to make estimates and evaluations of their own relation to extramural English exposure at earlier ages. The participants also underwent a vocabulary test to establish an estimate of their current vocabulary size.

In short, this is a well-conducted and admirably lucidly presented study, with a solid and interesting approach. It is a valuable contribution to the field as an exploratory study pointing towards the needs for more study, and especially longitudinal work, on the relationship between starting age of L2 exposure, as well as the overall impact over time of L2 exposure, on learning and proficiency development. The limitations of the study as a small-scale and explorative endeavor are suitably expressed.

The authors present a straight-forward and effective statistical analysis of their survey and vocabulary test data, supporting the overall argument. Participants experience a greater impact on their English proficiency from extramural activities than from formal education, and indeed regression analysis suggests that participants’ who started earlier and can be estimated to have experienced a longer “extramural exposure time”  over their lifetime, had significantly larger vocabulary size. The latter variable, formed by subtracting participants’ ages from their reported earliest exposure to extramural English, is a clever addition to the study.

In my view, this manuscript can be recommended for publication as-is, if the editors prefer. However, I would ask the authors to consider three suggestions for minor revision:

1)        The results section is fairly short, and one gets the feeling that more exploration could be teased out of the data. For instance, is there an association between earliest exposure and perceived importance of extramural English? Is there an association between type of early exposure and outcomes (e.g. comparing audiovisual entertainment to reading).

2)        Since the discussion alludes to the possibility of a “generational shift” (p. 9, line 270), could the association between participant age and earliest age of exposure be illustrated?

3)        Lines 266–268: The authors seem to express some surprise that participant age did not itself predict vocabulary size. I do not quite see why this would be surprising, given that earliest age of exposure was variable and, and I understand it, there was no other information in the data about 'amount' of exposure per year of life. It is only if all participants had a similar starting age of exposure, or could be assumed to have an overall similar amount of exposure per year, that it would be surprising, right?

Author Response

Thank you very much for taking the time to review this manuscript and for the kind words about the study. Please find the responses to the feedback below (in blue) and the revisions highlighted in the re-submitted manuscript.

In my view, this manuscript can be recommended for publication as-is, if the editors prefer. However, I would ask the authors to consider three suggestions for minor revision:

1)        The results section is fairly short, and one gets the feeling that more exploration could be teased out of the data. For instance, is there an association between earliest exposure and perceived importance of extramural English? Is there an association between type of early exposure and outcomes (e.g. comparing audiovisual entertainment to reading).

I agree that these would be great questions to investigate further. Since almost all participants rated extramural exposure as very important, I don’t think there would be enough variation in the data to see any patterns with age of earliest exposure. I also think it would be very interesting to investigate the effects of the type of input, but unfortunately, some categories like computer games and social media were not easy to classify as either written or audio-visual. If I were to conduct this study again, I would have asked for more details about the activities. I have added this as a suggestion for future research.

2)        Since the discussion alludes to the possibility of a “generational shift” (p. 9, line 270), could the association between participant age and earliest age of exposure be illustrated?

I did do some preliminary investigations into this, but the results were not very clear and I worried it was not very directly related to the goals of the study so decided not to include this in the final manuscript. I think this too would be an interesting avenue for future research.

3)        Lines 266–268: The authors seem to express some surprise that participant age did not itself predict vocabulary size. I do not quite see why this would be surprising, given that earliest age of exposure was variable and, and I understand it, there was no other information in the data about 'amount' of exposure per year of life. It is only if all participants had a similar starting age of exposure, or could be assumed to have an overall similar amount of exposure per year, that it would be surprising, right?

Thank you, this is a good point. I have removed ‘interestingly’ and added some discussion of potential influence of the quality and quantity of the input.

Reviewer 2 Report

Review:

Exploring the effects of early extramural English exposure on 2the vocabulary size of university students

Overall comment:

The study examines the relationship between adults' early exposure to English outside formal education and their English vocabulary size in adulthood. The sample size of 40 participants, all studying English at a Norwegian university, may be small and potentially positively biased toward English usage, which could impact some of the statistical comparisons. While the manuscript shows promise, significant revisions are necessary for it to progress to a publishable paper. Key areas for improvement include achieving a more coherent alignment between sections, providing clearer definitions of the investigated constructs (such as extramural, motivation, proficiency, incidental, etc.), offering a more comprehensive acknowledgment of previous research, and implementing additional procedures within the methods of data analysis to ensure valid interpretation. While these revisions may increase the length of the paper, they are essential to address its current inadequacies. Therefore, I strongly recommend major revisions.

Detailed comments: 

Line 25. ”often referred to as extramural”.

This comment relates to this line, but also is a comment to take into consideration throughout the paper. While I understand why the author relates the study to the notion of 'extramural' English, it's important to note that this term has primarily evolved from Sweden to the Nordic countries and, more recently, to some countries in Europe. Additionally, the author credits Sundqvist (2009) as the one introducing the term 'extramural,' but the first occurrence of 'extramural exposure' actually comes from Sylvén (2006a, 2006b). Historically and internationally, terms such as 'extracurricular,' 'out-of-school,' and 'out-of-class' have been applied to the same phenomenon. Other related terms used in the field include 'OILE' (Online Informal Learning of Englsh; which with its focus on ‘online’ seems relevant in a Norwegian context where young people have had access to online English for a long time back), as coined by Sockett (2014) and Jurkovič (2019), and 'the Digital Wild,' as proposed by Sauro and Zourou, reflecting the engagement of adolescents and young adults with English in various digital contexts. (Digital Wild reflects that adolescents and young adults engage in English in plethora of ways in the digital era, also, it entails that investigations need to be thorough in asking questions that include, e.g., reading both in press/print and online/digital as evidenced in Warnby, 2022). I recommend that the author acknowledges the variety of concepts used in the field and previous researchers' contributions, and provides justification for why 'extramural' is the optimal term in this context. Additionally, it would be helpful for the author to clarify whether 'extramural' is synonymous with 'out-of-class' learning. To address any conflict regarding the introduction of the 'extramural' concept, the author could refer to Sundqvist & Sylvén (2016), where they together expand on the term, its definition, and its history. References: (Jurkovič, 2019; Sauro & Zourou, 2019; Sockett, 2014; Sundqvist & Sylvén, 2016; Sylvén, 2006a, 2006b; Warnby, 2022).

Line 28-29 “L2 proficiency and quantity or frequency of input from various extramural English activities such as reading (Busby, 2021; Day et al., 1991; González Fernández & Schmitt, 2015; Nation, 2015)”

González Fernández & Schmitt (2015) used collocation knowledge, making this the (possibly) only adequate reference. The other references are not within the author’s scope: Busby (2021) correlated extramural English with vocabulary rather than reading. Day et al. (1991) conducted an in-class experiment without using extramural/out-of-school measures, focusing instead on vocabulary gain from reading. While Nation (2015) does not directly address extramural or out-of-school measures, it highlights how extensive reading within ESL instruction can drive vocabulary learning. The author is encouraged to carefully select references that specifically address extramural English and its relationship to L2 proficiency skills. Here, there are more adequate references to use (Fisher et al., 2012; Lee, 2019; Warnby, 2022). 

Line 36. “(Busby, 2021; Peters et al., 2019)”

To enhance clarity, the authors could specify that the L2 proficiency indicator used in these studies is vocabulary knowledge. Proficiency encompasses various dimensions, with vocabulary being just one aspect. For a more recent reference that aligns closely with university students' need for academic L2 proficiency, I recommend Warnby (2022), which complements Busby (2021).

Line 46. “Incidental acquisition”

The authors should provide a deeper analysis of the concepts of 'incidental' and 'intentional' learning. Given the complexity of these terms, it's important to acknowledge the difficulty in defining them clearly. Incidental learning often occurs also within the classroom environment, where students engage with content and simultaneously acquire language skills. Webb (2020) offers valuable insights into this topic, particularly emphasizing the notion of 'meaning-focused learning'. Additionally, Hulstijn (2003) discusses the distinction between intentional and incidental learning. I encourage the author to address these complexities in defining these concepts.(Hulstijn, 2003; Webb, 2020).

Lines 85-113 The context of Norway

I appreciated reading this part since it situates well the context of the study.

Line 114. Above current study

In the background section, the author should include more theoretical arguments and clear definitions of the constructs used in the study, such as proficiency in English and motivation. Additionally, there should be some discussion on what 'time-on-task' or a related term could signify in the context of extramural English. While formal education entails countable instruction hours, extramural English is often vague and occurs in various digital contexts, referred to as the 'digital wild.' Providing clarity on these concepts will strengthen the theoretical foundation of the study.

Line 128 RQs

While the research questions (RQs) are feasible to answer, the results section does not align with them. The initial part of the results focuses on participants' beliefs regarding the factors impacting their English proficiency the most (formal education vs. extramural exposure). Subsequently, a table is presented depicting participants' motivation (however unclear) and self-reported proficiency. However, none of the RQs are addressed in these sections. It's important to reassess and clearly define the RQs and then align the results accordingly.

Lines 141-142 “who were enrolled in a course in the English program”

How long have the students been taking English courses? Additionally, is there a potential bias issue to consider, wherein students who choose to study English may already possess a predisposition towards English proficiency due to earlier linguistic exposure in life?

Lines 151-153 ” Participants were asked to report using 10-point scales how important they considered formal education and out-of-class activities to have been to their current knowledge of English.”

Has the author utilized standard measurement instruments for these questions, or were the items developed by the author? If self-produced, has there been an investigation into existing standardized and validated instruments? With reference to previous studies, it's worth exploring whether there were established instruments that could have been utilized or adapted. Additionally, considering that the students are enrolled in an English program, what impact might this have had on the study's design?

Lines 156-158 ” They were also asked to rate their agreement with seven statements (reported in Table 1) relating to motivation and proficiency on a 5-point Likert-type scale (strongly agree – strongly disagree).”

Could the author elaborate on the development, piloting, and validation processes of these items/statements? Motivation is well researched, and I am sure the author is aware of this. So, how does the author ensure that these statements are relevant to the broader constructs of 'motivation' and 'proficiency'? Clarification on the inclusion of specific components within these constructs would be beneficial to understand how the author believes these constructs can be operationalized.

Lines 161-162 ” 10 words from every thousand-word frequency band”

The author should acknowledge that sampling 10 words from a pool of 1,000 words represents a very small sample, and thus, the valid inferences that can be drawn from these 10 words are limited. Recent research suggests that approximately 30 words per frequency band may be a minimum requirement for estimating knowledge of the 1,000 words. (Gyllstad et al., 2020)

Additionally, the Vocabulary Size Test (VST) is constructed based on word families as the unit of measurement. However, research strongly recommends the use of lemma-based vocabulary tests. To name a few, Kremmel (2016) and Warnby et al. (2023) have highlighted this limitation. Therefore, acknowledging this limitation in the VST is crucial, as it decreases the valid interpretations of the test scores. (Kremmel, 2016; Warnby et al., 2023)

Line 179 ”Analysis”

In the method section under analysis, I expect a more detailed description of the data analysis. For instance, what software program was used? Also, depending on the RQs one would have expected reliability coefficients to be presented here, factor loadings form the statements, etc. 

Line 186 ” feeling that extramural activities had made a greater”

How can this be understood in relation to factors such as time-on-task? Considering that participants have undergone compulsory and upper secondary education in Norway, it's crucial to examine the extent of their formal English education. The mandatory English education in Norway typically constitutes approximately 700 hours, with an additional 280 hours for voluntary courses at the upper secondary level (maximum approximately 1,000 hours). With the mean age of participants in the study being 23 years and the mean age of first encounter with extramural English being 8.8 years, this suggests approximately 11 years of possible spare time exposure to English. Assuming exposure to English for 30 minutes a day during these 11 years would amount to approximately (0.5 hour * 365 days * 11 years = ) 2,000 hours, double the formal English instruction hours. The mean for formal education was 6.07, while the mean for extramural exposure was 8.78. If we consider that 2,000 hours over 11 years is plausible, then a mean of 8.78 does not necessarily indicate stronger value compared to a mean of 6.07 from a maximum of 1,000 formal hours. The author should further explore and carefully consider these mean values in relation to factors such as time-on-task.

Lines 197-200 ” Participants also reported perceiving that extramural activities had played an important role in contributing to their current English proficiency, and that engaging in extramural activities early in life had enabled or encouraged them to do more activities in English later. See Table 1 for details.”

This investigation lacks thorough construct delineation regarding 'motivation' and 'proficiency', which should be clearly outlined in both the background and method sections. In the results section, the author fails to revisit the construct of motivation. Additionally, it is unclear how the seven statements could be divided into two constructs (motivation vs proficiency). Furthermore, if the author intends to capture attitudes about motivation and proficiency through a survey, a validation process, such as factor analysis or assessing internal consistency, should have been conducted. Moreover, while the author states that Table 1 provides statements about experiences with extramural English, it is not apparent how, for instance, Statement 2 'I feel that extramural activities have contributed to my knowledge of English vocabulary' significantly differs from the question presented in Figure 2. Similarly, the statement 'I feel comfortable reading in English' does not seem directly related to extramural English, and the expected pattern of responses may be skewed due to the participants' English studies. I urge the author to revise these results to ensure coherence with the background section and a methodological procedure.

Lines 225-236: the regression results

This section contains several uncertainties. To enhance clarity, state that the dependent variable was vocabulary size and specify the predictors used in the regression analysis. For example, the use of age of onset to extramural English as a predictor is mentioned, but it's unclear if a multiple regression was conducted. Additionally, in the following paragraph the author mentions age of participants as another predictor (without significance) and also introduces a computed 'extramural exposure time' predictor, which lacks clarity. Consider addressing issues of multicollinearity, as the predictors may be confounded with each other. Exploring moderation could be beneficial in this case.

Moreover, while the prediction may be statistically significant, it's essential to consider the practical significance of the effect. Discussing the interpretation of R-squared is crucial; it reflects the proportion of variance explained by the predictors. An R-squared of .018 suggests a small effect size, and it's important to assess whether the association is meaningful. In a multiple regression analysis, the contribution from early extramural involvement may diminish, and the significance could be lost. The author should address these issues and provide arguments for the meaningfulness of the association.

Line 240 “Discussion”

As I have requested revisions for all previous sections, it's highly likely that the Discussion section will require significant revision. Therefore, I won't provide detailed comments on this section. I recommend revisiting the research questions (RQs), aligning them with the results section, and reintroducing them at the beginning of the Discussion along with the study's purpose.

In discussing the findings, it's important to hedge claims due to various limitations. The small sample size, the brute measure of the Vocabulary Size Test (VST), the lack of validation for the developed statements, and the crude measure of extramural activities providing little information on the quality of involvement (an important aspect, cf. Hulstijn), all limit the depth of inference. Moreover, predictors are likely confounded, and the regression shows low R-squared values, which should inform the interpretation and the discussion. Hedging needed.

Avoid relying solely on the term 'significant' and instead discuss the magnitude of the relationship using effect size standards such as Cohen's. While referencing previous research is important, avoid introducing new information in the Discussion.

Regarding the conclusion, while the study ‘highlights the importance of considering […] the cumulative exposure” (Line 356), caution is needed in claiming impact. Furthermore, the study's design may not provide insights into the quality, timing, events, or content of early extramural English exposure.

Fisher, T., Sharples, M., Pemberton, R., Ogata, H., Uosaki, N., Edmonds, P., Hull, A., & Tschorn, P. (2012). Incidental second language vocabulary learning from reading novels: A comparison of three mobile modes. International Journal of Mobile and Blended Learning (IJMBL), 4(4), 47-61. 

Gyllstad, H., McLean, S., & Stewart, J. (2020). Using confidence intervals to determine adequate item sample sizes for vocabulary tests: An essential but overlooked practice. Language Testing, 0265532220979562. 

Hulstijn, J. H. (2003). Incidental and intentional learning. The handbook of second language acquisition, 349-381. 

Jurkovič, V. (2019). Online informal learning of English through smartphones in Slovenia. System, 80, 27-37. 

Kremmel, B. (2016). Word families and frequency bands in vocabulary tests: Challenging conventions. TESOL quarterly, 50(4), 976-987. https://doi.org/https://doi.org/10.1002/tesq.329

Lee, J. S. (2019). Informal digital learning of English and second language vocabulary outcomes: Can quantity conquer quality? British Journal of Educational Technology, 50(2), 767-778. https://doi.org/ https://doi.org/10.1111/bjet.12599

Sauro, S., & Zourou, K. (2019). What are the digital wilds? UMBC Education Department Collection. 

Sockett, G. (2014). The online informal learning of English. Springer. 

Sundqvist, P., & Sylvén, L. K. (2016). Extramural english in teaching and learning : from theory and research to practice. Basingstoke : Palgrave Macmillan. 

Sylvén, L. K. (2006a, 10-11 November 2005). Extramural exposure to English among Swedish school students. Approaches to teaching and learning in linguistic research: Papers from the ASLA symposium in Växjö, Växjö.

Sylvén, L. K. (2006b). How is extramural exposure to English among Swedish school students used in the CLIL classroom. VIEWS - Vienna English working papers, 15(3), 47-53. 

Warnby, M. (2022). Receptive academic vocabulary knowledge and extramural English involvement - is there a correlation? ITL - International Journal of Applied Linguistics, 173(1), 120–152. https://doi.org/https://doi.org/10.1075/itl.21021.war

Warnby, M., Malmström, H., & Hansen, K. Y. (2023). Linking scores from two written receptive English academic vocabulary tests—The VLT-Ac and the AVT. Language Testing, 40(3), 463–843. https://doi.org/https://doi.org/10.1177/02655322221145643

Webb, S. (2020). Incidental vocabulary learning (1 ed.). Routledge. https://doi.org/10.4324/9780429291586-15

The language is largely well structured, but form and content are always closely connected so a revision of the content may be a matter of language at a conceptual and syntactical level. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

This study explored the effects of early extramural English exposure on the vocabulary size of university students. Norwegian university students’ reported exposure to English through extramural activities at an early age and how this related to their current vocabulary size in L2 English. Participants (N = 40) completed an online survey 12 comprising items from the Vocabulary Size Test (VST: Beglar & Nation, 2007) and questions about 13 the earliest extramural activity they felt made an important contribution to their knowledge of L2 14 English and the age at which they engaged in this activity. There are some comments.

1.     The VST shows 11,246 words. That is quite high. I am not sure if it is too high. Because I work in an Asian context, I have not collected any data that shows such a high vocabulary size.

1.     The authors reported measures of L2 proficiency and quantity or frequency of input from various ex- tramural English activities such as reading (Busby, 2021; Day et al., 1991; González 29 Fernández & Schmitt, 2015; Nation, 2015), watching TV shows (Lindgren & Muñoz, 2013; 30 Peters, 2018; Peters & Webb, 2018; Puimège & Peters, 2019b, 2020), playing digital games (Brevik, 2016; Coxhead & Bytheway, 2015; Sundqvist, 2019; Sundqvist & Wikström, 2015;  Sylvén & Sundqvist, 2012), and online reading/social media (De Wilde et al., 2019; Peters, 33 2018; Peters et al., 2019). The authors seem to overcite Peters’ work. It is fine, but more need to be updated. Please check the following references for updating: Incidental vocabulary learning from listening, reading, and viewing captioned videos: Frequency and prior vocabulary knowledge. Applied Linguistics Review. Doi.org/10.1515/applirev-2023-0106 Comparing incidental learning of single words and collocations from different captioning conditions: The role of vocabulary knowledge and working memory. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12910
Incidental vocabulary learning from captioned video genres: Vocabulary knowledge, comprehension, repetition, and working memory. Computer Assisted Language Learning. DOI: 10.1080/09588221.2023.2275158 Effectiveness of captioned videos for incidental vocabulary learning and retention: The role of working memory. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2023.2173613

2.     Introduction is too long to follow. Possible to move some to literature review?

3.     Pay attention to format, like (Rindal, 2013, p. 1)

4.     Make sure the research questions reflect the gap.

5.     When the literature is updated, then the discussion can be updated too.

6.     Add implications and limitations.

7.     Overall, the language is fine. But can check it again before resubmission.

It is fine. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The author has addressed many of the issues raised in the first review. I am happy with that. Given that the author says this is an exploratory study I believe that it can be enough to publish, especially as the author also has used a more hedging language in the revised ms.

If the study intends to make stronger claims there are still constructs and methods to develop.