What Sentence Repetition Tasks Can Reveal about the Processing Effort Associated with Different Types of Code-Switching

Round 1

Reviewer 1 Report

The manuscript under review describes a novel method of studying processing cost associated with different types of code-switching. The topic is appropriate for publication Languages.

Summary: The study poses an interesting question of whether the degree of structural integration of a code-switch in a sentence affects the effort required to process the sentence and whether this effort is modulated by bilinguals’ experience with different types of code-switching (as well as a host of linguistic background variables and also inhibitory skills). Overall, the study has a sound design: Bilingual participants repeated model sentences they listened to on a computer. Processing effort was operationalized as the proportion of verbatim responses to the total number of responses. The bilinguals’ accuracy scores were predicted to change with the degree of structural integration of the code-switch into the sentence. The statistical analysis could be improved; however, the outcome is reported more or less clearly: Predictions were confirmed by the data although, rather intriguingly, single-language L2 sentences produced lower accuracy scores than bilingual sentences with “loosely” integrated CS. While the group accuracy scores at least partially coincided with the groups’ scores of self-reported experience with different types of CS (dense codeswitching was the least commonly encountered and yielded the lowest accuracy score), participants’ experience scores did not predict individual variation in repetition accuracy. None of the host of background variables (related to memory, executive control, bilingual profile) could predict variation dense CS, though age and memory could be related to the other types.   

I offer my comments and suggestions to the author(s) in two forms: a few comments (mostly about the form) are included directly into the paper. Below, I give comments mostly about the content. These are organized according to the sections of the paper.

1.      Introduction

·         Working Memory is tested based on the reasoning that existing literature reveals a lack of consensus about its role in the sentence repetition task (line 188-192). The controversy hinted at in the text deserves to be developed more, especially if it is to motivate including WM as a variable.

·         It is not clear why MacDonald & Christiansen 2002 is cited in support of the sentence repetition task as a measure of processing effort (lines 125-126). Could perhaps a better reference be found?

Research questions

While I find RQ1 to be clearly formulated, I’d suggest streamlining RQ2 (see the text).

·      It is not quite clear how the author(s) view the relationship between processing effort and exposure frequency. On one hand, the cited literature indicates that processing effort and exposure frequency are viewed as connected: “Gosselin & Sabourin (2021), which found that increased lexico-semantic processing costs (N400) following code-switches only occurred in bilinguals who did not habitually engage in code-switching. Bilinguals who regularly code-switched did not incur this cost” (line 71-75). On the other hand the authors pose an either/or question RQ2a, “Is it linguistic processing effort or exposure frequency that drives performance?”

·         Considered in detail, the formulation of RQ2a (lines 208-210) is somewhat odd: "Is it linguistic processing effort [expressed as repetition accuracy] or exposure frequency that drives performance [i.e. repetition accuracy]?" Wouldn't the simpler yes/no question suffice “Does bilinguals’ repetition accuracy converge with their sociolinguistic practices, i.e. exposure frequency?”

·         Isn’t the question RQ2b partially overlapping with RQ2a in asking about the impact of exposure, i.e. “regular code-switching habits” or “sociolinguistic practices”?

2.2. Materials and Procedure

Stimuli

The author(s) should make the description of the stimuli and the procedure more detailed.

·         The author(s) should provide a complete list of the stimulus sentences in an appendix. Even without glosses and translations, the list would allow the readers to assess the elicitation instrument better. Such appendix would not extend the length of the paper unreasonably (ca 70 lines if one sentence per line).

·         While it is made clear in the text that the stimulus sentences were matched in terms of number of words and syllables, the examples in Table 1 create a somewhat different impression (15 syllables in Alternation CS vs. 9 syllables in Dense CS). Having a list of stimuli in the appendix would help to see the balance.

·         In the Alternation condition, were the speakers always switching from L1 German into L2 English as is suggested by Table 1?

·         No examples of single language sentences are given though there is the information that “sentences were created to match the code-switching sentence”.  Again, a complete list of stimuli would be useful to the reader.  

·       The text indicates that a bilingual produced 28 responses in each condition (line 291-292) but the description of the stimuli refers to participants being presented with “14 code-switches of each type … 14 single-language sentences” (267-269). Was each stimulus repeated?

·       The stimuli were presented in an audio format. Please, give information on how the stimuli recordings were created (Were the stimuli spoken by a single speaker or multiple speakers? Were they produced by an L1 German-L2 English bilingual? Including the monolingual English stimuli?

·         Dense CS:  Perhaps give Muysken's term "congruent lexicalization" in brackets? (And add a reference for Dense CS?)

o  Give a better example (and a more detailed explanation) of what was considered as dense code-switching. If in dense CS “the two languages are so interwoven that it is impossible to identify a clear switch point (Alternation) or a matrix language (Insertion) (Hofweber et al. 2019)”, then the example Wir gehen BY CAR in die Berge with a single switch from L1-to-L2 and back does not seem to qualify.

o   Please describe explicitly the structural complexity of the (dense) codeswitching stimuli instead of referring outside the text.

Sentence repetition task

A few details could be added to the procedure:

·         Did the participants listen over the headphones? (Accurate perception of the stimuli is important in the task.)

·         The reader will probably assume that the model sentences could not be replayed but it might be worth saying.

·         What was the recording device capturing the responses, what were the recording conditions?

Scoring the repetition task

·         What was the scoring method? Was each response classified on the basis of listening to the recording or were the responses transcribed?

While the binary split between verbatim and non-verbatim sentences gives a neat, simple measure, readers are likely to wonder about the sources of “errors”, i.e. the diversions from the model, in this complex task.

·         Any evidence that some errors were due to misperception?

·         It would be interesting to know to what extent the diversions involved the switch itself and how accurate encoding and repetition of the rest of the sentences was.

·         What were the departures from the model sentences in the single language mode? Errors indicating cross-linguistics interaction (in L2, or even in L1), fossilized interlanguage errors?

Validity of the task
In the sentence repetition task, the processing cost associated with switching is experienced twice by the bilingual, once during sentence perception and encoding and once during sentence recall and production. It is not clear how each aspect of the task contributes to the bilinguals’ performance. In fact, the “processing effort” reflects some overall cost of repeating verbatim a comprehended message, which may not even be attributable to the bilingual mode of the CS sentences, as the low accuracy in single L2 sentences (and the high accuracy in the Alternation and G>E insertion conditions) might suggest.

3. Results

·         First, the significance of p-values reported in Tables 3 and 5 needs to be adjusted for running multiple pairwise comparisons.

·         I also wonder about the summary of Figure 1 (lines 374-376): “In fact, Figure 1 displays a gradual increase in accuracy from Dense code-switching to Insertion and to Alternation, which may reflect the processing difficulty associated with the three types of code-switching.” Dense CS is clearly different from the other types of CS and from L1 sentences. However, G>E insertion is not different from Alternation (and neither is different from L1 German sentences). 

·         Overall, I’d suggest that the data would be better analysed in a statistical model into which the various variables could be entered together, i.e. binomial generalized linear mixed model: sentence accuracy score as Dep. Variable and Predictors including the repetition type and the frequency-of-use score; Covariates (age, …); Random factors: subject and item.

4. Discussion

·         (467-468) “Bilinguals’ performance in L2 single-language sentences did not differ from their performance in mixed language sentences.” – According to the p-values, as they are currently reported in Table 3, L2 sentences were significantly worse compared to Insertion and Alternation.

·         (497-499) “However, there was also considerable divergence for Alternational code-switching and L2 single-language sentences, suggesting that bilinguals find these types of sentences harder to repeat, regardless of usage frequency.” This formulation sounds incorrect. They found the alternations easier to repeat despite medium frequency.

·         Dense CS is found to be overall least accurate as well as least frequent; in correspondence with what is found is corpora. However, the individual variation in accuracy scores does not align with the variation of frequency scores. Speakers who reported using dense CS more frequently did not repeat dense CS more accurately. The author(s) speculate that “It is possible that the lack of an observed relationship between sociolinguistic practices and repetition accuracy is due to the limitations of the frequency judgement task in capturing bilinguals’ regular code-switching habits” (line 502-503). Playing devil’s advocate: is it possible that the bilinguals’ codeswitching habits were accurately reflected in the frequency judgement scores and the lack of an observed relationship is due to the limitations of the sentence repetition task as a measure of processing effort in dense CS?

 

 

Comments for author File: Comments.pdf

Author Response

Dear reviewer,

Thank you for your constructive comments. Please find attached our detailed responses to each point.

Kind regards

Julia

 

 

 

Author Response File: Author Response.pdf

Reviewer 2 Report

The study uses a sentence repetition task to tap into the processing of code-switching by L1-German L2-English bilinguals. Overall, I find the study to be easy to follow and well executed. I have one major comment regarding the general goal of the study, as well as a few more minor comments. Finally, I have less important formatting/copyediting comments.

 

Major Comments

-       The biggest issue I have with the paper currently is that I don’t think it spends enough time clarifying what such a method really tells us about code-switching. Repetition is not the same as production, and so the cognitive processes involved are distinct. That is to say, what is accuracy in repetition really informing us about these bilinguals? I think the comparisons between the different types of code-switching is most promising, as you are indeed comparing (more so) apples to apples, and any inherent issue(s) with forcing a participant to repeat code-switching (which is much more unnatural than say elicited production where the participant has more autonomy) is held constant across conditions, but it’s unclear what comparing it to monolingual repetition is telling us. It could simply be a task effect, since repetition of monolingual utterances is almost certainly something that these individuals have done throughout their lifetime, whereas they likely have never been asked to repeat code-switching until participating in this study. Some more time unpacking this crucial backbone of the study (both in the background and the discussion) would solidify its impact.

 

Minor Comments

-       There is general inconsistency of how the specific bilinguals are referred to throughout the document. Although it becomes clearer while reading (especially once getting to the methods section), I would suggest using L1-German L2-English bilinguals from the onset to clarify. Also, I would suggest adding the language pair to the title somewhere to best orient the reader, otherwise one could assume that the different types of code-switching mentioned are different language pairs.

-       Line 52: Almost no detail is provided about the studies that have revealed increased naming latencies at switch points, which contrasts sharply with the studies that have found no difference.

-       Line 169: The use of authentic stimuli is a great addition to the study. However, I would spend some time clarifying how authenticity is being defined. The participants are a specific subset of German-English bilinguals, so are the sources coming from the same type of bilinguals? If they are the same type, are they from the same communities? And if they are not (for either question), would that still be considered authentic?

-       Line 248: Self-reported proficiency is used to argue that they are L1-dominant. Although that may be the case, it is not a particularly strong argument to make any strong claims about dominance. Since dominance does not play a specific role in the study overall, I would just not comment on language dominance and let the self-reported proficiency information speak for itself.

-       Table 1: At first glance, this table is hard to parse. Minimally, the three different examples should be labeled within each type. Also, within the narrative below, more information regarding how the stimuli were created would be helpful. For example, it’s not immediately clear whether the alternation stimuli included only one or both directions. Similarly, it’s not clear whether the dense stimuli compared to each other – did they have similar amounts of each language or did they vary? Also, when it mentions that artificial stimuli needed to be created, no explicit information is provided to clarify that process. Among other details, how were they ensured to be authentic, since that is an important point that has been clarified for the others?

-       Table 3: This is very hard to read. Try to find a way to reconfigure it to aid the reader’s parsing. (The same goes for Table 5.)

-       Table 6: I’m not convinced this table is necessary. Since it is so dense with text, it is a mere repetition of what is said in the narrative. If this information can be summarized differently, it could be a useful aid to the reader, but as it is it seems a bit redundant.

-       Lines 467-473: This is a very good point and a really interesting finding. I would love for you to expand a bit more on this.

-       Lines 502-503: Another good point.

Copyediting and Formatting

-       There was some inconsistently with Latin abbreviations (e.g./i.e.) be used in parenthetical material or not. (See Line 41 for an example.)

-       Line 44: Separate this as a standard linguistic example with the gloss – that is, on its own line with the number (1).

-       There was some inconsistency with three or more authors, as sometimes all three were listed and sometimes everyone but the first author was “et al.”. (See Line 53 for an example.)

-       There was some inconsistency with the use of “and” vs. “&” when referring to a co-authored paper in the narrative. (See Line 55 for an example.)

-       I realize that the three types of code-switching are specific types (Alternation, Dense, & Insertion), but capitalizing each throughout the document seems unnecessary. It seems more likely that italics is preferred when introducing them, as they are technical or key terms, but then just regular (non-capitalized) would suffice for the rest of the document.

-       Lines 89-102: The descriptions of each should be in the narrative of the text rather than in the linguistic example.

-       Line 247: No need to capitalize “age of onset.”

-       Line 506-507: As you list in the full references, the reference here is actually 4 authors (and as such should be Badiola et al., 2018), not 2.

 

Author Response

Dear Reviewer,

Thank you for your constructive comments.

Please find attached our detailed responses

Authors

 

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors addressed all of my concerns except for the recommendation regarding the statistical processing of the results. Although I believe an analysis in R would be comprehensible to the readership, I respect the authors' preference for SPSS.

Reviewer 2 Report

I appreciate the work that the author(s) put in to revising the manuscript, and I do not have any further comments regarding changes.