The Role of AI-Assisted Learning in Academic Writing: A Mixed-Methods Study on Chinese as a Second Language Students

Abstract

This mixed-methods study examines the role of artificial intelligence (AI)-assisted learning in academic writing for Chinese as a Second Language (CSL) students in a Chinese university context. Fifty international CSL students were randomly assigned to experimental—AI-assisted learning using ChatGPT—and control—traditional learning—groups. Writing samples from the participants were evaluated using established scoring rubrics for Chinese academic writing. Based on pre- and post-test quantitative data and supplementary qualitative interviews with six participants from the experimental group, this study reveals that AI-assisted learning can enhance student outcome by supporting knowledge acquisition, helping to create a supportive learning environment, and increasing student motivation. However, this study also highlights concerns regarding over-reliance on AI, particularly in relation to ethical concerns, technical and networking issues, and the unreliability of AI-generated content. These findings contribute to a nuanced understanding of the impact of AI on CSL learners’ academic writing performance. Finally, we also discuss practical implications for educational stakeholders regarding the integration of AI into language education.

1. Introduction

Over the past two decades, the teaching and learning of Chinese as a Second Language (CSL) has seen a significant increase in popularity around the world (Gong et al., 2018, 2020; Ma et al., 2017). An increasing number of international students are studying in China, and academic writing holds a pivotal role in the language development of CSL learners and in broader academic outcomes for international students in the country. It is essential for CSL learners to enhance their academic CSL writing skills in order to achieve academic excellence, effectively communicate their opinions, and articulate their thoughts in the context of a Chinese university. Academic writing in a foreign or second language (FL/SL) is more structurally elaborate and explicit than speech, which presents greater challenges to students (Biber & Gray, 2010). It requires proficiency in various aspects of language development, including writing organization, coherence, grammar, and vocabulary (Campbell, 2019). In this regard, the academic writing process for students often requires constant monitoring and insightful feedback from teachers, demanding significant time, effort, and consideration of subjectivity (Yu & Lee, 2014).

A desire to enhance students’ academic writing has prompted educators, researchers, and policymakers to explore innovative methods for enriching learning environments and providing effective instruction (MacIntyre et al., 2019). A range of recent digital technologies and online resources, including artificial intelligence (AI)-assisted tools, have been adopted to provide interactive and personalized instruction to improve students’ academic writing practices and skills (Jiang, 2022; Yan, 2023). Recognizing the transformative potential of AI, language teachers and researchers are increasingly embracing it as a tool to support students’ development of academic writing skills (Ji et al., 2023). With the increasingly widespread availability of AI technology and tools, students can now conveniently access these tools anytime and anywhere, with minimal physical constraints (Yan, 2023).

A key emerging research gap in CSL academic writing relates to the role and effectiveness of AI-assisted learning in improving international students’ writing proficiency and motivation (Jiang et al., 2024; Li et al., 2023). Thus, the current study aims to explore international students’ perceptions of AI-assisted learning in terms of their CSL academic writing and provide insights and guidance on the application of AI in the context of language education, especially in the context of CSL learners’ academic writing.

2. Literature Review

2.1. Academic Writing and Its Challenges

Writing is a cognitive process involving the generation of ideas and thoughts their arrangement into a text (Nunan, 2003). This process is commonly recognized as a demanding task for FL/SL learners, who may struggle to address both academic goals and communicative purposes (e.g., Lin & Morrison, 2021; Teng & Ma, 2024; Pilotti et al., 2024). Academic writing is a complex, demanding, and stressful task for students (Yu & Liu, 2021), differing from general writing in the sense that it “draws upon different grammatical, cognitive, and communicative knowledge” (Tadayyon & Farahani, 2017, p. 133). Academic writing includes specific features, such as a structured format, an academic vocabulary, and complex sentences, and requires arguments to be logically developed for the intended audience. Thus, the development of academic writing skills requires substantial integrated knowledge and competence from both teachers and learners.

Previous studies have investigated the potential challenges encountered by FL/SL learners in improving their academic writing skills, and offer relevant approaches to address these challenges. For example, Dang and Long (2024) found that the learning of core academic vocabulary increased steadily as more input was provided, in turn enhancing university EFL learners’ performance in academic writing. Tardy (2010) noted that SL undergraduate- and graduate-level writers must be able to select, evaluate, report, summarize, paraphrase, conclude, argue, select words and grammatical patterns, and avoid plagiarism in academic writing, all of which are demanding tasks.

Academic writing is more complex than many other learning tasks, presenting particular challenges for educational stakeholders, especially within FL/SL contexts (Bhowmik, 2009). Specifically, CSL learners usually encounter challenges different from those in EFL and other language education contexts, such as the complexity of the Chinese writing system, the intricacies of Chinese rhetorical patterns, specialized academic vocabulary and grammar, and language styles in Chinese academic writing (Shu, 2024). For instance, Shu (2024) highlighted the importance of academic writing skills for international students in a Chinese university and identified lexical and grammatical knowledge, pragmatic competence, and interpersonal communicative ability as challenges for them to succeed in academic writing. Liu (2018) and Li (2018) found that a lack of understanding of genre structure in academic writing significantly limited international students’ performance in their writing practices. At the same time, from a pedagogical perspective, Chen (2012) and Li et al. (2020) reported that structured training and timely feedback could support international students to improve their academic writing. The literature highlights both the importance and the challenges of academic writing in FL/SL education. Effective pedagogical approaches for developing and enhancing CSL learners’ academic writing competence are essential and needed for CSL education stakeholders.

2.2. Social Interactions and Feedback in Academic Writing

According to sociocultural theory, social interactions play a crucial role in the language learning process (Luan et al., 2023; Pica et al., 1991). This theory posits that learning occurs through the dynamic interplay between individuals and the social context in which they are situated, particularly collaborative efforts with peers who have varying levels of expertise (Vygotsky, 1978). Kim (2009), and suggests that a group of learners can mutually support each other’s learning by taking on complementary roles, alternating between positions as more or less skilled participants in various language tasks. Through collaborative group or pair work, learners pool their diverse language skills and knowledge, which helps them progress toward achieving their educational goals (Oxford, 1997). In FL/SL learning, students often find significant benefit in peer collaboration and social interactions as they develop their language and writing skills (Fathi et al., 2024).

Feedback and interactive experiences provide important scaffolding for students’ development of FL/SL skills, thus improving their experience of and performance in writing tasks (Banister, 2023; Lundstrom & Baker, 2009). Basturkmen (2003) noted that interactive processes around academic writing allow evaluators to provide students with additional information, justification, and support. These interactions facilitate a deeper mutual understanding of the written content. Crosthwaite (2017) listed patterns in academic feedback, including asking direct and/or indirect questions to others, making rebuttals of others’ claims, offering counterarguments to peers’ opinions, and presenting facts, opinions, or statistics from academic sources. He suggested that any of these interactions could positively influence students’ academic writing engagement and performances. Similarly, Su and Huang (2022) recognized interactions and feedback as critical components in enhancing writing performance and motivation in the context of language learning.

Thus, the extant literature emphasizes the role of interactions and feedback in shaping learners’ writing proficiency and motivation, and in guiding learners toward improvement (Loncar et al., 2023; Zhang & Zou, 2023). AI holds the potential to enhance student–teacher interaction by offering timely feedback and facilitating personalized communication, thereby addressing some of the limitations often associated with traditional instructional methods. This shift may empower both teachers and students to engage more effectively in the academic writing process and improve students’ motivation and proficiency in academic writing.

2.3. AI and Foreign Language Teaching and Learning

In the context of traditional in-class instruction, teachers and students alike face various challenges and constraints in building effective interactions and feedback processes, including time and space restrictions, financial issues, and interpersonal relationship dynamics (Chen, 2021). Recent studies report that emerging AI technologies offer a novel approach to addressing such challenges (e.g., Kohnke, 2023; Zawacki-Richter et al., 2019). In this context, AI-assisted tools can be understood as a viable substitute for human interaction partners when direct engagement is not feasible (Fathi et al., 2024). AI-assisted learning can incorporate automated feedback on various aspects of writing, including effectiveness, content organization, coherence, grammar, and vocabulary, facilitating students’ writing performance (Song & Song, 2023). These tools can help learners to identify and revise grammatical and lexical errors and suggest alternative sentence structures to enhance overall writing quality, structure, and presentation (Chen, 2023; Zhao, 2023).

Many prior studies have examined and identified the positive impact of AI-assisted tools and instruction in supporting FL/SL learning. Lu et al. (2006) reported that following the introduction of AI-powered instructional tools, and with less teacher supervision, foreign language learners with a range of target language skill levels could achieve greater academic learning outcomes and achieve better performances in academic tests. Among a group of college students with various proficiency levels, Kim (2016) also found that some foreign language learners felt more relaxed and were better able to overcome their anxiety when learning a foreign language with the help of AI tools. Huang et al. (2023) revealed that, compared to their counterparts in a traditional context, foreign language learners in an AI-assisted learning context showed superior achievement and demonstrated higher levels of participation in their learning tasks.

In the field of FL/SL writing, previous studies have explored the use of AI tools to enhance students’ writing performance and experiences. Song and Song (2023) investigated a group of Chinese undergraduate English as a foreign language (EFL) learners and found that AI-assisted learning could improve students’ learning autonomy and motivation. Similarly, Dale and Viethen (2021) reported that AI writing tools offered various language support functions, such as sentence or text generation, which could improve students’ writing performance. Link et al. (2022), Huang and Wilson (2021), and Nunes et al. (2022) consistently emphasized the role of AI-powered writing evaluation tools, identifying that these automated systems effectively enhanced learners’ writing quality. In addition, Godwin-Jones (2019), Urlaub and Dessein (2022), and Zhang and Torres-Hostench (2022) found that students frequently used AI tools for text translation, which aided them in generating, organizing, and revising their written work. Empirical studies have suggested that, compared to their peers relying on traditional paper-based tools, students using AI tools tend to demonstrate higher engagement in writing tasks and produce better writing outcomes. Other studies in this area have also focused on the potential role of AI-assisted tools in improving students’ FL/SL writing complexity, accuracy, and fluency (Fathi & Rahimi, 2022; Liu et al., 2021); writing motivation (Song & Song, 2023; Yan, 2023); and writing experiences (Wu et al., 2021).

Despite the growing body of research on the effectiveness of AI tools in enhancing FL/SL learners’ academic writing skills, there remains a notable gap in the literature concerning their application specifically to CSL students. While prior studies have demonstrated the positive impact of AI on writing performance, engagement, and motivation among various language learner groups, the unique needs, challenges, and experiences of CSL students in the AI-assisted writing context have not been adequately addressed. Given the increasing global population of CSL learners, further research is needed to understand how AI tools can support the development of CSL students’ academic writing skills and address their language learning process. In this regard, the current study examines the role of AI-assisted learning in international students’ CSL academic writing skill development. This study contributes to a nuanced understanding of how AI-assisted learning can facilitate CSL learning in a Chinese university context by addressing the following two research questions:

RQ1. How effective is AI-assisted learning in enhancing CSL students’ academic writing?

RQ2. How do international CSL students perceive the role of AI-assisted learning in their academic writing?

3. Methodology

The study adopts a two-phase mixed-methods design. Using test data from an experimental group and a control group, the first phase involves a quantitative analysis of CSL students’ performance in academic writing with and without AI-assisted learning. Using data from semi-structured interviews, the qualitative study set out to profile the CSL students’ perceptions of the effectiveness of AI-assisted learning, the roles of AI-assisted learning played in their academic writing, and their concerns about the use of AI-assisted tools. Methodological details and the findings of each phase are presented below.

3.1. Research Context and Participants

This study was conducted at a prestigious comprehensive university in mainland China. This institution has a reputation for academic excellence and a diverse population of students studying a wide range of disciplines. By situating the research in this setting, we aimed to capture the unique dynamics that can influence language learning performances, thereby contributing to a deeper understanding of AI-assisted learning. A total of 50 third-year international undergraduate CSL students, ranging in age from 20 to 27 years old (with a mean age of 21.3 years), participated in this study. The participants, majoring in Chinese Language and Literature or International Chinese Language Education, came from diverse linguistic and national backgrounds, including Thailand, Vietnam, Italy, Egypt, Kazakhstan, Nigeria, Indonesia, and other countries. The group consisted of 19 males and 31 females. Their prior exposure to Chinese language learning varied significantly, ranging from one year to more than ten years of formal study. All participants in the study had previously passed the Hanyu Shuiping Kaoshi (HSK) test at Level 5 or above. HSK serves as a standardized assessment of Chinese language proficiency for non-native speakers and is a requirement for international students to pursue their studies in China. Level 5 demonstrates a substantial command of the language, corresponding to an Advanced Intermediate to Advanced High level on the American Council on the Teaching of Foreign Languages (ACTFL) scale.

The participants were selected from four classes at the university using a convenience sampling strategy. This strategy was chosen to facilitate the efficient collection of data, as well as to capture a range of perspectives within relevant academic disciplines. By drawing participants from multiple classes, the study aimed to enhance the diversity of the sample and improve the robustness of the findings, thus providing a more comprehensive understanding of the impact of AI-assisted learning. The varied cultural and educational backgrounds of the participants provided comprehensive, in-depth insights into a wide range of possible experiences with AI-assisted academic writing instruction.

All participants were enrolled in a 16-week academic writing course, consisting of two 45 min sessions per week. The course was designed to develop students’ academic writing skills and prepare them for their final theses. It incorporated a variety of activities, including analyzing academic writing samples, drafting and revising essays, and participating in group discussions to enhance peer feedback and collaborative learning. Assignments were given to focus on key aspects of academic writing, such as argumentation, coherence, and formal language use. The teaching materials included academic journal articles, model essays, and the students’ writing samples. The course employed a combination of explicit instruction on academic writing conventions, scaffolded writing tasks, and interactive peer-learning approaches to support student learning. Before the present study started, each participant in the experimental group was asked about their prior experience of AI-assisted tools for academic purposes, including ChatGPT and other AI-powered resources.

Two learning contexts were created for this study in order to compare the effectiveness of AI-assisted learning and traditional in-class teacher instruction. The experimental group conducting AI-assisted learning consisted of 15 female and 10 male students (N₁ = 25), while the control group receiving traditional instruction consisted of 16 female and nine male students (N₂ = 25). The experimental group received the support of AI-assisted tools via computers and mobile devices instead of traditional activities.

3.2. Instruments

Two methods of data collection were employed to investigate the international students’ academic writing skill development when conducting AI-assisted learning. Established scoring rubrics for Chinese academic writing were used to assess the participants’ writing skills in pre-test and post-test evaluations. The scoring rubrics were developed based on the writing requirements of the Chinese Proficiency Grading Standards for International Chinese Language Education (Ministry of Education of the People’s Republic of China, 2021). The criteria include ideas, coherence and cohesion, lexicon, and grammatical range and accuracy. To ensure objectivity in the scoring process, two independent raters evaluated each writing sample at the pre-test and post-test stages. Both were experienced CSL teachers and HSK raters, who had received official HSK rating in instructional training at the Ministry of Education, and had the expertise necessary to accurately evaluate the participants’ academic writing skills. Inter-rater reliability was assessed, with the result demonstrating a high level of agreement based on a correlation coefficient of 0.86.

The experimental group used ChatGPT as the AI-assisted learning tool for CSL academic writing. Given that the participants in this study were international CSL students, their academic writing challenges extended beyond Chinese linguistic and cultural differences to include broader academic writing conventions. ChatGPT’s multilingual and cross-cultural capabilities can be suitable for them to provide feedback addressing both the structural and stylistic aspects of writing. At the same time, ChatGPT has been trained on a broad range of academic genres and writing styles, and thus has become suitable in terms of supporting students’ academic writing practice. Specifically, this is essential for international CSL students whose academic development mostly relates to both Chinese and global academic contexts. The student participants received feedback and revision suggestions on their writing samples, including specific suggestions for improvement. The tool also offered prompts to inspire writing on various topics. A notable feature of ChatGPT is its adaptability to accommodate both Chinese and other languages. Students could seek assistance and request hints in their first language when encountering difficulties with expressing themselves in Chinese.

3.3. Data Collection

To assess the change in the participants’ CSL academic writing skills, a written pre-test and post-test were administered in both the experimental and control groups. These were administered as course assignments, and were extracted during Weeks 5–6 for the pre-test (the mid-term assignment) and Weeks 15–16 (the final assignment) for the post-test, respectively. All participants were required to write a short but formal academic paper with a length of around 2000 Chinese characters on one of a selection of topics within two weeks for each assignment. The two raters then conducted a double-blind evaluation of the writing samples with reference to the established scoring rubrics.

To gather in-depth qualitative data on the participants’ perceptions of the role of AI-assisted learning in their academic writing, semi-structured individual interviews were conducted with six participants in the experimental group. The interviews were guided by a list of six questions, which were designed based on the research questions and the participants’ academic writing learning process (e.g., “Could you please describe your overall experience of using ChatGPT in your academic writing in Chinese?”) (See Appendix A). Demographic information on the six interviewees can be found in

Table 1. Demographic information of the interviewees (n = 6).

Pseudonym	Age	Gender	Nation of Origin	Chinese Language Level	Experience of Chinese Language Learning
Ava	20	Female	Colombia	HSK Level 5	5 years
Bella	21	Female	Egypt	HSK Level 5	3 years
Cindy	22	Female	Thailand	HSK Level 6	8 years
David	21	Male	Kazakhstan	HSK Level 5	3 years
Eric	27	Male	Vietnam	HSK Level 6	9 years
Fiona	22	Female	Nigeria	HSK Level 5	5 years

below.

3.4. Procedure

The study was conducted in the spring semester of 2024. Both groups attended the traditional teacher-instruction academic writing course for the first six weeks. For the experimental group, ChatGPT was integrated into the instructional design to assist academic writing tasks. The participants were guided to use it for specific purposes in an initial orientation session, such as generating ideas, refining writing statements, improving paragraph coherence, and correcting grammar. For instance, they could prompt ChatGPT with requests like 修改 (correct or revise), 缩写 (abbreviate), 换一种表达方式 (rephrase), etc., in their writing. Participants in the experimental group were encouraged to use the tool autonomously for their academic writing tasks. This ensured that they had regular exposure to AI-assisted learning throughout the 10-week intervention period. While the use of ChatGPT varied among participants in terms of time and frequency, all participants were required to document their usage and reflections in a learning portfolio. The portfolio served as both a record of their engagement with ChatGPT and a tool for self-assessment (See Appendix B for the requirements and examples).

Participants in the control group, meanwhile, continued their traditional teacher-instruction course. They used the same instructional materials as the experimental group. Face-to-face teacher-led interactions and feedback on the students’ writing performance were provided in the class, ensuring that participants received ongoing guidance and feedback throughout the 10-week intervention. The quantity and quality of instructions provided to these students were designed to be comparable to those provided to their counterparts in the experimental group.

All participants were required to submit their writing samples for the pre-test at the end of Week 6. Participants in both groups were required to complete the post-test after finishing the 16-week course. Six participants from the experimental group were then invited to participate in the interviews on a voluntary basis after the post-test. The interviewees were selected based on their varied educational and cultural backgrounds, as well as their different performances during the course, to capture a range of perspectives on the role of AI-assisted learning in the development of their academic writing. Each interview lasted approximately 40 min, and the interviews were digitally recorded and transcribed. All interviews were conducted in Mandarin Chinese, and the excerpts presented in this manuscript were translated into English. To ensure accuracy and reliability, the translations were initially completed by the first author, a native Mandarin speaker, with careful attention to cultural and contextual differences. These translations were then cross-checked by an independent reviewer proficient in both Mandarin and English. In cases of ambiguity, the translations were revised through discussion to ensure clarity. The full study procedure is shown in Figure 1 below.

3.5. Data Analysis

The current study used an independent samples T-test to compare the scores participants received on their writing assignments across the two groups. We then conducted repeated measures ANOVA analysis on the scores that participants obtained on the pre-tests and the post-tests in order to investigate the effectiveness of AI-assisted learning.

We conducted a thematic analysis for the qualitative data from six interviews based on the framework outlined by Boyatzis (1998). Guided by the research questions and the participants’ performance on the writing tasks in the earlier stages of this study, the interview data were categorized into three themes: (1) students’ perspectives on the effectiveness of AI-assisted learning on academic writing; (2) the role that AI-assisted learning played in facilitating students’ academic writing; and (3) challenges and concerns regarding effective AI-assisted learning. The coding process was completed by two independent coders, both of whom were experienced CSL teachers and HSK writing raters, to ensure the reliability of the data analysis.

4. Results

4.1. Effectiveness of AI-Assisted Learning on Students’ CSL Academic Writing

Levene’s Test for Equality of Variance was applied to the pre-test and post-test of academic writing ability. The results were below the threshold of significance (p₁ = 0.438, p₂ = 0.713, p > 0.05), indicating that equal variances could be assumed and allowing for further analysis. We employed an independent samples t-test to examine the latent differences between the pre-test and post-test scores of the two groups.

Table 2. Results of the independent samples t-test of the academic writing.

Test	Group	N	Mean	SD	t	p	Levene’s Test
Pre-test	AI-assisted	25	61.44/100	11.06	−1.437	0.78	F = 1.674, p1 = 0.438
	Traditional	25	64.06/100	8.42
Post-test	AI-assisted	25	89.74/100	9.08	6.53	0.00 *	F = 0.481, p2 = 0.713
	Traditional	25	82.15/100	10.23

* p < 0.05.

displays the descriptive statistical results of the participants’ academic writing performance.

At the pre-test stage, the mean score of the participants in the control group (M = 64.06, SD = 8.42) was slightly higher than that of their counterparts in the experimental group (M = 61.44, SD = 11.06). As the difference was not significant (p > 0.05), participants in the two groups were considered to have similar levels of knowledge and skill in CSL academic writing prior to the start of the intervention.

In the post-test, participants in the experimental group attained a mean score of 89.74, with a standard deviation of 9.08. The control group had a lower mean score of 82.15, with a standard deviation of 10.23. This constitutes a significant difference (p < 0.05) between the two groups in the post-test results, with participants in the AI-assisted learning group outperforming those who received a traditional instructional approach.

As illustrated in

Table 3. Results of a repeated measures ANOVA test of the academic writing of the experimental group.

Within-Subject Effects		Sum of Squares	Df	Mean Square	F	Sig.	Partial Eta-Squared
Test	Sphericity Assumed	104.326	1	104.326	3.483	0.00	0.317
	Greenhouse–Geisser	104.326	1.000	104.326	3.483	0.00	0.317
	Huynh–Feldt	104.326	1.000	104.326	3.483	0.00	0.317
	Lower Bound	104.326	1.000	104.326	3.483	0.00	0.317
Error (Test)	Sphericity Assumed	16.383	3	4.769
	Greenhouse-Geisser	16.383	2.104	6.322
	Huynh-Feldt	16.383	2.452	5.821
	Lower Bound	16.383	1.000	15.037

, the participants’ academic writing scores showed significant change between the pre-test and post-test (F3, 87 = 20.79, p < 0.05) at the probability level in this study. The Bonferroni post hoc test further confirmed a significant improvement in the academic writing performance of participants in the experimental group (p < 0.05,

Table 4. The results of a Bonferroni post hoc test on the academic writing of the experimental group.

Pairwise Comparisons
Measure: The Experimental Group (AI-Assisted Learning)
(I) Test	(J) Test	Mean Difference (I–J)	Std. Error	Sig.	95% Confidence Interval for Difference
					Lower Bound	Upper Bound
Pre-test	Post-test	−28.3	0.77	0.00 *	−36.41	−15.78
Post-test	Pre-test	28.3	0.77	0.00 *	15.78	36.41

* p < 0.05.

).

Overall, the six interviewees from the experimental group provided positive feedback on the effectiveness of AI-assisted learning in enhancing their academic writing skills. Specifically, their responses indicate that ChatGPT offered two main types of feedback to improve students’ academic writing: (1) identifying and correcting vocabulary and grammatical errors; and (2) inspiring thoughts and refining expression. For example, Cindy, who had passed HSK Level 6 and had extensive experience in CSL learning, explained that she had used ChatGPT to check and revise grammatical and lexical errors in her writing:

I wrote “书法作为一种书写工具已经逐渐慢慢地衰落了” (As a writing tool, calligraphy has gradually and slowly declined). ChatGPT told me it should be “书法作为一种艺术形式已经逐渐衰落了” (As an art form, calligraphy has gradually declined). I suddenly realized that calligraphy is not just a writing tool, and that “逐渐” (gradually) and “慢慢” (slowly) are semantically repetitive in this sentence.

(Interview)

Cindy further explained:

When I’m working on an academic paper, which usually involves writing several thousand characters, it’s impossible for me to check every word and sentence, even though I have some knowledge of vocabulary and grammar.

(Interview)

Other interviewees gave similarly positive responses and confirmed the effectiveness of using ChatGPT to identify and rectify errors in vocabulary and grammar in their academic writing. Examples included corrections such as “在图书馆查询相关文件/查询相关文件在图书馆” (looking up relevant files at the library) (Eric, Interview); correcting”他长年 (long years)” to “常年” (happening repeatedly or all the time) in the phrase “常年研究古代文学相关内容” (He has been studying ancient literature for long years) (Ava, Interview); and changing “了解” (know about) to “明白” (understand) in “非言语表达可以增进人与人之间” (Nonverbal expression can help people better understand each other)” (David, Interview). Interviewees also reported that they used ChatGPT “to search for commonly-used phrases” (David, Int. 3; Fiona, Interview), “to complete a sentence” (Ava, Interview), and “to look up a specific word” (Bella, Int. 2; David, Interview).

Two of the six interviewees highlighted their use of AI-assisted learning tools to inspire thoughts and refine expression in their academic writing. For instance, Bella described ChatGPT as a tool to “spark new ideas when I did not know what to write” (Int. 2). She provided a specific example in her interview accounts:

I was trying to write an academic paper on the Chinese language but didn’t know what specific topic to focus on since I wasn’t familiar with the subject. So, I opened ChatGPT and typed “请列出几个适合留学生的关于汉语本体的研究题目” (Please list several research topics suitable for international students on the ontology of the Chinese language). It immediately gave me a list of topics, and I found that research on pronouns might be interesting. … The tool further explained the topic in detail and provided some examples and references when I asked.

(Interview)

Like Bella, Eric, an experienced CSL learner with a level 6 HSK qualification, confirmed that ChatGPT was an effective tool in refining his academic writing. He said:

I’m pretty good at Chinese, but I still don’t know some of the academic expressions used in writing papers. For example, I used 我 (I) in my writing, because I was always taught to do so. ChatGPT told me that 笔者 (the author) as the first-person pronoun is more common in formal Chinese academic writing.

(Interview)

Eric also discussed his experience of using the AI-assisted learning tool to enhance the formality of specific expressions in his academic writing:

I wrote “我从问卷中看出, 学生更喜欢使用他们的手机查字” (I see from the questionnaire that students preferred using their mobile phones to look up words), which was a complete and correct sentence in Chinese. But after I typed it into ChatGPT, I realized that the revised sentence “从问卷结果分析, 学生更倾向于使用手机查找字词” (From the analysis of the questionnaire results, students tend to use their phones to look up words) was more formal and better for academic writing.

(Interview)

Thus, these two students both emphasized the importance of refining expressions in their academic writing with the support of the AI-assisted learning tool, noting that the task “could not be easily accomplished through the use of a dictionary or even with the help of a real teacher” (Bella, Interview).

4.2. The Role of AI-Assisted Learning in Facilitating Students’ CSL Academic Writing

Through a thematic analysis of the interview data, it was found that participants saw the AI-assisted learning tool as playing three main roles in their academic writing: (1) a knowledgeable instructor; (2) a friendly learning context creator; and (3) a passionate learning motivator. For instance, Ava described ChatGPT as a knowledgeable instructor, because “it seems that AI knows everything about writing and the Chinese language. … It’s like a knowledgeable instructor” (Ava, Interview). She further explained:

I’ve passed HSK Level 5, so I think my Chinese is pretty good. But whenever I write something, I still check it with ChatGPT. It’s always spot on.

(Interview)

David and Eric expressed similar views. In their interview accounts, they tended to define the AI-assisted learning tool as “an experienced teacher” (David, Interview; Eric, Interview), “an expert” (Eric, Interview), and “a senior peer who understands our [students’] difficulties and concerns in academic writing” (David, Interview).

Fiona, considered herself to be “a learner who has had dozens of Chinese teachers” (Interview), highlighted the perceived differences between the AI tool and those teachers:

Using an AI tool feels more like chatting with a friend. It’s way easier to ask it for help than to go to my teachers. I don’t get nervous, and I don’t have to worry about being judged by my teachers.

(Interview)

Fiona further explained:

Even when I ask a “stupid” question, I don’t feel embarrassed at all. For example, I’d never ask a teacher if 选题缘由 (Rationale for Topic Selection) is necessary in an academic paper though I am not clear about its difference from 研究背景 (Introduction). But I feel totally comfortable asking ChatGPT the same question.

(Interview)

Most interviewees expressed the feeling that the AI-assisted learning tool created a low-pressure learning environment, allowing them to interact with the tool without feeling self-conscious or anxious. ChatGPT was characterized as “a collaborator” in academic writing rather than “an instructor” (Cindy, Interview). The AI-assisted learning was seen as offering “a safe space” to practice academic writing in a “threat-free” (Ava, Interview), “peer competition-free” (Bella, Interview; Eric, Interview), and “judgment-free” (Cindy, Interview; Fiona, Interview) environment.

The interviewees also referred to the AI-assisted learning tool as “a passionate learning motivator”. David offered positive feedback on the tool, stating that it “never failed my expectations in learning.” As he explained:

I felt really motivated to learn because I knew I could always count on ChatGPT. Whenever I needed help, I could just open it up and type my questions. It made me feel super confident about my learning.

(Interview)

Eric, meanwhile, felt that the support he received from ChatGPT kept his writing process “under control.” He described the revision process with the AI-assisted learning tool as “sound and assured without many obstacles,” noting that “every time I used ChatGPT, I knew I could make my manuscript better,” and “I could get a better score” (Interview). Likewise, Cindy stated that ChatGPT could provide “personalized instruction” that was appropriate for her individual learning, rather than “correct but useless content for my own writing” (Interview). In addition, Eric noted the interactive nature of ChatGPT and believed that “communicating with ChatGPT is also an effective way to facilitate my writing” (Interview). Overall, the various roles that ChatGPT played in the participants’ development of academic writing skills enhanced their learning motivation and their engagement in writing activities.

4.3. Concerns About AI-Assisted Learning in CSL Academic Writing

Despite seeing advantages in the use of AI-assisted learning to facilitate academic writing skill development, the participants expressed three major concerns regarding the practice: the potential for over-reliance on AI; the risk of technical and networking issues; and the unreliable nature of AI-generated content. All six interviewees reported concerns about their own over-reliance on the AI-assisted learning tools in academic writing, as well as the associated ethical issues. For example, Cindy mentioned that the tool “was so good that I completely relied on its instruction” (Interview). Fiona also admitted that she had used ChatGPT to “check and revise almost every sentence or even every word I wrote” (Interview). Eric even claimed that “without the tool, I would write nothing correctly” (Interview).

The interviewees expressed concern about the ethical acceptability of their academic writing practices, such as “it might be wrong to ‘ChatGPT’ a paper” (David, Interview) and “I think it’s unacceptable to do so” (Bella, Interview). However, none of them decided to change their approach or quit using the AI tool. Instead, they showed a clear preference for the tool and claimed that they would “insist on using ChatGPT to help with my writing as long as it’s accessible” (David, Interview).

Four of the six interviewees reported encountering technical or networking obstacles while using the AI-assisted learning tool. Cindy observed that different prompts would lead to different outcomes, which made it “a great challenge for me to wisely select prompts” when using ChatGPT. She used the word “exhausting” to describe the process of selecting prompts, particularly “when I had to write a paper” (Cindy, Interview). Ava compared ChatGPT with an electronic dictionary, noting that access to ChatGPT remained an obstacle for her. David and Eric mentioned networking issues, as both had experienced unstable connections when they needed ChatGPT for academic writing. Eric noted, “I could do nothing but wait for several hours and almost missed the deadline for submission [of the assignment]” (Eric, Interview).

Two interviewees expressed concern that the AI tool might generate unverified or even fabricated content in response to their requests. For instance, David found that some references provided by ChatGPT were nonexistent. He searched for some of the references and found no results, noting that one was “so fake that its author was 张三 (Zhang San, a common male pseudonym in Chinese equivalent to John Smith)” (Interview). He added, “fortunately, I double-checked the provided reference. I can’t even imagine what would have happened if I had directly copied it into my paper” (Interview). Fiona expressed a similar concern. She shared an experience with ChatGPT which occurred as she was writing an academic paper on government policy regarding the education of international students in China. She noticed that some policy terms provided by ChatGPT were “significantly different from the government-issued documents I found online” (Interview). In the end, she decided to quote the official documents instead of using those provided by ChatGPT.

5. Discussion

This study explored the impact of AI-assisted learning on international CSL students’ academic writing in a Chinese university context. The results, drawn from pre-test and post-test assessments as well as six individual interviews, demonstrate that AI-assisted learning significantly improved participants’ writing performance. These findings align with previous studies (e.g., Liu et al., 2021; Song & Song, 2023), which highlight AI’s potential to enhance FL/SL learners’ writing abilities.

One key factor in this improvement was the AI tool’s ability to provide immediate, targeted feedback on grammar and vocabulary. This feedback helped students identify and correct errors, improving their writing quality. Students also used the AI tool to organize their writing, search for phrases, complete sentences, and look up specific words. For example, when one student wrote “学生需要, 所以老师调整了教学计划” (“Students needed it, so the teacher adjusted the teaching schedule”), the AI suggested a more formal revision: “由于学生的需求, 老师对教学计划进行了调整” (“Due to students’ needs, the teacher made adjustments to the teaching schedule”). This feedback, particularly in terms of grammar correction and vocabulary enhancement, addressed common challenges that CSL students face, such as refining sentence structure and selecting formal vocabulary. These findings contribute to the growing body of literature on AI-assisted writing in language education, offering new insights into the CSL context. While previous studies have examined EFL and other FL/SL groups (Barrot, 2023; Zhao, 2023), this study focuses on CSL learners in higher education, emphasizing the tool’s effectiveness in enhancing their academic writing.

In this study, the AI tool provided feedback that effectively addressed the participants’ writing needs, which led them to perceive the tool as a knowledgeable instructor (Ava, Interview). Beyond its instructional capabilities, the real-time response mechanism significantly contributed to the improvement of the CSL students’ academic writing. As Chen (2022) found, the delayed instruction typically encountered in traditional teaching contexts often results in less satisfactory learning outcomes. Some participants in the study reported that they might “leave the confusion behind” or even “ignore the puzzles” when unable to reach their teachers promptly (Ava, Interview; Eric, Interview). AI-assisted learning tools like ChatGPT can provide greater accessibility compared to teachers in the traditional classroom, without constraints like time, space, and interpersonal relationships. Thus, the participants were able to engage more actively in their individual learning processes and had more opportunities to practice and develop their writing skills.

AI-assisted learning also supported the participants in refining their CSL academic writing, an area where traditional tools like textbooks and dictionaries often fall short. CSL academic writing, with its distinct vocabulary, sentence structure, and logical coherence (Shu, 2024), poses unique challenges for learners. AI tools, such as ChatGPT, helped students better understand and apply academic writing conventions by generating sample texts and providing explanations based on their original writing. For example, when a participant wrote “这个研究结果很重要” (“The results of this study are important”), ChatGPT suggested a more formal revision, “该研究结果具有重要意义” (“The results of this study are of significant importance”), explaining that “具有重要意义” was more appropriate for academic contexts. This feedback was particularly helpful for CSL learners, who often struggle with distinguishing between casual and formal expressions, an issue not typically encountered in their native languages. Participants noted that AI tools like ChatGPT offered valuable alternatives for refining their texts (Bella, Interview). This finding supports Zhao’s (2023) report that AI tools are often more effective than traditional methods for meeting students’ needs in academic writing. AI tools in this study inspired ideas, generated grammatically accurate sentences, and adjusted language to meet academic style expectations, ultimately improving students’ performance in academic writing tasks.

An engaging learning context is crucial for maintaining student involvement in academic writing. In this study, AI-assisted learning offered a flexible, supportive environment that students found engaging and motivating. Participants described AI tools, particularly ChatGPT, as “a safe space” (Ava, Interview) and “a friendly collaborator” (Cindy, Interview), providing personalized and responsive support that is difficult to replicate in traditional teacher-led settings. In many Chinese universities, CSL students face intense peer competition and external pressures (Gao et al., 2014; Gong et al., 2021), which can discourage open communication and active participation in class (Fiona, Interview). AI-assisted learning alleviated these concerns by reducing anxiety related to making mistakes or “losing face” (Chen, 2022), creating a lower-stress environment that fostered engagement.

AI tools allowed students to learn at their own pace, with content tailored to their needs (David, Interview). This personalized support enabled students to receive specific feedback on their writing, helping them address individual challenges effectively, as noted by Song and Song (2023). The students’ positive experiences with AI-assisted learning, described as both effective and reliable, motivated them to actively engage in writing tasks (Eric, Interview). Thus, AI not only served as a knowledgeable instructor but also as a motivator, addressing some of the unique challenges faced by CSL students and facilitating more efficient and effective learning.

While AI-assisted learning offered benefits for academic writing development, participants also faced challenges and expressed concerns about its use. A major issue was the over-reliance on AI support in writing tasks, which can hinder the development of independent critical thinking and creativity, as noted by Utami and Winarni (2023). Despite these concerns, students continued to depend heavily on AI tools, regardless of their language proficiency or performance levels, raising questions about maintaining academic integrity. As emphasized in previous studies (Fathi et al., 2024; Koraishi, 2023; Mohamed, 2024), balancing AI use with fostering independent learning is crucial for preserving academic ethics in education.

Another challenge was the accuracy and authenticity of AI-generated content. Participants noted instances where AI tools provided inaccurate or unverified information, raising concerns about the potential negative impact on their writing (David, Interview; Fiona, Interview). The black-box nature of AI, combined with the participants’ limited ability to verify the correctness of responses, made it difficult for them to effectively refine their texts. In addition, technical issues, such as unstable internet connectivity, hindered the learning process, suggesting that technical training, support, and infrastructure are also crucial for the successful integration of AI tools in academic writing.

6. Conclusions and Implications

This study has investigated the effectiveness of AI-assisted learning and its role in enhancing international CSL students’ academic writing. Overall, the findings offer compelling evidence in favor of integrating AI-assisted learning tools into academic writing. The results indicate that AI-assisted learning significantly improved students’ academic writing performance by helping them to identify and correct errors, as well as inspiring them and helping them refine their expression. The participants reported positive learning experiences, noting that AI-assisted tools served as a knowledgeable instructor, a creator of a supportive learning environment, and a passionate motivator. These perceptions highlight the contribution of AI-assisted tools to enhancing learners’ academic writing motivation and skills. However, the results also identified challenges and concerns regarding the use of AI-assisted learning, including students’ over-reliance on AI, ethical concerns, technical and networking issues, and the potential unreliability of AI-generated content.

Informed by previous studies exploring the use of AI-assisted learning in language education (e.g., Fathi et al., 2024; Song & Song, 2023; Liu et al., 2021), this study focused on international CSL students’ academic writing in a Chinese university context. The findings suggest that AI-assisted learning can significantly enhance students’ academic writing skills by providing tailored feedback on language use, grammar, and structure. This is particularly valuable given the specific difficulties faced by CSL learners, such as mastering Chinese rhetorical norms and distinguishing between formal and informal language. The findings demonstrate the potential value of integrating AI-assisted learning into language education, particularly for enhancing students’ academic writing practices. Further adapting AI tools to incorporate explicit feedback on Chinese language and academic conventions could enhance their effectiveness in CSL writing instruction.

However, to ensure AI-assisted tools and resources are used appropriately, it is imperative for educational policymakers and institutions to address ethical concerns, especially relating to AI-driven academic dishonesty.

This study’s limitations include its small sample size; future research should examine the potential long-term impact of AI-assisted learning with a larger and more diverse learner population and incorporate follow-up evaluations. Additionally, investigating CSL teachers’ perspectives would help to triangulate the data collected and provide a more comprehensive understanding of the effectiveness and implications of AI-assisted learning in academic writing. Despite these limitations, the findings of this research demonstrate the value of integrating AI-assisted learning in university language programs. Future studies on exploring the use of Chinese local LLMs need to be conducted, which may offer more bespoke support in line with the linguistic and cultural awareness of Chinese academic writing. At the same time, incorporating a control group in the future phase of this study will be helpful to draw causal inferences about the effectiveness of AI-assisted learning on academic writing improvement. It may provide a more convincing comparison between AI-enhanced and traditional instructional methods. In addition, future research should consider CSL teachers’ perceptions and instructional practices regarding AI tools to understand how they integrate AI resources into their teaching, which can shed light on both the opportunities and challenges of AI-assisted learning in language education.