Exploring the Development of Student Teachers’ Knowledge Construction in Peer Assessment: A Quantitative Ethnography

Abstract

Peer assessment (PA) is a formative assessment tool that can effectively monitor the development process of knowledge construction. In comment-based PA, comments contain the evidence of how the assessors construct knowledge to conduct professional assessments, which initiates a research perspective to explore the dynamic knowledge construction of the assessors. Quantitative ethnography is both a method for the quantitative analysis of qualitative data and a technique for the network modelling of professional competencies, providing a new way of thinking about the analysis and evaluation of knowledge construction processes. In this paper, quantitative ethnography was used to mine the comments generated from comment-based PA activities to reveal the characteristics of student teachers’ knowledge construction and the developmental trajectories of knowledge structure at different learning stages. The experimental results show that the student teachers’ knowledge structures and knowledge levels evolve in the PA environment, and the cognitive network gradually tends to become more complex and balanced. The student teachers showed stage and gender differences in the level of knowledge progression during the learning process. The second PA was a turning point in knowledge progression. The knowledge structures of the male and female groups are biased towards different kinds of knowledge elements.

1. Introduction

Global trends in education view learning as the co-construction of knowledge, a process that facilitates the social-relational aspects of knowledge generation, the use of multiple resources and cross-contextual skills [1,2]. As described in Goal 4 of the Sustainable Development Goals, sustaining knowledge for all learners is an indispensable pillar to support the sustainable development in the world [3]. Knowledge construction research attempts to innovate from multiple theoretical, pedagogical and technological perspectives by placing learners in authentic problem situations and engaging them in active, purposeful and sustained knowledge construction activities, playing the role of contributors to knowledge creation [4,5,6]. For example, in task-driven situations, learners gradually gain cognitive sublimation by sharing and exchanging knowledge with each other, thus contributing to the collective cognitive level of the learning team [7,8].

With the advent of the knowledge economy, society has placed new demands on the comprehensive competencies and qualities of teachers [9]. Teachers have to continuously change their original knowledge structure and improve their professionalism through learning in order to adapt to the requirements of teachers in the new era [10]. Countries around the world have invested considerable efforts in teacher training and have adopted various methods to promote the development of teaching competencies [11,12], but the knowledge construction mode of the teachers in teachers training is not clear, which constrains the effectiveness of teachers training.

As a formative assessment tool and context-based social assessment, peer assessment (PA) is considered to be a method to monitor the process of knowledge construction [13] and an effective way to promote learners to construct knowledge [14,15]. In comment-based PA, assessors use professional knowledge to evaluate the performance of their peers and provide feedback in the form of comments [16,17]. The comments contain the implicit discursive content and cognitive characteristics of the assessor [14,18]. It was proved that assessors’ learning, logical thinking [19,20] and developmental cognitive skills [21,22] were enhanced when writing professional comments Research revealed some useful information, such as the cognitive structure [23] and the sequential pattern of knowledge construction, from the comments generated from learning activities [24,25].

To promote the effectiveness of teachers training, in this paper, we conducted several comment-based PA activities and explored the student teachers’ knowledge construction mode throughout the PA activities. The comments were collected and analysed with method and technology of quantitative ethnography. We examined the following questions to reveal the characteristics of knowledge construction in different PA stages and of different groups of student teachers.

• How does the student teachers’ knowledge construction change over a succession of PA activities?
• What are the characteristics and developmental trajectories of student teachers’ knowledge construction at different PA stages?
• What are the characteristics of the knowledge construction of different groups of student teachers?

2. Literature Review

2.1. The Process and Analysis of Knowledge Construction

Knowledge construction is based on constructivism, which emphasises the role of students as subjects, and group members engage in meaningful inquiry, knowledge construction and work creation through dialogue in order to facilitate the internalisation of meaning shared within the group by each member of the group [26]. Scardamalia and Bereiter specifically distinguish between shallow and deep construction in their knowledge construction studies and point out that the real meaning of constructs lies in deep constructs [26]. Deep constructs focus on the application and innovation of knowledge generation, with learning tasks and learning activities that explore the connotations and important laws implicit in the learning tasks [27]. Knowledge construction in a peer-assessment model is precisely deep construction [28]. The basic idea of knowledge construction is that learners develop higher-order learning skills, higher-order thinking skills and knowledge creation skills in the process of iteratively constructing and iteratively advancing knowledge based on problems. The most direct way to do this is not by designing learning tasks or inquiry activities that allow students to acquire knowledge and skills in the domain but by making them creators of knowledge [29,30].

For the analysis and evaluation of knowledge construction, the current field is usually based on content analysis, behavioural analysis, social interaction analysis, cognitive network analysis and other dimensions to explore the level of knowledge construction or the development patterns of learners in the process. Gunawardena et al. constructed a knowledge construction interaction analysis model, using content analysis to code the behavioural interactions between learners, and classified the learning team. The level of knowledge construction in a learning team was categorised into five levels: sharing and comparing information, discovering and exploring contradictions, negotiating meaning and constructing knowledge, validating or modifying the results of the construction and forming consensus and applying it [31]. Li et al. constructed a quality analysis model of collaborative knowledge construction using the behaviour of students’ knowledge construction in a wiki environment as a benchmark, in which knowledge construction was divided into three stages: knowledge sharing, knowledge linking and knowledge convergence [32]. Saqr & Alamro analysed the interaction data of different PBL groups through social network analysis to reveal the characteristics and patterns of learners’ social interactions in the process of knowledge construction, which provided a reference for subsequent in-depth analysis of the development of individual learners’ knowledge construction and the prediction of group knowledge construction trends [33]. Sullivan et al. used cognitive network analysis to analyse the collaborative learning activities by using cognitive network analysis to analyse the discourse elements of different groups in collaborative learning activities, Sullivan compares the differences in collaborative interactions between the high- and low-performing groups and thus provides a reference for later interventions in the learning styles of the low-performing groups [34].

2.2. Role of PA and the Current State of Research

PA is an activity in which learners evaluate the level, value or quality of the designated achievements or the performance of other equal-status learners [21]. Typically, two forms of PA exist: scoring- and comment-based PA [35]. Scoring-based PA involves providing quantitative feedback in terms of grades or scores for evaluation indicators on the basis of certain rules [36]. Comment-based PA is more flexible and has a wider scope than scoring-based PA, which is bound by evaluation indicators [37]. Comment-based PA can integrate the evaluator’s objective and comprehensive knowledge of educational objectives, evaluation rules and the evaluation object [38]. Compared with scoring, writing comments exhibits equal or higher learning potential [16,17].

PA has been applied in various education scenarios and has a positive effect on learners’ academic performance, thinking ability and autonomy [19,39,40]. Pope revealed that PA improved the academic performance and motivation of master trainees in a marketing course [41]. Wang et al. incorporated a teaching strategy based on online PA in programming training and found that, under this strategy, trainees acquired considerable programming knowledge, many programming skills, an active learning attitude and a sense of critical thinking [42]. Chien et al. implemented peer commenting activities in a spherical video–based virtual reality environment and found that these activities had a positive influence on learners’ spoken English skills, learning motivation and critical thinking [43].

PA is a complex learning task that requires a high level of cognitive processing [44]. By writing comments on other people’s reports, evaluators can reconceptualise what is right and wrong in their own reports [45]. Evaluators must recall and repeat material actively or apply learned concepts; thus, all participants become engaged in higher-level thinking and learning [46]. There were many researchers engaged in the study and application of PA to facilise learners’ learning outcomes, but the formative evidence related to the promotion is not so sufficient.

2.3. Rationale for Using Comment-Based PA: Retrieval-Based Learning Theory

Comment-based PA is a suitable formative assessment tool for learners’ learning [47]. Explaining the mechanisms underlying comment-based PA, which has been demonstrated to facilitate knowledge acquisition by learners [16,48], is crucial. Comment writing is a process of retrieving and reconstructing knowledge and is thus regarded as a form of retrieval learning [49].

In retrieval-based learning, knowledge retrieval is a key process for understanding knowledge and facilitating learning. All the knowledge representations in retrieval-based learning are retrieved using retrieval clues available in a given context [50]. Knowledge changes each time a person retrieves it because knowledge retrieval improves an individual’s ability to retrieve knowledge again in the future [51]. Even unsuccessful knowledge retrieval can facilitate learning [52]. The repeated retrieval of knowledge is key to long-term memory and, more importantly, improving later memory [53]. Chan et al. found that under conditions that simulated educational contexts, knowledge retrieval facilitated memory related not only to a test material but also (to a lesser extent) to a related non-test material [54]. Practice knowledge retrieval can produce meaningful, long-term learning [55]. Researchers have conducted experiments to verify the aforementioned statement. For example, Roediger reported that frequent testing in a classroom may increase educational achievement at all educational levels [56]. Karpicke and Grimaldi described two methods for instructing students to practice knowledge retrieval and found that incorporating knowledge retrieval into educational activities is an effective method for enhancing learning [57].

Comments are generated after the evaluator has retrieved relevant knowledge. Every retrieval is a learning experience and a process in which the evaluator reflects on their thinking and knowledge through the performance of peers to construct a dynamic cognitive structure. Therefore, comment-based PA can reveal the dynamic information such as the evaluator’s thinking, knowledge construction and cognition development in the learning process.

2.4. Quantitative Ethnography: Epistemic Network Analysis Approach Based on Epistemic Frames

In order to study the process of knowledge construction by learners in peer assessment, analysing the content of the comments written by the learners involved in the peer assessment is a useful method. Leung argues that ethnographic research can contribute to our understanding of social learning processes [58]. However, traditional ethnographic research is a time-consuming and labour-intensive qualitative analysis, which makes it difficult to conduct large-scale analytical assessments. To address the difficulty of analysing large amounts of comment data, Shaffer proposed an analytical method called quantitative ethnography [59]. A key component of quantitative ethnography is epistemic network analysis (ENA), a cognitive representation based on a cognitive framework and an evidence-based assessment method in digital learning environments [59].

Epistemic frames were introduced by Shaffer, who believes that each professional community of practice can be represented by a professional epistemic framework that encompasses its thinking methods, cognition and problem-solving abilities [60]. Epistemic frames consist of five interconnected components: skills, knowledge, identity, values and epistemology. Professionals in a particular field possess similar knowledge and skills. They have similar attitudes and approaches, thought habits and speaking styles when handling various tasks. Therefore, the discourse of professionals from a certain field can reveal their knowledge and skill structure. The core concept of epistemic frames is that the discourse of a community of practice can be modelled according to the connections between the elements of the epistemic framework to reflect the manner in which they think and behave when solving problems [61].

ENA was originally developed to model epistemic networks, which represent patterns of associations between knowledge, skills, values, thinking habits and other elements that characterise complex thinking [62]. Codes, analysis units and stanzas are three core aspects of ENA. A code represents a set of conceptual elements, and the purpose of ENA is to understand the interactions between these elements. An analysis unit represents the objects of ENA, such as gender or age groupings, or specific individuals. A stanza represents the range of code co-occurrences. ENA is conducted to model the connections between codes by quantifying the co-occurrences of codes in a conversation to generate a weighted network of co-occurrences and associated visualisations for each analysis unit in the data. The structure of the connections is determined, and the strength of the associations between elements in the weighted network is measured to quantify how the composition and strength of the connections change over time. For example, if the differences between individuals are to be compared, each individual should be the analysis unit. If the differences between groups are to be compared, each group should be the analysis unit. In ENA, elements in the same conversation are conceptually interconnected, whereas elements in different conversations are not [61].

ENA can be conducted for analysing and visualising unstructured data, such as discourse and text data, to compare cognitive differences between individuals and groups. For example, Sullivan et al. conducted ENA to model the connections between team discourse elements, with the aim of comparing the differences between a low-performing group and high-performing group in order to adjust the training for the low-performing group and to predict the performance level of both groups [34]. In ENA, domain expertise is considered not as an isolated set of knowledge, skills and processes but rather as a network of connections between knowledge, skills and decision-making processes. ENA is suitable for modelling patterns of association in any domain expertise characterised by complex dynamic relationships between a set of elements [63]. Thus, ENA can be used to compare the salient features of the cognitive networks of different individuals and groups [24].

In this study, we used ENA to visualise information about the knowledge status and cognitive structure of the assessors contained in the PA comments to explain the characteristics of the student teachers’ knowledge constructs and their developmental trajectory in teacher training.

3. Methodology

3.1. Research Design

A succession of PA activities was designed for a course titled ‘Teaching Skills Training’. In this course, two trainers guided student teachers (i.e., trainees) to promote their teaching competences. This research comprised three steps. In the first step, the trainees conducted PA activities that involved commenting on their peers’ teaching competences. The PA comments were collected. In the second step, the comment data were quantified according to a coding scheme to extract key information for analysis. In the third step, data processing and analysis were conducted to discover the characteristics of trainees’ knowledge constructs during the peer assessment activities.

ENA was performed to explore the specific cognitive characteristics of the comments, and statistical analysis was conducted to explore the differences in categorical information among the coded comments. The research framework is illustrated in Figure 1.

3.2. Participants and PA Activities

The participants were 20 junior student teachers (8 females and 12 males) in a university in eastern China. The mean (range) of their age was 20.60 (20–21) years old. They enrolled in a two-credit-hour course entitled ‘Teaching Skills Training’ that prepares them to become a teacher. The 20 participants were divided into 3 learning groups of 7, 7 and 6, with 3, 3 and 2 females in each group, respectively. Each learning group had roughly the same levels of participants, which was determined by the scores the participants received in a prerequisite course entitled “Instruction System Design”, which teaches theoretical and practical knowledge about teaching. Each learning group conducted a PA activity in a separate classroom.

Four comment-based PA activities were conducted during the course, with an interval of 1–2 weeks between each activity. The flow of the PA activity for each learning group is shown in Figure 2. Participants in the learning group took turns to conduct the trail teaching, with each participant acting as both the evaluator and the evaluatee. When a participant conducted a trail teaching as an evaluatee, the other members of the learning group were required to write PA comments for them as evaluators. Trail teaching videos and all of the comments were submitted to the online learning platform for the trainers to review.

3.3. Data Collection

All the trail teachings and PA activities were conducted within 8 weeks. We collected comments as the dataset for this study. In a learning group of 7, for example, each trainee in the group was required to write 6 comments during each PA activity, so the learning group was able to produce 42 comments for that activity. Each PA activity contained a total of three learning groups (7, 7, 6) and was able to generate 114 peer assessment comments. The four PA assessment activities collected a total of 456 comments, of which 17 were invalid. After the invalid comments were removed, 439 comments remained for analysis. These comments contained a total of 3615 sentences and 35,228 words.

3.4. Coding Scheme and Coding

We collected 580 comments written by 5 senior teachers and 2 trainers and used the comments to construct the epistemic framework of student teachers’ teaching competence. We adopted Grounded Theory [64] to obtain the categories, main categories and core categories in the context of teaching competence from the comments. With the NVIVO 12.0 tool, we formed 16 main categories from the categories implicated in the comments and abstracted three core categories, namely, the general knowledge category, the expertise category and the higher-order knowledge category, that can summarise all the main categories. We then randomly selected 44 comments from the 439 comments collected from the PA activities and coded them according to the grounded theory coding procedure to perform a saturation test. No new conceptual categories or relationships were found, which indicates that the model is theoretically saturated.

The main categories, the core categories and the relationships between them constructed the epistemic framework of student teachers’ teaching competence. The main categories represent the knowledge elements of teaching competence and were divided into three perspectives (core categories). We then assigned a code to each main category and formed a coding scheme for comments coding, with the specific interpretations shown in

Table 1. Overall coding framework of this study.

Core Category	Main Category	Code	Description	Example
General knowledge	Appropriate appearance	G1	Neat and tidy appearance, pleasant behaviour and appropriate body language.	Lecturing with a tablet and not very well groomed.
	Verbal expression	G2	Clear and appropriate language, accurate expression, clear diction, fluent speech, standard pronunciation, loud voice and appropriate speed of speech.	Not out of script, still stumbling a bit in trail teaching.
	Interactive questioning	G3	Focusing on the stimulation of students’ learning interest and interactions.	Highly interactive classroom with positive interaction with students.
	Blackboarding	G4	Appropriate representation of a suitable quantity of neat and attractive writing on a blackboard.	The blackboard could have a little more design.
	Teaching tools	G5	The tools, media or equipment used to deliver information and the specific environment in which teaching is conducted, including PowerPoint presentations, electronic whiteboards and multimedia classrooms.	In the Try It session, rather than using abstract descriptions in words, you can use Excel on your computer for a straightforward demonstration of the operation
Expertise	Teaching content	E1	Understanding the objectives and requirements of the course and obtaining an accurate picture of the teaching content.	Still a little empty in terms of teaching content.
	Teaching methods	E2	Selecting appropriate teaching formats and methods according to student needs.	The lecture process is still based on the traditional lecture method.
	Key teaching points	E3	Determining teaching objectives, teaching priorities, and difficulties in accordance with the requirements of the teaching content and curriculum standards.	Teaching difficulties are not sufficiently prominent.
	Arrangement of sessions	E4	Organising students’ learning activities effectively and controlling the teaching flow.	The lectures are well timed.
	Teaching evaluation	E5	Focusing on student assessment and feedback in the delivery of teaching.	Obtain feedback from students through assignments.
	Environment building	E6	Introducing course content and creating classroom situations that stimulate students’ learning interest and learning motivation.	Clever use of scenes from life to introduce course content.
	Teaching process	E7	Presenting teaching content in a scientifically accurate manner and controlling the time and speed of teaching.	It would have been nice to have more examples in the teaching process.
Higher-order knowledge	Student-centred teaching	H1	The teaching design reflects the subjectivity of the students.	I would suggest that your teaching design should be more student-centred.
	Professional ethics	H2	Love for education, care for students, respect for students and the fair and equal treatment of each student.	All levels of student acceptance should be taken into account.
	Quality of thinking	H3	Understanding and analysing problems quickly and accurately in a logical and flexible manner.	A little problem with the logic of the trail teaching.
	Psychological qualities	H4	Positive, cheerful and self-confident, with a determined and tenacious spirit, and not afraid of difficulties.	Acting nervous during the trail teaching.

. Five knowledge elements, namely, appropriate appearance (G1), verbal expression (G2), interactive questioning (G3), blackboarding (G4) and teaching tools (G5), belong to the general knowledge category. Teaching content (E1), teaching methods (E2), key teaching points (E3), the arrangement of sessions (E4), teaching evaluation (E5), environment building (E6) and teaching process (E7) are categorised under the expertise category. Student-centred teaching (H1), professional ethics (H2), quality of thinking (H3) and psychological qualities (H4) fall under the higher-order knowledge category.

We then used the coding scheme to abstract the key word from comments written by the student teachers in PAs. We used the binary values ‘0’ and ‘1’ to quantify the comment data to determine whether a comment contains the knowledge elements of the adopted coding scheme. If a knowledge element was represented in a sentence of a comment, the comment was coded ‘1’ on the corresponding item that represents this knowledge element; otherwise, ‘0’ was coded on the corresponding item.

To ensure coding accuracy, two researchers were trained to code the collected comments manually. A test set of 100 fragments was used to test the coding consistency between the two researchers. The intercoder reliability coefficient was 0.84 (Cohen’s Kappa), which indicates satisfactory reliability. Thus, the two researchers were asked to code all the comments. When their opinions on the comments differed, discussions were held until an agreement was reached. An example coded comment is presented in

Table 2. Coding format and an example coded comment.

Comment	Speech Was Fluent, and Arrangement Was Reasonable… [The Lecturer] Questioned the Students and Had Good Communication… The Teaching Process Indicated Clear Thinking and Always Focused on the Teaching Goals, and [the Lecturer] Grasped the Key Points…But He Was a Little Nervous at the Beginning.
Coded comment	Item	G1	G2	G3	G4	G5	E1	E2	E3	E4	E5	E6	E7	H1	H2	H3	H4
	Value	0	1	1	0	0	0	0	1	1	0	0	1	0	0	1	1

.

3.5. Data Analysis

ENA and statistical analysis were conducted to investigate the knowledge construction process of student teachers regarding teaching competence. As per the operation process of ENA, each comment for each individual was structured as a stanza. The knowledge elements within a stanza are specific to an evaluator and are therefore related to each other. The knowledge elements between stanzas are not specific to an evaluator and are therefore not correlated. The knowledge elements that appear and/or co-occur in a stanza reflect the evaluator’s cognition of the knowledge elements and the corresponding cognitive thinking during the writing of the comments.

All the comments of an evaluator were combined into an analysis unit. Each analysis unit comprised comments from a specific evaluator for different trainees. The concatenation of the set of knowledge elements of all stanzas within an analysis unit provides the overall picture of an evaluator’s level of knowledge within an epistemic framework and the cognition of that knowledge.

In order to obtain the general development information of student teachers’ cognition, we developed three measured parameters concerning the knowledge elements and the co-occurrences between the knowledge elements of the entire class during the four PA activities.

Table 3. The definition of the measured parameters.

Code	Definition	Description
$\mathsf{\alpha }$	Mean number of element types in each analysis unit	Represent the abundance of element types that the trainees used in the comments.
$\mathsf{\beta }$	Mean number of co-occurrence types in each analysis unit	Represent the abundance of co-occurrence types that the trainees used in the comments.
$\mathsf{\gamma }$	Mean number of co-occurrences in each analysis unit	Represent the strength of the connection between the co-occurrence elements the trainees used in the comments.

shows the definitions of the measured parameters.

We conducted statistical analysis to elaborate on the numbers and percentages of knowledge elements that appeared in the four PAs to detail the development information of student teachers’ knowledge constructs. Then, we conducted ENA to visualise the development of the cognitive structures and to identify differences and trends in the cognitive structures of the entire class throughout the PAs.

To explore the differences in knowledge construction between gender groups in the same PA setting. We divided all trainees into the male group (B, 12 trainees) and the female group (G, 8 trainees) based on gender characteristics for the generated comment data. The occurrence and co-occurrence of all knowledge elements for all trainees in each group were pooled to obtain common features that reflect the knowledge structure of the trainees in the group. The developmental patterns of the different groups were explored by comparing the centroid development diagrams of their epistemic networks through ENA and by calculating the mean number of element types that emerged in each PA for each group in every stage.

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results and their interpretation as well as the experimental conclusions that can be drawn.

4. Results

4.1. The Overall Developmental Characteristics of Cognition throughout the Four PAs

We conducted a statistical analysis of the content of the comments collected from the first to fourth PAs and determined the following factors: the mean number of element types, the mean number of co-occurrence types and the mean number of co-occurrences in each analysis unit.

Table 4. Numers of the measured parameters throughout the PAs.

PA Activity	$\mathsf{\alpha }$	$\mathsf{\beta }$	$\mathsf{\gamma }$
First	9.30	27.60	43.90
Second	11.17	40.00	67.94
Third	12.40	55.65	102.30
Fourth	13.25	65.55	133.85

presents statistics on the corresponding parameters throughout the four PAs.

As presented in Table 4, the student teachers’ cognition of teaching competence developed throughout the comment-based PA activities. In the first stage between the first and second PAs, the trainees focused on developing a diverse understanding of knowledge elements but did not understand them in sufficient depth. In the second stage between the second and third PAs, the trainees focused on the development of the diversity of the links between the various elements and the ability to integrate various elements for commenting. In the third stage between the third and fourth PAs, the trainees were more inclined to make deeper connections between knowledge elements from a fixed number of dimensions and developed a deeper understanding of individual knowledge elements.

We conducted a further statistical analysis on the comments collected in the first to fourth PAs.

Table 5. Numbers and percentages of the knowledge elements throughout the PAs.

	Codes	First PA		Second PA		Third PA		Fourth PA
		(N)	(%)	(N)	(%)	(N)	(%)	(N)	(%)
General knowledge	G1	7	1.56%	16	3.32%	32	4.77%	34	4.39%
	G2	38	8.48%	47	9.75%	63	9.39%	75	9.68%
	G3	63	14.06%	56	11.62%	60	8.94%	60	7.74%
	G4	13	2.90%	23	4.77%	40	5.96%	51	6.58%
	G5	50	11.16%	51	10.58%	57	8.49%	59	7.61%
	Total	171	38.16%	193	40.04%	252	37.55%	279	36.00%
Expertise	E1	57	12.72%	57	11.83%	76	11.33%	87	11.23%
	E2	53	11.83%	36	7.47%	53	7.90%	61	7.87%
	E3	7	1.56%	14	2.90%	29	4.32%	37	4.77%
	E4	59	13.17%	49	10.17%	57	8.49%	47	6.06%
	E5	6	1.34%	22	4.56%	33	4.92%	35	4.52%
	E6	25	5.58%	19	3.94%	35	5.22%	32	4.13%
	E7	23	5.13%	27	5.60%	41	6.11%	64	8.26%
	Total	230	51.33%	224	46.47%	324	48.29%	363	46.84%
Higher-order knowledge	H1	9	2.01%	13	2.70%	22	3.28%	40	5.16%
	H2	11	2.46%	21	4.36%	30	4.47%	31	4.00%
	H3	18	4.02%	19	3.94%	28	4.17%	37	4.77%
	H4	9	2.01%	12	2.49%	15	2.24%	25	3.23%
	Total	47	10.50%	65	13.49%	95	14.16%	133	17.16%

presents the numbers and percentages of knowledge elements used in the class-wide comments.

Table 6. Average numbers of knowledge elements in each comment throughout the PAs.

PA Activity	General Knowledge	Expertise	Higher-Order Knowledge
First	1.500	2.018	0.412
Second	1.990	2.309	0.670
Third	2.211	2.842	0.833
Fourth	2.447	3.184	1.667

presents the average number of knowledge elements in each element category that appeared in each comment.

Table 5 and Table 6 indicate that the student teachers increased their use of general knowledge (1.5 → 1.990, 38.16% → 40.04%) and higher-order knowledge elements (0.4123 → 0.6701, 10.50% → 13.49%) and decreased their use of expertise elements (51.33% → 46.47%) in the first stage. In the second stage, the student teachers decreased their use of general knowledge elements (40.04% → 37.55%) and increased their use of expertise elements (46.47% → 48.29%). In the third stage, the numbers of general knowledge (2.211 → 2.447, 37.55% → 36.00%) and expertise elements (2.842 → 3.184, 48.29% → 46.84%) used in the fourth PA were not significantly higher than those used in the third PA; however, the number and proportion of higher-order knowledge elements (0.833 → 1.667, 14.16% → 17.16%) used were significantly higher in the fourth PA.

4.2. The Detailed Developmental Characteristics through an Epistemic Network Analysis

On the basis of the coded comments’ data for all four PA activities, we constructed ENA to visualise the overall cognitive structures of the trainees in the four PA activities. Based on the results shown in Table 4, Table 5 and Table 6, the knowledge construction evolution over the four PAs can be identified. As suggested by Huang et al., the learning process can be divided into three periods: the initial period, the collision and sublimation period and the stabilisation period [65].

As illustrated in Figure 3, the largest confidence interval is achieved in the second PA, which indicates that the student teachers’ thinking was unstable and diffuse in the first PA to the second PA [66]. After that, the confidence interval gradually decreased, which indicates that the relevant cognition of the teaching competence tended to become more stable and cohesive. This suggests that the second PA was a turning point in the change in the cognitive network structure and the collision and sublimation period. It is also supported by the trend of the mean values, which initially move left–right to left and then move from left to right after the second PA.

For further exploring the developmental characteristics of the knowledge construction, we separately mapped the epistemic networks correspondences from the first to fourth PAs in Figure 3. The cognitive network implied in the first PA has a relatively simple structure, with many isolated and indiscernible points and only a few lines. The cognitive network in the second PA has more connecting lines than that in the first PA; however, these lines have a very light colour. Most of the lines in the cognitive network of the third PA are significantly thicker than those in the cognitive networks of the first two PAs. The distribution of the network began to equalise in the second stage. Some of the points in the cognitive network of the fourth PA (such as H1, H3 and H4) became larger than the previous interrater. The findings presented in this paragraph confirm that the student teachers focused on developing the diversity of knowledge elements, the complexity of the connections between knowledge elements and the depth of understanding of knowledge elements in the first, second and third stages, respectively.

In addition to the overall analysis of the network structure, we analysed the elements of the network diagram specifically. In the first PA, the knowledge construction process of the student teachers was focused on establishing connections between five knowledge elements: G5, E2, E1, G3 and E4. In the second PA, the student teachers established connections between the aforementioned five elements and increased the usage frequency of E5, G1 and G4 and their links to other knowledge elements. In the third PA, greater emphasis was placed on several knowledge elements located in the lower part of the figure, including E3, E6, E7 and H3. In the fourth PA, the teachers’ knowledge construction process was oriented towards establishing connections between E7, H3, H1, H4 and E1. The findings presented in this paragraph validate the conclusions presented in the first part of this section.

4.3. The Developmental Characteristics of the Knowledge Construction of Different Groups

We first conducted ENA to generate centroid development diagrams of the male and female groups throughout the four PA activities (Figure 4) and further mapped the differences between the epistemic networks of the two groups in each PA (Figure 5), as well as their corresponding tables of differences in statistical terms (

Table 7. t-test for differences in epistemic networks between the two groups.

	Group	X						Y
		Mean	SD	N	t	Effect Size (d)	p	Mean	SD	N	t	Effect Size (d)	p
1	B	−0.89	1.04	12	−5.87	2.44	0.00 *	0.00	1.15	12	0.00	0.00	1.00
	G	1.33	0.65	8				0.00	2.25	8
2	B	−1.24	0.94	10	−5.85	2.81	0.00 *	0.00	1.71	10	0.00	0.00	1.00
	G	1.55	1.06	8				0.00	2.41	8
3	B	−0.65	0.53	12	−5.59	2.71	0.00 *	0.00	1.40	12	0.00	0.00	1.00
	G	0.98	0.71	8				0.00	0.76	8
4	B	−0.88	0.89	12	−6.08	2.65	0.00 *	0.00	2.37	12	0.00	0.00	1.00
	G	1.32	0.72	8				0.00	1.28	8

* Significant at p ≤ 0.05.

).

Figure 4 presents the centroid development diagrams of the male and female groups throughout the PA activities. This figure indicates that the overall centroid movement trends of the male and female groups from the first to fourth PAs are the same along the X-axis and Y-axis. On the X-axis, the centroids of both groups first move from right to left and then move from left to right after the second PA. On the Y-axis, these centroids exhibit bottom-up movement throughout the PAs. The two dashed boxes in Figure 4 are completely independent, which indicates that equal distribution cannot occur in these boxes. The figure also illustrates that the confidence interval of the female group is larger than that of the male group, which indicates that the female student teachers’ thinking was more divergent than that of their male peers [66].

Figure 5 shows the differences between the structures of the epistemic networks of the two groups in each PA. In the first and second PAs, the distributions of the qualitative spaces on the Y-axis are greater in the female group than those in the male group, while the third and fourth PAs present the opposite situation. From a statistical point of view, there are statistically significant differences on the X-axis between the two groups on each of the PAs, as shown in Table 7.

We then further counted the average number of knowledge elements per category in each comment for the first through fourth PAs for both male and female groups. The relevant statistics are shown in Figure 5.

By referring to Figure 6 from the perspective of activity sequences, the following developmental characteristics of male and female groups can be identified. In the first stage, the male group focused on developing knowledge elements from the general knowledge and higher-order knowledge categories, whereas the female group maintained a steady development of the elements from each category. In the second stage, the male group focused on developing expertise elements and stopped developing general knowledge elements, whereas the female group made a strong effort to develop general knowledge and expertise elements. In the third stage, the male group maintained a steady development of elements from each category, whereas the female group made a strong effort to develop elements from the higher-order thinking category while maintaining a steady development of elements from the remaining categories. Overall, the knowledge structure of the male group tended to favour elements from the expertise and higher-order thinking categories, whereas that of the female group tended to favour elements from the general knowledge category.

5. Discussion

This study aimed to investigate the developmental characteristics of student teachers’ knowledge construction and their trajectories in PA. By conducting ENA and using mathematical statistics, we found that the development of student teachers’ knowledge construction differed in different stages and groups.

We first examined the general developmental trends of student teachers’ knowledge construction of teaching competences by analysing the coded comments for all trainees. We found that the student teachers’ knowledge construction was developed during the PA activities, and their cognition about teaching competencies was improved. This finding could be regarded as practical evidence for the research results of Farrell, who revealed that reflective practice can promote the development of teachers’ cognitive thinking [67], and as extending the findings of Cheng and Hou, who indicated that PA promotes the use of cognitive–metacognitive thinking by trainees when they provide feedback [22].

We further found that, in different stages of learning, the student teachers exhibited different development rates for distinct categories of knowledge elements. Moreover, the cognitive networks of the student teachers exhibited a general pattern of development from diffuse to tight and from simple to complex.

On the basis of the different characteristics of the student teachers’ three knowledge construction stages that were analysed in this study, instructors can adapt the curriculum process for the development of student teachers’ teaching competence. For example, in the early part of a course, instructors might place more emphasis on teaching trainees the definition of each knowledge element and other content regarding the knowledge elements. In the later part of the course, instructors might be more inclined to instruct the trainees on the connections between various knowledge elements. The most crucial aspect in this part of the course is to help advance trainees’ understanding of different knowledge elements.

However, the knowledge construction of the student teachers did not exhibit marked changes in the third stage. This result indicates that the trainees maintained a high development rate for the general knowledge and expertise elements during the first three PAs. Subsequently, their development rate for these elements decreased in the fourth PA. Moreover, their level of awareness of these knowledge elements reached a relatively constant range of values. The development of knowledge elements from the general knowledge and expertise categories tended to stagnate for trainees in the third stage. We speculate that this phenomenon might have been caused by the trainees becoming bored when writing a fourth comment on the same type of content, which might have led to a decrease in their motivation and initiative [68]. The trainees continued to deepen their understanding and thus increased their usage frequency of higher-order knowledge elements during PAs.

To address the aforementioned problem, we recommend that instructors place a limit on the number of times PA methods can be used for content based on similar types of topics within the same session. After limiting the usage frequency of these methods, instructors can intervene in the lesson through other methods. This approach prevents trainees from becoming bored and promotes the development of their knowledge structures at a deeper level.

We also examined the obtained data through statistical analysis and ENA to determine the developmental characteristics of different trainee groups. We found that, throughout the assessment activity, the knowledge structure of the male group was biased towards knowledge elements from the expertise and higher-order knowledge categories, whereas that of the female group was biased towards knowledge elements from the general knowledge category. In addition, the female group exhibited significantly less development in knowledge elements from the expertise and higher-order knowledge categories than the male group did. The results of this study are in agreement with the finding of Shen et al. that student teachers exhibit gender differences in the development of their cognitive thinking in a learning process [69].

The aforementioned conclusion is related to the fact that cultivating teaching competence is a kind of complex task, and task complexity may influence learners’ knowledge construction process and learning outcomes [70]. Accordingly, we suggest that instructors pay attention to gender differences in their daily teaching when training student teachers. Teaching interventions can be targeted according to these differences to achieve precision teaching and learning. For example, assessment scaffolds can be provided to female/male trainees before and during a teaching activity to help them break through their learning bottlenecks; a particular expertise framework can be provided, and additional discussion and reflection time can be designed during the second PA activity to give female/male trainees the opportunity to deeply understand knowledge and lead to better cognitive stability and cohesion.

6. Conclusions, Limitations and Future Work

This study used content analysis to examine the process of knowledge construction by student teachers in PAs. We used statistical methods and performed ENA to conduct an in-depth exploration on the data collected from four PA activities to investigate the development of the knowledge of teaching competence. First, we obtained statistics on all comments from different learning stages, which confirmed that the cognition of the teaching competence knowledge of the student teachers who participated in this study developed throughout the conducted course. Second, we classified knowledge elements into three different level categories: general knowledge, expertise and higher-order knowledge. Subsequently, we quantified comment data through ENA and statistical analysis to examine the developmental characteristics of the student teachers’ knowledge on different levels of knowledge in different stages. Finally, we compared the knowledge development trends of student teachers between different groups and in different stages. The results of this study provide evidence for instructors on the development of student teachers’ cognition of teaching competence and help them to carry out precision teaching and training.

The textual data generated in teaching and learning contain considerable personal cognitive information. The highlight of this study is our use of content analysis to mine the cognitive information in the PA comments to identify the characteristics and developmental patterns of student teachers’ knowledge construction in PAs. An increased quantity of such data can be analysed in future studies to build upon the current study and improve the accuracy and objectivity of the findings related to knowledge construction. This study has certain limitations. First, the selected research sample was insufficiently representative, and a relatively small number of comments was collected. The sample was only representative of junior student teachers from eastern China. It may be difficult to generalise the findings of this study to all grades of student teachers. Second, this study only analysed knowledge construction in a PA setting, and it is difficult to generalise the findings to all educational assessment settings. In a future study, we plan to examine the deeper links between student teachers’ cognition characteristics of teaching competence and their actual teaching performance in different grade levels during the process of knowledge construction. In addition, this study will combine the ENA and text mining to explore the cognitive differences and connections between teacher-assessment, student peer-assessment and self-assessment.