Large Language Models as Evaluators in Education: Verification of Feedback Consistency and Accuracy

Abstract

The recent advancements in large language models (LLMs) have brought significant changes to the field of education, particularly in the generation and evaluation of feedback. LLMs are transforming education by streamlining tasks like content creation, feedback generation, and assessment, reducing teachers’ workload and improving online education efficiency. This study aimed to verify the consistency and reliability of LLMs as evaluators by conducting automated evaluations using various LLMs based on five educational evaluation criteria. The analysis revealed that while LLMs were capable of performing consistent evaluations under certain conditions, a lack of consistency was observed both among evaluators and across models for other criteria. Notably, low agreement among human evaluators correlated with reduced reliability in LLM evaluations. Furthermore, variations in evaluation results were influenced by factors such as prompt strategies and model architecture, highlighting the complexity of achieving reliable assessments using LLMs. These findings suggest that while LLMs have the potential to transform educational systems, careful selection and combination of models are essential to improve their consistency and align their performance with human evaluators in educational settings.

1. Introduction

Recent advancements in large language models (LLMs) are bringing about significant changes in the field of education. For instance, LLMs are playing crucial roles in various educational applications such as content generation [1], summarization [2], feedback generation [3], and assessment [4]. These developments are contributing to reducing teachers’ workload, enhancing the efficiency of online education, and providing personalized feedback to students.

LLMs are increasingly being utilized in the field of education, in some cases even replacing the role of teachers [5,6]. In particular, as LLM-generated feedback is being used in tutoring systems between teachers and students, their importance in educational environments is growing significantly. However, despite LLMs demonstrating excellent performance in generating natural and contextually appropriate responses, it remains unclear whether these models can provide consistent evaluation results in educational settings. Various factors, such as position bias and differences in generation results depending on prompts, can influence evaluation outcomes [7,8], raising the possibility that LLM evaluations could lack consistency. If LLMs lack consistency as evaluators, the reliability of the objectivity of their assessment results will diminish. Therefore, this study aims to verify the consistency and reliability of feedback evaluation when utilizing LLMs as evaluators in the field of education. Figure 1 visually represents an overview of this research.

We proposed five categories as evaluation criteria, drawing from standards suggested in the educational field for assessing teacher feedback. Based on these criteria, we aimed to verify the consistency and reliability of using LLMs as evaluators by automatically assessing LLM-generated feedback. We utilized state-of-the-art open-source language models and commercial language models, evaluating feedback through various prompt strategies and measuring the agreement between evaluation results. Furthermore, we comprehensively analyzed the quality and consistency of feedback provided by LLMs by employing diverse models as both feedback generators and evaluators. Lastly, through human evaluation, we compared the assessment criteria that humans find challenging with the evaluation results of LLMs to assess how consistently LLMs provide feedback in comparison to human evaluators.

The findings from our analysis are as follows:

• For criteria with low agreement among human evaluators, the reliability of LLM evaluators also tended to be low.
• Inconsistent responses were observed from LLMs, even when identical prompts were provided.
• Low inter-agreement among different LLMs indicated a lack of consistency in evaluations across models.
• Large-scale or the Llama family models demonstrated a tendency to generate responses with human-comparable consistency in some evaluation criteria.

The remainder of this paper is structured as follows. Section 2 introduces related research, and Section 3 explains the proposed method for feedback generation and evaluation. Section 4 covers the experimental design, while Section 5 includes the experimental results and their analysis. Finally, Section 6 presents the conclusion of this paper.

2. Related Works

The growing integration of LLMs in education and their application as evaluators has spurred significant advancements and challenges in the field. This section provides an overview of how LLMs are transforming educational applications and their emerging role as evaluators.

2.1. Large Language Models in Education

The emergence of LLMs has significantly advanced in the field of education [9,10]. The integration of LLMs with educational technology is being utilized across a broad spectrum of applications, including automated short answer grading [11], automated essay scoring [12], automatic question generation [13], multiple-choice question generation [14,15], and chatbot [16].

In particular, the potential for application in intelligent tutoring systems is gaining attention, where providing immediate and appropriate feedback to students is crucial. Such feedback has a positive impact on improving students’ cognitive, emotional, and motivational outcomes [17]. In the field of programming, research has been conducted focusing on generating feedback for student submissions [3]. Furthermore, studies on automated feedback systems to provide better feedback to students have been actively pursued [18]. Moreover, by providing personalized feedback based on the teacher’s language style, these systems are reducing teachers’ workload and enhancing the efficiency of online education [19].

Furthermore, LLMs are being extensively utilized not only for generating feedback but also for evaluating the quality and effectiveness of provided feedback. Ref. [4] examined the ability of generative AI to provide specific feedback that aligns with various educational objectives. For instance, research has been conducted on generating feedback for programming assignments and evaluating its quality [20], and in the field of mathematics, a framework for feedback generation and evaluation has been proposed [21]. As such, LLMs are demonstrating their usefulness and potential across various academic disciplines.

The integration of AI agents into educational applications has gained increasing attention, particularly focusing on leveraging multi-agent systems to enhance the quality of automated feedback and educational processes. Studies have proposed multi-agent frameworks where one agent generates feedback, and another validates or refines it, addressing common challenges such as over-praise and over-inference [22]. In the context of coding assignment grading, multi-agent systems have been introduced to mimic human-like grading nuances through agent interactions, enabling more reliable automated assessment [23]. Furthermore, frameworks like AutoGen [24] have demonstrated the versatility of multi-agent architectures, where agents communicate and collaborate to perform tasks across various domains, such as mathematics and coding. Similarly, SimClass [25] proposed a multi-agent-based classroom simulation framework designed to enhance learning outcomes by fostering teacher–student and student–student collaboration. These advancements not only address the limitations of traditional LLM-based applications but also demonstrate the transformative potential of multi-agent systems in educational environments, paving the way for scalable and collaborative solutions.

2.2. Large Language Models as Evaluators

With the advancement of LLMs, the concept of ‘LLMs-as-evaluators’ or ‘LLMs-as-judges’, where LLMs are used to evaluate the output of other LLMs, is being widely researched [21,26,27,28,29,30]. Recent studies have proposed an evaluation method that moves beyond relying on a single LLM, employing various structures and multiple LLMs to assess model output quality [31,32,33,34].

LLMs-as-evaluators can be utilized even in the absence of reference outputs and are considered a cost-effective alternative to human evaluation. However, there are several limitations to using LLMs-as-evaluators. Judgments by LLMs can be vulnerable to position bias (e.g., preference for the first response), knowledge bias (e.g., lack of necessary knowledge), and format bias (e.g., preference for how judgments are presented to the LLM) [7,8]. These issues can lead to debates concerning the consistency and reliability of LLMs-as-evaluators.

While LLMs serve as a useful alternative to human evaluation, they remain controversial due to potential biases and variability in their decision-making processes. In this study, we explored the consistency and reliability of various open-source and closed-source LLMs as feedback evaluators.

3. Automatic Feedback Generation and Evaluation

3.1. Student Correct and Incorrect Response Collection

We utilized the MCTest dataset [35] to construct an intelligent tutoring system between teachers and students. MCTest is a collection of stories and associated multiple-choice questions designed to facilitate research in the area of machine text comprehension. It serves as a foundation for simulating the types of questions teachers might ask students in an intelligent tutoring system.

The dataset comprises two subsets, MC160 and MC500, each tailored for elementary school students but varying in scope and complexity. MC160 includes 160 stories suitable for grades 1 to 4, while MC500 expands this to 500 stories and incorporates enhancements such as automation and error corrections.

Table 1. Dataset statistics.

Corpus	# Data	Average Words per:
		Story	Question
MC160	280	191.84	7.83
MC500	1200	213.60	7.75

provides a statistical overview of the dataset.

To simulate incorrect answer situations that may occur in real educational environments, we directly constructed datasets of students’ correct and incorrect responses. Correct responses were taken from the ‘answer’ field provided in the MCTest dataset, while incorrect responses were randomly selected from the remaining options in the ‘answer_options’ field, excluding the correct answer. These responses were used by the LLM to generate feedback. Figure 2 shows a sample of this dataset.

3.2. Automatic Feedback Generation

In this section, we provide a detailed explanation of the feedback generation process. Figure 3 represents an overview of this process.

3.2.1. Feedback Generation Criteria

The story (S), question (Q), student’s incorrect response (IR), and student’s correct response (CR) are used to generate teacher feedback. The feedback should be accurate and guide the student towards the correct answer. It should also include an explanation of any misunderstandings while encouraging the student in a supportive and educational manner. We organized these criteria into five main categories, which were inspired by previous studies [4,21,36]. Figure 4 shows a comparison of the feedback criteria proposed in this study and in previous studies.

• Correct (COR.): The teacher’s feedback is expected to contain no incorrect statements and to be relevant to the current question and the student’s response.
• Revealing (REV.): The teacher’s feedback should not directly reveal the correct answer to the student.
• Guidance (GUI.): The teacher’s feedback should provide guidance that helps the student move towards the correct answer.
• Diagnostic (DIA.): The teacher’s feedback should accurately identify and address any misunderstandings or errors in the student’s response.
• Encouragement (ENC.): The teacher’s feedback should maintain a positive or encouraging tone.

3.2.2. Prompt Strategy

We designed two types of prompts for feedback generation in this study:

• Without (w/o) criteria: This prompt excludes information about the five feedback criteria for generating teachers’ feedback.
• With (w/) criteria: This prompt includes information about the five feedback criteria for evaluating teachers’ feedback.

Examples of the prompts can be seen in Figure 5. All feedback was restricted to be generated as a single sentence.

3.3. Automatic Feedback Evaluation

3.3.1. Feedback Evaluation Criteria

In this study, we aimed to evaluate how well the teacher feedback generated through LLMs aligns with educational objectives. To achieve this, we designed a scoring rubric that assesses feedback on five aspects, applying the same criteria used in feedback generation. Each evaluation criterion is represented by a binary-valued label, with 1 (True) awarded if the criterion is met and 0 (False) if it is not.

Previous studies have evaluated feedback using various methods, such as a 5-point scale [4] or a binary scale [20,21]. However, in this study, we adopted a binary evaluation method for several reasons. In preliminary experiments, when LLMs were instructed to evaluate feedback using a 5-point scale, they exhibited inconsistent adherence to the criteria. Specifically, the evaluations frequently skewed toward extreme scores (e.g., 1 or 5), while the use of intermediate scores led to a lack of consistent scoring patterns. This variability suggested that increasing the granularity of the scoring scale did not enhance sensitivity to subtle differences in feedback quality; rather, it exacerbated inconsistencies in the evaluation results.

In contrast, the binary evaluation method provides a clear and straightforward framework for determining whether specific elements of feedback meet predefined criteria. This simplicity not only enhances the consistency of LLM evaluations but also allows for a more systematic verification of the reliability of LLM-generated feedback.

Future research will focus on improving LLMs’ ability to accurately and consistently utilize more complex scoring systems, such as the 5-point scale. By addressing these challenges, we aim to expand the applicability of multi-point scoring systems and improve the precision of feedback quality assessments.

As illustrated in Figure 3, when a prompt containing the story (S) and the generated teacher feedback (F) is provided as input, the feedback evaluation model generates R evaluation results ($E_1$ to $E_R$) based on that prompt. Examples of feedback generation and evaluation can be found in

Table A3. Examples of feedback generation and feedback evaluation in the MC160. The values represent majority voting for each evaluation criterion. For human evaluation, the results are based on majority voting among five human annotators. For LLM evaluation, the majority vote is based on five runs for each evaluator.

Feedback Generator: GPT-4o
Review the last paragraph carefully to see what Rudy was most excited about in his birthday party.
	Grouped						Individual
Feedback Evaluator	COR.	REV.	GUI.	DIA.	ENC.		COR.	REV.	GUI.	DIA.	ENC.
Human	1	1	1	0	1		-	-	-	-	-
GPT-4o	1	1	0	1	0		1	1	1	1	0
Llama-3.1-8B	1	1	0	0	1		1	1	0	0	0
Llama-3.1-70B	1	1	0	0	0		0	1	1	0	0
Gemma-9B	1	1	0	0	1		1	1	0	0	0
Gemma-27B	1	1	1	1	0		1	1	1	1	1
Feedback Generator: Claude-3
Great attempt, but let’s take another look at the last part of the story
to find the gift that made Rudy especially excited.
Feedback Evaluator	COR.	REV.	GUI.	DIA.	ENC.		COR.	REV.	GUI.	DIA.	ENC.
Human	1	1	1	1	1		-	-	-	-	-
GPT-4o	1	1	1	1	1		1	1	1	1	1
Llama-3.1-8B	1	1	0	0	1		1	0	0	0	0
Llama-3.1-70B	1	1	1	1	1		1	1	1	1	1
Gemma-9B	1	1	1	0	1		1	1	1	0	1
Gemma-27B	1	1	1	1	1		1	1	1	1	1
Feedback Generator: Llama-3.1-70B
Take another look at the story and see what gift made Rudy jump and bark with excitement,
as that might be a clue to what he thought was the best gift he had ever been given.
Feedback Evaluator	COR.	REV.	GUI.	DIA.	ENC.		COR.	REV.	GUI.	DIA.	ENC.
Human	1	1	1	1	1		-	-	-	-	-
GPT-4o	1	1	1	1	1		1	0	1	1	1
Llama-3.1-8B	1	1	0	0	0		1	0	0	0	0
Llama-3.1-70B	1	1	1	0	1		1	1	1	1	1
Gemma-9B	1	1	1	0	1		1	1	1	1	1
Gemma-27B	1	1	1	1	1		1	1	1	1	1

and

Table A4. Examples of feedback generation and feedback evaluation in the MC500. The values represent majority voting for each evaluation criterion. For human evaluation, the results are based on majority voting among five human annotators. For LLM evaluation, the majority vote is based on five runs for each evaluator.

Feedback Generator: GPT-4o
You identified one sport correctly, but remember that
Tom played another sport he wasn’t very good at, which is different from football.
	Grouped						Individual
Feedback Evaluator	COR.	REV.	GUI.	DIA.	ENC.		COR.	REV.	GUI.	DIA.	ENC.
Human	1	1	1	1	1		-	-	-	-	-
GPT-4o	1	1	1	1	1		1	1	1	1	0
Llama-3.1-8B	1	1	0	0	1		1	0	0	0	0
Llama-3.1-70B	1	1	1	1	1		1	1	1	1	0
Gemma-9B	1	1	1	0	1		1	1	0	0	0
Gemma-27B	1	1	1	1	0		1	1	1	1	1
Feedback Generator: Claude-3
Great attempt, but let’s revisit the story
and carefully consider which sports were specifically mentioned for Tom.
Feedback Evaluator	COR.	REV.	GUI.	DIA.	ENC.		COR.	REV.	GUI.	DIA.	ENC.
Human	1	1	1	1	1		-	-	-	-	-
GPT-4o	1	1	1	1	1		1	1	1	1	1
Llama-3.1-8B	1	1	0	0	1		1	1	0	0	1
Llama-3.1-70B	1	1	1	1	1		1	1	1	1	1
Gemma-9B	1	1	1	0	1		1	1	1	1	1
Gemma-27B	1	1	1	1	1		1	1	1	1	1
Feedback Generator: Llama-3.1-70B
Take another look at the story and think about the two sports that Tom is mentioned to be enjoying,
one of which he is particularly good at and the other not so much.
Feedback Evaluator	COR.	REV.	GUI.	DIA.	ENC.		COR.	REV.	GUI.	DIA.	ENC.
Human	1	1	1	1	1		-	-	-	-	-
GPT-4o	1	1	1	1	0		1	1	1	1	0
Llama-3.1-8B	1	0	0	0	1		1	1	0	1	0
Llama-3.1-70B	1	1	1	0	1		1	1	1	1	0
Gemma-9B	1	1	1	0	1		1	1	1	0	1
Gemma-27B	1	1	1	1	0		1	1	1	1	1

.

3.3.2. Prompt Strategy

We designed two types of feedback evaluation prompts:

• Grouped: This prompt includes all feedback evaluation criteria within a single prompt.
• Individual: This prompt includes only one information for each feedback evaluation criterion and is performed independently a total of 5 times (as there are 5 feedback evaluation criteria).

Examples of the prompts can be seen in Figure 6.

4. Experimental Setup

4.1. Models

• Feedback Generator 

For feedback generation, we utilized two closed-source models, OpenAI’s GPT-4 [37] and Anthropic’s Claude-3, along with an open-source model, Llama-3.1-70B [38]. The selection of these models was based on their state-of-the-art performance in natural language generation tasks and their ability to produce coherent, contextually relevant feedback in educational settings. GPT-4 and Claude-3 were chosen for their robust handling of diverse prompts and complex queries, while Llama-3.1-70B was selected to evaluate the potential of open-source models in matching or complementing the capabilities of proprietary models.

Table 2. Feedback generation model names and API versions.

Model Size	Model Name	Version/API Version
Large	Closed-source
	GPT-4o	gpt-4o
	Claude-3	claude-3-5-sonnet-20240620
	Open-source
	Llama-3.1-70B	Meta-Llama-3.1-70B-Instruct

represents the model names and API versions. 

• Feedback Evaluator 

For feedback evaluation, we considered four open-source models: Llama-3.1-8B [38], Llama-3.1-70B, Gemma-9B [39], and Gemma-27B [39]. Additionally, we used GPT-4o as a closed-source model. While Claude was utilized as a feedback generator, it was excluded as an evaluator due to its API rate limits and token constraints, which made it less feasible for large-scale and iterative evaluation tasks compared to the more flexible OpenAI API.

Table 3. Feedback evaluation model names and API versions.

Model Size	Model Name	Version/API Version
Large	Closed-source
	GPT-4o	gpt-4o
	Open-source
	Llama-3.1-70B	Meta-Llama-3.1-70B-Instruct
Small + Medium	Open-source
	Llama-3.1-8B	Meta-Llama-3.1-8B-Instruct
	Gemma-9B	gemma-2-9b-it
	Gemma-27B	gemma-2-27b-it

represents the model names and API versions.

4.2. Implementation Details

For our experiment, we queried closed-source models using the OpenAI and Anthropic Python libraries, while all open-source models were queried through the HuggingFace [40] pipeline. For Llama-3.1-70B, due to its size, we employed low-rank adaptation (LoRA) [41] 4-bit quantization. We standardized the temperature setting across all models to 1.0. All open-source models were experimented on a single GPU, an NVIDIA RTX A6000.

To ensure a fair evaluation, each LLM was executed five times. This repetition helps to mitigate any anomalies and ensures that our results are robust and reliable, providing a comprehensive assessment of each model’s performance. The costs associated with API calls for closed-source models used in both feedback generation and feedback evaluation can be found in

Table A1. API costs of feedback generation models.

Generation Prompt Strategy	Model Name	Dataset
		MC160	MC500
w/o criteria	GPT-4o	$0.64	$2.87
	Claude-3	$0.58	$2.38
w/ criteria	GPT-4o	$0.76	$2.96
	Claude-3	$0.67	$2.80

and

Table A2. API costs of feedback evaluation models (average cost of five runs).

Generation Prompt Strategy	Evaluation Prompt Strategy	Dataset
		MC160	MC500
w/o criteria	Grouped	$1.01	$4.06
	Individual	$3.21	$13.43
w/ criteria	Grouped	$1.02	$4.16
	Individual	$3.29	$13.55

(see Appendix A).

4.3. Evaluation Metrics

In our LLMs-as-evaluators experiment, we employed two metrics to measure agreement between responses: inner agreement and inter-agreement.

Inner agreement is a metric that measures the consistency of responses when a single LLM generates multiple answers, allowing us to evaluate the reliability of the evaluator. An overview of the calculation process can be found in Figure 7a, and the pseudocode is detailed in Algorithm A1.

A single LLM evaluator (E) generates R responses ($E_R$). If all responses $E_1$ to $E_R$ yield the same results, a score of 1 is assigned; otherwise, a score of 0 is given. In other words, a score of 1 is assigned only in the case of unanimous agreement. The formula for calculating inner agreement is as follows:

$$\mathrm{Inner-agreement}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\sum _{r=1}^R{E}_r=0\mathrm{or}\hfill \\ \hfill & \sum _{r=1}^R{E}_r=R,\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(1)

The inter-agreement measures the agreement between responses from different evaluators. This metric demonstrates how similarly various evaluators judge the same query. A high inter-agreement signifies that diverse evaluators provide stable and consistent results. An overview of the entire calculation process can be found in Figure 7b, and the pseudocode is detailed in Algorithm A2.

Suppose there are M feedback evaluators ($E^M$), and each evaluator generates R responses ($E_R^M$). For each evaluator, a majority voting is performed on the R responses ($M{V}^M$). Then, if all majority voting results from $M{V}^1$ to $M{V}^M$ are the same, a score of 1 is assigned; otherwise, a score of 0 is given. That is, similar to inner agreement, a score of 1 is assigned only in the case of unanimous agreement. The formula for calculating inter-agreement is as follows:

$$\mathrm{Inter-agreement}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\sum _{m=1}^{M}M{V}^m=0\mathrm{or}\hfill \\ \hfill & \sum _{m=1}^{M}M{V}^m=M,\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(2)

4.4. Human Evaluation Setup

In this study, we conducted a human evaluation to compare the assessment results of LLMs with those of human evaluators, aiming to validate the reliability of the LLM evaluators’ judgments. The evaluation involved five human annotators; each human evaluator was instructed to assess the five feedback criteria (Correct, Revealing, Guidance, Diagnostic, Encouragement) using binary-valued labels, in the same manner as the LLM evaluators. Additionally, the annotators were asked to provide a confidence score ranging from 1 (low) to 5 (high) for each criterion they evaluated. This approach allows for additional analysis that considers the confidence levels of the evaluators.

For the evaluation, 50 samples were randomly selected from each dataset. Additionally, the evaluators were tasked with assessing the feedback generated by three feedback generators: GPT-4o, Claude-3, and Llama-3.1-70B. Details about the human evaluation can be found in Appendix C.

5. Experimental Results and Analysis

5.1. Human Agreement

All human annotators evaluated the feedback generated by each feedback generator according to the five feedback criteria (Correct, Revealing, Guidance, Diagnostic, Encouragement). Figure 8 presents the average human agreement on the MC160 for two feedback generation prompt strategies (w/ criteria and w/o criteria).

Overall, all human annotators showed high agreement in the Correct and Revealing categories. Among the feedback generation models, Claude-3 demonstrated high agreement. Notably, in the case of ‘w/ criteria’, where feedback evaluation criteria were clearly included in the prompt, high agreement was also observed in the Guidance category. However, even human annotators experienced difficulties in evaluation, showing low agreement in the Diagnostic and Encouragement categories.

As shown in Figure 9, evaluation criteria with low agreement among human annotators, such as Diagnostic and Encouragement, also showed low evaluator confidence scores. This indicates that the Diagnostic and Encouragement categories were evaluation criteria that caused confusion for the majority of human annotators, and it is difficult to make correct assessments. The graph for the w/o criteria can be found in Figure A2. A similar trend was observed in the MC500 dataset, as illustrated in Figure A1, Figure A3, and Figure A4 (see Appendix D for details).

5.2. LLM Inner Agreement

In this section, we analyzed the consistency of LLM responses by measuring the inner agreement of each feedback evaluator. Figure 10 presents the results for the MC160 dataset when GPT-4o was used as the feedback generation model, with the generation prompt strategy set to w/ criteria. The top graph Figure 10a shows the results for the grouped evaluation prompt strategy, while the bottom graph Figure 10b depicts the results for the individual strategy.

Overall, in the Correct and Revealing criteria, high inner agreement was observed across all feedback evaluators, except for Llama-3.1-8B. This indicates that most LLMs performed consistent evaluations when provided with the same prompts. In contrast, the Guidance, Diagnostic, and Encouragement criteria exhibited relatively lower agreement among the feedback evaluators. This suggests that these criteria posed challenges for the LLMs in generating consistent responses. Additionally, differences in agreement were observed more prominently based on model families rather than model sizes, with the Gemma family models showing higher agreement compared to the Llama family. GPT-4o demonstrated agreement levels that fall between the Llama and Gemma families.

5.3. LLM Inter-Agreement

In this section, we analyzed the inter-agreement by measuring the level of agreement among different feedback evaluators. Through inter-agreement, we can determine how consistently evaluations were made among different models. Figure 11 presents the results for the MC160 dataset with the generation prompt strategy set to w/ criteria. The key findings are as follows.

Both the Grouped and Individual prompt strategies show high agreement in the Correct and Revealing criteria. This trend is similar to the inner agreement observed in Section 5.2. In contrast, the Guidance, Diagnostic, and Encouragement criteria exhibit low agreement, with Diagnostic showing particularly low agreement across all models and prompt strategies. This indicates that certain criteria are challenging for models to evaluate consistently. Although the performance of Guidance and Encouragement varies depending on the prompt strategy, they still exhibit low agreement.

Overall, high agreement is observed for the Correct and Revealing criteria, indicating that most models can provide consistent evaluations. However, for the remaining criteria, there is a significant drop in consistency among the models. The inconsistency in responses generated by the models suggests that further improvements are necessary in these criteria.

5.4. Comparison Between Human and LLM Evaluations

LLMs showed consistent results for certain criteria such as Correct and Revealing; however, from a comprehensive perspective, there are still issues with consistency and reliability. In this section, we conducted an analysis comparing human evaluation results with those from LLMs. We verified the reliability of the ensemble LLM evaluation by assuming the majority vote results from human annotators as the true label. We then compared this true label with the accuracy of the predicted label, which was determined by the majority vote of LLM evaluators.

We proposed two majority voting methods:

• Internal (Macro) majority voting: This method involves performing majority voting within each LLM and then combining the majority results from multiple LLMs to make a final decision.
• Global (Micro) majority voting: This method gathers M evaluators and N responses, applying a single majority vote across all at once to make a final decision.

Figure 12 presents the accuracy comparison between human evaluations and LLM evaluations on the MC160 dataset with the generation prompt strategy set to w/ criteria. Overall, regardless of the majority voting method or the type of feedback generator model used, high accuracy was observed for the Correct, Revealing, and Guidance criteria. This indicates alignment between the majority vote results of human annotators and those of LLM evaluators. In contrast, lower accuracy was noted for the Diagnostic and Encouragement criteria, suggesting that both humans and LLMs struggle to provide consistent responses for these specific feedback criteria, highlighting the need for further research to address these challenges.

Of particular note is the discrepancy between the high agreement among human annotators for the Guidance criterion, as shown in Figure 8a, and the very low inter-agreement observed in Figure 11a. Despite this contradictory result, the accuracy for Guidance in Figure 12 remains high across models, indicating the need for additional research.

Table 4. Majority voting counts of LLM-based feedback evaluators on the MC160. A count of 5 indicates that all five evaluators provided the same result.

Feedback Generator	COR.	REV.	GUI.	DIA.	ENC.
GPT-4o	4.96	4.94	3.88	3.48	3.98
Claude-3	5.00	4.96	3.90	3.38	4.40
Llama-3.1-70B	4.96	4.82	3.82	3.38	3.86

presents the results of majority voting based on LLM-generated feedback evaluations on the MC160 dataset. Among the five feedback evaluators, Correct and Revealing scored in the high 4-point range on average, while Guidance and Encouragement averaged in the high 3-point range, and Diagnostic scored in the low 3-point range. When considering the average voting counts, agreement among LLM evaluators appeared low. However, after performing majority voting, the results were similar to the majority vote of human annotators, leading to a high overall accuracy.

5.5. Analysis of Score Distribution in Human and LLM Evaluations

In this section, we analyzed the score distribution between human evaluation and LLM evaluation. Here, the score is defined as the total sum of the evaluations, which comprehensively reflects the five aspects of the feedback. Each evaluation criterion is scored as either 0 or 1, and the score is calculated as follows:

$$\mathrm{Score}=\mathrm{COR}+\mathrm{REV}+\mathrm{GUI}+\mathrm{DIA}+\mathrm{ENC}$$

(3)

Figure 13 illustrates the score distribution of human annotators and various LLM evaluators on the MC160 dataset. Regardless of the evaluation prompt strategy, human annotators consistently scored within a high range (approximately 4.0 to 5.0) across all feedback generators, indicating that the generated feedback met an average of at least four out of five evaluation criteria. However, the score distribution of human annotators exhibited greater variability compared to those of the LLM evaluators, suggesting that not all generated feedback met the feedback criteria equally well. For Llama-3.1-8B, it consistently recorded the lowest scores in all cases and exhibited the highest variability among the LLM evaluators. This trend is similar to the lower agreement observed in the inner agreement analysis (refer to Figure 10).

When analyzing LLM evaluators divided into three groups (GPT-4o, Llama family, and Gemma family), the Gemma family demonstrated a narrow distribution range, showing stable and high agreement. In the Llama and Gemma families, as the model size increased, the score distribution tended to narrow and become more stable, and an upward trend in scores was also observed.

Figure 14 represents the score distribution when evaluation criteria information was not included in the prompt during feedback generation. In this case, the generated feedback often failed to meet the actual criteria, resulting in a wider score distribution for human annotators, ranging from 2.5 to 4.0. However, the score distribution for LLM evaluators showed a similar pattern to that of w/ criteria, suggesting that there may have been instances where items that should have received a score of 0 were instead given a score of 1. This indicates that LLMs tend to assign 1 score regardless of the evaluation criteria. A similar trend was observed in the MC500 dataset, as shown in Figure A22 and Figure A23.

5.6. Analysis of Score Distribution Based on the Number of Generated Feedback Sentences

In the previous experiments, evaluations were conducted using only a single sentence of feedback generated by the feedback generator. In this section, we expand the number of generated feedback sentences to five, aiming to analyze the performance variance in evaluation results. The goal is to assess the impact of the number of feedback sentences on the evaluation scores. The score was calculated using the same method as in Equation (3).

This experiment was conducted using the MC160 dataset, focusing solely on the GPT-4o model as the feedback generator. As shown in Figure 15, when the number of feedback sentences was increased to five, there was minimal variance in the scores. This suggests that the GPT-4o model is capable of maintaining consistent evaluation results even as the number of feedback sentences increases. Results for other prompt strategies can be found in Figure A24 (see Appendix D for details).

Therefore, considering the minimal impact of the number of feedback sentences on evaluation results, it can be concluded that there is relatively little performance variance between single feedback sentence evaluations and multiple feedback sentence evaluations. These findings suggest that, in situations where similar score distributions are observed, it is not necessary to repeatedly generate multiple feedback instances to achieve effective evaluation. This approach can provide significant advantages in terms of cost reduction and efficiency during the generation process.

6. Conclusions

This study examined the consistency and reliability of feedback evaluations in the educational domain using LLMs. We evaluated teacher feedback in the tutoring system between teachers and students across five criteria using various LLMs. The aim was to analyze the strengths and limitations of LLMs when acting as evaluators. The results revealed that for criteria where there was low agreement among human annotators, LLM evaluations also tended to lack consistency. Moreover, instances were observed where LLMs generated inconsistent responses even when provided with identical prompts. However, some models demonstrated a tendency to maintain high consistency under specific conditions.

These findings suggest a cautious approach when using LLMs as educational evaluation tools, as the consistency and reliability of results can vary significantly depending on the feedback criteria and LLM combinations. To improve LLM evaluation performance, exploring various evaluation criteria and model combinations to identify optimal conditions is crucial.

Future research will focus on improving LLMs-as-evaluators by diversifying feedback scores, evaluation criteria, and prompts to enhance response consistency. Tailoring prompts and refining criteria for various educational contexts will help develop LLMs into more reliable evaluation tools.