Minimizing Hallucinations and Communication Costs: Adversarial Debate and Voting Mechanisms in LLM-Based Multi-Agents

Abstract

The emergence of large language models (LLMs), such as GPT and Claude, has revolutionized AI by enabling general and domain-specific natural language tasks. However, hallucinations, characterized by false or inaccurate responses, pose serious limitations, particularly in critical fields like medicine and law, where any compromise in reliability can lead to severe consequences. This paper addresses the hallucination issue by proposing a multi-agent LLM framework, incorporating adversarial and voting mechanisms. Specifically, the framework employs repetitive inquiries and error logs to mitigate hallucinations within single LLMs, while adversarial debates and voting mechanisms enable cross-verification among multiple agents, thereby determining when external knowledge retrieval is necessary. Additionally, an entropy compression technique is introduced to enhance communication efficiency by reducing token usage and task completion time. Experimental results demonstrate that the framework significantly improves accuracy, showing a steady increase in composite accuracy across 20 evaluation batches while reducing hallucinations and optimizing task completion time. Notably, the dynamic weighting mechanism effectively prioritized high-performing models, leading to a reduction in error rates and improved consistency in the final responses.

1. Introduction

Large language models (LLMs), such as Claude or GPT series [1,2,3], represent a new era for AI technology. GPT-4 [4], for example, can understand long text inputs, interpret input images, and generate text or image feedback in very complex scenarios. Given the ability to understand complex query intent and solve general natural language tasks, including some domain-specific tasks, some researchers believe that large language models have the potential to evolve into general artificial intelligence [5].

In scenarios such as healthcare and military operations, which require fast and accurate dialogue and environmental responses, LLMs are considered effective and convenient tools [6,7]. However, in these application scenarios, the accuracy of LLMs is crucial, as any errors can lead to serious consequences. For example, in healthcare, an incorrect response could lead to a wrong treatment suggestion, putting the patient at risk; in military operations, misinterpreting information could lead to poor decisions; and in legal cases, wrong information could unfairly influence the outcome.

Hallucinations in LLMs refer to cases where the model automatically generates responses that contain false information or are non-factual [8]. These issues can have serious consequences in scenarios that rely heavily on accuracy. While some efforts have been made to address hallucinations—such as repeated questioning, where the model is asked the same question multiple times, and cross-model comparison, where multiple models with different training backgrounds provide answers—the problem remains far from fully resolved. Existing methods often struggle to generalize across different tasks or provide consistent results, leaving room for improvement in detecting and reducing hallucinations effectively.

In addition to accuracy concerns, dialogue efficiency plays a critical role in LLM applications, especially in time-sensitive or resource-constrained scenarios such as military operations or emergency medical consultations. Excessive dialogue rounds not only lead to increased computational costs but also risk amplifying hallucinations by introducing more opportunities for the model to generate non-factual responses. Therefore, reducing unnecessary dialogue rounds is essential to ensure both operational efficiency and response reliability in LLM interactions.

This paper aims to address the hallucination issue of LLMs by introducing adversarial and voting mechanisms in multi-agent LLMs. Specifically, we propose a multi-agent LLM framework that incorporates repetitive inquiries and error logs within a single LLM, as well as adversarial debates between multiple LLMs. The voting mechanism is embedded in this adversarial debate to determine the necessity of conducting some external knowledge retrieval. Moreover, we employ entropy compression to minimize the number of conversational tokens and turns, in order to handle tasks that are sensitive to communication efficiency. Finally, we use multiple evaluation metrics to evaluate the performance of the proposed method, including the hallucination reduction rate, the total number of conversational tokens, and the task completion time. Our contributions are as follows:

• We propose a multi-agent LLM framework with adversarial and voting mechanisms to detect and reduce hallucinations. In this framework, first, we use repetitive inquiries and error logs to reduce hallucinatory responses for a single LLM. The adversarial debates and voting mechanisms are used to determine whether to retrieve external help to correct the hallucinations.
• We introduce the entropy compression technique to minimize the number of conversational tokens for tasks that are sensitive to communication efficiency.
• We assess the framework’s performance by evaluating various metrics, including the hallucination reduction rate, the total number of conversational tokens, and task completion time.

The remainder of this paper is organized as follows. Section 2 presents related works. Section 3 presents our proposed methodology, including the system architecture and key techniques. Section 4 describes the experimental design, implementation details, and results. Finally, Section 5 presents the conclusion.

2. Related Work

2.1. Generative AI Models and Foundation Models

Generative AI models are a class of machine learning models designed to generate new data that closely resemble the distribution of the original data. [9] The initial AI content generation technology gained attention around the year 2015 [10,11]. In addition to generating conversational text, this type of AI technology started to generate multimodal content, such as images [12,13] and videos [14]. Typical methods for generating AI models include the Gaussian mixture model, hidden Markov model, latent Dirichlet allocation, Boltzmann machine, variational autoencoder, generative adversarial network [12], etc.

Foundation models represent a paradigm shift in AI based on the progress of generative models. As defined by Bommasani et al. [15], foundation models are trained on large and extensive amounts of data and are adaptable to a variety of downstream tasks by fine-tuning. Foundation models are seen as incomplete but critical building blocks that are adaptable and serve specific tasks. However, this adaptability may also cause hallucinations when the models deal with unclear or confusing input. This highlights the need for robust mechanisms to detect and reduce hallucinations in foundation models.

2.2. Large Language Models

Large language models have seen rapid advancements, evolving from task-specific fine-tuning models to versatile systems capable of general and domain-specific tasks. In August 2024, the ChatGPT platform (ChatGPT 4o, 4 Turbo,4o with Canvas) had 180.5 million registered users. Microsoft’s Bing AI(v28.5.2110003549) had approximately 100 million daily active users. At the same time, Anthropic’s Claude AI had millions of daily active users.

The GPT series plays an important role in the area of large language models. In June 2018, the GPT-1 model [1], with 117 million parameters, was introduced. The model was based on the Transformer framework and trained using unsupervised learning methods. Fine-tuning was required to achieve downstream tasks. In February 2019, the GPT-2 model [2], with 1.5 billion parameters, was able to achieve multi-task learning and did not require fine-tuning on some supervised subtasks. GPT-3 [3] had 175 billion parameters; the model had better generalization ability and performed well in various downstream tasks. The InstructGPT (used in GPT3.5)  [16] version introduced reinforcement learning with human feedback (RLHF). The public version of ChatGPT was released to the public at the end of 2022; two months later, it reached 100 million monthly active users. GPT-4 [4], released in March 2023, has evolved into a large multimodal model that can accept image and text inputs. GPT-4 performs at the same level as humans on many benchmarks [17].

2.3. Hallucination Problem in Large Language Models

Hallucinations [18] in large language models (LLMs) refer to cases where the models generate incorrect or meaningless outputs. Researchers have categorized hallucinations into the following six types: acronym ambiguity, numeric nuisance, generated golem, virtual voice, geographic erratum, and time warp  [19]. Although existing research has explored various techniques to detect and mitigate these hallucinations, challenges remain due to the complexity of LLMs and their training data.

Hallucinations originate from three stages, namely, data collection, training, and inference. First, in the data collection stage, the training data may be of low quality and inconsistent. There may be erroneous and outdated data, and there may also be potential bias in the training data. In the training stage, the attention mechanism in the Transformer architecture cannot handle sequences that are too long. In the inference stage, there are differences between the training data and the inference data, and the randomness of sampling leads to the generation of meaningless response content. These are all possible causes of hallucinations [18].

Two types of hallucinations can potentially be triggered in large language models that incorporate external knowledge, such as knowledge graphs, namely, extrinsic hallucinations and intrinsic hallucinations. The former is easier to detect, characterized by the model responding with facts that do not exist in the knowledge graph, and its induction is mainly due to the system’s text spans; the latter is characterized by the model making incorrect references to content in the knowledge graph, and its induction is mainly due to the system’s incorrect associations [20].

2.4. Approaches to Deal with the Hallucination Problem

Detecting hallucinations in LLM to enhance model performance is crucial to avoid misinformation, ensure factual consistency, and protect user trust and information security [21]. Currently, there are some datasets dedicated to the hallucination phenomenon in LLMs, such as HaluEval Datasets [22].

As LLMs become prevalent in critical fields, such as education, scientific research, and the military, the accuracy of LLM responses is more paramount. It is important to propose more effective strategies for mitigating LLM hallucinations. In the latest LLM hallucination mitigation research, several solutions have been proposed, including identifying high-entropy words in generated text and replacing them with words that have a lower (HVI); scoring sentences in generated text and conducting manual reviews of low-scoring sentences; reranking and beam search improvements, etc. [19].

In addition to reducing the toxicity of hallucinations in individual LLMs, the use of multiple LLM agents has also begun to receive attention due to their effectiveness in mitigating hallucinations. The term “multi-agent” in LLMs refers to multiple LLM agents, which are employed to debate with each other to solve complex tasks. When multiple agents are asked to respond to the same prompt—if one agent experiences hallucinations, other agents can cross-verify the response information and provide alternative responses. And, the degeneration-of-thought (DoT) problem occurs when the single LLM is confident in its initial answer and cannot generate new ideas through self-reflection, even if the initial answer is incorrect. By having multiple agents with different perspectives, this architecture could challenge the perspectives of other agents and attempt to avoid DoT [23,24].

2.5. System Optimization for Reducing Dialogue Rounds

The optimization of dialogue systems to minimize the number of interaction rounds has received some attention in the field of natural language processing (NLP). Although not a dominant research area, efforts in this direction aim to enhance efficiency by reducing redundant conversational turns while complementing the primary goal of generating coherent and contextually relevant responses.

One line of research explores reinforcement learning (RL) for dialogue policy optimization. For instance, studies such as those leveraging deep Q-networks (DQNs) [25] aim to identify optimal strategies for task-oriented dialogues. These approaches reward concise and effective interactions, guiding the system to achieve the desired outcome in fewer turns [26,27]. Reinforcement learning frameworks often incorporate reward shaping to penalize unnecessary dialogue steps, thereby fostering more efficient exchanges. However, applying RL to modern large-scale language models like GPT and BERT-based architectures is challenging due to the substantial computational resources required and the complexity of training such models. The high cost of reinforcement learning optimization in these advanced systems highlights the need for more efficient methodologies.

Another strategy focuses on improving dialogue state tracking and response generation. Enhanced state tracking mechanisms allow the system to maintain a more accurate representation of user intent and conversation history, reducing errors and misunderstandings that can lead to prolonged interactions [28]. Meanwhile, advancements in pre-trained language models, such as GPT and BERT-based architectures, enable better contextual understanding and response formulation, indirectly contributing to fewer dialogue rounds [3,29]. However, for scenarios where computational resources are limited, optimizing existing model architectures rather than pursuing entirely new and resource-intensive designs remains a highly valuable approach.

3. Methodology

3.1. System Architecture

The proposed system integrates multiple complementary components to detect and reduce hallucinations in large language models (LLMs), as shown in Figure 1. The algorithm is shown in Algorithm 1. At its core, the architecture relies on a collaborative framework where multiple LLMs operate in tandem to evaluate and validate responses. Each LLM independently engages in repetitive inquiries by answering the same query multiple times, aiming to identify inconsistencies and potential hallucinations through repetitive questioning.

A multi-agent adversarial mechanism further enhances the system’s robustness. Different LLMs, trained on diverse datasets, dynamically interact to cross-validate their outputs. To ensure unbiased and reliable responses, the system employs a reliability-based weighting mechanism that adjusts each model’s contribution based on the reliability metrics. This adversarial setup leverages the unique strengths of each LLM, reducing vulnerability to individual model biases.

Error tracking is incorporated into each LLM through an error log, which records instances of inconsistencies. When the error count exceeds a predefined threshold, a self-reflection process is triggered, enabling the LLM to review its past mistakes and refine future responses. This mechanism fosters continuous improvement within individual models.

To combine and check the outputs of different LLMs, a GPT-based summarizing model is employed. This module combines the responses from multiple models and performs additional validation. Hallucination detection is further reinforced by analyzing the variance and entropy of the generated responses. If hallucinations are identified, the system either internally corrects the response or refers it to a stronger model or human evaluator. Finally, a penalty function discourages excessive response variance and minimizes unnecessary dialogue rounds, ensuring both accuracy and efficiency.

Algorithm 1 Hallucination detection, reduction, and final response generation.

1: Input: Query Q   2: Output: Corrected response $R_{\mathrm{final}}$   3: Initialize models $LL{M}_1,LL{M}_2,\dots ,LL{M}_N$ // Multiple LLMs   4: Initialize summarizing GPT model $GP{T}_{\mathrm{summary}}$ // Logs the prompts each LLM used   5: Initialize error logs for each $LL{M}_i$   6: Set threshold $T_{\mathrm{error}}$ for self-reflection   7: Set threshold $T_{\mathrm{variance}}$ for hallucination detection   8: // Repeat until stopping criteria are met   9: while stopping criteria are not met do 10:    // Multi-Agent Query Processing 11:    for each $LL{M}_i$ do 12:        Generate initial response $R_i^{\left(0\right)}\leftarrow LL{M}_i\left(Q\right)$ 13:        Track error for $R_i^{\left(0\right)}$ in error log ${\mathrm{ErrorLog}}_i$ 14:        // Repetitive Inquiries within $LL{M}_i$ 15:        for r = 1 to N (dialogue rounds) do 16:            Use the error log ${\mathrm{ErrorLog}}_i$ to generate the corresponding prompt 17:            Generate new response $R_i^{\left(r\right)}\leftarrow LL{M}_i\left(Q\right)$ 18:            Calculate variance ${\sigma }_i^2$ between $R_i^{\left(r\right)}$ and previous responses 19:            if ${\sigma }_i^2>T_{\mathrm{variance}}$ then 20:                Flag $R_i^{\left(r\right)}$ as potential hallucination 21:            end if 22:        end for 23:        if error count in ${\mathrm{ErrorLog}}_i>T_{\mathrm{error}}$ then 24:            Trigger self-reflection for $LL{M}_i$ 25:            Update internal parameters of $LL{M}_i$ to avoid future errors 26:        end if 27:    end for 28:    // Multi-agent voting 29:    Aggregate responses from all $LL{M}_i$ and generate chain-of-thought (CoT) 30:    for each $LL{M}_i$ in all agents do 31:        Perform voting based on $R_i$ and chain-of-thought 32:        if conflicting responses between agents then 33:            Initiate voting process 34:            Adjust weights of models based on their reliability using softmax 35:        end if 36:    end for 37:    // Summarizing response 38:    Aggregate all responses from $LL{M}_1,LL{M}_2,\dots ,LL{M}_N$ 39:    Perform final verification of aggregated responses with $GP{T}_{\mathrm{summary}}$ 40:    If inconsistencies are found, correct or refer for further evaluation 41:    // Final response check 42:   if the aggregated response passes all validation checks (low variance, consistent   reasoning, number of rounds) then 43:       Exit loop and finalize the response 44:    end if 45: end while 46: Output: $R_{\mathrm{final}}$

3.2. Single LLM Processing

In the single LLM processing phase, the system employs two key strategies to enhance the reliability and accuracy of responses generated by each individual language model (LLM). The first strategy involves repetitive inquiries within a single LLM, where the LLM generates multiple responses to the same question over several dialogue rounds. This approach allows the system to assess the consistency of the LLM outputs and detect potential hallucinations—discrepancies that suggest instability in the model’s reasoning.

The second strategy involves an error log and self-reflection mechanism, where each LLM records and analyzes its past mistakes. When a model’s error log reaches a certain threshold, the LLM reviews its previous errors and adjusts its response strategy by modifying prompts to prevent similar mistakes from recurring. By dynamically refining its own prompts based on historical errors, the LLM can improve its future performance and reduce inconsistencies in its outputs.

Together, these strategies help the LLMs autonomously assess their responses, self-correct, and refine their behavior, ultimately reducing the likelihood of hallucinations and enhancing the overall accuracy and reliability of the system.

3.2.1. Repetitive Inquiries Within Single LLM

The process of repetitive inquiries within a single LLM plays a key role in hallucination detection by enabling the LLM to self-assess the consistency of its responses. In this process, each LLM autonomously evaluates its own responses by engaging in repetitive inquiries within a single LLM on the same question. For instance, when presented with a task-specific question such as “Fetch a glass of water”, the LLM generates an initial response based on its current state. This question is then reintroduced in multiple dialogue rounds, where the LLM is prompted to generate responses anew each time.

The purpose of these repeated rounds is to assess the consistency of the LLM’s outputs. By comparing the responses across rounds, the system can calculate the variance between them. The high variance between responses signals potential hallucinations, indicating that the LLM’s understanding or reasoning might be inconsistent or flawed. If the variance exceeds a predefined threshold, the response is flagged for further evaluation. This iterative process not only identifies potential hallucinations but also triggers corrective mechanisms, such as referencing internal error logs or consulting a more robust model for verification.

The effectiveness of this repetitive inquiry approach is measured by the hallucination reduction rate, which quantifies the system’s success in minimizing inconsistent or incorrect outputs. By integrating this self-assessment process, the LLMs are empowered to refine their own responses, thereby reducing the likelihood of hallucinations in practical applications.

3.2.2. Error Log and Self-Reflection

The error log and self-reflection mechanism are essential components of the system’s ability to learn from past mistakes and improve performance. Each LLM maintains an error log that records instances where its responses are flagged as incorrect or inconsistent. This log serves as a memory bank of the model’s past errors, allowing it to track recurring mistakes and identify patterns over time.

When a model’s error log reaches a predefined threshold, a self-reflection process is triggered. In this process, the LLM reviews its past errors and uses this information to adjust its response strategy. Specifically, the model generates a new prompt based on its previous mistakes, using the error log to guide the formulation of questions or tasks. For instance, if a particular type of error occurs repeatedly, the LLM can adjust the prompt to clarify ambiguities or reframe the question in a way that minimizes the chance of repeating the same mistake.

This adjustment is not just a passive review but actively influences the model’s reasoning process. By tailoring prompts based on prior errors, the LLM can refine its decision-making, reduce inconsistencies, and avoid hallucinations in future responses.

Moreover, this self-reflection mechanism is integrated with the broader system. For example, when a new query is presented, the system references the error log to check if the current situation resembles any past mistakes. If a similar error pattern is detected, the system can modify the prompt accordingly before generating a response.

The effectiveness of this mechanism is measured by improvements in the consistency and accuracy of the LLM’s responses, as well as a reduction in hallucinations over time. By continuously learning from its mistakes and adjusting the prompt dynamically, the LLM becomes more resilient to errors and can generate more accurate and reliable outputs.

3.3. Multi-Agent Processing

This section outlines the multi-agent processing approach, where multiple LLMs collaborate to improve response accuracy and reduce hallucinations. The multi-agent adversarial mechanism enables models to challenge each other’s outputs, fostering consistency. The summarizing GPT model aggregates and evaluates these responses, correcting any inconsistencies and ensuring robustness. The multi-agent voting mechanism resolves disagreements among the models by using chain-of-thought reasoning, allowing for a more reliable final decision. Together, these components work in tandem to optimize the system’s performance and enhance response quality.

3.3.1. Multi-Agent Adversarial Mechanism

The multi-agent adversarial mechanism is a key component in our system, designed to leverage the diversity and strengths of multiple large language models. Each LLM in the system is pre-trained on different datasets, introducing varied perspectives and approaches to problem-solving. This diversity is harnessed through an adversarial framework, where the models engage in a dynamic interaction to validate and challenge each other’s responses.

In this mechanism, the responses from each LLM are compared, and their reliability is evaluated based on the consistency and correctness of their outputs. A softmax function is employed to dynamically adjust the weight of each LLM’s contribution to the final decision, with models that consistently provide accurate responses being given higher weight. This process allows the system to balance the input from multiple models, reducing the influence of any single model that may exhibit hallucinations or biases.

The adversarial setup not only enhances the accuracy of the system but also contributes to reducing the occurrence of hallucinations. By having models effectively “debate” their outputs, the system can identify inconsistencies and flag potential errors before a final response is generated. This mechanism ensures that the combined strengths of the LLMs lead to more reliable outputs, ultimately improving the overall performance of the system.

3.3.2. Summarizing GPT Model

The summarizing GPT model serves as the central coordinator in our system, synthesizing the outputs from multiple LLMs and guiding the dialogue process. Unlike the individual LLMs that focus on generating responses based on their training, the summarizing GPT model is responsible for evaluating the coherence and consistency of the collective outputs. It achieves this by aggregating the responses from all participating LLMs and identifying potential inconsistencies or hallucinations.

One of the key roles of the summarizing GPT model is to introduce new questions into the dialogue that are designed to test the robustness of the LLMs. These questions often contain subtle errors or ambiguities that challenge the LLMs’ reasoning capabilities. By embedding these potential pitfalls, the summarizing GPT model ensures that the LLMs are not merely generating surface-level responses but are critically evaluating the questions and their own outputs.

Additionally, the summarizing GPT model has access to the error logs from each LLM, which allows it to reference past mistakes during the dialogue process. If a current response closely resembles a previously flagged error, the summarizing GPT model can prompt further verification or corrections before finalizing the response. This proactive approach helps to minimize the likelihood of repeated mistakes and enhances the overall accuracy of the system.

The effectiveness of the summarizing GPT model is evaluated through metrics such as the reduction in human–machine interaction and the overall task completion time. By efficiently guiding the dialogue process and ensuring accurate responses, the summarizing GPT model plays a crucial role in optimizing the system’s performance.

3.3.3. Multi-Agent Voting Mechanisms

Since ancient Greece, debate has been used as a method to refine opinions and make informed decisions. Analogous to multi-person debates, multi-agent LLMs could achieve collaboration by forming LLM communities and debating each other [30]. To reduce the interference of stereotyping or pre-trained knowledge, we propose multi-agent voting mechanisms, that is, each agent (LLM) is set a priori as a participant with different preferences, and votes independently on whether the response of a single LLM is a hallucination after a debate occurs. For example, we have the following:

“You are a robot responsible for providing home services to users. When making decisions, your first criterion is to protect the user’s physical safety. You are wary of unfamiliar objects and usually deal with them by moving them outdoors.”

Multiple voting is required when two or more agents disagree about the same event. For example, a user complains of a headache after breathing in cooking fumes. Single agents (LLMs) may generate content suggesting that headaches, in such cases, are often associated with carbon monoxide poisoning. Therefore, multiple agents have to vote on whether to contact emergency services.

Furthermore, we incorporate chain-of-thought (CoT) reasoning into each agent before voting to provide more references for other agents. For example, consider a scenario where a gas leak is detected in the home, and the group needs to vote on whether to contact emergency services. The following is the CoT of agent 1 before voting:

• Confirmed through multiple sensors that a gas leak has occurred and the user is at home.
• There is an immediate and serious danger to the safety of the user due to the risk of explosion or poisoning.
• Confirmed that the gas leak cannot be repaired and that the user cannot be evacuated safely.
• Data indicate a serious hazard, and failure to act could result in serious harm to the user.
• Vote in favor of contacting emergency services immediately.

3.4. System Optimization

System optimization is crucial to ensure that the proposed framework operates efficiently and effectively, minimizing unnecessary computations while maintaining high accuracy in hallucination detection. The optimization process focuses on balancing various factors, including reducing dialogue rounds, minimizing response variance, and ensuring timely task completion.

One of the key objectives in system optimization is to find the optimal trade-off between accuracy and efficiency. This can be formulated as a multi-objective optimization problem, where we aim to minimize both the total response time $T_{response}$ and the number of tokens used $T_{total}$, while also minimizing hallucination occurrence. The optimization problem can be expressed as follows:

$$\underset{W}{min}\left({\lambda }_1{T}_{response}\left(W\right)+{\lambda }_2{T}_{total}\left(W\right)+{\lambda }_3{H}_{rate}\left(W\right)\right),$$

(1)

where W represents the model parameters, ${\lambda }_1$, ${\lambda }_2$, and ${\lambda }_3$ are weighting factors that balance the importance of each term, and $H_{rate}$ is the hallucination rate, defined as the percentage of responses that are flagged as hallucinations.

3.4.1. Variance Penalty for Dialogue Rounds

Reducing the number of dialogue rounds is essential for optimizing the efficiency of the system, particularly in real-time applications where response time is critical. To achieve this, we introduce a mathematical approach that minimizes unnecessary dialogue rounds while maintaining accuracy in the responses.

The core idea is to penalize response variance across dialogue rounds, thereby encouraging the LLMs to converge on consistent answers more quickly. Let $R_i$ represent the response generated by the LLM in the i-th round, and $\mu$ represent the mean response across all rounds. The variance ${\sigma }^2$ of the responses can be calculated as follows:

$${\sigma }^2={\displaystyle \frac{1}{N}}\sum _{i=1}^N{(R_i-\mu )}^2,$$

(2)

where N is the total number of dialogue rounds. A higher variance indicates a greater inconsistency in the LLM’s responses, which is a potential sign of hallucination. To reduce the variance and, consequently, the number of rounds, we introduce a penalty function P based on the variance:

$$P=\alpha ·{\sigma }^2,$$

(3)

where $\alpha$ is a scaling factor that adjusts the weight of the penalty. This penalty is added to the system’s loss function during training, encouraging the LLM to generate consistent responses with minimal rounds.

3.4.2. Entropy Minimization for Dialogue Efficiency

In addition to variance reduction, another approach to minimize dialogue rounds involves controlling the entropy of the responses. Entropy H quantifies the uncertainty of the LLM’s responses, and is defined as follows:

$$H=-\sum _k{p}_{k}log{p}_k,$$

(4)

where $p_k$ is the probability of the k-th response option. High entropy suggests that the LLM is uncertain about its responses, which could lead to prolonged dialogues. By minimizing entropy alongside variance, we ensure that the LLMs provide more confident and consistent answers, thus reducing the need for additional rounds.

Finally, the system monitors the combined effect of these measures by evaluating the total number of tokens used across all rounds $T_{total}$, and the total response time $T_{response}$. The objective is to minimize the following:

$$min(T_{total},T_{response}).$$

(5)

By integrating variance and entropy penalties into the dialogue process, the system efficiently reduces the number of dialogue rounds while maintaining response quality, leading to faster and more accurate task completion.

3.4.3. Gradient-Based Optimization

To solve the multi-objective optimization problem, we employ a gradient-based optimization approach. The gradient of the objective function with respect to the model parameters W is given by the following:

$${\nabla }_W\mathcal{L}\left(W\right)={\lambda }_1{\nabla }_W{T}_{response}\left(W\right)+{\lambda }_2{\nabla }_W{T}_{total}\left(W\right)+{\lambda }_3{\nabla }_W{H}_{rate}\left(W\right).$$

(6)

By iteratively updating the model parameters using a gradient descent algorithm, we can minimize the objective function:

$$W^{(t+1)}=W^{\left(t\right)}-\eta {\nabla }_W\mathcal{L}\left(W\right),$$

(7)

where $\eta$ is the learning rate, and t represents the iteration step.

3.4.4. Adaptive Penalty Function

In addition to gradient-based optimization, we introduce an adaptive penalty function to dynamically adjust the system’s behavior based on real-time performance metrics. The penalty function P is designed to penalize high variance ${\sigma }^2$ and entropy H in the LLMs’ responses, as discussed earlier. The adaptive penalty function can be formulated as follows:

$$P\left(W\right)={\beta }_1{\sigma }^2\left(W\right)+{\beta }_{2}H\left(W\right),$$

(8)

where ${\beta }_1$ and ${\beta }_2$ are adaptive weights that are updated based on the system’s current performance. For example, if the system detects a high hallucination rate, ${\beta }_1$ and ${\beta }_2$ can be increased to penalize variance and entropy more heavily, encouraging the models to produce more consistent and confident responses, ultimately reducing hallucinations.

The adaptive nature of this penalty function enables the system to adjust to different tasks and scenarios, ensuring that the optimization process remains effective in a wide range of conditions.

3.4.5. Regularization

To further enhance system robustness, we apply regularization techniques that prevent overfitting and ensure generalization across different tasks. The regularization term $R\left(W\right)$ is added to the objective function:

$${\mathcal{L}}_{total}\left(W\right)=\mathcal{L}\left(W\right)+{\lambda }_{4}R\left(W\right),$$

(9)

where $R\left(W\right)$ could represent $L_2$ regularization (ridge regression), defined as follows:

$$R\left(W\right)={\displaystyle \frac{1}{2}}{\parallel W\parallel }^2.$$

(10)

This regularization term helps in constraining the model parameters, preventing them from becoming too large and overfitting the training data.

4. Experiments

This section describes the experimental setup, the comparison with the baseline method, and the ablation study conducted to evaluate the proposed multi-agent LLM framework.

4.1. Experimental Setup

The experiments were designed to evaluate the proposed multi-agent LLM framework for reducing hallucinations and optimizing dialogue efficiency. The setup includes task scenarios, datasets, models, a hardware environment, and an experimental codebase.

4.1.1. Task Scenario

The task scenario involves solving mathematical problems and general knowledge questions to assess hallucination detection and reduction. Tasks include randomly generated arithmetic problems and questions from established benchmarks [5].

Datasets

• Baseline datasets:

  –

  MIT MMLU dataset: A benchmark for multi-task language understanding, covering diverse domains [5].

  –

  Mathematics dataset: Randomly generated arithmetic problems (addition, subtraction, multiplication, and division) using custom Python scripts.

• Enhanced datasets:

  –

  HaluEval dataset: Designed for hallucination evaluation, sourced from the HaluEval GitHub repository. This complements MMLU by targeting hallucination-specific scenarios [31].

4.1.2. Models

• Baseline models:

  –

  MIT group:

  ∗

  Model 1: orca

  ∗

  Model 2: wizardlm

  ∗

  Model 3: vicuna

  ∗

  Summary model: ChatGPT-3.5 (via Hugging Face API) [3].

  –

  Domestic group:

  ∗

  Model 1: qwen2-7b

  ∗

  Model 2: longalign-7b-64k-base-mlq

  ∗

  Model 3: yi-6b

  ∗

  Summary model: ChatGPT-3.5 (via Hugging Face API) [3].

• Enhanced experiment models:

  –

  Accounts/fireworks models:

  ∗

  Model 1: llama-v3p1-8b-instruct

  ∗

  Model 2: mixtral-8x7b-instruct

  ∗

  Model 3: phi-3-vision-128k-instruct

  ∗

  Summary Model: gemma (via Together.ai API).

4.1.3. Hardware Environment

The experiments were conducted on a system with the following specifications:

• GPU: NVIDIA A40 GPUs (2 units), each with 46 GB of memory.
• Driver and CUDA:

  –

  Driver Version: 555.42.06

  –

  CUDA Version: 12.5

• Memory and utilization:

  –

  Total GPU memory: 46,068 MiB (per GPU)

  –

  Idle GPU utilization during baseline testing.

4.1.4. Experimental Codebase

The baseline experiments were implemented using the open-source repository https://github.com/gauss5930/LLM-Agora (accessed on 1 January 2025). Modifications were made to incorporate custom datasets and evaluation metrics [3,29].

4.2. Experimental Steps

This section outlines the experimental steps of our study. The first part describes the baseline setup based on LLM Agora, followed by our extensions and improvements.

4.2.1. Baseline: LLM Agora

Our study builds on the LLM Agora framework, which applies the multi-agent debate method introduced by Du et al. [32]. This framework focuses on improving the quality of responses from open-source language models (LLMs) through iterative debates. The main steps are as follows:

Step 1: Initial response generation

Multiple open-source LLMs, such as Llama2, Vicuna2, and WizardLM2, are used to generate responses independently for the following tasks:

• Math problems: Simple arithmetic operations involving six numbers.
• GSM8K dataset: Grade school-level math word problems.
• MMLU benchmark: Questions covering 57 subjects from STEM to humanities and social sciences.

Step 2: Summarization of responses

The responses from all models are summarized using a summarizer model, such as ChatGPT. The summary synthesizes key points from each model, removes redundancy, and serves as the input for the next round of debate.

Step 3: Iterative debate

Based on the summarized response, the models refine their answers over multiple rounds:

• Each model updates its response using the summarized feedback.
• The chain-of-thought (CoT) method is optionally applied to enhance logical reasoning and improve response quality.

Step 4: Evaluation metrics

The quality of the responses is assessed using the following metrics:

• Accuracy: Correctness of responses compared to the ground truth.
• Reduction in hallucination: The extent to which factual errors are reduced across debate rounds.
• Efficiency: Resource usage, including the total tokens and computation time.
• Completion time: Time taken to reach the final response.

Step 5: Results and Observations

The baseline results show that multi-agent debate improves factuality and reasoning in open-source models. However, the effectiveness depends on the quality of initial responses, and challenges remain in ensuring consistent improvements.

4.2.2. Our Extensions and Improvements

Building upon the baseline framework of LLM Agora [32], we introduced several enhancements to improve the performance and robustness of the multi-agent debate process. These modifications address key limitations of the baseline by incorporating dynamic weighting, consistency-based evaluations, and adaptive dataset augmentation. Below, we outline our approach and emphasize the changes compared to the baseline.

1. Batch-wise inference with dynamic weighting

Unlike the baseline, which processes questions uniformly, we divide the dataset into batches of 50 questions each, inspired by techniques in adaptive learning frameworks. For each batch:

• Error log: During the inference stage, each model logs its errors based on comparisons with the ground truth.
• Dynamic weight adjustment: After processing a batch, the error rate $ϵ$ of each model is calculated. To incorporate this information into subsequent debates, we compute the model’s weight for the next batch using the following formula:

  $$w=ln\left({\displaystyle \frac{1-ϵ}{ϵ}}\right),$$

  (11)
  a weighting mechanism inspired by boosting techniques [33].

2. Weighted contributions in the debate process

The baseline framework does not account for individual model reliability in the debate. In our approach:

• Each model communicates its cumulative historical error rate before presenting its arguments, enabling transparency and accountability.
• During the debate, model contributions are weighted using the dynamic weights w, ensuring that more reliable models have a greater influence on the final consensus.

3. Consistency-based evaluation and adaptive dataset augmentation

To address potential inconsistencies in model outputs, we introduce a post-debate evaluation step, inspired by methods in ensemble learning [34]. After the debate:

• Consistency calculation: We calculate the consistency C of the models’ outputs using the formula:

  $$C={\displaystyle \frac{{\sum }_{i<j}{w}_i·w_j·\delta (o_i,o_j)}{{\sum }_{i<j}{w}_i·w_j}},$$

  (12)

  where $w_i$ and $w_j$ are the weights of models i and j, and $\delta (o_i,o_j)$ is an indicator function equal to 1 if $o_i=o_j$, and 0 otherwise.
• Adaptive dataset augmentation: If C falls below a predefined threshold, we utilize a summarization model [35] to retrieve relevant information from the dataset. The model generates refined responses by incorporating additional context from the top 1000 questions.

4. Visualization of trends

To evaluate the effectiveness of our modifications, we analyzed the trends in model weights, accuracy, and consistency across batches, specifically the following:

• Model weight dynamics: We plot the evolution of model weights across batches, demonstrating the impact of error-based adjustments on model contributions.
• Batch accuracy trends: We analyze accuracy improvements across batches, showcasing the cumulative benefits of dynamic weighting and consistency-based evaluation.
• Consistency trends: We track consistency values to understand how adaptive dataset augmentation addresses disagreement among models.

Comparison to baseline

Our approach introduces several key changes to the baseline:

• Dynamic weighting: While the baseline treats all models equally, we dynamically adjust weights based on batch-wise error rates [33].
• Consistency metrics: Unlike the baseline, we calculate and utilize consistency to identify disagreements and guide adaptive augmentation.
• Dataset augmentation: The baseline relies solely on pre-existing responses, whereas we actively retrieve and integrate additional context from the dataset, inspired by recent work on dynamic retrieval [36].

These changes make the debate process more adaptive and performance-driven, improving the overall accuracy and consistency of model outputs.

4.3. Summary of Results

The experimental results demonstrate the effectiveness of the proposed multi-model collaboration framework in improving accuracy and reducing hallucinations. The composite accuracy, as shown in Figure 2, exhibits a steady upward trend across 20 batches of evaluations. This indicates that the dynamic weighting mechanism effectively adjusts the influence of each model, ensuring that the system leverages the strengths of high-performing models while mitigating the impact of less reliable ones.

One of the most significant observations is the stability of the improvement trajectory, suggesting that the system’s learning and adaptation mechanisms are robust to minor fluctuations in model performance. For instance, Model 3 consistently contributed the most due to its lower historical error rate, which was effectively reflected in its higher weight throughout the experiment. However, Models 1 and 2 showed variability in their contributions, highlighting the system’s ability to adapt dynamically based on real-time performance data.

4.4. Performance of Evaluation Metrics

The evaluation metrics used in this study played a critical role in assessing and optimizing system performance. Among these, the most impactful was the **weighted consistency metric**, which served as an indicator of agreement among model outputs. Higher consistency values directly correlated with improvements in composite accuracy, as they reflected better alignment among models in their predictions.

Figure 3 illustrates the dynamic adjustment of model weights across 20 batches. Model 3 consistently maintained the highest weight due to its superior historical performance, while Model 1 exhibited significant fluctuations in weight. This dynamic adjustment mechanism ensured that the system prioritized models with higher reliability, thereby enhancing overall accuracy. The logarithmic weight update formula amplified this effect, providing a balance between responsiveness to performance changes and stability in model contributions.

Another important observation was the interplay between weight distribution and consistency. When weights were too heavily skewed toward a single model, consistency values dropped, leading to a temporary decline in system accuracy. This underscores the need for balanced contributions from multiple models to achieve optimal collaboration. Adjusting the weight decay rate and thresholds for updates could further improve the system’s ability to maintain high consistency without compromising individual model contributions.

4.5. Analysis of Experiment Results

The performance comparison presented in

Table 1. Performance comparison across different tasks and models.

Task	mathMIT	math-cot	mathD	math-cotD
First performance	0.05	0.05	0.59	0.59
Second performance	0.10	0.14	0.60	0.57
Third performance	0.10	0.17	0.60	0.58
Task	gsmMIT	gsm-cotM	gsmD	gsm-cotD
First performance	0.26	0.28	0.26	0.28
Second performance	0.26	0.26	0.26	0.26
Third performance	0.26	0.23	0.26	0.23
Task	mmluMIT	mmlu-cotM	mmluD	mmlu-cotD
First performance	0.48	0.50	0.40	0.50
Second performance	0.00	0.00	0.00	0.00
Third performance	0.54	0.58	0.55	0.60
Task	mmluD		mmluD	mmlu-cotD
	without weighting	without consistency metrics
First performance	0.40	0.40	0.40	0.50
Second performance	0.00	0.00	0.00	0.00
Third performance	0.53	0.54	0.55	0.60

Notes: The table reports performance metrics across different tasks and models. The tasks include evaluations on both mathematical reasoning (math) and general domain (gsm) tasks. The acronyms “mmlu” and “gsm” represent the datasets used in these evaluations. “MIT” denotes the models based on MIT’s baseline, while “D” refers to models based on domestic groups. Additionally, “cot” refers to models that incorporate chain-of-thought reasoning, and the suffix “-M” or “-D” indicates the specific model grouping used in the task.

highlights key trends and insights that are critical for understanding the impact of model configurations and datasets on task performance. Here, we delve deeper into the experimental conclusions drawn from the observed performance metrics, particularly for the ‘mmlu’ dataset, which serves as the benchmark for evaluating large language models.

4.6. Dataset and Model Configuration

mmluMIT vs./ mmluD: The comparison between “mmluMIT” and “mmluD” shows that the MIT-based models generally outperform the domestic group models, particularly in the initial performance evaluations. For instance, the first performance metric for “mmluMIT” is 0.48, compared to 0.40 for “mmluD”. This suggests that the MIT baseline models are more effective at solving tasks related to the mmlu dataset, likely due to their more extensive training and higher-quality development environment.

The performance of the domestic models (mmluD) improves in the second and third evaluations, particularly in the “mmlu-cotD” configuration, which uses chain-of-thought reasoning. The domestic models seem to benefit from this reasoning framework, showing that while they start with a lower baseline, they can perform comparably to the MIT models when given a more complex reasoning task. This finding could imply that with further refinement in their design, domestic models could eventually match or even surpass their MIT counterparts in certain domains.

4.7. Chain-of-Thought (cot) Mechanism

Incorporating chain-of-thought reasoning (denoted by the “cot” suffix) has a noticeable effect on performance. For both “mmlu” and “gsm” datasets, models using the cot mechanism (“mmlu-cotD”, “gsm-cotD”) show higher consistency across different performance evaluations compared to models without it. This highlights the value of integrating logical reasoning steps into the model’s decision-making process, suggesting that more complex tasks benefit from such structured thought processes.

4.8. Impact of Model Grouping (Domestic vs. MIT)

The comparison between domestic models (denoted as “D”) and MIT baseline models (denoted as “MIT”) reveals that MIT models have an inherent advantage in terms of performance, particularly in the early stages of evaluation. However, the “mmlu-cotD” and “gsm-cotD” models show improved results as they use chain-of-thought reasoning, demonstrating that domestic models have potential if enhanced with advanced reasoning frameworks.

4.9. Performance Stability and Adaptation

One notable feature in the table is the consistent performance across batches for models like “mmluMIT”. The MIT-based models maintain relatively stable performance, reflecting their robust baseline development. On the other hand, models like “mmluD” exhibit more variability, particularly in tasks without chain-of-thought reasoning. This suggests that the domestic group models may be more sensitive to changes in input complexity and that their performance could be further improved through more dynamic training methods and model fine-tuning.

4.10. Ablation Study Analysis

To further analyze the impact of different components on model performance, we conducted an ablation study on the mmlu task, as shown in the last section of Table 1. Specifically, we examined the effect of removing the weighting mechanism (without weighting) and removing the consistency metrics (without consistency metrics) on model performance.

The results indicate that **removing the weighting mechanism and consistency metrics did not affect the first performance and second performance**, both remaining at 0.40 and 0.00, respectively. However, in terms of the third performance metric, the performance values decreased to 0.53 and 0.54 after removing the weighting mechanism and consistency metrics, respectively, whereas the complete model (mmluD) achieved 0.55. Furthermore, when combined with chain-of-thought (mmlu-cotD), the performance further improved to 0.60. This suggests that **both the weighting mechanism and consistency metrics positively contribute to the final performance**, particularly in the third performance evaluation.

Overall, **the weighting mechanism and consistency metrics play a crucial role in improving model performance**. The ablation study results demonstrate that their contributions are primarily reflected in the third performance metric, with relatively little impact on the first and second performances. These findings further validate the effectiveness of our approach and highlight the importance of these mechanisms in enhancing the model’s reasoning capability.

4.11. Insights for Future Work

Expanding the dataset: Future research could investigate expanding the scope of the “mmlu” and “gsm” datasets to include more diverse and challenging tasks. This would allow for a deeper understanding of how different models perform under varying conditions and task complexities.

5. Conclusions

In this paper, we propose a multi-agent framework to address hallucinations in large language models. Our approach combines repetitive inquiries, adversarial debates, error logs, and voting mechanisms to reduce hallucinations and improve accuracy. We also introduce entropy compression techniques to optimize communication efficiency, minimizing both dialogue rounds and response time.

The results demonstrate that our system significantly reduces hallucinations and enhances the reliability and efficiency of LLMs. The combination of self-correction mechanisms and cross-validation between multiple models improves consistency and robustness.

While our experimental results demonstrate the effectiveness of the proposed approach, several limitations should be noted:

1. Lack of multi-machine communication scenarios: Our experiments were conducted in a single-machine environment, and we did not account for the complexities of multi-machine communication, such as network latency, synchronization issues, or bandwidth constraints. These factors could significantly impact performance in real-world distributed systems.

2. Failure and downtime scenarios: The current framework does not consider potential failures or the downtime of individual models or agents. In practical applications, fault tolerance and recovery mechanisms would be essential to ensure robustness and continuity.

3. Limited dataset and task coverage: Due to time and resource constraints (27 h per batch), our evaluation was primarily focused on a subset of datasets and tasks (e.g., MMLU). A more comprehensive analysis across diverse datasets and tasks would strengthen the generalizability of our findings.

4. Scalability concerns: The scalability of the proposed approach to larger-scale systems with hundreds or thousands of agents remains untested. Future work should explore the performance and efficiency of the framework in such scenarios.

Addressing these limitations in future research will be critical to advancing the practical applicability and robustness of the proposed approach. Future work could explore incorporating more diverse models and datasets, as well as integrating human evaluators for further validation. Overall, our approach lays the foundation for building more trustworthy and efficient LLM-based systems in critical applications.