Fact-Aware Generative Text Summarization with Dependency Graphs

Abstract

Generative text summaries often suffer from factual inconsistencies, where the summary deviates from the original text. This significantly reduces their usefulness. To address this issue, we propose a novel method for improving the factual accuracy of Chinese summaries by leveraging dependency graphs. Our approach involves analyzing the input text to build a dependency graph. This graph, along with the original text, is then processed by separate models: a Relational Graph Attention Neural Network for the dependency graph and a Transformer model for the text itself. Finally, a Transformer decoder generates the summary. We evaluate the factual consistency of the generated summaries using various methods. Experiments demonstrate that our approach improves about 7.79 points compared to the baseline Transformer model on the Chinese LCSTS dataset using ROUGE-1 metric, and 4.48 points in the factual consistency assessment model StructBERT.

1. Introduction

Generative text summarization aims to create short, concise, and easy-to-read summaries by extracting key information from a text. This approach offers greater flexibility than extractive summarization, allowing models to generate new words with some probability. This capability brings summaries closer to those manually created, leading to a surge in research on generative models [1,2,3].

However, despite significant advancements achieved through neural networks, a critical challenge remains. Maynez et al. [4] demonstrated that sequence-to-sequence generative models often introduce “hallucinatory contents”—factual errors not present in the original text. Research by Falke et al. [5] further highlights this problem, finding that 25% of summaries from state-of-the-art models contain factual errors.

Table 1. Demonstration of erroneous sentences produced by different summarization models on the CNN-DM test set.

Source Documentation	Summarization	Model
$\left[\dots \right]$ jim jepps used a blog called the daily maybe to defend “rape fantasies”, describe paedophiles as “complex human beings” and question why teachers who have relationships with pupils are put on the sex offenders register.$\left[\dots \right]$	green party leader natalie bennett used a blog called the daily maybe to defend $\left[\dots \right]$	PGC [1]
(cnn) if newly revised nypd training materials are approved by a federal judge, new cadets could be taking courses reminding them “not to engage in racial profiling”. $\left[\dots \right]$	new nypd training materials are approved by a federal judge. $\left[\dots \right]$ [if missing]	FAS [6]
england’s first-choice right-back at the world cup looks set to leave liverpool after six years this summer. $\left[\dots \right]$	england’s premier league clubs set to leave liverpool after six years this summer. $\left[\dots \right]$	BUS [7]

showcases examples of these errors in three classical models [5]. As shown, the PGC model replaces the protagonist Jim Jeps with the unrelated Green Party leader Natalie Bainter, the FAS model transforms a viewpoint based on assumptions into definitive statements, and the BUS model incorrectly changes the protagonist of the document from a defender to a club. This issue plagues Chinese text summarization as well, significantly hindering the credibility and usability of generative text summarization models.

This paper proposes the Dependency-Augmented Summarization Model (DASUM) to address the issue of hallucination in Chinese text summarization, where summaries contain factual inconsistencies. DASUM leverages jieba (https://github.com/fxsjy/jieba, accessed on 30 June 2024) for initial segmentation of the source document. It then employs Baidu’s DDParser [8] to analyze the dependency relationships between the resulting tokens. Dependency analysis is crucial for capturing the relationships between words in Chinese text. This analysis results in a dependency graph where each edge represents a relationship between two words (head and dependent). DASUM then employs the Relational Graph Attention Network (RGAT) [9] within its encoder to encode the dependency graph extracted from source document to obtain a representation for each node in the graph. This representation captures both the node’s information and its connection to other words. This, in turn, helps the model capture the relationships between important words in the sentence, thereby reducing potential distortions during summarization. The node representations are then integrated into a standard Transformer-based [10] encoder–decoder architecture. During decoding, the decoder attends to each graph node within each Transformer block, allowing it to focus on relevant information throughout the summarization process. This approach improves the model’s understanding of factual relationships within the source text, leading to summaries with improved factual accuracy.

Experiments conducted on the benchmark LCSTS dataset [11] demonstrate that DASUM effectively improves both the quality and factual correctness of generated summaries. Evaluated using ROUGE [12] metrics, DASUM outperformed the baseline Transformer model, achieving a substantial 7.79-point increase in the ROUGE-1 score and an 8.73-point improvement in the ROUGE-L score on the LCSTS dataset. Additionally, StructBERT [13] evaluation revealed a notable 4.48-point improvement in factual correctness for DASUM-generated summaries compared to the baseline. Finally, these quantitative results are corroborated by human evaluation, confirming the model’s effectiveness.

In summary, the contribution of this paper is threefold:

• The Dependency-Augmented Summarization Model (DASUM) is proposed to tackle the hallucination problem in Chinese text summarization.
• A dependency graph is constructed based on the dependency relationships in the source document to extract the relationships between keywords in the article.
• The Relational Graph Attention Network (RGAT) is employed to encode the dependency graph, thereby integrating factual information from the source document into the summary generation model.

The rest of this paper is organized as follows: Related works are reviewed in Section 2. Section 3 briefly introduces the system model. Furthermore, our experimental and evaluation results are provided and analyzed in Section 4. Finally, Section 5 concludes the paper and outlines potential future directions.

2. Related Work

2.1. Generative Text Summaries

Generative text summarization has been extensively studied in recent years, and most generative summarization models use an encoder–decoder architecture (seq2seq) [14]. In this architecture, the source text is processed by an encoder to generate representations, which are then passed to a decoder that outputs the summary word by word. Researchers have proposed various techniques to improve upon this foundation. For example, see [1]; moreover, Gu [15] introduced the copy–generate mechanism, allowing the model to both generate new words and directly copy words from the source document. Gehrmann et al. [7] employed a masking technique to control which phrases from the source document that the model should include in the summary. Additionally, Paulus et al. [2] utilized a reinforcement learning approach to improve the quality of summarization. Dong et al. [3] introduced UNILM, a pre-trained model capable of handling both natural language understanding and generative tasks, which achieves state-of-the-art results in various tasks including generative summarization.

2.2. Fact-Aware Text Summaries

Recent years have seen a surge in research studies targeting the hallucination problem in generative text summarization models. A popular strategy focuses on architectural improvements to the widely used encoder–decoder models. Researchers have explored modifications to the encoder, decoder, and encoder–decoder structure to reduce this issue.

Among the encoder enhancement approaches, Zhu et al. [16] proposed using graph neural networks to encode factual relations extracted from source documents. Huang et al. [17] developed a reinforcement learning reward mechanism based on multiple-choice fill-in-the-blank questions to encourage the model to better understand the interactions between entities. In addition, Gunel et al. [18] used external knowledge from Wikipedia for knowledge embedding, which was shown to be useful in alleviating the hallucination problem.

In the decoder improvement approaches, Song et al. [19] proposed combining a sequential decoder with a tree-based decoder to generate text summaries. Aralikatte et al. [20] introduced the focal attention mechanism, which encourages the decoder to generate summaries that are similar or semantically related to the source document.

In approaches on encoder–decoder enhancements, Cao [21] extracted factual descriptions from the source text and applied a dual-attention framework to ensure summaries align with both source and the extracted factual information. Li [22] proposed an implication-aware architecture with multi-task learning that incorporates implication knowledge into the abstract summarization model.

Several research efforts have explored alternative training methods to mitigate the hallucination problem. Cao and Wang et al. [23] introduced a contrast learning method for training summarization models. This method utilizes positive training data consisting of reference summaries and negative data containing automatically generated hallucinations. A contrast learning system then distinguishes between these two types of summaries. Similarly, Tang et al. [24] proposed CONFIT, a contrast fine-tuning strategy that improves factual consistency and overall summary quality.

Some research works focus on post-editing techniques. Dong et al. [25] proposed SpanFact, a fact-correction model that leverages knowledge learned from QA models to rectify errors in summaries. Similarly, Cao [26] introduced a post-editing correction module to identify and correct hallucinatory content in generated summaries The correction module was trained on synthesizing data that were created by applying a series of heuristic transformations to reference summaries. Another research by Chen [27] introduced a contrast candidate generation and selection system for post-processing. This system first replaces named entities in the generated summaries with those from the source document, and then the system selects the best candidate as the final output summary.

3. System Models

Our model employs a Transformer-based architecture without pre-training. To leverage dependency relationships within the source document, we incorporate a Relational Graph Attention Network (R-GAT) [9] on top of the Transformer. Sentence embeddings are initially generated by the sentence encoder. Subsequently, R-GAT produces node embeddings for the dependency graph, capturing inter-word relationships. During decoding, the model attends to both sentence and node embeddings, enabling the generation of informative and factually accurate summaries. The model architecture is detailed in Figure 1.

3.1. Problem Formulation

Given a tokenized input sentence $X=\left(x_1,x_2,\cdots ,x_n\right)$, where n is the number of tokens, and a corresponding summary $Y=\left(y_1,y_2,\cdots ,y_m\right)$ with $m\le n$, we conduct dependency analysis to construct a dependency graph $G=\left(V,E\right)$. In this graph, V represents the set of nodes, E represents the set of edges, and each edge is defined as $\left(head,rel,tail\right)$, where $head\in V$, $tail\in V$, and $rel\in R$. $R=\{SBV,VOB,POB,\cdots \}$ denotes the set of dependency labels. Our objective is to generate the target summary Y given input sentence X and its dependency graph G. While the Transformer’s time complexity is $O\left(n^2\right)$, R-GAT’s complexity is $O\left(V{F}^2+EF\right)$ for a graph with V vertices, E edges, and feature dimension F. Despite this, parallel training maintains an overall $O\left(n^2\right)$ complexity. Our proposed method offers enhanced expressive power compared to a baseline Transformer model. Algorithm 1 shows the training process of DASUM.

Algorithm 1 Training Process of the Dependency-Augmented Summarization Model

Require : $D={\left\{(X^{\left(i\right)},Y^{\left(i\right)})\right\}}_{i=1}^K$ - training dataset. 1: Initialize network parameters $\theta$; 2: for each training epoch do 3:     for $i=1\mathrm{to}K$ do 4:        Dependency relation D = DDParser(X). 5:        $G\left(V,E\right)$ = graph_construct(D). 6:        $h_T=Transformer\left(X\right)$   and  $h_G=R\text{-}GAT\left(G\right)$ 7:        $h_{fin}=0.5\times h_T+0.5\times h_G$ 8:        $Y^{\prime }=decode\left(h_{fin}\right)$ 9:        compute and accumulate loss $\nabla loss\left(\theta \right)$. 10:   end for 11:   update network parameters $\theta \leftarrow \theta -\eta \nabla loss\left(\theta \right)$ 12: end for

3.2. Graph Encoder

We employed DDParser, a Chinese dependency analysis tool built on the PaddlePaddle platform, to extract dependencies between words. Directed graphs were then constructed from these dependencies (as shown in Figure 2). Subsequently, we obtained node representations using the Relational Graph Attention Neural Network (R-GAT).

R-GAT extends the Graph Attention Network (GAT) to incorporate edge labels. In this paper, a dependency tree is modeled as a graph G with n nodes representing words, and edges denoting dependency relations. GAT iteratively updates node representations by aggregating information from neighboring nodes using a multi-head attention mechanism:

$$h_{at{t}_i}^{l+1}={\left|\right|}_{k=1}^K\sum _{j\in N_i}{\alpha }_{ij}^{lk}{W}_k^l{h}_j^l$$

(1)

$${\alpha }_{ij}^{lk}=attention(i,j)$$

(2)

where $h_{at{t}_i}^{l+1}$ is the attention head of node i at layer $l+1$, and ${\left|\right|}_{k=1}^K{x}_i$ denotes the concatenation of vectors from $x_1$ to $x_K$. ${\alpha }_{ij}^{lk}$ represents the normalized attention coefficient computed by the $k_{th}$ attention head at layer l, and $W_k^l$ is the input transformation matrix. This paper employs dot-product attention for $attention(i,j)$.

While GAT aggregates information from neighboring nodes, it overlooks dependency relationships, potentially missing crucial information. To solve this problem, we introduce R-GAT, which has relation-specific attention heads, allowing for differentiated information flow based on dependency types. Intuitively, nodes connected by different dependency relations should have varying influences. By capturing and leveraging these fine-grained dependency details, our model excels at comprehending and representing sentence structure. This improved understanding benefits subsequent natural language processing tasks. The overall architecture is depicted in Figure 3.

Specifically, we first map the dependencies to vector representations and then compute the relation header as

$$h_{re{l}_i}^{l+1}={\left|\right|}_{m=1}^M\sum _{j\in N_i}{\beta }_{ij}^{lm}{W}_m^l{h}_j^l$$

(3)

$$g_{ij}^{lm}=\sigma (relu(r_{ij}{W}_{m1}+b_{m1})W_{m2}+b_{m2})$$

(4)

$${\beta }_{ij}^{lm}={\displaystyle \frac{exp\left(g_{ij}^{lm}\right)}{{\sum }_{j=1}^{N_i}exp\left(g_{ij}^{lm}\right)}}$$

(5)

where $r_{ij}$ denotes the relational embedding between node i and node j. R-GAT contains K attention heads and M relational heads. The final representation of each node is computed by

$$x_i^{l+1}=h_{at{t}_i}^{l+1}\left|\right|h_{re{l}_i}^{l+1}$$

(6)

$$h_i^{l+1}=relu(W_{l+1}{x}_i^{l+1}+b_{l+1})$$

(7)

3.3. Sentence Encoder

The sentence encoder processes the input sentence to generate contextual representations, mirroring the standard Transformer architecture. It comprises multiple layers, each consisting of multi-head attention followed by Add & Norm, and a feed-forward layer followed by another Add & Norm layer. To account for word order, essential for language understanding, positional encoding (PE) is incorporated. PE vectors, identical in dimension to word embeddings, are added to the word embeddings. These PE vectors are computed using a predetermined formula:

$$P{E}_{pos,2i}=sin(pos/10,{000}^{2i/d_{model}})$$

(8)

$$P{E}_{pos,2i+1}=cos(pos/10,{000}^{2i/d_{model}})$$

(9)

where $pos$ denotes the position of the word in the sentence and $d_{model}$ denotes the dimension of the PE.

In the multi-head self-attention mechanism, attention scores are computed from query (Q), key (K), and value (V) matrices. To mitigate the impact of large inner products between Q and K, the result is scaled by the square root of the key dimension ($d_k$). Subsequently, a softmax function is applied to obtain attention weights, which are then used to compute a weighted sum of the values:

$$Attention(Q,K,V)=softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(10)

Following the multi-head attention mechanism, a feed-forward network is applied. Both components are wrapped in residual connections and Layer Normalization (Add & Norm). The encoder processes the input matrix X through multiple layers of this structure, generating the following output representation:

$$h_T=LayerNorm(O+FFN\left(O\right))$$

(11)

$$O=LayerNorm\left(X+\mathrm{Multi}\text{-}\mathrm{Head}\text{-}\mathrm{Attention}\left(X\right)\right)$$

(12)

A single encoder processes the input matrix $X\in R^{n\ast d}$ and outputs a matrix O of identical dimensions. To enhance representation learning, multiple encoder layers can be stacked sequentially. The output of each layer becomes the input to the subsequent one, with the final output derived from the last encoder.

3.4. Decoder

The decoder combines the outputs of the sentence encoder and graph encoder into a unified representation:

$$h_{fin}=0.5\ast h_G+0.5\ast h_T$$

(13)

Specifically, the decoder employs two multi-head attention layers. The first layer utilizes masked self-attention to prevent the model from attending to future tokens during training, akin to the standard Transformer. The second layer’s K and V matrices are computed from a concatenation of the sentence encoder’s and graph encoder’s outputs, while the Q matrix is derived from the previous decoder layer’s output. This allows the decoder to access information from both the sentence and dependency graph for each generated token. Finally, a softmax layer predicts the next word based on the decoder’s output.

$$p_{vocab}=softmax(h_{fin}{w}_g+b_g)$$

(14)

4. Experimentation and Evaluation

This paper compares our proposed model, DASUM, to a baseline Transformer and a GAT+Transformer model to evaluate the effectiveness of the R-GAT module. While R-GAT and GAT models excel at modeling graph-structured data, they are rarely applied to natural language generation tasks. Therefore, in this study, we did not conduct separate comparative experiments on text summarization using R-GAT or GAT. Additional text summarization models will be incorporated for a comprehensive evaluation.

4.1. Datasets

We train and evaluate our model on the LCSTS [11] dataset, a benchmark for Chinese text summarization. Characterized by short text length and noise, LCSTS comprises three main parts. The first part of LCSTS, which contains 2,400,591 summary-text pairs across domains such as politics, athletics, military, movies, games, etc., offering diverse summarization topics and styles, is used as the training set of this paper. The second part of 10,666 manually annotated pairs including scores ranging from 1 to 5, indicating the correlation between the texts and the corresponding summary, is used as the validation set.

The third and final part of the dataset is a test set comprising 1106 samples. This set was manually curated based on the relevance between short texts and abstracts. To ensure data quality, duplicates, obvious mismatches, and other inconsistencies were removed, resulting in a final test set of 1012 samples.

Table 2. LCSTS dataset details.

Dataset Name	Dataset Size
Train dataset	2,400,591
Validation dataset	Human Score 1	942
	Human Score 2	1039
	Human Score 3	2019
	Human Score 4	3128
	Human Score 5	3538
	Total Number of Pairs	10,666
Test dataset	1012

presents the names and sizes of the three subsets within the LCSTS dataset.

4.2. Model Hyperparameters

Our model leverages the Transformer architecture as its foundation, integrating an R-GAT network for enhanced performance (the complete source code for the experiment described in this paper can be found on the following website: https://gitee.com/liyandewangzhan/dasum, accessed on 20 July 2024). Based on previous findings [28,29] which suggest that smaller dimensions are effective for low-resource tasks like short-text summarization, we set the word embedding and hidden layer dimensions for both the sentence encoder and decoder to 512. Additionally, we limit the maximum sentence length to 512 tokens. To efficiently capture inter-sentence relationships, we adopt a 6-layer, 8-head multi-head attention mechanism as our base model. Future experiments will explore larger-scale attention mechanisms. To ensure consistency with the Transformer module, we employ the same hyperparameter configuration for the R-GAT network. This includes a 6-layer, 8-head self-attention mechanism with a word embedding dimension and hidden state size of 512. To promote the model’s generalization ability, we introduce a dropout rate of 0.1. We utilize the Adam optimizer with a learning rate of 1e-3 and momentum parameters of (0.9, 0.98). These hyperparameter configurations improve our model’s effectiveness in processing text tasks, leading to enhanced performance and efficiency.

4.3. Assessment Criteria

To evaluate the quality of generated summaries, we employed the standard ROUGE-1, ROUGE-2, and ROUGE-L metrics. These metrics correlate well with human judgments and assess the accuracy of words, bi-gram matching, and longest common subsequences, respectively.

Table 3. Rouge scores for models on the LCSTS test set.

MODEL	RG-1	RG-2	RG-L
RNN	21.50	8.90	18.60
Transformer	34.42	20.44	30.19
CopyNet	34.40	21.60	31.30
GAT+Transformer	35.88	19.25	31.48
Gu et al. [15]	35.00	22.30	32.00
Bart et al. [30]	37.35	23.69	32.96
Li et al. [31]	36.99	24.15	34.21
Actor-Critic	37.51	24.68	35.02
DASUM (our model)	42.21	25.38	38.92

The bold numbers in the table are used to highlight the experimental results with the best performance.

presents the ROUGE scores for all experiments.

Experimental results demonstrate that the Transformer baseline significantly outperformed the RNN model, achieving 12.92, 11.54, and 11.59 points higher F1 scores on ROUGE-1, ROUGE-2, and ROUGE-L, respectively, demonstrating the Transformer’s superiority. Our model further improved upon this foundation, surpassing the Transformer by 7.79, 4.70, and 8.73 points on ROUGE-1, ROUGE-2, and ROUGE-L F1 scores, respectively. These findings underscore the effectiveness of the DASUM model.

Additionally, we evaluated the three models (DASUM, GAT+Transformer and Transformer) on the NLPCC test dataset (NLPCC 2017 Shared Task Test Data (Task 3), available at: http://tcci.ccf.org.cn/conference/2017/taskdata.php, accessed on 20 July 2024), which contains significantly longer text compared to the LCSTS dataset. Due to resource limitations, we directly tested the models trained on LCSTS without retraining them on NLPCC. As shown in

Table 4. Rouge scores for models on the NLPCC test set.

MODEL	RG-1	RG-2	RG-L
Transformer	12.33	6.47	9.91
GAT+Transformer	12.62	4.87	9.53
DASUM	$\mathbf{15}.\mathbf{20}$	$\mathbf{7}.\mathbf{67}$	$\mathbf{12}.\mathbf{52}$

The bold numbers in the table are used to highlight the experimental results with the best performance.

, model performance on NLPCC is notably lower than on LCSTS. This is primarily attributed to the discrepancy in data distribution, with NLPCC predominantly featuring longer texts. Nevertheless, the DASUM model consistently outperforms both the GAT+Transformer and Transformer models on the NLPCC dataset.

Meanwhile, we employed the StructBERT Chinese natural language inference model and ChatGPT-3.5-turbo large language model to evaluate factual correctness of summary texts. StructBERT, trained on CMNLI and OCNLI datasets, determines the semantic relationship between the source document and the generated summary. Using the dialogue template presented in

Table 5. ChatGPT dialogue template.

Q: I will present the data in the form of a (Source document, gold summary, generated summary) triplet, based on which please give a factual agreement score for the generated summary and the source document, with the highest score being 5 and the lowest score being 0. Please give the score directly.

A: Give three principles that need to be evaluated

Q: ({{Source_Document}}, {{Gold_Summary}}, {{Generated_Summary}})

A: “score 1”

$\cdots \cdots$

, we instructed the ChatGPT-3.5-turbo model to provide a factual correctness score between 0 and 5 for each generated summary. By averaging these scores across all summaries in the test set via the ChatGPT API, we determined the model’s overall factual correctness.

A comparative analysis of DASUM, the baseline Transformer, and GAT+Transformer across these evaluation metrics (presented in

Table 6. Results of the factual consistency assessment.

MODEL	StructBERT Score	Chatgpt
Transformer	56.90	3.58
GAT+Transformer	57.32	3.59
DASUM (our model)	61.38	3.64

The bold numbers in the table are used to highlight the experimental results with the best performance.

) demonstrates the superior factual correctness of our proposed model. Furthermore, manual evaluations of randomly selected samples support these findings.

4.4. Manual Assessment

To further assess our model’s ability to enhance semantic relevance and mitigate content bias, we conducted a manual evaluation focusing on fidelity, informativeness, and fluency. Representative examples are presented in

Table 7. Examples of manual assessments.

Source Doc:	最近，一位房地产领域的专家语出惊人：“过去10年里，北京这样的大城市房价增速每年超过15%。按照 这个速度，25年后，北京每平方米房价超过80万元非常容易。你要做的是什么？买。”专家言之凿凿，着实吓人，仿佛不赶紧买房就错过了天上掉下的馅饼。
	Recently, a real estate expert made a startling statement: “In the past 10 years, housing prices in big cities like Beijing have increased by more than 15% annually. At this rate, in 25 years, it will be very easy for the price of housing in Beijing to exceed 800,000 yuan per square meter. What should you do? Buy now!” The expert’s words were so convincing and terrifying that it seemed like you would miss out on a windfall if you didn’t buy a house right away.
Transformer:	专家称过去10年北京房价涨了15%
	An expert claims that Beijing’s housing prices have surged by 15% over the past decade.
DASUM:	专家称北京每平方房价将超过80万元
	An expert predicts that housing prices in Beijing will exceed 800,000 yuan per square meter.
Gold:	房价会80万元一平方米吗
	Will housing prices reach 800,000 yuan per square meter?
Source Doc:	2014年，51信用卡管家跟宜信等P2P公司合作，推出线上信贷产品“瞬时贷”，其是一种纯在线操作的信贷 模式。51信用卡管家创始人孙海涛说，51目前每天放贷1000万，预计2015年，自营产品加上瞬时贷，放 贷额度将远超30亿。
	In 2014, 51 Credit Card partnered with P2P companies like CreditEase to launch an online credit product called “Instant Loan”, a purely online credit model. Sun Haitao, founder of 51 Credit Card, stated that 51 currently disburses 10 million yuan daily and expects the total lending amount to far exceed 3 billion yuan in 2015, including self-operated products and Instant Loan.
Transformer:	51信用卡管家推“瞬时贷”每天放贷30亿
	51 Credit Card’s “Instant Loan” disburses 3 billion yuan daily
DASUM:	51信用卡贷，2015年或将超30亿
	51 Credit Card’s lending volume is expected to exceed 3 billion yuan in 2015
Gold:	51信用卡管家，预计2015年放贷额度远超30亿
	51 Credit Card projects lending volume to far exceed 3 billion yuan in 2015

.

5. Conclusions and Future Work

This paper explores the utilization of dependency graphs to enhance Chinese text summarization models. We introduce DASUM, a novel model that effectively integrates original text with dependency information to guide summary generation. Experimental results on the LCSTS dataset demonstrate the model’s effectiveness in improving semantic relevance and overall summary quality. While dependency relations offer valuable insights, challenges such as parsing errors in complex sentences and limited handling of cross-sentence dependencies persist. Future work will attempt to address the aforementioned issues, including the following:

• Incorporating multiple dependency parsers to mitigate the impact of errors from individual parsers;
• Enhancing dependency graph construction by considering the unique syntactic patterns and idiomatic expressions of the Chinese language;
• Leveraging human-in-the-loop reinforcement learning to iteratively refine summary generation quality.

Furthermore, we aim to develop more comprehensive evaluation metrics to assess summary quality from multiple perspectives.