The rise of social media platforms has revolutionized communication, enabling the exchange of vast amounts of data through text, audio, images, and videos. These platforms have become critical for sharing opinions and...The rise of social media platforms has revolutionized communication, enabling the exchange of vast amounts of data through text, audio, images, and videos. These platforms have become critical for sharing opinions and insights, influencing daily habits, and driving business, political, and economic decisions. Text posts are particularly significant, and natural language processing (NLP) has emerged as a powerful tool for analyzing such data. While traditional NLP methods have been effective for structured media, social media content poses unique challenges due to its informal and diverse nature. This has spurred the development of new techniques tailored for processing and extracting insights from unstructured user-generated text. One key application of NLP is the summarization of user comments to manage overwhelming content volumes. Abstractive summarization has proven highly effective in generating concise, human-like summaries, offering clear overviews of key themes and sentiments. This enhances understanding and engagement while reducing cognitive effort for users. For businesses, summarization provides actionable insights into customer preferences and feedback, enabling faster trend analysis, improved responsiveness, and strategic adaptability. By distilling complex data into manageable insights, summarization plays a vital role in improving user experiences and empowering informed decision-making in a data-driven landscape. This paper proposes a new implementation framework by fine-tuning and parameterizing Transformer Large Language Models to manage and maintain linguistic and semantic components in abstractive summary generation. The system excels in transforming large volumes of data into meaningful summaries, as evidenced by its strong performance across metrics like fluency, consistency, readability, and semantic coherence.展开更多
A large variety of complaint reports reflect subjective information expressed by citizens.A key challenge of text summarization for complaint reports is to ensure the factual consistency of generated summary.Therefore...A large variety of complaint reports reflect subjective information expressed by citizens.A key challenge of text summarization for complaint reports is to ensure the factual consistency of generated summary.Therefore,in this paper,a simple and weakly supervised framework considering factual consistency is proposed to generate a summary of city-based complaint reports without pre-labeled sentences/words.Furthermore,it considers the importance of entity in complaint reports to ensure factual consistency of summary.Experimental results on the customer review datasets(Yelp and Amazon)and complaint report dataset(complaint reports of Shenyang in China)show that the proposed framework outperforms state-of-the-art approaches in ROUGE scores and human evaluation.It unveils the effectiveness of our approach to helping in dealing with complaint reports.展开更多
Text summarization aims to generate a concise version of the original text.The longer the summary text is,themore detailed it will be fromthe original text,and this depends on the intended use.Therefore,the problem of...Text summarization aims to generate a concise version of the original text.The longer the summary text is,themore detailed it will be fromthe original text,and this depends on the intended use.Therefore,the problem of generating summary texts with desired lengths is a vital task to put the research into practice.To solve this problem,in this paper,we propose a new method to integrate the desired length of the summarized text into the encoder-decoder model for the abstractive text summarization problem.This length parameter is integrated into the encoding phase at each self-attention step and the decoding process by preserving the remaining length for calculating headattention in the generation process and using it as length embeddings added to theword embeddings.We conducted experiments for the proposed model on the two data sets,Cable News Network(CNN)Daily and NEWSROOM,with different desired output lengths.The obtained results show the proposed model’s effectiveness compared with related studies.展开更多
In recent research,deep learning algorithms have presented effective representation learning models for natural languages.The deep learningbased models create better data representation than classical models.They are ...In recent research,deep learning algorithms have presented effective representation learning models for natural languages.The deep learningbased models create better data representation than classical models.They are capable of automated extraction of distributed representation of texts.In this research,we introduce a new tree Extractive text summarization that is characterized by fitting the text structure representation in knowledge base training module,and also addresses memory issues that were not addresses before.The proposed model employs a tree structured mechanism to generate the phrase and text embedding.The proposed architecture mimics the tree configuration of the text-texts and provide better feature representation.It also incorporates an attention mechanism that offers an additional information source to conduct better summary extraction.The novel model addresses text summarization as a classification process,where the model calculates the probabilities of phrase and text-summary association.The model classification is divided into multiple features recognition such as information entropy,significance,redundancy and position.The model was assessed on two datasets,on the Multi-Doc Composition Query(MCQ)and Dual Attention Composition dataset(DAC)dataset.The experimental results prove that our proposed model has better summarization precision vs.other models by a considerable margin.展开更多
With the continuous growth of online news articles,there arises the necessity for an efficient abstractive summarization technique for the problem of information overloading.Abstractive summarization is highly complex...With the continuous growth of online news articles,there arises the necessity for an efficient abstractive summarization technique for the problem of information overloading.Abstractive summarization is highly complex and requires a deeper understanding and proper reasoning to come up with its own summary outline.Abstractive summarization task is framed as seq2seq modeling.Existing seq2seq methods perform better on short sequences;however,for long sequences,the performance degrades due to high computation and hence a two-phase self-normalized deep neural document summarization model consisting of improvised extractive cosine normalization and seq2seq abstractive phases has been proposed in this paper.The novelty is to parallelize the sequence computation training by incorporating feed-forward,the self-normalized neural network in the Extractive phase using Intra Cosine Attention Similarity(Ext-ICAS)with sentence dependency position.Also,it does not require any normalization technique explicitly.Our proposed abstractive Bidirectional Long Short Term Memory(Bi-LSTM)encoder sequence model performs better than the Bidirectional Gated Recurrent Unit(Bi-GRU)encoder with minimum training loss and with fast convergence.The proposed model was evaluated on the Cable News Network(CNN)/Daily Mail dataset and an average rouge score of 0.435 was achieved also computational training in the extractive phase was reduced by 59%with an average number of similarity computations.展开更多
Nowadays,data is very rapidly increasing in every domain such as social media,news,education,banking,etc.Most of the data and information is in the form of text.Most of the text contains little invaluable information ...Nowadays,data is very rapidly increasing in every domain such as social media,news,education,banking,etc.Most of the data and information is in the form of text.Most of the text contains little invaluable information and knowledge with lots of unwanted contents.To fetch this valuable information out of the huge text document,we need summarizer which is capable to extract data automatically and at the same time capable to summarize the document,particularly textual text in novel document,without losing its any vital information.The summarization could be in the form of extractive and abstractive summarization.The extractive summarization includes picking sentences of high rank from the text constructed by using sentence and word features and then putting them together to produced summary.An abstractive summarization is based on understanding the key ideas in the given text and then expressing those ideas in pure natural language.The abstractive summarization is the latest problem area for NLP(natural language processing),ML(Machine Learning)and NN(Neural Network)In this paper,the foremost techniques for automatic text summarization processes are defined.The different existing methods have been reviewed.Their effectiveness and limitations are described.Further the novel approach based on Neural Network and LSTM has been discussed.In Machine Learning approach the architecture of the underlying concept is called Encoder-Decoder.展开更多
Text summarization is an important task in natural language processing and it has been applied in many applications.Recently,abstractive summarization has attracted many attentions.However,the traditional evaluation m...Text summarization is an important task in natural language processing and it has been applied in many applications.Recently,abstractive summarization has attracted many attentions.However,the traditional evaluation metrics that consider little semantic information,are unsuitable for evaluating the quality of deep learning based abstractive summarization models,since these models may generate new words that do not exist in the original text.Moreover,the out-of-vocabulary(OOV)problem that affects the evaluation results,has not been well solved yet.To address these issues,we propose a novel model called ENMS,to enhance existing N-gram based evaluation metrics with semantics.To be specific,we present two types of methods:N-gram based Semantic Matching(NSM for short),and N-gram based Semantic Similarity(NSS for short),to improve several widely-used evaluation metrics including ROUGE(Recall-Oriented Understudy for Gisting Evaluation),BLEU(Bilingual Evaluation Understudy),etc.NSM and NSS work in different ways.The former calculates the matching degree directly,while the latter mainly improves the similarity measurement.Moreover we propose an N-gram representation mechanism to explore the vector representation of N-grams(including skip-grams).It serves as the basis of our ENMS model,in which we exploit some simple but effective integration methods to solve the OOV problem efficiently.Experimental results over the TAC AESOP dataset show that the metrics improved by our methods are well correlated with human judgements and can be used to better evaluate abstractive summarization methods.展开更多
Automatically generating a brief summary for legal-related public opinion news(LPO-news,which contains legal words or phrases)plays an important role in rapid and effective public opinion disposal.For LPO-news,the cri...Automatically generating a brief summary for legal-related public opinion news(LPO-news,which contains legal words or phrases)plays an important role in rapid and effective public opinion disposal.For LPO-news,the critical case elements which are significant parts of the summary may be mentioned several times in the reader comments.Consequently,we investigate the task of comment-aware abstractive text summarization for LPO-news,which can generate salient summary by learning pivotal case elements from the reader comments.In this paper,we present a hierarchical comment-aware encoder(HCAE),which contains four components:1)a traditional sequenceto-sequence framework as our baseline;2)a selective denoising module to filter the noisy of comments and distinguish the case elements;3)a merge module by coupling the source article and comments to yield comment-aware context representation;4)a recoding module to capture the interaction among the source article words conditioned on the comments.Extensive experiments are conducted on a large dataset of legal public opinion news collected from micro-blog,and results show that the proposed model outperforms several existing state-of-the-art baseline models under the ROUGE metrics.展开更多
任务的目的是识别对话中的关键信息并生成一段简短的文本.由于对话具有非正式化和动态交互性质,导致对话文本信息稀疏、关键信息分散.然而,现有模型未能实现对对话中主题特征信息的有效挖掘,缺乏对核心话语的识别,忽略了附加特征融合过...任务的目的是识别对话中的关键信息并生成一段简短的文本.由于对话具有非正式化和动态交互性质,导致对话文本信息稀疏、关键信息分散.然而,现有模型未能实现对对话中主题特征信息的有效挖掘,缺乏对核心话语的识别,忽略了附加特征融合过程中的噪声问题.针对上述问题,本文提出一种结合主题挖掘与话语中心性的对话摘要模型DS-TMUC(Dialogue Summarization model combining Topic Mining and Utterance Centrality).首先,提出一种主题特征提取模块,该模块引入嵌入式主题模型来有效地挖掘对话中可解释的潜在主题信息,为抽象对话摘要过程提供更丰富的语义信息.其次,提出一种特征动态融合模块,设计特征感知网络为融合特征去除噪声以增强特征的表征能力,利用多头注意力捕捉特征之间的语义关联性,并且使用门控机制进行过滤融合,从而增强特征之间的有效融合.再次,提出一种话语赋权模块,设计无监督聚类方法计算话语中心性权重为话语赋权,通过引导模型选择核心话语,进而提高模型对对话上下文建模的有效性.在SAMSum和DialogSum数据集上的实验结果表明,DS-TMUC模型的总体性能优于对比模型.展开更多
模型的编码器输出中包含冗余信息,导致生成内容存在语义不相关和偏离主旨等问题,提出了一个结合关键词信息和门控单元的预训练文本摘要模型BGUK(BERT with Gated Unit and Keywords)。首先,该模型使用BERT对源文本进行编码,并引入了门...模型的编码器输出中包含冗余信息,导致生成内容存在语义不相关和偏离主旨等问题,提出了一个结合关键词信息和门控单元的预训练文本摘要模型BGUK(BERT with Gated Unit and Keywords)。首先,该模型使用BERT对源文本进行编码,并引入了门控单元进行语义提取和冗余信息的过滤。其次,将主题关键词信息合并到模型中解决生成摘要偏离主旨的问题。最后,加入覆盖率机制来减少生成摘要时出现的重复。实验结果表明BGUK生成了更符合主题的高质量的摘要,同时ROUGE得分也超过了基线模型。展开更多
目前,基于BERT预训练的文本摘要模型效果良好。然而,预训练模型内部使用的自注意力机制倾向于关注文本中字与字之间的相关信息,对词信息关注度较低,并且在解码时存在语义理解不充分的情况。针对上述问题,该文提出了一种基于BERT的语义...目前,基于BERT预训练的文本摘要模型效果良好。然而,预训练模型内部使用的自注意力机制倾向于关注文本中字与字之间的相关信息,对词信息关注度较低,并且在解码时存在语义理解不充分的情况。针对上述问题,该文提出了一种基于BERT的语义增强文本摘要模型CBSUM-Aux(Convolution and BERT Based Summarization Model with Auxiliary Information)。首先,使用窗口大小不同的卷积神经网络模块提取原文中的词特征信息,并与输入的字嵌入进行特征融合,之后通过预训练模型对融合特征进行深度特征挖掘。然后,在解码输出阶段,将卷积之后的词特征信息作为解码辅助信息输入解码器中指导模型解码。最后,针对束搜索算法倾向于输出短句的问题对其进行优化。该文使用LCSTS和CSTSD数据集对模型进行验证,实验结果表明,该文模型在ROUGE指标上有明显提升,生成的摘要与原文语义更加贴合。展开更多
文摘The rise of social media platforms has revolutionized communication, enabling the exchange of vast amounts of data through text, audio, images, and videos. These platforms have become critical for sharing opinions and insights, influencing daily habits, and driving business, political, and economic decisions. Text posts are particularly significant, and natural language processing (NLP) has emerged as a powerful tool for analyzing such data. While traditional NLP methods have been effective for structured media, social media content poses unique challenges due to its informal and diverse nature. This has spurred the development of new techniques tailored for processing and extracting insights from unstructured user-generated text. One key application of NLP is the summarization of user comments to manage overwhelming content volumes. Abstractive summarization has proven highly effective in generating concise, human-like summaries, offering clear overviews of key themes and sentiments. This enhances understanding and engagement while reducing cognitive effort for users. For businesses, summarization provides actionable insights into customer preferences and feedback, enabling faster trend analysis, improved responsiveness, and strategic adaptability. By distilling complex data into manageable insights, summarization plays a vital role in improving user experiences and empowering informed decision-making in a data-driven landscape. This paper proposes a new implementation framework by fine-tuning and parameterizing Transformer Large Language Models to manage and maintain linguistic and semantic components in abstractive summary generation. The system excels in transforming large volumes of data into meaningful summaries, as evidenced by its strong performance across metrics like fluency, consistency, readability, and semantic coherence.
基金supported by National Natural Science Foundation of China(62276058,61902057,41774063)Fundamental Research Funds for the Central Universities(N2217003)Joint Fund of Science&Technology Department of Liaoning Province and State Key Laboratory of Robotics,China(2020-KF-12-11).
文摘A large variety of complaint reports reflect subjective information expressed by citizens.A key challenge of text summarization for complaint reports is to ensure the factual consistency of generated summary.Therefore,in this paper,a simple and weakly supervised framework considering factual consistency is proposed to generate a summary of city-based complaint reports without pre-labeled sentences/words.Furthermore,it considers the importance of entity in complaint reports to ensure factual consistency of summary.Experimental results on the customer review datasets(Yelp and Amazon)and complaint report dataset(complaint reports of Shenyang in China)show that the proposed framework outperforms state-of-the-art approaches in ROUGE scores and human evaluation.It unveils the effectiveness of our approach to helping in dealing with complaint reports.
基金funded by Vietnam National Foundation for Science and Technology Development(NAFOSTED)under Grant Number 102.05-2020.26。
文摘Text summarization aims to generate a concise version of the original text.The longer the summary text is,themore detailed it will be fromthe original text,and this depends on the intended use.Therefore,the problem of generating summary texts with desired lengths is a vital task to put the research into practice.To solve this problem,in this paper,we propose a new method to integrate the desired length of the summarized text into the encoder-decoder model for the abstractive text summarization problem.This length parameter is integrated into the encoding phase at each self-attention step and the decoding process by preserving the remaining length for calculating headattention in the generation process and using it as length embeddings added to theword embeddings.We conducted experiments for the proposed model on the two data sets,Cable News Network(CNN)Daily and NEWSROOM,with different desired output lengths.The obtained results show the proposed model’s effectiveness compared with related studies.
基金This research was funded by Princess Nourah bint Abdulrahman University Researchers Supporting Project Number(PNURSP2022R113),Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘In recent research,deep learning algorithms have presented effective representation learning models for natural languages.The deep learningbased models create better data representation than classical models.They are capable of automated extraction of distributed representation of texts.In this research,we introduce a new tree Extractive text summarization that is characterized by fitting the text structure representation in knowledge base training module,and also addresses memory issues that were not addresses before.The proposed model employs a tree structured mechanism to generate the phrase and text embedding.The proposed architecture mimics the tree configuration of the text-texts and provide better feature representation.It also incorporates an attention mechanism that offers an additional information source to conduct better summary extraction.The novel model addresses text summarization as a classification process,where the model calculates the probabilities of phrase and text-summary association.The model classification is divided into multiple features recognition such as information entropy,significance,redundancy and position.The model was assessed on two datasets,on the Multi-Doc Composition Query(MCQ)and Dual Attention Composition dataset(DAC)dataset.The experimental results prove that our proposed model has better summarization precision vs.other models by a considerable margin.
文摘With the continuous growth of online news articles,there arises the necessity for an efficient abstractive summarization technique for the problem of information overloading.Abstractive summarization is highly complex and requires a deeper understanding and proper reasoning to come up with its own summary outline.Abstractive summarization task is framed as seq2seq modeling.Existing seq2seq methods perform better on short sequences;however,for long sequences,the performance degrades due to high computation and hence a two-phase self-normalized deep neural document summarization model consisting of improvised extractive cosine normalization and seq2seq abstractive phases has been proposed in this paper.The novelty is to parallelize the sequence computation training by incorporating feed-forward,the self-normalized neural network in the Extractive phase using Intra Cosine Attention Similarity(Ext-ICAS)with sentence dependency position.Also,it does not require any normalization technique explicitly.Our proposed abstractive Bidirectional Long Short Term Memory(Bi-LSTM)encoder sequence model performs better than the Bidirectional Gated Recurrent Unit(Bi-GRU)encoder with minimum training loss and with fast convergence.The proposed model was evaluated on the Cable News Network(CNN)/Daily Mail dataset and an average rouge score of 0.435 was achieved also computational training in the extractive phase was reduced by 59%with an average number of similarity computations.
文摘Nowadays,data is very rapidly increasing in every domain such as social media,news,education,banking,etc.Most of the data and information is in the form of text.Most of the text contains little invaluable information and knowledge with lots of unwanted contents.To fetch this valuable information out of the huge text document,we need summarizer which is capable to extract data automatically and at the same time capable to summarize the document,particularly textual text in novel document,without losing its any vital information.The summarization could be in the form of extractive and abstractive summarization.The extractive summarization includes picking sentences of high rank from the text constructed by using sentence and word features and then putting them together to produced summary.An abstractive summarization is based on understanding the key ideas in the given text and then expressing those ideas in pure natural language.The abstractive summarization is the latest problem area for NLP(natural language processing),ML(Machine Learning)and NN(Neural Network)In this paper,the foremost techniques for automatic text summarization processes are defined.The different existing methods have been reviewed.Their effectiveness and limitations are described.Further the novel approach based on Neural Network and LSTM has been discussed.In Machine Learning approach the architecture of the underlying concept is called Encoder-Decoder.
基金This work was supported by the National Natural Science Foundation of China under Grant Nos.62172149,61632009,62172159,and 62172372the Natural Science Foundation of Hunan Province of China under Grant No.2021JJ30137the Open Project of ZHEJIANG LAB under Grant No.2019KE0AB02.
文摘Text summarization is an important task in natural language processing and it has been applied in many applications.Recently,abstractive summarization has attracted many attentions.However,the traditional evaluation metrics that consider little semantic information,are unsuitable for evaluating the quality of deep learning based abstractive summarization models,since these models may generate new words that do not exist in the original text.Moreover,the out-of-vocabulary(OOV)problem that affects the evaluation results,has not been well solved yet.To address these issues,we propose a novel model called ENMS,to enhance existing N-gram based evaluation metrics with semantics.To be specific,we present two types of methods:N-gram based Semantic Matching(NSM for short),and N-gram based Semantic Similarity(NSS for short),to improve several widely-used evaluation metrics including ROUGE(Recall-Oriented Understudy for Gisting Evaluation),BLEU(Bilingual Evaluation Understudy),etc.NSM and NSS work in different ways.The former calculates the matching degree directly,while the latter mainly improves the similarity measurement.Moreover we propose an N-gram representation mechanism to explore the vector representation of N-grams(including skip-grams).It serves as the basis of our ENMS model,in which we exploit some simple but effective integration methods to solve the OOV problem efficiently.Experimental results over the TAC AESOP dataset show that the metrics improved by our methods are well correlated with human judgements and can be used to better evaluate abstractive summarization methods.
基金supported by the National Key Research and Development Program of China (2018YFC0830105,2018YFC 0830101,2018YFC0830100)the National Natural Science Foundation of China (Grant Nos.61972186,61762056,61472168)+1 种基金the Yunnan Provincial Major Science and Technology Special Plan Projects (202002AD080001)the General Projects of Basic Research in Yunnan Province (202001AT070046,202001AT070047).
文摘Automatically generating a brief summary for legal-related public opinion news(LPO-news,which contains legal words or phrases)plays an important role in rapid and effective public opinion disposal.For LPO-news,the critical case elements which are significant parts of the summary may be mentioned several times in the reader comments.Consequently,we investigate the task of comment-aware abstractive text summarization for LPO-news,which can generate salient summary by learning pivotal case elements from the reader comments.In this paper,we present a hierarchical comment-aware encoder(HCAE),which contains four components:1)a traditional sequenceto-sequence framework as our baseline;2)a selective denoising module to filter the noisy of comments and distinguish the case elements;3)a merge module by coupling the source article and comments to yield comment-aware context representation;4)a recoding module to capture the interaction among the source article words conditioned on the comments.Extensive experiments are conducted on a large dataset of legal public opinion news collected from micro-blog,and results show that the proposed model outperforms several existing state-of-the-art baseline models under the ROUGE metrics.
文摘任务的目的是识别对话中的关键信息并生成一段简短的文本.由于对话具有非正式化和动态交互性质,导致对话文本信息稀疏、关键信息分散.然而,现有模型未能实现对对话中主题特征信息的有效挖掘,缺乏对核心话语的识别,忽略了附加特征融合过程中的噪声问题.针对上述问题,本文提出一种结合主题挖掘与话语中心性的对话摘要模型DS-TMUC(Dialogue Summarization model combining Topic Mining and Utterance Centrality).首先,提出一种主题特征提取模块,该模块引入嵌入式主题模型来有效地挖掘对话中可解释的潜在主题信息,为抽象对话摘要过程提供更丰富的语义信息.其次,提出一种特征动态融合模块,设计特征感知网络为融合特征去除噪声以增强特征的表征能力,利用多头注意力捕捉特征之间的语义关联性,并且使用门控机制进行过滤融合,从而增强特征之间的有效融合.再次,提出一种话语赋权模块,设计无监督聚类方法计算话语中心性权重为话语赋权,通过引导模型选择核心话语,进而提高模型对对话上下文建模的有效性.在SAMSum和DialogSum数据集上的实验结果表明,DS-TMUC模型的总体性能优于对比模型.
文摘模型的编码器输出中包含冗余信息,导致生成内容存在语义不相关和偏离主旨等问题,提出了一个结合关键词信息和门控单元的预训练文本摘要模型BGUK(BERT with Gated Unit and Keywords)。首先,该模型使用BERT对源文本进行编码,并引入了门控单元进行语义提取和冗余信息的过滤。其次,将主题关键词信息合并到模型中解决生成摘要偏离主旨的问题。最后,加入覆盖率机制来减少生成摘要时出现的重复。实验结果表明BGUK生成了更符合主题的高质量的摘要,同时ROUGE得分也超过了基线模型。
文摘目前,基于BERT预训练的文本摘要模型效果良好。然而,预训练模型内部使用的自注意力机制倾向于关注文本中字与字之间的相关信息,对词信息关注度较低,并且在解码时存在语义理解不充分的情况。针对上述问题,该文提出了一种基于BERT的语义增强文本摘要模型CBSUM-Aux(Convolution and BERT Based Summarization Model with Auxiliary Information)。首先,使用窗口大小不同的卷积神经网络模块提取原文中的词特征信息,并与输入的字嵌入进行特征融合,之后通过预训练模型对融合特征进行深度特征挖掘。然后,在解码输出阶段,将卷积之后的词特征信息作为解码辅助信息输入解码器中指导模型解码。最后,针对束搜索算法倾向于输出短句的问题对其进行优化。该文使用LCSTS和CSTSD数据集对模型进行验证,实验结果表明,该文模型在ROUGE指标上有明显提升,生成的摘要与原文语义更加贴合。
