An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during t...An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during the model training,which are essential but result in grossly imbalanced data distributions and in turn cause suboptimal model performance.In order to address the above issues,we propose a two-phase paradigm for the span-based joint entity and relation extraction,which involves classifying the entities and relations in the first phase,and predicting the types of these entities and relations in the second phase.The two-phase paradigm enables our model to significantly reduce the data distribution gap,including the gap between negative entities and other entities,aswell as the gap between negative relations and other relations.In addition,we make the first attempt at combining entity type and entity distance as global features,which has proven effective,especially for the relation extraction.Experimental results on several datasets demonstrate that the span-based joint extraction model augmented with the two-phase paradigm and the global features consistently outperforms previous state-ofthe-art span-based models for the joint extraction task,establishing a new standard benchmark.Qualitative and quantitative analyses further validate the effectiveness the proposed paradigm and the global features.展开更多
With the rapid expansion of social media,analyzing emotions and their causes in texts has gained significant importance.Emotion-cause pair extraction enables the identification of causal relationships between emotions...With the rapid expansion of social media,analyzing emotions and their causes in texts has gained significant importance.Emotion-cause pair extraction enables the identification of causal relationships between emotions and their triggers within a text,facilitating a deeper understanding of expressed sentiments and their underlying reasons.This comprehension is crucial for making informed strategic decisions in various business and societal contexts.However,recent research approaches employing multi-task learning frameworks for modeling often face challenges such as the inability to simultaneouslymodel extracted features and their interactions,or inconsistencies in label prediction between emotion-cause pair extraction and independent assistant tasks like emotion and cause extraction.To address these issues,this study proposes an emotion-cause pair extraction methodology that incorporates joint feature encoding and task alignment mechanisms.The model consists of two primary components:First,joint feature encoding simultaneously generates features for emotion-cause pairs and clauses,enhancing feature interactions between emotion clauses,cause clauses,and emotion-cause pairs.Second,the task alignment technique is applied to reduce the labeling distance between emotion-cause pair extraction and the two assistant tasks,capturing deep semantic information interactions among tasks.The proposed method is evaluated on a Chinese benchmark corpus using 10-fold cross-validation,assessing key performance metrics such as precision,recall,and F1 score.Experimental results demonstrate that the model achieves an F1 score of 76.05%,surpassing the state-of-the-art by 1.03%.The proposed model exhibits significant improvements in emotion-cause pair extraction(ECPE)and cause extraction(CE)compared to existing methods,validating its effectiveness.This research introduces a novel approach based on joint feature encoding and task alignment mechanisms,contributing to advancements in emotion-cause pair extraction.However,the study’s limitation lies in the data sources,potentially restricting the generalizability of the findings.展开更多
In recent years,with the rapid development of deep learning technology,relational triplet extraction techniques have also achieved groundbreaking progress.Traditional pipeline models have certain limitations due to er...In recent years,with the rapid development of deep learning technology,relational triplet extraction techniques have also achieved groundbreaking progress.Traditional pipeline models have certain limitations due to error propagation.To overcome the limitations of traditional pipeline models,recent research has focused on jointly modeling the two key subtasks-named entity recognition and relation extraction-within a unified framework.To support future research,this paper provides a comprehensive review of recently published studies in the field of relational triplet extraction.The review examines commonly used public datasets for relational triplet extraction techniques and systematically reviews current mainstream joint extraction methods,including joint decoding methods and parameter sharing methods,with joint decoding methods further divided into table filling,tagging,and sequence-to-sequence approaches.In addition,this paper also conducts small-scale replication experiments on models that have performed well in recent years for each method to verify the reproducibility of the code and to compare the performance of different models under uniform conditions.Each method has its own advantages in terms of model design,task handling,and application scenarios,but also faces challenges such as processing complex sentence structures,cross-sentence relation extraction,and adaptability in low-resource environments.Finally,this paper systematically summarizes each method and discusses the future development prospects of joint extraction of relational triples.展开更多
Extracting valuable information frombiomedical texts is one of the current research hotspots of concern to a wide range of scholars.The biomedical corpus contains numerous complex long sentences and overlapping relati...Extracting valuable information frombiomedical texts is one of the current research hotspots of concern to a wide range of scholars.The biomedical corpus contains numerous complex long sentences and overlapping relational triples,making most generalized domain joint modeling methods difficult to apply effectively in this field.For a complex semantic environment in biomedical texts,in this paper,we propose a novel perspective to perform joint entity and relation extraction;existing studies divide the relation triples into several steps or modules.However,the three elements in the relation triples are interdependent and inseparable,so we regard joint extraction as a tripartite classification problem.At the same time,fromthe perspective of triple classification,we design amulti-granularity 2D convolution to refine the word pair table and better utilize the dependencies between biomedical word pairs.Finally,we use a biaffine predictor to assist in predicting the labels of word pairs for relation extraction.Our model(MCTPL)Multi-granularity Convolutional Tokens Pairs of Labeling better utilizes the elements of triples and improves the ability to extract overlapping triples compared to previous approaches.Finally,we evaluated our model on two publicly accessible datasets.The experimental results show that our model’s ability to extract relation triples on the CPI dataset improves the F1 score by 2.34%compared to the current optimal model.On the DDI dataset,the F1 value improves the F1 value by 1.68%compared to the current optimal model.Our model achieved state-of-the-art performance compared to other baseline models in biomedical text entity relation extraction.展开更多
Emotion cause extraction(ECE)task that aims at extracting potential trigger events of certain emotions has attracted extensive attention recently.However,current work neglects the implicit emotion expressed without an...Emotion cause extraction(ECE)task that aims at extracting potential trigger events of certain emotions has attracted extensive attention recently.However,current work neglects the implicit emotion expressed without any explicit emotional keywords,which appears more frequently in application scenarios.The lack of explicit emotion information makes it extremely hard to extract emotion causes only with the local context.Moreover,an entire event is usually across multiple clauses,while existing work merely extracts cause events at clause level and cannot effectively capture complete cause event information.To address these issues,the events are first redefined at the tuple level and a span-based tuple-level algorithm is proposed to extract events from different clauses.Based on it,a corpus for implicit emotion cause extraction that tries to extract causes of implicit emotions is constructed.The authors propose a knowledge-enriched jointlearning model of implicit emotion recognition and implicit emotion cause extraction tasks(KJ-IECE),which leverages commonsense knowledge from ConceptNet and NRC_VAD to better capture connections between emotion and corresponding cause events.Experiments on both implicit and explicit emotion cause extraction datasets demonstrate the effectiveness of the proposed model.展开更多
Edge computing,a novel paradigm for performing computations at the network edge,holds significant relevance in the healthcare domain for extracting medical knowledge from traditional Uygur medical texts.Medical knowle...Edge computing,a novel paradigm for performing computations at the network edge,holds significant relevance in the healthcare domain for extracting medical knowledge from traditional Uygur medical texts.Medical knowledge extraction methods based on edge computing deploy deep learning models on edge devices to achieve localized entity and relation extraction.This approach avoids transferring substantial sensitive data to cloud data centers,effectively safeguarding the privacy of healthcare services.However,existing relation extraction methods mainly employ a sequential pipeline approach,which classifies relations between determined entities after entity recognition.This mode faces challenges such as error propagation between tasks,insufficient consideration of dependencies between the two subtasks,and the neglect of interrelations between different relations within a sentence.To address these challenges,a joint extraction model with parameter sharing in edge computing is proposed,named CoEx-Bert.This model leverages shared parameterization between two models to jointly extract entities and relations.Specifically,CoEx-Bert employs two models,each separately sharing hidden layer parameters,and combines these two loss functions for joint backpropagation to optimize the model parameters.Additionally,it effectively resolves the issue of entity overlapping when extracting knowledge from unstructured Uygur medical texts by considering contextual relations.Finally,this model is deployed on edge devices for real-time extraction and inference of Uygur medical knowledge.Experimental results demonstrate that CoEx-Bert outperforms existing state-of-the-art methods,achieving accuracy,recall,and F1-score of 90.65%,92.45%,and 91.54%,respectively,in the Uygur traditional medical literature dataset.These improvements represent a 6.45%increase in accuracy,a 9.45%increase in recall,and a 7.95%increase in F1-score compared to the baseline.展开更多
Spatial relation extraction is the process of identifying geographic entities from text and determining their corresponding spatial relations.Traditional spatial relation extraction mainly uses rule-based pattern matc...Spatial relation extraction is the process of identifying geographic entities from text and determining their corresponding spatial relations.Traditional spatial relation extraction mainly uses rule-based pattern matching,supervised learning-based or unsupervised learning-based methods.However,these methods suffer from poor time-sensitive,high labor cost and high dependence on large-scale data.With the development of pre-trained language models greatly alleviating the shortcomings of traditional methods,supervised learning methods incorporating pre-trained language models have become the mainstream relation extraction methods.Pipeline extraction and joint extraction,as the two most dominant ideas of relation extraction,both have obtained good performance on different datasets,and whether to share the contextual information of entities and relations is the main differences between the two ideas.In this paper,we compare the performance of two ideas oriented to spatial relation extraction based on Chinese corpus data in the field of geography and verify which method based on pre-trained language models is more suitable for Chinese spatial relation extraction.We fine-tuned the hyperparameters of the two models to optimize the extraction accuracy before the comparison experiments.The results of the comparison experiments show that pipeline extraction performs better than joint extraction of spatial relation extraction for Chinese text data with sentence granularity,because different tasks have different focus on contextual information,and it is difficult to take account into the needs of both tasks by sharing contextual information.In addition,we further compare the performance of the two models with the rule-based template approach in extracting topological,directional and distance relations,summarize the shortcomings of this experiment and provide an outlook for future work.展开更多
设备运维是保障生产正常进行的重要基础,现有的智能运维技术主要依赖信号分析、数据挖掘或专家知识重用。然而,随着设备自动化和集成化程度的提高,其各类运行异常的表征信号、多源致因和维护方案之间的关系呈现出更高的模糊性和复杂性,...设备运维是保障生产正常进行的重要基础,现有的智能运维技术主要依赖信号分析、数据挖掘或专家知识重用。然而,随着设备自动化和集成化程度的提高,其各类运行异常的表征信号、多源致因和维护方案之间的关系呈现出更高的模糊性和复杂性,将信号、数据和知识进行融合分析是提高设备运维精度和效率的关键。为此,采用知识图谱技术将“人”、“机”、“物”三元数据融合来支撑复杂设备的异常诊断和维护方案决策,提高运维智能化程度、避免决策片面性。首先,对设备运维领域人机物三元数据进行定义并完成三元本体设计,指导知识图数据层的构建。其次,对人机物三元数据进行预处理并搭建了统一混合注意力机制联合抽取模型(Joint entity and relation extraction model with mixed attention,MAREL)从三元数据中自动抽取知识,并建立三元知识之间的关联关系,以此实现人机物三元数据的融合;MAREL模型将任务拆解为两个关联的解码模块来解决实体重叠问题,利用混合注意力机制增强模型的长文本处理能力,在中文数据集SKE上的测试证明MAREL具有优异的性能。最后,以某汽车生产机器人设备运维人机物知识图谱的构建为例,验证了所提方法的有效性,结果表明知识图谱能够将人机物三元数据有效融合,为工业设备的智能运维提供支持。展开更多
针对目前方法大多未能充分利用跨度语义信息和局部上下文隐含信息等问题,提出基于跨度和多层次特征融合的实体关系联合抽取模型。该模型首先将文本输入到预训练语言模型(Bidirectional Encoder Representations from Transformer,BERT)...针对目前方法大多未能充分利用跨度语义信息和局部上下文隐含信息等问题,提出基于跨度和多层次特征融合的实体关系联合抽取模型。该模型首先将文本输入到预训练语言模型(Bidirectional Encoder Representations from Transformer,BERT)转换为词向量后,将其与通过图卷积获得的句法依赖信息进行融合,形成更丰富的文本特征;然后通过多头注意力层对文本特征进行加权处理,以此抑制噪声特征的干扰,并促进特征之间的交互,随后根据跨度将文本信息分割成跨度序列进行实体识别;最后使用双向门控循环单元提取局部上下文隐含信息,将与实体类型信息融合到候选实体跨度对并使用sigmoid函数进行关系分类。实验表明,该模型在SciERC数据集和CoNLL04数据集上取得良好的提升效果。展开更多
基金supported by the National Key Research and Development Program[2020YFB1006302].
文摘An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during the model training,which are essential but result in grossly imbalanced data distributions and in turn cause suboptimal model performance.In order to address the above issues,we propose a two-phase paradigm for the span-based joint entity and relation extraction,which involves classifying the entities and relations in the first phase,and predicting the types of these entities and relations in the second phase.The two-phase paradigm enables our model to significantly reduce the data distribution gap,including the gap between negative entities and other entities,aswell as the gap between negative relations and other relations.In addition,we make the first attempt at combining entity type and entity distance as global features,which has proven effective,especially for the relation extraction.Experimental results on several datasets demonstrate that the span-based joint extraction model augmented with the two-phase paradigm and the global features consistently outperforms previous state-ofthe-art span-based models for the joint extraction task,establishing a new standard benchmark.Qualitative and quantitative analyses further validate the effectiveness the proposed paradigm and the global features.
文摘With the rapid expansion of social media,analyzing emotions and their causes in texts has gained significant importance.Emotion-cause pair extraction enables the identification of causal relationships between emotions and their triggers within a text,facilitating a deeper understanding of expressed sentiments and their underlying reasons.This comprehension is crucial for making informed strategic decisions in various business and societal contexts.However,recent research approaches employing multi-task learning frameworks for modeling often face challenges such as the inability to simultaneouslymodel extracted features and their interactions,or inconsistencies in label prediction between emotion-cause pair extraction and independent assistant tasks like emotion and cause extraction.To address these issues,this study proposes an emotion-cause pair extraction methodology that incorporates joint feature encoding and task alignment mechanisms.The model consists of two primary components:First,joint feature encoding simultaneously generates features for emotion-cause pairs and clauses,enhancing feature interactions between emotion clauses,cause clauses,and emotion-cause pairs.Second,the task alignment technique is applied to reduce the labeling distance between emotion-cause pair extraction and the two assistant tasks,capturing deep semantic information interactions among tasks.The proposed method is evaluated on a Chinese benchmark corpus using 10-fold cross-validation,assessing key performance metrics such as precision,recall,and F1 score.Experimental results demonstrate that the model achieves an F1 score of 76.05%,surpassing the state-of-the-art by 1.03%.The proposed model exhibits significant improvements in emotion-cause pair extraction(ECPE)and cause extraction(CE)compared to existing methods,validating its effectiveness.This research introduces a novel approach based on joint feature encoding and task alignment mechanisms,contributing to advancements in emotion-cause pair extraction.However,the study’s limitation lies in the data sources,potentially restricting the generalizability of the findings.
基金funding from Key Areas Science and Technology Research Plan of Xinjiang Production And Construction Corps Financial Science and Technology Plan Project under Grant Agreement No.2023AB048 for the project:Research and Application Demonstration of Data-driven Elderly Care System.
文摘In recent years,with the rapid development of deep learning technology,relational triplet extraction techniques have also achieved groundbreaking progress.Traditional pipeline models have certain limitations due to error propagation.To overcome the limitations of traditional pipeline models,recent research has focused on jointly modeling the two key subtasks-named entity recognition and relation extraction-within a unified framework.To support future research,this paper provides a comprehensive review of recently published studies in the field of relational triplet extraction.The review examines commonly used public datasets for relational triplet extraction techniques and systematically reviews current mainstream joint extraction methods,including joint decoding methods and parameter sharing methods,with joint decoding methods further divided into table filling,tagging,and sequence-to-sequence approaches.In addition,this paper also conducts small-scale replication experiments on models that have performed well in recent years for each method to verify the reproducibility of the code and to compare the performance of different models under uniform conditions.Each method has its own advantages in terms of model design,task handling,and application scenarios,but also faces challenges such as processing complex sentence structures,cross-sentence relation extraction,and adaptability in low-resource environments.Finally,this paper systematically summarizes each method and discusses the future development prospects of joint extraction of relational triples.
基金supported by the National Natural Science Foundation of China(Nos.62002206 and 62202373)the open topic of the Green Development Big Data Decision-Making Key Laboratory(DM202003).
文摘Extracting valuable information frombiomedical texts is one of the current research hotspots of concern to a wide range of scholars.The biomedical corpus contains numerous complex long sentences and overlapping relational triples,making most generalized domain joint modeling methods difficult to apply effectively in this field.For a complex semantic environment in biomedical texts,in this paper,we propose a novel perspective to perform joint entity and relation extraction;existing studies divide the relation triples into several steps or modules.However,the three elements in the relation triples are interdependent and inseparable,so we regard joint extraction as a tripartite classification problem.At the same time,fromthe perspective of triple classification,we design amulti-granularity 2D convolution to refine the word pair table and better utilize the dependencies between biomedical word pairs.Finally,we use a biaffine predictor to assist in predicting the labels of word pairs for relation extraction.Our model(MCTPL)Multi-granularity Convolutional Tokens Pairs of Labeling better utilizes the elements of triples and improves the ability to extract overlapping triples compared to previous approaches.Finally,we evaluated our model on two publicly accessible datasets.The experimental results show that our model’s ability to extract relation triples on the CPI dataset improves the F1 score by 2.34%compared to the current optimal model.On the DDI dataset,the F1 value improves the F1 value by 1.68%compared to the current optimal model.Our model achieved state-of-the-art performance compared to other baseline models in biomedical text entity relation extraction.
基金National Natural Science Foundation of China,Grant/Award Numbers:61671064,61732005National Key Research&Development Program,Grant/Award Number:2018YFC0831700。
文摘Emotion cause extraction(ECE)task that aims at extracting potential trigger events of certain emotions has attracted extensive attention recently.However,current work neglects the implicit emotion expressed without any explicit emotional keywords,which appears more frequently in application scenarios.The lack of explicit emotion information makes it extremely hard to extract emotion causes only with the local context.Moreover,an entire event is usually across multiple clauses,while existing work merely extracts cause events at clause level and cannot effectively capture complete cause event information.To address these issues,the events are first redefined at the tuple level and a span-based tuple-level algorithm is proposed to extract events from different clauses.Based on it,a corpus for implicit emotion cause extraction that tries to extract causes of implicit emotions is constructed.The authors propose a knowledge-enriched jointlearning model of implicit emotion recognition and implicit emotion cause extraction tasks(KJ-IECE),which leverages commonsense knowledge from ConceptNet and NRC_VAD to better capture connections between emotion and corresponding cause events.Experiments on both implicit and explicit emotion cause extraction datasets demonstrate the effectiveness of the proposed model.
文摘Edge computing,a novel paradigm for performing computations at the network edge,holds significant relevance in the healthcare domain for extracting medical knowledge from traditional Uygur medical texts.Medical knowledge extraction methods based on edge computing deploy deep learning models on edge devices to achieve localized entity and relation extraction.This approach avoids transferring substantial sensitive data to cloud data centers,effectively safeguarding the privacy of healthcare services.However,existing relation extraction methods mainly employ a sequential pipeline approach,which classifies relations between determined entities after entity recognition.This mode faces challenges such as error propagation between tasks,insufficient consideration of dependencies between the two subtasks,and the neglect of interrelations between different relations within a sentence.To address these challenges,a joint extraction model with parameter sharing in edge computing is proposed,named CoEx-Bert.This model leverages shared parameterization between two models to jointly extract entities and relations.Specifically,CoEx-Bert employs two models,each separately sharing hidden layer parameters,and combines these two loss functions for joint backpropagation to optimize the model parameters.Additionally,it effectively resolves the issue of entity overlapping when extracting knowledge from unstructured Uygur medical texts by considering contextual relations.Finally,this model is deployed on edge devices for real-time extraction and inference of Uygur medical knowledge.Experimental results demonstrate that CoEx-Bert outperforms existing state-of-the-art methods,achieving accuracy,recall,and F1-score of 90.65%,92.45%,and 91.54%,respectively,in the Uygur traditional medical literature dataset.These improvements represent a 6.45%increase in accuracy,a 9.45%increase in recall,and a 7.95%increase in F1-score compared to the baseline.
基金supported by the National Key Research and Development Program of China under[Grant number 2021YFB3900903]the National Natural Science Foundation of China under[Grant number 41971337].
文摘Spatial relation extraction is the process of identifying geographic entities from text and determining their corresponding spatial relations.Traditional spatial relation extraction mainly uses rule-based pattern matching,supervised learning-based or unsupervised learning-based methods.However,these methods suffer from poor time-sensitive,high labor cost and high dependence on large-scale data.With the development of pre-trained language models greatly alleviating the shortcomings of traditional methods,supervised learning methods incorporating pre-trained language models have become the mainstream relation extraction methods.Pipeline extraction and joint extraction,as the two most dominant ideas of relation extraction,both have obtained good performance on different datasets,and whether to share the contextual information of entities and relations is the main differences between the two ideas.In this paper,we compare the performance of two ideas oriented to spatial relation extraction based on Chinese corpus data in the field of geography and verify which method based on pre-trained language models is more suitable for Chinese spatial relation extraction.We fine-tuned the hyperparameters of the two models to optimize the extraction accuracy before the comparison experiments.The results of the comparison experiments show that pipeline extraction performs better than joint extraction of spatial relation extraction for Chinese text data with sentence granularity,because different tasks have different focus on contextual information,and it is difficult to take account into the needs of both tasks by sharing contextual information.In addition,we further compare the performance of the two models with the rule-based template approach in extracting topological,directional and distance relations,summarize the shortcomings of this experiment and provide an outlook for future work.
文摘设备运维是保障生产正常进行的重要基础,现有的智能运维技术主要依赖信号分析、数据挖掘或专家知识重用。然而,随着设备自动化和集成化程度的提高,其各类运行异常的表征信号、多源致因和维护方案之间的关系呈现出更高的模糊性和复杂性,将信号、数据和知识进行融合分析是提高设备运维精度和效率的关键。为此,采用知识图谱技术将“人”、“机”、“物”三元数据融合来支撑复杂设备的异常诊断和维护方案决策,提高运维智能化程度、避免决策片面性。首先,对设备运维领域人机物三元数据进行定义并完成三元本体设计,指导知识图数据层的构建。其次,对人机物三元数据进行预处理并搭建了统一混合注意力机制联合抽取模型(Joint entity and relation extraction model with mixed attention,MAREL)从三元数据中自动抽取知识,并建立三元知识之间的关联关系,以此实现人机物三元数据的融合;MAREL模型将任务拆解为两个关联的解码模块来解决实体重叠问题,利用混合注意力机制增强模型的长文本处理能力,在中文数据集SKE上的测试证明MAREL具有优异的性能。最后,以某汽车生产机器人设备运维人机物知识图谱的构建为例,验证了所提方法的有效性,结果表明知识图谱能够将人机物三元数据有效融合,为工业设备的智能运维提供支持。
文摘针对目前方法大多未能充分利用跨度语义信息和局部上下文隐含信息等问题,提出基于跨度和多层次特征融合的实体关系联合抽取模型。该模型首先将文本输入到预训练语言模型(Bidirectional Encoder Representations from Transformer,BERT)转换为词向量后,将其与通过图卷积获得的句法依赖信息进行融合,形成更丰富的文本特征;然后通过多头注意力层对文本特征进行加权处理,以此抑制噪声特征的干扰,并促进特征之间的交互,随后根据跨度将文本信息分割成跨度序列进行实体识别;最后使用双向门控循环单元提取局部上下文隐含信息,将与实体类型信息融合到候选实体跨度对并使用sigmoid函数进行关系分类。实验表明,该模型在SciERC数据集和CoNLL04数据集上取得良好的提升效果。
