Entity relation extraction,a fundamental and essential task in natural language processing(NLP),has garnered significant attention over an extended period.,aiming to extract the core of semantic knowledge from unstruc...Entity relation extraction,a fundamental and essential task in natural language processing(NLP),has garnered significant attention over an extended period.,aiming to extract the core of semantic knowledge from unstructured text,i.e.,entities and the relations between them.At present,the main dilemma of Chinese entity relation extraction research lies in nested entities,relation overlap,and lack of entity relation interaction.This dilemma is particularly prominent in complex knowledge extraction tasks with high-density knowledge,imprecise syntactic structure,and lack of semantic roles.To address these challenges,this paper presents an innovative“character-level”Chinese part-of-speech(CN-POS)tagging approach and incorporates part-of-speech(POS)information into the pre-trained model,aiming to improve its semantic understanding and syntactic information processing capabilities.Additionally,A relation reference filling mechanism(RF)is proposed to enhance the semantic interaction between relations and entities,utilize relations to guide entity modeling,improve the boundary prediction ability of entity models for nested entity phenomena,and increase the cascading accuracy of entity-relation triples.Meanwhile,the“Queue”sub-task connection strategy is adopted to alleviate triplet cascading errors caused by overlapping relations,and a Syntax-enhanced entity relation extraction model(SE-RE)is constructed.The model showed excellent performance on the self-constructed E-commerce Product Information dataset(EPI)in this article.The results demonstrate that integrating POS enhancement into the pre-trained encoding model significantly boosts the performance of entity relation extraction models compared to baseline methods.Specifically,the F1-score fluctuation in subtasks caused by error accumulation was reduced by 3.21%,while the F1-score for entity-relation triplet extraction improved by 1.91%.展开更多
Given the scarcity of Satellite Frequency and Orbit(SFO)resources,it holds paramount importance to establish a comprehensive knowledge graph of SFO field(SFO-KG)and employ knowledge reasoning technology to automatical...Given the scarcity of Satellite Frequency and Orbit(SFO)resources,it holds paramount importance to establish a comprehensive knowledge graph of SFO field(SFO-KG)and employ knowledge reasoning technology to automatically mine available SFO resources.An essential aspect of constructing SFO-KG is the extraction of Chinese entity relations.Unfortunately,there is currently no publicly available Chinese SFO entity Relation Extraction(RE)dataset.Moreover,publicly available SFO text data contain numerous NA(representing for“No Answer”)relation category sentences that resemble other relation sentences and pose challenges in accurate classification,resulting in low recall and precision for the NA relation category in entity RE.Consequently,this issue adversely affects both the accuracy of constructing the knowledge graph and the efficiency of RE processes.To address these challenges,this paper proposes a method for extracting Chinese SFO text entity relations based on dynamic integrated learning.This method includes the construction of a manually annotated Chinese SFO entity RE dataset and a classifier combining features of SFO resource data.The proposed approach combines integrated learning and pre-training models,specifically utilizing Bidirectional Encoder Representation from Transformers(BERT).In addition,it incorporates one-class classification,attention mechanisms,and dynamic feedback mechanisms to improve the performance of the RE model.Experimental results show that the proposed method outperforms the traditional methods in terms of F1 value when extracting entity relations from both balanced and long-tailed datasets.展开更多
Relation extraction plays a crucial role in numerous downstream tasks.Dialogue relation extraction focuses on identifying relations between two arguments within a given dialogue.To tackle the problem of low informatio...Relation extraction plays a crucial role in numerous downstream tasks.Dialogue relation extraction focuses on identifying relations between two arguments within a given dialogue.To tackle the problem of low information density in dialogues,methods based on trigger enhancement have been proposed,yielding positive results.However,trigger enhancement faces challenges,which cause suboptimal model performance.First,the proportion of annotated triggers is low in DialogRE.Second,feature representations of triggers and arguments often contain conflicting information.In this paper,we propose a novel Multi-Feature Filtering and Fusion trigger enhancement approach to overcome these limitations.We first obtain representations of arguments,and triggers that contain rich semantic information through attention and gate methods.Then,we design a feature filtering mechanism that eliminates conflicting features in the encoding of trigger prototype representations and their corresponding argument pairs.Additionally,we utilize large language models to create prompts based on Chain-of-Thought and In-context Learning for automated trigger extraction.Experiments show that our model increases the average F1 score by 1.3%in the dialogue relation extraction task.Ablation and case studies confirm the effectiveness of our model.Furthermore,the feature filtering method effectively integrates with other trigger enhancement models,enhancing overall performance and demonstrating its ability to resolve feature conflicts.展开更多
The joint entity relation extraction model which integrates the semantic information of relation is favored by relevant researchers because of its effectiveness in solving the overlapping of entities,and the method of...The joint entity relation extraction model which integrates the semantic information of relation is favored by relevant researchers because of its effectiveness in solving the overlapping of entities,and the method of defining the semantic template of relation manually is particularly prominent in the extraction effect because it can obtain the deep semantic information of relation.However,this method has some problems,such as relying on expert experience and poor portability.Inspired by the rule-based entity relation extraction method,this paper proposes a joint entity relation extraction model based on a relation semantic template automatically constructed,which is abbreviated as RSTAC.This model refines the extraction rules of relation semantic templates from relation corpus through dependency parsing and realizes the automatic construction of relation semantic templates.Based on the relation semantic template,the process of relation classification and triplet extraction is constrained,and finally,the entity relation triplet is obtained.The experimental results on the three major Chinese datasets of DuIE,SanWen,and FinRE showthat the RSTAC model successfully obtains rich deep semantics of relation,improves the extraction effect of entity relation triples,and the F1 scores are increased by an average of 0.96% compared with classical joint extraction models such as CasRel,TPLinker,and RFBFN.展开更多
Deep neural network-based relational extraction research has made significant progress in recent years,andit provides data support for many natural language processing downstream tasks such as building knowledgegraph,...Deep neural network-based relational extraction research has made significant progress in recent years,andit provides data support for many natural language processing downstream tasks such as building knowledgegraph,sentiment analysis and question-answering systems.However,previous studies ignored much unusedstructural information in sentences that could enhance the performance of the relation extraction task.Moreover,most existing dependency-based models utilize self-attention to distinguish the importance of context,whichhardly deals withmultiple-structure information.To efficiently leverage multiple structure information,this paperproposes a dynamic structure attention mechanism model based on textual structure information,which deeplyintegrates word embedding,named entity recognition labels,part of speech,dependency tree and dependency typeinto a graph convolutional network.Specifically,our model extracts text features of different structures from theinput sentence.Textual Structure information Graph Convolutional Networks employs the dynamic structureattention mechanism to learn multi-structure attention,effectively distinguishing important contextual features invarious structural information.In addition,multi-structure weights are carefully designed as amergingmechanismin the different structure attention to dynamically adjust the final attention.This paper combines these featuresand trains a graph convolutional network for relation extraction.We experiment on supervised relation extractiondatasets including SemEval 2010 Task 8,TACRED,TACREV,and Re-TACED,the result significantly outperformsthe previous.展开更多
The relation is a semantic expression relevant to two named entities in a sentence.Since a sentence usually contains several named entities,it is essential to learn a structured sentence representation that encodes de...The relation is a semantic expression relevant to two named entities in a sentence.Since a sentence usually contains several named entities,it is essential to learn a structured sentence representation that encodes dependency information specific to the two named entities.In related work,graph convolutional neural networks are widely adopted to learn semantic dependencies,where a dependency tree initializes the adjacency matrix.However,this approach has two main issues.First,parsing a sentence heavily relies on external toolkits,which can be errorprone.Second,the dependency tree only encodes the syntactical structure of a sentence,which may not align with the relational semantic expression.In this paper,we propose an automatic graph learningmethod to autonomously learn a sentence’s structural information.Instead of using a fixed adjacency matrix initialized by a dependency tree,we introduce an Adaptive Adjacency Matrix to encode the semantic dependency between tokens.The elements of thismatrix are dynamically learned during the training process and optimized by task-relevant learning objectives,enabling the construction of task-relevant semantic dependencies within a sentence.Our model demonstrates superior performance on the TACRED and SemEval 2010 datasets,surpassing previous works by 1.3%and 0.8%,respectively.These experimental results show that our model excels in the relation extraction task,outperforming prior models.展开更多
Recently,many researchers have concentrated on using neural networks to learn features for Distant Supervised Relation Extraction(DSRE).These approaches generally use a softmax classifier with cross-entropy loss,which...Recently,many researchers have concentrated on using neural networks to learn features for Distant Supervised Relation Extraction(DSRE).These approaches generally use a softmax classifier with cross-entropy loss,which inevitably brings the noise of artificial class NA into classification process.To address the shortcoming,the classifier with ranking loss is employed to DSRE.Uniformly randomly selecting a relation or heuristically selecting the highest score among all incorrect relations are two common methods for generating a negative class in the ranking loss function.However,the majority of the generated negative class can be easily discriminated from positive class and will contribute little towards the training.Inspired by Generative Adversarial Networks(GANs),we use a neural network as the negative class generator to assist the training of our desired model,which acts as the discriminator in GANs.Through the alternating optimization of generator and discriminator,the generator is learning to produce more and more discriminable negative classes and the discriminator has to become better as well.This framework is independent of the concrete form of generator and discriminator.In this paper,we use a two layers fully-connected neural network as the generator and the Piecewise Convolutional Neural Networks(PCNNs)as the discriminator.Experiment results show that our proposed GAN-based method is effective and performs better than state-of-the-art methods.展开更多
The joint extraction of entities and their relations from certain texts plays a significant role in most natural language processes.For entity and relation extraction in a specific domain,we propose a hybrid neural fr...The joint extraction of entities and their relations from certain texts plays a significant role in most natural language processes.For entity and relation extraction in a specific domain,we propose a hybrid neural framework consisting of two parts:a span-based model and a graph-based model.The span-based model can tackle overlapping problems compared with BILOU methods,whereas the graph-based model treats relation prediction as graph classification.Our main contribution is to incorporate external lexical and syntactic knowledge of a specific domain,such as domain dictionaries and dependency structures from texts,into end-to-end neural models.We conducted extensive experiments on a Chinese military entity and relation extraction corpus.The results show that the proposed framework outperforms the baselines with better performance in terms of entity and relation prediction.The proposed method provides insight into problems with the joint extraction of entities and their relations.展开更多
An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during t...An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during the model training,which are essential but result in grossly imbalanced data distributions and in turn cause suboptimal model performance.In order to address the above issues,we propose a two-phase paradigm for the span-based joint entity and relation extraction,which involves classifying the entities and relations in the first phase,and predicting the types of these entities and relations in the second phase.The two-phase paradigm enables our model to significantly reduce the data distribution gap,including the gap between negative entities and other entities,aswell as the gap between negative relations and other relations.In addition,we make the first attempt at combining entity type and entity distance as global features,which has proven effective,especially for the relation extraction.Experimental results on several datasets demonstrate that the span-based joint extraction model augmented with the two-phase paradigm and the global features consistently outperforms previous state-ofthe-art span-based models for the joint extraction task,establishing a new standard benchmark.Qualitative and quantitative analyses further validate the effectiveness the proposed paradigm and the global features.展开更多
Log-linear models and more recently neural network models used forsupervised relation extraction requires substantial amounts of training data andtime, limiting the portability to new relations and domains. To this en...Log-linear models and more recently neural network models used forsupervised relation extraction requires substantial amounts of training data andtime, limiting the portability to new relations and domains. To this end, we propose a training representation based on the dependency paths between entities in adependency tree which we call lexicalized dependency paths (LDPs). We showthat this representation is fast, efficient and transparent. We further propose representations utilizing entity types and its subtypes to refine our model and alleviatethe data sparsity problem. We apply lexicalized dependency paths to supervisedlearning using the ACE corpus and show that it can achieve similar performancelevel to other state-of-the-art methods and even surpass them on severalcategories.展开更多
Cross-document relation extraction(RE),as an extension of information extraction,requires integrating information from multiple documents retrieved from open domains with a large number of irrelevant or confusing nois...Cross-document relation extraction(RE),as an extension of information extraction,requires integrating information from multiple documents retrieved from open domains with a large number of irrelevant or confusing noisy texts.Previous studies focus on the attention mechanism to construct the connection between different text features through semantic similarity.However,similarity-based methods cannot distinguish valid information from highly similar retrieved documents well.How to design an effective algorithm to implement aggregated reasoning in confusing information with similar features still remains an open issue.To address this problem,we design a novel local-toglobal causal reasoning(LGCR)network for cross-document RE,which enables efficient distinguishing,filtering and global reasoning on complex information from a causal perspective.Specifically,we propose a local causal estimation algorithm to estimate the causal effect,which is the first trial to use the causal reasoning independent of feature similarity to distinguish between confusing and valid information in cross-document RE.Furthermore,based on the causal effect,we propose a causality guided global reasoning algorithm to filter the confusing information and achieve global reasoning.Experimental results under the closed and the open settings of the large-scale dataset Cod RED demonstrate our LGCR network significantly outperforms the state-ofthe-art methods and validate the effectiveness of causal reasoning in confusing information processing.展开更多
Relation extraction is an important task in NLP community.However,some models often fail in capturing Long-distance dependence on semantics,and the interaction between semantics of two entities is ignored.In this pape...Relation extraction is an important task in NLP community.However,some models often fail in capturing Long-distance dependence on semantics,and the interaction between semantics of two entities is ignored.In this paper,we propose a novel neural network model for semantic relation classification called joint self-attention bi-LSTM(SA-Bi-LSTM)to model the internal structure of the sentence to obtain the importance of each word of the sentence without relying on additional information,and capture Long-distance dependence on semantics.We conduct experiments using the SemEval-2010 Task 8 dataset.Extensive experiments and the results demonstrated that the proposed method is effective against relation classification,which can obtain state-ofthe-art classification accuracy just with minimal feature engineering.展开更多
Spatial relation extraction is the process of identifying geographic entities from text and determining their corresponding spatial relations.Traditional spatial relation extraction mainly uses rule-based pattern matc...Spatial relation extraction is the process of identifying geographic entities from text and determining their corresponding spatial relations.Traditional spatial relation extraction mainly uses rule-based pattern matching,supervised learning-based or unsupervised learning-based methods.However,these methods suffer from poor time-sensitive,high labor cost and high dependence on large-scale data.With the development of pre-trained language models greatly alleviating the shortcomings of traditional methods,supervised learning methods incorporating pre-trained language models have become the mainstream relation extraction methods.Pipeline extraction and joint extraction,as the two most dominant ideas of relation extraction,both have obtained good performance on different datasets,and whether to share the contextual information of entities and relations is the main differences between the two ideas.In this paper,we compare the performance of two ideas oriented to spatial relation extraction based on Chinese corpus data in the field of geography and verify which method based on pre-trained language models is more suitable for Chinese spatial relation extraction.We fine-tuned the hyperparameters of the two models to optimize the extraction accuracy before the comparison experiments.The results of the comparison experiments show that pipeline extraction performs better than joint extraction of spatial relation extraction for Chinese text data with sentence granularity,because different tasks have different focus on contextual information,and it is difficult to take account into the needs of both tasks by sharing contextual information.In addition,we further compare the performance of the two models with the rule-based template approach in extracting topological,directional and distance relations,summarize the shortcomings of this experiment and provide an outlook for future work.展开更多
A qualia role-based entity-dependency graph(EDG)is proposed to represent and extract quantity relations for solving algebra story problems stated in Chinese.Traditional neural solvers use end-to-end models to translat...A qualia role-based entity-dependency graph(EDG)is proposed to represent and extract quantity relations for solving algebra story problems stated in Chinese.Traditional neural solvers use end-to-end models to translate problem texts into math expressions,which lack quantity relation acquisition in sophisticated scenarios.To address the problem,the proposed method leverages EDG to represent quantity relations hidden in qualia roles of math objects.Algorithms were designed for EDG generation and quantity relation extraction for solving algebra story problems.Experimental result shows that the proposedmethod achieved an average accuracy of 82.2%on quantity relation extraction compared to 74.5%of baseline method.Another prompt learning result shows a 5%increase obtained in problem solving by injecting the extracted quantity relations into the baseline neural solvers.展开更多
Spatial relations in text refer to how a geographical entity is located in space in relation to a reference entity.Extracting spatial relations from text is a fundamental task in natural language understanding.Previou...Spatial relations in text refer to how a geographical entity is located in space in relation to a reference entity.Extracting spatial relations from text is a fundamental task in natural language understanding.Previous studies have only focused on generic fine-tuning methods with additional classifiers,ignoring the importance of the semantic correlation between different spatial elements and the large offset between the relation extraction task and the pre-trained models.To address the above two issues,we propose a spatial relation extraction model based on Dual-view Prompt and Element Correlation(DPEC).Specifically,we first reformulate spatial relation extraction as a mask language model with a Dual-view Prompt(i.e.,Link Prompt and Confidence Prompt).Link Prompt can not only guide the model to incorporate more contextual information related to the spatial relation extraction task,but also better adapt to the original pre-training task of the language models.Meanwhile,Confidence Prompt can measure the confidence of candidate triplets in Link Prompt and work as a supplement to identify those easily confused examples in Link Prompt.Moreover,we incorporate the element correlation to measure the consistency between different spatial elements,which is an effective cue for identifying the rationality of spatial relations.Experimental results on the popular SpaceEval show that our DPEC significantly outperforms the SOTA baselines.展开更多
In recent years,with the rapid development of deep learning technology,relational triplet extraction techniques have also achieved groundbreaking progress.Traditional pipeline models have certain limitations due to er...In recent years,with the rapid development of deep learning technology,relational triplet extraction techniques have also achieved groundbreaking progress.Traditional pipeline models have certain limitations due to error propagation.To overcome the limitations of traditional pipeline models,recent research has focused on jointly modeling the two key subtasks-named entity recognition and relation extraction-within a unified framework.To support future research,this paper provides a comprehensive review of recently published studies in the field of relational triplet extraction.The review examines commonly used public datasets for relational triplet extraction techniques and systematically reviews current mainstream joint extraction methods,including joint decoding methods and parameter sharing methods,with joint decoding methods further divided into table filling,tagging,and sequence-to-sequence approaches.In addition,this paper also conducts small-scale replication experiments on models that have performed well in recent years for each method to verify the reproducibility of the code and to compare the performance of different models under uniform conditions.Each method has its own advantages in terms of model design,task handling,and application scenarios,but also faces challenges such as processing complex sentence structures,cross-sentence relation extraction,and adaptability in low-resource environments.Finally,this paper systematically summarizes each method and discusses the future development prospects of joint extraction of relational triples.展开更多
Relation extraction is a pivotal task within the field of natural language processing,boasting numerous realworld applications.Existing research predominantly centers on monolingual relation extraction or cross-lingua...Relation extraction is a pivotal task within the field of natural language processing,boasting numerous realworld applications.Existing research predominantly centers on monolingual relation extraction or cross-lingual enhancement for relation extraction.However,there exists a notable gap in understanding relation extraction within mix-lingual(or code-switching)scenarios.In these scenarios,individuals blend content from different languages within sentences,generating mix-lingual content.The effectiveness of existing relation extraction models in such scenarios remains largely unexplored due to the absence of dedicated datasets.To address this gap,we introduce the Mix-Lingual Relation Extraction(MixRE)task and construct a human-annotated dataset MixRED to support this task.Additionally,we propose a hierarchical training approach for the mix-lingual scenario named Mix-Lingual Training(MixTrain),designed to enhance the performance of large language models(LLMs)when capturing relational dependencies from mix-lingual content spanning different semantic levels.Our experiments involve evaluating state-of-the-art supervised models and LLMs on the constructed dataset,with results indicating that MixTrain notably improves model performance.Moreover,we investigate the effectiveness of using mix-lingual content as a tool to transfer learned relational dependencies across different languages.Additionally,we delve into factors influencing model performance for both supervised models and LLMs in the novel MixREtask.展开更多
As Satellite Frequency and Orbit(SFO)constitute scarce natural resources,constructing a Satellite Frequency and Orbit Knowledge Graph(SFO-KG)becomes crucial for optimizing their utilization.In the process of building ...As Satellite Frequency and Orbit(SFO)constitute scarce natural resources,constructing a Satellite Frequency and Orbit Knowledge Graph(SFO-KG)becomes crucial for optimizing their utilization.In the process of building the SFO-KG from Chinese unstructured data,extracting Chinese entity relations is the fundamental step.Although Relation Extraction(RE)methods in the English field have been extensively studied and developed earlier than their Chinese counterparts,their direct application to Chinese texts faces significant challenges due to linguistic distinctions such as unique grammar,pictographic characters,and prevalent polysemy.The absence of comprehensive reviews on Chinese RE research progress necessitates a systematic investigation.A thorough review of Chinese RE has been conducted from four methodological approaches:pipeline RE,joint entityrelation extraction,open domain RE,and multimodal RE techniques.In addition,we further analyze the essential research infrastructure,including specialized datasets,evaluation benchmarks,and competitions within Chinese RE research.Finally,the current research challenges and development trends in the field of Chinese RE were summarized and analyzed from the perspectives of ecological construction methods for datasets,open domain RE,N-ary RE,and RE based on large language models.This comprehensive review aims to facilitate SFO-KG construction and its practical applications in SFO resource management.展开更多
Knowledge Graphs(KGs)are pivotal for effectively organizing and managing structured information across various applications.Financial KGs have been successfully employed in advancing applications such as audit,anti-fr...Knowledge Graphs(KGs)are pivotal for effectively organizing and managing structured information across various applications.Financial KGs have been successfully employed in advancing applications such as audit,anti-fraud,and anti-money laundering.Despite their success,the construction of Chinese financial KGs has seen limited research due to the complex semantics.A significant challenge is the overlap triples problem,where entities feature in multiple relations within a sentence,hampering extraction accuracy-more than 39%of the triples in Chinese datasets exhibit the overlap triples.To address this,we propose the Entity-type-Enriched Cascaded Neural Network(E^(2)CNN),leveraging special tokens for entity boundaries and types.E^(2)CNN ensures consistency in entity types and excludes specific relations,mitigating overlap triple problems and enhancing relation extraction.Besides,we introduce the available Chinese financial dataset FINCORPUS.CN,annotated from annual reports of 2,000 companies,containing 48,389 entities and 23,368 triples.Experimental results on the DUIE dataset and FINCORPUS.CN underscore E^(2)CNN’s superiority over state-of-the-art models.展开更多
With the escalating complexity in production scenarios, vast amounts of production information are retained within enterprises in the industrial domain. Probing questions of how to meticulously excavate value from com...With the escalating complexity in production scenarios, vast amounts of production information are retained within enterprises in the industrial domain. Probing questions of how to meticulously excavate value from complex document information and establish coherent information links arise. In this work, we present a framework for knowledge graph construction in the industrial domain, predicated on knowledge-enhanced document-level entity and relation extraction. This approach alleviates the shortage of annotated data in the industrial domain and models the interplay of industrial documents. To augment the accuracy of named entity recognition, domain-specific knowledge is incorporated into the initialization of the word embedding matrix within the bidirectional long short-term memory conditional random field (BiLSTM-CRF) framework. For relation extraction, this paper introduces the knowledge-enhanced graph inference (KEGI) network, a pioneering method designed for long paragraphs in the industrial domain. This method discerns intricate interactions among entities by constructing a document graph and innovatively integrates knowledge representation into both node construction and path inference through TransR. On the application stratum, BiLSTM-CRF and KEGI are utilized to craft a knowledge graph from a knowledge representation model and Chinese fault reports for a steel production line, specifically SPOnto and SPFRDoc. The F1 value for entity and relation extraction has been enhanced by 2% to 6%. The quality of the extracted knowledge graph complies with the requirements of real-world production environment applications. The results demonstrate that KEGI can profoundly delve into production reports, extracting a wealth of knowledge and patterns, thereby providing a comprehensive solution for production management.展开更多
基金funded by the National Key Technology R&D Program of China under Grant No.2021YFD2100605the National Natural Science Foundation of China under Grant No.62433002+1 种基金the Project of Construction and Support for High-Level Innovative Teams of Beijing Municipal Institutions under Grant No.BPHR20220104Beijing Scholars Program under Grant No.099.
文摘Entity relation extraction,a fundamental and essential task in natural language processing(NLP),has garnered significant attention over an extended period.,aiming to extract the core of semantic knowledge from unstructured text,i.e.,entities and the relations between them.At present,the main dilemma of Chinese entity relation extraction research lies in nested entities,relation overlap,and lack of entity relation interaction.This dilemma is particularly prominent in complex knowledge extraction tasks with high-density knowledge,imprecise syntactic structure,and lack of semantic roles.To address these challenges,this paper presents an innovative“character-level”Chinese part-of-speech(CN-POS)tagging approach and incorporates part-of-speech(POS)information into the pre-trained model,aiming to improve its semantic understanding and syntactic information processing capabilities.Additionally,A relation reference filling mechanism(RF)is proposed to enhance the semantic interaction between relations and entities,utilize relations to guide entity modeling,improve the boundary prediction ability of entity models for nested entity phenomena,and increase the cascading accuracy of entity-relation triples.Meanwhile,the“Queue”sub-task connection strategy is adopted to alleviate triplet cascading errors caused by overlapping relations,and a Syntax-enhanced entity relation extraction model(SE-RE)is constructed.The model showed excellent performance on the self-constructed E-commerce Product Information dataset(EPI)in this article.The results demonstrate that integrating POS enhancement into the pre-trained encoding model significantly boosts the performance of entity relation extraction models compared to baseline methods.Specifically,the F1-score fluctuation in subtasks caused by error accumulation was reduced by 3.21%,while the F1-score for entity-relation triplet extraction improved by 1.91%.
文摘Given the scarcity of Satellite Frequency and Orbit(SFO)resources,it holds paramount importance to establish a comprehensive knowledge graph of SFO field(SFO-KG)and employ knowledge reasoning technology to automatically mine available SFO resources.An essential aspect of constructing SFO-KG is the extraction of Chinese entity relations.Unfortunately,there is currently no publicly available Chinese SFO entity Relation Extraction(RE)dataset.Moreover,publicly available SFO text data contain numerous NA(representing for“No Answer”)relation category sentences that resemble other relation sentences and pose challenges in accurate classification,resulting in low recall and precision for the NA relation category in entity RE.Consequently,this issue adversely affects both the accuracy of constructing the knowledge graph and the efficiency of RE processes.To address these challenges,this paper proposes a method for extracting Chinese SFO text entity relations based on dynamic integrated learning.This method includes the construction of a manually annotated Chinese SFO entity RE dataset and a classifier combining features of SFO resource data.The proposed approach combines integrated learning and pre-training models,specifically utilizing Bidirectional Encoder Representation from Transformers(BERT).In addition,it incorporates one-class classification,attention mechanisms,and dynamic feedback mechanisms to improve the performance of the RE model.Experimental results show that the proposed method outperforms the traditional methods in terms of F1 value when extracting entity relations from both balanced and long-tailed datasets.
基金supported by the National Key Research and Development Program of China(No.2023YFF0905400)the National Natural Science Foundation of China(No.U2341229).
文摘Relation extraction plays a crucial role in numerous downstream tasks.Dialogue relation extraction focuses on identifying relations between two arguments within a given dialogue.To tackle the problem of low information density in dialogues,methods based on trigger enhancement have been proposed,yielding positive results.However,trigger enhancement faces challenges,which cause suboptimal model performance.First,the proportion of annotated triggers is low in DialogRE.Second,feature representations of triggers and arguments often contain conflicting information.In this paper,we propose a novel Multi-Feature Filtering and Fusion trigger enhancement approach to overcome these limitations.We first obtain representations of arguments,and triggers that contain rich semantic information through attention and gate methods.Then,we design a feature filtering mechanism that eliminates conflicting features in the encoding of trigger prototype representations and their corresponding argument pairs.Additionally,we utilize large language models to create prompts based on Chain-of-Thought and In-context Learning for automated trigger extraction.Experiments show that our model increases the average F1 score by 1.3%in the dialogue relation extraction task.Ablation and case studies confirm the effectiveness of our model.Furthermore,the feature filtering method effectively integrates with other trigger enhancement models,enhancing overall performance and demonstrating its ability to resolve feature conflicts.
基金supported by the National Natural Science Foundation of China(Nos.U1804263,U1736214,62172435)the Zhongyuan Science and Technology Innovation Leading Talent Project(No.214200510019).
文摘The joint entity relation extraction model which integrates the semantic information of relation is favored by relevant researchers because of its effectiveness in solving the overlapping of entities,and the method of defining the semantic template of relation manually is particularly prominent in the extraction effect because it can obtain the deep semantic information of relation.However,this method has some problems,such as relying on expert experience and poor portability.Inspired by the rule-based entity relation extraction method,this paper proposes a joint entity relation extraction model based on a relation semantic template automatically constructed,which is abbreviated as RSTAC.This model refines the extraction rules of relation semantic templates from relation corpus through dependency parsing and realizes the automatic construction of relation semantic templates.Based on the relation semantic template,the process of relation classification and triplet extraction is constrained,and finally,the entity relation triplet is obtained.The experimental results on the three major Chinese datasets of DuIE,SanWen,and FinRE showthat the RSTAC model successfully obtains rich deep semantics of relation,improves the extraction effect of entity relation triples,and the F1 scores are increased by an average of 0.96% compared with classical joint extraction models such as CasRel,TPLinker,and RFBFN.
文摘Deep neural network-based relational extraction research has made significant progress in recent years,andit provides data support for many natural language processing downstream tasks such as building knowledgegraph,sentiment analysis and question-answering systems.However,previous studies ignored much unusedstructural information in sentences that could enhance the performance of the relation extraction task.Moreover,most existing dependency-based models utilize self-attention to distinguish the importance of context,whichhardly deals withmultiple-structure information.To efficiently leverage multiple structure information,this paperproposes a dynamic structure attention mechanism model based on textual structure information,which deeplyintegrates word embedding,named entity recognition labels,part of speech,dependency tree and dependency typeinto a graph convolutional network.Specifically,our model extracts text features of different structures from theinput sentence.Textual Structure information Graph Convolutional Networks employs the dynamic structureattention mechanism to learn multi-structure attention,effectively distinguishing important contextual features invarious structural information.In addition,multi-structure weights are carefully designed as amergingmechanismin the different structure attention to dynamically adjust the final attention.This paper combines these featuresand trains a graph convolutional network for relation extraction.We experiment on supervised relation extractiondatasets including SemEval 2010 Task 8,TACRED,TACREV,and Re-TACED,the result significantly outperformsthe previous.
基金supported by the Technology Projects of Guizhou Province under Grant[2024]003National Natural Science Foundation of China(GrantNos.62166007,62066008,62066007)Guizhou Provincial Science and Technology Projects under Grant No.ZK[2023]300.
文摘The relation is a semantic expression relevant to two named entities in a sentence.Since a sentence usually contains several named entities,it is essential to learn a structured sentence representation that encodes dependency information specific to the two named entities.In related work,graph convolutional neural networks are widely adopted to learn semantic dependencies,where a dependency tree initializes the adjacency matrix.However,this approach has two main issues.First,parsing a sentence heavily relies on external toolkits,which can be errorprone.Second,the dependency tree only encodes the syntactical structure of a sentence,which may not align with the relational semantic expression.In this paper,we propose an automatic graph learningmethod to autonomously learn a sentence’s structural information.Instead of using a fixed adjacency matrix initialized by a dependency tree,we introduce an Adaptive Adjacency Matrix to encode the semantic dependency between tokens.The elements of thismatrix are dynamically learned during the training process and optimized by task-relevant learning objectives,enabling the construction of task-relevant semantic dependencies within a sentence.Our model demonstrates superior performance on the TACRED and SemEval 2010 datasets,surpassing previous works by 1.3%and 0.8%,respectively.These experimental results show that our model excels in the relation extraction task,outperforming prior models.
基金This research work is supported by the National Natural Science Foundation of China(NO.61772454,6171101570,61602059)Hunan Provincial Natural Science Foundation of China(No.2017JJ3334)+1 种基金the Research Foundation of Education Bureau of Hunan Province,China(No.16C0045)the Open Project Program of the National Laboratory of Pattern Recognition(NLPR).Professor Jin Wang is the corresponding author.
文摘Recently,many researchers have concentrated on using neural networks to learn features for Distant Supervised Relation Extraction(DSRE).These approaches generally use a softmax classifier with cross-entropy loss,which inevitably brings the noise of artificial class NA into classification process.To address the shortcoming,the classifier with ranking loss is employed to DSRE.Uniformly randomly selecting a relation or heuristically selecting the highest score among all incorrect relations are two common methods for generating a negative class in the ranking loss function.However,the majority of the generated negative class can be easily discriminated from positive class and will contribute little towards the training.Inspired by Generative Adversarial Networks(GANs),we use a neural network as the negative class generator to assist the training of our desired model,which acts as the discriminator in GANs.Through the alternating optimization of generator and discriminator,the generator is learning to produce more and more discriminable negative classes and the discriminator has to become better as well.This framework is independent of the concrete form of generator and discriminator.In this paper,we use a two layers fully-connected neural network as the generator and the Piecewise Convolutional Neural Networks(PCNNs)as the discriminator.Experiment results show that our proposed GAN-based method is effective and performs better than state-of-the-art methods.
基金supported by the Jiangsu Province“333”project BRA2020418the NSFC under Grant Number 71901215+2 种基金the National University of Defense Technology Research Project ZK20-46the Outstanding Young Talents Program of National University of Defense Technologythe National University of Defense Technology Youth Innovation Project。
文摘The joint extraction of entities and their relations from certain texts plays a significant role in most natural language processes.For entity and relation extraction in a specific domain,we propose a hybrid neural framework consisting of two parts:a span-based model and a graph-based model.The span-based model can tackle overlapping problems compared with BILOU methods,whereas the graph-based model treats relation prediction as graph classification.Our main contribution is to incorporate external lexical and syntactic knowledge of a specific domain,such as domain dictionaries and dependency structures from texts,into end-to-end neural models.We conducted extensive experiments on a Chinese military entity and relation extraction corpus.The results show that the proposed framework outperforms the baselines with better performance in terms of entity and relation prediction.The proposed method provides insight into problems with the joint extraction of entities and their relations.
基金supported by the National Key Research and Development Program[2020YFB1006302].
文摘An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during the model training,which are essential but result in grossly imbalanced data distributions and in turn cause suboptimal model performance.In order to address the above issues,we propose a two-phase paradigm for the span-based joint entity and relation extraction,which involves classifying the entities and relations in the first phase,and predicting the types of these entities and relations in the second phase.The two-phase paradigm enables our model to significantly reduce the data distribution gap,including the gap between negative entities and other entities,aswell as the gap between negative relations and other relations.In addition,we make the first attempt at combining entity type and entity distance as global features,which has proven effective,especially for the relation extraction.Experimental results on several datasets demonstrate that the span-based joint extraction model augmented with the two-phase paradigm and the global features consistently outperforms previous state-ofthe-art span-based models for the joint extraction task,establishing a new standard benchmark.Qualitative and quantitative analyses further validate the effectiveness the proposed paradigm and the global features.
文摘Log-linear models and more recently neural network models used forsupervised relation extraction requires substantial amounts of training data andtime, limiting the portability to new relations and domains. To this end, we propose a training representation based on the dependency paths between entities in adependency tree which we call lexicalized dependency paths (LDPs). We showthat this representation is fast, efficient and transparent. We further propose representations utilizing entity types and its subtypes to refine our model and alleviatethe data sparsity problem. We apply lexicalized dependency paths to supervisedlearning using the ACE corpus and show that it can achieve similar performancelevel to other state-of-the-art methods and even surpass them on severalcategories.
基金supported in part by the National Key Research and Development Program of China(2022ZD0116405)the Strategic Priority Research Program of the Chinese Academy of Sciences(XDA27030300)the Key Research Program of the Chinese Academy of Sciences(ZDBS-SSW-JSC006)。
文摘Cross-document relation extraction(RE),as an extension of information extraction,requires integrating information from multiple documents retrieved from open domains with a large number of irrelevant or confusing noisy texts.Previous studies focus on the attention mechanism to construct the connection between different text features through semantic similarity.However,similarity-based methods cannot distinguish valid information from highly similar retrieved documents well.How to design an effective algorithm to implement aggregated reasoning in confusing information with similar features still remains an open issue.To address this problem,we design a novel local-toglobal causal reasoning(LGCR)network for cross-document RE,which enables efficient distinguishing,filtering and global reasoning on complex information from a causal perspective.Specifically,we propose a local causal estimation algorithm to estimate the causal effect,which is the first trial to use the causal reasoning independent of feature similarity to distinguish between confusing and valid information in cross-document RE.Furthermore,based on the causal effect,we propose a causality guided global reasoning algorithm to filter the confusing information and achieve global reasoning.Experimental results under the closed and the open settings of the large-scale dataset Cod RED demonstrate our LGCR network significantly outperforms the state-ofthe-art methods and validate the effectiveness of causal reasoning in confusing information processing.
文摘Relation extraction is an important task in NLP community.However,some models often fail in capturing Long-distance dependence on semantics,and the interaction between semantics of two entities is ignored.In this paper,we propose a novel neural network model for semantic relation classification called joint self-attention bi-LSTM(SA-Bi-LSTM)to model the internal structure of the sentence to obtain the importance of each word of the sentence without relying on additional information,and capture Long-distance dependence on semantics.We conduct experiments using the SemEval-2010 Task 8 dataset.Extensive experiments and the results demonstrated that the proposed method is effective against relation classification,which can obtain state-ofthe-art classification accuracy just with minimal feature engineering.
基金supported by the National Key Research and Development Program of China under[Grant number 2021YFB3900903]the National Natural Science Foundation of China under[Grant number 41971337].
文摘Spatial relation extraction is the process of identifying geographic entities from text and determining their corresponding spatial relations.Traditional spatial relation extraction mainly uses rule-based pattern matching,supervised learning-based or unsupervised learning-based methods.However,these methods suffer from poor time-sensitive,high labor cost and high dependence on large-scale data.With the development of pre-trained language models greatly alleviating the shortcomings of traditional methods,supervised learning methods incorporating pre-trained language models have become the mainstream relation extraction methods.Pipeline extraction and joint extraction,as the two most dominant ideas of relation extraction,both have obtained good performance on different datasets,and whether to share the contextual information of entities and relations is the main differences between the two ideas.In this paper,we compare the performance of two ideas oriented to spatial relation extraction based on Chinese corpus data in the field of geography and verify which method based on pre-trained language models is more suitable for Chinese spatial relation extraction.We fine-tuned the hyperparameters of the two models to optimize the extraction accuracy before the comparison experiments.The results of the comparison experiments show that pipeline extraction performs better than joint extraction of spatial relation extraction for Chinese text data with sentence granularity,because different tasks have different focus on contextual information,and it is difficult to take account into the needs of both tasks by sharing contextual information.In addition,we further compare the performance of the two models with the rule-based template approach in extracting topological,directional and distance relations,summarize the shortcomings of this experiment and provide an outlook for future work.
基金supported by the National Natural Science Foundation of China (Nos.62177024,62007014)the Humanities and Social Sciences Youth Fund of the Ministry of Education (No.20YJC880024)+1 种基金China Post Doctoral Science Foundation (No.2019M652678)the Fundamental Research Funds for the Central Universities (No.CCNU20ZT019).
文摘A qualia role-based entity-dependency graph(EDG)is proposed to represent and extract quantity relations for solving algebra story problems stated in Chinese.Traditional neural solvers use end-to-end models to translate problem texts into math expressions,which lack quantity relation acquisition in sophisticated scenarios.To address the problem,the proposed method leverages EDG to represent quantity relations hidden in qualia roles of math objects.Algorithms were designed for EDG generation and quantity relation extraction for solving algebra story problems.Experimental result shows that the proposedmethod achieved an average accuracy of 82.2%on quantity relation extraction compared to 74.5%of baseline method.Another prompt learning result shows a 5%increase obtained in problem solving by injecting the extracted quantity relations into the baseline neural solvers.
基金supported by the National Natural Science Foundation of China(Grant Nos.62276177 and 62376181).
文摘Spatial relations in text refer to how a geographical entity is located in space in relation to a reference entity.Extracting spatial relations from text is a fundamental task in natural language understanding.Previous studies have only focused on generic fine-tuning methods with additional classifiers,ignoring the importance of the semantic correlation between different spatial elements and the large offset between the relation extraction task and the pre-trained models.To address the above two issues,we propose a spatial relation extraction model based on Dual-view Prompt and Element Correlation(DPEC).Specifically,we first reformulate spatial relation extraction as a mask language model with a Dual-view Prompt(i.e.,Link Prompt and Confidence Prompt).Link Prompt can not only guide the model to incorporate more contextual information related to the spatial relation extraction task,but also better adapt to the original pre-training task of the language models.Meanwhile,Confidence Prompt can measure the confidence of candidate triplets in Link Prompt and work as a supplement to identify those easily confused examples in Link Prompt.Moreover,we incorporate the element correlation to measure the consistency between different spatial elements,which is an effective cue for identifying the rationality of spatial relations.Experimental results on the popular SpaceEval show that our DPEC significantly outperforms the SOTA baselines.
基金funding from Key Areas Science and Technology Research Plan of Xinjiang Production And Construction Corps Financial Science and Technology Plan Project under Grant Agreement No.2023AB048 for the project:Research and Application Demonstration of Data-driven Elderly Care System.
文摘In recent years,with the rapid development of deep learning technology,relational triplet extraction techniques have also achieved groundbreaking progress.Traditional pipeline models have certain limitations due to error propagation.To overcome the limitations of traditional pipeline models,recent research has focused on jointly modeling the two key subtasks-named entity recognition and relation extraction-within a unified framework.To support future research,this paper provides a comprehensive review of recently published studies in the field of relational triplet extraction.The review examines commonly used public datasets for relational triplet extraction techniques and systematically reviews current mainstream joint extraction methods,including joint decoding methods and parameter sharing methods,with joint decoding methods further divided into table filling,tagging,and sequence-to-sequence approaches.In addition,this paper also conducts small-scale replication experiments on models that have performed well in recent years for each method to verify the reproducibility of the code and to compare the performance of different models under uniform conditions.Each method has its own advantages in terms of model design,task handling,and application scenarios,but also faces challenges such as processing complex sentence structures,cross-sentence relation extraction,and adaptability in low-resource environments.Finally,this paper systematically summarizes each method and discusses the future development prospects of joint extraction of relational triples.
文摘Relation extraction is a pivotal task within the field of natural language processing,boasting numerous realworld applications.Existing research predominantly centers on monolingual relation extraction or cross-lingual enhancement for relation extraction.However,there exists a notable gap in understanding relation extraction within mix-lingual(or code-switching)scenarios.In these scenarios,individuals blend content from different languages within sentences,generating mix-lingual content.The effectiveness of existing relation extraction models in such scenarios remains largely unexplored due to the absence of dedicated datasets.To address this gap,we introduce the Mix-Lingual Relation Extraction(MixRE)task and construct a human-annotated dataset MixRED to support this task.Additionally,we propose a hierarchical training approach for the mix-lingual scenario named Mix-Lingual Training(MixTrain),designed to enhance the performance of large language models(LLMs)when capturing relational dependencies from mix-lingual content spanning different semantic levels.Our experiments involve evaluating state-of-the-art supervised models and LLMs on the constructed dataset,with results indicating that MixTrain notably improves model performance.Moreover,we investigate the effectiveness of using mix-lingual content as a tool to transfer learned relational dependencies across different languages.Additionally,we delve into factors influencing model performance for both supervised models and LLMs in the novel MixREtask.
文摘As Satellite Frequency and Orbit(SFO)constitute scarce natural resources,constructing a Satellite Frequency and Orbit Knowledge Graph(SFO-KG)becomes crucial for optimizing their utilization.In the process of building the SFO-KG from Chinese unstructured data,extracting Chinese entity relations is the fundamental step.Although Relation Extraction(RE)methods in the English field have been extensively studied and developed earlier than their Chinese counterparts,their direct application to Chinese texts faces significant challenges due to linguistic distinctions such as unique grammar,pictographic characters,and prevalent polysemy.The absence of comprehensive reviews on Chinese RE research progress necessitates a systematic investigation.A thorough review of Chinese RE has been conducted from four methodological approaches:pipeline RE,joint entityrelation extraction,open domain RE,and multimodal RE techniques.In addition,we further analyze the essential research infrastructure,including specialized datasets,evaluation benchmarks,and competitions within Chinese RE research.Finally,the current research challenges and development trends in the field of Chinese RE were summarized and analyzed from the perspectives of ecological construction methods for datasets,open domain RE,N-ary RE,and RE based on large language models.This comprehensive review aims to facilitate SFO-KG construction and its practical applications in SFO resource management.
基金supported in part by the National Key R&D Program of China(Grant No.2020AAA0108501).
文摘Knowledge Graphs(KGs)are pivotal for effectively organizing and managing structured information across various applications.Financial KGs have been successfully employed in advancing applications such as audit,anti-fraud,and anti-money laundering.Despite their success,the construction of Chinese financial KGs has seen limited research due to the complex semantics.A significant challenge is the overlap triples problem,where entities feature in multiple relations within a sentence,hampering extraction accuracy-more than 39%of the triples in Chinese datasets exhibit the overlap triples.To address this,we propose the Entity-type-Enriched Cascaded Neural Network(E^(2)CNN),leveraging special tokens for entity boundaries and types.E^(2)CNN ensures consistency in entity types and excludes specific relations,mitigating overlap triple problems and enhancing relation extraction.Besides,we introduce the available Chinese financial dataset FINCORPUS.CN,annotated from annual reports of 2,000 companies,containing 48,389 entities and 23,368 triples.Experimental results on the DUIE dataset and FINCORPUS.CN underscore E^(2)CNN’s superiority over state-of-the-art models.
基金supported by the National Science and Technology Innovation 2030 New Generation Artificial Intelligence Major Project(Grant No.2018AAA0101800)the National Natural Science Foundation of China(Grant No.72271188).
文摘With the escalating complexity in production scenarios, vast amounts of production information are retained within enterprises in the industrial domain. Probing questions of how to meticulously excavate value from complex document information and establish coherent information links arise. In this work, we present a framework for knowledge graph construction in the industrial domain, predicated on knowledge-enhanced document-level entity and relation extraction. This approach alleviates the shortage of annotated data in the industrial domain and models the interplay of industrial documents. To augment the accuracy of named entity recognition, domain-specific knowledge is incorporated into the initialization of the word embedding matrix within the bidirectional long short-term memory conditional random field (BiLSTM-CRF) framework. For relation extraction, this paper introduces the knowledge-enhanced graph inference (KEGI) network, a pioneering method designed for long paragraphs in the industrial domain. This method discerns intricate interactions among entities by constructing a document graph and innovatively integrates knowledge representation into both node construction and path inference through TransR. On the application stratum, BiLSTM-CRF and KEGI are utilized to craft a knowledge graph from a knowledge representation model and Chinese fault reports for a steel production line, specifically SPOnto and SPFRDoc. The F1 value for entity and relation extraction has been enhanced by 2% to 6%. The quality of the extracted knowledge graph complies with the requirements of real-world production environment applications. The results demonstrate that KEGI can profoundly delve into production reports, extracting a wealth of knowledge and patterns, thereby providing a comprehensive solution for production management.
