Since Google introduced the concept of Knowledge Graphs(KGs)in 2012,their construction technologies have evolved into a comprehensive methodological framework encompassing knowledge acquisition,extraction,representati...Since Google introduced the concept of Knowledge Graphs(KGs)in 2012,their construction technologies have evolved into a comprehensive methodological framework encompassing knowledge acquisition,extraction,representation,modeling,fusion,computation,and storage.Within this framework,knowledge extraction,as the core component,directly determines KG quality.In military domains,traditional manual curation models face efficiency constraints due to data fragmentation,complex knowledge architectures,and confidentiality protocols.Meanwhile,crowdsourced ontology construction approaches from general domains prove non-transferable,while human-crafted ontologies struggle with generalization deficiencies.To address these challenges,this study proposes an OntologyAware LLM Methodology for Military Domain Knowledge Extraction(LLM-KE).This approach leverages the deep semantic comprehension capabilities of Large Language Models(LLMs)to simulate human experts’cognitive processes in crowdsourced ontology construction,enabling automated extraction of military textual knowledge.It concurrently enhances knowledge processing efficiency and improves KG completeness.Empirical analysis demonstrates that this method effectively resolves scalability and dynamic adaptation challenges in military KG construction,establishing a novel technological pathway for advancing military intelligence development.展开更多
In the international shipping industry, digital intelligence transformation has become essential, with both governments and enterprises actively working to integrate diverse datasets. The domain of maritime and shippi...In the international shipping industry, digital intelligence transformation has become essential, with both governments and enterprises actively working to integrate diverse datasets. The domain of maritime and shipping is characterized by a vast array of document types, filled with complex, large-scale, and often chaotic knowledge and relationships. Effectively managing these documents is crucial for developing a Large Language Model (LLM) in the maritime domain, enabling practitioners to access and leverage valuable information. A Knowledge Graph (KG) offers a state-of-the-art solution for enhancing knowledge retrieval, providing more accurate responses and enabling context-aware reasoning. This paper presents a framework for utilizing maritime and shipping documents to construct a knowledge graph using GraphRAG, a hybrid tool combining graph-based retrieval and generation capabilities. The extraction of entities and relationships from these documents and the KG construction process are detailed. Furthermore, the KG is integrated with an LLM to develop a Q&A system, demonstrating that the system significantly improves answer accuracy compared to traditional LLMs. Additionally, the KG construction process is up to 50% faster than conventional LLM-based approaches, underscoring the efficiency of our method. This study provides a promising approach to digital intelligence in shipping, advancing knowledge accessibility and decision-making.展开更多
Entity relation extraction,a fundamental and essential task in natural language processing(NLP),has garnered significant attention over an extended period.,aiming to extract the core of semantic knowledge from unstruc...Entity relation extraction,a fundamental and essential task in natural language processing(NLP),has garnered significant attention over an extended period.,aiming to extract the core of semantic knowledge from unstructured text,i.e.,entities and the relations between them.At present,the main dilemma of Chinese entity relation extraction research lies in nested entities,relation overlap,and lack of entity relation interaction.This dilemma is particularly prominent in complex knowledge extraction tasks with high-density knowledge,imprecise syntactic structure,and lack of semantic roles.To address these challenges,this paper presents an innovative“character-level”Chinese part-of-speech(CN-POS)tagging approach and incorporates part-of-speech(POS)information into the pre-trained model,aiming to improve its semantic understanding and syntactic information processing capabilities.Additionally,A relation reference filling mechanism(RF)is proposed to enhance the semantic interaction between relations and entities,utilize relations to guide entity modeling,improve the boundary prediction ability of entity models for nested entity phenomena,and increase the cascading accuracy of entity-relation triples.Meanwhile,the“Queue”sub-task connection strategy is adopted to alleviate triplet cascading errors caused by overlapping relations,and a Syntax-enhanced entity relation extraction model(SE-RE)is constructed.The model showed excellent performance on the self-constructed E-commerce Product Information dataset(EPI)in this article.The results demonstrate that integrating POS enhancement into the pre-trained encoding model significantly boosts the performance of entity relation extraction models compared to baseline methods.Specifically,the F1-score fluctuation in subtasks caused by error accumulation was reduced by 3.21%,while the F1-score for entity-relation triplet extraction improved by 1.91%.展开更多
Objectives Medical knowledge extraction (MKE) plays a key role in natural language processing (NLP) research in electronic medical records (EMR),which are the important digital carriers for recording medical activitie...Objectives Medical knowledge extraction (MKE) plays a key role in natural language processing (NLP) research in electronic medical records (EMR),which are the important digital carriers for recording medical activities of patients.Named entity recognition (NER) and medical relation extraction (MRE) are two basic tasks of MKE.This study aims to improve the recognition accuracy of these two tasks by exploring deep learning methods.Methods This study discussed and built two application scenes of bidirectional long short-term memory combined conditional random field (BiLSTM-CRF) model for NER and MRE tasks.In the data preprocessing of both tasks,a GloVe word embedding model was used to vectorize words.In the NER task,a sequence labeling strategy was used to classify each word tag by the joint probability distribution through the CRF layer.In the MRE task,the medical entity relation category was predicted by transforming the classification problem of a single entity into a sequence classification problem and linking the feature combinations between entities also through the CRF layer.Results Through the validation on the I2B2 2010 public dataset,the BiLSTM-CRF models built in this study got much better results than the baseline methods in the two tasks,where the F1-measure was up to 0.88 in NER task and 0.78 in MRE task.Moreover,the model converged faster and avoided problems such as overfitting.Conclusion This study proved the good performance of deep learning on medical knowledge extraction.It also verified the feasibility of the BiLSTM-CRF model in different application scenarios,laying the foundation for the subsequent work in the EMR field.展开更多
Due to the complex structural hierarchy,with deeply nested associative relations between entities such as equipment,specifications,and business processes,intelligent power grid engineering is challenging.Meanwhile,lim...Due to the complex structural hierarchy,with deeply nested associative relations between entities such as equipment,specifications,and business processes,intelligent power grid engineering is challenging.Meanwhile,limited by the fragmented data and loss of contextual information,the generated reports are prone to the problems such as content redundancy and omission of critical information,failing to meet the demands of efficient decision-making and accurate management in modern power systems.To address these issues,this paper proposes a knowledge graph(KG)-enhanced framework to automatically generate electric power engineering reports.In the KG construction phase,a feature-fused entity recognition model named BERT-BiLSTM-CRF is adopted to improve the accuracy of entity recognition in scenarios involving power engineering professional terminology,thereby solving the problem of ambiguous entity boundaries in traditional models;then a BERT-attention relation extraction model is proposed to enhance the completeness of extracting complex hierarchical and implicit relations in power grid data.In the report generation phase,an improved Transformer architecture is adopted to accurately transform structured knowledge into natural language reports that comply with engineering specifications,addressing the issue of semantic inconsistency caused by the loss of structural information in existing models.By validating with real-world projects,the results show that the proposed framework significantly outperforms existing baseline models in entity recognition,confirming its superiority and applicability in practical engineering.展开更多
Entity recognition and extraction are the foundations of knowledge graph construction.Entity data in the field of software engineering come from different platforms and communities,and have different formats.This pape...Entity recognition and extraction are the foundations of knowledge graph construction.Entity data in the field of software engineering come from different platforms and communities,and have different formats.This paper divides multi-source software knowledge entities into unstructured data,semi-structured data and code data.For these different types of data,Bi-directional Long Short-Term Memory(Bi-LSTM)with Conditional Random Field(CRF),template matching,and abstract syntax tree are used and integrated into a multi-source software knowledge entity extraction integration model(MEIM)to extract software entities.The model can be updated continuously based on user’s feedbacks to improve the accuracy.To deal with the shortage of entity annotation datasets,keyword extraction methods based on Term Frequency–Inverse Document Frequency(TF-IDF),TextRank,and K-Means are applied to annotate tasks.The proposed MEIM model is applied to the Spring Boot framework,which demonstrates good adaptability.The extracted entities are used to construct a knowledge graph,which is applied to association retrieval and association visualization.展开更多
Hidden Web provides great amount of domain-specific data for constructing knowledge services. Most previous knowledge extraction researches ignore the valuable data hidden in Web database, and related works do not ref...Hidden Web provides great amount of domain-specific data for constructing knowledge services. Most previous knowledge extraction researches ignore the valuable data hidden in Web database, and related works do not refer how to make extracted information available for knowledge system. This paper describes a novel approach to build a domain-specific knowledge service with the data retrieved from Hidden Web. Ontology serves to model the domain knowledge. Queries forms of different Web sites are translated into machine-understandable format, defined knowledge concepts, so that they can be accessed automatically. Also knowledge data are extracted from Web pages and organized in ontology format knowledge. The experiment proves the algorithm achieves high accuracy and the system facilitates constructing knowledge services greatly.展开更多
Event extraction stands as a significant endeavor within the realm of information extraction,aspiring to automatically extract structured event information from vast volumes of unstructured text.Extracting event eleme...Event extraction stands as a significant endeavor within the realm of information extraction,aspiring to automatically extract structured event information from vast volumes of unstructured text.Extracting event elements from multi-modal data remains a challenging task due to the presence of a large number of images and overlapping event elements in the data.Although researchers have proposed various methods to accomplish this task,most existing event extraction models cannot address these challenges because they are only applicable to text scenarios.To solve the above issues,this paper proposes a multi-modal event extraction method based on knowledge fusion.Specifically,for event-type recognition,we use a meticulous pipeline approach that integrates multiple pre-trained models.This approach enables a more comprehensive capture of the multidimensional event semantic features present in military texts,thereby enhancing the interconnectedness of information between trigger words and events.For event element extraction,we propose a method for constructing a priori templates that combine event types with corresponding trigger words.This approach facilitates the acquisition of fine-grained input samples containing event trigger words,thus enabling the model to understand the semantic relationships between elements in greater depth.Furthermore,a fusion method for spatial mapping of textual event elements and image elements is proposed to reduce the category number overload and effectively achieve multi-modal knowledge fusion.The experimental results based on the CCKS 2022 dataset show that our method has achieved competitive results,with a comprehensive evaluation value F1-score of 53.4%for the model.These results validate the effectiveness of our method in extracting event elements from multi-modal data.展开更多
Emotion cause extraction(ECE)task that aims at extracting potential trigger events of certain emotions has attracted extensive attention recently.However,current work neglects the implicit emotion expressed without an...Emotion cause extraction(ECE)task that aims at extracting potential trigger events of certain emotions has attracted extensive attention recently.However,current work neglects the implicit emotion expressed without any explicit emotional keywords,which appears more frequently in application scenarios.The lack of explicit emotion information makes it extremely hard to extract emotion causes only with the local context.Moreover,an entire event is usually across multiple clauses,while existing work merely extracts cause events at clause level and cannot effectively capture complete cause event information.To address these issues,the events are first redefined at the tuple level and a span-based tuple-level algorithm is proposed to extract events from different clauses.Based on it,a corpus for implicit emotion cause extraction that tries to extract causes of implicit emotions is constructed.The authors propose a knowledge-enriched jointlearning model of implicit emotion recognition and implicit emotion cause extraction tasks(KJ-IECE),which leverages commonsense knowledge from ConceptNet and NRC_VAD to better capture connections between emotion and corresponding cause events.Experiments on both implicit and explicit emotion cause extraction datasets demonstrate the effectiveness of the proposed model.展开更多
To address the difficulty of training high-quality models in some specific domains due to the lack of fine-grained annotation resources, we propose in this paper a knowledge-integrated cross-domain data generation met...To address the difficulty of training high-quality models in some specific domains due to the lack of fine-grained annotation resources, we propose in this paper a knowledge-integrated cross-domain data generation method for unsupervised domain adaptation tasks. Specifically, we extract domain features, lexical and syntactic knowledge from source-domain and target-domain data, and use a masking model with an extended masking strategy and a re-masking strategy to obtain domain-specific data that remove domain-specific features. Finally, we improve the sequence generation model BART and use it to generate high-quality target domain data for the task of aspect and opinion co-extraction from the target domain. Experiments were performed on three conventional English datasets from different domains, and our method generates more accurate and diverse target domain data with the best results compared to previous methods.展开更多
Given the scarcity of Satellite Frequency and Orbit(SFO)resources,it holds paramount importance to establish a comprehensive knowledge graph of SFO field(SFO-KG)and employ knowledge reasoning technology to automatical...Given the scarcity of Satellite Frequency and Orbit(SFO)resources,it holds paramount importance to establish a comprehensive knowledge graph of SFO field(SFO-KG)and employ knowledge reasoning technology to automatically mine available SFO resources.An essential aspect of constructing SFO-KG is the extraction of Chinese entity relations.Unfortunately,there is currently no publicly available Chinese SFO entity Relation Extraction(RE)dataset.Moreover,publicly available SFO text data contain numerous NA(representing for“No Answer”)relation category sentences that resemble other relation sentences and pose challenges in accurate classification,resulting in low recall and precision for the NA relation category in entity RE.Consequently,this issue adversely affects both the accuracy of constructing the knowledge graph and the efficiency of RE processes.To address these challenges,this paper proposes a method for extracting Chinese SFO text entity relations based on dynamic integrated learning.This method includes the construction of a manually annotated Chinese SFO entity RE dataset and a classifier combining features of SFO resource data.The proposed approach combines integrated learning and pre-training models,specifically utilizing Bidirectional Encoder Representation from Transformers(BERT).In addition,it incorporates one-class classification,attention mechanisms,and dynamic feedback mechanisms to improve the performance of the RE model.Experimental results show that the proposed method outperforms the traditional methods in terms of F1 value when extracting entity relations from both balanced and long-tailed datasets.展开更多
The rapid development of electricity retail market has prompted an increasing number of electricity consumers to sign green electricity contracts with retail electricity companies,which poses greater challenges for th...The rapid development of electricity retail market has prompted an increasing number of electricity consumers to sign green electricity contracts with retail electricity companies,which poses greater challenges for the market service for green energy consumers.This study proposed a two-stage feature extraction approach for green energy consumers leveraging clustering and termfrequency-inverse document frequency(TF-IDF)algorithms within a knowledge graph framework to provide an information basis that supports the green development of the retail electricity market.First,the multi-source heterogeneous data of green energy consumers under an actual market environment is systematically introduced and the information is categorized into discrete,interval,and relational features.A clustering algorithm was employed to extract features of the trading behavior of green energy consumers in the first stage using the parameter data of green retail electricity contracts.Then,TF-IDF algorithm was applied in the second stage to extract features for green energy consumers in different clusters.Finally,the effectiveness of the proposed approach was validated based on the actual operational data in a southern province of China.It is shown that the most significant discrepancy between the retail trading behaviors of green energy consumers is the power share of green retail packages,whose averaged values are 25.64%,50%,39.66%,and 24.89%in four different clusters,respectively.Additionally,power supply bureaus and electricity retail companies affects the behavior of the green energy consumers most significantly.展开更多
The cost and strict input format requirements of GraphRAG make it less efficient for processing large documents. This paper proposes an alternative approach for constructing a knowledge graph (KG) from a PDF document ...The cost and strict input format requirements of GraphRAG make it less efficient for processing large documents. This paper proposes an alternative approach for constructing a knowledge graph (KG) from a PDF document with a focus on simplicity and cost-effectiveness. The process involves splitting the document into chunks, extracting concepts within each chunk using a large language model (LLM), and building relationships based on the proximity of concepts in the same chunk. Unlike traditional named entity recognition (NER), which identifies entities like “Shanghai”, the proposed method identifies concepts, such as “Convenient transportation in Shanghai” which is found to be more meaningful for KG construction. Each edge in the KG represents a relationship between concepts occurring in the same text chunk. The process is computationally inexpensive, leveraging locally set up tools like Mistral 7B openorca instruct and Ollama for model inference, ensuring the entire graph generation process is cost-free. A method of assigning weights to relationships, grouping similar pairs, and summarizing multiple relationships into a single edge with associated weight and relation details is introduced. Additionally, node degrees and communities are calculated for node sizing and coloring. This approach offers a scalable, cost-effective solution for generating meaningful knowledge graphs from large documents, achieving results comparable to GraphRAG while maintaining accessibility for personal machines.展开更多
The article is based on language model,through the cue word engineering and agent thinking method,automatic knowledge extraction,with China accounting standards support to complete the corresponding knowledge map cons...The article is based on language model,through the cue word engineering and agent thinking method,automatic knowledge extraction,with China accounting standards support to complete the corresponding knowledge map construction.Through the way of extracting the accounting entities and their connections in the pattern layer,the data layer is provided for the fine-tuning and optimization of the large model.Studies found that,through the reasonable application of language model,knowledge can be realized in massive financial data neural five effective extracted tuples,and complete accounting knowledge map construction.展开更多
With the popularization of social media,public opi-nion information on emergencies spreads rapidly on the Internet,the impact of negative public opinions on an event has become more significant.Based on the organizati...With the popularization of social media,public opi-nion information on emergencies spreads rapidly on the Internet,the impact of negative public opinions on an event has become more significant.Based on the organizational form of public opinion information,the knowledge graph is used to construct the knowledge base of public opinion risk cases on the emer-gency network.The emotion recognition model of negative pub-lic opinion information based on the bi-directional long short-term memory(BiLSTM)network is studied in the model layer design,and a linear discriminant analysis(LDA)topic extraction method combined with association rules is proposed to extract and mine the semantics of negative public opinion topics to real-ize further in-depth analysis of information topics.Focusing on public health emergencies,knowledge acquisition and knowl-edge processing of public opinion information are conducted,and the experimental results show that the knowledge graph framework based on the construction can facilitate in-depth theme evolution analysis of public opinion events,thus demon-strating important research significance for reducing online pub-lic opinion risks.展开更多
Psychological counseling Q&A system is enjoying a remarkable and increasing popularity in recent years. Knowledge base is the important component for such kind of systems, but it is difficult and time-consuming to...Psychological counseling Q&A system is enjoying a remarkable and increasing popularity in recent years. Knowledge base is the important component for such kind of systems, but it is difficult and time-consuming to construct the knowledge base manually. Fortunately, there emerges large number of Q&A pairs in many psychological counseling websites, which can provide good source enriching the knowledge base. This paper presents the method of knowledge extraction from psychological consulting Q&A pairs of on-line psychological counseling websites, which include keywords, semantic extension and word sequence. P-XML, which is the knowledge template based on XML, is also designed to store the knowledge. The extracted knowledge has been successfully used in our non-obstructive psychologycal counseling system, called P.A.L., and the experimental results also demonstrated the feasibility and effectiveness of our approach.展开更多
Geological knowledge can provide support for knowledge discovery, knowledge inference and mineralization predictions of geological big data. Entity identification and relationship extraction from geological data descr...Geological knowledge can provide support for knowledge discovery, knowledge inference and mineralization predictions of geological big data. Entity identification and relationship extraction from geological data description text are the key links for constructing knowledge graphs. Given the lack of publicly annotated datasets in the geology domain, this paper illustrates the construction process of geological entity datasets, defines the types of entities and interconceptual relationships by using the geological entity concept system, and completes the construction of the geological corpus. To address the shortcomings of existing language models(such as Word2vec and Glove) that cannot solve polysemous words and have a poor ability to fuse contexts, we propose a geological named entity recognition and relationship extraction model jointly with Bidirectional Encoder Representation from Transformers(BERT) pretrained language model. To effectively represent the text features, we construct a BERT-bidirectional gated recurrent unit network(BiGRU)-conditional random field(CRF)-based architecture to extract the named entities and the BERT-BiGRU-Attention-based architecture to extract the entity relations. The results show that the F1-score of the BERT-BiGRU-CRF named entity recognition model is 0.91 and the F1-score of the BERT-BiGRU-Attention relationship extraction model is 0.84, which are significant performance improvements when compared to classic language models(e.g., word2vec and Embedding from Language Models(ELMo)).展开更多
Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information ...Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.展开更多
The acquisition of valuable design knowledge from massive fragmentary data is challenging for designers in conceptual product design.This study proposes a novel method for acquiring design knowledge by combining deep ...The acquisition of valuable design knowledge from massive fragmentary data is challenging for designers in conceptual product design.This study proposes a novel method for acquiring design knowledge by combining deep learning with knowledge graph.Specifically,the design knowledge acquisition method utilises the knowledge extraction model to extract design-related entities and relations from fragmentary data,and further constructs the knowledge graph to support design knowledge acquisition for conceptual product design.Moreover,the knowledge extraction model introduces ALBERT to solve memory limitation and communication overhead in the entity extraction module,and uses multi-granularity information to overcome segmentation errors and polysemy ambiguity in the relation extraction module.Experimental comparison verified the effectiveness and accuracy of the proposed knowledge extraction model.The case study demonstrated the feasibility of the knowledge graph construction with real fragmentary porcelain data and showed the capability to provide designers with interconnected and visualised design knowledge.展开更多
文摘Since Google introduced the concept of Knowledge Graphs(KGs)in 2012,their construction technologies have evolved into a comprehensive methodological framework encompassing knowledge acquisition,extraction,representation,modeling,fusion,computation,and storage.Within this framework,knowledge extraction,as the core component,directly determines KG quality.In military domains,traditional manual curation models face efficiency constraints due to data fragmentation,complex knowledge architectures,and confidentiality protocols.Meanwhile,crowdsourced ontology construction approaches from general domains prove non-transferable,while human-crafted ontologies struggle with generalization deficiencies.To address these challenges,this study proposes an OntologyAware LLM Methodology for Military Domain Knowledge Extraction(LLM-KE).This approach leverages the deep semantic comprehension capabilities of Large Language Models(LLMs)to simulate human experts’cognitive processes in crowdsourced ontology construction,enabling automated extraction of military textual knowledge.It concurrently enhances knowledge processing efficiency and improves KG completeness.Empirical analysis demonstrates that this method effectively resolves scalability and dynamic adaptation challenges in military KG construction,establishing a novel technological pathway for advancing military intelligence development.
文摘In the international shipping industry, digital intelligence transformation has become essential, with both governments and enterprises actively working to integrate diverse datasets. The domain of maritime and shipping is characterized by a vast array of document types, filled with complex, large-scale, and often chaotic knowledge and relationships. Effectively managing these documents is crucial for developing a Large Language Model (LLM) in the maritime domain, enabling practitioners to access and leverage valuable information. A Knowledge Graph (KG) offers a state-of-the-art solution for enhancing knowledge retrieval, providing more accurate responses and enabling context-aware reasoning. This paper presents a framework for utilizing maritime and shipping documents to construct a knowledge graph using GraphRAG, a hybrid tool combining graph-based retrieval and generation capabilities. The extraction of entities and relationships from these documents and the KG construction process are detailed. Furthermore, the KG is integrated with an LLM to develop a Q&A system, demonstrating that the system significantly improves answer accuracy compared to traditional LLMs. Additionally, the KG construction process is up to 50% faster than conventional LLM-based approaches, underscoring the efficiency of our method. This study provides a promising approach to digital intelligence in shipping, advancing knowledge accessibility and decision-making.
基金funded by the National Key Technology R&D Program of China under Grant No.2021YFD2100605the National Natural Science Foundation of China under Grant No.62433002+1 种基金the Project of Construction and Support for High-Level Innovative Teams of Beijing Municipal Institutions under Grant No.BPHR20220104Beijing Scholars Program under Grant No.099.
文摘Entity relation extraction,a fundamental and essential task in natural language processing(NLP),has garnered significant attention over an extended period.,aiming to extract the core of semantic knowledge from unstructured text,i.e.,entities and the relations between them.At present,the main dilemma of Chinese entity relation extraction research lies in nested entities,relation overlap,and lack of entity relation interaction.This dilemma is particularly prominent in complex knowledge extraction tasks with high-density knowledge,imprecise syntactic structure,and lack of semantic roles.To address these challenges,this paper presents an innovative“character-level”Chinese part-of-speech(CN-POS)tagging approach and incorporates part-of-speech(POS)information into the pre-trained model,aiming to improve its semantic understanding and syntactic information processing capabilities.Additionally,A relation reference filling mechanism(RF)is proposed to enhance the semantic interaction between relations and entities,utilize relations to guide entity modeling,improve the boundary prediction ability of entity models for nested entity phenomena,and increase the cascading accuracy of entity-relation triples.Meanwhile,the“Queue”sub-task connection strategy is adopted to alleviate triplet cascading errors caused by overlapping relations,and a Syntax-enhanced entity relation extraction model(SE-RE)is constructed.The model showed excellent performance on the self-constructed E-commerce Product Information dataset(EPI)in this article.The results demonstrate that integrating POS enhancement into the pre-trained encoding model significantly boosts the performance of entity relation extraction models compared to baseline methods.Specifically,the F1-score fluctuation in subtasks caused by error accumulation was reduced by 3.21%,while the F1-score for entity-relation triplet extraction improved by 1.91%.
基金Supported by the Zhejiang Provincial Natural Science Foundation(No.LQ16H180004)~~
文摘Objectives Medical knowledge extraction (MKE) plays a key role in natural language processing (NLP) research in electronic medical records (EMR),which are the important digital carriers for recording medical activities of patients.Named entity recognition (NER) and medical relation extraction (MRE) are two basic tasks of MKE.This study aims to improve the recognition accuracy of these two tasks by exploring deep learning methods.Methods This study discussed and built two application scenes of bidirectional long short-term memory combined conditional random field (BiLSTM-CRF) model for NER and MRE tasks.In the data preprocessing of both tasks,a GloVe word embedding model was used to vectorize words.In the NER task,a sequence labeling strategy was used to classify each word tag by the joint probability distribution through the CRF layer.In the MRE task,the medical entity relation category was predicted by transforming the classification problem of a single entity into a sequence classification problem and linking the feature combinations between entities also through the CRF layer.Results Through the validation on the I2B2 2010 public dataset,the BiLSTM-CRF models built in this study got much better results than the baseline methods in the two tasks,where the F1-measure was up to 0.88 in NER task and 0.78 in MRE task.Moreover,the model converged faster and avoided problems such as overfitting.Conclusion This study proved the good performance of deep learning on medical knowledge extraction.It also verified the feasibility of the BiLSTM-CRF model in different application scenarios,laying the foundation for the subsequent work in the EMR field.
基金supported by State Grid Shanghai Economic Research Institute under Grant No.SGTYHT/23-JS-004.
文摘Due to the complex structural hierarchy,with deeply nested associative relations between entities such as equipment,specifications,and business processes,intelligent power grid engineering is challenging.Meanwhile,limited by the fragmented data and loss of contextual information,the generated reports are prone to the problems such as content redundancy and omission of critical information,failing to meet the demands of efficient decision-making and accurate management in modern power systems.To address these issues,this paper proposes a knowledge graph(KG)-enhanced framework to automatically generate electric power engineering reports.In the KG construction phase,a feature-fused entity recognition model named BERT-BiLSTM-CRF is adopted to improve the accuracy of entity recognition in scenarios involving power engineering professional terminology,thereby solving the problem of ambiguous entity boundaries in traditional models;then a BERT-attention relation extraction model is proposed to enhance the completeness of extracting complex hierarchical and implicit relations in power grid data.In the report generation phase,an improved Transformer architecture is adopted to accurately transform structured knowledge into natural language reports that comply with engineering specifications,addressing the issue of semantic inconsistency caused by the loss of structural information in existing models.By validating with real-world projects,the results show that the proposed framework significantly outperforms existing baseline models in entity recognition,confirming its superiority and applicability in practical engineering.
基金Zhifang Liao:Ministry of Science and Technology:Key Research and Development Project(2018YFB003800),Hunan Provincial Key Laboratory of Finance&Economics Big Data Scienceand Technology(Hunan University of Finance and Economics)2017TP1025,HNNSF 2018JJ2535Shengzong Liu:NSF61802120.
文摘Entity recognition and extraction are the foundations of knowledge graph construction.Entity data in the field of software engineering come from different platforms and communities,and have different formats.This paper divides multi-source software knowledge entities into unstructured data,semi-structured data and code data.For these different types of data,Bi-directional Long Short-Term Memory(Bi-LSTM)with Conditional Random Field(CRF),template matching,and abstract syntax tree are used and integrated into a multi-source software knowledge entity extraction integration model(MEIM)to extract software entities.The model can be updated continuously based on user’s feedbacks to improve the accuracy.To deal with the shortage of entity annotation datasets,keyword extraction methods based on Term Frequency–Inverse Document Frequency(TF-IDF),TextRank,and K-Means are applied to annotate tasks.The proposed MEIM model is applied to the Spring Boot framework,which demonstrates good adaptability.The extracted entities are used to construct a knowledge graph,which is applied to association retrieval and association visualization.
基金This project is supported by Major International Cooperation Program of NSFC Grant 60221120145 Chinese Folk Music Digital Library.
文摘Hidden Web provides great amount of domain-specific data for constructing knowledge services. Most previous knowledge extraction researches ignore the valuable data hidden in Web database, and related works do not refer how to make extracted information available for knowledge system. This paper describes a novel approach to build a domain-specific knowledge service with the data retrieved from Hidden Web. Ontology serves to model the domain knowledge. Queries forms of different Web sites are translated into machine-understandable format, defined knowledge concepts, so that they can be accessed automatically. Also knowledge data are extracted from Web pages and organized in ontology format knowledge. The experiment proves the algorithm achieves high accuracy and the system facilitates constructing knowledge services greatly.
基金supported by the National Natural Science Foundation of China(Grant No.81973695)Discipline with Strong Characteristics of Liaocheng University-Intelligent Science and Technology(Grant No.319462208).
文摘Event extraction stands as a significant endeavor within the realm of information extraction,aspiring to automatically extract structured event information from vast volumes of unstructured text.Extracting event elements from multi-modal data remains a challenging task due to the presence of a large number of images and overlapping event elements in the data.Although researchers have proposed various methods to accomplish this task,most existing event extraction models cannot address these challenges because they are only applicable to text scenarios.To solve the above issues,this paper proposes a multi-modal event extraction method based on knowledge fusion.Specifically,for event-type recognition,we use a meticulous pipeline approach that integrates multiple pre-trained models.This approach enables a more comprehensive capture of the multidimensional event semantic features present in military texts,thereby enhancing the interconnectedness of information between trigger words and events.For event element extraction,we propose a method for constructing a priori templates that combine event types with corresponding trigger words.This approach facilitates the acquisition of fine-grained input samples containing event trigger words,thus enabling the model to understand the semantic relationships between elements in greater depth.Furthermore,a fusion method for spatial mapping of textual event elements and image elements is proposed to reduce the category number overload and effectively achieve multi-modal knowledge fusion.The experimental results based on the CCKS 2022 dataset show that our method has achieved competitive results,with a comprehensive evaluation value F1-score of 53.4%for the model.These results validate the effectiveness of our method in extracting event elements from multi-modal data.
基金National Natural Science Foundation of China,Grant/Award Numbers:61671064,61732005National Key Research&Development Program,Grant/Award Number:2018YFC0831700。
文摘Emotion cause extraction(ECE)task that aims at extracting potential trigger events of certain emotions has attracted extensive attention recently.However,current work neglects the implicit emotion expressed without any explicit emotional keywords,which appears more frequently in application scenarios.The lack of explicit emotion information makes it extremely hard to extract emotion causes only with the local context.Moreover,an entire event is usually across multiple clauses,while existing work merely extracts cause events at clause level and cannot effectively capture complete cause event information.To address these issues,the events are first redefined at the tuple level and a span-based tuple-level algorithm is proposed to extract events from different clauses.Based on it,a corpus for implicit emotion cause extraction that tries to extract causes of implicit emotions is constructed.The authors propose a knowledge-enriched jointlearning model of implicit emotion recognition and implicit emotion cause extraction tasks(KJ-IECE),which leverages commonsense knowledge from ConceptNet and NRC_VAD to better capture connections between emotion and corresponding cause events.Experiments on both implicit and explicit emotion cause extraction datasets demonstrate the effectiveness of the proposed model.
文摘To address the difficulty of training high-quality models in some specific domains due to the lack of fine-grained annotation resources, we propose in this paper a knowledge-integrated cross-domain data generation method for unsupervised domain adaptation tasks. Specifically, we extract domain features, lexical and syntactic knowledge from source-domain and target-domain data, and use a masking model with an extended masking strategy and a re-masking strategy to obtain domain-specific data that remove domain-specific features. Finally, we improve the sequence generation model BART and use it to generate high-quality target domain data for the task of aspect and opinion co-extraction from the target domain. Experiments were performed on three conventional English datasets from different domains, and our method generates more accurate and diverse target domain data with the best results compared to previous methods.
文摘Given the scarcity of Satellite Frequency and Orbit(SFO)resources,it holds paramount importance to establish a comprehensive knowledge graph of SFO field(SFO-KG)and employ knowledge reasoning technology to automatically mine available SFO resources.An essential aspect of constructing SFO-KG is the extraction of Chinese entity relations.Unfortunately,there is currently no publicly available Chinese SFO entity Relation Extraction(RE)dataset.Moreover,publicly available SFO text data contain numerous NA(representing for“No Answer”)relation category sentences that resemble other relation sentences and pose challenges in accurate classification,resulting in low recall and precision for the NA relation category in entity RE.Consequently,this issue adversely affects both the accuracy of constructing the knowledge graph and the efficiency of RE processes.To address these challenges,this paper proposes a method for extracting Chinese SFO text entity relations based on dynamic integrated learning.This method includes the construction of a manually annotated Chinese SFO entity RE dataset and a classifier combining features of SFO resource data.The proposed approach combines integrated learning and pre-training models,specifically utilizing Bidirectional Encoder Representation from Transformers(BERT).In addition,it incorporates one-class classification,attention mechanisms,and dynamic feedback mechanisms to improve the performance of the RE model.Experimental results show that the proposed method outperforms the traditional methods in terms of F1 value when extracting entity relations from both balanced and long-tailed datasets.
基金support by the Science and Technology Project of Guangdong Power Exchange Center Co.,Ltd.(No.GDKJXM20222599)National Natural Science Foundation of China(No.52207104)Natural Science Foundation of Guangdong Province(No.2024A1515010426).
文摘The rapid development of electricity retail market has prompted an increasing number of electricity consumers to sign green electricity contracts with retail electricity companies,which poses greater challenges for the market service for green energy consumers.This study proposed a two-stage feature extraction approach for green energy consumers leveraging clustering and termfrequency-inverse document frequency(TF-IDF)algorithms within a knowledge graph framework to provide an information basis that supports the green development of the retail electricity market.First,the multi-source heterogeneous data of green energy consumers under an actual market environment is systematically introduced and the information is categorized into discrete,interval,and relational features.A clustering algorithm was employed to extract features of the trading behavior of green energy consumers in the first stage using the parameter data of green retail electricity contracts.Then,TF-IDF algorithm was applied in the second stage to extract features for green energy consumers in different clusters.Finally,the effectiveness of the proposed approach was validated based on the actual operational data in a southern province of China.It is shown that the most significant discrepancy between the retail trading behaviors of green energy consumers is the power share of green retail packages,whose averaged values are 25.64%,50%,39.66%,and 24.89%in four different clusters,respectively.Additionally,power supply bureaus and electricity retail companies affects the behavior of the green energy consumers most significantly.
文摘The cost and strict input format requirements of GraphRAG make it less efficient for processing large documents. This paper proposes an alternative approach for constructing a knowledge graph (KG) from a PDF document with a focus on simplicity and cost-effectiveness. The process involves splitting the document into chunks, extracting concepts within each chunk using a large language model (LLM), and building relationships based on the proximity of concepts in the same chunk. Unlike traditional named entity recognition (NER), which identifies entities like “Shanghai”, the proposed method identifies concepts, such as “Convenient transportation in Shanghai” which is found to be more meaningful for KG construction. Each edge in the KG represents a relationship between concepts occurring in the same text chunk. The process is computationally inexpensive, leveraging locally set up tools like Mistral 7B openorca instruct and Ollama for model inference, ensuring the entire graph generation process is cost-free. A method of assigning weights to relationships, grouping similar pairs, and summarizing multiple relationships into a single edge with associated weight and relation details is introduced. Additionally, node degrees and communities are calculated for node sizing and coloring. This approach offers a scalable, cost-effective solution for generating meaningful knowledge graphs from large documents, achieving results comparable to GraphRAG while maintaining accessibility for personal machines.
文摘The article is based on language model,through the cue word engineering and agent thinking method,automatic knowledge extraction,with China accounting standards support to complete the corresponding knowledge map construction.Through the way of extracting the accounting entities and their connections in the pattern layer,the data layer is provided for the fine-tuning and optimization of the large model.Studies found that,through the reasonable application of language model,knowledge can be realized in massive financial data neural five effective extracted tuples,and complete accounting knowledge map construction.
基金supported by the National Social Science Foundation Major Project(22&ZD135)the National Social Science Fund National Emergency Management System Construction Research Project(20VYJ061).
文摘With the popularization of social media,public opi-nion information on emergencies spreads rapidly on the Internet,the impact of negative public opinions on an event has become more significant.Based on the organizational form of public opinion information,the knowledge graph is used to construct the knowledge base of public opinion risk cases on the emer-gency network.The emotion recognition model of negative pub-lic opinion information based on the bi-directional long short-term memory(BiLSTM)network is studied in the model layer design,and a linear discriminant analysis(LDA)topic extraction method combined with association rules is proposed to extract and mine the semantics of negative public opinion topics to real-ize further in-depth analysis of information topics.Focusing on public health emergencies,knowledge acquisition and knowl-edge processing of public opinion information are conducted,and the experimental results show that the knowledge graph framework based on the construction can facilitate in-depth theme evolution analysis of public opinion events,thus demon-strating important research significance for reducing online pub-lic opinion risks.
文摘Psychological counseling Q&A system is enjoying a remarkable and increasing popularity in recent years. Knowledge base is the important component for such kind of systems, but it is difficult and time-consuming to construct the knowledge base manually. Fortunately, there emerges large number of Q&A pairs in many psychological counseling websites, which can provide good source enriching the knowledge base. This paper presents the method of knowledge extraction from psychological consulting Q&A pairs of on-line psychological counseling websites, which include keywords, semantic extension and word sequence. P-XML, which is the knowledge template based on XML, is also designed to store the knowledge. The extracted knowledge has been successfully used in our non-obstructive psychologycal counseling system, called P.A.L., and the experimental results also demonstrated the feasibility and effectiveness of our approach.
基金financially supported by the National Key R&D Program of China (No.2022YFF0711601)the Natural Science Foundation of Hubei Province of China (No.2022CFB640)+2 种基金the Open Fund of Key Laboratory of Urban Land Resources Monitoring and Simulation,Ministry of Natural Resources (No.KF-2022-07-014)the Opening Fund of Hubei Key Laboratory of Intelligent Vision-Based Monitoring for Hydroelectric Engineering (No.2022SDSJ04)the Beijing Key Laboratory of Urban Spatial Information Engineering (No.20220108)。
文摘Geological knowledge can provide support for knowledge discovery, knowledge inference and mineralization predictions of geological big data. Entity identification and relationship extraction from geological data description text are the key links for constructing knowledge graphs. Given the lack of publicly annotated datasets in the geology domain, this paper illustrates the construction process of geological entity datasets, defines the types of entities and interconceptual relationships by using the geological entity concept system, and completes the construction of the geological corpus. To address the shortcomings of existing language models(such as Word2vec and Glove) that cannot solve polysemous words and have a poor ability to fuse contexts, we propose a geological named entity recognition and relationship extraction model jointly with Bidirectional Encoder Representation from Transformers(BERT) pretrained language model. To effectively represent the text features, we construct a BERT-bidirectional gated recurrent unit network(BiGRU)-conditional random field(CRF)-based architecture to extract the named entities and the BERT-BiGRU-Attention-based architecture to extract the entity relations. The results show that the F1-score of the BERT-BiGRU-CRF named entity recognition model is 0.91 and the F1-score of the BERT-BiGRU-Attention relationship extraction model is 0.84, which are significant performance improvements when compared to classic language models(e.g., word2vec and Embedding from Language Models(ELMo)).
基金supported in part by the Beijing Natural Science Foundation under Grants L211020 and M21032in part by the National Natural Science Foundation of China under Grants U1836106 and 62271045in part by the Scientific and Technological Innovation Foundation of Foshan under Grants BK21BF001 and BK20BF010。
文摘Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.
基金This research is supported by the Chinese Special Projects of the National Key Research and Development Plan(2019YFB1405702).
文摘The acquisition of valuable design knowledge from massive fragmentary data is challenging for designers in conceptual product design.This study proposes a novel method for acquiring design knowledge by combining deep learning with knowledge graph.Specifically,the design knowledge acquisition method utilises the knowledge extraction model to extract design-related entities and relations from fragmentary data,and further constructs the knowledge graph to support design knowledge acquisition for conceptual product design.Moreover,the knowledge extraction model introduces ALBERT to solve memory limitation and communication overhead in the entity extraction module,and uses multi-granularity information to overcome segmentation errors and polysemy ambiguity in the relation extraction module.Experimental comparison verified the effectiveness and accuracy of the proposed knowledge extraction model.The case study demonstrated the feasibility of the knowledge graph construction with real fragmentary porcelain data and showed the capability to provide designers with interconnected and visualised design knowledge.
