Learning discriminative representations with deep neural networks often relies on massive labeled data, which is expensive and difficult to obtain in many real scenarios. As an alternative, self-supervised learning th...Learning discriminative representations with deep neural networks often relies on massive labeled data, which is expensive and difficult to obtain in many real scenarios. As an alternative, self-supervised learning that leverages input itself as supervision is strongly preferred for its soaring performance on visual representation learning. This paper introduces a contrastive self-supervised framework for learning generalizable representations on the synthetic data that can be obtained easily with complete controllability.Specifically, we propose to optimize a contrastive learning task and a physical property prediction task simultaneously. Given the synthetic scene, the first task aims to maximize agreement between a pair of synthetic images generated by our proposed view sampling module, while the second task aims to predict three physical property maps, i.e., depth, instance contour maps, and surface normal maps. In addition, a feature-level domain adaptation technique with adversarial training is applied to reduce the domain difference between the realistic and the synthetic data. Experiments demonstrate that our proposed method achieves state-of-the-art performance on several visual recognition datasets.展开更多
Computed Tomography(CT)reconstruction is essential inmedical imaging and other engineering fields.However,blurring of the projection during CT imaging can lead to artifacts in the reconstructed images.Projection blur ...Computed Tomography(CT)reconstruction is essential inmedical imaging and other engineering fields.However,blurring of the projection during CT imaging can lead to artifacts in the reconstructed images.Projection blur combines factors such as larger ray sources,scattering and imaging system vibration.To address the problem,we propose DeblurTomo,a novel self-supervised learning-based deblurring and reconstruction algorithm that efficiently reconstructs sharp CT images from blurry input without needing external data and blur measurement.Specifically,we constructed a coordinate-based implicit neural representation reconstruction network,which can map the coordinates to the attenuation coefficient in the reconstructed space formore convenient ray representation.Then,wemodel the blur as aweighted sumof offset rays and design the RayCorrectionNetwork(RCN)andWeight ProposalNetwork(WPN)to fit these rays and their weights bymulti-view consistency and geometric information,thereby extending 2D deblurring to 3D space.In the training phase,we use the blurry input as the supervision signal to optimize the reconstruction network,the RCN,and the WPN simultaneously.Extensive experiments on the widely used synthetic dataset show that DeblurTomo performs superiorly on the limited-angle and sparse-view in the simulated blurred scenarios.Further experiments on real datasets demonstrate the superiority of our method in practical scenarios.展开更多
The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of netw...The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.展开更多
Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based o...Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based on graph convolutional network(GCN).Methods Clauses that contain symptoms,formulas,and herbs were abstracted from Treatise on Febrile Diseases to construct symptom-formula-herb heterogeneous graphs,which were used to propose a node representation learning method based on GCN−the Traditional Chinese Medicine Graph Convolution Network(TCM-GCN).The symptom-formula,symptom-herb,and formula-herb heterogeneous graphs were processed with the TCM-GCN to realize high-order propagating message passing and neighbor aggregation to obtain new node representation attributes,and thus acquiring the nodes’sum-aggregations of symptoms,formulas,and herbs to lay a foundation for the downstream tasks of the prediction models.Results Comparisons among the node representations with multi-hot encoding,non-fusion encoding,and fusion encoding showed that the Precision@10,Recall@10,and F1-score@10 of the fusion encoding were 9.77%,6.65%,and 8.30%,respectively,higher than those of the non-fusion encoding in the prediction studies of the model.Conclusion Node representations by fusion encoding achieved comparatively ideal results,indicating the TCM-GCN is effective in realizing node-level representations of heterogeneous graph structured Treatise on Febrile Diseases dataset and is able to elevate the performance of the downstream tasks of the diagnosis model.展开更多
Few-shot learning has emerged as a crucial technique for coral species classification,addressing the challenge of limited labeled data in underwater environments.This study introduces an optimized few-shot learning mo...Few-shot learning has emerged as a crucial technique for coral species classification,addressing the challenge of limited labeled data in underwater environments.This study introduces an optimized few-shot learning model that enhances classification accuracy while minimizing reliance on extensive data collection.The proposed model integrates a hybrid similarity measure combining Euclidean distance and cosine similarity,effectively capturing both feature magnitude and directional relationships.This approach achieves a notable accuracy of 71.8%under a 5-way 5-shot evaluation,outperforming state-of-the-art models such as Prototypical Networks,FEAT,and ESPT by up to 10%.Notably,the model demonstrates high precision in classifying Siderastreidae(87.52%)and Fungiidae(88.95%),underscoring its effectiveness in distinguishing subtle morphological differences.To further enhance performance,we incorporate a self-supervised learning mechanism based on contrastive learning,enabling the model to extract robust representations by leveraging local structural patterns in corals.This enhancement significantly improves classification accuracy,particularly for species with high intra-class variation,leading to an overall accuracy of 76.52%under a 5-way 10-shot evaluation.Additionally,the model exploits the repetitive structures inherent in corals,introducing a local feature aggregation strategy that refines classification through spatial information integration.Beyond its technical contributions,this study presents a scalable and efficient approach for automated coral reef monitoring,reducing annotation costs while maintaining high classification accuracy.By improving few-shot learning performance in underwater environments,our model enhances monitoring accuracy by up to 15%compared to traditional methods,offering a practical solution for large-scale coral conservation efforts.展开更多
Unsupervised learning methods such as graph contrastive learning have been used for dynamic graph represen-tation learning to eliminate the dependence of labels.However,existing studies neglect positional information ...Unsupervised learning methods such as graph contrastive learning have been used for dynamic graph represen-tation learning to eliminate the dependence of labels.However,existing studies neglect positional information when learning discrete snapshots,resulting in insufficient network topology learning.At the same time,due to the lack of appropriate data augmentation methods,it is difficult to capture the evolving patterns of the network effectively.To address the above problems,a position-aware and subgraph enhanced dynamic graph contrastive learning method is proposed for discrete-time dynamic graphs.Firstly,the global snapshot is built based on the historical snapshots to express the stable pattern of the dynamic graph,and the random walk is used to obtain the position representation by learning the positional information of the nodes.Secondly,a new data augmentation method is carried out from the perspectives of short-term changes and long-term stable structures of dynamic graphs.Specifically,subgraph sampling based on snapshots and global snapshots is used to obtain two structural augmentation views,and node structures and evolving patterns are learned by combining graph neural network,gated recurrent unit,and attention mechanism.Finally,the quality of node representation is improved by combining the contrastive learning between different structural augmentation views and between the two representations of structure and position.Experimental results on four real datasets show that the performance of the proposed method is better than the existing unsupervised methods,and it is more competitive than the supervised learning method under a semi-supervised setting.展开更多
Graph representation learning often faces knowledge scarcity in real-world applications,including limited labels and sparse relationships.Although a range of methods have been proposed to address these problems,such a...Graph representation learning often faces knowledge scarcity in real-world applications,including limited labels and sparse relationships.Although a range of methods have been proposed to address these problems,such as graph few-shot learning,they mainly rely on inadequate knowledge within the task graph,which would limit their effectiveness.Moreover,they fail to consider other potentially useful task-related graphs.To overcome these limitations,domain adaptation for graph representation learning has emerged as an effective paradigm for transferring knowledge across graphs.It is also recognized as graph domain adaptation(GDA).In particular,to enhance model performance on target graphs with specific tasks,GDA introduces a bunch of task-related graphs as source graphs and adapts the knowledge learnt from source graphs to the target graphs.Since GDA combines the advantages of graph representation learning and domain adaptation,it has become a promising direction of transfer learning on graphs and has attracted an increasing amount of research interest in recent years.In this paper,we comprehensively overview the studies of GDA and present a detailed survey of recent advances.Specifically,we outline the current research status,analyze key challenges,propose a taxonomy,introduce representative work and practical applications,and discuss future prospects.To the best of our knowledge,this paper is the first survey for graph domain adaptation.展开更多
In view of the low interpretability of existing collaborative filtering recommendation algorithms and the difficulty of extracting information from content-based recommendation algorithms,we propose an efficient KGRS ...In view of the low interpretability of existing collaborative filtering recommendation algorithms and the difficulty of extracting information from content-based recommendation algorithms,we propose an efficient KGRS model.KGRS first obtains reasoning paths of knowledge graph and embeds the entities of paths into vectors based on knowledge representation learning TransD algorithm,then uses LSTM and soft attention mechanism to capture the semantic of each path reasoning,then uses convolution operation and pooling operation to distinguish the importance of different paths reasoning.Finally,through the full connection layer and sigmoid function to get the prediction ratings,and the items are sorted according to the prediction ratings to get the user’s recommendation list.KGRS is tested on the movielens-100k dataset.Compared with the related representative algorithm,including the state-of-the-art interpretable recommendation models RKGE and RippleNet,the experimental results show that KGRS has good recommendation interpretation and higher recommendation accuracy.展开更多
With the popularity of online learning in educational settings, knowledge tracing(KT) plays an increasingly significant role. The task of KT is to help students learn more effectively by predicting their next mastery ...With the popularity of online learning in educational settings, knowledge tracing(KT) plays an increasingly significant role. The task of KT is to help students learn more effectively by predicting their next mastery of knowledge based on their historical exercise sequences. Nowadays, many related works have emerged in this field, such as Bayesian knowledge tracing and deep knowledge tracing methods. Despite the progress that has been made in KT, existing techniques still have the following limitations: 1) Previous studies address KT by only exploring the observational sparsity data distribution, and the counterfactual data distribution has been largely ignored. 2) Current works designed for KT only consider either the entity relationships between questions and concepts, or the relations between two concepts, and none of them investigates the relations among students, questions, and concepts, simultaneously, leading to inaccurate student modeling. To address the above limitations,we propose a graph counterfactual augmentation method for knowledge tracing. Concretely, to consider the multiple relationships among different entities, we first uniform students, questions, and concepts in graphs, and then leverage a heterogeneous graph convolutional network to conduct representation learning.To model the counterfactual world, we conduct counterfactual transformations on students’ learning graphs by changing the corresponding treatments and then exploit the counterfactual outcomes in a contrastive learning framework. We conduct extensive experiments on three real-world datasets, and the experimental results demonstrate the superiority of our proposed Graph CA method compared with several state-of-the-art baselines.展开更多
Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of t...Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.展开更多
Smart contracts have led to more efficient development in finance and healthcare,but vulnerabilities in contracts pose high risks to their future applications.The current vulnerability detection methods for contracts ...Smart contracts have led to more efficient development in finance and healthcare,but vulnerabilities in contracts pose high risks to their future applications.The current vulnerability detection methods for contracts are either based on fixed expert rules,which are inefficient,or rely on simplistic deep learning techniques that do not fully leverage contract semantic information.Therefore,there is ample room for improvement in terms of detection precision.To solve these problems,this paper proposes a vulnerability detector based on deep learning techniques,graph representation,and Transformer,called GRATDet.The method first performs swapping,insertion,and symbolization operations for contract functions,increasing the amount of small sample data.Each line of code is then treated as a basic semantic element,and information such as control and data relationships is extracted to construct a new representation in the form of a Line Graph(LG),which shows more structural features that differ from the serialized presentation of the contract.Finally,the node information and edge information of the graph are jointly learned using an improved Transformer-GP model to extract information globally and locally,and the fused features are used for vulnerability detection.The effectiveness of the method in reentrancy vulnerability detection is verified in experiments,where the F1 score reaches 95.16%,exceeding stateof-the-art methods.展开更多
The proliferation of internet traffic encryption has become a double-edged sword. While it significantly enhances user privacy, it also inadvertently shields cyber-attacks from detection, presenting a formidable chall...The proliferation of internet traffic encryption has become a double-edged sword. While it significantly enhances user privacy, it also inadvertently shields cyber-attacks from detection, presenting a formidable challenge to cybersecurity. Traditional machine learning and deep learning techniques often fall short in identifying encrypted malicious traffic due to their inability to fully extract and utilize the implicit relational and positional information embedded within data packets. This limitation has led to an unresolved challenge in the cybersecurity community: how to effectively extract valuable insights from the complex patterns of traffic packet transmission. Consequently, this paper introduces the TB-Graph model, an encrypted malicious traffic classification model based on a relational graph attention network. The model is a heterogeneous traffic burst graph that embeds side-channel features, which are unaffected by encryption, into the graph nodes and connects them with three different types of burst edges. Subsequently, we design a relational positional coding that prevents the loss of temporal relationships between the original traffic flows during graph transformation. Ultimately, TB-Graph leverages the powerful graph representation learning capabilities of Relational Graph Attention Network (RGAT) to extract latent behavioral features from the burst graph nodes and edge relationships. Experimental results show that TB-Graph outperforms various state-of-the-art methods in fine-grained encrypted malicious traffic classification tasks on two public datasets, indicating its enhanced capability for identifying encrypted malicious traffic.展开更多
Entity alignment,which aims to identify entities with the same meaning in different Knowledge Graphs(KGs),is a key step in knowledge integration.Despite the promising results achieved by existing methods,they often fa...Entity alignment,which aims to identify entities with the same meaning in different Knowledge Graphs(KGs),is a key step in knowledge integration.Despite the promising results achieved by existing methods,they often fail to fully leverage the structure information of KGs for entity alignment.Therefore,our goal is to thoroughly explore the features of entity neighbors and relationships to obtain better entity embeddings.In this work,we propose DCEA,an effective dual-context representation learning framework for entity alignment.Specifically,the neighbor-level embedding module introduces relation information to more accurately aggregate neighbor context.The relation-level embedding module utilizes neighbor context to enhance relation-level embeddings.To eliminate semantic gaps between neighbor-level and relation-level embeddings,and fully exploit their complementarity,we design a hybrid embedding fusion model that adaptively performs embedding fusion to obtain powerful joint entity embeddings.We also jointly optimize the contrastive loss of multi-level embeddings,enhancing their mutual reinforcement while preserving the characteristics of neighbor and relation embeddings.Additionally,the decision fusion module combines the similarity scores calculated between entities based on embeddings at different levels to make the final alignment decision.Extensive experimental results on public datasets indicate that our DCEA performs better than state-of-the-art baselines.展开更多
The escalating complexity and heterogeneity of modern energy systems—particularly in smart grid and distributed energy infrastructures—has intensified the need for intelligent and scalable security vulnerability cla...The escalating complexity and heterogeneity of modern energy systems—particularly in smart grid and distributed energy infrastructures—has intensified the need for intelligent and scalable security vulnerability classification.To address this challenge,we propose Vulnerability2Vec,a graph-embedding-based framework designed to enhance the automated classification of security vulnerabilities that threaten energy system resilience.Vulnerability2Vec converts Common Vulnerabilities and Exposures(CVE)text explanations to semantic graphs,where nodes represent CVE IDs and key terms(nouns,verbs,and adjectives),and edges capture co-occurrence relationships.Then,it embeds the semantic graphs to a low-dimensional vector space with random-walk sampling and skip-gram with negative sampling.It is possible to identify the latent relationships and structural patterns that traditional sparse vector methods fail to capture.Experimental results demonstrate a classification accuracy of up to 80%,significantly outperforming baseline methods.This approach offers a theoretical basis for classifying vulnerability types as structured semantic patterns in complex software systems.The proposed method models the semantic structure of vulnerabilities,providing a theoretical foundation for their classification.展开更多
With the development of the Semantic Web,the number of ontologies grows exponentially and the semantic relationships between ontologies become more and more complex,understanding the true semantics of specific terms o...With the development of the Semantic Web,the number of ontologies grows exponentially and the semantic relationships between ontologies become more and more complex,understanding the true semantics of specific terms or concepts in an ontology is crucial for the matching task.At present,the main challenges facing ontology matching tasks based on representation learning methods are how to improve the embedding quality of ontology knowledge and how to integrate multiple features of ontology efficiently.Therefore,we propose an Ontology Matching Method Based on the Gated Graph Attention Model(OM-GGAT).Firstly,the semantic knowledge related to concepts in the ontology is encoded into vectors using the OWL2Vec^(*)method,and the relevant path information from the root node to the concept is embedded to understand better the true meaning of the concept itself and the relationship between concepts.Secondly,the ontology is transformed into the corresponding graph structure according to the semantic relation.Then,when extracting the features of the ontology graph nodes,different attention weights are assigned to each adjacent node of the central concept with the help of the attention mechanism idea.Finally,gated networks are designed to further fuse semantic and structural embedding representations efficiently.To verify the effectiveness of the proposed method,comparative experiments on matching tasks were carried out on public datasets.The results show that the OM-GGAT model can effectively improve the efficiency of ontology matching.展开更多
In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with l...In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.展开更多
Various graph representation learning models convert graph nodes into vectors using techniques like matrix factorization,random walk,and deep learning.However,choosing the right method for different tasks can be chall...Various graph representation learning models convert graph nodes into vectors using techniques like matrix factorization,random walk,and deep learning.However,choosing the right method for different tasks can be challenging.Communities within networks help reveal underlying structures and correlations.Investigating how different models preserve community properties is crucial for identifying the best graph representation for data analysis.This paper defines indicators to explore the perceptual quality of community properties in representation learning spaces,including the consistency of community structure,node distribution within and between communities,and central node distribution.A visualization system presents these indicators,allowing users to evaluate models based on community structures.Case studies demonstrate the effectiveness of the indicators for the visual evaluation of graph representation learning models.展开更多
Graph contrastive learning(GCL)has attracted extensive research interest due to its powerful ability to capture latent structural and semantic information of graphs in a self-supervised manner.Existing GCL methods com...Graph contrastive learning(GCL)has attracted extensive research interest due to its powerful ability to capture latent structural and semantic information of graphs in a self-supervised manner.Existing GCL methods commonly adopt predefined graph augmentations to generate two contrastive views.Subsequently,they design a contrastive pretext task between these views with the goal of maximizing their agreement.These methods as-sume the augmented graph can fully preserve the semantics of the original.However,typical data augmentation strategies in GCL,such as random edge dropping,may alter the properties of the original graph.As a result,previous GCL methods overlooked graph differences,potentially leading to difficulty distinguishing between graphs that are structurally similar but semantically different.Therefore,we argue that it is necessary to design a method that can quantify the dissimilarity between the original and augmented graphs to more accurately capture the relationships between samples.In this work,we propose a novel graph contrastive learning framework,named Accurate Difference-based Node-Level Graph Contrastive Learning(DNGCL),which helps the model distinguish similar graphs with slight differences by learning node-level differences between graphs.Specifically,we train the model to distinguish between original and augmented nodes via a node discriminator and employ cosine dissimilarity to accurately measure the difference between each node.Furthermore,we employ multiple types of data augmentation commonly used in current GCL methods on the original graph,aiming to learn the differences between nodes under different augmentation strategies and help the model learn richer local information.We conduct extensive experiments on six benchmark datasets and the results show that our DNGCL outperforms most state-of-the-art baselines,which strongly validates the effectiveness of our model.展开更多
基金by National Natural Science Foundation of China(Nos.61822204 and 61521002).
文摘Learning discriminative representations with deep neural networks often relies on massive labeled data, which is expensive and difficult to obtain in many real scenarios. As an alternative, self-supervised learning that leverages input itself as supervision is strongly preferred for its soaring performance on visual representation learning. This paper introduces a contrastive self-supervised framework for learning generalizable representations on the synthetic data that can be obtained easily with complete controllability.Specifically, we propose to optimize a contrastive learning task and a physical property prediction task simultaneously. Given the synthetic scene, the first task aims to maximize agreement between a pair of synthetic images generated by our proposed view sampling module, while the second task aims to predict three physical property maps, i.e., depth, instance contour maps, and surface normal maps. In addition, a feature-level domain adaptation technique with adversarial training is applied to reduce the domain difference between the realistic and the synthetic data. Experiments demonstrate that our proposed method achieves state-of-the-art performance on several visual recognition datasets.
基金supported in part by the National Natural Science Foundation of China under Grants 62472434 and 62402171in part by the National Key Research and Development Program of China under Grant 2022YFF1203001+1 种基金in part by the Science and Technology Innovation Program of Hunan Province under Grant 2022RC3061in part by the Sci-Tech Innovation 2030 Agenda under Grant 2023ZD0508600.
文摘Computed Tomography(CT)reconstruction is essential inmedical imaging and other engineering fields.However,blurring of the projection during CT imaging can lead to artifacts in the reconstructed images.Projection blur combines factors such as larger ray sources,scattering and imaging system vibration.To address the problem,we propose DeblurTomo,a novel self-supervised learning-based deblurring and reconstruction algorithm that efficiently reconstructs sharp CT images from blurry input without needing external data and blur measurement.Specifically,we constructed a coordinate-based implicit neural representation reconstruction network,which can map the coordinates to the attenuation coefficient in the reconstructed space formore convenient ray representation.Then,wemodel the blur as aweighted sumof offset rays and design the RayCorrectionNetwork(RCN)andWeight ProposalNetwork(WPN)to fit these rays and their weights bymulti-view consistency and geometric information,thereby extending 2D deblurring to 3D space.In the training phase,we use the blurry input as the supervision signal to optimize the reconstruction network,the RCN,and the WPN simultaneously.Extensive experiments on the widely used synthetic dataset show that DeblurTomo performs superiorly on the limited-angle and sparse-view in the simulated blurred scenarios.Further experiments on real datasets demonstrate the superiority of our method in practical scenarios.
基金This research is partially supported by the National Natural Science Foundation of China(Grant No.61772098)Chongqing Technology Innovation and Application Development Project(Grant No.cstc2020jscxmsxmX0150)+2 种基金Chongqing Science and Technology Innovation Leading Talent Support Program(CSTCCXLJRC201908)Basic and Advanced Research Projects of CSTC(No.cstc2019jcyj-zdxmX0008)Science and Technology Research Program of Chongqing Municipal Education Commission(Grant No.KJZD-K201900605).
文摘The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.
基金New-Generation Artificial Intelligence-Major Program in the Sci-Tech Innovation 2030 Agenda from the Ministry of Science and Technology of China(2018AAA0102100)Hunan Provincial Department of Education key project(21A0250)The First Class Discipline Open Fund of Hunan University of Traditional Chinese Medicine(2022ZYX08)。
文摘Objective To construct symptom-formula-herb heterogeneous graphs structured Treatise on Febrile Diseases(Shang Han Lun,《伤寒论》)dataset and explore an optimal learning method represented with node attributes based on graph convolutional network(GCN).Methods Clauses that contain symptoms,formulas,and herbs were abstracted from Treatise on Febrile Diseases to construct symptom-formula-herb heterogeneous graphs,which were used to propose a node representation learning method based on GCN−the Traditional Chinese Medicine Graph Convolution Network(TCM-GCN).The symptom-formula,symptom-herb,and formula-herb heterogeneous graphs were processed with the TCM-GCN to realize high-order propagating message passing and neighbor aggregation to obtain new node representation attributes,and thus acquiring the nodes’sum-aggregations of symptoms,formulas,and herbs to lay a foundation for the downstream tasks of the prediction models.Results Comparisons among the node representations with multi-hot encoding,non-fusion encoding,and fusion encoding showed that the Precision@10,Recall@10,and F1-score@10 of the fusion encoding were 9.77%,6.65%,and 8.30%,respectively,higher than those of the non-fusion encoding in the prediction studies of the model.Conclusion Node representations by fusion encoding achieved comparatively ideal results,indicating the TCM-GCN is effective in realizing node-level representations of heterogeneous graph structured Treatise on Febrile Diseases dataset and is able to elevate the performance of the downstream tasks of the diagnosis model.
基金funded by theNational Science and TechnologyCouncil(NSTC),Taiwan,under grant numbers NSTC 112-2634-F-019-001 and NSTC 113-2634-F-A49-007.
文摘Few-shot learning has emerged as a crucial technique for coral species classification,addressing the challenge of limited labeled data in underwater environments.This study introduces an optimized few-shot learning model that enhances classification accuracy while minimizing reliance on extensive data collection.The proposed model integrates a hybrid similarity measure combining Euclidean distance and cosine similarity,effectively capturing both feature magnitude and directional relationships.This approach achieves a notable accuracy of 71.8%under a 5-way 5-shot evaluation,outperforming state-of-the-art models such as Prototypical Networks,FEAT,and ESPT by up to 10%.Notably,the model demonstrates high precision in classifying Siderastreidae(87.52%)and Fungiidae(88.95%),underscoring its effectiveness in distinguishing subtle morphological differences.To further enhance performance,we incorporate a self-supervised learning mechanism based on contrastive learning,enabling the model to extract robust representations by leveraging local structural patterns in corals.This enhancement significantly improves classification accuracy,particularly for species with high intra-class variation,leading to an overall accuracy of 76.52%under a 5-way 10-shot evaluation.Additionally,the model exploits the repetitive structures inherent in corals,introducing a local feature aggregation strategy that refines classification through spatial information integration.Beyond its technical contributions,this study presents a scalable and efficient approach for automated coral reef monitoring,reducing annotation costs while maintaining high classification accuracy.By improving few-shot learning performance in underwater environments,our model enhances monitoring accuracy by up to 15%compared to traditional methods,offering a practical solution for large-scale coral conservation efforts.
文摘Unsupervised learning methods such as graph contrastive learning have been used for dynamic graph represen-tation learning to eliminate the dependence of labels.However,existing studies neglect positional information when learning discrete snapshots,resulting in insufficient network topology learning.At the same time,due to the lack of appropriate data augmentation methods,it is difficult to capture the evolving patterns of the network effectively.To address the above problems,a position-aware and subgraph enhanced dynamic graph contrastive learning method is proposed for discrete-time dynamic graphs.Firstly,the global snapshot is built based on the historical snapshots to express the stable pattern of the dynamic graph,and the random walk is used to obtain the position representation by learning the positional information of the nodes.Secondly,a new data augmentation method is carried out from the perspectives of short-term changes and long-term stable structures of dynamic graphs.Specifically,subgraph sampling based on snapshots and global snapshots is used to obtain two structural augmentation views,and node structures and evolving patterns are learned by combining graph neural network,gated recurrent unit,and attention mechanism.Finally,the quality of node representation is improved by combining the contrastive learning between different structural augmentation views and between the two representations of structure and position.Experimental results on four real datasets show that the performance of the proposed method is better than the existing unsupervised methods,and it is more competitive than the supervised learning method under a semi-supervised setting.
基金supported by the Strategic Priority Research Program of the Chinese Academy of Sciences(CAS)under Grant No.XDB0680302the National Key Research and Development Program of China under Grant No.2023YFC3305303+2 种基金the National Natural Science Foundation of China under Grant Nos.62372434 and 62302485the China Postdoctoral Science Foundation under Grant No.2022M713206the CAS Special Research Assistant Program,and the Key Research Project of Chinese Academy of Sciences under Grant No.RCJJ-145-24-21.
文摘Graph representation learning often faces knowledge scarcity in real-world applications,including limited labels and sparse relationships.Although a range of methods have been proposed to address these problems,such as graph few-shot learning,they mainly rely on inadequate knowledge within the task graph,which would limit their effectiveness.Moreover,they fail to consider other potentially useful task-related graphs.To overcome these limitations,domain adaptation for graph representation learning has emerged as an effective paradigm for transferring knowledge across graphs.It is also recognized as graph domain adaptation(GDA).In particular,to enhance model performance on target graphs with specific tasks,GDA introduces a bunch of task-related graphs as source graphs and adapts the knowledge learnt from source graphs to the target graphs.Since GDA combines the advantages of graph representation learning and domain adaptation,it has become a promising direction of transfer learning on graphs and has attracted an increasing amount of research interest in recent years.In this paper,we comprehensively overview the studies of GDA and present a detailed survey of recent advances.Specifically,we outline the current research status,analyze key challenges,propose a taxonomy,introduce representative work and practical applications,and discuss future prospects.To the best of our knowledge,this paper is the first survey for graph domain adaptation.
基金supported by the National Science Foundation of China Grant No.61762092“Dynamic multi-objective requirement optimization based on transfer learning”,No.61762089+2 种基金“The key research of high order tensor decomposition in distributed environment”the Open Foundation of the Key Laboratory in Software Engineering of Yunnan Province,Grant No.2017SE204,”Research on extracting software feature models using transfer learning”.
文摘In view of the low interpretability of existing collaborative filtering recommendation algorithms and the difficulty of extracting information from content-based recommendation algorithms,we propose an efficient KGRS model.KGRS first obtains reasoning paths of knowledge graph and embeds the entities of paths into vectors based on knowledge representation learning TransD algorithm,then uses LSTM and soft attention mechanism to capture the semantic of each path reasoning,then uses convolution operation and pooling operation to distinguish the importance of different paths reasoning.Finally,through the full connection layer and sigmoid function to get the prediction ratings,and the items are sorted according to the prediction ratings to get the user’s recommendation list.KGRS is tested on the movielens-100k dataset.Compared with the related representative algorithm,including the state-of-the-art interpretable recommendation models RKGE and RippleNet,the experimental results show that KGRS has good recommendation interpretation and higher recommendation accuracy.
基金supported by the Natural Science Foundation of China (62372277)the Natural Science Foundation of Shandong Province (ZR2022MF257, ZR2022MF295)Humanities and Social Sciences Fund of the Ministry of Education (21YJC630157)。
文摘With the popularity of online learning in educational settings, knowledge tracing(KT) plays an increasingly significant role. The task of KT is to help students learn more effectively by predicting their next mastery of knowledge based on their historical exercise sequences. Nowadays, many related works have emerged in this field, such as Bayesian knowledge tracing and deep knowledge tracing methods. Despite the progress that has been made in KT, existing techniques still have the following limitations: 1) Previous studies address KT by only exploring the observational sparsity data distribution, and the counterfactual data distribution has been largely ignored. 2) Current works designed for KT only consider either the entity relationships between questions and concepts, or the relations between two concepts, and none of them investigates the relations among students, questions, and concepts, simultaneously, leading to inaccurate student modeling. To address the above limitations,we propose a graph counterfactual augmentation method for knowledge tracing. Concretely, to consider the multiple relationships among different entities, we first uniform students, questions, and concepts in graphs, and then leverage a heterogeneous graph convolutional network to conduct representation learning.To model the counterfactual world, we conduct counterfactual transformations on students’ learning graphs by changing the corresponding treatments and then exploit the counterfactual outcomes in a contrastive learning framework. We conduct extensive experiments on three real-world datasets, and the experimental results demonstrate the superiority of our proposed Graph CA method compared with several state-of-the-art baselines.
基金support by the National Natural Science Foundation of China(NSFC)under grant number 61873274.
文摘Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.
基金supported by the Science and Technology Program Project(No.2020A02001-1)of Xinjiang Autonomous Region,China.
文摘Smart contracts have led to more efficient development in finance and healthcare,but vulnerabilities in contracts pose high risks to their future applications.The current vulnerability detection methods for contracts are either based on fixed expert rules,which are inefficient,or rely on simplistic deep learning techniques that do not fully leverage contract semantic information.Therefore,there is ample room for improvement in terms of detection precision.To solve these problems,this paper proposes a vulnerability detector based on deep learning techniques,graph representation,and Transformer,called GRATDet.The method first performs swapping,insertion,and symbolization operations for contract functions,increasing the amount of small sample data.Each line of code is then treated as a basic semantic element,and information such as control and data relationships is extracted to construct a new representation in the form of a Line Graph(LG),which shows more structural features that differ from the serialized presentation of the contract.Finally,the node information and edge information of the graph are jointly learned using an improved Transformer-GP model to extract information globally and locally,and the fused features are used for vulnerability detection.The effectiveness of the method in reentrancy vulnerability detection is verified in experiments,where the F1 score reaches 95.16%,exceeding stateof-the-art methods.
文摘The proliferation of internet traffic encryption has become a double-edged sword. While it significantly enhances user privacy, it also inadvertently shields cyber-attacks from detection, presenting a formidable challenge to cybersecurity. Traditional machine learning and deep learning techniques often fall short in identifying encrypted malicious traffic due to their inability to fully extract and utilize the implicit relational and positional information embedded within data packets. This limitation has led to an unresolved challenge in the cybersecurity community: how to effectively extract valuable insights from the complex patterns of traffic packet transmission. Consequently, this paper introduces the TB-Graph model, an encrypted malicious traffic classification model based on a relational graph attention network. The model is a heterogeneous traffic burst graph that embeds side-channel features, which are unaffected by encryption, into the graph nodes and connects them with three different types of burst edges. Subsequently, we design a relational positional coding that prevents the loss of temporal relationships between the original traffic flows during graph transformation. Ultimately, TB-Graph leverages the powerful graph representation learning capabilities of Relational Graph Attention Network (RGAT) to extract latent behavioral features from the burst graph nodes and edge relationships. Experimental results show that TB-Graph outperforms various state-of-the-art methods in fine-grained encrypted malicious traffic classification tasks on two public datasets, indicating its enhanced capability for identifying encrypted malicious traffic.
基金supported by the“pioneer”and“Leading Goose”Key R&D Program of Zhejiang Province under Grant No.2022C03106the Zhejiang Provincial Natural Science Foundation of China under Grant No.LY23F020010the National Natural Science Foundation of China under Grant No.62077015.
文摘Entity alignment,which aims to identify entities with the same meaning in different Knowledge Graphs(KGs),is a key step in knowledge integration.Despite the promising results achieved by existing methods,they often fail to fully leverage the structure information of KGs for entity alignment.Therefore,our goal is to thoroughly explore the features of entity neighbors and relationships to obtain better entity embeddings.In this work,we propose DCEA,an effective dual-context representation learning framework for entity alignment.Specifically,the neighbor-level embedding module introduces relation information to more accurately aggregate neighbor context.The relation-level embedding module utilizes neighbor context to enhance relation-level embeddings.To eliminate semantic gaps between neighbor-level and relation-level embeddings,and fully exploit their complementarity,we design a hybrid embedding fusion model that adaptively performs embedding fusion to obtain powerful joint entity embeddings.We also jointly optimize the contrastive loss of multi-level embeddings,enhancing their mutual reinforcement while preserving the characteristics of neighbor and relation embeddings.Additionally,the decision fusion module combines the similarity scores calculated between entities based on embeddings at different levels to make the final alignment decision.Extensive experimental results on public datasets indicate that our DCEA performs better than state-of-the-art baselines.
基金supported by the MSIT(Ministry of Science and ICT),Republic of Korea,under the Convergence Security Core Talent Training Business Support Program(IITP-2025-RS-2023-00266605,50%)in part by the Institute of Information&Communications Technology Planning&Evaluation(lITP)grant funded by the Korea government(MSIT)(RS-2025-02305436,Development of Digital Innovative Element Technologies for Rapid Prediction of Potential Complex Disasters and Continuous Disaster Prevention,30%)supported by the Chung-Ang University Graduate Research Scholar-ship in 2023(20%).
文摘The escalating complexity and heterogeneity of modern energy systems—particularly in smart grid and distributed energy infrastructures—has intensified the need for intelligent and scalable security vulnerability classification.To address this challenge,we propose Vulnerability2Vec,a graph-embedding-based framework designed to enhance the automated classification of security vulnerabilities that threaten energy system resilience.Vulnerability2Vec converts Common Vulnerabilities and Exposures(CVE)text explanations to semantic graphs,where nodes represent CVE IDs and key terms(nouns,verbs,and adjectives),and edges capture co-occurrence relationships.Then,it embeds the semantic graphs to a low-dimensional vector space with random-walk sampling and skip-gram with negative sampling.It is possible to identify the latent relationships and structural patterns that traditional sparse vector methods fail to capture.Experimental results demonstrate a classification accuracy of up to 80%,significantly outperforming baseline methods.This approach offers a theoretical basis for classifying vulnerability types as structured semantic patterns in complex software systems.The proposed method models the semantic structure of vulnerabilities,providing a theoretical foundation for their classification.
基金supported by the National Natural Science Foundation of China(grant numbers 62267005 and 42365008)the Guangxi Collaborative Innovation Center of Multi-Source Information Integration and Intelligent Processing.
文摘With the development of the Semantic Web,the number of ontologies grows exponentially and the semantic relationships between ontologies become more and more complex,understanding the true semantics of specific terms or concepts in an ontology is crucial for the matching task.At present,the main challenges facing ontology matching tasks based on representation learning methods are how to improve the embedding quality of ontology knowledge and how to integrate multiple features of ontology efficiently.Therefore,we propose an Ontology Matching Method Based on the Gated Graph Attention Model(OM-GGAT).Firstly,the semantic knowledge related to concepts in the ontology is encoded into vectors using the OWL2Vec^(*)method,and the relevant path information from the root node to the concept is embedded to understand better the true meaning of the concept itself and the relationship between concepts.Secondly,the ontology is transformed into the corresponding graph structure according to the semantic relation.Then,when extracting the features of the ontology graph nodes,different attention weights are assigned to each adjacent node of the central concept with the help of the attention mechanism idea.Finally,gated networks are designed to further fuse semantic and structural embedding representations efficiently.To verify the effectiveness of the proposed method,comparative experiments on matching tasks were carried out on public datasets.The results show that the OM-GGAT model can effectively improve the efficiency of ontology matching.
基金supported by the National Science and Technology Council(NSTC),Taiwan,under Grants Numbers 112-2622-E-029-009 and 112-2221-E-029-019.
文摘In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.
文摘Various graph representation learning models convert graph nodes into vectors using techniques like matrix factorization,random walk,and deep learning.However,choosing the right method for different tasks can be challenging.Communities within networks help reveal underlying structures and correlations.Investigating how different models preserve community properties is crucial for identifying the best graph representation for data analysis.This paper defines indicators to explore the perceptual quality of community properties in representation learning spaces,including the consistency of community structure,node distribution within and between communities,and central node distribution.A visualization system presents these indicators,allowing users to evaluate models based on community structures.Case studies demonstrate the effectiveness of the indicators for the visual evaluation of graph representation learning models.
基金supported in part by the Zhejiang Provincial Natural Science Foundation of China(LDT23F01012F01 and LDT23F01015F01)in part by the Fundamental Research Funds for the Provincial Universities of Zhejiang Grant GK229909299001-008the National Natural Science Foundation of China(62372146 and 61806061).
文摘Graph contrastive learning(GCL)has attracted extensive research interest due to its powerful ability to capture latent structural and semantic information of graphs in a self-supervised manner.Existing GCL methods commonly adopt predefined graph augmentations to generate two contrastive views.Subsequently,they design a contrastive pretext task between these views with the goal of maximizing their agreement.These methods as-sume the augmented graph can fully preserve the semantics of the original.However,typical data augmentation strategies in GCL,such as random edge dropping,may alter the properties of the original graph.As a result,previous GCL methods overlooked graph differences,potentially leading to difficulty distinguishing between graphs that are structurally similar but semantically different.Therefore,we argue that it is necessary to design a method that can quantify the dissimilarity between the original and augmented graphs to more accurately capture the relationships between samples.In this work,we propose a novel graph contrastive learning framework,named Accurate Difference-based Node-Level Graph Contrastive Learning(DNGCL),which helps the model distinguish similar graphs with slight differences by learning node-level differences between graphs.Specifically,we train the model to distinguish between original and augmented nodes via a node discriminator and employ cosine dissimilarity to accurately measure the difference between each node.Furthermore,we employ multiple types of data augmentation commonly used in current GCL methods on the original graph,aiming to learn the differences between nodes under different augmentation strategies and help the model learn richer local information.We conduct extensive experiments on six benchmark datasets and the results show that our DNGCL outperforms most state-of-the-art baselines,which strongly validates the effectiveness of our model.
