The majority of existing graph-network-based few-shot models focus on a node-similarity update mode.The lack of adequate information intensies the risk of overtraining.In this paper,we propose a novel Multihead Attent...The majority of existing graph-network-based few-shot models focus on a node-similarity update mode.The lack of adequate information intensies the risk of overtraining.In this paper,we propose a novel Multihead Attention Graph Network to excavate discriminative relation and fulll effective information propagation.For edge update,the node-level attention is used to evaluate the similarities between the two nodes and the distributionlevel attention extracts more in-deep global relation.The cooperation between those two parts provides a discriminative and comprehensive expression for edge feature.For node update,we embrace the label-level attention to soften the noise of irrelevant nodes and optimize the update direction.Our proposed model is veried through extensive experiments on two few-shot benchmark MiniImageNet and CIFAR-FS dataset.The results suggest that our method has a strong capability of noise immunity and quick convergence.The classication accuracy outperforms most state-of-the-art approaches.展开更多
1 Introduction Molecular Property Prediction aims to identify molecules sharing similar efficacious properties[1],which is a foundational task in drug discovery,materials science and bioinformatics.Graph neural networ...1 Introduction Molecular Property Prediction aims to identify molecules sharing similar efficacious properties[1],which is a foundational task in drug discovery,materials science and bioinformatics.Graph neural networks(GNNs)have shown significant success in this field.However,GNN-based methods often face label scarcity,limiting their performance in predicting molecular properties.Besides,GNNs trained on specific datasets frequently struggle with generalization due to domain shift[2].展开更多
Fine-tuning is a popular approach to solve the few-shot object detection problem.In this paper,we attempt to introduce a new perspective on it.We formulate the few-shot novel tasks as a type of distribution shifted fr...Fine-tuning is a popular approach to solve the few-shot object detection problem.In this paper,we attempt to introduce a new perspective on it.We formulate the few-shot novel tasks as a type of distribution shifted from its ground-truth distribution.We introduce the concept of imaginary placeholder masks to show that this distribution shift is essentially a composite of in-distribution(ID)and out-of-distribution(OOD)shifts.Our empirical investigation results show that it is significant to balance the trade-off between adapting to the available few-shot distribution and keeping the distribution-shift robustness of the pre-trained model.We explore improvements in the few-shot finetuning transfer in the few-shot object detection(FSOD)settings from three aspects.First,we explore the LinearProbe-Finetuning(LP-FT)technique to balance this trade-off to mitigate the feature distortion problem.Second,we explore the effectiveness of utilizing the protection freezing strategy for querybased object detectors to keep their OOD robustness.Third,we try to utilize ensembling methods to circumvent the feature distortion.All these techniques are integrated into a whole method called BIOT(Balanced ID-OOD Transfer).Evaluation results show that our method is simple yet effective and general to tap the FSOD potential of query-based object detectors.It outperforms the current SOTA method in many FSOD settings and has a promising scaling capability.展开更多
Taking the real part and the imaginary part of complex sound pressure of the sound field as features,a transfer learning model is constructed.Based on the pre-training of a large amount of underwater acoustic data in ...Taking the real part and the imaginary part of complex sound pressure of the sound field as features,a transfer learning model is constructed.Based on the pre-training of a large amount of underwater acoustic data in the preselected sea area using the convolutional neural network(CNN),the few-shot underwater acoustic data in the test sea area are retrained to study the underwater sound source ranging problem.The S5 voyage data of SWellEX-96 experiment is used to verify the proposed method,realize the range estimation for the shallow source in the experiment,and compare the range estimation performance of the underwater target sound source of four methods:matched field processing(MFP),generalized regression neural network(GRNN),traditional CNN,and transfer learning.Experimental data processing results show that the transfer learning model based on residual CNN can effectively realize range estimation in few-shot scenes,and the estimation performance is remarkably better than that of other methods.展开更多
Now object detection based on deep learning tries different strategies.It uses fewer data training networks to achieve the effect of large dataset training.However,the existing methods usually do not achieve the balan...Now object detection based on deep learning tries different strategies.It uses fewer data training networks to achieve the effect of large dataset training.However,the existing methods usually do not achieve the balance between network parameters and training data.It makes the information provided by a small amount of picture data insufficient to optimize model parameters,resulting in unsatisfactory detection results.To improve the accuracy of few shot object detection,this paper proposes a network based on the transformer and high-resolution feature extraction(THR).High-resolution feature extractionmaintains the resolution representation of the image.Channels and spatial attention are used to make the network focus on features that are more useful to the object.In addition,the recently popular transformer is used to fuse the features of the existing object.This compensates for the previous network failure by making full use of existing object features.Experiments on the Pascal VOC and MS-COCO datasets prove that the THR network has achieved better results than previous mainstream few shot object detection.展开更多
针对利用海量数据构建分类模型时训练数据规模大、训练时间长且碳排放量大的问题,提出面向低能耗高性能的分类器两阶段数据选择方法TSDS(Two-Stage Data Selection)。首先,通过修正余弦相似度确定聚类中心,并将样本数据进行基于不相似...针对利用海量数据构建分类模型时训练数据规模大、训练时间长且碳排放量大的问题,提出面向低能耗高性能的分类器两阶段数据选择方法TSDS(Two-Stage Data Selection)。首先,通过修正余弦相似度确定聚类中心,并将样本数据进行基于不相似点的分裂层次聚类;其次,对聚类结果按数据分布自适应抽样以组成高质量的子样本集;最后,利用子样本集在分类模型上训练,在加速训练过程的同时提升模型精度。在Spambase、Bupa和Phoneme等6个数据集上构建支持向量机(SVM)和多层感知机(MLP)分类模型,验证TSDS的性能。实验结果表明在样本数据压缩比达到85.00%的情况下,TSDS能将分类模型准确率提升3~10个百分点,同时加速模型训练,使训练SVM分类器的能耗平均降低93.76%,训练MLP分类器的能耗平均降低75.41%。可见,TSDS在大数据场景的分类任务上既能缩短训练时间和减少能耗,又能提升分类器性能,从而助力实现“双碳”目标。展开更多
基金supported in part by the Natural Science Foundation of China under Grant 61972169 and U1536203in part by the National key research and developm program of China(2016QY01W0200)in part by the Major Scientic and Technological Project of Hubei Province(2018AAA068 and 2019AAA051).
文摘The majority of existing graph-network-based few-shot models focus on a node-similarity update mode.The lack of adequate information intensies the risk of overtraining.In this paper,we propose a novel Multihead Attention Graph Network to excavate discriminative relation and fulll effective information propagation.For edge update,the node-level attention is used to evaluate the similarities between the two nodes and the distributionlevel attention extracts more in-deep global relation.The cooperation between those two parts provides a discriminative and comprehensive expression for edge feature.For node update,we embrace the label-level attention to soften the noise of irrelevant nodes and optimize the update direction.Our proposed model is veried through extensive experiments on two few-shot benchmark MiniImageNet and CIFAR-FS dataset.The results suggest that our method has a strong capability of noise immunity and quick convergence.The classication accuracy outperforms most state-of-the-art approaches.
基金sponsored in part by the National Key Research and Development Program of China(No.2023YFB3307500)the Science and Technology Innovation Project of Hunan Province(No.2023RC4014)the National Natural Science Foundation of China(NSFC)(Grant Nos.62076146,62021002,U20A6003,6212780016).
文摘1 Introduction Molecular Property Prediction aims to identify molecules sharing similar efficacious properties[1],which is a foundational task in drug discovery,materials science and bioinformatics.Graph neural networks(GNNs)have shown significant success in this field.However,GNN-based methods often face label scarcity,limiting their performance in predicting molecular properties.Besides,GNNs trained on specific datasets frequently struggle with generalization due to domain shift[2].
文摘Fine-tuning is a popular approach to solve the few-shot object detection problem.In this paper,we attempt to introduce a new perspective on it.We formulate the few-shot novel tasks as a type of distribution shifted from its ground-truth distribution.We introduce the concept of imaginary placeholder masks to show that this distribution shift is essentially a composite of in-distribution(ID)and out-of-distribution(OOD)shifts.Our empirical investigation results show that it is significant to balance the trade-off between adapting to the available few-shot distribution and keeping the distribution-shift robustness of the pre-trained model.We explore improvements in the few-shot finetuning transfer in the few-shot object detection(FSOD)settings from three aspects.First,we explore the LinearProbe-Finetuning(LP-FT)technique to balance this trade-off to mitigate the feature distortion problem.Second,we explore the effectiveness of utilizing the protection freezing strategy for querybased object detectors to keep their OOD robustness.Third,we try to utilize ensembling methods to circumvent the feature distortion.All these techniques are integrated into a whole method called BIOT(Balanced ID-OOD Transfer).Evaluation results show that our method is simple yet effective and general to tap the FSOD potential of query-based object detectors.It outperforms the current SOTA method in many FSOD settings and has a promising scaling capability.
基金supported by the National Natural Science Foundation of China(1197428611904274)+1 种基金the Shaanxi Young Science and Technology Star Program(2021KJXX-07)the fundamental research funding for characteristic disciplines(G2022WD0235)。
文摘Taking the real part and the imaginary part of complex sound pressure of the sound field as features,a transfer learning model is constructed.Based on the pre-training of a large amount of underwater acoustic data in the preselected sea area using the convolutional neural network(CNN),the few-shot underwater acoustic data in the test sea area are retrained to study the underwater sound source ranging problem.The S5 voyage data of SWellEX-96 experiment is used to verify the proposed method,realize the range estimation for the shallow source in the experiment,and compare the range estimation performance of the underwater target sound source of four methods:matched field processing(MFP),generalized regression neural network(GRNN),traditional CNN,and transfer learning.Experimental data processing results show that the transfer learning model based on residual CNN can effectively realize range estimation in few-shot scenes,and the estimation performance is remarkably better than that of other methods.
基金the National Natural Science Foundation of China under grant 62172059 and 62072055Hunan Provincial Natural Science Foundations of China under Grant 2020JJ4626+2 种基金Scientific Research Fund of Hunan Provincial Education Department of China under Grant 19B004“Double First-class”International Cooperation and Development Scientific Research Project of Changsha University of Science and Technology under Grant 2018IC25the Young Teacher Growth Plan Project of Changsha University of Science and Technology under Grant 2019QJCZ076.
文摘Now object detection based on deep learning tries different strategies.It uses fewer data training networks to achieve the effect of large dataset training.However,the existing methods usually do not achieve the balance between network parameters and training data.It makes the information provided by a small amount of picture data insufficient to optimize model parameters,resulting in unsatisfactory detection results.To improve the accuracy of few shot object detection,this paper proposes a network based on the transformer and high-resolution feature extraction(THR).High-resolution feature extractionmaintains the resolution representation of the image.Channels and spatial attention are used to make the network focus on features that are more useful to the object.In addition,the recently popular transformer is used to fuse the features of the existing object.This compensates for the previous network failure by making full use of existing object features.Experiments on the Pascal VOC and MS-COCO datasets prove that the THR network has achieved better results than previous mainstream few shot object detection.
文摘针对利用海量数据构建分类模型时训练数据规模大、训练时间长且碳排放量大的问题,提出面向低能耗高性能的分类器两阶段数据选择方法TSDS(Two-Stage Data Selection)。首先,通过修正余弦相似度确定聚类中心,并将样本数据进行基于不相似点的分裂层次聚类;其次,对聚类结果按数据分布自适应抽样以组成高质量的子样本集;最后,利用子样本集在分类模型上训练,在加速训练过程的同时提升模型精度。在Spambase、Bupa和Phoneme等6个数据集上构建支持向量机(SVM)和多层感知机(MLP)分类模型,验证TSDS的性能。实验结果表明在样本数据压缩比达到85.00%的情况下,TSDS能将分类模型准确率提升3~10个百分点,同时加速模型训练,使训练SVM分类器的能耗平均降低93.76%,训练MLP分类器的能耗平均降低75.41%。可见,TSDS在大数据场景的分类任务上既能缩短训练时间和减少能耗,又能提升分类器性能,从而助力实现“双碳”目标。
