Objective To address the dual challenges of long-tail distribution and feature sparsity in traditional Chinese medicine(TCM)syndrome differentiation within real clinical settings,we propose a data-efficient learning f...Objective To address the dual challenges of long-tail distribution and feature sparsity in traditional Chinese medicine(TCM)syndrome differentiation within real clinical settings,we propose a data-efficient learning framework enhanced by knowledge graphs.Methods We developed Agent-GNN,a three-stage decoupled learning framework,and validated it on the Traditional Chinese Medicine Syndrome Diagnosis(TCM-SD)dataset containing 54152 clinical records across 148 syndrome categories.First,we constructed a comprehensive medical knowledge graph encoding the complete TCM reasoning system.Second,we proposed a Functional Patient Profiling(FPP)method that utilizes large language models(LLMs)combined with Graph Retrieval-Augmented Generation(RAG)to extract structured symptom-etiology-pathogenesis subgraphs from medical records.Third,we employed heterogeneous graph neural networks to learn structured combination patterns explicitly.We compared our method against multiple baselines including BERT,ZY-BERT,ZY-BERT+Know,GAT,and GPT-4 Few-shot,using macro-F1 score as the primary evaluation metric.Additionally,ablation experiments were conducted to validate the contribution of each key component to model performance.Results Agent-GNN achieved an overall macro-F1 score of 72.4%,representing an 8.7 percentage points improvement over ZY-BERT+Know(63.7%),the strongest baseline among traditional methods.For long-tail syndromes with fewer than 10 samples,Agent-GNN reached a macro-F1 score of 58.6%,compared with 39.3%for ZY-BERT+Know and 41.2%for GPT-4 Few-shot,representing relative improvements of 49.2%and 42.2%,respectively.Ablation experiments confirmed that the explicit modeling of etiology-pathogenesis nodes contributed 12.4 percentage points to this enhanced long-tail syndrome performance.Conclusion This study proposes Agent-GNN,a knowledge graph-enhanced framework that effectively addresses the long-tail distribution challenge in TCM syndrome differentiation.By explicitly modeling manifestation-mechanism-essence patterns through structured knowledge graphs,our approach achieves superior performance in data-scarce scenarios while providing interpretable reasoning paths for TCM intelligent diagnosis.展开更多
Continual learning fault diagnosis(CLFD)has gained growing interest in mechanical systems for its ability to accumulate and transfer knowledge in dynamic fault diagnosis scenarios.However,existing CLFD methods typical...Continual learning fault diagnosis(CLFD)has gained growing interest in mechanical systems for its ability to accumulate and transfer knowledge in dynamic fault diagnosis scenarios.However,existing CLFD methods typically assume balanced task distributions,neglecting the long-tailed nature of real-world fault occurrences,where certain faults dominate while others are rare.Due to the long-tailed distribution among different me-chanical conditions,excessive attention has been focused on the dominant type,leading to performance de-gradation in rarer types.In this paper,decoupling incremental classifier and representation learning(DICRL)is proposed to address the dual challenges of catastrophic forgetting introduced by incremental tasks and the bias in long-tailed CLFD(LT-CLFD).The core innovation lies in the structural decoupling of incremental classifier learning and representation learning.An instance-balanced sampling strategy is employed to learn more dis-criminative deep representations from the exemplars selected by the herding algorithm and new data.Then,the previous classifiers are frozen to prevent damage to representation learning during backward propagation.Cosine normalization classifier with learnable weight scaling is trained using a class-balanced sampling strategy to enhance classification accuracy.Experimental results demonstrate that DICRL outperforms existing continual learning methods across multiple benchmarks,demonstrating superior performance and robustness in both LT-CLFD and conventional CLFD.DICRL effectively tackles both catastrophic forgetting and long-tailed distribution in CLFD,enabling more reliable fault diagnosis in industrial applications.展开更多
Real-world data always exhibit an imbalanced and long-tailed distribution,which leads to poor performance for neural network-based classification.Existing methods mainly tackle this problem by reweighting the loss fun...Real-world data always exhibit an imbalanced and long-tailed distribution,which leads to poor performance for neural network-based classification.Existing methods mainly tackle this problem by reweighting the loss function or rebalancing the classifier.However,one crucial aspect overlooked by previous research studies is the imbalanced feature space problem caused by the imbalanced angle distribution.In this paper,the authors shed light on the significance of the angle distribution in achieving a balanced feature space,which is essential for improving model performance under long-tailed distributions.Nevertheless,it is challenging to effectively balance both the classifier norms and angle distribution due to problems such as the low feature norm.To tackle these challenges,the authors first thoroughly analyse the classifier and feature space by decoupling the classification logits into three key components:classifier norm(i.e.the magnitude of the classifier vector),feature norm(i.e.the magnitude of the feature vector),and cosine similarity between the classifier vector and feature vector.In this way,the authors analyse the change of each component in the training process and reveal three critical problems that should be solved,that is,the imbalanced angle distribution,the lack of feature discrimination,and the low feature norm.Drawing from this analysis,the authors propose a novel loss function that incorporates hyperspherical uniformity,additive angular margin,and feature norm regularisation.Each component of the loss function addresses a specific problem and synergistically contributes to achieving a balanced classifier and feature space.The authors conduct extensive experiments on three popular benchmark datasets including CIFAR-10/100-LT,ImageNet-LT,and iNaturalist 2018.The experimental results demonstrate that the authors’loss function outperforms several previous state-of-the-art methods in addressing the challenges posed by imbalanced and longtailed datasets,that is,by improving upon the best-performing baselines on CIFAR-100-LT by 1.34,1.41,1.41 and 1.33,respectively.展开更多
Brooding is a major breeding investment of parental birds during the early nestling stage, and has important effects on the development and survival of nestlings. Investigating brooding behavior can help to understand...Brooding is a major breeding investment of parental birds during the early nestling stage, and has important effects on the development and survival of nestlings. Investigating brooding behavior can help to understand avian breeding investment strategies. From January to June in 2013 and 2014, we studied the brooding behaviors of long-tailed tits （Aegithalos caudatus glaucogularis） in Dongzhai National Nature Reserve, Henan Province, China. We analyzed the relationships between parental diurnal brooding duration and nestling age, brood size, temperature, relative breeding season, time of day and nestling frequencies during brooding duration. Results showed that female and male long-tailed tit parents had different breeding investment strategies during the early nestling stage. Female parents bore most of the brooding investment, while male parents performed most of the nestling feedings. In addition, helpers were not found to brood nestlings at the two cooperative breeding nests. Parental brooding duration was significantly associated with the food delivered to nestlings （F=86.10, dr=l, 193.94, P〈0.001）, and was longer when the nestlings received more food. We found that parental brooding duration declined significantly as nestlings aged （F=5.99, dr=-1, 50.13, P=0.018）. When nestlings were six days old, daytime parental brooding almost ceased, implying that long- tailed tit nestlings might be able to maintain their own body temperature by this age. In addition, brooding duration was affected by both brood size （F=12.74, dr=-1,32.08, P=0.001） and temperature （F=5.83, df=-l, 39.59, P=-0.021）, with it being shorter in larger broods and when ambient temperature was higher.展开更多
Jinfengopteryx is a newly uncovered Archaeopteryx-like avialan bird outside Germany, which was found from the Jehol Biota of northern Hebei in northeastern China. It shares many characters only with Archaeopteryx by t...Jinfengopteryx is a newly uncovered Archaeopteryx-like avialan bird outside Germany, which was found from the Jehol Biota of northern Hebei in northeastern China. It shares many characters only with Archaeopteryx by the possession of three fenestrae in the antorbital cavity, 23 caudal vertebrae and long tail feathers attached to all the caudal vertebrae. But the former differs from the latter in the relatively short and high preorbital region of skull, more and closely packed teeth, much shorter forelimb compared to hindlimb. Such differences indicate Jinfengopteryx is even slightly more primitive than Archaeopteryx, although both birds can be placed at the root position of the avialan tree based on cladistic analysis. Shenzhouraptor is suggested to be slightly more advanced than Jinfengopteryx ＋ Archaeopteryx, supported by some derived features in teeth, shoulder girdles and forelimbs such as the reduction of tooth number, dorsolaterally directed glenoid facet, very long forelimb and comparatively short manus. Meanwhile, the tail of Shenzhouraptor shows more primitive characters than those of Jinfengopteryx and Archaeopteryx, e.g., the strikingly longer tail composed of more caudal vertebrae and the long tail feathers attached only to distal caudal segments. The mixed primitive and advanced characters reveal the evident mosaic evolution among long-tailed avialan birds.展开更多
Objective To explore the semi-supervised learning(SSL) algorithm for long-tail endoscopic image classification with limited annotations.Method We explored semi-supervised long-tail endoscopic image classification in H...Objective To explore the semi-supervised learning(SSL) algorithm for long-tail endoscopic image classification with limited annotations.Method We explored semi-supervised long-tail endoscopic image classification in HyperKvasir,the largest gastrointestinal public dataset with 23 diverse classes.Semi-supervised learning algorithm FixMatch was applied based on consistency regularization and pseudo-labeling.After splitting the training dataset and the test dataset at a ratio of 4:1,we sampled 20%,50%,and 100% labeled training data to test the classification with limited annotations.Results The classification performance was evaluated by micro-average and macro-average evaluation metrics,with the Mathews correlation coefficient(MCC) as the overall evaluation.SSL algorithm improved the classification performance,with MCC increasing from 0.8761 to 0.8850,from 0.8983 to 0.8994,and from 0.9075 to 0.9095 with 20%,50%,and 100% ratio of labeled training data,respectively.With a 20% ratio of labeled training data,SSL improved both the micro-average and macro-average classification performance;while for the ratio of 50% and 100%,SSL improved the micro-average performance but hurt macro-average performance.Through analyzing the confusion matrix and labeling bias in each class,we found that the pseudo-based SSL algorithm exacerbated the classifier’ s preference for the head class,resulting in improved performance in the head class and degenerated performance in the tail class.Conclusion SSL can improve the classification performance for semi-supervised long-tail endoscopic image classification,especially when the labeled data is extremely limited,which may benefit the building of assisted diagnosis systems for low-volume hospitals.However,the pseudo-labeling strategy may amplify the effect of class imbalance,which hurts the classification performance for the tail class.展开更多
Leveraging deep learning-based techniques to classify diseases has attracted extensive research interest in recent years.Nevertheless,most of the current studies only consider single-modal medical images,and the numbe...Leveraging deep learning-based techniques to classify diseases has attracted extensive research interest in recent years.Nevertheless,most of the current studies only consider single-modal medical images,and the number of ophthalmic diseases that can be classified is relatively small.Moreover,imbalanced data distribution of different ophthalmic diseases is not taken into consideration,which limits the application of deep learning techniques in realistic clinical scenes.In this paper,we propose a Multimodal Multi-disease Long-tailed Classification Network(M^(2)LC-Net)in response to the challenges mentioned above.M^(2)LC-Net leverages ResNet18-CBAM to extract features from fundus images and Optical Coherence Tomography(OCT)images,respectively,and conduct feature fusion to classify 11 common ophthalmic diseases.Moreover,Class Activation Mapping(CAM)is employed to visualize each mode to improve interpretability of M^(2)LC-Net.We conduct comprehensive experiments on realistic dataset collected from a Grade III Level A ophthalmology hospital in China,including 34,396 images of 11 disease labels.Experimental results demonstrate effectiveness of our proposed model M^(2)LC-Net.Compared with the stateof-the-art,various performance metrics have been improved significantly.Specifically,Cohen’s kappa coefficient κ has been improved by 3.21%,which is a remarkable improvement.展开更多
With the rapid increase of large-scale problems, the distribution of real-world datasets tends to be long-tailed. Existing solutions typically involve re-balancing strategies (i.e., re-sampling and re-weighting). Alth...With the rapid increase of large-scale problems, the distribution of real-world datasets tends to be long-tailed. Existing solutions typically involve re-balancing strategies (i.e., re-sampling and re-weighting). Although they can significantly promote the classifier learning of deep networks, they will unexpectedly impair the representative ability of the learned deep features to a certain extent. Therefore, this paper proposes a dual-channel learning algorithm with involution neural networks (DC-Invo) to take care of representation learning and classifier learning concurrently. In this work, the most important thing is to combine ResNet and involution to obtain higher classification accuracy because of involution’s wider coverage in the spatial dimension. The paper conducted extensive experiments on several benchmark vision tasks including Cifar-LT, Imagenet-LT, and Places-LT, showing that DC-Invo is able to achieve significant performance gained on long-tailed datasets.展开更多
The long-tailed data distribution poses an enormous challenge for training neural networks in classification.A classification network can be decoupled into a feature extractor and a classifier.This paper takes a semi-...The long-tailed data distribution poses an enormous challenge for training neural networks in classification.A classification network can be decoupled into a feature extractor and a classifier.This paper takes a semi-discrete optimal transport(OT)perspective to analyze the long-tailed classification problem,where the feature space is viewed as a continuous source domain,and the classifier weights are viewed as a discrete target domain.The classifier is indeed to find a cell decomposition of the feature space with each cell corresponding to one class.An imbalanced training set causes the more frequent classes to have larger volume cells,which means that the classifier's decision boundary is biased towards less frequent classes,resulting in reduced classification performance in the inference phase.Therefore,we propose a novel OTdynamic softmax loss,which dynamically adjusts the decision boundary in the training phase to avoid overfitting in the tail classes.In addition,our method incorporates the supervised contrastive loss so that the feature space can satisfy the uniform distribution condition.Extensive and comprehensive experiments demonstrate that our method achieves state-ofthe-art performance on multiple long-tailed recognition benchmarks,including CIFAR-LT,ImageNet-LT,iNaturalist 2018,and Places-LT.展开更多
Skeleton-based action recognition has recently made significant progress.However,data imbalance is still a great challenge in real-world scenarios.The performance of current action recognition algorithms declines shar...Skeleton-based action recognition has recently made significant progress.However,data imbalance is still a great challenge in real-world scenarios.The performance of current action recognition algorithms declines sharply when training data suffers from heavy class imbalance.The imbalanced data actually degrades the representations learned by these methods and becomes the bottleneck for action recognition.How to learn unbiased representations from imbalanced action data is the key to long-tailed action recognition.In this paper,we propose a novel balanced representation learning method to address the long-tailed problem in action recognition.Firstly,a spatial-temporal action exploration strategy is presented to expand the sample space effectively,generating more valuable samples in a rebalanced manner.Secondly,we design a detached action-aware learning schedule to further mitigate the bias in the representation space.The schedule detaches the representation learning of tail classes from training and proposes an action-aware loss to impose more effective constraints.Additionally,a skip-type representation is proposed to provide complementary structural information.The proposed method is validated on four skeleton datasets,NTU RGB+D 60,NTU RGB+D 120,NW-UCLA and Kinetics.It not only achieves consistently large improvement compared to the state-of-the-art(SOTA)methods,but also demonstrates a superior generalization capacity through extensive experiments.Our code is available at https://github.com/firework8/BRL.展开更多
The collection and annotation of lar ge-scale bird datasets are resource-intensive and time-consuming processes that significantly limit the scalability and accuracy of biodiversity monitoring systems.While self-super...The collection and annotation of lar ge-scale bird datasets are resource-intensive and time-consuming processes that significantly limit the scalability and accuracy of biodiversity monitoring systems.While self-supervised learning(SSL)has emerged as a promising approach for leveraging unannotated data,current SSL methods face two critical challenges in bird species recognition:(1)long-tailed data distributions that result in poor performance on underrepresented species;and(2)domain shift issues caused by data augmentation strategies designed to mitigate class imbalance.Here we present SDNet,a novel SSL-based bird recognition framework that integrates diffusion models with large language models(LLMs)to overcome these limitations.SDNet employs LLMs to generate semantically rich textual descriptions for tail-class species by prompting the models with species taxonomy,morphological attributes,and habitat information,producing detailed natural language priors that capture fine-grained visual characteristics(e.g.,plumage patterns,body proportions,and distinctive markings).These textual descriptions are subsequently used by a conditional diffusion model to synthesize new bird image samples through cross-attention mechanisms that fuse textual embeddings with intermediate visual feature representations during the denoising process,ensuring generated images preserve species-specific morphological details while maintaining photorealistic quality.Additionally,we incorporate a Swin Transformer as the feature extraction backbone whose hierarchical window-based attention mechanism and shifted windowing scheme enable multi-scale local feature extraction that proves particularly effective at capturing finegrained discriminative patterns(such as beak shape and feather texture)while mitigating domain shift between synthetic and original images through consistent feature representations across both data sources.SDNet is validated on both a self-constructed dataset(Bird_BXS)an d a publicly available benchmark(Birds_25),demonstrating substantial improvements over conventional SSL approaches.Our results indicate that the synergistic integration of LLMs,diffusion models,and the Swin Transformer architecture contributes significantly to recognition accuracy,particularly for rare and morphologically similar species.These findings highlight the potential of SDNet for addressing fundamental limitations of existing SSL methods in avian recognition tasks and establishing a new paradigm for efficient self-supervised learning in large-scale ornithological vision applications.展开更多
Intelligent detection and classification of kitchen waste can promote ecological sustainability by replacing inefficient manual processes.However,the presence of non-degradable waste mixed in kitchen waste often follo...Intelligent detection and classification of kitchen waste can promote ecological sustainability by replacing inefficient manual processes.However,the presence of non-degradable waste mixed in kitchen waste often follows a long-tailed distribution,making it challenging to train convolutional neural network-based object detectors,which results in the unsatisfactory detection of tailclass waste.To address this challenge,we propose a class-instance balanced detector(CIB-Det) for intelligent detection and classification of kitchen waste.CIB-Det implements two strategies for the loss function:the class-balanced strategy(CBS)and the instance-balanced strategy(IBS).The CBS focuses more on tail classes,and the IBS concentrates on hard-to-classify instances adaptively during training.Consequently,CIB-Det comprehensively and adaptively addresses the long-tailed issue.Our experiments on a real dataset of kitchen waste images support the effectiveness of CIB-Det for kitchen waste detection.展开更多
Federated learning(FL),a cutting-edge distributed machine learning training paradigm,aims to generate a global model by collaborating on the training of client models without revealing local private data.The co-occurr...Federated learning(FL),a cutting-edge distributed machine learning training paradigm,aims to generate a global model by collaborating on the training of client models without revealing local private data.The co-occurrence of non-independent and identically distributed(non-IID)and long-tailed distribution in FL is one challenge that substantially degrades aggregate performance.In this paper,we present a corresponding solution called federated dual-decoupling via model and logit calibration(FedDDC)for non-IID and long-tailed distributions.The model is characterized by three aspects.First,we decouple the global model into the feature extractor and the classifier to fine-tune the components affected by the joint problem.For the biased feature extractor,we propose a client confidence re-weighting scheme to assist calibration,which assigns optimal weights to each client.For the biased classifier,we apply the classifier re-balancing method for fine-tuning.Then,we calibrate and integrate the client confidence re-weighted logits with the re-balanced logits to obtain the unbiased logits.Finally,we use decoupled knowledge distillation for the first time in the joint problem to enhance the accuracy of the global model by extracting the knowledge of the unbiased model.Numerous experiments demonstrate that on non-IID and long-tailed data in FL,our approach outperforms state-of-the-art methods.展开更多
The prevalence of long-tailed distributions in real-world data often results in classification models favoring the dominant classes,neglecting the less frequent ones.Current approaches address the issues in long-taile...The prevalence of long-tailed distributions in real-world data often results in classification models favoring the dominant classes,neglecting the less frequent ones.Current approaches address the issues in long-tailed image classification by rebalancing data,optimizing weights,and augmenting information.However,these methods often struggle to balance the performance between dominant and minority classes because of inadequate representation learning of the latter.To address these problems,we introduce descriptional words into images as cross-modal privileged information and propose a cross-modal enhanced method for long-tailed image classification,referred to as CMLTNet.CMLTNet improves the learning of intraclass similarity of tail-class representations by cross-modal alignment and captures the difference between the head and tail classes in semantic space by cross-modal inference.After fusing the above information,CMLTNet achieved an overall performance that was better than those of benchmark long-tailed and cross-modal learning methods on the long-tailed cross-modal datasets,NUS-WIDE and VireoFood-172.The effectiveness of the proposed modules was further studied through ablation experiments.In a case study of feature distribution,the proposed model was better in learning representations of tail classes,and in the experiments on model attention,CMLTNet has the potential to help learn some rare concepts in the tail class through mapping to the semantic space.展开更多
基金Sichuan TCM Culture Coordinated Development Research Center Project(2023XT131)National Key Science and Technology Project of China(2023ZD0509405)National Natural Science Foundation of China(82174236).
文摘Objective To address the dual challenges of long-tail distribution and feature sparsity in traditional Chinese medicine(TCM)syndrome differentiation within real clinical settings,we propose a data-efficient learning framework enhanced by knowledge graphs.Methods We developed Agent-GNN,a three-stage decoupled learning framework,and validated it on the Traditional Chinese Medicine Syndrome Diagnosis(TCM-SD)dataset containing 54152 clinical records across 148 syndrome categories.First,we constructed a comprehensive medical knowledge graph encoding the complete TCM reasoning system.Second,we proposed a Functional Patient Profiling(FPP)method that utilizes large language models(LLMs)combined with Graph Retrieval-Augmented Generation(RAG)to extract structured symptom-etiology-pathogenesis subgraphs from medical records.Third,we employed heterogeneous graph neural networks to learn structured combination patterns explicitly.We compared our method against multiple baselines including BERT,ZY-BERT,ZY-BERT+Know,GAT,and GPT-4 Few-shot,using macro-F1 score as the primary evaluation metric.Additionally,ablation experiments were conducted to validate the contribution of each key component to model performance.Results Agent-GNN achieved an overall macro-F1 score of 72.4%,representing an 8.7 percentage points improvement over ZY-BERT+Know(63.7%),the strongest baseline among traditional methods.For long-tail syndromes with fewer than 10 samples,Agent-GNN reached a macro-F1 score of 58.6%,compared with 39.3%for ZY-BERT+Know and 41.2%for GPT-4 Few-shot,representing relative improvements of 49.2%and 42.2%,respectively.Ablation experiments confirmed that the explicit modeling of etiology-pathogenesis nodes contributed 12.4 percentage points to this enhanced long-tail syndrome performance.Conclusion This study proposes Agent-GNN,a knowledge graph-enhanced framework that effectively addresses the long-tail distribution challenge in TCM syndrome differentiation.By explicitly modeling manifestation-mechanism-essence patterns through structured knowledge graphs,our approach achieves superior performance in data-scarce scenarios while providing interpretable reasoning paths for TCM intelligent diagnosis.
基金Supported by National Natural Science Foundation of China(Grant No.52272440)Suzhou Science Foundation(Grant Nos.SYG202323,ZXL2022027).
文摘Continual learning fault diagnosis(CLFD)has gained growing interest in mechanical systems for its ability to accumulate and transfer knowledge in dynamic fault diagnosis scenarios.However,existing CLFD methods typically assume balanced task distributions,neglecting the long-tailed nature of real-world fault occurrences,where certain faults dominate while others are rare.Due to the long-tailed distribution among different me-chanical conditions,excessive attention has been focused on the dominant type,leading to performance de-gradation in rarer types.In this paper,decoupling incremental classifier and representation learning(DICRL)is proposed to address the dual challenges of catastrophic forgetting introduced by incremental tasks and the bias in long-tailed CLFD(LT-CLFD).The core innovation lies in the structural decoupling of incremental classifier learning and representation learning.An instance-balanced sampling strategy is employed to learn more dis-criminative deep representations from the exemplars selected by the herding algorithm and new data.Then,the previous classifiers are frozen to prevent damage to representation learning during backward propagation.Cosine normalization classifier with learnable weight scaling is trained using a class-balanced sampling strategy to enhance classification accuracy.Experimental results demonstrate that DICRL outperforms existing continual learning methods across multiple benchmarks,demonstrating superior performance and robustness in both LT-CLFD and conventional CLFD.DICRL effectively tackles both catastrophic forgetting and long-tailed distribution in CLFD,enabling more reliable fault diagnosis in industrial applications.
基金National Key Research and Development Program of China,Grant/Award Numbers:2022YFB3103900,2023YFB3106504Major Key Project of PCL,Grant/Award Numbers:PCL2022A03,PCL2023A09+5 种基金Shenzhen Basic Research,Grant/Award Number:JCYJ20220531095214031Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies,Grant/Award Number:2022B1212010005Shenzhen International Science and Technology Cooperation Project,Grant/Award Number:GJHZ20220913143008015Natural Science Foundation of Guangdong Province,Grant/Award Number:2023A1515011959Shenzhen-Hong Kong Jointly Funded Project,Grant/Award Number:SGDX20230116091246007Shenzhen Science and Technology Program,Grant/Award Numbers:RCBS20221008093131089,ZDSYS20210623091809029。
文摘Real-world data always exhibit an imbalanced and long-tailed distribution,which leads to poor performance for neural network-based classification.Existing methods mainly tackle this problem by reweighting the loss function or rebalancing the classifier.However,one crucial aspect overlooked by previous research studies is the imbalanced feature space problem caused by the imbalanced angle distribution.In this paper,the authors shed light on the significance of the angle distribution in achieving a balanced feature space,which is essential for improving model performance under long-tailed distributions.Nevertheless,it is challenging to effectively balance both the classifier norms and angle distribution due to problems such as the low feature norm.To tackle these challenges,the authors first thoroughly analyse the classifier and feature space by decoupling the classification logits into three key components:classifier norm(i.e.the magnitude of the classifier vector),feature norm(i.e.the magnitude of the feature vector),and cosine similarity between the classifier vector and feature vector.In this way,the authors analyse the change of each component in the training process and reveal three critical problems that should be solved,that is,the imbalanced angle distribution,the lack of feature discrimination,and the low feature norm.Drawing from this analysis,the authors propose a novel loss function that incorporates hyperspherical uniformity,additive angular margin,and feature norm regularisation.Each component of the loss function addresses a specific problem and synergistically contributes to achieving a balanced classifier and feature space.The authors conduct extensive experiments on three popular benchmark datasets including CIFAR-10/100-LT,ImageNet-LT,and iNaturalist 2018.The experimental results demonstrate that the authors’loss function outperforms several previous state-of-the-art methods in addressing the challenges posed by imbalanced and longtailed datasets,that is,by improving upon the best-performing baselines on CIFAR-100-LT by 1.34,1.41,1.41 and 1.33,respectively.
基金Foundation item： This study was supported by the National Natural Science Foundation of China （31472011）ACKNOWLEDGEMENTS We are grateful to Peng ZHANG, Zheng CHEN, Jia-Hui WANG, and Hui-Jia YUAN of Beijing Normal University for field assistance, and staff from Henan Dongzhai National Nature Reserve for help during field work. We also thank editor for revising the English, and the two reviewers for their constructive comments, which have helped to improve the manuscript.
文摘Brooding is a major breeding investment of parental birds during the early nestling stage, and has important effects on the development and survival of nestlings. Investigating brooding behavior can help to understand avian breeding investment strategies. From January to June in 2013 and 2014, we studied the brooding behaviors of long-tailed tits （Aegithalos caudatus glaucogularis） in Dongzhai National Nature Reserve, Henan Province, China. We analyzed the relationships between parental diurnal brooding duration and nestling age, brood size, temperature, relative breeding season, time of day and nestling frequencies during brooding duration. Results showed that female and male long-tailed tit parents had different breeding investment strategies during the early nestling stage. Female parents bore most of the brooding investment, while male parents performed most of the nestling feedings. In addition, helpers were not found to brood nestlings at the two cooperative breeding nests. Parental brooding duration was significantly associated with the food delivered to nestlings （F=86.10, dr=l, 193.94, P〈0.001）, and was longer when the nestlings received more food. We found that parental brooding duration declined significantly as nestlings aged （F=5.99, dr=-1, 50.13, P=0.018）. When nestlings were six days old, daytime parental brooding almost ceased, implying that long- tailed tit nestlings might be able to maintain their own body temperature by this age. In addition, brooding duration was affected by both brood size （F=12.74, dr=-1,32.08, P=0.001） and temperature （F=5.83, df=-l, 39.59, P=-0.021）, with it being shorter in larger broods and when ambient temperature was higher.
基金financially supported by the National Basic Research Program of China(973 Project,Grant No.2006CB701405)the China Geological Survey,and the National Natural Science Foundation of China(Grant No.40272008).
文摘Jinfengopteryx is a newly uncovered Archaeopteryx-like avialan bird outside Germany, which was found from the Jehol Biota of northern Hebei in northeastern China. It shares many characters only with Archaeopteryx by the possession of three fenestrae in the antorbital cavity, 23 caudal vertebrae and long tail feathers attached to all the caudal vertebrae. But the former differs from the latter in the relatively short and high preorbital region of skull, more and closely packed teeth, much shorter forelimb compared to hindlimb. Such differences indicate Jinfengopteryx is even slightly more primitive than Archaeopteryx, although both birds can be placed at the root position of the avialan tree based on cladistic analysis. Shenzhouraptor is suggested to be slightly more advanced than Jinfengopteryx ＋ Archaeopteryx, supported by some derived features in teeth, shoulder girdles and forelimbs such as the reduction of tooth number, dorsolaterally directed glenoid facet, very long forelimb and comparatively short manus. Meanwhile, the tail of Shenzhouraptor shows more primitive characters than those of Jinfengopteryx and Archaeopteryx, e.g., the strikingly longer tail composed of more caudal vertebrae and the long tail feathers attached only to distal caudal segments. The mixed primitive and advanced characters reveal the evident mosaic evolution among long-tailed avialan birds.
文摘Objective To explore the semi-supervised learning(SSL) algorithm for long-tail endoscopic image classification with limited annotations.Method We explored semi-supervised long-tail endoscopic image classification in HyperKvasir,the largest gastrointestinal public dataset with 23 diverse classes.Semi-supervised learning algorithm FixMatch was applied based on consistency regularization and pseudo-labeling.After splitting the training dataset and the test dataset at a ratio of 4:1,we sampled 20%,50%,and 100% labeled training data to test the classification with limited annotations.Results The classification performance was evaluated by micro-average and macro-average evaluation metrics,with the Mathews correlation coefficient(MCC) as the overall evaluation.SSL algorithm improved the classification performance,with MCC increasing from 0.8761 to 0.8850,from 0.8983 to 0.8994,and from 0.9075 to 0.9095 with 20%,50%,and 100% ratio of labeled training data,respectively.With a 20% ratio of labeled training data,SSL improved both the micro-average and macro-average classification performance;while for the ratio of 50% and 100%,SSL improved the micro-average performance but hurt macro-average performance.Through analyzing the confusion matrix and labeling bias in each class,we found that the pseudo-based SSL algorithm exacerbated the classifier’ s preference for the head class,resulting in improved performance in the head class and degenerated performance in the tail class.Conclusion SSL can improve the classification performance for semi-supervised long-tail endoscopic image classification,especially when the labeled data is extremely limited,which may benefit the building of assisted diagnosis systems for low-volume hospitals.However,the pseudo-labeling strategy may amplify the effect of class imbalance,which hurts the classification performance for the tail class.
基金the National Natural Science Foundation of China(No.62076035)。
文摘Leveraging deep learning-based techniques to classify diseases has attracted extensive research interest in recent years.Nevertheless,most of the current studies only consider single-modal medical images,and the number of ophthalmic diseases that can be classified is relatively small.Moreover,imbalanced data distribution of different ophthalmic diseases is not taken into consideration,which limits the application of deep learning techniques in realistic clinical scenes.In this paper,we propose a Multimodal Multi-disease Long-tailed Classification Network(M^(2)LC-Net)in response to the challenges mentioned above.M^(2)LC-Net leverages ResNet18-CBAM to extract features from fundus images and Optical Coherence Tomography(OCT)images,respectively,and conduct feature fusion to classify 11 common ophthalmic diseases.Moreover,Class Activation Mapping(CAM)is employed to visualize each mode to improve interpretability of M^(2)LC-Net.We conduct comprehensive experiments on realistic dataset collected from a Grade III Level A ophthalmology hospital in China,including 34,396 images of 11 disease labels.Experimental results demonstrate effectiveness of our proposed model M^(2)LC-Net.Compared with the stateof-the-art,various performance metrics have been improved significantly.Specifically,Cohen’s kappa coefficient κ has been improved by 3.21%,which is a remarkable improvement.
文摘With the rapid increase of large-scale problems, the distribution of real-world datasets tends to be long-tailed. Existing solutions typically involve re-balancing strategies (i.e., re-sampling and re-weighting). Although they can significantly promote the classifier learning of deep networks, they will unexpectedly impair the representative ability of the learned deep features to a certain extent. Therefore, this paper proposes a dual-channel learning algorithm with involution neural networks (DC-Invo) to take care of representation learning and classifier learning concurrently. In this work, the most important thing is to combine ResNet and involution to obtain higher classification accuracy because of involution’s wider coverage in the spatial dimension. The paper conducted extensive experiments on several benchmark vision tasks including Cifar-LT, Imagenet-LT, and Places-LT, showing that DC-Invo is able to achieve significant performance gained on long-tailed datasets.
基金supported by the National Key Research and Development Program of China under Grant No.2021YFA1003003the National Natural Science Foundation of China under Grant Nos.61936002 and T2225012.
文摘The long-tailed data distribution poses an enormous challenge for training neural networks in classification.A classification network can be decoupled into a feature extractor and a classifier.This paper takes a semi-discrete optimal transport(OT)perspective to analyze the long-tailed classification problem,where the feature space is viewed as a continuous source domain,and the classifier weights are viewed as a discrete target domain.The classifier is indeed to find a cell decomposition of the feature space with each cell corresponding to one class.An imbalanced training set causes the more frequent classes to have larger volume cells,which means that the classifier's decision boundary is biased towards less frequent classes,resulting in reduced classification performance in the inference phase.Therefore,we propose a novel OTdynamic softmax loss,which dynamically adjusts the decision boundary in the training phase to avoid overfitting in the tail classes.In addition,our method incorporates the supervised contrastive loss so that the feature space can satisfy the uniform distribution condition.Extensive and comprehensive experiments demonstrate that our method achieves state-ofthe-art performance on multiple long-tailed recognition benchmarks,including CIFAR-LT,ImageNet-LT,iNaturalist 2018,and Places-LT.
基金supported by the National Natural Science Foundation of China(Nos.62276263,62006225 and 62071468)the Strategic Priority Research Program of Chinese Academy of Sciences(CAS),China(No.XDA27040700)the National Key Research and Development Program of China(No.2022YFC3310400).
文摘Skeleton-based action recognition has recently made significant progress.However,data imbalance is still a great challenge in real-world scenarios.The performance of current action recognition algorithms declines sharply when training data suffers from heavy class imbalance.The imbalanced data actually degrades the representations learned by these methods and becomes the bottleneck for action recognition.How to learn unbiased representations from imbalanced action data is the key to long-tailed action recognition.In this paper,we propose a novel balanced representation learning method to address the long-tailed problem in action recognition.Firstly,a spatial-temporal action exploration strategy is presented to expand the sample space effectively,generating more valuable samples in a rebalanced manner.Secondly,we design a detached action-aware learning schedule to further mitigate the bias in the representation space.The schedule detaches the representation learning of tail classes from training and proposes an action-aware loss to impose more effective constraints.Additionally,a skip-type representation is proposed to provide complementary structural information.The proposed method is validated on four skeleton datasets,NTU RGB+D 60,NTU RGB+D 120,NW-UCLA and Kinetics.It not only achieves consistently large improvement compared to the state-of-the-art(SOTA)methods,but also demonstrates a superior generalization capacity through extensive experiments.Our code is available at https://github.com/firework8/BRL.
基金supported by the National Natural Science Foundation of China(32471964)。
文摘The collection and annotation of lar ge-scale bird datasets are resource-intensive and time-consuming processes that significantly limit the scalability and accuracy of biodiversity monitoring systems.While self-supervised learning(SSL)has emerged as a promising approach for leveraging unannotated data,current SSL methods face two critical challenges in bird species recognition:(1)long-tailed data distributions that result in poor performance on underrepresented species;and(2)domain shift issues caused by data augmentation strategies designed to mitigate class imbalance.Here we present SDNet,a novel SSL-based bird recognition framework that integrates diffusion models with large language models(LLMs)to overcome these limitations.SDNet employs LLMs to generate semantically rich textual descriptions for tail-class species by prompting the models with species taxonomy,morphological attributes,and habitat information,producing detailed natural language priors that capture fine-grained visual characteristics(e.g.,plumage patterns,body proportions,and distinctive markings).These textual descriptions are subsequently used by a conditional diffusion model to synthesize new bird image samples through cross-attention mechanisms that fuse textual embeddings with intermediate visual feature representations during the denoising process,ensuring generated images preserve species-specific morphological details while maintaining photorealistic quality.Additionally,we incorporate a Swin Transformer as the feature extraction backbone whose hierarchical window-based attention mechanism and shifted windowing scheme enable multi-scale local feature extraction that proves particularly effective at capturing finegrained discriminative patterns(such as beak shape and feather texture)while mitigating domain shift between synthetic and original images through consistent feature representations across both data sources.SDNet is validated on both a self-constructed dataset(Bird_BXS)an d a publicly available benchmark(Birds_25),demonstrating substantial improvements over conventional SSL approaches.Our results indicate that the synergistic integration of LLMs,diffusion models,and the Swin Transformer architecture contributes significantly to recognition accuracy,particularly for rare and morphologically similar species.These findings highlight the potential of SDNet for addressing fundamental limitations of existing SSL methods in avian recognition tasks and establishing a new paradigm for efficient self-supervised learning in large-scale ornithological vision applications.
基金supported by the National Key Research and Development Program of China (Grant No. 2021YFC1910402)。
文摘Intelligent detection and classification of kitchen waste can promote ecological sustainability by replacing inefficient manual processes.However,the presence of non-degradable waste mixed in kitchen waste often follows a long-tailed distribution,making it challenging to train convolutional neural network-based object detectors,which results in the unsatisfactory detection of tailclass waste.To address this challenge,we propose a class-instance balanced detector(CIB-Det) for intelligent detection and classification of kitchen waste.CIB-Det implements two strategies for the loss function:the class-balanced strategy(CBS)and the instance-balanced strategy(IBS).The CBS focuses more on tail classes,and the IBS concentrates on hard-to-classify instances adaptively during training.Consequently,CIB-Det comprehensively and adaptively addresses the long-tailed issue.Our experiments on a real dataset of kitchen waste images support the effectiveness of CIB-Det for kitchen waste detection.
基金supported by the National Natural Science Foundation of China(No.61702321)。
文摘Federated learning(FL),a cutting-edge distributed machine learning training paradigm,aims to generate a global model by collaborating on the training of client models without revealing local private data.The co-occurrence of non-independent and identically distributed(non-IID)and long-tailed distribution in FL is one challenge that substantially degrades aggregate performance.In this paper,we present a corresponding solution called federated dual-decoupling via model and logit calibration(FedDDC)for non-IID and long-tailed distributions.The model is characterized by three aspects.First,we decouple the global model into the feature extractor and the classifier to fine-tune the components affected by the joint problem.For the biased feature extractor,we propose a client confidence re-weighting scheme to assist calibration,which assigns optimal weights to each client.For the biased classifier,we apply the classifier re-balancing method for fine-tuning.Then,we calibrate and integrate the client confidence re-weighted logits with the re-balanced logits to obtain the unbiased logits.Finally,we use decoupled knowledge distillation for the first time in the joint problem to enhance the accuracy of the global model by extracting the knowledge of the unbiased model.Numerous experiments demonstrate that on non-IID and long-tailed data in FL,our approach outperforms state-of-the-art methods.
基金supported in part by the National Natural Science Foundation of China(62006141)the National Key R&D Program of China(2021YFC3300203)+1 种基金the Overseas Innovation Team Project of the“20 Regulations for New Universities”Funding Program of Jinan(2021GXRC073)the Excellent Youth Scholars Program of Shandong Province(2022HWYQ-048).
文摘The prevalence of long-tailed distributions in real-world data often results in classification models favoring the dominant classes,neglecting the less frequent ones.Current approaches address the issues in long-tailed image classification by rebalancing data,optimizing weights,and augmenting information.However,these methods often struggle to balance the performance between dominant and minority classes because of inadequate representation learning of the latter.To address these problems,we introduce descriptional words into images as cross-modal privileged information and propose a cross-modal enhanced method for long-tailed image classification,referred to as CMLTNet.CMLTNet improves the learning of intraclass similarity of tail-class representations by cross-modal alignment and captures the difference between the head and tail classes in semantic space by cross-modal inference.After fusing the above information,CMLTNet achieved an overall performance that was better than those of benchmark long-tailed and cross-modal learning methods on the long-tailed cross-modal datasets,NUS-WIDE and VireoFood-172.The effectiveness of the proposed modules was further studied through ablation experiments.In a case study of feature distribution,the proposed model was better in learning representations of tail classes,and in the experiments on model attention,CMLTNet has the potential to help learn some rare concepts in the tail class through mapping to the semantic space.
