Skeleton-based action recognition has recently made significant progress.However,data imbalance is still a great challenge in real-world scenarios.The performance of current action recognition algorithms declines shar...Skeleton-based action recognition has recently made significant progress.However,data imbalance is still a great challenge in real-world scenarios.The performance of current action recognition algorithms declines sharply when training data suffers from heavy class imbalance.The imbalanced data actually degrades the representations learned by these methods and becomes the bottleneck for action recognition.How to learn unbiased representations from imbalanced action data is the key to long-tailed action recognition.In this paper,we propose a novel balanced representation learning method to address the long-tailed problem in action recognition.Firstly,a spatial-temporal action exploration strategy is presented to expand the sample space effectively,generating more valuable samples in a rebalanced manner.Secondly,we design a detached action-aware learning schedule to further mitigate the bias in the representation space.The schedule detaches the representation learning of tail classes from training and proposes an action-aware loss to impose more effective constraints.Additionally,a skip-type representation is proposed to provide complementary structural information.The proposed method is validated on four skeleton datasets,NTU RGB+D 60,NTU RGB+D 120,NW-UCLA and Kinetics.It not only achieves consistently large improvement compared to the state-of-the-art(SOTA)methods,but also demonstrates a superior generalization capacity through extensive experiments.Our code is available at https://github.com/firework8/BRL.展开更多
The prediction of molecular properties is a fundamental task in the field of drug discovery.Recently,graph neural networks(GNNs)have been gaining prominence in this area.Since a molecule tends to have multiple correla...The prediction of molecular properties is a fundamental task in the field of drug discovery.Recently,graph neural networks(GNNs)have been gaining prominence in this area.Since a molecule tends to have multiple correlated properties,there is a great need to develop the multi-task learning ability of GNNs.However,limited by expensive and time-consuming human annotations,collecting complete labels for each task is difficult.As a result,most existing benchmarks involve many missing labels in training data,and the performance of GNNs is impaired due to the lack of sufficient supervision information.To overcome this obstacle,we propose to improve multi-task molecular property prediction by missing label imputation.Specifically,a bipartite graph is first introduced to model the molecule-task co-occurrence relationships.Then,the imputation of missing labels is transformed into predicting missing edges on this bipartite graph.To predict the missing edges,a graph neural network is devised,which can learn the complex molecule-task co-occurrence relationships.After that,we select reliable pseudo labels according to the uncertainty of the prediction results.Boosting with enough and reliable supervision information,our approach achieves state-of-the-art performance on a variety of real-world datasets.展开更多
Real-world objects exhibit intricate semantic properties that can be characterized from a multitude of perspectives, which necessitates the development of a model capable of discerning multiple patterns within data, w...Real-world objects exhibit intricate semantic properties that can be characterized from a multitude of perspectives, which necessitates the development of a model capable of discerning multiple patterns within data, while concurrently predicting several Labeling Dimensions (LDs) — a task known as Multi-dimensional Classification (MDC). While the class imbalance issue has been extensively investigated within the multi-class paradigm, its study in the MDC context has been limited due to the imbalance shift phenomenon. A sample’s classification as a minor or major class instance becomes ambiguous when it belongs to a minor class in one LD and a major class in another. Previous MDC methodologies predominantly emphasized instance-wise criteria, neglecting prediction capabilities from a dimension aspect, i.e., the average classification performance across LDs. We assert the significance of dimension-wise metrics in real-world MDC applications and introduce two such metrics. Furthermore, we observe imbalanced class distributions within each LD and propose a novel Imbalance-Aware fusion Model (IMAM) for addressing the MDC problem. Specifically, we first decompose the task into multiple multi-class classification problems, creating imbalance-aware deep models for each LD separately. This straightforward method performs well across LDs without sacrificing performance in instance-wise criteria. Subsequently, we employ LD-wise models as multiple teachers and transfer their knowledge across all LDs to a unified student model. Experimental results on several real-world datasets demonstrate that our IMAM approach excels in both instance-wise evaluations and the proposed dimension-wise metrics.展开更多
基金supported by the National Natural Science Foundation of China(Nos.62276263,62006225 and 62071468)the Strategic Priority Research Program of Chinese Academy of Sciences(CAS),China(No.XDA27040700)the National Key Research and Development Program of China(No.2022YFC3310400).
文摘Skeleton-based action recognition has recently made significant progress.However,data imbalance is still a great challenge in real-world scenarios.The performance of current action recognition algorithms declines sharply when training data suffers from heavy class imbalance.The imbalanced data actually degrades the representations learned by these methods and becomes the bottleneck for action recognition.How to learn unbiased representations from imbalanced action data is the key to long-tailed action recognition.In this paper,we propose a novel balanced representation learning method to address the long-tailed problem in action recognition.Firstly,a spatial-temporal action exploration strategy is presented to expand the sample space effectively,generating more valuable samples in a rebalanced manner.Secondly,we design a detached action-aware learning schedule to further mitigate the bias in the representation space.The schedule detaches the representation learning of tail classes from training and proposes an action-aware loss to impose more effective constraints.Additionally,a skip-type representation is proposed to provide complementary structural information.The proposed method is validated on four skeleton datasets,NTU RGB+D 60,NTU RGB+D 120,NW-UCLA and Kinetics.It not only achieves consistently large improvement compared to the state-of-the-art(SOTA)methods,but also demonstrates a superior generalization capacity through extensive experiments.Our code is available at https://github.com/firework8/BRL.
基金supported by the National Natural Science Foundation of China(Nos.62141608 and U19B 2038),the CAAI Huawei MindSpore Open Fund.
文摘The prediction of molecular properties is a fundamental task in the field of drug discovery.Recently,graph neural networks(GNNs)have been gaining prominence in this area.Since a molecule tends to have multiple correlated properties,there is a great need to develop the multi-task learning ability of GNNs.However,limited by expensive and time-consuming human annotations,collecting complete labels for each task is difficult.As a result,most existing benchmarks involve many missing labels in training data,and the performance of GNNs is impaired due to the lack of sufficient supervision information.To overcome this obstacle,we propose to improve multi-task molecular property prediction by missing label imputation.Specifically,a bipartite graph is first introduced to model the molecule-task co-occurrence relationships.Then,the imputation of missing labels is transformed into predicting missing edges on this bipartite graph.To predict the missing edges,a graph neural network is devised,which can learn the complex molecule-task co-occurrence relationships.After that,we select reliable pseudo labels according to the uncertainty of the prediction results.Boosting with enough and reliable supervision information,our approach achieves state-of-the-art performance on a variety of real-world datasets.
基金supported by the National Key R&D Program of China(2020AAA0109401,2020AAA0109405),(62376118,62006112,62250069,62206245)the Young Elite Scientists Sponsorship Program of Jiangsu Association for Science and the Technology 2021-020Collaborative Innovation Center of Novel Software Technology and Industrialization.
文摘Real-world objects exhibit intricate semantic properties that can be characterized from a multitude of perspectives, which necessitates the development of a model capable of discerning multiple patterns within data, while concurrently predicting several Labeling Dimensions (LDs) — a task known as Multi-dimensional Classification (MDC). While the class imbalance issue has been extensively investigated within the multi-class paradigm, its study in the MDC context has been limited due to the imbalance shift phenomenon. A sample’s classification as a minor or major class instance becomes ambiguous when it belongs to a minor class in one LD and a major class in another. Previous MDC methodologies predominantly emphasized instance-wise criteria, neglecting prediction capabilities from a dimension aspect, i.e., the average classification performance across LDs. We assert the significance of dimension-wise metrics in real-world MDC applications and introduce two such metrics. Furthermore, we observe imbalanced class distributions within each LD and propose a novel Imbalance-Aware fusion Model (IMAM) for addressing the MDC problem. Specifically, we first decompose the task into multiple multi-class classification problems, creating imbalance-aware deep models for each LD separately. This straightforward method performs well across LDs without sacrificing performance in instance-wise criteria. Subsequently, we employ LD-wise models as multiple teachers and transfer their knowledge across all LDs to a unified student model. Experimental results on several real-world datasets demonstrate that our IMAM approach excels in both instance-wise evaluations and the proposed dimension-wise metrics.
