Mixture of Experts(MoE)regression models are widely studied in statistics and machine learning for modeling heterogeneity in data for regression,clustering and classification.Laplace distribution is one of the most im...Mixture of Experts(MoE)regression models are widely studied in statistics and machine learning for modeling heterogeneity in data for regression,clustering and classification.Laplace distribution is one of the most important statistical tools to analyze thick and tail data.Laplace Mixture of Linear Experts(LMoLE)regression models are based on the Laplace distribution which is more robust.Similar to modelling variance parameter in a homogeneous population,we propose and study a new novel class of models:heteroscedastic Laplace mixture of experts regression models to analyze the heteroscedastic data coming from a heterogeneous population in this paper.The issues of maximum likelihood estimation are addressed.In particular,Minorization-Maximization(MM)algorithm for estimating the regression parameters is developed.Properties of the estimators of the regression coefficients are evaluated through Monte Carlo simulations.Results from the analysis of two real data sets are presented.展开更多
Assigning standardized International Classification of Disease(ICD)codes to Electronic Medical Records(EMR)is crucial for enhancing the efficiency and accuracy of medical coding processes.However,existing methods face...Assigning standardized International Classification of Disease(ICD)codes to Electronic Medical Records(EMR)is crucial for enhancing the efficiency and accuracy of medical coding processes.However,existing methods face challenges in effectively capturing,integrating,and amalgamating specialized medical knowledge from complex textual data.In this study,we propose GoM-ICD,an advanced automatic ICD coding framework that integrates multiple gap schemes with a Mixture of Experts(MoE)architecture.GoM-ICD is designed to address the extreme multilabel text classification in ICD coding.It segments and reassembles text to facilitate seamless information exchange across different chunks,employing various segmentation methods derived from different gap schemes.Then the model-level MoE consolidates the predictions of these methods to enhance the prediction performance.Specifically,the segmented text is input to a Pretrained Language Model(PLM)to extract textual features.Next,a Bidirectional Long Short-Term Memory network(BiLSTM)is employed to capture long-term contextual information from the textual features.Finally,a text-level MoE,followed by a label-level MoE,enables precise attention matching between text and labels,thereby improving the fidelity of the coding process.The three levels of MoE leverage the collective insights of diverse expert models,effectively processing multi-dimensional text features and unifying model-level insights from various gap schemes.Extensive experimental results demonstrate that GoM-ICD achieves the state-of-the-art performance in automatic ICD coding tasks,reaching micro-F1 of 0.617,0.620,and 0.613 on datasets MIMIC III full,MIMIC-III clean,and MIMIC-IV ICD-10,respectively.The source code can be obtained from https://github.com/CSUBioGroup/GoM-ICD.展开更多
Knowledge Bases （KBs） are valuable resources of human knowledge which contribute to many applications. However, since they are manually maintained, there is a big lag between their contents and the upto-date informa...Knowledge Bases （KBs） are valuable resources of human knowledge which contribute to many applications. However, since they are manually maintained, there is a big lag between their contents and the upto-date information of entities. Considering a target entity in KBs, this paper investigates how Cumulative Citation Recommendation （CCR） can be used to effectively detect its worthy-citation documents in large volumes of stream data. Most global relevant models only consider semantic and temporat features of entity-document instances, which does not sufficiently exploit prior knowledge underlying entity-document instances. To tackle this problem, we present a Mixture of Experts （ME） model by introducing a latent layer to capture relationships between the entity-document instances and their latent class information. An extensive set of experiments was conducted on TREC-KBA-2013 dataset. The results show that the model can significantly achieve a better performance gain compared to state-of-the-art models in CCR.展开更多
To better complete various missions, it is necessary to plan an optimal trajectory or provide the optimal control law for the multirole missile according to the actual situation, including launch conditions and target...To better complete various missions, it is necessary to plan an optimal trajectory or provide the optimal control law for the multirole missile according to the actual situation, including launch conditions and target location. Since trajectory optimization struggles to meet real-time requirements, the emergence of data-based generation methods has become a significant focus in contemporary research. However, due to the large differences in the characteristics of the optimal control laws caused by the diversity of tasks, it is difficult to achieve good prediction results by modeling all data with one single model.Therefore, the modeling idea of the mixture of experts(MoE) is adopted. Firstly, the K-means clustering algorithm is used to partition the sample data set, and the corresponding neural network classification model is established as the gate switch of MoE. Then, the expert models, i.e., the mappings from the generation conditions to the optimal control law represented by the results of principal component analysis(PCA), are represented by Kriging models. Finally, multiple rounds of accuracy evaluation, sample supplementation, and model updating are conducted to improve the generation accuracy. The Monte Carlo simulation shows that the accuracy of the proposed model reaches 96% and the generation efficiency meets the real-time requirement.展开更多
Psychiatric disorders constitute a complex health issue,primarily manifesting as significant disturbances in cognition,emotional regulation,and behavior.However,due to limited resources within health care systems,only...Psychiatric disorders constitute a complex health issue,primarily manifesting as significant disturbances in cognition,emotional regulation,and behavior.However,due to limited resources within health care systems,only a minority of patients can access effective treatment and care services,highlighting an urgent need for improvement.large language models(LLMs),with their natural language understanding and generation capabilities,are gradually penetrating the entire process of psychiatric diagnosis and treatment,including outpatient reception,diagnosis and therapy,clinical nursing,medication safety,and prognosis follow-up.They hold promise for improving the current severe shortage of health system resources and promoting equal access to mental health care.This article reviews the application scenarios and research progress of LLMs.It explores optimization methods for LLMs in psychiatry.Based on the research findings,we propose a clinical LLM for mental health using the Mixture of Experts framework to improve the accuracy of psychiatric diagnosis and therapeutic interventions.展开更多
Current works of environmental perception for connected autonomous electrified vehicles(CAEVs)mainly focus on the object detection task in good weather and illumination conditions,they often perform poorly in adverse ...Current works of environmental perception for connected autonomous electrified vehicles(CAEVs)mainly focus on the object detection task in good weather and illumination conditions,they often perform poorly in adverse scenarios and have a vague scene parsing ability.This paper aims to develop an end-to-end sharpening mixture of experts(SMoE)fusion framework to improve the robustness and accuracy of the perception systems for CAEVs in complex illumination and weather conditions.Three original contributions make our work distinctive from the existing relevant literature.The Complex KITTI dataset is introduced which consists of 7481 pairs of modified KITTI RGB images and the generated LiDAR dense depth maps,and this dataset is fine annotated in instance-level with the proposed semi-automatic annotation method.The SMoE fusion approach is devised to adaptively learn the robust kernels from complementary modalities.Comprehensive comparative experiments are implemented,and the results show that the proposed SMoE framework yield significant improvements over the other fusion techniques in adverse environmental conditions.This research proposes a SMoE fusion framework to improve the scene parsing ability of the perception systems for CAEVs in adverse conditions.展开更多
The prediction of molecular properties is a fundamental task in the field of drug discovery.Recently,graph neural networks(GNNs)have been gaining prominence in this area.Since a molecule tends to have multiple correla...The prediction of molecular properties is a fundamental task in the field of drug discovery.Recently,graph neural networks(GNNs)have been gaining prominence in this area.Since a molecule tends to have multiple correlated properties,there is a great need to develop the multi-task learning ability of GNNs.However,limited by expensive and time-consuming human annotations,collecting complete labels for each task is difficult.As a result,most existing benchmarks involve many missing labels in training data,and the performance of GNNs is impaired due to the lack of sufficient supervision information.To overcome this obstacle,we propose to improve multi-task molecular property prediction by missing label imputation.Specifically,a bipartite graph is first introduced to model the molecule-task co-occurrence relationships.Then,the imputation of missing labels is transformed into predicting missing edges on this bipartite graph.To predict the missing edges,a graph neural network is devised,which can learn the complex molecule-task co-occurrence relationships.After that,we select reliable pseudo labels according to the uncertainty of the prediction results.Boosting with enough and reliable supervision information,our approach achieves state-of-the-art performance on a variety of real-world datasets.展开更多
Space-time video super-resolution(STVSR)serves the purpose to reconstruct high-resolution high-frame-rate videos from their low-resolution low-frame-rate counterparts.Recent approaches utilize end-to-end deep learning...Space-time video super-resolution(STVSR)serves the purpose to reconstruct high-resolution high-frame-rate videos from their low-resolution low-frame-rate counterparts.Recent approaches utilize end-to-end deep learning models to achieve STVSR.They first interpolate intermediate frame features between given frames,then perform local and global refinement among the feature sequence,and finally increase the spatial resolutions of these features.However,in the most important feature interpolation phase,they only capture spatial-temporal information from the most adjacent frame features,ignoring modelling long-term spatial-temporal correlations between multiple neighbouring frames to restore variable-speed object movements and maintain long-term motion continuity.In this paper,we propose a novel long-term temporal feature aggregation network(LTFA-Net)for STVSR.Specifically,we design a long-term mixture of experts(LTMoE)module for feature interpolation.LTMoE contains multiple experts to extract mutual and complementary spatial-temporal information from multiple consecutive adjacent frame features,which are then combined with different weights to obtain interpolation results using several gating nets.Next,we perform local and global feature refinement using the Locally-temporal Feature Comparison(LFC)module and bidirectional deformable ConvLSTM layer,respectively.Experimental results on two standard benchmarks,Adobe240 and GoPro,indicate the effectiveness and superiority of our approach over state of the art.展开更多
基金the National Natural Science Foundation of China(11861041,11261025).
文摘Mixture of Experts(MoE)regression models are widely studied in statistics and machine learning for modeling heterogeneity in data for regression,clustering and classification.Laplace distribution is one of the most important statistical tools to analyze thick and tail data.Laplace Mixture of Linear Experts(LMoLE)regression models are based on the Laplace distribution which is more robust.Similar to modelling variance parameter in a homogeneous population,we propose and study a new novel class of models:heteroscedastic Laplace mixture of experts regression models to analyze the heteroscedastic data coming from a heterogeneous population in this paper.The issues of maximum likelihood estimation are addressed.In particular,Minorization-Maximization(MM)algorithm for estimating the regression parameters is developed.Properties of the estimators of the regression coefficients are evaluated through Monte Carlo simulations.Results from the analysis of two real data sets are presented.
文摘Assigning standardized International Classification of Disease(ICD)codes to Electronic Medical Records(EMR)is crucial for enhancing the efficiency and accuracy of medical coding processes.However,existing methods face challenges in effectively capturing,integrating,and amalgamating specialized medical knowledge from complex textual data.In this study,we propose GoM-ICD,an advanced automatic ICD coding framework that integrates multiple gap schemes with a Mixture of Experts(MoE)architecture.GoM-ICD is designed to address the extreme multilabel text classification in ICD coding.It segments and reassembles text to facilitate seamless information exchange across different chunks,employing various segmentation methods derived from different gap schemes.Then the model-level MoE consolidates the predictions of these methods to enhance the prediction performance.Specifically,the segmented text is input to a Pretrained Language Model(PLM)to extract textual features.Next,a Bidirectional Long Short-Term Memory network(BiLSTM)is employed to capture long-term contextual information from the textual features.Finally,a text-level MoE,followed by a label-level MoE,enables precise attention matching between text and labels,thereby improving the fidelity of the coding process.The three levels of MoE leverage the collective insights of diverse expert models,effectively processing multi-dimensional text features and unifying model-level insights from various gap schemes.Extensive experimental results demonstrate that GoM-ICD achieves the state-of-the-art performance in automatic ICD coding tasks,reaching micro-F1 of 0.617,0.620,and 0.613 on datasets MIMIC III full,MIMIC-III clean,and MIMIC-IV ICD-10,respectively.The source code can be obtained from https://github.com/CSUBioGroup/GoM-ICD.
基金supported by the National Key Research and Development Program of China(No.2016YFB1000902)the National Natural Science Foundation of China(Nos.61472040,61751217,and 61866038)+1 种基金Natural Science Basic Research Plan in Shaanxi Province of China(No.2016JM6082)PhD start project of Yan’an University(No.YDBK2018-09)
文摘Knowledge Bases （KBs） are valuable resources of human knowledge which contribute to many applications. However, since they are manually maintained, there is a big lag between their contents and the upto-date information of entities. Considering a target entity in KBs, this paper investigates how Cumulative Citation Recommendation （CCR） can be used to effectively detect its worthy-citation documents in large volumes of stream data. Most global relevant models only consider semantic and temporat features of entity-document instances, which does not sufficiently exploit prior knowledge underlying entity-document instances. To tackle this problem, we present a Mixture of Experts （ME） model by introducing a latent layer to capture relationships between the entity-document instances and their latent class information. An extensive set of experiments was conducted on TREC-KBA-2013 dataset. The results show that the model can significantly achieve a better performance gain compared to state-of-the-art models in CCR.
基金Defense Industrial Technology Development Program (JCKY2020204B016)National Natural Science Foundation of China (92471206)。
文摘To better complete various missions, it is necessary to plan an optimal trajectory or provide the optimal control law for the multirole missile according to the actual situation, including launch conditions and target location. Since trajectory optimization struggles to meet real-time requirements, the emergence of data-based generation methods has become a significant focus in contemporary research. However, due to the large differences in the characteristics of the optimal control laws caused by the diversity of tasks, it is difficult to achieve good prediction results by modeling all data with one single model.Therefore, the modeling idea of the mixture of experts(MoE) is adopted. Firstly, the K-means clustering algorithm is used to partition the sample data set, and the corresponding neural network classification model is established as the gate switch of MoE. Then, the expert models, i.e., the mappings from the generation conditions to the optimal control law represented by the results of principal component analysis(PCA), are represented by Kriging models. Finally, multiple rounds of accuracy evaluation, sample supplementation, and model updating are conducted to improve the generation accuracy. The Monte Carlo simulation shows that the accuracy of the proposed model reaches 96% and the generation efficiency meets the real-time requirement.
基金Supported by the STI2030-Major Projects,No.2021ZD0203400 and No.2021ZD0200800the National Natural Science Foundation of China,No.82171477。
文摘Psychiatric disorders constitute a complex health issue,primarily manifesting as significant disturbances in cognition,emotional regulation,and behavior.However,due to limited resources within health care systems,only a minority of patients can access effective treatment and care services,highlighting an urgent need for improvement.large language models(LLMs),with their natural language understanding and generation capabilities,are gradually penetrating the entire process of psychiatric diagnosis and treatment,including outpatient reception,diagnosis and therapy,clinical nursing,medication safety,and prognosis follow-up.They hold promise for improving the current severe shortage of health system resources and promoting equal access to mental health care.This article reviews the application scenarios and research progress of LLMs.It explores optimization methods for LLMs in psychiatry.Based on the research findings,we propose a clinical LLM for mental health using the Mixture of Experts framework to improve the accuracy of psychiatric diagnosis and therapeutic interventions.
基金Supported by National Natural Science Foundation of China(Grant Nos.51975118,52025121,51975103,51905095)National Natural Science Foundation of Jiangsu Province(Grant No.BK20180401).
文摘Current works of environmental perception for connected autonomous electrified vehicles(CAEVs)mainly focus on the object detection task in good weather and illumination conditions,they often perform poorly in adverse scenarios and have a vague scene parsing ability.This paper aims to develop an end-to-end sharpening mixture of experts(SMoE)fusion framework to improve the robustness and accuracy of the perception systems for CAEVs in complex illumination and weather conditions.Three original contributions make our work distinctive from the existing relevant literature.The Complex KITTI dataset is introduced which consists of 7481 pairs of modified KITTI RGB images and the generated LiDAR dense depth maps,and this dataset is fine annotated in instance-level with the proposed semi-automatic annotation method.The SMoE fusion approach is devised to adaptively learn the robust kernels from complementary modalities.Comprehensive comparative experiments are implemented,and the results show that the proposed SMoE framework yield significant improvements over the other fusion techniques in adverse environmental conditions.This research proposes a SMoE fusion framework to improve the scene parsing ability of the perception systems for CAEVs in adverse conditions.
基金supported by the National Natural Science Foundation of China(Nos.62141608 and U19B 2038),the CAAI Huawei MindSpore Open Fund.
文摘The prediction of molecular properties is a fundamental task in the field of drug discovery.Recently,graph neural networks(GNNs)have been gaining prominence in this area.Since a molecule tends to have multiple correlated properties,there is a great need to develop the multi-task learning ability of GNNs.However,limited by expensive and time-consuming human annotations,collecting complete labels for each task is difficult.As a result,most existing benchmarks involve many missing labels in training data,and the performance of GNNs is impaired due to the lack of sufficient supervision information.To overcome this obstacle,we propose to improve multi-task molecular property prediction by missing label imputation.Specifically,a bipartite graph is first introduced to model the molecule-task co-occurrence relationships.Then,the imputation of missing labels is transformed into predicting missing edges on this bipartite graph.To predict the missing edges,a graph neural network is devised,which can learn the complex molecule-task co-occurrence relationships.After that,we select reliable pseudo labels according to the uncertainty of the prediction results.Boosting with enough and reliable supervision information,our approach achieves state-of-the-art performance on a variety of real-world datasets.
文摘Space-time video super-resolution(STVSR)serves the purpose to reconstruct high-resolution high-frame-rate videos from their low-resolution low-frame-rate counterparts.Recent approaches utilize end-to-end deep learning models to achieve STVSR.They first interpolate intermediate frame features between given frames,then perform local and global refinement among the feature sequence,and finally increase the spatial resolutions of these features.However,in the most important feature interpolation phase,they only capture spatial-temporal information from the most adjacent frame features,ignoring modelling long-term spatial-temporal correlations between multiple neighbouring frames to restore variable-speed object movements and maintain long-term motion continuity.In this paper,we propose a novel long-term temporal feature aggregation network(LTFA-Net)for STVSR.Specifically,we design a long-term mixture of experts(LTMoE)module for feature interpolation.LTMoE contains multiple experts to extract mutual and complementary spatial-temporal information from multiple consecutive adjacent frame features,which are then combined with different weights to obtain interpolation results using several gating nets.Next,we perform local and global feature refinement using the Locally-temporal Feature Comparison(LFC)module and bidirectional deformable ConvLSTM layer,respectively.Experimental results on two standard benchmarks,Adobe240 and GoPro,indicate the effectiveness and superiority of our approach over state of the art.
