Many spore-forming Bacillus species can cause serious human diseases,because of accidental Bacillusspore infection.Thus,developing an identification strategy with both high sensitivity and specificity is greatly in de...Many spore-forming Bacillus species can cause serious human diseases,because of accidental Bacillusspore infection.Thus,developing an identification strategy with both high sensitivity and specificity is greatly in demand.In this work,we proposed a novel approach named multi-head self-attention mechanism-guided neural network Raman platform to identify living Bacillus spores within a single-cell resolution.The multi-head self-attention mechanism-guided neural network Raman platform was created by combining single-cell Raman spectroscopy,convolutional neural network(CNN),and multi-head self-attention mechanism.To address the limited size of the original spectra dataset,Gaussian noise-based spectra augmentation was employed to increase the number of single-cell Raman spectra datasets for CNN training.Owing to the assistance of both spectra augmentation and multi-head self-attention mechanism,the obtained prediction accuracy of five Bacillus spore species was further improved from 92.29±0.82%to 99.43±0.15%.To figure out the spectra differences covered by the multi-head self-attention mechanism-guided CNN,the relative classification weight from typical Raman bands was visualized via multi-head self-attention mechanism curve.In the process of spectra augmentation from 0 to 1000,the distribution of relative classification weight varied from a discrete state to a more concentrated phase.More importantly,these highlighted four Raman bands(1017,1449,1576,and 1660 cm^(-1))were assigned large weights,showing that the spectra differences in the Raman bands produced the largest contribution to prediction accuracy.It can be foreseen that,our proposed sorting platform has great potential in accurately identifying Bacillus and its related genera species at a single-cell level.展开更多
Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract loc...Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract local and global features, as well as the lack of effective mechanisms to capture complex interactions between features;Additionally, when increasing the receptive field to obtain deeper feature representations, the reliance on increasing network depth leads to a significant increase in computational resource consumption, affecting the efficiency and performance of detection. Based on these issues, firstly, this paper proposes a network traffic anomaly detection model based on parallel dilated convolution and residual learning (Res-PDC). To better explore the interactive relationships between features, the traffic samples are converted into two-dimensional matrix. A module combining parallel dilated convolutions and residual learning (res-pdc) was designed to extract local and global features of traffic at different scales. By utilizing res-pdc modules with different dilation rates, we can effectively capture spatial features at different scales and explore feature dependencies spanning wider regions without increasing computational resources. Secondly, to focus and integrate the information in different feature subspaces, further enhance and extract the interactions among the features, multi-head attention is added to Res-PDC, resulting in the final model: multi-head attention enhanced parallel dilated convolution and residual learning (MHA-Res-PDC) for network traffic anomaly detection. Finally, comparisons with other machine learning and deep learning algorithms are conducted on the NSL-KDD and CIC-IDS-2018 datasets. The experimental results demonstrate that the proposed method in this paper can effectively improve the detection performance.展开更多
Sarcasm detection is a complex and challenging task,particularly in the context of Chinese social media,where it exhibits strong contextual dependencies and cultural specificity.To address the limitations of existing ...Sarcasm detection is a complex and challenging task,particularly in the context of Chinese social media,where it exhibits strong contextual dependencies and cultural specificity.To address the limitations of existing methods in capturing the implicit semantics and contextual associations in sarcastic expressions,this paper proposes an event-aware model for Chinese sarcasm detection,leveraging a multi-head attention(MHA)mechanism and contrastive learning(CL)strategies.The proposed model employs a dual-path Bidirectional Encoder Representations from Transformers(BERT)encoder to process comment text and event context separately and integrates an MHA mechanism to facilitate deep interactions between the two,thereby capturing multidimensional semantic associations.Additionally,a CL strategy is introduced to enhance feature representation capabilities,further improving the model’s performance in handling class imbalance and complex contextual scenarios.The model achieves state-of-the-art performance on the Chinese sarcasm dataset,with significant improvements in accuracy(79.55%),F1-score(84.22%),and an area under the curve(AUC,84.35%).展开更多
As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as...As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as a new challenge in the field of recommendation systems.This paper introduces a group-buying recommendation model based on multi-head attention mechanisms and multi-task learning,termed the Multi-head Attention Mechanisms and Multi-task Learning Group-Buying Recommendation(MAMGBR)model,specifically designed to optimize group-buying recommendations on e-commerce platforms.The core dataset of this study comes from the Chinese maternal and infant e-commerce platform“Beibei,”encompassing approximately 430,000 successful groupbuying actions and over 120,000 users.Themodel focuses on twomain tasks:recommending items for group organizers(Task Ⅰ)and recommending participants for a given group-buying event(Task Ⅱ).In model evaluation,MAMGBR achieves an MRR@10 of 0.7696 for Task I,marking a 20.23%improvement over baseline models.Furthermore,in Task II,where complex interaction patterns prevail,MAMGBR utilizes auxiliary loss functions to effectively model the multifaceted roles of users,items,and participants,leading to a 24.08%increase in MRR@100 under a 1:99 sample ratio.Experimental results show that compared to benchmark models,such as NGCF and EATNN,MAMGBR’s integration ofmulti-head attentionmechanisms,expert networks,and gating mechanisms enables more accurate modeling of user preferences and social associations within group-buying scenarios,significantly enhancing recommendation accuracy and platform group-buying success rates.展开更多
Safety maintenance of power equipment is of great importance in power grids,in which image-processing-based defect recognition is supposed to classify abnormal conditions during daily inspection.However,owing to the b...Safety maintenance of power equipment is of great importance in power grids,in which image-processing-based defect recognition is supposed to classify abnormal conditions during daily inspection.However,owing to the blurred features of defect images,the current defect recognition algorithm has poor fine-grained recognition ability.Visual attention can achieve fine-grained recognition with its abil-ity to model long-range dependencies while introducing extra computational complexity,especially for multi-head attention in vision transformer structures.Under these circumstances,this paper proposes a self-reduction multi-head attention module that can reduce computational complexity and be easily combined with a Convolutional Neural Network(CNN).In this manner,local and global fea-tures can be calculated simultaneously in our proposed structure,aiming to improve the defect recognition performance.Specifically,the proposed self-reduction multi-head attention can reduce redundant parameters,thereby solving the problem of limited computational resources.Experimental results were obtained based on the defect dataset collected from the substation.The results demonstrated the efficiency and superiority of the proposed method over other advanced algorithms.展开更多
Lightweight deep learning models are increasingly required in resource-constrained environments such as mobile devices and the Internet of Medical Things(IoMT).Multi-head convolution with channel attention can facilit...Lightweight deep learning models are increasingly required in resource-constrained environments such as mobile devices and the Internet of Medical Things(IoMT).Multi-head convolution with channel attention can facilitate learning activations relevant to different kernel sizes within a multi-head convolutional layer.Therefore,this study investigates the capability of novel lightweight models incorporating residual multi-head convolution with channel attention(ResMHCNN)blocks to classify medical images.We introduced three novel lightweight deep learning models(BT-Net,LCC-Net,and BC-Net)utilizing the ResMHCNN block as their backbone.These models were crossvalidated and tested on three publicly available medical image datasets:a brain tumor dataset from Figshare consisting of T1-weighted magnetic resonance imaging slices of meningioma,glioma,and pituitary tumors;the LC25000 dataset,which includes microscopic images of lung and colon cancers;and the BreaKHis dataset,containing benign and malignant breast microscopic images.The lightweight models achieved accuracies of 96.9%for 3-class brain tumor classification using BT-Net,and 99.7%for 5-class lung and colon cancer classification using LCC-Net.For 2-class breast cancer classification,BC-Net achieved an accuracy of 96.7%.The parameter counts for the proposed lightweight models—LCC-Net,BC-Net,and BT-Net—are 0.528,0.226,and 1.154 million,respectively.The presented lightweight models,featuring ResMHCNN blocks,may be effectively employed for accurate medical image classification.In the future,these models might be tested for viability in resource-constrained systems such as mobile devices and IoMT platforms.展开更多
Coal dust explosions are severe safety accidents in coal mine production,posing significant threats to life and property.Predicting the maximum explosion pressure(Pm)of coal dust using deep learning models can effecti...Coal dust explosions are severe safety accidents in coal mine production,posing significant threats to life and property.Predicting the maximum explosion pressure(Pm)of coal dust using deep learning models can effectively assess potential risks and provide a scientific basis for preventing coal dust explosions.In this study,a 20-L explosion sphere apparatus was used to test the maximum explosion pressure of coal dust under seven different particle sizes and ten mass concentrations(Cdust),resulting in a dataset of 70 experimental groups.Through Spearman correlation analysis and random forest feature selection methods,particle size(D_(10),D_(20),D_(50))and mass concentration(Cdust)were identified as critical feature parameters from the ten initial parameters of the coal dust samples.Based on this,a hybrid Long Short-Term Memory(LSTM)network model incorporating a Multi-Head Attention Mechanism and the Sparrow Search Algorithm(SSA)was proposed to predict the maximum explosion pressure of coal dust.The results demonstrate that the SSA-LSTM-Multi-Head Attention model excels in predicting the maximum explosion pressure of coal dust.The four evaluation metrics indicate that the model achieved a coefficient of determination(R^(2)),root mean square error(RMSE),mean absolute percentage error(MAPE),and mean absolute error(MAE)of 0.9841,0.0030,0.0074,and 0.0049,respectively,in the training set.In the testing set,these values were 0.9743,0.0087,0.0108,and 0.0069,respectively.Compared to artificial neural networks(ANN),random forest(RF),support vector machines(SVM),particle swarm optimized-SVM(PSO-SVM)neural networks,and the traditional single-model LSTM,the SSA-LSTM-Multi-Head Attention model demonstrated superior generalization capability and prediction accuracy.The findings of this study not only advance the application of deep learning in coal dust explosion prediction but also provide robust technical support for the prevention and risk assessment of coal dust explosions.展开更多
The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships ...The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships between pixels requires further improvement.Previous methods face challenges in efficiently managing multi-scale fea-tures of different granularities from the encoder backbone,leaving room for improvement in their global representation and feature extraction capabilities.To address these challenges,we propose a novel Decoder with Multi-Head Feature Receptors(DMHFR),which receives multi-scale features from the encoder backbone and organizes them into three feature groups with different granularities:coarse,fine-grained,and full set.These groups are subsequently processed by Multi-Head Feature Receptors(MHFRs)after feature capture and modeling operations.MHFRs include two Three-Head Feature Receptors(THFRs)and one Four-Head Feature Receptor(FHFR).Each group of features is passed through these MHFRs and then fed into axial transformers,which help the model capture long-range dependencies within the features.The three MHFRs produce three distinct feature outputs.The output from the FHFR serves as auxiliary auxiliary features in the prediction head,and the prediction output and their losses will eventually be aggregated.Experimental results show that the Transformer using DMHFR outperforms 15 state of the arts(SOTA)methods on five public datasets.Specifically,it achieved significant improvements in mean DICE scores over the classic Parallel Reverse Attention Network(PraNet)method,with gains of 4.1%,2.2%,1.4%,8.9%,and 16.3%on the CVC-ClinicDB,Kvasir-SEG,CVC-T,CVC-ColonDB,and ETIS-LaribPolypDB datasets,respectively.展开更多
Traffic flow prediction is a crucial element of intelligent transportation systems.However,accu-rate traffic flow prediction is quite challenging because of its highly nonlinear,complex,and dynam-ic characteristics.To...Traffic flow prediction is a crucial element of intelligent transportation systems.However,accu-rate traffic flow prediction is quite challenging because of its highly nonlinear,complex,and dynam-ic characteristics.To address the difficulties in simultaneously capturing local and global dynamic spatiotemporal correlations in traffic flow,as well as the high time complexity of existing models,a multi-head flow attention-based local-global dynamic hypergraph convolution(MFA-LGDHC)pre-diction model is proposed.which consists of multi-head flow attention(MHFA)mechanism,graph convolution network(GCN),and local-global dynamic hypergraph convolution(LGHC).MHFA is utilized to extract the time dependency of traffic flow and reduce the time complexity of the model.GCN is employed to catch the spatial dependency of traffic flow.LGHC utilizes down-sampling con-volution and isometric convolution to capture the local and global spatial dependencies of traffic flow.And dynamic hypergraph convolution is used to model the dynamic higher-order relationships of the traffic road network.Experimental results indicate that the MFA-LGDHC model outperforms current popular baseline models and exhibits good prediction performance.展开更多
Reliable traffic flow prediction is crucial for mitigating urban congestion.This paper proposes Attentionbased spatiotemporal Interactive Dynamic Graph Convolutional Network(AIDGCN),a novel architecture integrating In...Reliable traffic flow prediction is crucial for mitigating urban congestion.This paper proposes Attentionbased spatiotemporal Interactive Dynamic Graph Convolutional Network(AIDGCN),a novel architecture integrating Interactive Dynamic Graph Convolution Network(IDGCN)with Temporal Multi-Head Trend-Aware Attention.Its core innovation lies in IDGCN,which uniquely splits sequences into symmetric intervals for interactive feature sharing via dynamic graphs,and a novel attention mechanism incorporating convolutional operations to capture essential local traffic trends—addressing a critical gap in standard attention for continuous data.For 15-and 60-min forecasting on METR-LA,AIDGCN achieves MAEs of 0.75%and 0.39%,and RMSEs of 1.32%and 0.14%,respectively.In the 60-min long-term forecasting of the PEMS-BAY dataset,the AIDGCN out-performs the MRA-BGCN method by 6.28%,4.93%,and 7.17%in terms of MAE,RMSE,and MAPE,respectively.Experimental results demonstrate the superiority of our pro-posed model over state-of-the-art methods.展开更多
Accurate and reliable power system data are fundamental for critical operations such as gridmonitoring,fault diagnosis,and load forecasting,underpinned by increasing intelligentization and digitalization.However,data ...Accurate and reliable power system data are fundamental for critical operations such as gridmonitoring,fault diagnosis,and load forecasting,underpinned by increasing intelligentization and digitalization.However,data loss and anomalies frequently compromise data integrity in practical settings,significantly impacting system operational efficiency and security.Most existing data recovery methods require complete datasets for training,leading to substantial data and computational demands and limited generalization.To address these limitations,this study proposes a missing data imputation model based on an improved Generative Adversarial Network(BAC-GAN).Within the BAC-GAN framework,the generator utilizes Bidirectional Long Short-Term Memory(BiLSTM)networks and Multi-Head Attention mechanisms to capture temporal dependencies and complex relationships within power system data.The discriminator employs a Convolutional Neural Network(CNN)architecture to integrate local features with global structures,effectivelymitigating the generation of implausible imputations.Experimental results on two public datasets demonstrate that the BAC-GAN model achieves superior data recovery accuracy compared to five state-of-the-art and classical benchmarkmethods,with an average improvement of 17.7%in reconstruction accuracy.The proposedmethod significantly enhances the accuracy of grid fault diagnosis and provides reliable data support for the stable operation of smart grids,showing great potential for practical applications in power systems.展开更多
Groundwater level(GWL)is a key indicator used to accurately assess groundwater resources and form the foundation for ef-fective groundwater management.This paper integrates a Gate Recurrent Unit(GRU)model with a Multi...Groundwater level(GWL)is a key indicator used to accurately assess groundwater resources and form the foundation for ef-fective groundwater management.This paper integrates a Gate Recurrent Unit(GRU)model with a Multi-head Self-attention mechan-ism(MSAM-GRU)to simulate GWLs in both confined and unconfined aquifers simultaneously.The model innovatively captures the lag times between GWLs in the unconfined aquifer and precipitation,as well as between GWLs in the confined aquifer and the upper aquifer.We have assessed the effectiveness of the proposed model using a case study in the Beijing Plain,China from January 2005 to December 2020.With the consideration of lag times,the results indicated that the MSAM-GRU model exhibits a maximum 67%and 73%reduction in RMSE compared to the Attention mechanism-GRU(AM-GRU)and GRU model,respectively.MSAM-GRU model exhibited a 31%reduction in RMSE and a 0.12 increase in R^(2) compared to the same model that do not account for lag time.In Region I,the shortest lag time of GWL in the unconfined aquifer was two months,while that in the confined aquifer was three months,indicating a longer delayed response in the confined aquifer.MSAM-GRU model considering lag time,was then applied to simulate the GWLs in the unconfined aquifer under different scenarios and to analyze whether GWL fluctuations affect subway operations.The simulation res-ults showed that under the scenario 1,the GWL in the unconfined aquifer would rise above the depth of subway station floor,threaten-ing the operation of subways.This study can provide reliable technical support for the accurate simulation of GWLs in multi-aquifer systems.展开更多
基金partially supported by the National Natural Science Foundation of China(62075137)the Guangdong Basic and Applied Basic Research Foundation(2023A1515140161)+3 种基金the Guangxi Natural Science Foundation of China(2021JJB 110003)the Dongguan Science and Technology of Social Development Program(20231800936312)the high-level talent program of Dongguan University of Technology(No.221110080)the Sanming Project of Medicine in Shenzhen(No.SZSM202103014).
文摘Many spore-forming Bacillus species can cause serious human diseases,because of accidental Bacillusspore infection.Thus,developing an identification strategy with both high sensitivity and specificity is greatly in demand.In this work,we proposed a novel approach named multi-head self-attention mechanism-guided neural network Raman platform to identify living Bacillus spores within a single-cell resolution.The multi-head self-attention mechanism-guided neural network Raman platform was created by combining single-cell Raman spectroscopy,convolutional neural network(CNN),and multi-head self-attention mechanism.To address the limited size of the original spectra dataset,Gaussian noise-based spectra augmentation was employed to increase the number of single-cell Raman spectra datasets for CNN training.Owing to the assistance of both spectra augmentation and multi-head self-attention mechanism,the obtained prediction accuracy of five Bacillus spore species was further improved from 92.29±0.82%to 99.43±0.15%.To figure out the spectra differences covered by the multi-head self-attention mechanism-guided CNN,the relative classification weight from typical Raman bands was visualized via multi-head self-attention mechanism curve.In the process of spectra augmentation from 0 to 1000,the distribution of relative classification weight varied from a discrete state to a more concentrated phase.More importantly,these highlighted four Raman bands(1017,1449,1576,and 1660 cm^(-1))were assigned large weights,showing that the spectra differences in the Raman bands produced the largest contribution to prediction accuracy.It can be foreseen that,our proposed sorting platform has great potential in accurately identifying Bacillus and its related genera species at a single-cell level.
基金supported by the Xiamen Science and Technology Subsidy Project(No.2023CXY0318).
文摘Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract local and global features, as well as the lack of effective mechanisms to capture complex interactions between features;Additionally, when increasing the receptive field to obtain deeper feature representations, the reliance on increasing network depth leads to a significant increase in computational resource consumption, affecting the efficiency and performance of detection. Based on these issues, firstly, this paper proposes a network traffic anomaly detection model based on parallel dilated convolution and residual learning (Res-PDC). To better explore the interactive relationships between features, the traffic samples are converted into two-dimensional matrix. A module combining parallel dilated convolutions and residual learning (res-pdc) was designed to extract local and global features of traffic at different scales. By utilizing res-pdc modules with different dilation rates, we can effectively capture spatial features at different scales and explore feature dependencies spanning wider regions without increasing computational resources. Secondly, to focus and integrate the information in different feature subspaces, further enhance and extract the interactions among the features, multi-head attention is added to Res-PDC, resulting in the final model: multi-head attention enhanced parallel dilated convolution and residual learning (MHA-Res-PDC) for network traffic anomaly detection. Finally, comparisons with other machine learning and deep learning algorithms are conducted on the NSL-KDD and CIC-IDS-2018 datasets. The experimental results demonstrate that the proposed method in this paper can effectively improve the detection performance.
基金granted by Qin Xin Talents Cultivation Program(No.QXTCP C202115),Beijing Information Science&Technology Universitythe Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing Fund(No.GJJ-23),National Social Science Foundation,China(No.21BTQ079).
文摘Sarcasm detection is a complex and challenging task,particularly in the context of Chinese social media,where it exhibits strong contextual dependencies and cultural specificity.To address the limitations of existing methods in capturing the implicit semantics and contextual associations in sarcastic expressions,this paper proposes an event-aware model for Chinese sarcasm detection,leveraging a multi-head attention(MHA)mechanism and contrastive learning(CL)strategies.The proposed model employs a dual-path Bidirectional Encoder Representations from Transformers(BERT)encoder to process comment text and event context separately and integrates an MHA mechanism to facilitate deep interactions between the two,thereby capturing multidimensional semantic associations.Additionally,a CL strategy is introduced to enhance feature representation capabilities,further improving the model’s performance in handling class imbalance and complex contextual scenarios.The model achieves state-of-the-art performance on the Chinese sarcasm dataset,with significant improvements in accuracy(79.55%),F1-score(84.22%),and an area under the curve(AUC,84.35%).
基金supported by the Key Research and Development Program of Heilongjiang Province(No.2022ZX01A35).
文摘As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as a new challenge in the field of recommendation systems.This paper introduces a group-buying recommendation model based on multi-head attention mechanisms and multi-task learning,termed the Multi-head Attention Mechanisms and Multi-task Learning Group-Buying Recommendation(MAMGBR)model,specifically designed to optimize group-buying recommendations on e-commerce platforms.The core dataset of this study comes from the Chinese maternal and infant e-commerce platform“Beibei,”encompassing approximately 430,000 successful groupbuying actions and over 120,000 users.Themodel focuses on twomain tasks:recommending items for group organizers(Task Ⅰ)and recommending participants for a given group-buying event(Task Ⅱ).In model evaluation,MAMGBR achieves an MRR@10 of 0.7696 for Task I,marking a 20.23%improvement over baseline models.Furthermore,in Task II,where complex interaction patterns prevail,MAMGBR utilizes auxiliary loss functions to effectively model the multifaceted roles of users,items,and participants,leading to a 24.08%increase in MRR@100 under a 1:99 sample ratio.Experimental results show that compared to benchmark models,such as NGCF and EATNN,MAMGBR’s integration ofmulti-head attentionmechanisms,expert networks,and gating mechanisms enables more accurate modeling of user preferences and social associations within group-buying scenarios,significantly enhancing recommendation accuracy and platform group-buying success rates.
基金supported in part by Major Program of the National Natural Science Foundation of China under Grant 62127803.
文摘Safety maintenance of power equipment is of great importance in power grids,in which image-processing-based defect recognition is supposed to classify abnormal conditions during daily inspection.However,owing to the blurred features of defect images,the current defect recognition algorithm has poor fine-grained recognition ability.Visual attention can achieve fine-grained recognition with its abil-ity to model long-range dependencies while introducing extra computational complexity,especially for multi-head attention in vision transformer structures.Under these circumstances,this paper proposes a self-reduction multi-head attention module that can reduce computational complexity and be easily combined with a Convolutional Neural Network(CNN).In this manner,local and global fea-tures can be calculated simultaneously in our proposed structure,aiming to improve the defect recognition performance.Specifically,the proposed self-reduction multi-head attention can reduce redundant parameters,thereby solving the problem of limited computational resources.Experimental results were obtained based on the defect dataset collected from the substation.The results demonstrated the efficiency and superiority of the proposed method over other advanced algorithms.
基金supported by the Institute of Information&Communications Technology Planning&Evaluation(IITP)-Innovative Human Resource Development for Local Intellectualization program grant funded by the Korea government(MSIT)(IITP-2025-RS-2023-00259678)by INHA UNIVERSITY Research Grant.
文摘Lightweight deep learning models are increasingly required in resource-constrained environments such as mobile devices and the Internet of Medical Things(IoMT).Multi-head convolution with channel attention can facilitate learning activations relevant to different kernel sizes within a multi-head convolutional layer.Therefore,this study investigates the capability of novel lightweight models incorporating residual multi-head convolution with channel attention(ResMHCNN)blocks to classify medical images.We introduced three novel lightweight deep learning models(BT-Net,LCC-Net,and BC-Net)utilizing the ResMHCNN block as their backbone.These models were crossvalidated and tested on three publicly available medical image datasets:a brain tumor dataset from Figshare consisting of T1-weighted magnetic resonance imaging slices of meningioma,glioma,and pituitary tumors;the LC25000 dataset,which includes microscopic images of lung and colon cancers;and the BreaKHis dataset,containing benign and malignant breast microscopic images.The lightweight models achieved accuracies of 96.9%for 3-class brain tumor classification using BT-Net,and 99.7%for 5-class lung and colon cancer classification using LCC-Net.For 2-class breast cancer classification,BC-Net achieved an accuracy of 96.7%.The parameter counts for the proposed lightweight models—LCC-Net,BC-Net,and BT-Net—are 0.528,0.226,and 1.154 million,respectively.The presented lightweight models,featuring ResMHCNN blocks,may be effectively employed for accurate medical image classification.In the future,these models might be tested for viability in resource-constrained systems such as mobile devices and IoMT platforms.
基金funded by the Research on Intelligent Mining Geological Model and Ventilation Model for Extremely Thin Coal Seam in Heilongjiang Province,China(2021ZXJ02A03)the Demonstration of Intelligent Mining for Comprehensive Mining Face in Extremely Thin Coal Seam in Heilongjiang Province,China(2021ZXJ02A04)the Natural Science Foundation of Heilongjiang Province,China(LH2024E112).
文摘Coal dust explosions are severe safety accidents in coal mine production,posing significant threats to life and property.Predicting the maximum explosion pressure(Pm)of coal dust using deep learning models can effectively assess potential risks and provide a scientific basis for preventing coal dust explosions.In this study,a 20-L explosion sphere apparatus was used to test the maximum explosion pressure of coal dust under seven different particle sizes and ten mass concentrations(Cdust),resulting in a dataset of 70 experimental groups.Through Spearman correlation analysis and random forest feature selection methods,particle size(D_(10),D_(20),D_(50))and mass concentration(Cdust)were identified as critical feature parameters from the ten initial parameters of the coal dust samples.Based on this,a hybrid Long Short-Term Memory(LSTM)network model incorporating a Multi-Head Attention Mechanism and the Sparrow Search Algorithm(SSA)was proposed to predict the maximum explosion pressure of coal dust.The results demonstrate that the SSA-LSTM-Multi-Head Attention model excels in predicting the maximum explosion pressure of coal dust.The four evaluation metrics indicate that the model achieved a coefficient of determination(R^(2)),root mean square error(RMSE),mean absolute percentage error(MAPE),and mean absolute error(MAE)of 0.9841,0.0030,0.0074,and 0.0049,respectively,in the training set.In the testing set,these values were 0.9743,0.0087,0.0108,and 0.0069,respectively.Compared to artificial neural networks(ANN),random forest(RF),support vector machines(SVM),particle swarm optimized-SVM(PSO-SVM)neural networks,and the traditional single-model LSTM,the SSA-LSTM-Multi-Head Attention model demonstrated superior generalization capability and prediction accuracy.The findings of this study not only advance the application of deep learning in coal dust explosion prediction but also provide robust technical support for the prevention and risk assessment of coal dust explosions.
基金supported by Xiamen Medical and Health Guidance Project in 2021(No.3502Z20214ZD1070)supported by a grant from Guangxi Key Laboratory of Machine Vision and Intelligent Control,China(No.2023B02).
文摘The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships between pixels requires further improvement.Previous methods face challenges in efficiently managing multi-scale fea-tures of different granularities from the encoder backbone,leaving room for improvement in their global representation and feature extraction capabilities.To address these challenges,we propose a novel Decoder with Multi-Head Feature Receptors(DMHFR),which receives multi-scale features from the encoder backbone and organizes them into three feature groups with different granularities:coarse,fine-grained,and full set.These groups are subsequently processed by Multi-Head Feature Receptors(MHFRs)after feature capture and modeling operations.MHFRs include two Three-Head Feature Receptors(THFRs)and one Four-Head Feature Receptor(FHFR).Each group of features is passed through these MHFRs and then fed into axial transformers,which help the model capture long-range dependencies within the features.The three MHFRs produce three distinct feature outputs.The output from the FHFR serves as auxiliary auxiliary features in the prediction head,and the prediction output and their losses will eventually be aggregated.Experimental results show that the Transformer using DMHFR outperforms 15 state of the arts(SOTA)methods on five public datasets.Specifically,it achieved significant improvements in mean DICE scores over the classic Parallel Reverse Attention Network(PraNet)method,with gains of 4.1%,2.2%,1.4%,8.9%,and 16.3%on the CVC-ClinicDB,Kvasir-SEG,CVC-T,CVC-ColonDB,and ETIS-LaribPolypDB datasets,respectively.
基金Supported by the Key R&D Program of Gansu Province(No.23YFGA0063)the Key Talent Project of Gansu Province(No.2024RCXM57,2024RCXM22)the Major Science and Technology Special Program of Gansu Province(No.25ZYJA037).
文摘Traffic flow prediction is a crucial element of intelligent transportation systems.However,accu-rate traffic flow prediction is quite challenging because of its highly nonlinear,complex,and dynam-ic characteristics.To address the difficulties in simultaneously capturing local and global dynamic spatiotemporal correlations in traffic flow,as well as the high time complexity of existing models,a multi-head flow attention-based local-global dynamic hypergraph convolution(MFA-LGDHC)pre-diction model is proposed.which consists of multi-head flow attention(MHFA)mechanism,graph convolution network(GCN),and local-global dynamic hypergraph convolution(LGHC).MHFA is utilized to extract the time dependency of traffic flow and reduce the time complexity of the model.GCN is employed to catch the spatial dependency of traffic flow.LGHC utilizes down-sampling con-volution and isometric convolution to capture the local and global spatial dependencies of traffic flow.And dynamic hypergraph convolution is used to model the dynamic higher-order relationships of the traffic road network.Experimental results indicate that the MFA-LGDHC model outperforms current popular baseline models and exhibits good prediction performance.
文摘Reliable traffic flow prediction is crucial for mitigating urban congestion.This paper proposes Attentionbased spatiotemporal Interactive Dynamic Graph Convolutional Network(AIDGCN),a novel architecture integrating Interactive Dynamic Graph Convolution Network(IDGCN)with Temporal Multi-Head Trend-Aware Attention.Its core innovation lies in IDGCN,which uniquely splits sequences into symmetric intervals for interactive feature sharing via dynamic graphs,and a novel attention mechanism incorporating convolutional operations to capture essential local traffic trends—addressing a critical gap in standard attention for continuous data.For 15-and 60-min forecasting on METR-LA,AIDGCN achieves MAEs of 0.75%and 0.39%,and RMSEs of 1.32%and 0.14%,respectively.In the 60-min long-term forecasting of the PEMS-BAY dataset,the AIDGCN out-performs the MRA-BGCN method by 6.28%,4.93%,and 7.17%in terms of MAE,RMSE,and MAPE,respectively.Experimental results demonstrate the superiority of our pro-posed model over state-of-the-art methods.
基金supported by the National Natural Science Foundation of China(No.51977113)the Science and Technology Project of State Grid Zhejiang Electric Power Co.,Ltd.(No.5211JX240001).
文摘Accurate and reliable power system data are fundamental for critical operations such as gridmonitoring,fault diagnosis,and load forecasting,underpinned by increasing intelligentization and digitalization.However,data loss and anomalies frequently compromise data integrity in practical settings,significantly impacting system operational efficiency and security.Most existing data recovery methods require complete datasets for training,leading to substantial data and computational demands and limited generalization.To address these limitations,this study proposes a missing data imputation model based on an improved Generative Adversarial Network(BAC-GAN).Within the BAC-GAN framework,the generator utilizes Bidirectional Long Short-Term Memory(BiLSTM)networks and Multi-Head Attention mechanisms to capture temporal dependencies and complex relationships within power system data.The discriminator employs a Convolutional Neural Network(CNN)architecture to integrate local features with global structures,effectivelymitigating the generation of implausible imputations.Experimental results on two public datasets demonstrate that the BAC-GAN model achieves superior data recovery accuracy compared to five state-of-the-art and classical benchmarkmethods,with an average improvement of 17.7%in reconstruction accuracy.The proposedmethod significantly enhances the accuracy of grid fault diagnosis and provides reliable data support for the stable operation of smart grids,showing great potential for practical applications in power systems.
基金Under the auspices of the National Key Research and Development Program of China(No.2024YFC3713102)。
文摘Groundwater level(GWL)is a key indicator used to accurately assess groundwater resources and form the foundation for ef-fective groundwater management.This paper integrates a Gate Recurrent Unit(GRU)model with a Multi-head Self-attention mechan-ism(MSAM-GRU)to simulate GWLs in both confined and unconfined aquifers simultaneously.The model innovatively captures the lag times between GWLs in the unconfined aquifer and precipitation,as well as between GWLs in the confined aquifer and the upper aquifer.We have assessed the effectiveness of the proposed model using a case study in the Beijing Plain,China from January 2005 to December 2020.With the consideration of lag times,the results indicated that the MSAM-GRU model exhibits a maximum 67%and 73%reduction in RMSE compared to the Attention mechanism-GRU(AM-GRU)and GRU model,respectively.MSAM-GRU model exhibited a 31%reduction in RMSE and a 0.12 increase in R^(2) compared to the same model that do not account for lag time.In Region I,the shortest lag time of GWL in the unconfined aquifer was two months,while that in the confined aquifer was three months,indicating a longer delayed response in the confined aquifer.MSAM-GRU model considering lag time,was then applied to simulate the GWLs in the unconfined aquifer under different scenarios and to analyze whether GWL fluctuations affect subway operations.The simulation res-ults showed that under the scenario 1,the GWL in the unconfined aquifer would rise above the depth of subway station floor,threaten-ing the operation of subways.This study can provide reliable technical support for the accurate simulation of GWLs in multi-aquifer systems.
