Sarcasm detection is a complex and challenging task,particularly in the context of Chinese social media,where it exhibits strong contextual dependencies and cultural specificity.To address the limitations of existing ...Sarcasm detection is a complex and challenging task,particularly in the context of Chinese social media,where it exhibits strong contextual dependencies and cultural specificity.To address the limitations of existing methods in capturing the implicit semantics and contextual associations in sarcastic expressions,this paper proposes an event-aware model for Chinese sarcasm detection,leveraging a multi-head attention(MHA)mechanism and contrastive learning(CL)strategies.The proposed model employs a dual-path Bidirectional Encoder Representations from Transformers(BERT)encoder to process comment text and event context separately and integrates an MHA mechanism to facilitate deep interactions between the two,thereby capturing multidimensional semantic associations.Additionally,a CL strategy is introduced to enhance feature representation capabilities,further improving the model’s performance in handling class imbalance and complex contextual scenarios.The model achieves state-of-the-art performance on the Chinese sarcasm dataset,with significant improvements in accuracy(79.55%),F1-score(84.22%),and an area under the curve(AUC,84.35%).展开更多
As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as...As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as a new challenge in the field of recommendation systems.This paper introduces a group-buying recommendation model based on multi-head attention mechanisms and multi-task learning,termed the Multi-head Attention Mechanisms and Multi-task Learning Group-Buying Recommendation(MAMGBR)model,specifically designed to optimize group-buying recommendations on e-commerce platforms.The core dataset of this study comes from the Chinese maternal and infant e-commerce platform“Beibei,”encompassing approximately 430,000 successful groupbuying actions and over 120,000 users.Themodel focuses on twomain tasks:recommending items for group organizers(Task Ⅰ)and recommending participants for a given group-buying event(Task Ⅱ).In model evaluation,MAMGBR achieves an MRR@10 of 0.7696 for Task I,marking a 20.23%improvement over baseline models.Furthermore,in Task II,where complex interaction patterns prevail,MAMGBR utilizes auxiliary loss functions to effectively model the multifaceted roles of users,items,and participants,leading to a 24.08%increase in MRR@100 under a 1:99 sample ratio.Experimental results show that compared to benchmark models,such as NGCF and EATNN,MAMGBR’s integration ofmulti-head attentionmechanisms,expert networks,and gating mechanisms enables more accurate modeling of user preferences and social associations within group-buying scenarios,significantly enhancing recommendation accuracy and platform group-buying success rates.展开更多
Safety maintenance of power equipment is of great importance in power grids,in which image-processing-based defect recognition is supposed to classify abnormal conditions during daily inspection.However,owing to the b...Safety maintenance of power equipment is of great importance in power grids,in which image-processing-based defect recognition is supposed to classify abnormal conditions during daily inspection.However,owing to the blurred features of defect images,the current defect recognition algorithm has poor fine-grained recognition ability.Visual attention can achieve fine-grained recognition with its abil-ity to model long-range dependencies while introducing extra computational complexity,especially for multi-head attention in vision transformer structures.Under these circumstances,this paper proposes a self-reduction multi-head attention module that can reduce computational complexity and be easily combined with a Convolutional Neural Network(CNN).In this manner,local and global fea-tures can be calculated simultaneously in our proposed structure,aiming to improve the defect recognition performance.Specifically,the proposed self-reduction multi-head attention can reduce redundant parameters,thereby solving the problem of limited computational resources.Experimental results were obtained based on the defect dataset collected from the substation.The results demonstrated the efficiency and superiority of the proposed method over other advanced algorithms.展开更多
Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract loc...Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract local and global features, as well as the lack of effective mechanisms to capture complex interactions between features;Additionally, when increasing the receptive field to obtain deeper feature representations, the reliance on increasing network depth leads to a significant increase in computational resource consumption, affecting the efficiency and performance of detection. Based on these issues, firstly, this paper proposes a network traffic anomaly detection model based on parallel dilated convolution and residual learning (Res-PDC). To better explore the interactive relationships between features, the traffic samples are converted into two-dimensional matrix. A module combining parallel dilated convolutions and residual learning (res-pdc) was designed to extract local and global features of traffic at different scales. By utilizing res-pdc modules with different dilation rates, we can effectively capture spatial features at different scales and explore feature dependencies spanning wider regions without increasing computational resources. Secondly, to focus and integrate the information in different feature subspaces, further enhance and extract the interactions among the features, multi-head attention is added to Res-PDC, resulting in the final model: multi-head attention enhanced parallel dilated convolution and residual learning (MHA-Res-PDC) for network traffic anomaly detection. Finally, comparisons with other machine learning and deep learning algorithms are conducted on the NSL-KDD and CIC-IDS-2018 datasets. The experimental results demonstrate that the proposed method in this paper can effectively improve the detection performance.展开更多
Lightweight deep learning models are increasingly required in resource-constrained environments such as mobile devices and the Internet of Medical Things(IoMT).Multi-head convolution with channel attention can facilit...Lightweight deep learning models are increasingly required in resource-constrained environments such as mobile devices and the Internet of Medical Things(IoMT).Multi-head convolution with channel attention can facilitate learning activations relevant to different kernel sizes within a multi-head convolutional layer.Therefore,this study investigates the capability of novel lightweight models incorporating residual multi-head convolution with channel attention(ResMHCNN)blocks to classify medical images.We introduced three novel lightweight deep learning models(BT-Net,LCC-Net,and BC-Net)utilizing the ResMHCNN block as their backbone.These models were crossvalidated and tested on three publicly available medical image datasets:a brain tumor dataset from Figshare consisting of T1-weighted magnetic resonance imaging slices of meningioma,glioma,and pituitary tumors;the LC25000 dataset,which includes microscopic images of lung and colon cancers;and the BreaKHis dataset,containing benign and malignant breast microscopic images.The lightweight models achieved accuracies of 96.9%for 3-class brain tumor classification using BT-Net,and 99.7%for 5-class lung and colon cancer classification using LCC-Net.For 2-class breast cancer classification,BC-Net achieved an accuracy of 96.7%.The parameter counts for the proposed lightweight models—LCC-Net,BC-Net,and BT-Net—are 0.528,0.226,and 1.154 million,respectively.The presented lightweight models,featuring ResMHCNN blocks,may be effectively employed for accurate medical image classification.In the future,these models might be tested for viability in resource-constrained systems such as mobile devices and IoMT platforms.展开更多
Coal dust explosions are severe safety accidents in coal mine production,posing significant threats to life and property.Predicting the maximum explosion pressure(Pm)of coal dust using deep learning models can effecti...Coal dust explosions are severe safety accidents in coal mine production,posing significant threats to life and property.Predicting the maximum explosion pressure(Pm)of coal dust using deep learning models can effectively assess potential risks and provide a scientific basis for preventing coal dust explosions.In this study,a 20-L explosion sphere apparatus was used to test the maximum explosion pressure of coal dust under seven different particle sizes and ten mass concentrations(Cdust),resulting in a dataset of 70 experimental groups.Through Spearman correlation analysis and random forest feature selection methods,particle size(D_(10),D_(20),D_(50))and mass concentration(Cdust)were identified as critical feature parameters from the ten initial parameters of the coal dust samples.Based on this,a hybrid Long Short-Term Memory(LSTM)network model incorporating a Multi-Head Attention Mechanism and the Sparrow Search Algorithm(SSA)was proposed to predict the maximum explosion pressure of coal dust.The results demonstrate that the SSA-LSTM-Multi-Head Attention model excels in predicting the maximum explosion pressure of coal dust.The four evaluation metrics indicate that the model achieved a coefficient of determination(R^(2)),root mean square error(RMSE),mean absolute percentage error(MAPE),and mean absolute error(MAE)of 0.9841,0.0030,0.0074,and 0.0049,respectively,in the training set.In the testing set,these values were 0.9743,0.0087,0.0108,and 0.0069,respectively.Compared to artificial neural networks(ANN),random forest(RF),support vector machines(SVM),particle swarm optimized-SVM(PSO-SVM)neural networks,and the traditional single-model LSTM,the SSA-LSTM-Multi-Head Attention model demonstrated superior generalization capability and prediction accuracy.The findings of this study not only advance the application of deep learning in coal dust explosion prediction but also provide robust technical support for the prevention and risk assessment of coal dust explosions.展开更多
The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships ...The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships between pixels requires further improvement.Previous methods face challenges in efficiently managing multi-scale fea-tures of different granularities from the encoder backbone,leaving room for improvement in their global representation and feature extraction capabilities.To address these challenges,we propose a novel Decoder with Multi-Head Feature Receptors(DMHFR),which receives multi-scale features from the encoder backbone and organizes them into three feature groups with different granularities:coarse,fine-grained,and full set.These groups are subsequently processed by Multi-Head Feature Receptors(MHFRs)after feature capture and modeling operations.MHFRs include two Three-Head Feature Receptors(THFRs)and one Four-Head Feature Receptor(FHFR).Each group of features is passed through these MHFRs and then fed into axial transformers,which help the model capture long-range dependencies within the features.The three MHFRs produce three distinct feature outputs.The output from the FHFR serves as auxiliary auxiliary features in the prediction head,and the prediction output and their losses will eventually be aggregated.Experimental results show that the Transformer using DMHFR outperforms 15 state of the arts(SOTA)methods on five public datasets.Specifically,it achieved significant improvements in mean DICE scores over the classic Parallel Reverse Attention Network(PraNet)method,with gains of 4.1%,2.2%,1.4%,8.9%,and 16.3%on the CVC-ClinicDB,Kvasir-SEG,CVC-T,CVC-ColonDB,and ETIS-LaribPolypDB datasets,respectively.展开更多
Reliable traffic flow prediction is crucial for mitigating urban congestion.This paper proposes Attentionbased spatiotemporal Interactive Dynamic Graph Convolutional Network(AIDGCN),a novel architecture integrating In...Reliable traffic flow prediction is crucial for mitigating urban congestion.This paper proposes Attentionbased spatiotemporal Interactive Dynamic Graph Convolutional Network(AIDGCN),a novel architecture integrating Interactive Dynamic Graph Convolution Network(IDGCN)with Temporal Multi-Head Trend-Aware Attention.Its core innovation lies in IDGCN,which uniquely splits sequences into symmetric intervals for interactive feature sharing via dynamic graphs,and a novel attention mechanism incorporating convolutional operations to capture essential local traffic trends—addressing a critical gap in standard attention for continuous data.For 15-and 60-min forecasting on METR-LA,AIDGCN achieves MAEs of 0.75%and 0.39%,and RMSEs of 1.32%and 0.14%,respectively.In the 60-min long-term forecasting of the PEMS-BAY dataset,the AIDGCN out-performs the MRA-BGCN method by 6.28%,4.93%,and 7.17%in terms of MAE,RMSE,and MAPE,respectively.Experimental results demonstrate the superiority of our pro-posed model over state-of-the-art methods.展开更多
Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,...Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.展开更多
Fraud cases have been a risk in society and people’s property security has been greatly threatened.In recent studies,many promising algorithms have been developed for social media offensive text recognition as well a...Fraud cases have been a risk in society and people’s property security has been greatly threatened.In recent studies,many promising algorithms have been developed for social media offensive text recognition as well as sentiment analysis.These algorithms are also suitable for fraudulent phone text recognition.Compared to these tasks,the semantics of fraudulent words are more complex and more difficult to distinguish.Recurrent Neural Networks(RNN),the variants ofRNN,ConvolutionalNeuralNetworks(CNN),and hybrid neural networks to extract text features are used by most text classification research.However,a single network or a simple network combination cannot obtain rich characteristic knowledge of fraudulent phone texts relatively.Therefore,a new model is proposed in this paper.In the fraudulent phone text,the knowledge that can be learned by the model includes the sequence structure of sentences,the correlation between words,the correlation of contextual semantics,the feature of keywords in sentences,etc.The new model combines a bidirectional Long-Short Term Memory Neural Network(BiLSTM)or a bidirectional Gate Recurrent United(BiGRU)and a Multi-Head attention mechanism module with convolution.A normalization layer is added after the output of the final hidden layer.BiLSTM or BiGRU is used to build the encoding and decoding layer.Multi-head attention mechanism module with convolution(MHAC)enhances the ability of the model to learn global interaction information and multi-granularity local interaction information in fraudulent sentences.A fraudulent phone text dataset is produced by us in this paper.The THUCNews data sets and fraudulent phone text data sets are used in experiments.Experiment results show that compared with the baseline model,the proposed model(LMHACL)has the best experiment results in terms of Accuracy,Precision,Recall,and F1 score on the two data sets.And the performance indexes on fraudulent phone text data sets are all above 0.94.展开更多
Due to the time-varying topology and possible disturbances in a conflict environment,it is still challenging to maintain the mission performance of flying Ad hoc networks(FANET),which limits the application of Unmanne...Due to the time-varying topology and possible disturbances in a conflict environment,it is still challenging to maintain the mission performance of flying Ad hoc networks(FANET),which limits the application of Unmanned Aerial Vehicle(UAV)swarms in harsh environments.This paper proposes an intelligent framework to quickly recover the cooperative coveragemission by aggregating the historical spatio-temporal network with the attention mechanism.The mission resilience metric is introduced in conjunction with connectivity and coverage status information to simplify the optimization model.A spatio-temporal node pooling method is proposed to ensure all node location features can be updated after destruction by capturing the temporal network structure.Combined with the corresponding Laplacian matrix as the hyperparameter,a recovery algorithm based on the multi-head attention graph network is designed to achieve rapid recovery.Simulation results showed that the proposed framework can facilitate rapid recovery of the connectivity and coverage more effectively compared to the existing studies.The results demonstrate that the average connectivity and coverage results is improved by 17.92%and 16.96%,respectively compared with the state-of-the-art model.Furthermore,by the ablation study,the contributions of each different improvement are compared.The proposed model can be used to support resilient network design for real-time mission execution.展开更多
Worldwide,many elders are suffering from Alzheimer’s disease(AD).The elders with AD exhibit various abnormalities in their activities,such as sleep disturbances,wandering aimlessly,forgetting activities,etc.,which ar...Worldwide,many elders are suffering from Alzheimer’s disease(AD).The elders with AD exhibit various abnormalities in their activities,such as sleep disturbances,wandering aimlessly,forgetting activities,etc.,which are the strong signs and symptoms of AD progression.Recognizing these symptoms in advance could assist to a quicker diagnosis and treatment and to prevent the progression of Disease to the next stage.The proposed method aims to detect the behavioral abnormalities found in Daily activities of AD patients(ADP)using wearables.In the proposed work,a publicly available dataset collected using wearables is applied.Currently,no real-world data is available to illustrate the daily activities of ADP.Hence,the proposed method has synthesized the wearables data according to the abnormal activities of ADP.In the proposed work,multi-headed(MH)architectures such as MH Convolutional Neural Network-Long Short-Term Mem-ory Network(CNN-LSTM),MH one-dimensional Convolutional Neural Network(1D-CNN)and MH two dimensional Convolutional Neural Network(2D-CNN)as well as conventional methods,namely CNN-LSTM,1D-CNN,2D-CNN have been implemented to model activity pattern.A multi-label prediction technique is applied to detect abnormal activities.The results obtained show that the proposed MH architectures achieve improved performance than the conventional methods.Moreover,the MH models for activity recognition perform better than the abnormality detection.展开更多
基金granted by Qin Xin Talents Cultivation Program(No.QXTCP C202115),Beijing Information Science&Technology Universitythe Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing Fund(No.GJJ-23),National Social Science Foundation,China(No.21BTQ079).
文摘Sarcasm detection is a complex and challenging task,particularly in the context of Chinese social media,where it exhibits strong contextual dependencies and cultural specificity.To address the limitations of existing methods in capturing the implicit semantics and contextual associations in sarcastic expressions,this paper proposes an event-aware model for Chinese sarcasm detection,leveraging a multi-head attention(MHA)mechanism and contrastive learning(CL)strategies.The proposed model employs a dual-path Bidirectional Encoder Representations from Transformers(BERT)encoder to process comment text and event context separately and integrates an MHA mechanism to facilitate deep interactions between the two,thereby capturing multidimensional semantic associations.Additionally,a CL strategy is introduced to enhance feature representation capabilities,further improving the model’s performance in handling class imbalance and complex contextual scenarios.The model achieves state-of-the-art performance on the Chinese sarcasm dataset,with significant improvements in accuracy(79.55%),F1-score(84.22%),and an area under the curve(AUC,84.35%).
基金supported by the Key Research and Development Program of Heilongjiang Province(No.2022ZX01A35).
文摘As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as a new challenge in the field of recommendation systems.This paper introduces a group-buying recommendation model based on multi-head attention mechanisms and multi-task learning,termed the Multi-head Attention Mechanisms and Multi-task Learning Group-Buying Recommendation(MAMGBR)model,specifically designed to optimize group-buying recommendations on e-commerce platforms.The core dataset of this study comes from the Chinese maternal and infant e-commerce platform“Beibei,”encompassing approximately 430,000 successful groupbuying actions and over 120,000 users.Themodel focuses on twomain tasks:recommending items for group organizers(Task Ⅰ)and recommending participants for a given group-buying event(Task Ⅱ).In model evaluation,MAMGBR achieves an MRR@10 of 0.7696 for Task I,marking a 20.23%improvement over baseline models.Furthermore,in Task II,where complex interaction patterns prevail,MAMGBR utilizes auxiliary loss functions to effectively model the multifaceted roles of users,items,and participants,leading to a 24.08%increase in MRR@100 under a 1:99 sample ratio.Experimental results show that compared to benchmark models,such as NGCF and EATNN,MAMGBR’s integration ofmulti-head attentionmechanisms,expert networks,and gating mechanisms enables more accurate modeling of user preferences and social associations within group-buying scenarios,significantly enhancing recommendation accuracy and platform group-buying success rates.
基金supported in part by Major Program of the National Natural Science Foundation of China under Grant 62127803.
文摘Safety maintenance of power equipment is of great importance in power grids,in which image-processing-based defect recognition is supposed to classify abnormal conditions during daily inspection.However,owing to the blurred features of defect images,the current defect recognition algorithm has poor fine-grained recognition ability.Visual attention can achieve fine-grained recognition with its abil-ity to model long-range dependencies while introducing extra computational complexity,especially for multi-head attention in vision transformer structures.Under these circumstances,this paper proposes a self-reduction multi-head attention module that can reduce computational complexity and be easily combined with a Convolutional Neural Network(CNN).In this manner,local and global fea-tures can be calculated simultaneously in our proposed structure,aiming to improve the defect recognition performance.Specifically,the proposed self-reduction multi-head attention can reduce redundant parameters,thereby solving the problem of limited computational resources.Experimental results were obtained based on the defect dataset collected from the substation.The results demonstrated the efficiency and superiority of the proposed method over other advanced algorithms.
基金supported by the Xiamen Science and Technology Subsidy Project(No.2023CXY0318).
文摘Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract local and global features, as well as the lack of effective mechanisms to capture complex interactions between features;Additionally, when increasing the receptive field to obtain deeper feature representations, the reliance on increasing network depth leads to a significant increase in computational resource consumption, affecting the efficiency and performance of detection. Based on these issues, firstly, this paper proposes a network traffic anomaly detection model based on parallel dilated convolution and residual learning (Res-PDC). To better explore the interactive relationships between features, the traffic samples are converted into two-dimensional matrix. A module combining parallel dilated convolutions and residual learning (res-pdc) was designed to extract local and global features of traffic at different scales. By utilizing res-pdc modules with different dilation rates, we can effectively capture spatial features at different scales and explore feature dependencies spanning wider regions without increasing computational resources. Secondly, to focus and integrate the information in different feature subspaces, further enhance and extract the interactions among the features, multi-head attention is added to Res-PDC, resulting in the final model: multi-head attention enhanced parallel dilated convolution and residual learning (MHA-Res-PDC) for network traffic anomaly detection. Finally, comparisons with other machine learning and deep learning algorithms are conducted on the NSL-KDD and CIC-IDS-2018 datasets. The experimental results demonstrate that the proposed method in this paper can effectively improve the detection performance.
基金supported by the Institute of Information&Communications Technology Planning&Evaluation(IITP)-Innovative Human Resource Development for Local Intellectualization program grant funded by the Korea government(MSIT)(IITP-2025-RS-2023-00259678)by INHA UNIVERSITY Research Grant.
文摘Lightweight deep learning models are increasingly required in resource-constrained environments such as mobile devices and the Internet of Medical Things(IoMT).Multi-head convolution with channel attention can facilitate learning activations relevant to different kernel sizes within a multi-head convolutional layer.Therefore,this study investigates the capability of novel lightweight models incorporating residual multi-head convolution with channel attention(ResMHCNN)blocks to classify medical images.We introduced three novel lightweight deep learning models(BT-Net,LCC-Net,and BC-Net)utilizing the ResMHCNN block as their backbone.These models were crossvalidated and tested on three publicly available medical image datasets:a brain tumor dataset from Figshare consisting of T1-weighted magnetic resonance imaging slices of meningioma,glioma,and pituitary tumors;the LC25000 dataset,which includes microscopic images of lung and colon cancers;and the BreaKHis dataset,containing benign and malignant breast microscopic images.The lightweight models achieved accuracies of 96.9%for 3-class brain tumor classification using BT-Net,and 99.7%for 5-class lung and colon cancer classification using LCC-Net.For 2-class breast cancer classification,BC-Net achieved an accuracy of 96.7%.The parameter counts for the proposed lightweight models—LCC-Net,BC-Net,and BT-Net—are 0.528,0.226,and 1.154 million,respectively.The presented lightweight models,featuring ResMHCNN blocks,may be effectively employed for accurate medical image classification.In the future,these models might be tested for viability in resource-constrained systems such as mobile devices and IoMT platforms.
基金funded by the Research on Intelligent Mining Geological Model and Ventilation Model for Extremely Thin Coal Seam in Heilongjiang Province,China(2021ZXJ02A03)the Demonstration of Intelligent Mining for Comprehensive Mining Face in Extremely Thin Coal Seam in Heilongjiang Province,China(2021ZXJ02A04)the Natural Science Foundation of Heilongjiang Province,China(LH2024E112).
文摘Coal dust explosions are severe safety accidents in coal mine production,posing significant threats to life and property.Predicting the maximum explosion pressure(Pm)of coal dust using deep learning models can effectively assess potential risks and provide a scientific basis for preventing coal dust explosions.In this study,a 20-L explosion sphere apparatus was used to test the maximum explosion pressure of coal dust under seven different particle sizes and ten mass concentrations(Cdust),resulting in a dataset of 70 experimental groups.Through Spearman correlation analysis and random forest feature selection methods,particle size(D_(10),D_(20),D_(50))and mass concentration(Cdust)were identified as critical feature parameters from the ten initial parameters of the coal dust samples.Based on this,a hybrid Long Short-Term Memory(LSTM)network model incorporating a Multi-Head Attention Mechanism and the Sparrow Search Algorithm(SSA)was proposed to predict the maximum explosion pressure of coal dust.The results demonstrate that the SSA-LSTM-Multi-Head Attention model excels in predicting the maximum explosion pressure of coal dust.The four evaluation metrics indicate that the model achieved a coefficient of determination(R^(2)),root mean square error(RMSE),mean absolute percentage error(MAPE),and mean absolute error(MAE)of 0.9841,0.0030,0.0074,and 0.0049,respectively,in the training set.In the testing set,these values were 0.9743,0.0087,0.0108,and 0.0069,respectively.Compared to artificial neural networks(ANN),random forest(RF),support vector machines(SVM),particle swarm optimized-SVM(PSO-SVM)neural networks,and the traditional single-model LSTM,the SSA-LSTM-Multi-Head Attention model demonstrated superior generalization capability and prediction accuracy.The findings of this study not only advance the application of deep learning in coal dust explosion prediction but also provide robust technical support for the prevention and risk assessment of coal dust explosions.
基金supported by Xiamen Medical and Health Guidance Project in 2021(No.3502Z20214ZD1070)supported by a grant from Guangxi Key Laboratory of Machine Vision and Intelligent Control,China(No.2023B02).
文摘The self-attention mechanism of Transformers,which captures long-range contextual information,has demonstrated significant potential in image segmentation.However,their ability to learn local,contextual relationships between pixels requires further improvement.Previous methods face challenges in efficiently managing multi-scale fea-tures of different granularities from the encoder backbone,leaving room for improvement in their global representation and feature extraction capabilities.To address these challenges,we propose a novel Decoder with Multi-Head Feature Receptors(DMHFR),which receives multi-scale features from the encoder backbone and organizes them into three feature groups with different granularities:coarse,fine-grained,and full set.These groups are subsequently processed by Multi-Head Feature Receptors(MHFRs)after feature capture and modeling operations.MHFRs include two Three-Head Feature Receptors(THFRs)and one Four-Head Feature Receptor(FHFR).Each group of features is passed through these MHFRs and then fed into axial transformers,which help the model capture long-range dependencies within the features.The three MHFRs produce three distinct feature outputs.The output from the FHFR serves as auxiliary auxiliary features in the prediction head,and the prediction output and their losses will eventually be aggregated.Experimental results show that the Transformer using DMHFR outperforms 15 state of the arts(SOTA)methods on five public datasets.Specifically,it achieved significant improvements in mean DICE scores over the classic Parallel Reverse Attention Network(PraNet)method,with gains of 4.1%,2.2%,1.4%,8.9%,and 16.3%on the CVC-ClinicDB,Kvasir-SEG,CVC-T,CVC-ColonDB,and ETIS-LaribPolypDB datasets,respectively.
文摘Reliable traffic flow prediction is crucial for mitigating urban congestion.This paper proposes Attentionbased spatiotemporal Interactive Dynamic Graph Convolutional Network(AIDGCN),a novel architecture integrating Interactive Dynamic Graph Convolution Network(IDGCN)with Temporal Multi-Head Trend-Aware Attention.Its core innovation lies in IDGCN,which uniquely splits sequences into symmetric intervals for interactive feature sharing via dynamic graphs,and a novel attention mechanism incorporating convolutional operations to capture essential local traffic trends—addressing a critical gap in standard attention for continuous data.For 15-and 60-min forecasting on METR-LA,AIDGCN achieves MAEs of 0.75%and 0.39%,and RMSEs of 1.32%and 0.14%,respectively.In the 60-min long-term forecasting of the PEMS-BAY dataset,the AIDGCN out-performs the MRA-BGCN method by 6.28%,4.93%,and 7.17%in terms of MAE,RMSE,and MAPE,respectively.Experimental results demonstrate the superiority of our pro-posed model over state-of-the-art methods.
基金Supported by National Natural Science Foundation of China(Grant Nos.52005103,71801046,51775112,51975121)Guangdong Province Basic and Applied Basic Research Foundation of China(Grant No.2019B1515120095)+1 种基金Intelligent Manufacturing PHM Innovation Team Program(Grant Nos.2018KCXTD029,TDYB2019010)MoST International Cooperation Program(6-14).
文摘Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.
基金This researchwas funded by the Major Science and Technology Innovation Project of Shandong Province in China(2019JZZY010120).
文摘Fraud cases have been a risk in society and people’s property security has been greatly threatened.In recent studies,many promising algorithms have been developed for social media offensive text recognition as well as sentiment analysis.These algorithms are also suitable for fraudulent phone text recognition.Compared to these tasks,the semantics of fraudulent words are more complex and more difficult to distinguish.Recurrent Neural Networks(RNN),the variants ofRNN,ConvolutionalNeuralNetworks(CNN),and hybrid neural networks to extract text features are used by most text classification research.However,a single network or a simple network combination cannot obtain rich characteristic knowledge of fraudulent phone texts relatively.Therefore,a new model is proposed in this paper.In the fraudulent phone text,the knowledge that can be learned by the model includes the sequence structure of sentences,the correlation between words,the correlation of contextual semantics,the feature of keywords in sentences,etc.The new model combines a bidirectional Long-Short Term Memory Neural Network(BiLSTM)or a bidirectional Gate Recurrent United(BiGRU)and a Multi-Head attention mechanism module with convolution.A normalization layer is added after the output of the final hidden layer.BiLSTM or BiGRU is used to build the encoding and decoding layer.Multi-head attention mechanism module with convolution(MHAC)enhances the ability of the model to learn global interaction information and multi-granularity local interaction information in fraudulent sentences.A fraudulent phone text dataset is produced by us in this paper.The THUCNews data sets and fraudulent phone text data sets are used in experiments.Experiment results show that compared with the baseline model,the proposed model(LMHACL)has the best experiment results in terms of Accuracy,Precision,Recall,and F1 score on the two data sets.And the performance indexes on fraudulent phone text data sets are all above 0.94.
基金the National Natural Science Foundation of China(NNSFC)(Grant Nos.72001213 and 72301292)the National Social Science Fund of China(Grant No.19BGL297)the Basic Research Program of Natural Science in Shaanxi Province(Grant No.2021JQ-369).
文摘Due to the time-varying topology and possible disturbances in a conflict environment,it is still challenging to maintain the mission performance of flying Ad hoc networks(FANET),which limits the application of Unmanned Aerial Vehicle(UAV)swarms in harsh environments.This paper proposes an intelligent framework to quickly recover the cooperative coveragemission by aggregating the historical spatio-temporal network with the attention mechanism.The mission resilience metric is introduced in conjunction with connectivity and coverage status information to simplify the optimization model.A spatio-temporal node pooling method is proposed to ensure all node location features can be updated after destruction by capturing the temporal network structure.Combined with the corresponding Laplacian matrix as the hyperparameter,a recovery algorithm based on the multi-head attention graph network is designed to achieve rapid recovery.Simulation results showed that the proposed framework can facilitate rapid recovery of the connectivity and coverage more effectively compared to the existing studies.The results demonstrate that the average connectivity and coverage results is improved by 17.92%and 16.96%,respectively compared with the state-of-the-art model.Furthermore,by the ablation study,the contributions of each different improvement are compared.The proposed model can be used to support resilient network design for real-time mission execution.
文摘Worldwide,many elders are suffering from Alzheimer’s disease(AD).The elders with AD exhibit various abnormalities in their activities,such as sleep disturbances,wandering aimlessly,forgetting activities,etc.,which are the strong signs and symptoms of AD progression.Recognizing these symptoms in advance could assist to a quicker diagnosis and treatment and to prevent the progression of Disease to the next stage.The proposed method aims to detect the behavioral abnormalities found in Daily activities of AD patients(ADP)using wearables.In the proposed work,a publicly available dataset collected using wearables is applied.Currently,no real-world data is available to illustrate the daily activities of ADP.Hence,the proposed method has synthesized the wearables data according to the abnormal activities of ADP.In the proposed work,multi-headed(MH)architectures such as MH Convolutional Neural Network-Long Short-Term Mem-ory Network(CNN-LSTM),MH one-dimensional Convolutional Neural Network(1D-CNN)and MH two dimensional Convolutional Neural Network(2D-CNN)as well as conventional methods,namely CNN-LSTM,1D-CNN,2D-CNN have been implemented to model activity pattern.A multi-label prediction technique is applied to detect abnormal activities.The results obtained show that the proposed MH architectures achieve improved performance than the conventional methods.Moreover,the MH models for activity recognition perform better than the abnormality detection.
