Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global featu...Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global feature.The transformer can extract the global information well but adapting it to small medical datasets is challenging and its computational complexity can be heavy.In this work,a serial and parallel network is proposed for the accurate 3D medical image segmentation by combining CNN and transformer and promoting feature interactions across various semantic levels.The core components of the proposed method include the cross window self-attention based transformer(CWST)and multi-scale local enhanced(MLE)modules.The CWST module enhances the global context understanding by partitioning 3D images into non-overlapping windows and calculating sparse global attention between windows.The MLE module selectively fuses features by computing the voxel attention between different branch features,and uses convolution to strengthen the dense local information.The experiments on the prostate,atrium,and pancreas MR/CT image datasets consistently demonstrate the advantage of the proposed method over six popular segmentation models in both qualitative evaluation and quantitative indexes such as dice similarity coefficient,Intersection over Union,95%Hausdorff distance and average symmetric surface distance.展开更多
As the complexity of scientific satellite missions increases,the requirements for their magnetic fields,magnetic field fluctuations,and even magnetic field gradients and variations become increasingly stringent.Additi...As the complexity of scientific satellite missions increases,the requirements for their magnetic fields,magnetic field fluctuations,and even magnetic field gradients and variations become increasingly stringent.Additionally,there is a growing need to address the alternating magnetic fields produced by the spacecraft itself.This paper introduces a novel modeling method for spacecraft magnetic dipoles using an integrated self-attention mechanism and a transformer combined with Kolmogorov-Arnold Networks.The self-attention mechanism captures correlations among globally sparse data,establishing dependencies b.etween sparse magnetometer readings.Concurrently,the Kolmogorov-Arnold Network,proficient in modeling implicit numerical relationships between data features,enhances the ability to learn subtle patterns.Comparative experiments validate the capability of the proposed method to precisely model magnetic dipoles,achieving maximum Root Mean Square Errors of 24.06 mA·m^(2)and 0.32 cm for size and location modeling,respectively.The spacecraft magnetic model established using this method accurately computes magnetic fields and alternating magnetic fields at designated surfaces or points.This approach facilitates the rapid and precise construction of individual and complete spacecraft magnetic models,enabling the verification of magnetic specifications from the spacecraft design phase.展开更多
The development of deep learning has made non-biochemical methods for molecular property prediction screening a reality,which can increase the experimental speed and reduce the experimental cost of relevant experiment...The development of deep learning has made non-biochemical methods for molecular property prediction screening a reality,which can increase the experimental speed and reduce the experimental cost of relevant experiments.There are currently two main approaches to representing molecules:(a)representing molecules by fixing molecular descriptors,and(b)representing molecules by graph convolutional neural networks.Currently,both of these Representative methods have achieved some results in their respective experiments.Based on past efforts,we propose a Dual Self-attention Fusion Message Neural Network(DSFMNN).DSFMNN uses a combination of dual self-attention mechanism and graph convolutional neural network.Advantages of DSFMNN:(1)The dual self-attention mechanism focuses not only on the relationship between individual subunits in a molecule but also on the relationship between the atoms and chemical bonds contained in each subunit.(2)On the directed molecular graph,a message delivery approach centered on directed molecular bonds is used.We test the performance of the model on eight publicly available datasets and compare the performance with several models.Based on the current experimental results,DSFMNN has superior performance compared to previous models on the datasets applied in this paper.展开更多
Located in northern China,the Hetao Plain is an important agro-economic zone and population centre.The deterioration of local groundwater quality has had a serious impact on human health and economic development.Nowad...Located in northern China,the Hetao Plain is an important agro-economic zone and population centre.The deterioration of local groundwater quality has had a serious impact on human health and economic development.Nowadays,the groundwater vulnerability assessment(GVA)has become an essential task to identify the current status and development trend of groundwater quality.In this study,the Convolutional Neural Network(CNN)and Long Short-Term Memory(LSTM)models are integrated to realize the spatio-temporal prediction of regional groundwater vulnerability by introducing the Self-attention mechanism.The study firstly builds the CNN-LSTM modelwith self-attention(SA)mechanism and evaluates the prediction accuracy of the model for groundwater vulnerability compared to other common machine learning models such as Support Vector Machine(SVM),Random Forest(RF),and Extreme Gradient Boosting(XGBoost).The results indicate that the CNNLSTM model outperforms thesemodels,demonstrating its significance in groundwater vulnerability assessment.It can be posited that the predictions indicate an increased risk of groundwater vulnerability in the study area over the coming years.This increase can be attributed to the synergistic impact of global climate anomalies and intensified local human activities.Moreover,the overall groundwater vulnerability risk in the entire region has increased,evident fromboth the notably high value and standard deviation.This suggests that the spatial variability of groundwater vulnerability in the area is expected to expand in the future due to the sustained progression of climate change and human activities.The model can be optimized for diverse applications across regional environmental assessment,pollution prediction,and risk statistics.This study holds particular significance for ecological protection and groundwater resource management.展开更多
The satellite-terrestrial networks possess the ability to transcend geographical constraints inherent in traditional communication networks,enabling global coverage and offering users ubiquitous computing power suppor...The satellite-terrestrial networks possess the ability to transcend geographical constraints inherent in traditional communication networks,enabling global coverage and offering users ubiquitous computing power support,which is an important development direction of future communications.In this paper,we take into account a multi-scenario network model under the coverage of low earth orbit(LEO)satellite,which can provide computing resources to users in faraway areas to improve task processing efficiency.However,LEO satellites experience limitations in computing and communication resources and the channels are time-varying and complex,which makes the extraction of state information a daunting task.Therefore,we explore the dynamic resource management issue pertaining to joint computing,communication resource allocation and power control for multi-access edge computing(MEC).In order to tackle this formidable issue,we undertake the task of transforming the issue into a Markov decision process(MDP)problem and propose the self-attention based dynamic resource management(SABDRM)algorithm,which effectively extracts state information features to enhance the training process.Simulation results show that the proposed algorithm is capable of effectively reducing the long-term average delay and energy consumption of the tasks.展开更多
The Ultra-Wideband(UWB)Location-Based Service is receiving more and more attention due to its high ranging accuracy and good time resolution.However,the None-Line-of-Sight(NLOS)propagation may reduce the ranging accur...The Ultra-Wideband(UWB)Location-Based Service is receiving more and more attention due to its high ranging accuracy and good time resolution.However,the None-Line-of-Sight(NLOS)propagation may reduce the ranging accuracy for UWB localization system in indoor environment.So it is important to identify LOS and NLOS propagations before taking proper measures to improve the UWB localization accuracy.In this paper,a deep learning-based UWB NLOS/LOS classification algorithm called FCN-Attention is proposed.The proposed FCN-Attention algorithm utilizes a Fully Convolution Network(FCN)for improving feature extraction ability and a self-attention mechanism for enhancing feature description from the data to improve the classification accuracy.The proposed algorithm is evaluated using an open-source dataset,a local collected dataset and a mixed dataset created from these two datasets.The experiment result shows that the proposed FCN-Attention algorithm achieves classification accuracy of 88.24%on the open-source dataset,100%on the local collected dataset and 92.01%on the mixed dataset,which is better than the results from other evaluated NLOS/LOS classification algorithms in most scenarios in this paper.展开更多
Currently,most trains are equipped with dedicated cameras for capturing pantograph videos.Pantographs are core to the high-speed-railway pantograph-catenary system,and their failure directly affects the normal operati...Currently,most trains are equipped with dedicated cameras for capturing pantograph videos.Pantographs are core to the high-speed-railway pantograph-catenary system,and their failure directly affects the normal operation of high-speed trains.However,given the complex and variable real-world operational conditions of high-speed railways,there is no real-time and robust pantograph fault-detection method capable of handling large volumes of surveillance video.Hence,it is of paramount importance to maintain real-time monitoring and analysis of pantographs.Our study presents a real-time intelligent detection technology for identifying faults in high-speed railway pantographs,utilizing a fusion of self-attention and convolution features.We delved into lightweight multi-scale feature-extraction and fault-detection models based on deep learning to detect pantograph anomalies.Compared with traditional methods,this approach achieves high recall and accuracy in pantograph recognition,accurately pinpointing issues like discharge sparks,pantograph horns,and carbon pantograph-slide malfunctions.After experimentation and validation with actual surveillance videos of electric multiple-unit train,our algorithmic model demonstrates real-time,high-accuracy performance even under complex operational conditions.展开更多
Industrial Internet of Things(IIoT)is a pervasive network of interlinked smart devices that provide a variety of intelligent computing services in industrial environments.Several IIoT nodes operate confidential data(s...Industrial Internet of Things(IIoT)is a pervasive network of interlinked smart devices that provide a variety of intelligent computing services in industrial environments.Several IIoT nodes operate confidential data(such as medical,transportation,military,etc.)which are reachable targets for hostile intruders due to their openness and varied structure.Intrusion Detection Systems(IDS)based on Machine Learning(ML)and Deep Learning(DL)techniques have got significant attention.However,existing ML and DL-based IDS still face a number of obstacles that must be overcome.For instance,the existing DL approaches necessitate a substantial quantity of data for effective performance,which is not feasible to run on low-power and low-memory devices.Imbalanced and fewer data potentially lead to low performance on existing IDS.This paper proposes a self-attention convolutional neural network(SACNN)architecture for the detection of malicious activity in IIoT networks and an appropriate feature extraction method to extract the most significant features.The proposed architecture has a self-attention layer to calculate the input attention and convolutional neural network(CNN)layers to process the assigned attention features for prediction.The performance evaluation of the proposed SACNN architecture has been done with the Edge-IIoTset and X-IIoTID datasets.These datasets encompassed the behaviours of contemporary IIoT communication protocols,the operations of state-of-the-art devices,various attack types,and diverse attack scenarios.展开更多
This study introduces the Smart Exponential-Threshold-Linear with Double Deep Q-learning Network(SETL-DDQN)and an extended Gumbel distribution method,designed to optimize the Contention Window(CW)in IEEE 802.11 networ...This study introduces the Smart Exponential-Threshold-Linear with Double Deep Q-learning Network(SETL-DDQN)and an extended Gumbel distribution method,designed to optimize the Contention Window(CW)in IEEE 802.11 networks.Unlike conventional Deep Reinforcement Learning(DRL)-based approaches for CW size adjustment,which often suffer from overestimation bias and limited exploration diversity,leading to suboptimal throughput and collision performance.Our framework integrates the Gumbel distribution and extreme value theory to systematically enhance action selection under varying network conditions.First,SETL adopts a DDQN architecture(SETL-DDQN)to improve Q-value estimation accuracy and enhance training stability.Second,we incorporate a Gumbel distribution-driven exploration mechanism,forming SETL-DDQN(Gumbel),which employs the extreme value theory to promote diverse action selection,replacing the conventional-greedy exploration that undergoes early convergence to suboptimal solutions.Both models are evaluated through extensive simulations in static and time-varying IEEE 802.11 network scenarios.The results demonstrate that our approach consistently achieves higher throughput,lower collision rates,and improved adaptability,even under abrupt fluctuations in traffic load and network conditions.In particular,the Gumbel-based mechanism enhances the balance between exploration and exploitation,facilitating faster adaptation to varying congestion levels.These findings position Gumbel-enhanced DRL as an effective and robust solution for CW optimization in wireless networks,offering notable gains in efficiency and reliability over existing methods.展开更多
Early and timely diagnosis of stroke is critical for effective treatment,and the electroencephalogram(EEG)offers a low-cost,non-invasive solution.However,the shortage of high-quality patient EEG data often hampers the...Early and timely diagnosis of stroke is critical for effective treatment,and the electroencephalogram(EEG)offers a low-cost,non-invasive solution.However,the shortage of high-quality patient EEG data often hampers the accuracy of diagnostic classification methods based on deep learning.To address this issue,our study designed a deep data amplification model named Progressive Conditional Generative Adversarial Network with Efficient Approximating Self Attention(PCGAN-EASA),which incrementally improves the quality of generated EEG features.This network can yield full-scale,fine-grained EEG features from the low-scale,coarse ones.Specially,to overcome the limitations of traditional generative models that fail to generate features tailored to individual patient characteristics,we developed an encoder with an effective approximating self-attention mechanism.This encoder not only automatically extracts relevant features across different patients but also reduces the computational resource consumption.Furthermore,the adversarial loss and reconstruction loss functions were redesigned to better align with the training characteristics of the network and the spatial correlations among electrodes.Extensive experimental results demonstrate that PCGAN-EASA provides the highest generation quality and the lowest computational resource usage compared to several existing approaches.Additionally,it significantly improves the accuracy of subsequent stroke classification tasks.展开更多
In the application of aerial target recognition,on the one hand,the recognition error produced by the single measurement of the sensor is relatively large due to the impact of noise.On the other hand,it is difficult t...In the application of aerial target recognition,on the one hand,the recognition error produced by the single measurement of the sensor is relatively large due to the impact of noise.On the other hand,it is difficult to apply machine learning methods to improve the intelligence and recognition effect due to few or no actual measurement samples.Aiming at these problems,an aerial target recognition algorithm based on self-attention and Long Short-Term Memory Network(LSTM)is proposed.LSTM can effectively extract temporal dependencies.The attention mechanism calculates the weight of each input element and applies the weight to the hidden state of the LSTM,thereby adjusting the LSTM’s attention to the input.This combination retains the learning ability of LSTM and introduces the advantages of the attention mechanism,making the model have stronger feature extraction ability and adaptability when processing sequence data.In addition,based on the prior information of the multidimensional characteristics of the target,the three-point estimation method is adopted to simulate an aerial target recognition dataset to train the recognition model.The experimental results show that the proposed algorithm achieves more than 91%recognition accuracy,lower false alarm rate and higher robustness compared with the multi-attribute decision-making(MADM)based on fuzzy numbers.展开更多
On Twitter,people often use hashtags to mark the subject of a tweet.Tweets have specific themes or content that are easy for people to manage.With the increase in the number of tweets,how to automatically recommend ha...On Twitter,people often use hashtags to mark the subject of a tweet.Tweets have specific themes or content that are easy for people to manage.With the increase in the number of tweets,how to automatically recommend hashtags for tweets has received wide attention.The previous hashtag recommendation methods were to convert the task into a multi-class classification problem.However,these methods can only recommend hashtags that appeared in historical information,and cannot recommend the new ones.In this work,we extend the self-attention mechanism to turn the hashtag recommendation task into a sequence labeling task.To train and evaluate the proposed method,we used the real tweet data which is collected from Twitter.Experimental results show that the proposed method can be significantly better than the most advanced method.Compared with the state-of-the-art methods,the accuracy of our method has been increased 4%.展开更多
Infrared image recognition plays an important role in the inspection of power equipment.Existing technologies dedicated to this purpose often require manually selected features,which are not transferable and interpret...Infrared image recognition plays an important role in the inspection of power equipment.Existing technologies dedicated to this purpose often require manually selected features,which are not transferable and interpretable,and have limited training data.To address these limitations,this paper proposes an automatic infrared image recognition framework,which includes an object recognition module based on a deep self-attention network and a temperature distribution identification module based on a multi-factor similarity calculation.First,the features of an input image are extracted and embedded using a multi-head attention encoding-decoding mechanism.Thereafter,the embedded features are used to predict the equipment component category and location.In the located area,preliminary segmentation is performed.Finally,similar areas are gradually merged,and the temperature distribution of the equipment is obtained to identify a fault.Our experiments indicate that the proposed method demonstrates significantly improved accuracy compared with other related methods and,hence,provides a good reference for the automation of power equipment inspection.展开更多
Aiming at the problem of insufficient consideration of the correlation between components in the prediction of the remaining life of mechanical equipment,the method of remaining life prediction that combines the self-...Aiming at the problem of insufficient consideration of the correlation between components in the prediction of the remaining life of mechanical equipment,the method of remaining life prediction that combines the self-attention mechanism with the long short-term memory neural network(LSTM-NN)is proposed,called Self-Attention-LSTM.First,the auto-encoder is used to obtain the component-level state information;second,the state information of each component is input into the self-attention mechanism to learn the correlation between components;then,the multi-component correlation matrix is added to the LSTM input gate,and the LSTM-NN is used for life prediction.Finally,combined with the commercial modular aero-propulsion system simulation data set(C-MAPSS),the experiment was carried out and compared with the existing methods.Research results show that the proposed method can achieve better prediction accuracy and verify the feasibility of the method.展开更多
Relation extraction is an important task in NLP community.However,some models often fail in capturing Long-distance dependence on semantics,and the interaction between semantics of two entities is ignored.In this pape...Relation extraction is an important task in NLP community.However,some models often fail in capturing Long-distance dependence on semantics,and the interaction between semantics of two entities is ignored.In this paper,we propose a novel neural network model for semantic relation classification called joint self-attention bi-LSTM(SA-Bi-LSTM)to model the internal structure of the sentence to obtain the importance of each word of the sentence without relying on additional information,and capture Long-distance dependence on semantics.We conduct experiments using the SemEval-2010 Task 8 dataset.Extensive experiments and the results demonstrated that the proposed method is effective against relation classification,which can obtain state-ofthe-art classification accuracy just with minimal feature engineering.展开更多
Background A crucial element of human-machine interaction,the automatic detection of emotional states from human speech has long been regarded as a challenging task for machine learning models.One vital challenge in s...Background A crucial element of human-machine interaction,the automatic detection of emotional states from human speech has long been regarded as a challenging task for machine learning models.One vital challenge in speech emotion recognition(SER)is learning robust and discriminative representations from speech.Although machine learning methods have been widely applied in SER research,the inadequate amount of available annotated data has become a bottleneck impeding the extended application of such techniques(e.g.,deep neural networks).To address this issue,we present a deep learning method that combines knowledge transfer and self-attention for SER tasks.Herein,we apply the log-Mel spectrogram with deltas and delta-deltas as inputs.Moreover,given that emotions are time dependent,we apply temporal convolutional neural networks to model the variations in emotions.We further introduce an attention transfer mechanism,which is based on a self-attention algorithm to learn long-term dependencies.The self-attention transfer network(SATN)in our proposed approach takes advantage of attention transfer to learn attention from speech recognition,followed by transferring this knowledge into SER.An evaluation built on Interactive Emotional Dyadic Motion Capture(IEMOCAP)dataset demonstrates the effectiveness of the proposed model.展开更多
Objective N6-methyladenosine(m6A),the most prevalent epigenetic modification in eukaryotic RNA,plays a pivotal role in regulating cellular differentiation and developmental processes,with its dysregulation implicated ...Objective N6-methyladenosine(m6A),the most prevalent epigenetic modification in eukaryotic RNA,plays a pivotal role in regulating cellular differentiation and developmental processes,with its dysregulation implicated in diverse pathological conditions.Accurate prediction of m6A sites is critical for elucidating their regulatory mechanisms and informing drug development.However,traditional experimental methods are time-consuming and costly.Although various computational approaches have been proposed,challenges remain in feature learning,predictive accuracy,and generalization.Here,we present m6A-PSRA,a dual-branch residual-network-based predictor that fully exploits RNA sequence information to enhance prediction performance and model generalization.Methods m6A-PSRA adopts a parallel dual-branch network architecture to comprehensively extract RNA sequence features via two independent pathways.The first branch applies one-hot encoding to transform the RNA sequence into a numerical matrix while strictly preserving positional information and sequence continuity.This ensures that the biological context conveyed by nucleotide order is retained.A bidirectional long short-term memory network(BiLSTM)then processes the encoded matrix,capturing both forward and backward dependencies between bases to resolve contextual correlations.The second branch employs a k-mer tokenization strategy(k=3),decomposing the sequence into overlapping 3-mer subsequences to capture local sequence patterns.A pre-trained Doc2vec model maps these subsequences into fixeddimensional vectors,reducing feature dimensionality while extracting latent global semantic information via context learning.Both branches integrate residual networks(ResNet)and a self-attention mechanism:ResNet mitigates vanishing gradients through skip connections,preserving feature integrity,while self-attention adaptively assigns weights to focus on sequence regions most relevant to methylation prediction.This synergy enhances both feature learning and generalization capability.Results Across 11 tissues from humans,mice,and rats,m6A-PSRA consistently outperformed existing methods in accuracy(ACC)and area under the curve(AUC),achieving>90%ACC and>95%AUC in every tissue tested,indicating strong cross-species and cross-tissue adaptability.Validation on independent datasets—including three human cell lines(MOLM1,HEK293,A549)and a long-sequence dataset(m6A_IND,1001 nt)—confirmed stable performance across varied biological contexts and sequence lengths.Ablation studies demonstrated that the dual-branch architecture,residual network,and self-attention mechanism each contribute critically to performance,with their combination reducing interference between pathways.Motif analysis revealed an enrichment of m6A sites in guanine(G)and cytosine(C),consistent with known regulatory patterns,supporting the model’s biological plausibility.Conclusion m6A-PSRA effectively captures RNA sequence features,achieving high prediction accuracy and robust generalization across tissues and species,providing an efficient computational tool for m6A methylation site prediction.展开更多
Traditional based deep learning intrusion detection methods face problems such as insufficient cloud storage,data privacy leaks,high com-munication costs,unsatisfactory detection rates,and false positive rate.To addre...Traditional based deep learning intrusion detection methods face problems such as insufficient cloud storage,data privacy leaks,high com-munication costs,unsatisfactory detection rates,and false positive rate.To address existing issues in intrusion detection,this paper presents a novel approach called CS-FL,which combines Federated Learning and a Self-Attention Fusion Convolutional Neural Network.Federated Learning is a new distributed computing model that enables individual training of client data without uploading local data to a central server.at the same time,local training results are uploaded and integrated across all participating clients to produce a global model.The sharing model reduces communication costs,protects data privacy,and solves problems such as insufficient cloud storage and“data islands”for each client.In the proposed method,a hybrid model is formed by integrating the self-Attention and similar parts of the Convolutional Neural Network in the local data processing.This approach not only enhances the performance of the hybrid model but also reduces computational overhead compared to pure hybrid neural networks.Results from experiments on the NSL-KDD dataset show that the proposed method outperforms other intrusion detection techniques,resulting in a significant improvement in performance.This demonstrates the effectiveness of the proposed approach in improving intrusion detection accuracy.展开更多
We extract some physical and chemical features re-lated to the occurrence of single nucleotide polymorphism （SNP） from three groups of sliding windows around SNP site,and then make the predictions about accuracy by ...We extract some physical and chemical features re-lated to the occurrence of single nucleotide polymorphism （SNP） from three groups of sliding windows around SNP site,and then make the predictions about accuracy by using radial basis function （RBF） networks. The result of the forward sliding windows sug-gests that the accuracies and Matthews correlation coefficient （MCC values） ascend with the increasing of length of sliding windows. The accuracies range from 73.27 % to 80.69 %,and MCC values range from 0.465 to 0.614. The backward sliding windows and the sliding windows with fixed length three are de-signed to find the crucial sites related to SNP. The results imply that the occurrence possibility of SNP relies heavily on the above physical and chemical features of sites which are at a distance around 20 bases from the SNP site. Compared with the support vector machine （SVM）,our RBF network approach has achieved more satisfactory results.展开更多
基金National Key Research and Development Program of China,Grant/Award Number:2018YFE0206900China Postdoctoral Science Foundation,Grant/Award Number:2023M731204+2 种基金The Open Project of Key Laboratory for Quality Evaluation of Ultrasound Surgical Equipment of National Medical Products Administration,Grant/Award Number:SMDTKL-2023-1-01The Hubei Province Key Research and Development Project,Grant/Award Number:2023BCB007CAAI-Huawei MindSpore Open Fund。
文摘Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global feature.The transformer can extract the global information well but adapting it to small medical datasets is challenging and its computational complexity can be heavy.In this work,a serial and parallel network is proposed for the accurate 3D medical image segmentation by combining CNN and transformer and promoting feature interactions across various semantic levels.The core components of the proposed method include the cross window self-attention based transformer(CWST)and multi-scale local enhanced(MLE)modules.The CWST module enhances the global context understanding by partitioning 3D images into non-overlapping windows and calculating sparse global attention between windows.The MLE module selectively fuses features by computing the voxel attention between different branch features,and uses convolution to strengthen the dense local information.The experiments on the prostate,atrium,and pancreas MR/CT image datasets consistently demonstrate the advantage of the proposed method over six popular segmentation models in both qualitative evaluation and quantitative indexes such as dice similarity coefficient,Intersection over Union,95%Hausdorff distance and average symmetric surface distance.
基金supported by the National Key Research and Development Program of China(2020YFC2200901)。
文摘As the complexity of scientific satellite missions increases,the requirements for their magnetic fields,magnetic field fluctuations,and even magnetic field gradients and variations become increasingly stringent.Additionally,there is a growing need to address the alternating magnetic fields produced by the spacecraft itself.This paper introduces a novel modeling method for spacecraft magnetic dipoles using an integrated self-attention mechanism and a transformer combined with Kolmogorov-Arnold Networks.The self-attention mechanism captures correlations among globally sparse data,establishing dependencies b.etween sparse magnetometer readings.Concurrently,the Kolmogorov-Arnold Network,proficient in modeling implicit numerical relationships between data features,enhances the ability to learn subtle patterns.Comparative experiments validate the capability of the proposed method to precisely model magnetic dipoles,achieving maximum Root Mean Square Errors of 24.06 mA·m^(2)and 0.32 cm for size and location modeling,respectively.The spacecraft magnetic model established using this method accurately computes magnetic fields and alternating magnetic fields at designated surfaces or points.This approach facilitates the rapid and precise construction of individual and complete spacecraft magnetic models,enabling the verification of magnetic specifications from the spacecraft design phase.
文摘The development of deep learning has made non-biochemical methods for molecular property prediction screening a reality,which can increase the experimental speed and reduce the experimental cost of relevant experiments.There are currently two main approaches to representing molecules:(a)representing molecules by fixing molecular descriptors,and(b)representing molecules by graph convolutional neural networks.Currently,both of these Representative methods have achieved some results in their respective experiments.Based on past efforts,we propose a Dual Self-attention Fusion Message Neural Network(DSFMNN).DSFMNN uses a combination of dual self-attention mechanism and graph convolutional neural network.Advantages of DSFMNN:(1)The dual self-attention mechanism focuses not only on the relationship between individual subunits in a molecule but also on the relationship between the atoms and chemical bonds contained in each subunit.(2)On the directed molecular graph,a message delivery approach centered on directed molecular bonds is used.We test the performance of the model on eight publicly available datasets and compare the performance with several models.Based on the current experimental results,DSFMNN has superior performance compared to previous models on the datasets applied in this paper.
基金supported by the National Key Research and Development Program of China(No.2021YFA0715900).
文摘Located in northern China,the Hetao Plain is an important agro-economic zone and population centre.The deterioration of local groundwater quality has had a serious impact on human health and economic development.Nowadays,the groundwater vulnerability assessment(GVA)has become an essential task to identify the current status and development trend of groundwater quality.In this study,the Convolutional Neural Network(CNN)and Long Short-Term Memory(LSTM)models are integrated to realize the spatio-temporal prediction of regional groundwater vulnerability by introducing the Self-attention mechanism.The study firstly builds the CNN-LSTM modelwith self-attention(SA)mechanism and evaluates the prediction accuracy of the model for groundwater vulnerability compared to other common machine learning models such as Support Vector Machine(SVM),Random Forest(RF),and Extreme Gradient Boosting(XGBoost).The results indicate that the CNNLSTM model outperforms thesemodels,demonstrating its significance in groundwater vulnerability assessment.It can be posited that the predictions indicate an increased risk of groundwater vulnerability in the study area over the coming years.This increase can be attributed to the synergistic impact of global climate anomalies and intensified local human activities.Moreover,the overall groundwater vulnerability risk in the entire region has increased,evident fromboth the notably high value and standard deviation.This suggests that the spatial variability of groundwater vulnerability in the area is expected to expand in the future due to the sustained progression of climate change and human activities.The model can be optimized for diverse applications across regional environmental assessment,pollution prediction,and risk statistics.This study holds particular significance for ecological protection and groundwater resource management.
基金supported by the National Key Research and Development Plan(No.2022YFB2902701)the key Natural Science Foundation of Shenzhen(No.JCYJ20220818102209020).
文摘The satellite-terrestrial networks possess the ability to transcend geographical constraints inherent in traditional communication networks,enabling global coverage and offering users ubiquitous computing power support,which is an important development direction of future communications.In this paper,we take into account a multi-scenario network model under the coverage of low earth orbit(LEO)satellite,which can provide computing resources to users in faraway areas to improve task processing efficiency.However,LEO satellites experience limitations in computing and communication resources and the channels are time-varying and complex,which makes the extraction of state information a daunting task.Therefore,we explore the dynamic resource management issue pertaining to joint computing,communication resource allocation and power control for multi-access edge computing(MEC).In order to tackle this formidable issue,we undertake the task of transforming the issue into a Markov decision process(MDP)problem and propose the self-attention based dynamic resource management(SABDRM)algorithm,which effectively extracts state information features to enhance the training process.Simulation results show that the proposed algorithm is capable of effectively reducing the long-term average delay and energy consumption of the tasks.
基金supported by the National Key Research and Development Program of China[grant No.2016YF B0502200]the Postdoctoral Research Foundation of China[grant No.2020M682480]the Fundamental Research Funds for the Central Universities[grant No.2042021kf0009]。
文摘The Ultra-Wideband(UWB)Location-Based Service is receiving more and more attention due to its high ranging accuracy and good time resolution.However,the None-Line-of-Sight(NLOS)propagation may reduce the ranging accuracy for UWB localization system in indoor environment.So it is important to identify LOS and NLOS propagations before taking proper measures to improve the UWB localization accuracy.In this paper,a deep learning-based UWB NLOS/LOS classification algorithm called FCN-Attention is proposed.The proposed FCN-Attention algorithm utilizes a Fully Convolution Network(FCN)for improving feature extraction ability and a self-attention mechanism for enhancing feature description from the data to improve the classification accuracy.The proposed algorithm is evaluated using an open-source dataset,a local collected dataset and a mixed dataset created from these two datasets.The experiment result shows that the proposed FCN-Attention algorithm achieves classification accuracy of 88.24%on the open-source dataset,100%on the local collected dataset and 92.01%on the mixed dataset,which is better than the results from other evaluated NLOS/LOS classification algorithms in most scenarios in this paper.
基金supported by the National Key R&D Program of China(No.2022YFB4301102).
文摘Currently,most trains are equipped with dedicated cameras for capturing pantograph videos.Pantographs are core to the high-speed-railway pantograph-catenary system,and their failure directly affects the normal operation of high-speed trains.However,given the complex and variable real-world operational conditions of high-speed railways,there is no real-time and robust pantograph fault-detection method capable of handling large volumes of surveillance video.Hence,it is of paramount importance to maintain real-time monitoring and analysis of pantographs.Our study presents a real-time intelligent detection technology for identifying faults in high-speed railway pantographs,utilizing a fusion of self-attention and convolution features.We delved into lightweight multi-scale feature-extraction and fault-detection models based on deep learning to detect pantograph anomalies.Compared with traditional methods,this approach achieves high recall and accuracy in pantograph recognition,accurately pinpointing issues like discharge sparks,pantograph horns,and carbon pantograph-slide malfunctions.After experimentation and validation with actual surveillance videos of electric multiple-unit train,our algorithmic model demonstrates real-time,high-accuracy performance even under complex operational conditions.
基金Deputy for Research and Innovation-Ministry of Education,Kingdom of Saudi Arabia,Grant/Award Number:NU/IFC/02/SERC/-/31Institutional Funding Committee at Najran University,Kingdom of Saudi Arabia。
文摘Industrial Internet of Things(IIoT)is a pervasive network of interlinked smart devices that provide a variety of intelligent computing services in industrial environments.Several IIoT nodes operate confidential data(such as medical,transportation,military,etc.)which are reachable targets for hostile intruders due to their openness and varied structure.Intrusion Detection Systems(IDS)based on Machine Learning(ML)and Deep Learning(DL)techniques have got significant attention.However,existing ML and DL-based IDS still face a number of obstacles that must be overcome.For instance,the existing DL approaches necessitate a substantial quantity of data for effective performance,which is not feasible to run on low-power and low-memory devices.Imbalanced and fewer data potentially lead to low performance on existing IDS.This paper proposes a self-attention convolutional neural network(SACNN)architecture for the detection of malicious activity in IIoT networks and an appropriate feature extraction method to extract the most significant features.The proposed architecture has a self-attention layer to calculate the input attention and convolutional neural network(CNN)layers to process the assigned attention features for prediction.The performance evaluation of the proposed SACNN architecture has been done with the Edge-IIoTset and X-IIoTID datasets.These datasets encompassed the behaviours of contemporary IIoT communication protocols,the operations of state-of-the-art devices,various attack types,and diverse attack scenarios.
文摘This study introduces the Smart Exponential-Threshold-Linear with Double Deep Q-learning Network(SETL-DDQN)and an extended Gumbel distribution method,designed to optimize the Contention Window(CW)in IEEE 802.11 networks.Unlike conventional Deep Reinforcement Learning(DRL)-based approaches for CW size adjustment,which often suffer from overestimation bias and limited exploration diversity,leading to suboptimal throughput and collision performance.Our framework integrates the Gumbel distribution and extreme value theory to systematically enhance action selection under varying network conditions.First,SETL adopts a DDQN architecture(SETL-DDQN)to improve Q-value estimation accuracy and enhance training stability.Second,we incorporate a Gumbel distribution-driven exploration mechanism,forming SETL-DDQN(Gumbel),which employs the extreme value theory to promote diverse action selection,replacing the conventional-greedy exploration that undergoes early convergence to suboptimal solutions.Both models are evaluated through extensive simulations in static and time-varying IEEE 802.11 network scenarios.The results demonstrate that our approach consistently achieves higher throughput,lower collision rates,and improved adaptability,even under abrupt fluctuations in traffic load and network conditions.In particular,the Gumbel-based mechanism enhances the balance between exploration and exploitation,facilitating faster adaptation to varying congestion levels.These findings position Gumbel-enhanced DRL as an effective and robust solution for CW optimization in wireless networks,offering notable gains in efficiency and reliability over existing methods.
基金supported by the General Program under grant funded by the National Natural Science Foundation of China(NSFC)(No.62171307)the Basic Research Program of Shanxi Province under grant funded by the Department of Science and Technology of Shanxi Province(China)(No.202103021224113).
文摘Early and timely diagnosis of stroke is critical for effective treatment,and the electroencephalogram(EEG)offers a low-cost,non-invasive solution.However,the shortage of high-quality patient EEG data often hampers the accuracy of diagnostic classification methods based on deep learning.To address this issue,our study designed a deep data amplification model named Progressive Conditional Generative Adversarial Network with Efficient Approximating Self Attention(PCGAN-EASA),which incrementally improves the quality of generated EEG features.This network can yield full-scale,fine-grained EEG features from the low-scale,coarse ones.Specially,to overcome the limitations of traditional generative models that fail to generate features tailored to individual patient characteristics,we developed an encoder with an effective approximating self-attention mechanism.This encoder not only automatically extracts relevant features across different patients but also reduces the computational resource consumption.Furthermore,the adversarial loss and reconstruction loss functions were redesigned to better align with the training characteristics of the network and the spatial correlations among electrodes.Extensive experimental results demonstrate that PCGAN-EASA provides the highest generation quality and the lowest computational resource usage compared to several existing approaches.Additionally,it significantly improves the accuracy of subsequent stroke classification tasks.
文摘In the application of aerial target recognition,on the one hand,the recognition error produced by the single measurement of the sensor is relatively large due to the impact of noise.On the other hand,it is difficult to apply machine learning methods to improve the intelligence and recognition effect due to few or no actual measurement samples.Aiming at these problems,an aerial target recognition algorithm based on self-attention and Long Short-Term Memory Network(LSTM)is proposed.LSTM can effectively extract temporal dependencies.The attention mechanism calculates the weight of each input element and applies the weight to the hidden state of the LSTM,thereby adjusting the LSTM’s attention to the input.This combination retains the learning ability of LSTM and introduces the advantages of the attention mechanism,making the model have stronger feature extraction ability and adaptability when processing sequence data.In addition,based on the prior information of the multidimensional characteristics of the target,the three-point estimation method is adopted to simulate an aerial target recognition dataset to train the recognition model.The experimental results show that the proposed algorithm achieves more than 91%recognition accuracy,lower false alarm rate and higher robustness compared with the multi-attribute decision-making(MADM)based on fuzzy numbers.
文摘On Twitter,people often use hashtags to mark the subject of a tweet.Tweets have specific themes or content that are easy for people to manage.With the increase in the number of tweets,how to automatically recommend hashtags for tweets has received wide attention.The previous hashtag recommendation methods were to convert the task into a multi-class classification problem.However,these methods can only recommend hashtags that appeared in historical information,and cannot recommend the new ones.In this work,we extend the self-attention mechanism to turn the hashtag recommendation task into a sequence labeling task.To train and evaluate the proposed method,we used the real tweet data which is collected from Twitter.Experimental results show that the proposed method can be significantly better than the most advanced method.Compared with the state-of-the-art methods,the accuracy of our method has been increased 4%.
基金This work was supported by National Key R&D Program of China(2019YFE0102900).
文摘Infrared image recognition plays an important role in the inspection of power equipment.Existing technologies dedicated to this purpose often require manually selected features,which are not transferable and interpretable,and have limited training data.To address these limitations,this paper proposes an automatic infrared image recognition framework,which includes an object recognition module based on a deep self-attention network and a temperature distribution identification module based on a multi-factor similarity calculation.First,the features of an input image are extracted and embedded using a multi-head attention encoding-decoding mechanism.Thereafter,the embedded features are used to predict the equipment component category and location.In the located area,preliminary segmentation is performed.Finally,similar areas are gradually merged,and the temperature distribution of the equipment is obtained to identify a fault.Our experiments indicate that the proposed method demonstrates significantly improved accuracy compared with other related methods and,hence,provides a good reference for the automation of power equipment inspection.
基金the National Natural Science Foundation of China(Nos.51875451 and 51834006)。
文摘Aiming at the problem of insufficient consideration of the correlation between components in the prediction of the remaining life of mechanical equipment,the method of remaining life prediction that combines the self-attention mechanism with the long short-term memory neural network(LSTM-NN)is proposed,called Self-Attention-LSTM.First,the auto-encoder is used to obtain the component-level state information;second,the state information of each component is input into the self-attention mechanism to learn the correlation between components;then,the multi-component correlation matrix is added to the LSTM input gate,and the LSTM-NN is used for life prediction.Finally,combined with the commercial modular aero-propulsion system simulation data set(C-MAPSS),the experiment was carried out and compared with the existing methods.Research results show that the proposed method can achieve better prediction accuracy and verify the feasibility of the method.
文摘Relation extraction is an important task in NLP community.However,some models often fail in capturing Long-distance dependence on semantics,and the interaction between semantics of two entities is ignored.In this paper,we propose a novel neural network model for semantic relation classification called joint self-attention bi-LSTM(SA-Bi-LSTM)to model the internal structure of the sentence to obtain the importance of each word of the sentence without relying on additional information,and capture Long-distance dependence on semantics.We conduct experiments using the SemEval-2010 Task 8 dataset.Extensive experiments and the results demonstrated that the proposed method is effective against relation classification,which can obtain state-ofthe-art classification accuracy just with minimal feature engineering.
基金the National Natural Science Foundation of China(62071330)the National Science Fund for Distinguished Young Scholars(61425017)+3 种基金the Key Program of the National Natural Science Foundation(61831022)the Key Program of the Natural Science Foundation of Tianjin(18JCZDJC36300)the Open Projects Program of the National Laboratory of Pattern Recognition and the Senior Visiting Scholar Program of Tianjin Normal Universitythe Innovative Medicines Initiative 2 Joint Undertaking(115902),which receives support from the European Union's Horizon 2020 research and innovation program and EFPIA.
文摘Background A crucial element of human-machine interaction,the automatic detection of emotional states from human speech has long been regarded as a challenging task for machine learning models.One vital challenge in speech emotion recognition(SER)is learning robust and discriminative representations from speech.Although machine learning methods have been widely applied in SER research,the inadequate amount of available annotated data has become a bottleneck impeding the extended application of such techniques(e.g.,deep neural networks).To address this issue,we present a deep learning method that combines knowledge transfer and self-attention for SER tasks.Herein,we apply the log-Mel spectrogram with deltas and delta-deltas as inputs.Moreover,given that emotions are time dependent,we apply temporal convolutional neural networks to model the variations in emotions.We further introduce an attention transfer mechanism,which is based on a self-attention algorithm to learn long-term dependencies.The self-attention transfer network(SATN)in our proposed approach takes advantage of attention transfer to learn attention from speech recognition,followed by transferring this knowledge into SER.An evaluation built on Interactive Emotional Dyadic Motion Capture(IEMOCAP)dataset demonstrates the effectiveness of the proposed model.
基金supported by grants from The National Natural Science Foundation of China(12361104)Yunnan Fundamental Research Projects(202301AT070016,202401AT070036)+2 种基金the Youth Talent Program of Xingdian Talent Support Plan(XDYC-QNRC-2022-0514)the Yunnan Province International Joint Laboratory for Intelligent Integration and Application of Ethnic Multilingualism(202403AP140014)the Open Research Fund of Yunnan Key Laboratory of Statistical Modeling and Data Analysis,Yunnan University(SMDAYB2023004)。
文摘Objective N6-methyladenosine(m6A),the most prevalent epigenetic modification in eukaryotic RNA,plays a pivotal role in regulating cellular differentiation and developmental processes,with its dysregulation implicated in diverse pathological conditions.Accurate prediction of m6A sites is critical for elucidating their regulatory mechanisms and informing drug development.However,traditional experimental methods are time-consuming and costly.Although various computational approaches have been proposed,challenges remain in feature learning,predictive accuracy,and generalization.Here,we present m6A-PSRA,a dual-branch residual-network-based predictor that fully exploits RNA sequence information to enhance prediction performance and model generalization.Methods m6A-PSRA adopts a parallel dual-branch network architecture to comprehensively extract RNA sequence features via two independent pathways.The first branch applies one-hot encoding to transform the RNA sequence into a numerical matrix while strictly preserving positional information and sequence continuity.This ensures that the biological context conveyed by nucleotide order is retained.A bidirectional long short-term memory network(BiLSTM)then processes the encoded matrix,capturing both forward and backward dependencies between bases to resolve contextual correlations.The second branch employs a k-mer tokenization strategy(k=3),decomposing the sequence into overlapping 3-mer subsequences to capture local sequence patterns.A pre-trained Doc2vec model maps these subsequences into fixeddimensional vectors,reducing feature dimensionality while extracting latent global semantic information via context learning.Both branches integrate residual networks(ResNet)and a self-attention mechanism:ResNet mitigates vanishing gradients through skip connections,preserving feature integrity,while self-attention adaptively assigns weights to focus on sequence regions most relevant to methylation prediction.This synergy enhances both feature learning and generalization capability.Results Across 11 tissues from humans,mice,and rats,m6A-PSRA consistently outperformed existing methods in accuracy(ACC)and area under the curve(AUC),achieving>90%ACC and>95%AUC in every tissue tested,indicating strong cross-species and cross-tissue adaptability.Validation on independent datasets—including three human cell lines(MOLM1,HEK293,A549)and a long-sequence dataset(m6A_IND,1001 nt)—confirmed stable performance across varied biological contexts and sequence lengths.Ablation studies demonstrated that the dual-branch architecture,residual network,and self-attention mechanism each contribute critically to performance,with their combination reducing interference between pathways.Motif analysis revealed an enrichment of m6A sites in guanine(G)and cytosine(C),consistent with known regulatory patterns,supporting the model’s biological plausibility.Conclusion m6A-PSRA effectively captures RNA sequence features,achieving high prediction accuracy and robust generalization across tissues and species,providing an efficient computational tool for m6A methylation site prediction.
基金sponsored by the National Natural Science Foundation of China under Grants 62271264,61972207,and 42175194the Project through the Priority Academic Program Development(PAPD)of Jiangsu Higher Education Institution.
文摘Traditional based deep learning intrusion detection methods face problems such as insufficient cloud storage,data privacy leaks,high com-munication costs,unsatisfactory detection rates,and false positive rate.To address existing issues in intrusion detection,this paper presents a novel approach called CS-FL,which combines Federated Learning and a Self-Attention Fusion Convolutional Neural Network.Federated Learning is a new distributed computing model that enables individual training of client data without uploading local data to a central server.at the same time,local training results are uploaded and integrated across all participating clients to produce a global model.The sharing model reduces communication costs,protects data privacy,and solves problems such as insufficient cloud storage and“data islands”for each client.In the proposed method,a hybrid model is formed by integrating the self-Attention and similar parts of the Convolutional Neural Network in the local data processing.This approach not only enhances the performance of the hybrid model but also reduces computational overhead compared to pure hybrid neural networks.Results from experiments on the NSL-KDD dataset show that the proposed method outperforms other intrusion detection techniques,resulting in a significant improvement in performance.This demonstrates the effectiveness of the proposed approach in improving intrusion detection accuracy.
基金Supported by Discipline-Crossing Research Foundation of Huazhong Agricultural University(2008XKJC006)the Fundamental Research Funds for the Central Universities of China
文摘We extract some physical and chemical features re-lated to the occurrence of single nucleotide polymorphism （SNP） from three groups of sliding windows around SNP site,and then make the predictions about accuracy by using radial basis function （RBF） networks. The result of the forward sliding windows sug-gests that the accuracies and Matthews correlation coefficient （MCC values） ascend with the increasing of length of sliding windows. The accuracies range from 73.27 % to 80.69 %,and MCC values range from 0.465 to 0.614. The backward sliding windows and the sliding windows with fixed length three are de-signed to find the crucial sites related to SNP. The results imply that the occurrence possibility of SNP relies heavily on the above physical and chemical features of sites which are at a distance around 20 bases from the SNP site. Compared with the support vector machine （SVM）,our RBF network approach has achieved more satisfactory results.
