Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global featu...Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global feature.The transformer can extract the global information well but adapting it to small medical datasets is challenging and its computational complexity can be heavy.In this work,a serial and parallel network is proposed for the accurate 3D medical image segmentation by combining CNN and transformer and promoting feature interactions across various semantic levels.The core components of the proposed method include the cross window self-attention based transformer(CWST)and multi-scale local enhanced(MLE)modules.The CWST module enhances the global context understanding by partitioning 3D images into non-overlapping windows and calculating sparse global attention between windows.The MLE module selectively fuses features by computing the voxel attention between different branch features,and uses convolution to strengthen the dense local information.The experiments on the prostate,atrium,and pancreas MR/CT image datasets consistently demonstrate the advantage of the proposed method over six popular segmentation models in both qualitative evaluation and quantitative indexes such as dice similarity coefficient,Intersection over Union,95%Hausdorff distance and average symmetric surface distance.展开更多
Currently,most trains are equipped with dedicated cameras for capturing pantograph videos.Pantographs are core to the high-speed-railway pantograph-catenary system,and their failure directly affects the normal operati...Currently,most trains are equipped with dedicated cameras for capturing pantograph videos.Pantographs are core to the high-speed-railway pantograph-catenary system,and their failure directly affects the normal operation of high-speed trains.However,given the complex and variable real-world operational conditions of high-speed railways,there is no real-time and robust pantograph fault-detection method capable of handling large volumes of surveillance video.Hence,it is of paramount importance to maintain real-time monitoring and analysis of pantographs.Our study presents a real-time intelligent detection technology for identifying faults in high-speed railway pantographs,utilizing a fusion of self-attention and convolution features.We delved into lightweight multi-scale feature-extraction and fault-detection models based on deep learning to detect pantograph anomalies.Compared with traditional methods,this approach achieves high recall and accuracy in pantograph recognition,accurately pinpointing issues like discharge sparks,pantograph horns,and carbon pantograph-slide malfunctions.After experimentation and validation with actual surveillance videos of electric multiple-unit train,our algorithmic model demonstrates real-time,high-accuracy performance even under complex operational conditions.展开更多
Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address ...Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address this problem, a Multi-head Self-attention and Spatial-Temporal Graph Convolutional Network (MSSTGCN) for multiscale traffic flow prediction is proposed. Firstly, to capture the hidden traffic periodicity of traffic flow, traffic flow is divided into three kinds of periods, including hourly, daily, and weekly data. Secondly, a graph attention residual layer is constructed to learn the global spatial features across regions. Local spatial-temporal dependence is captured by using a T-GCN module. Thirdly, a transformer layer is introduced to learn the long-term dependence in time. A position embedding mechanism is introduced to label position information for all traffic sequences. Thus, this multi-head self-attention mechanism can recognize the sequence order and allocate weights for different time nodes. Experimental results on four real-world datasets show that the MSSTGCN performs better than the baseline methods and can be successfully adapted to traffic prediction tasks.展开更多
Traditional data-driven fault diagnosis methods depend on expert experience to manually extract effective fault features of signals,which has certain limitations.Conversely,deep learning techniques have gained promine...Traditional data-driven fault diagnosis methods depend on expert experience to manually extract effective fault features of signals,which has certain limitations.Conversely,deep learning techniques have gained prominence as a central focus of research in the field of fault diagnosis by strong fault feature extraction ability and end-to-end fault diagnosis efficiency.Recently,utilizing the respective advantages of convolution neural network(CNN)and Transformer in local and global feature extraction,research on cooperating the two have demonstrated promise in the field of fault diagnosis.However,the cross-channel convolution mechanism in CNN and the self-attention calculations in Transformer contribute to excessive complexity in the cooperative model.This complexity results in high computational costs and limited industrial applicability.To tackle the above challenges,this paper proposes a lightweight CNN-Transformer named as SEFormer for rotating machinery fault diagnosis.First,a separable multiscale depthwise convolution block is designed to extract and integrate multiscale feature information from different channel dimensions of vibration signals.Then,an efficient self-attention block is developed to capture critical fine-grained features of the signal from a global perspective.Finally,experimental results on the planetary gearbox dataset and themotor roller bearing dataset prove that the proposed framework can balance the advantages of robustness,generalization and lightweight compared to recent state-of-the-art fault diagnosis models based on CNN and Transformer.This study presents a feasible strategy for developing a lightweight rotating machinery fault diagnosis framework aimed at economical deployment.展开更多
Many spore-forming Bacillus species can cause serious human diseases,because of accidental Bacillusspore infection.Thus,developing an identification strategy with both high sensitivity and specificity is greatly in de...Many spore-forming Bacillus species can cause serious human diseases,because of accidental Bacillusspore infection.Thus,developing an identification strategy with both high sensitivity and specificity is greatly in demand.In this work,we proposed a novel approach named multi-head self-attention mechanism-guided neural network Raman platform to identify living Bacillus spores within a single-cell resolution.The multi-head self-attention mechanism-guided neural network Raman platform was created by combining single-cell Raman spectroscopy,convolutional neural network(CNN),and multi-head self-attention mechanism.To address the limited size of the original spectra dataset,Gaussian noise-based spectra augmentation was employed to increase the number of single-cell Raman spectra datasets for CNN training.Owing to the assistance of both spectra augmentation and multi-head self-attention mechanism,the obtained prediction accuracy of five Bacillus spore species was further improved from 92.29±0.82%to 99.43±0.15%.To figure out the spectra differences covered by the multi-head self-attention mechanism-guided CNN,the relative classification weight from typical Raman bands was visualized via multi-head self-attention mechanism curve.In the process of spectra augmentation from 0 to 1000,the distribution of relative classification weight varied from a discrete state to a more concentrated phase.More importantly,these highlighted four Raman bands(1017,1449,1576,and 1660 cm^(-1))were assigned large weights,showing that the spectra differences in the Raman bands produced the largest contribution to prediction accuracy.It can be foreseen that,our proposed sorting platform has great potential in accurately identifying Bacillus and its related genera species at a single-cell level.展开更多
Sustainable energy systems will entail a change in the carbon intensity projections,which should be carried out in a proper manner to facilitate the smooth running of the grid and reduce greenhouse emissions.The prese...Sustainable energy systems will entail a change in the carbon intensity projections,which should be carried out in a proper manner to facilitate the smooth running of the grid and reduce greenhouse emissions.The present article outlines the TransCarbonNet,a novel hybrid deep learning framework with self-attention characteristics added to the bidirectional Long Short-Term Memory(Bi-LSTM)network to forecast the carbon intensity of the grid several days.The proposed temporal fusion model not only learns the local temporal interactions but also the long-term patterns of the carbon emission data;hence,it is able to give suitable forecasts over a period of seven days.TransCarbonNet takes advantage of a multi-head self-attention element to identify significant temporal connections,which means the Bi-LSTM element calculates sequential dependencies in both directions.Massive tests on two actual data sets indicate much improved results in comparison with the existing results,with mean relative errors of 15.3 percent and 12.7 percent,respectively.The framework has given explicable weights of attention that reveal critical periods that influence carbon intensity alterations,and informed decisions on the management of carbon sustainability.The effectiveness of the proposed solution has been validated in numerous cases of operations,and TransCarbonNet is established to be an effective tool when it comes to carbon-friendly optimization of the grid.展开更多
The development of brain-computer interfaces(BCI)based on motor imagery(MI)has greatly improved patients’quality of life with movement disorders.The classification of upper limb MI has been widely studied and applied...The development of brain-computer interfaces(BCI)based on motor imagery(MI)has greatly improved patients’quality of life with movement disorders.The classification of upper limb MI has been widely studied and applied in many fields,including rehabilitation.However,the physiological representations of left and right lower limb movements are too close and activated deep in the cerebral cortex,making it difficult to distinguish their features.Therefore,classifying lower limbs motor imagery is more challenging.In this study,we propose a feature extraction method based on functional connectivity,which utilizes phase-locked values to construct a functional connectivity matrix as the features of the left and right legs,which can effectively avoid the problem of physiological representations of the left and right lower limbs being too close to each other during movement.In addition,considering the topology and the temporal characteristics of the electroencephalogram(EEG),we designed a temporal-spatial convolutional network(TSGCN)to capture the spatiotemporal information for classification.Experimental results show that the accuracy of the proposed method is higher than that of existing methods,achieving an average classification accuracy of 73.58%on the internal dataset.Finally,this study explains the network mechanism of left and right foot MI from the perspective of graph theoretic features and demonstrates the feasibility of decoding lower limb MI.展开更多
With network attack technology continuing to develop,traditional anomaly traffic detection methods that rely on feature engineering are increasingly insufficient in efficiency and accuracy.Graph Neural Network(GNN),a ...With network attack technology continuing to develop,traditional anomaly traffic detection methods that rely on feature engineering are increasingly insufficient in efficiency and accuracy.Graph Neural Network(GNN),a promising Deep Learning(DL)approach,has proven to be highly effective in identifying intricate patterns in graph⁃structured data and has already found wide applications in the field of network security.In this paper,we propose a hybrid Graph Convolutional Network(GCN)⁃GraphSAGE model for Anomaly Traffic Detection,namely HGS⁃ATD,which aims to improve the accuracy of anomaly traffic detection by leveraging edge feature learning to better capture the relationships between network entities.We validate the HGS⁃ATD model on four publicly available datasets,including NF⁃UNSW⁃NB15⁃v2.The experimental results show that the enhanced hybrid model is 5.71%to 10.25%higher than the baseline model in terms of accuracy,and the F1⁃score is 5.53%to 11.63%higher than the baseline model,proving that the model can effectively distinguish normal traffic from attack traffic and accurately classify various types of attacks.展开更多
Graph neural networks(GNN)have shown strong performance in node classification tasks,yet most existing models rely on uniform or shared weight aggregation,lacking flexibility in modeling the varying strength of relati...Graph neural networks(GNN)have shown strong performance in node classification tasks,yet most existing models rely on uniform or shared weight aggregation,lacking flexibility in modeling the varying strength of relationships among nodes.This paper proposes a novel graph coupling convolutional model that introduces an adaptive weighting mechanism to assign distinct importance to neighboring nodes based on their similarity to the central node.Unlike traditional methods,the proposed coupling strategy enhances the interpretability of node interactions while maintaining competitive classification performance.The model operates in the spatial domain,utilizing adjacency list structures for efficient convolution and addressing the limitations of weight sharing through a coupling-based similarity computation.Extensive experiments are conducted on five graph-structured datasets,including Cora,Citeseer,PubMed,Reddit,and BlogCatalog,as well as a custom topology dataset constructed from the Open University Learning Analytics Dataset(OULAD)educational platform.Results demonstrate that the proposed model achieves good classification accuracy,while significantly reducing training time through direct second-order neighbor fusion and data preprocessing.Moreover,analysis of neighborhood order reveals that considering third-order neighbors offers limited accuracy gains but introduces considerable computational overhead,confirming the efficiency of first-and second-order convolution in practical applications.Overall,the proposed graph coupling model offers a lightweight,interpretable,and effective framework for multi-label node classification in complex networks.展开更多
With the rapid development of the Artificial Intelligence of Things(AIoT),convolutional neural networks(CNNs)have demonstrated potential and remarkable performance in AIoT applications due to their excellent performan...With the rapid development of the Artificial Intelligence of Things(AIoT),convolutional neural networks(CNNs)have demonstrated potential and remarkable performance in AIoT applications due to their excellent performance in various inference tasks.However,the users have concerns about privacy leakage for the use of AI and the performance and efficiency of computing on resource-constrained IoT edge devices.Therefore,this paper proposes an efficient privacy-preserving CNN framework(i.e.,EPPA)based on the Fully Homomorphic Encryption(FHE)scheme for AIoT application scenarios.In the plaintext domain,we verify schemes with different activation structures to determine the actual activation functions applicable to the corresponding ciphertext domain.Within the encryption domain,we integrate batch normalization(BN)into the convolutional layers to simplify the computation process.For nonlinear activation functions,we use composite polynomials for approximate calculation.Regarding the noise accumulation caused by homomorphic multiplication operations,we realize the refreshment of ciphertext noise through minimal“decryption-encryption”interactions,instead of adopting bootstrapping operations.Additionally,in practical implementation,we convert three-dimensional convolution into two-dimensional convolution to reduce the amount of computation in the encryption domain.Finally,we conduct extensive experiments on four IoT datasets,different CNN architectures,and two platforms with different resource configurations to evaluate the performance of EPPA in detail.展开更多
With the increasing complexity of industrial automation,planetary gearboxes play a vital role in largescale equipment transmission systems,directly impacting operational efficiency and safety.Traditional maintenance s...With the increasing complexity of industrial automation,planetary gearboxes play a vital role in largescale equipment transmission systems,directly impacting operational efficiency and safety.Traditional maintenance strategies often struggle to accurately predict the degradation process of equipment,leading to excessive maintenance costs or potential failure risks.However,existing prediction methods based on statistical models are difficult to adapt to nonlinear degradation processes.To address these challenges,this study proposes a novel condition-based maintenance framework for planetary gearboxes.A comprehensive full-lifecycle degradation experiment was conducted to collect raw vibration signals,which were then processed using a temporal convolutional network autoencoder with multi-scale perception capability to extract deep temporal degradation features,enabling the collaborative extraction of longperiod meshing frequencies and short-term impact features from the vibration signals.Kernel principal component analysis was employed to fuse and normalize these features,enhancing the characterization of degradation progression.A nonlinear Wiener process was used to model the degradation trajectory,with a threshold decay function introduced to dynamically adjust maintenance strategies,and model parameters optimized through maximum likelihood estimation.Meanwhile,the maintenance strategy was optimized to minimize costs per unit time,determining the optimal maintenance timing and preventive maintenance threshold.The comprehensive indicator of degradation trends extracted by this method reaches 0.756,which is 41.2%higher than that of traditional time-domain features;the dynamic threshold strategy reduces the maintenance cost per unit time to 55.56,which is 8.9%better than that of the static threshold optimization.Experimental results demonstrate significant reductions in maintenance costs while enhancing system reliability and safety.This study realizes the organic integration of deep learning and reliability theory in the maintenance of planetary gearboxes,provides an interpretable solution for the predictive maintenance of complex mechanical systems,and promotes the development of condition-based maintenance strategies for planetary gearboxes.展开更多
The prediction of sea surface partial pressure of carbon dioxide(pCO_(2))in the South China Sea is crucial for understanding the region’s contribution to the global carbon budget and its interactions with climate cha...The prediction of sea surface partial pressure of carbon dioxide(pCO_(2))in the South China Sea is crucial for understanding the region’s contribution to the global carbon budget and its interactions with climate change.We applied the Spatiotemporal Convolutional Long Short-Term Memory(STConvLSTM)model,integrating key environmental factors including sea surface temperature(SST),sea surface salinity(SSS),and chlorophyll a(Chl a),to predict and analyze sea surface pCO_(2)in the South China Sea.The model demonstrated high accuracy in short-term predictions(1 month),with a mean absolute error(MAE)of 0.394,a root mean square error(RMSE)of 0.659,and a coefficient of determination(R^(2))of 0.998.For long-term predictions(12 months),the model maintained its predictive capability,with an MAE of 0.667,RMSE of 1.255,and R^(2)of 0.994.Feature importance analysis revealed that sea surface pCO_(2)and SST were the main drivers of the model’s predictions,whereas Chl a and SSS had relatively minor impacts.The model’s generalization ability was further validated in the northwest Pacific Ocean and tropical Pacific Ocean,where it successfully captured the spatiotemporal variation in pCO_(2)with small prediction errors.The ST-ConvLSTM model provides an efficient and accurate tool for forecasting and analyzing sea surface pCO_(2)in the South China Sea,offering new insights into global carbon cycling and climate change.This study demonstrates the potential of deep learning in marine science and provides a significant technical support for global changes and marine ecosystem research.展开更多
critical for guiding treatment and improving patient outcomes.Traditional molecular subtyping via immuno-histochemistry(IHC)test is invasive,time-consuming,and may not fully represent tumor heterogeneity.This study pr...critical for guiding treatment and improving patient outcomes.Traditional molecular subtyping via immuno-histochemistry(IHC)test is invasive,time-consuming,and may not fully represent tumor heterogeneity.This study proposes a non-invasive approach using digital mammography images and deep learning algorithm for classifying breast cancer molecular subtypes.Four pretrained models,including two Convolutional Neural Networks(MobileNet_V3_Large and VGG-16)and two Vision Transformers(ViT_B_16 and ViT_Base_Patch16_Clip_224)were fine-tuned to classify images into HER2-enriched,Luminal,Normal-like,and Triple Negative subtypes.Hyperparameter tuning,including learning rate adjustment and layer freezing strategies,was applied to optimize performance.Among the evaluated models,ViT_Base_Patch16_Clip_224 achieved the highest test accuracy(94.44%),with equally high precision,recall,and F1-score of 0.94,demonstrating excellent generalization.MobileNet_V3_Large achieved the same accuracy but showed less training stability.In contrast,VGG-16 recorded the lowest performance,indicating a limitation in its generalizability for this classification task.The study also highlighted the superior performance of the Vision Transformer models over CNNs,particularly due to their ability to capture global contextual features and the benefit of CLIP-based pretraining in ViT_Base_Patch16_Clip_224.To enhance clinical applicability,a graphical user interface(GUI)named“BCMS Dx”was developed for streamlined subtype prediction.Deep learning applied to mammography has proven effective for accurate and non-invasive molecular subtyping.The proposed Vision Transformer-based model and supporting GUI offer a promising direction for augmenting diagnostic workflows,minimizing the need for invasive procedures,and advancing personalized breast cancer management.展开更多
Parkinson’s disease(PD)is a debilitating neurological disorder affecting over 10 million people worldwide.PD classification models using voice signals as input are common in the literature.It is believed that using d...Parkinson’s disease(PD)is a debilitating neurological disorder affecting over 10 million people worldwide.PD classification models using voice signals as input are common in the literature.It is believed that using deep learning algorithms further enhances performance;nevertheless,it is challenging due to the nature of small-scale and imbalanced PD datasets.This paper proposed a convolutional neural network-based deep support vector machine(CNN-DSVM)to automate the feature extraction process using CNN and extend the conventional SVM to a DSVM for better classification performance in small-scale PD datasets.A customized kernel function reduces the impact of biased classification towards the majority class(healthy candidates in our consideration).An improved generative adversarial network(IGAN)was designed to generate additional training data to enhance the model’s performance.For performance evaluation,the proposed algorithm achieves a sensitivity of 97.6%and a specificity of 97.3%.The performance comparison is evaluated from five perspectives,including comparisons with different data generation algorithms,feature extraction techniques,kernel functions,and existing works.Results reveal the effectiveness of the IGAN algorithm,which improves the sensitivity and specificity by 4.05%–4.72%and 4.96%–5.86%,respectively;and the effectiveness of the CNN-DSVM algorithm,which improves the sensitivity by 1.24%–57.4%and specificity by 1.04%–163%and reduces biased detection towards the majority class.The ablation experiments confirm the effectiveness of individual components.Two future research directions have also been suggested.展开更多
Lightweight convolutional neural networks(CNNs)have simple structures but struggle to comprehensively and accurately extract important semantic information from images.While attention mechanisms can enhance CNNs by le...Lightweight convolutional neural networks(CNNs)have simple structures but struggle to comprehensively and accurately extract important semantic information from images.While attention mechanisms can enhance CNNs by learning distinctive representations,most existing spatial and hybrid attention methods focus on local regions with extensive parameters,making them unsuitable for lightweight CNNs.In this paper,we propose a self-attention mechanism tailored for lightweight networks,namely the brief self-attention module(BSAM).BSAM consists of the brief spatial attention(BSA)and advanced channel attention blocks.Unlike conventional self-attention methods with many parameters,our BSA block improves the performance of lightweight networks by effectively learning global semantic representations.Moreover,BSAM can be seamlessly integrated into lightweight CNNs for end-to-end training,maintaining the network’s lightweight and mobile characteristics.We validate the effectiveness of the proposed method on image classification tasks using the Food-101,Caltech-256,and Mini-ImageNet datasets.展开更多
Industrial Internet of Things(IIoT)is a pervasive network of interlinked smart devices that provide a variety of intelligent computing services in industrial environments.Several IIoT nodes operate confidential data(s...Industrial Internet of Things(IIoT)is a pervasive network of interlinked smart devices that provide a variety of intelligent computing services in industrial environments.Several IIoT nodes operate confidential data(such as medical,transportation,military,etc.)which are reachable targets for hostile intruders due to their openness and varied structure.Intrusion Detection Systems(IDS)based on Machine Learning(ML)and Deep Learning(DL)techniques have got significant attention.However,existing ML and DL-based IDS still face a number of obstacles that must be overcome.For instance,the existing DL approaches necessitate a substantial quantity of data for effective performance,which is not feasible to run on low-power and low-memory devices.Imbalanced and fewer data potentially lead to low performance on existing IDS.This paper proposes a self-attention convolutional neural network(SACNN)architecture for the detection of malicious activity in IIoT networks and an appropriate feature extraction method to extract the most significant features.The proposed architecture has a self-attention layer to calculate the input attention and convolutional neural network(CNN)layers to process the assigned attention features for prediction.The performance evaluation of the proposed SACNN architecture has been done with the Edge-IIoTset and X-IIoTID datasets.These datasets encompassed the behaviours of contemporary IIoT communication protocols,the operations of state-of-the-art devices,various attack types,and diverse attack scenarios.展开更多
Traditional based deep learning intrusion detection methods face problems such as insufficient cloud storage,data privacy leaks,high com-munication costs,unsatisfactory detection rates,and false positive rate.To addre...Traditional based deep learning intrusion detection methods face problems such as insufficient cloud storage,data privacy leaks,high com-munication costs,unsatisfactory detection rates,and false positive rate.To address existing issues in intrusion detection,this paper presents a novel approach called CS-FL,which combines Federated Learning and a Self-Attention Fusion Convolutional Neural Network.Federated Learning is a new distributed computing model that enables individual training of client data without uploading local data to a central server.at the same time,local training results are uploaded and integrated across all participating clients to produce a global model.The sharing model reduces communication costs,protects data privacy,and solves problems such as insufficient cloud storage and“data islands”for each client.In the proposed method,a hybrid model is formed by integrating the self-Attention and similar parts of the Convolutional Neural Network in the local data processing.This approach not only enhances the performance of the hybrid model but also reduces computational overhead compared to pure hybrid neural networks.Results from experiments on the NSL-KDD dataset show that the proposed method outperforms other intrusion detection techniques,resulting in a significant improvement in performance.This demonstrates the effectiveness of the proposed approach in improving intrusion detection accuracy.展开更多
Located in northern China,the Hetao Plain is an important agro-economic zone and population centre.The deterioration of local groundwater quality has had a serious impact on human health and economic development.Nowad...Located in northern China,the Hetao Plain is an important agro-economic zone and population centre.The deterioration of local groundwater quality has had a serious impact on human health and economic development.Nowadays,the groundwater vulnerability assessment(GVA)has become an essential task to identify the current status and development trend of groundwater quality.In this study,the Convolutional Neural Network(CNN)and Long Short-Term Memory(LSTM)models are integrated to realize the spatio-temporal prediction of regional groundwater vulnerability by introducing the Self-attention mechanism.The study firstly builds the CNN-LSTM modelwith self-attention(SA)mechanism and evaluates the prediction accuracy of the model for groundwater vulnerability compared to other common machine learning models such as Support Vector Machine(SVM),Random Forest(RF),and Extreme Gradient Boosting(XGBoost).The results indicate that the CNNLSTM model outperforms thesemodels,demonstrating its significance in groundwater vulnerability assessment.It can be posited that the predictions indicate an increased risk of groundwater vulnerability in the study area over the coming years.This increase can be attributed to the synergistic impact of global climate anomalies and intensified local human activities.Moreover,the overall groundwater vulnerability risk in the entire region has increased,evident fromboth the notably high value and standard deviation.This suggests that the spatial variability of groundwater vulnerability in the area is expected to expand in the future due to the sustained progression of climate change and human activities.The model can be optimized for diverse applications across regional environmental assessment,pollution prediction,and risk statistics.This study holds particular significance for ecological protection and groundwater resource management.展开更多
The Ultra-Wideband(UWB)Location-Based Service is receiving more and more attention due to its high ranging accuracy and good time resolution.However,the None-Line-of-Sight(NLOS)propagation may reduce the ranging accur...The Ultra-Wideband(UWB)Location-Based Service is receiving more and more attention due to its high ranging accuracy and good time resolution.However,the None-Line-of-Sight(NLOS)propagation may reduce the ranging accuracy for UWB localization system in indoor environment.So it is important to identify LOS and NLOS propagations before taking proper measures to improve the UWB localization accuracy.In this paper,a deep learning-based UWB NLOS/LOS classification algorithm called FCN-Attention is proposed.The proposed FCN-Attention algorithm utilizes a Fully Convolution Network(FCN)for improving feature extraction ability and a self-attention mechanism for enhancing feature description from the data to improve the classification accuracy.The proposed algorithm is evaluated using an open-source dataset,a local collected dataset and a mixed dataset created from these two datasets.The experiment result shows that the proposed FCN-Attention algorithm achieves classification accuracy of 88.24%on the open-source dataset,100%on the local collected dataset and 92.01%on the mixed dataset,which is better than the results from other evaluated NLOS/LOS classification algorithms in most scenarios in this paper.展开更多
In recent years,gait-based emotion recognition has been widely applied in the field of computer vision.However,existing gait emotion recognition methods typically rely on complete human skeleton data,and their accurac...In recent years,gait-based emotion recognition has been widely applied in the field of computer vision.However,existing gait emotion recognition methods typically rely on complete human skeleton data,and their accuracy significantly declines when the data is occluded.To enhance the accuracy of gait emotion recognition under occlusion,this paper proposes a Multi-scale Suppression Graph ConvolutionalNetwork(MS-GCN).TheMS-GCN consists of three main components:Joint Interpolation Module(JI Moudle),Multi-scale Temporal Convolution Network(MS-TCN),and Suppression Graph Convolutional Network(SGCN).The JI Module completes the spatially occluded skeletal joints using the(K-Nearest Neighbors)KNN interpolation method.The MS-TCN employs convolutional kernels of various sizes to comprehensively capture the emotional information embedded in the gait,compensating for the temporal occlusion of gait information.The SGCN extracts more non-prominent human gait features by suppressing the extraction of key body part features,thereby reducing the negative impact of occlusion on emotion recognition results.The proposed method is evaluated on two comprehensive datasets:Emotion-Gait,containing 4227 real gaits from sources like BML,ICT-Pollick,and ELMD,and 1000 synthetic gaits generated using STEP-Gen technology,and ELMB,consisting of 3924 gaits,with 1835 labeled with emotions such as“Happy,”“Sad,”“Angry,”and“Neutral.”On the standard datasets Emotion-Gait and ELMB,the proposed method achieved accuracies of 0.900 and 0.896,respectively,attaining performance comparable to other state-ofthe-artmethods.Furthermore,on occlusion datasets,the proposedmethod significantly mitigates the performance degradation caused by occlusion compared to other methods,the accuracy is significantly higher than that of other methods.展开更多
基金National Key Research and Development Program of China,Grant/Award Number:2018YFE0206900China Postdoctoral Science Foundation,Grant/Award Number:2023M731204+2 种基金The Open Project of Key Laboratory for Quality Evaluation of Ultrasound Surgical Equipment of National Medical Products Administration,Grant/Award Number:SMDTKL-2023-1-01The Hubei Province Key Research and Development Project,Grant/Award Number:2023BCB007CAAI-Huawei MindSpore Open Fund。
文摘Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global feature.The transformer can extract the global information well but adapting it to small medical datasets is challenging and its computational complexity can be heavy.In this work,a serial and parallel network is proposed for the accurate 3D medical image segmentation by combining CNN and transformer and promoting feature interactions across various semantic levels.The core components of the proposed method include the cross window self-attention based transformer(CWST)and multi-scale local enhanced(MLE)modules.The CWST module enhances the global context understanding by partitioning 3D images into non-overlapping windows and calculating sparse global attention between windows.The MLE module selectively fuses features by computing the voxel attention between different branch features,and uses convolution to strengthen the dense local information.The experiments on the prostate,atrium,and pancreas MR/CT image datasets consistently demonstrate the advantage of the proposed method over six popular segmentation models in both qualitative evaluation and quantitative indexes such as dice similarity coefficient,Intersection over Union,95%Hausdorff distance and average symmetric surface distance.
基金supported by the National Key R&D Program of China(No.2022YFB4301102).
文摘Currently,most trains are equipped with dedicated cameras for capturing pantograph videos.Pantographs are core to the high-speed-railway pantograph-catenary system,and their failure directly affects the normal operation of high-speed trains.However,given the complex and variable real-world operational conditions of high-speed railways,there is no real-time and robust pantograph fault-detection method capable of handling large volumes of surveillance video.Hence,it is of paramount importance to maintain real-time monitoring and analysis of pantographs.Our study presents a real-time intelligent detection technology for identifying faults in high-speed railway pantographs,utilizing a fusion of self-attention and convolution features.We delved into lightweight multi-scale feature-extraction and fault-detection models based on deep learning to detect pantograph anomalies.Compared with traditional methods,this approach achieves high recall and accuracy in pantograph recognition,accurately pinpointing issues like discharge sparks,pantograph horns,and carbon pantograph-slide malfunctions.After experimentation and validation with actual surveillance videos of electric multiple-unit train,our algorithmic model demonstrates real-time,high-accuracy performance even under complex operational conditions.
基金supported by the National Natural Science Foundation of China(Grant Nos.62472149,62376089,62202147)Hubei Provincial Science and Technology Plan Project(2023BCB04100).
文摘Accurate traffic flow prediction has a profound impact on modern traffic management. Traffic flow has complex spatial-temporal correlations and periodicity, which poses difficulties for precise prediction. To address this problem, a Multi-head Self-attention and Spatial-Temporal Graph Convolutional Network (MSSTGCN) for multiscale traffic flow prediction is proposed. Firstly, to capture the hidden traffic periodicity of traffic flow, traffic flow is divided into three kinds of periods, including hourly, daily, and weekly data. Secondly, a graph attention residual layer is constructed to learn the global spatial features across regions. Local spatial-temporal dependence is captured by using a T-GCN module. Thirdly, a transformer layer is introduced to learn the long-term dependence in time. A position embedding mechanism is introduced to label position information for all traffic sequences. Thus, this multi-head self-attention mechanism can recognize the sequence order and allocate weights for different time nodes. Experimental results on four real-world datasets show that the MSSTGCN performs better than the baseline methods and can be successfully adapted to traffic prediction tasks.
基金supported by the National Natural Science Foundation of China(No.52277055).
文摘Traditional data-driven fault diagnosis methods depend on expert experience to manually extract effective fault features of signals,which has certain limitations.Conversely,deep learning techniques have gained prominence as a central focus of research in the field of fault diagnosis by strong fault feature extraction ability and end-to-end fault diagnosis efficiency.Recently,utilizing the respective advantages of convolution neural network(CNN)and Transformer in local and global feature extraction,research on cooperating the two have demonstrated promise in the field of fault diagnosis.However,the cross-channel convolution mechanism in CNN and the self-attention calculations in Transformer contribute to excessive complexity in the cooperative model.This complexity results in high computational costs and limited industrial applicability.To tackle the above challenges,this paper proposes a lightweight CNN-Transformer named as SEFormer for rotating machinery fault diagnosis.First,a separable multiscale depthwise convolution block is designed to extract and integrate multiscale feature information from different channel dimensions of vibration signals.Then,an efficient self-attention block is developed to capture critical fine-grained features of the signal from a global perspective.Finally,experimental results on the planetary gearbox dataset and themotor roller bearing dataset prove that the proposed framework can balance the advantages of robustness,generalization and lightweight compared to recent state-of-the-art fault diagnosis models based on CNN and Transformer.This study presents a feasible strategy for developing a lightweight rotating machinery fault diagnosis framework aimed at economical deployment.
基金partially supported by the National Natural Science Foundation of China(62075137)the Guangdong Basic and Applied Basic Research Foundation(2023A1515140161)+3 种基金the Guangxi Natural Science Foundation of China(2021JJB 110003)the Dongguan Science and Technology of Social Development Program(20231800936312)the high-level talent program of Dongguan University of Technology(No.221110080)the Sanming Project of Medicine in Shenzhen(No.SZSM202103014).
文摘Many spore-forming Bacillus species can cause serious human diseases,because of accidental Bacillusspore infection.Thus,developing an identification strategy with both high sensitivity and specificity is greatly in demand.In this work,we proposed a novel approach named multi-head self-attention mechanism-guided neural network Raman platform to identify living Bacillus spores within a single-cell resolution.The multi-head self-attention mechanism-guided neural network Raman platform was created by combining single-cell Raman spectroscopy,convolutional neural network(CNN),and multi-head self-attention mechanism.To address the limited size of the original spectra dataset,Gaussian noise-based spectra augmentation was employed to increase the number of single-cell Raman spectra datasets for CNN training.Owing to the assistance of both spectra augmentation and multi-head self-attention mechanism,the obtained prediction accuracy of five Bacillus spore species was further improved from 92.29±0.82%to 99.43±0.15%.To figure out the spectra differences covered by the multi-head self-attention mechanism-guided CNN,the relative classification weight from typical Raman bands was visualized via multi-head self-attention mechanism curve.In the process of spectra augmentation from 0 to 1000,the distribution of relative classification weight varied from a discrete state to a more concentrated phase.More importantly,these highlighted four Raman bands(1017,1449,1576,and 1660 cm^(-1))were assigned large weights,showing that the spectra differences in the Raman bands produced the largest contribution to prediction accuracy.It can be foreseen that,our proposed sorting platform has great potential in accurately identifying Bacillus and its related genera species at a single-cell level.
基金funded by the Deanship of Scientific Research and Libraries at Princess Nourah bint Abdulrahman University,through the“Nafea”Program,Grant No.(NP-45-082).
文摘Sustainable energy systems will entail a change in the carbon intensity projections,which should be carried out in a proper manner to facilitate the smooth running of the grid and reduce greenhouse emissions.The present article outlines the TransCarbonNet,a novel hybrid deep learning framework with self-attention characteristics added to the bidirectional Long Short-Term Memory(Bi-LSTM)network to forecast the carbon intensity of the grid several days.The proposed temporal fusion model not only learns the local temporal interactions but also the long-term patterns of the carbon emission data;hence,it is able to give suitable forecasts over a period of seven days.TransCarbonNet takes advantage of a multi-head self-attention element to identify significant temporal connections,which means the Bi-LSTM element calculates sequential dependencies in both directions.Massive tests on two actual data sets indicate much improved results in comparison with the existing results,with mean relative errors of 15.3 percent and 12.7 percent,respectively.The framework has given explicable weights of attention that reveal critical periods that influence carbon intensity alterations,and informed decisions on the management of carbon sustainability.The effectiveness of the proposed solution has been validated in numerous cases of operations,and TransCarbonNet is established to be an effective tool when it comes to carbon-friendly optimization of the grid.
基金supported in part by the National Natural Science Foundation of China under Grant 62172368the Natural Science Foundation of Zhejiang Province under Grant LR22F020003.
文摘The development of brain-computer interfaces(BCI)based on motor imagery(MI)has greatly improved patients’quality of life with movement disorders.The classification of upper limb MI has been widely studied and applied in many fields,including rehabilitation.However,the physiological representations of left and right lower limb movements are too close and activated deep in the cerebral cortex,making it difficult to distinguish their features.Therefore,classifying lower limbs motor imagery is more challenging.In this study,we propose a feature extraction method based on functional connectivity,which utilizes phase-locked values to construct a functional connectivity matrix as the features of the left and right legs,which can effectively avoid the problem of physiological representations of the left and right lower limbs being too close to each other during movement.In addition,considering the topology and the temporal characteristics of the electroencephalogram(EEG),we designed a temporal-spatial convolutional network(TSGCN)to capture the spatiotemporal information for classification.Experimental results show that the accuracy of the proposed method is higher than that of existing methods,achieving an average classification accuracy of 73.58%on the internal dataset.Finally,this study explains the network mechanism of left and right foot MI from the perspective of graph theoretic features and demonstrates the feasibility of decoding lower limb MI.
基金National Natural Science Foundation of China(Grant No.62103434)National Science Fund for Distinguished Young Scholars(Grant No.62176263).
文摘With network attack technology continuing to develop,traditional anomaly traffic detection methods that rely on feature engineering are increasingly insufficient in efficiency and accuracy.Graph Neural Network(GNN),a promising Deep Learning(DL)approach,has proven to be highly effective in identifying intricate patterns in graph⁃structured data and has already found wide applications in the field of network security.In this paper,we propose a hybrid Graph Convolutional Network(GCN)⁃GraphSAGE model for Anomaly Traffic Detection,namely HGS⁃ATD,which aims to improve the accuracy of anomaly traffic detection by leveraging edge feature learning to better capture the relationships between network entities.We validate the HGS⁃ATD model on four publicly available datasets,including NF⁃UNSW⁃NB15⁃v2.The experimental results show that the enhanced hybrid model is 5.71%to 10.25%higher than the baseline model in terms of accuracy,and the F1⁃score is 5.53%to 11.63%higher than the baseline model,proving that the model can effectively distinguish normal traffic from attack traffic and accurately classify various types of attacks.
基金Support by Sichuan Science and Technology Program[2023YFSY0026,2023YFH0004]Guangzhou Huashang University[2024HSZD01,HS2023JYSZH01].
文摘Graph neural networks(GNN)have shown strong performance in node classification tasks,yet most existing models rely on uniform or shared weight aggregation,lacking flexibility in modeling the varying strength of relationships among nodes.This paper proposes a novel graph coupling convolutional model that introduces an adaptive weighting mechanism to assign distinct importance to neighboring nodes based on their similarity to the central node.Unlike traditional methods,the proposed coupling strategy enhances the interpretability of node interactions while maintaining competitive classification performance.The model operates in the spatial domain,utilizing adjacency list structures for efficient convolution and addressing the limitations of weight sharing through a coupling-based similarity computation.Extensive experiments are conducted on five graph-structured datasets,including Cora,Citeseer,PubMed,Reddit,and BlogCatalog,as well as a custom topology dataset constructed from the Open University Learning Analytics Dataset(OULAD)educational platform.Results demonstrate that the proposed model achieves good classification accuracy,while significantly reducing training time through direct second-order neighbor fusion and data preprocessing.Moreover,analysis of neighborhood order reveals that considering third-order neighbors offers limited accuracy gains but introduces considerable computational overhead,confirming the efficiency of first-and second-order convolution in practical applications.Overall,the proposed graph coupling model offers a lightweight,interpretable,and effective framework for multi-label node classification in complex networks.
基金supported by the Natural Science Foundation of China No.62362008the Major Scientific and Technological Special Project of Guizhou Province([2024]014).
文摘With the rapid development of the Artificial Intelligence of Things(AIoT),convolutional neural networks(CNNs)have demonstrated potential and remarkable performance in AIoT applications due to their excellent performance in various inference tasks.However,the users have concerns about privacy leakage for the use of AI and the performance and efficiency of computing on resource-constrained IoT edge devices.Therefore,this paper proposes an efficient privacy-preserving CNN framework(i.e.,EPPA)based on the Fully Homomorphic Encryption(FHE)scheme for AIoT application scenarios.In the plaintext domain,we verify schemes with different activation structures to determine the actual activation functions applicable to the corresponding ciphertext domain.Within the encryption domain,we integrate batch normalization(BN)into the convolutional layers to simplify the computation process.For nonlinear activation functions,we use composite polynomials for approximate calculation.Regarding the noise accumulation caused by homomorphic multiplication operations,we realize the refreshment of ciphertext noise through minimal“decryption-encryption”interactions,instead of adopting bootstrapping operations.Additionally,in practical implementation,we convert three-dimensional convolution into two-dimensional convolution to reduce the amount of computation in the encryption domain.Finally,we conduct extensive experiments on four IoT datasets,different CNN architectures,and two platforms with different resource configurations to evaluate the performance of EPPA in detail.
基金funded by scientific research projects under Grant JY2024B011.
文摘With the increasing complexity of industrial automation,planetary gearboxes play a vital role in largescale equipment transmission systems,directly impacting operational efficiency and safety.Traditional maintenance strategies often struggle to accurately predict the degradation process of equipment,leading to excessive maintenance costs or potential failure risks.However,existing prediction methods based on statistical models are difficult to adapt to nonlinear degradation processes.To address these challenges,this study proposes a novel condition-based maintenance framework for planetary gearboxes.A comprehensive full-lifecycle degradation experiment was conducted to collect raw vibration signals,which were then processed using a temporal convolutional network autoencoder with multi-scale perception capability to extract deep temporal degradation features,enabling the collaborative extraction of longperiod meshing frequencies and short-term impact features from the vibration signals.Kernel principal component analysis was employed to fuse and normalize these features,enhancing the characterization of degradation progression.A nonlinear Wiener process was used to model the degradation trajectory,with a threshold decay function introduced to dynamically adjust maintenance strategies,and model parameters optimized through maximum likelihood estimation.Meanwhile,the maintenance strategy was optimized to minimize costs per unit time,determining the optimal maintenance timing and preventive maintenance threshold.The comprehensive indicator of degradation trends extracted by this method reaches 0.756,which is 41.2%higher than that of traditional time-domain features;the dynamic threshold strategy reduces the maintenance cost per unit time to 55.56,which is 8.9%better than that of the static threshold optimization.Experimental results demonstrate significant reductions in maintenance costs while enhancing system reliability and safety.This study realizes the organic integration of deep learning and reliability theory in the maintenance of planetary gearboxes,provides an interpretable solution for the predictive maintenance of complex mechanical systems,and promotes the development of condition-based maintenance strategies for planetary gearboxes.
基金Supported by the National Key Research and Development Program of China(No.2023YFC3008202)the National Natural Science Foundation of China(No.42406019)the Scientific Research Fund of Zhejiang Provincial Education Department(No.Y202353066)。
文摘The prediction of sea surface partial pressure of carbon dioxide(pCO_(2))in the South China Sea is crucial for understanding the region’s contribution to the global carbon budget and its interactions with climate change.We applied the Spatiotemporal Convolutional Long Short-Term Memory(STConvLSTM)model,integrating key environmental factors including sea surface temperature(SST),sea surface salinity(SSS),and chlorophyll a(Chl a),to predict and analyze sea surface pCO_(2)in the South China Sea.The model demonstrated high accuracy in short-term predictions(1 month),with a mean absolute error(MAE)of 0.394,a root mean square error(RMSE)of 0.659,and a coefficient of determination(R^(2))of 0.998.For long-term predictions(12 months),the model maintained its predictive capability,with an MAE of 0.667,RMSE of 1.255,and R^(2)of 0.994.Feature importance analysis revealed that sea surface pCO_(2)and SST were the main drivers of the model’s predictions,whereas Chl a and SSS had relatively minor impacts.The model’s generalization ability was further validated in the northwest Pacific Ocean and tropical Pacific Ocean,where it successfully captured the spatiotemporal variation in pCO_(2)with small prediction errors.The ST-ConvLSTM model provides an efficient and accurate tool for forecasting and analyzing sea surface pCO_(2)in the South China Sea,offering new insights into global carbon cycling and climate change.This study demonstrates the potential of deep learning in marine science and provides a significant technical support for global changes and marine ecosystem research.
基金funded by the Ministry of Higher Education(MoHE)Malaysia through the Fundamental Research Grant Scheme—Early Career Researcher(FRGS-EC),grant number FRGSEC/1/2024/ICT02/UNIMAP/02/8.
文摘critical for guiding treatment and improving patient outcomes.Traditional molecular subtyping via immuno-histochemistry(IHC)test is invasive,time-consuming,and may not fully represent tumor heterogeneity.This study proposes a non-invasive approach using digital mammography images and deep learning algorithm for classifying breast cancer molecular subtypes.Four pretrained models,including two Convolutional Neural Networks(MobileNet_V3_Large and VGG-16)and two Vision Transformers(ViT_B_16 and ViT_Base_Patch16_Clip_224)were fine-tuned to classify images into HER2-enriched,Luminal,Normal-like,and Triple Negative subtypes.Hyperparameter tuning,including learning rate adjustment and layer freezing strategies,was applied to optimize performance.Among the evaluated models,ViT_Base_Patch16_Clip_224 achieved the highest test accuracy(94.44%),with equally high precision,recall,and F1-score of 0.94,demonstrating excellent generalization.MobileNet_V3_Large achieved the same accuracy but showed less training stability.In contrast,VGG-16 recorded the lowest performance,indicating a limitation in its generalizability for this classification task.The study also highlighted the superior performance of the Vision Transformer models over CNNs,particularly due to their ability to capture global contextual features and the benefit of CLIP-based pretraining in ViT_Base_Patch16_Clip_224.To enhance clinical applicability,a graphical user interface(GUI)named“BCMS Dx”was developed for streamlined subtype prediction.Deep learning applied to mammography has proven effective for accurate and non-invasive molecular subtyping.The proposed Vision Transformer-based model and supporting GUI offer a promising direction for augmenting diagnostic workflows,minimizing the need for invasive procedures,and advancing personalized breast cancer management.
基金The work described in this paper was fully supported by a grant from Hong Kong Metropolitan University(RIF/2021/05).
文摘Parkinson’s disease(PD)is a debilitating neurological disorder affecting over 10 million people worldwide.PD classification models using voice signals as input are common in the literature.It is believed that using deep learning algorithms further enhances performance;nevertheless,it is challenging due to the nature of small-scale and imbalanced PD datasets.This paper proposed a convolutional neural network-based deep support vector machine(CNN-DSVM)to automate the feature extraction process using CNN and extend the conventional SVM to a DSVM for better classification performance in small-scale PD datasets.A customized kernel function reduces the impact of biased classification towards the majority class(healthy candidates in our consideration).An improved generative adversarial network(IGAN)was designed to generate additional training data to enhance the model’s performance.For performance evaluation,the proposed algorithm achieves a sensitivity of 97.6%and a specificity of 97.3%.The performance comparison is evaluated from five perspectives,including comparisons with different data generation algorithms,feature extraction techniques,kernel functions,and existing works.Results reveal the effectiveness of the IGAN algorithm,which improves the sensitivity and specificity by 4.05%–4.72%and 4.96%–5.86%,respectively;and the effectiveness of the CNN-DSVM algorithm,which improves the sensitivity by 1.24%–57.4%and specificity by 1.04%–163%and reduces biased detection towards the majority class.The ablation experiments confirm the effectiveness of individual components.Two future research directions have also been suggested.
文摘Lightweight convolutional neural networks(CNNs)have simple structures but struggle to comprehensively and accurately extract important semantic information from images.While attention mechanisms can enhance CNNs by learning distinctive representations,most existing spatial and hybrid attention methods focus on local regions with extensive parameters,making them unsuitable for lightweight CNNs.In this paper,we propose a self-attention mechanism tailored for lightweight networks,namely the brief self-attention module(BSAM).BSAM consists of the brief spatial attention(BSA)and advanced channel attention blocks.Unlike conventional self-attention methods with many parameters,our BSA block improves the performance of lightweight networks by effectively learning global semantic representations.Moreover,BSAM can be seamlessly integrated into lightweight CNNs for end-to-end training,maintaining the network’s lightweight and mobile characteristics.We validate the effectiveness of the proposed method on image classification tasks using the Food-101,Caltech-256,and Mini-ImageNet datasets.
基金Deputy for Research and Innovation-Ministry of Education,Kingdom of Saudi Arabia,Grant/Award Number:NU/IFC/02/SERC/-/31Institutional Funding Committee at Najran University,Kingdom of Saudi Arabia。
文摘Industrial Internet of Things(IIoT)is a pervasive network of interlinked smart devices that provide a variety of intelligent computing services in industrial environments.Several IIoT nodes operate confidential data(such as medical,transportation,military,etc.)which are reachable targets for hostile intruders due to their openness and varied structure.Intrusion Detection Systems(IDS)based on Machine Learning(ML)and Deep Learning(DL)techniques have got significant attention.However,existing ML and DL-based IDS still face a number of obstacles that must be overcome.For instance,the existing DL approaches necessitate a substantial quantity of data for effective performance,which is not feasible to run on low-power and low-memory devices.Imbalanced and fewer data potentially lead to low performance on existing IDS.This paper proposes a self-attention convolutional neural network(SACNN)architecture for the detection of malicious activity in IIoT networks and an appropriate feature extraction method to extract the most significant features.The proposed architecture has a self-attention layer to calculate the input attention and convolutional neural network(CNN)layers to process the assigned attention features for prediction.The performance evaluation of the proposed SACNN architecture has been done with the Edge-IIoTset and X-IIoTID datasets.These datasets encompassed the behaviours of contemporary IIoT communication protocols,the operations of state-of-the-art devices,various attack types,and diverse attack scenarios.
基金sponsored by the National Natural Science Foundation of China under Grants 62271264,61972207,and 42175194the Project through the Priority Academic Program Development(PAPD)of Jiangsu Higher Education Institution.
文摘Traditional based deep learning intrusion detection methods face problems such as insufficient cloud storage,data privacy leaks,high com-munication costs,unsatisfactory detection rates,and false positive rate.To address existing issues in intrusion detection,this paper presents a novel approach called CS-FL,which combines Federated Learning and a Self-Attention Fusion Convolutional Neural Network.Federated Learning is a new distributed computing model that enables individual training of client data without uploading local data to a central server.at the same time,local training results are uploaded and integrated across all participating clients to produce a global model.The sharing model reduces communication costs,protects data privacy,and solves problems such as insufficient cloud storage and“data islands”for each client.In the proposed method,a hybrid model is formed by integrating the self-Attention and similar parts of the Convolutional Neural Network in the local data processing.This approach not only enhances the performance of the hybrid model but also reduces computational overhead compared to pure hybrid neural networks.Results from experiments on the NSL-KDD dataset show that the proposed method outperforms other intrusion detection techniques,resulting in a significant improvement in performance.This demonstrates the effectiveness of the proposed approach in improving intrusion detection accuracy.
基金supported by the National Key Research and Development Program of China(No.2021YFA0715900).
文摘Located in northern China,the Hetao Plain is an important agro-economic zone and population centre.The deterioration of local groundwater quality has had a serious impact on human health and economic development.Nowadays,the groundwater vulnerability assessment(GVA)has become an essential task to identify the current status and development trend of groundwater quality.In this study,the Convolutional Neural Network(CNN)and Long Short-Term Memory(LSTM)models are integrated to realize the spatio-temporal prediction of regional groundwater vulnerability by introducing the Self-attention mechanism.The study firstly builds the CNN-LSTM modelwith self-attention(SA)mechanism and evaluates the prediction accuracy of the model for groundwater vulnerability compared to other common machine learning models such as Support Vector Machine(SVM),Random Forest(RF),and Extreme Gradient Boosting(XGBoost).The results indicate that the CNNLSTM model outperforms thesemodels,demonstrating its significance in groundwater vulnerability assessment.It can be posited that the predictions indicate an increased risk of groundwater vulnerability in the study area over the coming years.This increase can be attributed to the synergistic impact of global climate anomalies and intensified local human activities.Moreover,the overall groundwater vulnerability risk in the entire region has increased,evident fromboth the notably high value and standard deviation.This suggests that the spatial variability of groundwater vulnerability in the area is expected to expand in the future due to the sustained progression of climate change and human activities.The model can be optimized for diverse applications across regional environmental assessment,pollution prediction,and risk statistics.This study holds particular significance for ecological protection and groundwater resource management.
基金supported by the National Key Research and Development Program of China[grant No.2016YF B0502200]the Postdoctoral Research Foundation of China[grant No.2020M682480]the Fundamental Research Funds for the Central Universities[grant No.2042021kf0009]。
文摘The Ultra-Wideband(UWB)Location-Based Service is receiving more and more attention due to its high ranging accuracy and good time resolution.However,the None-Line-of-Sight(NLOS)propagation may reduce the ranging accuracy for UWB localization system in indoor environment.So it is important to identify LOS and NLOS propagations before taking proper measures to improve the UWB localization accuracy.In this paper,a deep learning-based UWB NLOS/LOS classification algorithm called FCN-Attention is proposed.The proposed FCN-Attention algorithm utilizes a Fully Convolution Network(FCN)for improving feature extraction ability and a self-attention mechanism for enhancing feature description from the data to improve the classification accuracy.The proposed algorithm is evaluated using an open-source dataset,a local collected dataset and a mixed dataset created from these two datasets.The experiment result shows that the proposed FCN-Attention algorithm achieves classification accuracy of 88.24%on the open-source dataset,100%on the local collected dataset and 92.01%on the mixed dataset,which is better than the results from other evaluated NLOS/LOS classification algorithms in most scenarios in this paper.
基金supported by the National Natural Science Foundation of China(62272049,62236006,62172045)the Key Projects of Beijing Union University(ZKZD202301).
文摘In recent years,gait-based emotion recognition has been widely applied in the field of computer vision.However,existing gait emotion recognition methods typically rely on complete human skeleton data,and their accuracy significantly declines when the data is occluded.To enhance the accuracy of gait emotion recognition under occlusion,this paper proposes a Multi-scale Suppression Graph ConvolutionalNetwork(MS-GCN).TheMS-GCN consists of three main components:Joint Interpolation Module(JI Moudle),Multi-scale Temporal Convolution Network(MS-TCN),and Suppression Graph Convolutional Network(SGCN).The JI Module completes the spatially occluded skeletal joints using the(K-Nearest Neighbors)KNN interpolation method.The MS-TCN employs convolutional kernels of various sizes to comprehensively capture the emotional information embedded in the gait,compensating for the temporal occlusion of gait information.The SGCN extracts more non-prominent human gait features by suppressing the extraction of key body part features,thereby reducing the negative impact of occlusion on emotion recognition results.The proposed method is evaluated on two comprehensive datasets:Emotion-Gait,containing 4227 real gaits from sources like BML,ICT-Pollick,and ELMD,and 1000 synthetic gaits generated using STEP-Gen technology,and ELMB,consisting of 3924 gaits,with 1835 labeled with emotions such as“Happy,”“Sad,”“Angry,”and“Neutral.”On the standard datasets Emotion-Gait and ELMB,the proposed method achieved accuracies of 0.900 and 0.896,respectively,attaining performance comparable to other state-ofthe-artmethods.Furthermore,on occlusion datasets,the proposedmethod significantly mitigates the performance degradation caused by occlusion compared to other methods,the accuracy is significantly higher than that of other methods.
