In the current digital era,new technologies are becoming an essential part of our lives.Consequently,the number ofmalicious software ormalware attacks is rapidly growing.There is no doubt,themajority ofmalware attacks...In the current digital era,new technologies are becoming an essential part of our lives.Consequently,the number ofmalicious software ormalware attacks is rapidly growing.There is no doubt,themajority ofmalware attacks can be detected by most antivirus programs.However,such types of antivirus programs are one step behind malicious software.Due to these dilemmas,deep learning become popular in the detection and classification of malicious data.Therefore,researchers have significantly focused on finding solutions for malware attacks by analyzing malicious samples with the help of different techniques and models.In this research,we presented a lightweight attention-based novel deep Convolutional Neural Network(DNN-CNN)model for binary and multi-class malware classification,including benign,trojan horse,ransomware,and spyware.We applied the Principal Component Analysis(PCA)technique for feature extraction for binary classification.We used the Synthetic Minority Oversampling Technique(SMOTE)to handle the imbalanced data during multi-class classification.Our proposed attention-based malware detectionmodel is trained on the benchmarkmalware memory dataset named CIC-MalMem-2022.Theresults indicate that our model obtained high accuracy for binary and multi-class classification,99.5% and 97.9%,respectively.展开更多
Dynamic sign language recognition holds significant importance, particularly with the application of deep learning to address its complexity. However, existing methods face several challenges. Firstly, recognizing dyn...Dynamic sign language recognition holds significant importance, particularly with the application of deep learning to address its complexity. However, existing methods face several challenges. Firstly, recognizing dynamic sign language requires identifying keyframes that best represent the signs, and missing these keyframes reduces accuracy. Secondly, some methods do not focus enough on hand regions, which are small within the overall frame, leading to information loss. To address these challenges, we propose a novel Video Transformer Attention-based Network (VTAN) for dynamic sign language recognition. Our approach prioritizes informative frames and hand regions effectively. To tackle the first issue, we designed a keyframe extraction module enhanced by a convolutional autoencoder, which focuses on selecting information-rich frames and eliminating redundant ones from the video sequences. For the second issue, we developed a soft attention-based transformer module that emphasizes extracting features from hand regions, ensuring that the network pays more attention to hand information within sequences. This dual-focus approach improves effective dynamic sign language recognition by addressing the key challenges of identifying critical frames and emphasizing hand regions. Experimental results on two public benchmark datasets demonstrate the effectiveness of our network, outperforming most of the typical methods in sign language recognition tasks.展开更多
Electrocardiogram(ECG)signal is one of the noninvasive physiological measurement techniques commonly usedin cardiac diagnosis.However,in real scenarios,the ECGsignal is susceptible to various noise erosion,which affec...Electrocardiogram(ECG)signal is one of the noninvasive physiological measurement techniques commonly usedin cardiac diagnosis.However,in real scenarios,the ECGsignal is susceptible to various noise erosion,which affectsthe subsequent pathological analysis.Therefore,the effective removal of the noise from ECG signals has becomea top priority in cardiac diagnostic research.Aiming at the problem of incomplete signal shape retention andlow signal-to-noise ratio(SNR)after denoising,a novel ECG denoising network,named attention-based residualdense shrinkage network(ARDSN),is proposed in this paper.Firstly,the shallow ECG characteristics are extractedby a shallow feature extraction network(SFEN).Then,the residual dense shrinkage attention block(RDSAB)isused for adaptive noise suppression.Finally,feature fusion representation(FFR)is performed on the hierarchicalfeatures extracted by a series of RDSABs to reconstruct the de-noised ECG signal.Experiments on the MIT-BIHarrhythmia database and MIT-BIH noise stress test database indicate that the proposed scheme can effectively resistthe interference of different sources of noise on the ECG signal.展开更多
The growing number of COVID-19 cases puts pressure on healthcare services and public institutions worldwide.The pandemic has brought much uncertainty to the global economy and the situation in general.Forecasting meth...The growing number of COVID-19 cases puts pressure on healthcare services and public institutions worldwide.The pandemic has brought much uncertainty to the global economy and the situation in general.Forecasting methods and modeling techniques are important tools for governments to manage critical situations caused by pandemics,which have negative impact on public health.The main purpose of this study is to obtain short-term forecasts of disease epidemiology that could be useful for policymakers and public institutions to make necessary short-term decisions.To evaluate the effectiveness of the proposed attention-based method combining certain data mining algorithms and the classical ARIMA model for short-term forecasts,data on the spread of the COVID-19 virus in Lithuania is used,the forecasts of epidemic dynamics were examined,and the results were presented in the study.Nevertheless,the approach presented might be applied to any country and other pandemic situations.The COVID-19 outbreak started at different times in different countries,hence some countries have a longer history of the disease with more historical data than others.The paper proposes a novel approach to data registration and machine learning-based analysis using data from attention-based countries for forecast validation to predict trends of the spread of COVID-19 and assess risks.展开更多
In the present technological world,surveillance cameras generate an immense amount of video data from various sources,making its scrutiny tough for computer vision specialists.It is difficult to search for anomalous e...In the present technological world,surveillance cameras generate an immense amount of video data from various sources,making its scrutiny tough for computer vision specialists.It is difficult to search for anomalous events manually in thesemassive video records since they happen infrequently and with a low probability in real-world monitoring systems.Therefore,intelligent surveillance is a requirement of the modern day,as it enables the automatic identification of normal and aberrant behavior using artificial intelligence and computer vision technologies.In this article,we introduce an efficient Attention-based deep-learning approach for anomaly detection in surveillance video(ADSV).At the input of the ADSV,a shots boundary detection technique is used to segment prominent frames.Next,The Lightweight ConvolutionNeuralNetwork(LWCNN)model receives the segmented frames to extract spatial and temporal information from the intermediate layer.Following that,spatial and temporal features are learned using Long Short-Term Memory(LSTM)cells and Attention Network from a series of frames for each anomalous activity in a sample.To detect motion and action,the LWCNN received chronologically sorted frames.Finally,the anomaly activity in the video is identified using the proposed trained ADSV model.Extensive experiments are conducted on complex and challenging benchmark datasets.In addition,the experimental results have been compared to state-ofthe-artmethodologies,and a significant improvement is attained,demonstrating the efficiency of our ADSV method.展开更多
Background Accurate early warning of sepsis onset is crucial for reducing mortality.However,the inter-individual heterogeneity in clinical manifestations of sepsis leads to significant sparsity of data.The current tim...Background Accurate early warning of sepsis onset is crucial for reducing mortality.However,the inter-individual heterogeneity in clinical manifestations of sepsis leads to significant sparsity of data.The current time series analysis methods attempt to interpolate highly sparse sepsis data,yielding unsatisfactory results.In this study,we aimed to develop an efficient artificial intelligence approach for early warning of sepsis onset.Methods The I2former model,an incident-induced attention-based architecture,was proposed to address the challenges posed by sparse medical data.This model employs a novel increment entropy encoding strategy to extract clinically significant features from sparse data,effectively transforming the unavailable data into valuable insights.The training data were sourced from MIMIC-IV v2.2 and eICU v2.0,with external validation from Beijing Tsinghua Changgung Hospital.Five advanced models,including the Autoformer,Timesnet,Informer,Reformer,and DLinear,currently in use were used for comparison.Results Five metrics used for classification indicated that the I2former significantly outperformed the 5 advanced time series analysis methods,achieving area under the receiver operating characteristic(AUROC),area under the precision-recall curve(AUPRC),Matthews correlation coefficient(MCC),F1-score,and accuracy of 0.886,0.529,0.449,and 0.917,respectively.Furthermore,external validation using the data from Beijing Tsinghua Changgung Hospital demonstrated that the model provides accurate early warnings,on average of 15.5 h prior to sepsis onset.Conclusion Therefore,I2former is proposed for accurate early warning of sepsis onset.Five crucial metrics for classification underscored the substantial advantages of I2former in managing sparse data,while highlighting its potential application and value in the field of medical data analysis.展开更多
Intra refresh is an efficient and simple technique for suppressing temporal error propagation in video transmission over error-prone networks. However, most existing intra refresh algorithms do not make good use of th...Intra refresh is an efficient and simple technique for suppressing temporal error propagation in video transmission over error-prone networks. However, most existing intra refresh algorithms do not make good use of the visual perceptual mechanism of the Human Visual System （HVS）. This paper presents an intra refresh algorithm based on an attention model and a loss impact model. Intra refresh rates are allowed to vary for different image regions according to the HVS characteristic and the channel conditions to protect the most important macro-blocks against packet loss. A joint source-channel rate-distortion model was de- veloped taking into account the HVS characteristic to achieve an optimal end-to-end distortion for a better subjective quality. Tests demonstrate that, for the same bit rate and packet loss rate, this method provides a much better subjective feeling than existing intra refresh methods.展开更多
Environmental parameter data collected by sensors for monitoring the environment of agricultural facility operations are usually incomplete due to external environmental disturbances and device failures.And the missin...Environmental parameter data collected by sensors for monitoring the environment of agricultural facility operations are usually incomplete due to external environmental disturbances and device failures.And the missing of collected data is completely at random.In practice,missing data could create biased estimations and make multivariate time series predictions of environmental parameters difficult,leading to imprecise environmental control.A multivariate time series imputation model based on generative adversarial networks and multi-head attention(ATTN-GAN)is proposed in this work to reducing the negative consequence of missing data.ATTN-GAN can capture the temporal and spatial correlation of time series,and has a good capacity to learn data distribution.In the downstream experiments,we used ATTN-GAN and baseline models for data imputation,and predicted the imputed data,respectively.For the imputation of missing data,over the 20%,50%and 80%missing rate,ATTN-GAN had the lowest RMSE,0.1593,0.2012 and 0.2688 respectively.For water temperature prediction,data processed with ATTN-GAN over MLP,LSTM,DA-RNN prediction methods had the lowest MSE,0.6816,0.8375 and 0.3736 respectively.Those results revealed that ATTN-GAN outperformed all baseline models in terms of data imputation accuracy.The data processed by ATTN-GAN is the best for time series prediction.展开更多
The attention-based encoder-decoder technique,known as the trans-former,is used to enhance the performance of end-to-end automatic speech recognition(ASR).This research focuses on applying ASR end-toend transformer-ba...The attention-based encoder-decoder technique,known as the trans-former,is used to enhance the performance of end-to-end automatic speech recognition(ASR).This research focuses on applying ASR end-toend transformer-based models for the Arabic language,as the researchers’community pays little attention to it.The Muslims Holy Qur’an book is written using Arabic diacritized text.In this paper,an end-to-end transformer model to building a robust Qur’an vs.recognition is proposed.The acoustic model was built using the transformer-based model as deep learning by the PyTorch framework.A multi-head attention mechanism is utilized to represent the encoder and decoder in the acoustic model.AMel filter bank is used for feature extraction.To build a language model(LM),the Recurrent Neural Network(RNN)and Long short-term memory(LSTM)were used to train an n-gram word-based LM.As a part of this research,a new dataset of Qur’an verses and their associated transcripts were collected and processed for training and evaluating the proposed model,consisting of 10 h of.wav recitations performed by 60 reciters.The experimental results showed that the proposed end-to-end transformer-based model achieved a significant low character error rate(CER)of 1.98%and a word error rate(WER)of 6.16%.We have achieved state-of-the-art end-to-end transformer-based recognition for Qur’an reciters.展开更多
We advance the attention-based view by presenting empirical evidence that the attention of headquarters and subsidiary managers in a multi-business organization is,at times,out of sync.Based on empirical material that...We advance the attention-based view by presenting empirical evidence that the attention of headquarters and subsidiary managers in a multi-business organization is,at times,out of sync.Based on empirical material that allows us to differentiate between what is attended to and what is ignored by management,we analyze the focus of managerial attention,environmental and subsidiary stimuli,and actions taken in the decision-making process over 15 years,during a period of strategic transformation.We suggest that attentional mismatches occur not only between strategic issues but also between what are considered relevant responses or actions to be taken at any particular time.We analyze the origins of the attentional mismatches and explore ways to avoid such nonalignment in strategic decision making.展开更多
基金funded by Naif Arab University for Security Sciences under grant No.NAUSS-23-R11.
文摘In the current digital era,new technologies are becoming an essential part of our lives.Consequently,the number ofmalicious software ormalware attacks is rapidly growing.There is no doubt,themajority ofmalware attacks can be detected by most antivirus programs.However,such types of antivirus programs are one step behind malicious software.Due to these dilemmas,deep learning become popular in the detection and classification of malicious data.Therefore,researchers have significantly focused on finding solutions for malware attacks by analyzing malicious samples with the help of different techniques and models.In this research,we presented a lightweight attention-based novel deep Convolutional Neural Network(DNN-CNN)model for binary and multi-class malware classification,including benign,trojan horse,ransomware,and spyware.We applied the Principal Component Analysis(PCA)technique for feature extraction for binary classification.We used the Synthetic Minority Oversampling Technique(SMOTE)to handle the imbalanced data during multi-class classification.Our proposed attention-based malware detectionmodel is trained on the benchmarkmalware memory dataset named CIC-MalMem-2022.Theresults indicate that our model obtained high accuracy for binary and multi-class classification,99.5% and 97.9%,respectively.
基金supported by the National Natural Science Foundation of China under Grant Nos.62076117 and 62166026the Jiangxi Provincial Key Laboratory of Virtual Reality under Grant No.2024SSY03151.
文摘Dynamic sign language recognition holds significant importance, particularly with the application of deep learning to address its complexity. However, existing methods face several challenges. Firstly, recognizing dynamic sign language requires identifying keyframes that best represent the signs, and missing these keyframes reduces accuracy. Secondly, some methods do not focus enough on hand regions, which are small within the overall frame, leading to information loss. To address these challenges, we propose a novel Video Transformer Attention-based Network (VTAN) for dynamic sign language recognition. Our approach prioritizes informative frames and hand regions effectively. To tackle the first issue, we designed a keyframe extraction module enhanced by a convolutional autoencoder, which focuses on selecting information-rich frames and eliminating redundant ones from the video sequences. For the second issue, we developed a soft attention-based transformer module that emphasizes extracting features from hand regions, ensuring that the network pays more attention to hand information within sequences. This dual-focus approach improves effective dynamic sign language recognition by addressing the key challenges of identifying critical frames and emphasizing hand regions. Experimental results on two public benchmark datasets demonstrate the effectiveness of our network, outperforming most of the typical methods in sign language recognition tasks.
基金the National Natural Science Foundation of China under Grant 62172059 and 62072055Hunan Provincial Natural Science Foundations of China under Grant 2022JJ50318 and 2022JJ30621Scientific Research Fund of Hunan Provincial Education Department of China under Grant 22A0200 and 20K098。
文摘Electrocardiogram(ECG)signal is one of the noninvasive physiological measurement techniques commonly usedin cardiac diagnosis.However,in real scenarios,the ECGsignal is susceptible to various noise erosion,which affectsthe subsequent pathological analysis.Therefore,the effective removal of the noise from ECG signals has becomea top priority in cardiac diagnostic research.Aiming at the problem of incomplete signal shape retention andlow signal-to-noise ratio(SNR)after denoising,a novel ECG denoising network,named attention-based residualdense shrinkage network(ARDSN),is proposed in this paper.Firstly,the shallow ECG characteristics are extractedby a shallow feature extraction network(SFEN).Then,the residual dense shrinkage attention block(RDSAB)isused for adaptive noise suppression.Finally,feature fusion representation(FFR)is performed on the hierarchicalfeatures extracted by a series of RDSABs to reconstruct the de-noised ECG signal.Experiments on the MIT-BIHarrhythmia database and MIT-BIH noise stress test database indicate that the proposed scheme can effectively resistthe interference of different sources of noise on the ECG signal.
基金This project has received funding from the Research Council of Lithuania(LMTLT),agreement No S-COV-20-4.
文摘The growing number of COVID-19 cases puts pressure on healthcare services and public institutions worldwide.The pandemic has brought much uncertainty to the global economy and the situation in general.Forecasting methods and modeling techniques are important tools for governments to manage critical situations caused by pandemics,which have negative impact on public health.The main purpose of this study is to obtain short-term forecasts of disease epidemiology that could be useful for policymakers and public institutions to make necessary short-term decisions.To evaluate the effectiveness of the proposed attention-based method combining certain data mining algorithms and the classical ARIMA model for short-term forecasts,data on the spread of the COVID-19 virus in Lithuania is used,the forecasts of epidemic dynamics were examined,and the results were presented in the study.Nevertheless,the approach presented might be applied to any country and other pandemic situations.The COVID-19 outbreak started at different times in different countries,hence some countries have a longer history of the disease with more historical data than others.The paper proposes a novel approach to data registration and machine learning-based analysis using data from attention-based countries for forecast validation to predict trends of the spread of COVID-19 and assess risks.
基金This research was supported by the Chung-Ang University Research Scholarship Grants in 2021 and the Culture,Sports and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture,Sports,and Tourism in 2022(Project Name:Development of Digital Quarantine and Operation Technologies for Creation of Safe Viewing Environment in Cultural Facilities,Project Number:R2021040028,Contribution Rate:100%).
文摘In the present technological world,surveillance cameras generate an immense amount of video data from various sources,making its scrutiny tough for computer vision specialists.It is difficult to search for anomalous events manually in thesemassive video records since they happen infrequently and with a low probability in real-world monitoring systems.Therefore,intelligent surveillance is a requirement of the modern day,as it enables the automatic identification of normal and aberrant behavior using artificial intelligence and computer vision technologies.In this article,we introduce an efficient Attention-based deep-learning approach for anomaly detection in surveillance video(ADSV).At the input of the ADSV,a shots boundary detection technique is used to segment prominent frames.Next,The Lightweight ConvolutionNeuralNetwork(LWCNN)model receives the segmented frames to extract spatial and temporal information from the intermediate layer.Following that,spatial and temporal features are learned using Long Short-Term Memory(LSTM)cells and Attention Network from a series of frames for each anomalous activity in a sample.To detect motion and action,the LWCNN received chronologically sorted frames.Finally,the anomaly activity in the video is identified using the proposed trained ADSV model.Extensive experiments are conducted on complex and challenging benchmark datasets.In addition,the experimental results have been compared to state-ofthe-artmethodologies,and a significant improvement is attained,demonstrating the efficiency of our ADSV method.
基金sponsored by National Key Research and Development Program of China(Grant No.2021YFC2500803)Capital's Funds for Health Improvement and Research(Grant No.2024-1-1202).
文摘Background Accurate early warning of sepsis onset is crucial for reducing mortality.However,the inter-individual heterogeneity in clinical manifestations of sepsis leads to significant sparsity of data.The current time series analysis methods attempt to interpolate highly sparse sepsis data,yielding unsatisfactory results.In this study,we aimed to develop an efficient artificial intelligence approach for early warning of sepsis onset.Methods The I2former model,an incident-induced attention-based architecture,was proposed to address the challenges posed by sparse medical data.This model employs a novel increment entropy encoding strategy to extract clinically significant features from sparse data,effectively transforming the unavailable data into valuable insights.The training data were sourced from MIMIC-IV v2.2 and eICU v2.0,with external validation from Beijing Tsinghua Changgung Hospital.Five advanced models,including the Autoformer,Timesnet,Informer,Reformer,and DLinear,currently in use were used for comparison.Results Five metrics used for classification indicated that the I2former significantly outperformed the 5 advanced time series analysis methods,achieving area under the receiver operating characteristic(AUROC),area under the precision-recall curve(AUPRC),Matthews correlation coefficient(MCC),F1-score,and accuracy of 0.886,0.529,0.449,and 0.917,respectively.Furthermore,external validation using the data from Beijing Tsinghua Changgung Hospital demonstrated that the model provides accurate early warnings,on average of 15.5 h prior to sepsis onset.Conclusion Therefore,I2former is proposed for accurate early warning of sepsis onset.Five crucial metrics for classification underscored the substantial advantages of I2former in managing sparse data,while highlighting its potential application and value in the field of medical data analysis.
基金Supported by the National Natural Science Foundation of China(No. 60572081)
文摘Intra refresh is an efficient and simple technique for suppressing temporal error propagation in video transmission over error-prone networks. However, most existing intra refresh algorithms do not make good use of the visual perceptual mechanism of the Human Visual System （HVS）. This paper presents an intra refresh algorithm based on an attention model and a loss impact model. Intra refresh rates are allowed to vary for different image regions according to the HVS characteristic and the channel conditions to protect the most important macro-blocks against packet loss. A joint source-channel rate-distortion model was de- veloped taking into account the HVS characteristic to achieve an optimal end-to-end distortion for a better subjective quality. Tests demonstrate that, for the same bit rate and packet loss rate, this method provides a much better subjective feeling than existing intra refresh methods.
基金supported by the National Natural Science Foundation of China:“Regularity and prediction model of juvenile fish growth under synergistic effect of water temperature and flow fields in recirculating aquaculture”(Grant No.32373185)2115 Talent Development Program of China Agricultural University,Overseas High-level Youth Talents Program(China Agricultural University,China,Grant No.62339001)+2 种基金China Agricultural University Excellent Talents Plan(Grant No.31051015)Major Science and Technology Innovation Fund 2019 of Shandong Province(Grant No.2019JZZY010703)National Innovation Center for Digital Fishery,and Beijing Engineering and Technology Research Center for Internet of Things in Agriculture.
文摘Environmental parameter data collected by sensors for monitoring the environment of agricultural facility operations are usually incomplete due to external environmental disturbances and device failures.And the missing of collected data is completely at random.In practice,missing data could create biased estimations and make multivariate time series predictions of environmental parameters difficult,leading to imprecise environmental control.A multivariate time series imputation model based on generative adversarial networks and multi-head attention(ATTN-GAN)is proposed in this work to reducing the negative consequence of missing data.ATTN-GAN can capture the temporal and spatial correlation of time series,and has a good capacity to learn data distribution.In the downstream experiments,we used ATTN-GAN and baseline models for data imputation,and predicted the imputed data,respectively.For the imputation of missing data,over the 20%,50%and 80%missing rate,ATTN-GAN had the lowest RMSE,0.1593,0.2012 and 0.2688 respectively.For water temperature prediction,data processed with ATTN-GAN over MLP,LSTM,DA-RNN prediction methods had the lowest MSE,0.6816,0.8375 and 0.3736 respectively.Those results revealed that ATTN-GAN outperformed all baseline models in terms of data imputation accuracy.The data processed by ATTN-GAN is the best for time series prediction.
基金the Chair of Prince Faisal for Artificial Intelligent research(CPFIA),Qassim University through the Project Number QU-CPFAI-2-10-5.
文摘The attention-based encoder-decoder technique,known as the trans-former,is used to enhance the performance of end-to-end automatic speech recognition(ASR).This research focuses on applying ASR end-toend transformer-based models for the Arabic language,as the researchers’community pays little attention to it.The Muslims Holy Qur’an book is written using Arabic diacritized text.In this paper,an end-to-end transformer model to building a robust Qur’an vs.recognition is proposed.The acoustic model was built using the transformer-based model as deep learning by the PyTorch framework.A multi-head attention mechanism is utilized to represent the encoder and decoder in the acoustic model.AMel filter bank is used for feature extraction.To build a language model(LM),the Recurrent Neural Network(RNN)and Long short-term memory(LSTM)were used to train an n-gram word-based LM.As a part of this research,a new dataset of Qur’an verses and their associated transcripts were collected and processed for training and evaluating the proposed model,consisting of 10 h of.wav recitations performed by 60 reciters.The experimental results showed that the proposed end-to-end transformer-based model achieved a significant low character error rate(CER)of 1.98%and a word error rate(WER)of 6.16%.We have achieved state-of-the-art end-to-end transformer-based recognition for Qur’an reciters.
文摘We advance the attention-based view by presenting empirical evidence that the attention of headquarters and subsidiary managers in a multi-business organization is,at times,out of sync.Based on empirical material that allows us to differentiate between what is attended to and what is ignored by management,we analyze the focus of managerial attention,environmental and subsidiary stimuli,and actions taken in the decision-making process over 15 years,during a period of strategic transformation.We suggest that attentional mismatches occur not only between strategic issues but also between what are considered relevant responses or actions to be taken at any particular time.We analyze the origins of the attentional mismatches and explore ways to avoid such nonalignment in strategic decision making.
