Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract loc...Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract local and global features, as well as the lack of effective mechanisms to capture complex interactions between features;Additionally, when increasing the receptive field to obtain deeper feature representations, the reliance on increasing network depth leads to a significant increase in computational resource consumption, affecting the efficiency and performance of detection. Based on these issues, firstly, this paper proposes a network traffic anomaly detection model based on parallel dilated convolution and residual learning (Res-PDC). To better explore the interactive relationships between features, the traffic samples are converted into two-dimensional matrix. A module combining parallel dilated convolutions and residual learning (res-pdc) was designed to extract local and global features of traffic at different scales. By utilizing res-pdc modules with different dilation rates, we can effectively capture spatial features at different scales and explore feature dependencies spanning wider regions without increasing computational resources. Secondly, to focus and integrate the information in different feature subspaces, further enhance and extract the interactions among the features, multi-head attention is added to Res-PDC, resulting in the final model: multi-head attention enhanced parallel dilated convolution and residual learning (MHA-Res-PDC) for network traffic anomaly detection. Finally, comparisons with other machine learning and deep learning algorithms are conducted on the NSL-KDD and CIC-IDS-2018 datasets. The experimental results demonstrate that the proposed method in this paper can effectively improve the detection performance.展开更多
Due to the huge difference of noise distribution,the result of a mixture of multiple noises becomes very complicated.Under normal circumstances,the most common type of mixed noise is to add impulse noise(IN)and then w...Due to the huge difference of noise distribution,the result of a mixture of multiple noises becomes very complicated.Under normal circumstances,the most common type of mixed noise is to add impulse noise(IN)and then white Gaussian noise(AWGN).From the reduction of cascaded IN and AWGN to the latest sparse representation,a great deal of methods has been proposed to reduce this form of mixed noise.However,when the mixed noise is very strong,most methods often produce a lot of artifacts.In order to solve the above problems,we propose a method based on residual learning for the removal of AWGN-IN noise in this paper.By training,our model can obtain stable nonlinear mapping from the images with mixed noise to the clean images.After a series of experiments under different noise settings,the results show that our method is obviously better than the traditional sparse representation and patch based method.Meanwhile,the time of model training and image denoising is greatly reduced.展开更多
Amodel that can obtain rapid and accurate detection of coronavirus disease 2019(COVID-19)plays a significant role in treating and preventing the spread of disease transmission.However,designing such amodel that can ba...Amodel that can obtain rapid and accurate detection of coronavirus disease 2019(COVID-19)plays a significant role in treating and preventing the spread of disease transmission.However,designing such amodel that can balance the detection accuracy andweight parameters ofmemorywell to deploy a mobile device is challenging.Taking this point into account,this paper fuses the convolutional neural network and residual learning operations to build a multi-class classification model,which improves COVID-19 pneumonia detection performance and keeps a trade-off between the weight parameters and accuracy.The convolutional neural network can extract the COVID-19 feature information by repeated convolutional operations.The residual learning operations alleviate the gradient problems caused by stacking convolutional layers and enhance the ability of feature extraction.The ability further enables the proposed model to acquire effective feature information at a lowcost,which canmake ourmodel keep smallweight parameters.Extensive validation and comparison with other models of COVID-19 pneumonia detection on the well-known COVIDx dataset show that(1)the sensitivity of COVID-19 pneumonia detection is improved from 88.2%(non-COVID-19)and 77.5%(COVID-19)to 95.3%(non-COVID-19)and 96.5%(COVID-19),respectively.The positive predictive value is also respectively increased from72.8%(non-COVID-19)and 89.0%(COVID-19)to 88.8%(non-COVID-19)and 95.1%(COVID-19).(2)Compared with the weight parameters of the COVIDNet-small network,the value of the proposed model is 13 M,which is slightly higher than that(11.37 M)of the COVIDNet-small network.But,the corresponding accuracy is improved from 85.2%to 93.0%.The above results illustrate the proposed model can gain an efficient balance between accuracy and weight parameters.展开更多
Brain magnetic resonance images(MRI)are used to diagnose the different diseases of the brain,such as swelling and tumor detection.The quality of the brain MR images is degraded by different noises,usually salt&pep...Brain magnetic resonance images(MRI)are used to diagnose the different diseases of the brain,such as swelling and tumor detection.The quality of the brain MR images is degraded by different noises,usually salt&pepper and Gaussian noises,which are added to the MR images during the acquisition process.In the presence of these noises,medical experts are facing problems in diagnosing diseases from noisy brain MR images.Therefore,we have proposed a de-noising method by mixing concatenation,and residual deep learning techniques called the MCR de-noising method.Our proposed MCR method is to eliminate salt&pepper and gaussian noises as much as possible from the brain MRI images.The MCR method has been trained and tested on the noise quantity levels 2%to 20%for both salt&pepper and gaussian noise.The experiments have been done on publically available brain MRI image datasets,which can easily be accessible in the experiments and result section.The Structure Similarity Index Measure(SSIM)and Peak Signal-to-Noise Ratio(PSNR)calculate the similarity score between the denoised images by the proposed MCR method and the original clean images.Also,the Mean Squared Error(MSE)measures the error or difference between generated denoised and the original images.The proposed MCR denoising method has a 0.9763 SSIM score,84.3182 PSNR,and 0.0004 MSE for salt&pepper noise;similarly,0.7402 SSIM score,72.7601 PSNR,and 0.0041 MSE for Gaussian noise at the highest level of 20%noise.In the end,we have compared the MCR method with the state-of-the-art de-noising filters such as median and wiener de-noising filters.展开更多
Trajectory prediction for heterogeneous traffic agents plays a crucial role in ensuring the safety and efficiency of automated driving in highly interactive traffic environments.Numerous studies in this area have focu...Trajectory prediction for heterogeneous traffic agents plays a crucial role in ensuring the safety and efficiency of automated driving in highly interactive traffic environments.Numerous studies in this area have focused on physicsbased approaches because they can clearly interpret the dynamic evolution of trajectories.However,physics-based methods often suffer from limited accuracy.Recent learning-based methods have demonstrated better performance,but they cannot be fully trusted due to the insufficient incorporation of physical constraints.To mitigate the limitations of purely physics-based and learning-based approaches,this study proposes a kinematics-aware multigraph attention network(KAMGAT)that incorporates physics models into a deep learning framework to improve the learning process of neural networks.Besides,we propose a residual prediction module to further refine the trajectory predictions and address the limitations arising from simplified assumptions in kinematic models.We evaluate our proposed model through experiments on two challenging trajectory datasets,namely,ApolloScape and NGSIM.Our findings from the experiments demonstrate that our model outperforms various kinematics-agnostic models with respect to prediction accuracy and learning efficiency.展开更多
To improve the accuracy and efficiency of time-varying channels estimation algorithms for millimeter Wave(mmWave)massive Multiple-Input Multiple-Output(MIMO)systems in Internet of Vehicles(IoV)scenarios,the paper prop...To improve the accuracy and efficiency of time-varying channels estimation algorithms for millimeter Wave(mmWave)massive Multiple-Input Multiple-Output(MIMO)systems in Internet of Vehicles(IoV)scenarios,the paper proposes a deep learning(DL)algorithm,Squeeze-and-Excitation Attention Residual Network(SEARNet),which integrates Squeeze-and-Excitation Attention(SEAttention)mechanism and residual module.Specifically,SEARNet considers the channel information as an image matrix,and embeds a SEAttention module in residual module to construct the SEAttention-Residual block.Through a data-driven approach,SEARNet can effectively extract key information from the channel matrix using the SEAttention mechanism,thereby reducing noise interference and estimating the channel in an accurate and efficient manner.The simulation results show that compared to two traditional and two DL channel estimation algorithms,the proposed SEARNet can achieve a maximum reduction in normalized mean square error(NMSE)of 97.66%and 84.49%at SNR of-10 dB,78.18%at SNR of 5 dB,and 43.51%at SNR of 10 dB,respectively.展开更多
Terahertz imaging technology has great potential applications in areas,such as remote sensing,navigation,security checks,and so on.However,terahertz images usually have the problems of heavy noises and low resolution....Terahertz imaging technology has great potential applications in areas,such as remote sensing,navigation,security checks,and so on.However,terahertz images usually have the problems of heavy noises and low resolution.Previous terahertz image denoising methods are mainly based on traditional image processing methods,which have limited denoising effects on the terahertz noise.Existing deep learning-based image denoising methods are mostly used in natural images and easily cause a large amount of detail loss when denoising terahertz images.Here,a residual-learning-based multiscale hybridconvolution residual network(MHRNet)is proposed for terahertz image denoising,which can remove noises while preserving detail features in terahertz images.Specifically,a multiscale hybrid-convolution residual block(MHRB)is designed to extract rich detail features and local prediction residual noise from terahertz images.Specifically,MHRB is a residual structure composed of a multiscale dilated convolution block,a bottleneck layer,and a multiscale convolution block.MHRNet uses the MHRB and global residual learning to achieve terahertz image denoising.Ablation studies are performed to validate the effectiveness of MHRB.A series of experiments are conducted on the public terahertz image datasets.The experimental results demonstrate that MHRNet has an excellent denoising effect on synthetic and real noisy terahertz images.Compared with existing methods,MHRNet achieves comprehensive competitive results.展开更多
Fine-grained weather forecasting data,i.e.,the grid data with high-resolution,have attracted increasing attention in recent years,especially for some specific applications such as the Winter Olympic Games.Although Eur...Fine-grained weather forecasting data,i.e.,the grid data with high-resolution,have attracted increasing attention in recent years,especially for some specific applications such as the Winter Olympic Games.Although European Centre for Medium-Range Weather Forecasts(ECMWF)provides grid prediction up to 240 hours,the coarse data are unable to meet high requirements of these major events.In this paper,we propose a method,called model residual machine learning(MRML),to generate grid prediction with high-resolution based on high-precision stations forecasting.MRML applies model output machine learning(MOML)for stations forecasting.Subsequently,MRML utilizes these forecasts to improve the quality of the grid data by fitting a machine learning(ML)model to the residuals.We demonstrate that MRML achieves high capability at diverse meteorological elements,specifically,temperature,relative humidity,and wind speed.In addition,MRML could be easily extended to other post-processing methods by invoking different techniques.In our experiments,MRML outperforms the traditional downscaling methods such as piecewise linear interpolation(PLI)on the testing data.展开更多
A heart attack disrupts the normal flow of blood to the heart muscle,potentially causing severe damage or death if not treated promptly.It can lead to long-term health complications,reduce quality of life,and signific...A heart attack disrupts the normal flow of blood to the heart muscle,potentially causing severe damage or death if not treated promptly.It can lead to long-term health complications,reduce quality of life,and significantly impact daily activities and overall well-being.Despite the growing popularity of deep learning,several drawbacks persist,such as complexity and the limitation of single-model learning.In this paper,we introduce a residual learning-based feature fusion technique to achieve high accuracy in differentiating abnormal cardiac rhythms heart sound.Combining MobileNet with DenseNet201 for feature fusion leverages MobileNet lightweight,efficient architecture with DenseNet201,dense connections,resulting in enhanced feature extraction and improved model performance with reduced computational cost.To further enhance the fusion,we employed residual learning to optimize the hierarchical features of heart abnormal sounds during training.The experimental results demonstrate that the proposed fusion method achieved an accuracy of 95.67%on the benchmark PhysioNet-2016 Spectrogram dataset.To further validate the performance,we applied it to the BreakHis dataset with a magnification level of 100X.The results indicate that the model maintains robust performance on the second dataset,achieving an accuracy of 96.55%.it highlights its consistent performance,making it a suitable for various applications.展开更多
Deep learning-based systems have succeeded in many computer vision tasks.However,it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks.These attacks can qu...Deep learning-based systems have succeeded in many computer vision tasks.However,it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks.These attacks can quickly spoil deep learning models,e.g.,different convolutional neural networks(CNNs),used in various computer vision tasks from image classification to object detection.The adversarial examples are carefully designed by injecting a slight perturbation into the clean images.The proposed CRU-Net defense model is inspired by state-of-the-art defense mechanisms such as MagNet defense,Generative Adversarial Net-work Defense,Deep Regret Analytic Generative Adversarial Networks Defense,Deep Denoising Sparse Autoencoder Defense,and Condtional Generattive Adversarial Network Defense.We have experimentally proved that our approach is better than previous defensive techniques.Our proposed CRU-Net model maps the adversarial image examples into clean images by eliminating the adversarial perturbation.The proposed defensive approach is based on residual and U-Net learning.Many experiments are done on the datasets MNIST and CIFAR10 to prove that our proposed CRU-Net defense model prevents adversarial example attacks in WhiteBox and BlackBox settings and improves the robustness of the deep learning algorithms especially in the computer visionfield.We have also reported similarity(SSIM and PSNR)between the original and restored clean image examples by the proposed CRU-Net defense model.展开更多
Research shows that deep learning algorithms can ffectivelyimprove a single image's super-resolution quality.However,if the algorithmis solely focused on increasing network depth and the desired result is not achi...Research shows that deep learning algorithms can ffectivelyimprove a single image's super-resolution quality.However,if the algorithmis solely focused on increasing network depth and the desired result is not achieved,difficulties in the training process are more likely to arise.Simultaneously,the function space that can be transferred from a iow-resolution image to a high-resolution image is enormous,making finding a satisfactory solution difficult.In this paper,we propose a deep learning method for single image super-resolution.The MDRN network framework uses multi-scale residual blocks and dual learning to fully acquire features in low-resolution images.Finally,these features will be sent to the image reconstruction module torestore high-quality images.The function space is constrained by the closedloop formed by dual learning,which provides additional supervision forthe super-resolution reconstruction of the image.The up-sampling processincludes residual blocks with short-hop connections,so that the networkfocuses on learning high-frequency information,and strives to reconstructimages with richer feature details.The experimental results of ×4 and ×8 super-resolution reconstruction of the image show that the quality of thereconstructed image with this method is better than some existing experimental results of image super-resolution reconstruction in subjective visual ffectsand objective evaluation indicators.展开更多
Model-based reinforcement learning(RL)is anticipated to exhibit higher sample efficiency than model-free RL by utilizing a virtual environment model.However,obtaining sufficiently accurate representations of environme...Model-based reinforcement learning(RL)is anticipated to exhibit higher sample efficiency than model-free RL by utilizing a virtual environment model.However,obtaining sufficiently accurate representations of environmental dynamics is challenging because of uncertainties in complex systems and environments.An inaccurate environment model may degrade the sample efficiency and performance of model-based RL.Furthermore,while model-based RL can improve sample efficiency,it often still requires substantial training time to learn from scratch,potentially limiting its advantages over model-free approaches.To address these challenges,this paper introduces a knowledge-informed model-based residual reinforcement learning framework aimed at enhancing learning efficiency by infusing established expert knowledge into the learning process and avoiding the issue of beginning from zero.Our approach integrates traffic expert knowledge into a virtual environment model,employing the intelligent driver model(IDM)for basic dynamics and neural networks for residual dynamics,thus ensuring adaptability to complex scenarios.We propose a novel strategy that combines traditional control methods with residual RL,facilitating efficient learning and policy optimization without the need to learn from scratch.The proposed approach is applied to connected automated vehicle(CAV)trajectory control tasks for the dissipation of stop-and-go waves in mixed traffic flows.The experimental results demonstrate that our proposed approach enables the CAV agent to achieve superior performance in trajectory control compared with the baseline agents in terms of sample efficiency,traffic flow smoothness and traffic mobility.展开更多
Low-frequency band-shaped swell noise with strong amplitude is common in marine seismic data.The conventional high-pass fi ltering algorithm widely used to suppress swell noise often results in serious damage of effec...Low-frequency band-shaped swell noise with strong amplitude is common in marine seismic data.The conventional high-pass fi ltering algorithm widely used to suppress swell noise often results in serious damage of effective information.This paper introduces the residual learning strategy of denoising convolutional neural network(DnCNN)into a U-shaped convolutional neural network(U-Net)to develop a new U-Net with more generalization,which can eliminate low-frequency swell noise with high precision.The results of both model date tests and real data processing show that the new U-Net is capable of effi cient learning and high-precision noise removal,and can avoid the overfi tting problem which is very common in conventional neural network methods.This new U-Net can also be generalized to some extent and can eff ectively preserve low-frequency eff ective information.Compared with the conventional high-pass fi ltering method commonly used in the industry,the new U-Net can eliminate low-frequency swell noise with higher precision while eff ectively preserving low-frequency eff ective information,which is of great signifi cance for subsequent processing such as amplitude-preserving imaging and full waveform inversion.展开更多
This research article proposes an automatic frame work for detectingCOVID -19 at the early stage using chest X-ray image. It is an undeniable factthat coronovirus is a serious disease but the early detection of the vi...This research article proposes an automatic frame work for detectingCOVID -19 at the early stage using chest X-ray image. It is an undeniable factthat coronovirus is a serious disease but the early detection of the virus presentin human bodies can save lives. In recent times, there are so many research solutions that have been presented for early detection, but there is still a lack in needof right and even rich technology for its early detection. The proposed deeplearning model analysis the pixels of every image and adjudges the presence ofvirus. The classifier is designed in such a way so that, it automatically detectsthe virus present in lungs using chest image. This approach uses an imagetexture analysis technique called granulometric mathematical model. Selectedfeatures are heuristically processed for optimization using novel multi scaling deep learning called light weight residual–atrous spatial pyramid pooling(LightRES-ASPP-Unet) Unet model. The proposed deep LightRES-ASPPUnet technique has a higher level of contracting solution by extracting majorlevel of image features. Moreover, the corona virus has been detected usinghigh resolution output. In the framework, atrous spatial pyramid pooling(ASPP) method is employed at its bottom level for incorporating the deepmulti scale features in to the discriminative mode. The architectural workingstarts from the selecting the features from the image using granulometricmathematical model and the selected features are optimized using LightRESASPP-Unet. ASPP in the analysis of images has performed better than theexisting Unet model. The proposed algorithm has achieved 99.6% of accuracyin detecting the virus at its early stage.展开更多
Rutting of asphalt pavements is a crucial design criterion in various pavement design guides. A good road transportation base can provide security for the transportation of oil and gas in road transportation. This stu...Rutting of asphalt pavements is a crucial design criterion in various pavement design guides. A good road transportation base can provide security for the transportation of oil and gas in road transportation. This study attempts to develop a robust artificial intelligence model to estimate different asphalt pavements’ rutting depth clips, temperature, and load axes as primary characteristics. The experiment data were obtained from19 asphalt pavements with different crude oil sources on a 2.038km long full-scale field accelerated pavement test track(Road Track Institute, RIOHTrack) in Tongzhou, Beijing. In addition,this paper also proposes to build complex networks with different pavement rutting depths through complex network methods and the Louvain algorithm for community detection. The most critical structural elements can be selected from different asphalt pavement rutting data, and similar structural elements can be found. An extreme learning machine algorithm with residual correction(RELM) is designed and optimized using an independent adaptive particle swarm algorithm. The experimental results of the proposed method are compared with several classical machine learning algorithms, with predictions of average root mean squared error(MSE), average mean absolute error(MAE), and a verage mean absolute percentage error(MAPE) for 19 asphalt pavements reaching 1.742, 1.363, and 1.94% respectively. The experiments demonstrate that the RELM algorithm has an advantage over classical machine learning methods in dealing with non-linear problems in road engineering. Notably, the method ensures the adaptation of the simulated environment to different levels of abstraction through the cognitive analysis of the production environment parameters. It is a promising alternative method that facilitates the rapid assessment of pavement conditions and could be applied in the future to production processes in the oil and gas industry.展开更多
This paper develops a physics-guided graph network to enhance the robustness of distribution system state estimation(DSSE)against anomalous real-time measurements,as well as a deep auto-encoder(DAE)-based detector and...This paper develops a physics-guided graph network to enhance the robustness of distribution system state estimation(DSSE)against anomalous real-time measurements,as well as a deep auto-encoder(DAE)-based detector and a Gaussian process-aided residual learning(GARL)to deal with challenges arising from topology changes.A global-scanning jumping knowledge network(GSJKN)is first designed to establish the regression rule between the measurement data and state variables.The structural information of distribution system(DS)and a global-scanning module are incorporated to guide the propagation of scarce measurements in the graph topology,contributing to valid estimation precision in sparsely measured DSs.To monitor the topology changes of the network,a DAE network is employed to learn an efficient representation of the measurements of the system under a certain topology,which can achieve online monitoring of the network structure by observing the variation tendency of the reconstruction error.When the topology change occurs,a Gaussian process with a composite kernel is applied to the modeling of the pre-trained GSJKN residual to adapt to the new topology.The embedding of the physical structural knowledge enables the proposed GSJKN method to restore the missing/noisy values utilizing the adjacent measurements,which enhances the robustness to typical data acquisition errors.The adopted DAE network and special GARL-based transfer method further allow the DSSE method to rapidly detect and adapt to the topology change,as well as achieve effective quantification of the estimation uncertainties.Comparative tests on balanced and unbalanced systems demonstrate the accuracy,robustness,and adaptability of the proposed DSSE method.展开更多
Aiming to solve the poor performance of low illumination enhancement algorithms on uneven illumination images,a low-light image enhancement(LIME)algorithm based on a residual network was proposed.The algorithm constru...Aiming to solve the poor performance of low illumination enhancement algorithms on uneven illumination images,a low-light image enhancement(LIME)algorithm based on a residual network was proposed.The algorithm constructs a deep network that uses residual modules to extract image feature information and semantic modules to extract image semantic information from different levels.Moreover,a composite loss function was also designed for the process of low illumination image enhancement,which dynamically evaluated the loss of an enhanced image from three factors of color,structure,and gradient.It ensures that the model can correctly enhance the image features according to the image semantics,so that the enhancement results are more in line with the human visual experience.Experimental results show that compared with the state-of-the-art algorithms,the semantic-driven residual low-light network(SRLLN)can effectively improve the quality of low illumination images,and achieve better subjective and objective evaluation indexes on different types of images.展开更多
An end-to-end channel attention and pixel attention network(CP-Net)is proposed to produce dehazed image directly in the paper.The CP-Net structure contains three critical components.Firstly,the double attention(DA)mod...An end-to-end channel attention and pixel attention network(CP-Net)is proposed to produce dehazed image directly in the paper.The CP-Net structure contains three critical components.Firstly,the double attention(DA)module consisting of channel attention(CA)and pixel attention(PA).Different channel features contain different levels of important information,and CA can give more weight to relevant information,so the network can learn more useful information.Meanwhile,haze is unevenly distributed on different pixels,and PA is able to filter out haze with varying weights for different pixels.It sums the outputs of the two attention modules to improve further feature representation which contributes to better dehazing result.Secondly,local residual learning and DA module constitute another important component,namely basic block structure.Local residual learning can transfer the feature information in the shallow part of the network to the deep part of the network through multiple local residual connections and enhance the expressive ability of CP-Net.Thirdly,CP-Net mainly uses its core component,DA module,to automatically assign different weights to different features to achieve satisfactory dehazing effect.The experiment results on synthetic datasets and real hazy images indicate that many state-of-the-art single image dehazing methods have been surpassed by the CP-Net both quantitatively and qualitatively.展开更多
Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities.However,conventional deep networks assume that the training and test ...Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities.However,conventional deep networks assume that the training and test data are sampled from the same distribution,and this assumption is often violated in real-world scenarios.To address the domain shift or data bias problems,we introduce layer-wise domain correction(LDC),a new unsupervised domain adaptation algorithm which adapts an existing deep network through additive correction layers spaced throughout the network.Through the additive layers,the representations of source and target domains can be perfectly aligned.The corrections that are trained via maximum mean discrepancy,adapt to the target domain while increasing the representational capacity of the network.LDC requires no target labels,achieves state-of-the-art performance across several adaptation benchmarks,and requires significantly less training time than existing adaptation methods.展开更多
基金supported by the Xiamen Science and Technology Subsidy Project(No.2023CXY0318).
文摘Abnormal network traffic, as a frequent security risk, requires a series of techniques to categorize and detect it. Existing network traffic anomaly detection still faces challenges: the inability to fully extract local and global features, as well as the lack of effective mechanisms to capture complex interactions between features;Additionally, when increasing the receptive field to obtain deeper feature representations, the reliance on increasing network depth leads to a significant increase in computational resource consumption, affecting the efficiency and performance of detection. Based on these issues, firstly, this paper proposes a network traffic anomaly detection model based on parallel dilated convolution and residual learning (Res-PDC). To better explore the interactive relationships between features, the traffic samples are converted into two-dimensional matrix. A module combining parallel dilated convolutions and residual learning (res-pdc) was designed to extract local and global features of traffic at different scales. By utilizing res-pdc modules with different dilation rates, we can effectively capture spatial features at different scales and explore feature dependencies spanning wider regions without increasing computational resources. Secondly, to focus and integrate the information in different feature subspaces, further enhance and extract the interactions among the features, multi-head attention is added to Res-PDC, resulting in the final model: multi-head attention enhanced parallel dilated convolution and residual learning (MHA-Res-PDC) for network traffic anomaly detection. Finally, comparisons with other machine learning and deep learning algorithms are conducted on the NSL-KDD and CIC-IDS-2018 datasets. The experimental results demonstrate that the proposed method in this paper can effectively improve the detection performance.
基金supported in part by the National Natural Science Foundation of China under Grant 61601235,in part by the Natural Science Foundation of Jiangsu Province of China under Grant BK20160972.
文摘Due to the huge difference of noise distribution,the result of a mixture of multiple noises becomes very complicated.Under normal circumstances,the most common type of mixed noise is to add impulse noise(IN)and then white Gaussian noise(AWGN).From the reduction of cascaded IN and AWGN to the latest sparse representation,a great deal of methods has been proposed to reduce this form of mixed noise.However,when the mixed noise is very strong,most methods often produce a lot of artifacts.In order to solve the above problems,we propose a method based on residual learning for the removal of AWGN-IN noise in this paper.By training,our model can obtain stable nonlinear mapping from the images with mixed noise to the clean images.After a series of experiments under different noise settings,the results show that our method is obviously better than the traditional sparse representation and patch based method.Meanwhile,the time of model training and image denoising is greatly reduced.
基金This work was supported in part by the science and technology research project of Henan Provincial Department of science and technology(No.222102110366)the Science and Technology Innovation Team of Henan University(No.22IRTSTHN016)the grants from the teaching reform research and practice project of higher education in Henan Province in 2021[2021SJGLX502].
文摘Amodel that can obtain rapid and accurate detection of coronavirus disease 2019(COVID-19)plays a significant role in treating and preventing the spread of disease transmission.However,designing such amodel that can balance the detection accuracy andweight parameters ofmemorywell to deploy a mobile device is challenging.Taking this point into account,this paper fuses the convolutional neural network and residual learning operations to build a multi-class classification model,which improves COVID-19 pneumonia detection performance and keeps a trade-off between the weight parameters and accuracy.The convolutional neural network can extract the COVID-19 feature information by repeated convolutional operations.The residual learning operations alleviate the gradient problems caused by stacking convolutional layers and enhance the ability of feature extraction.The ability further enables the proposed model to acquire effective feature information at a lowcost,which canmake ourmodel keep smallweight parameters.Extensive validation and comparison with other models of COVID-19 pneumonia detection on the well-known COVIDx dataset show that(1)the sensitivity of COVID-19 pneumonia detection is improved from 88.2%(non-COVID-19)and 77.5%(COVID-19)to 95.3%(non-COVID-19)and 96.5%(COVID-19),respectively.The positive predictive value is also respectively increased from72.8%(non-COVID-19)and 89.0%(COVID-19)to 88.8%(non-COVID-19)and 95.1%(COVID-19).(2)Compared with the weight parameters of the COVIDNet-small network,the value of the proposed model is 13 M,which is slightly higher than that(11.37 M)of the COVIDNet-small network.But,the corresponding accuracy is improved from 85.2%to 93.0%.The above results illustrate the proposed model can gain an efficient balance between accuracy and weight parameters.
文摘Brain magnetic resonance images(MRI)are used to diagnose the different diseases of the brain,such as swelling and tumor detection.The quality of the brain MR images is degraded by different noises,usually salt&pepper and Gaussian noises,which are added to the MR images during the acquisition process.In the presence of these noises,medical experts are facing problems in diagnosing diseases from noisy brain MR images.Therefore,we have proposed a de-noising method by mixing concatenation,and residual deep learning techniques called the MCR de-noising method.Our proposed MCR method is to eliminate salt&pepper and gaussian noises as much as possible from the brain MRI images.The MCR method has been trained and tested on the noise quantity levels 2%to 20%for both salt&pepper and gaussian noise.The experiments have been done on publically available brain MRI image datasets,which can easily be accessible in the experiments and result section.The Structure Similarity Index Measure(SSIM)and Peak Signal-to-Noise Ratio(PSNR)calculate the similarity score between the denoised images by the proposed MCR method and the original clean images.Also,the Mean Squared Error(MSE)measures the error or difference between generated denoised and the original images.The proposed MCR denoising method has a 0.9763 SSIM score,84.3182 PSNR,and 0.0004 MSE for salt&pepper noise;similarly,0.7402 SSIM score,72.7601 PSNR,and 0.0041 MSE for Gaussian noise at the highest level of 20%noise.In the end,we have compared the MCR method with the state-of-the-art de-noising filters such as median and wiener de-noising filters.
基金the University of Wisconsin-Madison’s Center for Connected and Automated Transportation(CCAT),a part of the larger CCAT consortium,a USDOT Region 5 University Transportation Center funded by the U.S.Department of Transportation,Award#69A3552348305.
文摘Trajectory prediction for heterogeneous traffic agents plays a crucial role in ensuring the safety and efficiency of automated driving in highly interactive traffic environments.Numerous studies in this area have focused on physicsbased approaches because they can clearly interpret the dynamic evolution of trajectories.However,physics-based methods often suffer from limited accuracy.Recent learning-based methods have demonstrated better performance,but they cannot be fully trusted due to the insufficient incorporation of physical constraints.To mitigate the limitations of purely physics-based and learning-based approaches,this study proposes a kinematics-aware multigraph attention network(KAMGAT)that incorporates physics models into a deep learning framework to improve the learning process of neural networks.Besides,we propose a residual prediction module to further refine the trajectory predictions and address the limitations arising from simplified assumptions in kinematic models.We evaluate our proposed model through experiments on two challenging trajectory datasets,namely,ApolloScape and NGSIM.Our findings from the experiments demonstrate that our model outperforms various kinematics-agnostic models with respect to prediction accuracy and learning efficiency.
基金supported in part by the National Natural Science Foundation of China under Grants U2001213 and 62261024in part by National Key Research and Development Project under Grant 2020YFB1807204in part by Key Laboratory of Universal Wireless Communications(BUPT),Ministry of Education under Grant KFKT2022101.
文摘To improve the accuracy and efficiency of time-varying channels estimation algorithms for millimeter Wave(mmWave)massive Multiple-Input Multiple-Output(MIMO)systems in Internet of Vehicles(IoV)scenarios,the paper proposes a deep learning(DL)algorithm,Squeeze-and-Excitation Attention Residual Network(SEARNet),which integrates Squeeze-and-Excitation Attention(SEAttention)mechanism and residual module.Specifically,SEARNet considers the channel information as an image matrix,and embeds a SEAttention module in residual module to construct the SEAttention-Residual block.Through a data-driven approach,SEARNet can effectively extract key information from the channel matrix using the SEAttention mechanism,thereby reducing noise interference and estimating the channel in an accurate and efficient manner.The simulation results show that compared to two traditional and two DL channel estimation algorithms,the proposed SEARNet can achieve a maximum reduction in normalized mean square error(NMSE)of 97.66%and 84.49%at SNR of-10 dB,78.18%at SNR of 5 dB,and 43.51%at SNR of 10 dB,respectively.
基金National Natural Science Foundation of China,Grant/Award Number:62173098,62104047Guangdong Provincial Key Laboratory of Cyber-Physical System,Grant/Award Number:2020B1212060069。
文摘Terahertz imaging technology has great potential applications in areas,such as remote sensing,navigation,security checks,and so on.However,terahertz images usually have the problems of heavy noises and low resolution.Previous terahertz image denoising methods are mainly based on traditional image processing methods,which have limited denoising effects on the terahertz noise.Existing deep learning-based image denoising methods are mostly used in natural images and easily cause a large amount of detail loss when denoising terahertz images.Here,a residual-learning-based multiscale hybridconvolution residual network(MHRNet)is proposed for terahertz image denoising,which can remove noises while preserving detail features in terahertz images.Specifically,a multiscale hybrid-convolution residual block(MHRB)is designed to extract rich detail features and local prediction residual noise from terahertz images.Specifically,MHRB is a residual structure composed of a multiscale dilated convolution block,a bottleneck layer,and a multiscale convolution block.MHRNet uses the MHRB and global residual learning to achieve terahertz image denoising.Ablation studies are performed to validate the effectiveness of MHRB.A series of experiments are conducted on the public terahertz image datasets.The experimental results demonstrate that MHRNet has an excellent denoising effect on synthetic and real noisy terahertz images.Compared with existing methods,MHRNet achieves comprehensive competitive results.
基金Project supported by the National Natural Science Foundation of China(Nos.12101072 and 11421101)the National Key Research and Development Program of China(No.2018YFF0300104)+1 种基金the Beijing Municipal Science and Technology Project(No.Z201100005820002)the Open Research Fund of Shenzhen Research Institute of Big Data(No.2019ORF01001)。
文摘Fine-grained weather forecasting data,i.e.,the grid data with high-resolution,have attracted increasing attention in recent years,especially for some specific applications such as the Winter Olympic Games.Although European Centre for Medium-Range Weather Forecasts(ECMWF)provides grid prediction up to 240 hours,the coarse data are unable to meet high requirements of these major events.In this paper,we propose a method,called model residual machine learning(MRML),to generate grid prediction with high-resolution based on high-precision stations forecasting.MRML applies model output machine learning(MOML)for stations forecasting.Subsequently,MRML utilizes these forecasts to improve the quality of the grid data by fitting a machine learning(ML)model to the residuals.We demonstrate that MRML achieves high capability at diverse meteorological elements,specifically,temperature,relative humidity,and wind speed.In addition,MRML could be easily extended to other post-processing methods by invoking different techniques.In our experiments,MRML outperforms the traditional downscaling methods such as piecewise linear interpolation(PLI)on the testing data.
文摘A heart attack disrupts the normal flow of blood to the heart muscle,potentially causing severe damage or death if not treated promptly.It can lead to long-term health complications,reduce quality of life,and significantly impact daily activities and overall well-being.Despite the growing popularity of deep learning,several drawbacks persist,such as complexity and the limitation of single-model learning.In this paper,we introduce a residual learning-based feature fusion technique to achieve high accuracy in differentiating abnormal cardiac rhythms heart sound.Combining MobileNet with DenseNet201 for feature fusion leverages MobileNet lightweight,efficient architecture with DenseNet201,dense connections,resulting in enhanced feature extraction and improved model performance with reduced computational cost.To further enhance the fusion,we employed residual learning to optimize the hierarchical features of heart abnormal sounds during training.The experimental results demonstrate that the proposed fusion method achieved an accuracy of 95.67%on the benchmark PhysioNet-2016 Spectrogram dataset.To further validate the performance,we applied it to the BreakHis dataset with a magnification level of 100X.The results indicate that the model maintains robust performance on the second dataset,achieving an accuracy of 96.55%.it highlights its consistent performance,making it a suitable for various applications.
文摘Deep learning-based systems have succeeded in many computer vision tasks.However,it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks.These attacks can quickly spoil deep learning models,e.g.,different convolutional neural networks(CNNs),used in various computer vision tasks from image classification to object detection.The adversarial examples are carefully designed by injecting a slight perturbation into the clean images.The proposed CRU-Net defense model is inspired by state-of-the-art defense mechanisms such as MagNet defense,Generative Adversarial Net-work Defense,Deep Regret Analytic Generative Adversarial Networks Defense,Deep Denoising Sparse Autoencoder Defense,and Condtional Generattive Adversarial Network Defense.We have experimentally proved that our approach is better than previous defensive techniques.Our proposed CRU-Net model maps the adversarial image examples into clean images by eliminating the adversarial perturbation.The proposed defensive approach is based on residual and U-Net learning.Many experiments are done on the datasets MNIST and CIFAR10 to prove that our proposed CRU-Net defense model prevents adversarial example attacks in WhiteBox and BlackBox settings and improves the robustness of the deep learning algorithms especially in the computer visionfield.We have also reported similarity(SSIM and PSNR)between the original and restored clean image examples by the proposed CRU-Net defense model.
基金funded by National Key R&D Program of China(2021YFC3320302)Network threat depth analysis software(KY10800210013).
文摘Research shows that deep learning algorithms can ffectivelyimprove a single image's super-resolution quality.However,if the algorithmis solely focused on increasing network depth and the desired result is not achieved,difficulties in the training process are more likely to arise.Simultaneously,the function space that can be transferred from a iow-resolution image to a high-resolution image is enormous,making finding a satisfactory solution difficult.In this paper,we propose a deep learning method for single image super-resolution.The MDRN network framework uses multi-scale residual blocks and dual learning to fully acquire features in low-resolution images.Finally,these features will be sent to the image reconstruction module torestore high-quality images.The function space is constrained by the closedloop formed by dual learning,which provides additional supervision forthe super-resolution reconstruction of the image.The up-sampling processincludes residual blocks with short-hop connections,so that the networkfocuses on learning high-frequency information,and strives to reconstructimages with richer feature details.The experimental results of ×4 and ×8 super-resolution reconstruction of the image show that the quality of thereconstructed image with this method is better than some existing experimental results of image super-resolution reconstruction in subjective visual ffectsand objective evaluation indicators.
基金University of Wisconsin-Madison's Center for Connected and Automated Transportation(CCAT),a part of the larger CCAT consortium,a USDOT Region 5 University Transportation Center funded by the U.S.Department of Transportation,Award#69A3552348305The contents of this paper reflect the views of the authors,who are responsible for the facts and the accuracy of the data presented herein,and do not necessarily reflect the official views or policies of the sponsoring organization.
文摘Model-based reinforcement learning(RL)is anticipated to exhibit higher sample efficiency than model-free RL by utilizing a virtual environment model.However,obtaining sufficiently accurate representations of environmental dynamics is challenging because of uncertainties in complex systems and environments.An inaccurate environment model may degrade the sample efficiency and performance of model-based RL.Furthermore,while model-based RL can improve sample efficiency,it often still requires substantial training time to learn from scratch,potentially limiting its advantages over model-free approaches.To address these challenges,this paper introduces a knowledge-informed model-based residual reinforcement learning framework aimed at enhancing learning efficiency by infusing established expert knowledge into the learning process and avoiding the issue of beginning from zero.Our approach integrates traffic expert knowledge into a virtual environment model,employing the intelligent driver model(IDM)for basic dynamics and neural networks for residual dynamics,thus ensuring adaptability to complex scenarios.We propose a novel strategy that combines traditional control methods with residual RL,facilitating efficient learning and policy optimization without the need to learn from scratch.The proposed approach is applied to connected automated vehicle(CAV)trajectory control tasks for the dissipation of stop-and-go waves in mixed traffic flows.The experimental results demonstrate that our proposed approach enables the CAV agent to achieve superior performance in trajectory control compared with the baseline agents in terms of sample efficiency,traffic flow smoothness and traffic mobility.
基金the Key R&D project of Shandong Province(No.2019JZZY010803)the Central Universities(No.201964016),the National Natural Science Foundation of China(No.41704114)+2 种基金the National Science and Technology Major Project of China(No.2016ZX05027-002)Taishan Scholar Project Funding(No.tspd20161007)the China Scholarship Council(No.201906335010).
文摘Low-frequency band-shaped swell noise with strong amplitude is common in marine seismic data.The conventional high-pass fi ltering algorithm widely used to suppress swell noise often results in serious damage of effective information.This paper introduces the residual learning strategy of denoising convolutional neural network(DnCNN)into a U-shaped convolutional neural network(U-Net)to develop a new U-Net with more generalization,which can eliminate low-frequency swell noise with high precision.The results of both model date tests and real data processing show that the new U-Net is capable of effi cient learning and high-precision noise removal,and can avoid the overfi tting problem which is very common in conventional neural network methods.This new U-Net can also be generalized to some extent and can eff ectively preserve low-frequency eff ective information.Compared with the conventional high-pass fi ltering method commonly used in the industry,the new U-Net can eliminate low-frequency swell noise with higher precision while eff ectively preserving low-frequency eff ective information,which is of great signifi cance for subsequent processing such as amplitude-preserving imaging and full waveform inversion.
文摘This research article proposes an automatic frame work for detectingCOVID -19 at the early stage using chest X-ray image. It is an undeniable factthat coronovirus is a serious disease but the early detection of the virus presentin human bodies can save lives. In recent times, there are so many research solutions that have been presented for early detection, but there is still a lack in needof right and even rich technology for its early detection. The proposed deeplearning model analysis the pixels of every image and adjudges the presence ofvirus. The classifier is designed in such a way so that, it automatically detectsthe virus present in lungs using chest image. This approach uses an imagetexture analysis technique called granulometric mathematical model. Selectedfeatures are heuristically processed for optimization using novel multi scaling deep learning called light weight residual–atrous spatial pyramid pooling(LightRES-ASPP-Unet) Unet model. The proposed deep LightRES-ASPPUnet technique has a higher level of contracting solution by extracting majorlevel of image features. Moreover, the corona virus has been detected usinghigh resolution output. In the framework, atrous spatial pyramid pooling(ASPP) method is employed at its bottom level for incorporating the deepmulti scale features in to the discriminative mode. The architectural workingstarts from the selecting the features from the image using granulometricmathematical model and the selected features are optimized using LightRESASPP-Unet. ASPP in the analysis of images has performed better than theexisting Unet model. The proposed algorithm has achieved 99.6% of accuracyin detecting the virus at its early stage.
基金supported by the Analytical Center for the Government of the Russian Federation (IGK 000000D730321P5Q0002) and Agreement Nos.(70-2021-00141)。
文摘Rutting of asphalt pavements is a crucial design criterion in various pavement design guides. A good road transportation base can provide security for the transportation of oil and gas in road transportation. This study attempts to develop a robust artificial intelligence model to estimate different asphalt pavements’ rutting depth clips, temperature, and load axes as primary characteristics. The experiment data were obtained from19 asphalt pavements with different crude oil sources on a 2.038km long full-scale field accelerated pavement test track(Road Track Institute, RIOHTrack) in Tongzhou, Beijing. In addition,this paper also proposes to build complex networks with different pavement rutting depths through complex network methods and the Louvain algorithm for community detection. The most critical structural elements can be selected from different asphalt pavement rutting data, and similar structural elements can be found. An extreme learning machine algorithm with residual correction(RELM) is designed and optimized using an independent adaptive particle swarm algorithm. The experimental results of the proposed method are compared with several classical machine learning algorithms, with predictions of average root mean squared error(MSE), average mean absolute error(MAE), and a verage mean absolute percentage error(MAPE) for 19 asphalt pavements reaching 1.742, 1.363, and 1.94% respectively. The experiments demonstrate that the RELM algorithm has an advantage over classical machine learning methods in dealing with non-linear problems in road engineering. Notably, the method ensures the adaptation of the simulated environment to different levels of abstraction through the cognitive analysis of the production environment parameters. It is a promising alternative method that facilitates the rapid assessment of pavement conditions and could be applied in the future to production processes in the oil and gas industry.
基金supported in part by Fundamental Research Funds for the Central Universities(No.ZYGX2024J014)in part by the National Natural Science Foundation of China(No.52277083).
文摘This paper develops a physics-guided graph network to enhance the robustness of distribution system state estimation(DSSE)against anomalous real-time measurements,as well as a deep auto-encoder(DAE)-based detector and a Gaussian process-aided residual learning(GARL)to deal with challenges arising from topology changes.A global-scanning jumping knowledge network(GSJKN)is first designed to establish the regression rule between the measurement data and state variables.The structural information of distribution system(DS)and a global-scanning module are incorporated to guide the propagation of scarce measurements in the graph topology,contributing to valid estimation precision in sparsely measured DSs.To monitor the topology changes of the network,a DAE network is employed to learn an efficient representation of the measurements of the system under a certain topology,which can achieve online monitoring of the network structure by observing the variation tendency of the reconstruction error.When the topology change occurs,a Gaussian process with a composite kernel is applied to the modeling of the pre-trained GSJKN residual to adapt to the new topology.The embedding of the physical structural knowledge enables the proposed GSJKN method to restore the missing/noisy values utilizing the adjacent measurements,which enhances the robustness to typical data acquisition errors.The adopted DAE network and special GARL-based transfer method further allow the DSSE method to rapidly detect and adapt to the topology change,as well as achieve effective quantification of the estimation uncertainties.Comparative tests on balanced and unbalanced systems demonstrate the accuracy,robustness,and adaptability of the proposed DSSE method.
基金supported by the Promotion Plan of Jiangsu Association for Science and Technology(TJ215039)the Research Foundation of Nanjing University of Posts and Telecommunications(NY219076)。
文摘Aiming to solve the poor performance of low illumination enhancement algorithms on uneven illumination images,a low-light image enhancement(LIME)algorithm based on a residual network was proposed.The algorithm constructs a deep network that uses residual modules to extract image feature information and semantic modules to extract image semantic information from different levels.Moreover,a composite loss function was also designed for the process of low illumination image enhancement,which dynamically evaluated the loss of an enhanced image from three factors of color,structure,and gradient.It ensures that the model can correctly enhance the image features according to the image semantics,so that the enhancement results are more in line with the human visual experience.Experimental results show that compared with the state-of-the-art algorithms,the semantic-driven residual low-light network(SRLLN)can effectively improve the quality of low illumination images,and achieve better subjective and objective evaluation indexes on different types of images.
文摘An end-to-end channel attention and pixel attention network(CP-Net)is proposed to produce dehazed image directly in the paper.The CP-Net structure contains three critical components.Firstly,the double attention(DA)module consisting of channel attention(CA)and pixel attention(PA).Different channel features contain different levels of important information,and CA can give more weight to relevant information,so the network can learn more useful information.Meanwhile,haze is unevenly distributed on different pixels,and PA is able to filter out haze with varying weights for different pixels.It sums the outputs of the two attention modules to improve further feature representation which contributes to better dehazing result.Secondly,local residual learning and DA module constitute another important component,namely basic block structure.Local residual learning can transfer the feature information in the shallow part of the network to the deep part of the network through multiple local residual connections and enhance the expressive ability of CP-Net.Thirdly,CP-Net mainly uses its core component,DA module,to automatically assign different weights to different features to achieve satisfactory dehazing effect.The experiment results on synthetic datasets and real hazy images indicate that many state-of-the-art single image dehazing methods have been surpassed by the CP-Net both quantitatively and qualitatively.
基金supported by the National Key R&D Program of China(No.2016YFB1200203)the National Natural Science Foundation of China(Nos.41427806 and 61273233)
文摘Deep neural networks have been successfully applied to numerous machine learning tasks because of their impressive feature abstraction capabilities.However,conventional deep networks assume that the training and test data are sampled from the same distribution,and this assumption is often violated in real-world scenarios.To address the domain shift or data bias problems,we introduce layer-wise domain correction(LDC),a new unsupervised domain adaptation algorithm which adapts an existing deep network through additive correction layers spaced throughout the network.Through the additive layers,the representations of source and target domains can be perfectly aligned.The corrections that are trained via maximum mean discrepancy,adapt to the target domain while increasing the representational capacity of the network.LDC requires no target labels,achieves state-of-the-art performance across several adaptation benchmarks,and requires significantly less training time than existing adaptation methods.
