Geochemical survey data are essential across Earth Science disciplines but are often affected by noise,which can obscure important geological signals and compromise subsequent prediction and interpretation.Quantifying...Geochemical survey data are essential across Earth Science disciplines but are often affected by noise,which can obscure important geological signals and compromise subsequent prediction and interpretation.Quantifying prediction uncertainty is hence crucial for robust geoscientific decision-making.This study proposes a novel deep learning framework,the Spatially Constrained Variational Autoencoder(SC-VAE),for denoising geochemical survey data with integrated uncertainty quantification.The SC-VAE incorporates spatial regularization,which enforces spatial coherence by modeling inter-sample relationships directly within the latent space.The performance of the SC-VAE was systematically evaluated against a standard Variational Autoencoder(VAE)using geochemical data from the gold polymetallic district in the northwestern part of Sichuan Province,China.Both models were optimized using Bayesian optimization,with objective functions specifically designed to maintain essential geostatistical characteristics.Evaluation metrics include variogram analysis,quantitative measures of spatial interpolation accuracy,visual assessment of denoised maps,and statistical analysis of data distributions,as well as decomposition of uncertainties.Results show that the SC-VAE achieves superior noise suppression and better preservation of spatial structure compared to the standard VAE,as demonstrated by a significant reduction in the variogram nugget effect and an increased partial sill.The SC-VAE produces denoised maps with clearer anomaly delineation and more regularized data distributions,effectively mitigating outliers and reducing kurtosis.Additionally,it delivers improved interpolation accuracy and spatially explicit uncertainty estimates,facilitating more reliable and interpretable assessments of prediction confidence.The SC-VAE framework thus provides a robust,geostatistically informed solution for enhancing the quality and interpretability of geochemical data,with broad applicability in mineral exploration,environmental geochemistry,and other Earth Science domains.展开更多
The aerial deployment method enables Unmanned Aerial Vehicles(UAVs)to be directly positioned at the required altitude for their mission.This method typically employs folding technology to improve loading efficiency,wi...The aerial deployment method enables Unmanned Aerial Vehicles(UAVs)to be directly positioned at the required altitude for their mission.This method typically employs folding technology to improve loading efficiency,with applications such as the gravity-only aerial deployment of high-aspect-ratio solar-powered UAVs,and aerial takeoff of fixed-wing drones in Mars research.However,the significant morphological changes during deployment are accompanied by strong nonlinear dynamic aerodynamic forces,which result in multiple degrees of freedom and an unstable character.This hinders the description and analysis of unknown dynamic behaviors,further leading to difficulties in the design of deployment strategies and flight control.To address this issue,this paper proposes an analysis method for dynamic behaviors during aerial deployment based on the Variational Autoencoder(VAE).Focusing on the gravity-only deployment problem of highaspect-ratio foldable-wing UAVs,the method encodes the multi-degree-of-freedom unstable motion signals into a low-dimensional feature space through a data-driven approach.By clustering in the feature space,this paper identifies and studies several dynamic behaviors during aerial deployment.The research presented in this paper offers a new method and perspective for feature extraction and analysis of complex and difficult-to-describe extreme flight dynamics,guiding the research on aerial deployment drones design and control strategies.展开更多
In the face of the unrelenting challenge posed by earthquakes-a natural hazard of unpredictable nature with a legacy of significant loss of life,destruction of infrastructure,and profound economic and social impacts-t...In the face of the unrelenting challenge posed by earthquakes-a natural hazard of unpredictable nature with a legacy of significant loss of life,destruction of infrastructure,and profound economic and social impacts-the scientific community has pursued advancements in earthquake early warning systems(EEWSs).These systems are vital for pre-emptive actions and decision-making that can save lives and safeguard critical infrastructure.This study proposes and validates a domain-informed deep learning-based EEWS called the hybrid earthquake early warning framework for estimating response spectra(HEWFERS),which represents a significant leap forward in the capabilities to predict ground shaking intensity in real-time,aligning with the United Nations’disaster risk reduction goals.HEWFERS ingeniously integrates a domain-informed variational autoencoder for physics-based latent variable(LV)extraction,a feed-forward neural network for on-site prediction,and Gaussian process regression for spatial prediction.Adopting explainable artificial intelligence-based Shapley explanations further elucidates the predictive mechanisms,ensuring stakeholder-informed decisions.By conducting an extensive analysis of the proposed framework under a large database of approximately 14000 recorded ground motions,this study offers insights into the potential of integrating machine learning with seismology to revolutionize earthquake preparedness and response,thus paving the way for a safer and more resilient future.展开更多
To overcome the challenges of limited experimental data and improve the accuracy of empirical formulas,we propose a low-cycle fatigue(LCF)life prediction model for nickel-based superalloys using a data augmentation me...To overcome the challenges of limited experimental data and improve the accuracy of empirical formulas,we propose a low-cycle fatigue(LCF)life prediction model for nickel-based superalloys using a data augmentation method.This method utilizes a variational autoencoder(VAE)to generate low-cycle fatigue data and form an augmented dataset.The Pearson correlation coefficient(PCC)is employed to verify the similarity of feature distributions between the original and augmented datasets.Six machine learning models,namely random forest(RF),artificial neural network(ANN),support vector machine(SVM),gradient-boosted decision tree(GBDT),eXtreme Gradient Boosting(XGBoost),and Categorical Boosting(CatBoost),are utilized to predict the LCF life of nickel-based superalloys.Results indicate that the proposed data augmentation method based on VAE can effectively expand the dataset,and the mean absolute error(MAE),root mean square error(RMSE),and R-squared(R^(2))values achieved using the CatBoost model,with respective values of 0.0242,0.0391,and 0.9538,are superior to those of the other models.The proposed method reduces the cost and time associated with LCF experiments and accurately establishes the relationship between fatigue characteristics and LCF life of nickel-based superalloys.展开更多
Direct coal liquefaction products offer a considerable quantity of cycloalkanes, which are the valuable candidates for making the high energy density fuels. The creation of such fuels depends on designing molecular st...Direct coal liquefaction products offer a considerable quantity of cycloalkanes, which are the valuable candidates for making the high energy density fuels. The creation of such fuels depends on designing molecular structures and calculating their properties, which can be expedited with computer-aided techniques. In this study, a dataset containing 367 fuel molecules was constructed based on the analysis of direct coal liquefied oil. Three convolutional neural network property prediction models have been created based on molecular structure-physical and chemical property data from the library. All the models have good fitting ability with R2 values above 0.97. Then, a variational autoencoder generation model has been established using the molecular structures from the library, focusing on the structure of saturated cycloalkanes. The structure-property prediction model was then applied to the newly generated molecules, assessing their density, volumetric calorific value, and melting point. As a result, 70000 novel molecular structures were generated, and 25 molecular structures meeting the criteria for high energy density fuels were identified. The established variational autoencoder model in this study effectively assimilates the structural information from the sample set and autonomously generates novel high energy density fuels, which is difficult to achieve in traditional experimental methods.展开更多
Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer i...Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.展开更多
In this study,the hourly directions of eight banking stocks in Borsa Istanbul were predicted using linear-based,deep-learning(LSTM)and ensemble learning(Light-GBM)models.These models were trained with four different f...In this study,the hourly directions of eight banking stocks in Borsa Istanbul were predicted using linear-based,deep-learning(LSTM)and ensemble learning(Light-GBM)models.These models were trained with four different feature sets and their performances were evaluated in terms of accuracy and F-measure metrics.While the first experiments directly used the own stock features as the model inputs,the second experiments utilized reduced stock features through Variational AutoEncoders(VAE).In the last experiments,in order to grasp the effects of the other banking stocks on individual stock performance,the features belonging to other stocks were also given as inputs to our models.While combining other stock features was done for both own(named as allstock_own)and VAE-reduced(named as allstock_VAE)stock features,the expanded dimensions of the feature sets were reduced by Recursive Feature Elimination.As the highest success rate increased up to 0.685 with allstock_own and LSTM with attention model,the combination of allstock_VAE and LSTM with the attention model obtained an accuracy rate of 0.675.Although the classification results achieved with both feature types was close,allstock_VAE achieved these results using nearly 16.67%less features compared to allstock_own.When all experimental results were examined,it was found out that the models trained with allstock_own and allstock_VAE achieved higher accuracy rates than those using individual stock features.It was also concluded that the results obtained with the VAE-reduced stock features were similar to those obtained by own stock features.展开更多
In modern industry,process monitoring plays a significant role in improving the quality of process conduct.With the higher dimensional of the industrial data,the monitoring methods based on the latent variables have b...In modern industry,process monitoring plays a significant role in improving the quality of process conduct.With the higher dimensional of the industrial data,the monitoring methods based on the latent variables have been widely applied in order to decrease the wasting of the industrial database.Nevertheless,these latent variables do not usually follow the Gaussian distribution and thus perform unsuitable when applying some statistics indices,especially the T^(2) on them.Variational AutoEncoders(VAE),an unsupervised deep learning algorithm using the hierarchy study method,has the ability to make the latent variables follow the Gaussian distribution.The partial least squares(PLS)are used to obtain the information between the dependent variables and independent variables.In this paper,we will integrate these two methods and make a comparison with other methods.The superiority of this proposed method will be verified by the simulation and the Trimethylchlorosilane purification process in terms of the multivariate control charts.展开更多
Supervised machine learning algorithms have been widely used in seismic exploration processing,but the lack of labeled examples complicates its application.Therefore,we propose a seismic labeled data expansion method ...Supervised machine learning algorithms have been widely used in seismic exploration processing,but the lack of labeled examples complicates its application.Therefore,we propose a seismic labeled data expansion method based on deep variational Autoencoders(VAE),which are made of neural networks and contains two partsEncoder and Decoder.Lack of training samples leads to overfitting of the network.We training the VAE with whole seismic data,which is a data-driven process and greatly alleviates the risk of overfitting.The Encoder captures the ability to map the seismic waveform Y to latent deep features z,and the Decoder captures the ability to reconstruct high-dimensional waveform Yb from latent deep features z.Later,we put the labeled seismic data into Encoders and get the latent deep features.We can easily use gaussian mixture model to fit the deep feature distribution of each class labeled data.We resample a mass of expansion deep features z* according to the Gaussian mixture model,and put the expansion deep features into the decoder to generate expansion seismic data.The experiments in synthetic and real data show that our method alleviates the problem of lacking labeled seismic data for supervised seismic facies analysis.展开更多
Learning disentangled representation of data is a key problem in deep learning.Specifically,disentangling 2D facial landmarks into different factors(e.g.,identity and expression)is widely used in the applications of f...Learning disentangled representation of data is a key problem in deep learning.Specifically,disentangling 2D facial landmarks into different factors(e.g.,identity and expression)is widely used in the applications of face reconstruction,face reenactment and talking head et al..However,due to the sparsity of landmarks and the lack of accurate labels for the factors,it is hard to learn the disentangled representation of landmarks.To address these problem,we propose a simple and effective model named FLD-VAE to disentangle arbitrary facial landmarks into identity and expression latent representations,which is based on a Variational Autoencoder framework.Besides,we propose three invariant loss functions in both latent and data levels to constrain the invariance of representations during training stage.Moreover,we implement an identity preservation loss to further enhance the representation ability of identity factor.To the best of our knowledge,this is the first work to end-to-end disentangle identity and expression factors simultaneously from one single facial landmark.展开更多
As the popularity and dependence on the Internet increase,DDoS(distributed denial of service)attacks seriously threaten network security.By accurately distinguishing between different types of DDoS attacks,targeted de...As the popularity and dependence on the Internet increase,DDoS(distributed denial of service)attacks seriously threaten network security.By accurately distinguishing between different types of DDoS attacks,targeted defense strategies can be formulated,significantly improving network protection efficiency.DDoS attacks usually manifest as an abnormal increase in network traffic,and their diverse types of attacks,along with a severe data imbalance,make it difficult for traditional classification methods to effectively identify a small number of attack types.To solve this problem,this paper proposes a DDoS recognition method CVWGG(Conditional Variational Autoencoder-Wasserstein Generative Adversarial Network-gradient penalty-Gated Recurrent Unit)for unbalanced data,which generates less noisy data and high data quality compared with existing methods.CVWGG mainly includes unbalanced data processing for CVWG,feature extraction,and classification.CVWGG uses the CVAE(Conditional Variational Autoencoder)to improve the WGAN(Wasserstein Generative Adversarial Network)and introduces a GP(gradient penalty)term to design the loss function to generate balanced data,which enhances the learning ability and stability of the data.Subsequently,the GRU(Gated Recurrent Units)are used to capture the temporal features and patterns of the data.Finally,the logsoftmax function is used to differentiate DDoS attack categories.Using PyCharm and Python 3.10 for programming and evaluating performance with metrics such as accuracy and precision,the results show that the method achieved accuracy rates of 96.0%and 97.3%on two datasets,respectively.Additionally,comparison and ablation experiment results demonstrate that CVWGG effectively mitigates the imbalance between DDoS attack categories,significantly improves the classification accuracy of different types of attacks and provides a valuable reference for network security defense.展开更多
Future 6G communications will open up opportunities for innovative applications,including Cyber-Physical Systems,edge computing,supporting Industry 5.0,and digital agriculture.While automation is creating efficiencies...Future 6G communications will open up opportunities for innovative applications,including Cyber-Physical Systems,edge computing,supporting Industry 5.0,and digital agriculture.While automation is creating efficiencies,it can also create new cyber threats,such as vulnerabilities in trust and malicious node injection.Denialof-Service(DoS)attacks can stop many forms of operations by overwhelming networks and systems with data noise.Current anomaly detection methods require extensive software changes and only detect static threats.Data collection is important for being accurate,but it is often a slow,tedious,and sometimes inefficient process.This paper proposes a new wavelet transformassisted Bayesian deep learning based probabilistic(WT-BDLP)approach tomitigate malicious data injection attacks in 6G edge networks.The proposed approach combines outlier detection based on a Bayesian learning conditional variational autoencoder(Bay-LCVariAE)and traffic pattern analysis based on continuous wavelet transform(CWT).The Bay-LCVariAE framework allows for probabilistic modelling of generative features to facilitate capturing how features of interest change over time,spatially,and for recognition of anomalies.Similarly,CWT allows emphasizing the multi-resolution spectral analysis and permits temporally relevant frequency pattern recognition.Experimental testing showed that the flexibility of the Bayesian probabilistic framework offers a vast improvement in anomaly detection accuracy over existing methods,with a maximum accuracy of 98.21%recognizing anomalies.展开更多
Inverse design has long been an efficient and powerful design tool in the aircraft industry.In this paper,a novel inverse design method for supercritical airfoils is proposed based on generative models in deep learnin...Inverse design has long been an efficient and powerful design tool in the aircraft industry.In this paper,a novel inverse design method for supercritical airfoils is proposed based on generative models in deep learning.A Conditional Variational Auto Encoder(CVAE)and an integrated generative network CVAE-GAN that combines the CVAE with the Wasserstein Generative Adversarial Networks(WGAN),are conducted as generative models.They are used to generate target wall Mach distributions for the inverse design that matches specified features,such as locations of suction peak,shock and aft loading.Qualitative and quantitative results show that both adopted generative models can generate diverse and realistic wall Mach number distributions satisfying the given features.The CVAE-GAN model outperforms the CVAE model and achieves better reconstruction accuracies for all the samples in the dataset.Furthermore,a deep neural network for nonlinear mapping is adopted to obtain the airfoil shape corresponding to the target wall Mach number distribution.The performances of the designed deep neural network are fully demonstrated and a smoothness measurement is proposed to quantify small oscillations in the airfoil surface,proving the authenticity and accuracy of the generated airfoil shapes.展开更多
Stocks that are fundamentally connected with each other tend to move together.Considering such common trends is believed to benefit stock movement forecasting tasks.However,such signals are not trivial to model becaus...Stocks that are fundamentally connected with each other tend to move together.Considering such common trends is believed to benefit stock movement forecasting tasks.However,such signals are not trivial to model because the connections among stocks are not physically presented and need to be estimated from volatile data.Motivated by this observation,we propose a framework that incorporates the inter-connection of firms to forecast stock prices.To effectively utilize a large set of fundamental features,we further design a novel pipeline.First,we use variational autoencoder(VAE)to reduce the dimension of stock fundamental information and then cluster stocks into a graph structure(fundamentally clustering).Second,a hybrid model of graph convolutional network and long-short term memory network(GCN-LSTM)with an adjacency graph matrix(learnt from VAE)is proposed for graph-structured stock market forecasting.Experiments on minute-level U.S.stock market data demonstrate that our model effectively captures both spatial and temporal signals and achieves superior improvement over baseline methods.The proposed model is promising for other applications in which there is a possible but hidden spatial dependency to improve time-series prediction.展开更多
Weak feature extraction is of great importance for condition monitoring and intelligent diagnosis of aeroengine.Aimed at achieving intelligent diagnosis of aero-engine main shaft bearing,an enhanced sparsity-assisted ...Weak feature extraction is of great importance for condition monitoring and intelligent diagnosis of aeroengine.Aimed at achieving intelligent diagnosis of aero-engine main shaft bearing,an enhanced sparsity-assisted intelligent condition monitoring method is proposed in this paper.Through analyzing the weakness of convex sparse model,i.e.the tradeoff between noise reduction and feature reconstruction,this paper proposes an enhanced-sparsity nonconvex regularized convex model based on Moreau envelope to achieve weak feature extraction.Accordingly,a sparsity-assisted deep convolutional variational autoencoders network is proposed,which achieves the intelligent identification of fault state through training denoised normal data.Finally,the effectiveness of the proposed method is verified through aero-engine bearing run-to-failure experiment.The comparison results show that the proposed method is good at abnormal pattern recognition,showing a good potential for weak fault intelligent diagnosis of aero-engine main shaft bearings.展开更多
With the development of intelligent agents pursuing humanisation,artificial intelligence must consider emotion,the most basic spiritual need in human interaction.Traditional emotional dialogue systems usually use an e...With the development of intelligent agents pursuing humanisation,artificial intelligence must consider emotion,the most basic spiritual need in human interaction.Traditional emotional dialogue systems usually use an external emotional dictionary to select appropriate emotional words to add to the response or concatenate emotional tags and semantic features in the decoding step to generate appropriate responses.However,selecting emotional words from a fixed emotional dictionary may result in loss of the diversity and consistency of the response.We propose a semantic and emotion-based dual latent variable generation model(Dual-LVG)for dialogue systems,which is able to generate appropriate emotional responses without an emotional dictionary.Different from previous work,the conditional variational autoencoder(CVAE)adopts the standard transformer structure.Then,Dual-LVG regularises the CVAE latent space by introducing a dual latent space of semantics and emotion.The content diversity and emotional accuracy of the generated responses are improved by learning emotion and semantic features respectively.Moreover,the average attention mechanism is adopted to better extract semantic features at the sequence level,and the semi-supervised attention mechanism is used in the decoding step to strengthen the fusion of emotional features of the model.Experimental results show that Dual-LVG can successfully achieve the effect of generating different content by controlling emotional factors.展开更多
Ensuring the safe and efficient operation of self-driving vehicles relies heavily on accurately predicting their future trajectories.Existing approaches commonly employ an encoder-decoder neural network structure to e...Ensuring the safe and efficient operation of self-driving vehicles relies heavily on accurately predicting their future trajectories.Existing approaches commonly employ an encoder-decoder neural network structure to enhance information extraction during the encoding phase.However,these methods often neglect the inclusion of road rule constraints during trajectory formulation in the decoding phase.This paper proposes a novel method that combines neural networks and rule-based constraints in the decoder stage to improve trajectory prediction accuracy while ensuring compliance with vehicle kinematics and road rules.The approach separates vehicle trajectories into lateral and longitudinal routes and utilizes conditional variational autoencoder(CVAE)to capture trajectory uncertainty.The evaluation results demonstrate a reduction of 32.4%and 27.6%in the average displacement error(ADE)for predicting the top five and top ten trajectories,respectively,compared to the baseline method.展开更多
Contemporary attackers,mainly motivated by financial gain,consistently devise sophisticated penetration techniques to access important information or data.The growing use of Internet of Things(IoT)technology in the co...Contemporary attackers,mainly motivated by financial gain,consistently devise sophisticated penetration techniques to access important information or data.The growing use of Internet of Things(IoT)technology in the contemporary convergence environment to connect to corporate networks and cloud-based applications only worsens this situation,as it facilitates multiple new attack vectors to emerge effortlessly.As such,existing intrusion detection systems suffer from performance degradation mainly because of insufficient considerations and poorly modeled detection systems.To address this problem,we designed a blended threat detection approach,considering the possible impact and dimensionality of new attack surfaces due to the aforementioned convergence.We collectively refer to the convergence of different technology sectors as the internet of blended environment.The proposed approach encompasses an ensemble of heterogeneous probabilistic autoencoders that leverage the corresponding advantages of a convolutional variational autoencoder and long short-term memory variational autoencoder.An extensive experimental analysis conducted on the TON_IoT dataset demonstrated 96.02%detection accuracy.Furthermore,performance of the proposed approach was compared with various single model(autoencoder)-based network intrusion detection approaches:autoencoder,variational autoencoder,convolutional variational autoencoder,and long short-term memory variational autoencoder.The proposed model outperformed all compared models,demonstrating F1-score improvements of 4.99%,2.25%,1.92%,and 3.69%,respectively.展开更多
<div style="text-align:justify;"> Most existing image dehazing methods based learning are less able to perform well to real hazy images. An important reason is that they are trained on synthetic hazy i...<div style="text-align:justify;"> Most existing image dehazing methods based learning are less able to perform well to real hazy images. An important reason is that they are trained on synthetic hazy images whose distribution is different from real hazy images. To relieve this issue, this paper proposes a new hazy scene generation model based on domain adaptation, which uses a variational autoencoder to encode the synthetic hazy image pairs and the real hazy images into the latent space to adapt. The synthetic hazy image pairs guide the model to learn the mapping of clear images to hazy images, the real hazy images are used to adapt the synthetic hazy images’ latent space to real hazy images through generative adversarial loss, so as to make the generative hazy images’ distribution as close to the real hazy images’ distribution as possible. By comparing the results of the model with traditional physical scattering models and Adobe Lightroom CC software, the hazy images generated in this paper is more realistic. Our end-to-end domain adaptation model is also very convenient to synthesize hazy images without depth map. Using traditional method to dehaze the synthetic hazy images generated by this paper, both SSIM and PSNR have been improved, proved that the effectiveness of our method. The non-reference haze density evaluation algorithm and other quantitative evaluation also illustrate the advantages of our method in synthetic hazy images. </div>展开更多
Anomaly detection is one of the many challenging areas in cybersecurity.The anomaly can occur in many forms,such as fraudulent credit card transactions,network intrusions,and anomalous imageries or documents.One of th...Anomaly detection is one of the many challenging areas in cybersecurity.The anomaly can occur in many forms,such as fraudulent credit card transactions,network intrusions,and anomalous imageries or documents.One of the most common challenges in anomaly detection is the obscurity of the normal state and the lack of anomalous samples.Traditionally,this problem is tackled by using resampling techniques or choosing models that approximate the distribution of the normal states.Variational AutoEncoder(VAE)has been studied in anomaly detections despite being more suitable in generative tasks.This study aims to explore the usage of VAE in credit card anomaly detection and evaluate latent space sampling techniques.In this study,we evaluate the usage of the convolutional network-based VAE model on a credit card transaction dataset.We train two VAE models,one with a large number of normal data and one with a small number of anomalous data.We compare the performance of both VAE models and evaluate the latent space of both VAE models by rescaling them with reconstruction error vectors.We also compare the effectiveness of the VAE model with other anomaly detection models when they are trained on imbalanced dataset.展开更多
基金supported by the National Natural Science Foundation of China(Nos.42530801,42425208)the Natural Science Foundation of Hubei Province(China)(No.2023AFA001)+1 种基金the MOST Special Fund from State Key Laboratory of Geological Processes and Mineral Resources,China University of Geosciences(No.MSFGPMR2025-401)the China Scholarship Council(No.202306410181)。
文摘Geochemical survey data are essential across Earth Science disciplines but are often affected by noise,which can obscure important geological signals and compromise subsequent prediction and interpretation.Quantifying prediction uncertainty is hence crucial for robust geoscientific decision-making.This study proposes a novel deep learning framework,the Spatially Constrained Variational Autoencoder(SC-VAE),for denoising geochemical survey data with integrated uncertainty quantification.The SC-VAE incorporates spatial regularization,which enforces spatial coherence by modeling inter-sample relationships directly within the latent space.The performance of the SC-VAE was systematically evaluated against a standard Variational Autoencoder(VAE)using geochemical data from the gold polymetallic district in the northwestern part of Sichuan Province,China.Both models were optimized using Bayesian optimization,with objective functions specifically designed to maintain essential geostatistical characteristics.Evaluation metrics include variogram analysis,quantitative measures of spatial interpolation accuracy,visual assessment of denoised maps,and statistical analysis of data distributions,as well as decomposition of uncertainties.Results show that the SC-VAE achieves superior noise suppression and better preservation of spatial structure compared to the standard VAE,as demonstrated by a significant reduction in the variogram nugget effect and an increased partial sill.The SC-VAE produces denoised maps with clearer anomaly delineation and more regularized data distributions,effectively mitigating outliers and reducing kurtosis.Additionally,it delivers improved interpolation accuracy and spatially explicit uncertainty estimates,facilitating more reliable and interpretable assessments of prediction confidence.The SC-VAE framework thus provides a robust,geostatistically informed solution for enhancing the quality and interpretability of geochemical data,with broad applicability in mineral exploration,environmental geochemistry,and other Earth Science domains.
基金co-supported by the Natural Science Basic Research Program of Shaanxi,China(No.2023-JC-QN-0043)the ND Basic Research Funds,China(No.G2022WD).
文摘The aerial deployment method enables Unmanned Aerial Vehicles(UAVs)to be directly positioned at the required altitude for their mission.This method typically employs folding technology to improve loading efficiency,with applications such as the gravity-only aerial deployment of high-aspect-ratio solar-powered UAVs,and aerial takeoff of fixed-wing drones in Mars research.However,the significant morphological changes during deployment are accompanied by strong nonlinear dynamic aerodynamic forces,which result in multiple degrees of freedom and an unstable character.This hinders the description and analysis of unknown dynamic behaviors,further leading to difficulties in the design of deployment strategies and flight control.To address this issue,this paper proposes an analysis method for dynamic behaviors during aerial deployment based on the Variational Autoencoder(VAE).Focusing on the gravity-only deployment problem of highaspect-ratio foldable-wing UAVs,the method encodes the multi-degree-of-freedom unstable motion signals into a low-dimensional feature space through a data-driven approach.By clustering in the feature space,this paper identifies and studies several dynamic behaviors during aerial deployment.The research presented in this paper offers a new method and perspective for feature extraction and analysis of complex and difficult-to-describe extreme flight dynamics,guiding the research on aerial deployment drones design and control strategies.
基金the financial support from the Chilean National Research and Development Agency(Agencia Nacional de Investigación y Desarrollo,ANID)through Fondo Nacional de Desarrollo Científico y Tecnológico(FONDECYT)Regular 1240503Fondo de Valorización de la Investigación(FOVI)230030 projectsthe financial support from the ANID through FONDECYT Reg-ular 1240501.
文摘In the face of the unrelenting challenge posed by earthquakes-a natural hazard of unpredictable nature with a legacy of significant loss of life,destruction of infrastructure,and profound economic and social impacts-the scientific community has pursued advancements in earthquake early warning systems(EEWSs).These systems are vital for pre-emptive actions and decision-making that can save lives and safeguard critical infrastructure.This study proposes and validates a domain-informed deep learning-based EEWS called the hybrid earthquake early warning framework for estimating response spectra(HEWFERS),which represents a significant leap forward in the capabilities to predict ground shaking intensity in real-time,aligning with the United Nations’disaster risk reduction goals.HEWFERS ingeniously integrates a domain-informed variational autoencoder for physics-based latent variable(LV)extraction,a feed-forward neural network for on-site prediction,and Gaussian process regression for spatial prediction.Adopting explainable artificial intelligence-based Shapley explanations further elucidates the predictive mechanisms,ensuring stakeholder-informed decisions.By conducting an extensive analysis of the proposed framework under a large database of approximately 14000 recorded ground motions,this study offers insights into the potential of integrating machine learning with seismology to revolutionize earthquake preparedness and response,thus paving the way for a safer and more resilient future.
基金Financial support from the Fundamental Research Funds for the Central Universities(ZJ2022-003,JG2022-27,J2020-060,and J2021-060)Sichuan Province Engineering Technology Research Center of General Aircraft Maintenance(GAMRC2021YB08)the Young Scientists Fund of the National Natural Science Foundation of China(No.52105417)is acknowledged.
文摘To overcome the challenges of limited experimental data and improve the accuracy of empirical formulas,we propose a low-cycle fatigue(LCF)life prediction model for nickel-based superalloys using a data augmentation method.This method utilizes a variational autoencoder(VAE)to generate low-cycle fatigue data and form an augmented dataset.The Pearson correlation coefficient(PCC)is employed to verify the similarity of feature distributions between the original and augmented datasets.Six machine learning models,namely random forest(RF),artificial neural network(ANN),support vector machine(SVM),gradient-boosted decision tree(GBDT),eXtreme Gradient Boosting(XGBoost),and Categorical Boosting(CatBoost),are utilized to predict the LCF life of nickel-based superalloys.Results indicate that the proposed data augmentation method based on VAE can effectively expand the dataset,and the mean absolute error(MAE),root mean square error(RMSE),and R-squared(R^(2))values achieved using the CatBoost model,with respective values of 0.0242,0.0391,and 0.9538,are superior to those of the other models.The proposed method reduces the cost and time associated with LCF experiments and accurately establishes the relationship between fatigue characteristics and LCF life of nickel-based superalloys.
基金the National Natural Science Foundation of China(22178243 and 22038008).
文摘Direct coal liquefaction products offer a considerable quantity of cycloalkanes, which are the valuable candidates for making the high energy density fuels. The creation of such fuels depends on designing molecular structures and calculating their properties, which can be expedited with computer-aided techniques. In this study, a dataset containing 367 fuel molecules was constructed based on the analysis of direct coal liquefied oil. Three convolutional neural network property prediction models have been created based on molecular structure-physical and chemical property data from the library. All the models have good fitting ability with R2 values above 0.97. Then, a variational autoencoder generation model has been established using the molecular structures from the library, focusing on the structure of saturated cycloalkanes. The structure-property prediction model was then applied to the newly generated molecules, assessing their density, volumetric calorific value, and melting point. As a result, 70000 novel molecular structures were generated, and 25 molecular structures meeting the criteria for high energy density fuels were identified. The established variational autoencoder model in this study effectively assimilates the structural information from the sample set and autonomously generates novel high energy density fuels, which is difficult to achieve in traditional experimental methods.
基金National Natural Science Foundation of China(61976209,62020106015,U21A20388)in part by the CAS International Collaboration Key Project(173211KYSB20190024)in part by the Strategic Priority Research Program of CAS(XDB32040000)。
文摘Traditional electroencephalograph(EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject,which restricts the application of the affective brain computer interface(BCI)in practice.We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples.To solve this problem,we propose a multimodal domain adaptive variational autoencoder(MMDA-VAE)method,which learns shared cross-domain latent representations of the multi-modal data.Our method builds a multi-modal variational autoencoder(MVAE)to project the data of multiple modalities into a common space.Through adversarial learning and cycle-consistency regularization,our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge.Extensive experiments are conducted on two public datasets,SEED and SEED-IV,and the results show the superiority of our proposed method.Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.
文摘In this study,the hourly directions of eight banking stocks in Borsa Istanbul were predicted using linear-based,deep-learning(LSTM)and ensemble learning(Light-GBM)models.These models were trained with four different feature sets and their performances were evaluated in terms of accuracy and F-measure metrics.While the first experiments directly used the own stock features as the model inputs,the second experiments utilized reduced stock features through Variational AutoEncoders(VAE).In the last experiments,in order to grasp the effects of the other banking stocks on individual stock performance,the features belonging to other stocks were also given as inputs to our models.While combining other stock features was done for both own(named as allstock_own)and VAE-reduced(named as allstock_VAE)stock features,the expanded dimensions of the feature sets were reduced by Recursive Feature Elimination.As the highest success rate increased up to 0.685 with allstock_own and LSTM with attention model,the combination of allstock_VAE and LSTM with the attention model obtained an accuracy rate of 0.675.Although the classification results achieved with both feature types was close,allstock_VAE achieved these results using nearly 16.67%less features compared to allstock_own.When all experimental results were examined,it was found out that the models trained with allstock_own and allstock_VAE achieved higher accuracy rates than those using individual stock features.It was also concluded that the results obtained with the VAE-reduced stock features were similar to those obtained by own stock features.
文摘In modern industry,process monitoring plays a significant role in improving the quality of process conduct.With the higher dimensional of the industrial data,the monitoring methods based on the latent variables have been widely applied in order to decrease the wasting of the industrial database.Nevertheless,these latent variables do not usually follow the Gaussian distribution and thus perform unsuitable when applying some statistics indices,especially the T^(2) on them.Variational AutoEncoders(VAE),an unsupervised deep learning algorithm using the hierarchy study method,has the ability to make the latent variables follow the Gaussian distribution.The partial least squares(PLS)are used to obtain the information between the dependent variables and independent variables.In this paper,we will integrate these two methods and make a comparison with other methods.The superiority of this proposed method will be verified by the simulation and the Trimethylchlorosilane purification process in terms of the multivariate control charts.
基金Supported by National Natural Science Foundation of China(41804126,41604107).
文摘Supervised machine learning algorithms have been widely used in seismic exploration processing,but the lack of labeled examples complicates its application.Therefore,we propose a seismic labeled data expansion method based on deep variational Autoencoders(VAE),which are made of neural networks and contains two partsEncoder and Decoder.Lack of training samples leads to overfitting of the network.We training the VAE with whole seismic data,which is a data-driven process and greatly alleviates the risk of overfitting.The Encoder captures the ability to map the seismic waveform Y to latent deep features z,and the Decoder captures the ability to reconstruct high-dimensional waveform Yb from latent deep features z.Later,we put the labeled seismic data into Encoders and get the latent deep features.We can easily use gaussian mixture model to fit the deep feature distribution of each class labeled data.We resample a mass of expansion deep features z* according to the Gaussian mixture model,and put the expansion deep features into the decoder to generate expansion seismic data.The experiments in synthetic and real data show that our method alleviates the problem of lacking labeled seismic data for supervised seismic facies analysis.
基金Supported by the National Natural Science Foundation of China(61210007).
文摘Learning disentangled representation of data is a key problem in deep learning.Specifically,disentangling 2D facial landmarks into different factors(e.g.,identity and expression)is widely used in the applications of face reconstruction,face reenactment and talking head et al..However,due to the sparsity of landmarks and the lack of accurate labels for the factors,it is hard to learn the disentangled representation of landmarks.To address these problem,we propose a simple and effective model named FLD-VAE to disentangle arbitrary facial landmarks into identity and expression latent representations,which is based on a Variational Autoencoder framework.Besides,we propose three invariant loss functions in both latent and data levels to constrain the invariance of representations during training stage.Moreover,we implement an identity preservation loss to further enhance the representation ability of identity factor.To the best of our knowledge,this is the first work to end-to-end disentangle identity and expression factors simultaneously from one single facial landmark.
基金The financial support from the Fundamental Research Funds for Higher Education Institutions of Heilongjiang Province(Grant No.145209126)the Research and Innovation Platform Project(Grant No.145309314)is acknowledged.
文摘As the popularity and dependence on the Internet increase,DDoS(distributed denial of service)attacks seriously threaten network security.By accurately distinguishing between different types of DDoS attacks,targeted defense strategies can be formulated,significantly improving network protection efficiency.DDoS attacks usually manifest as an abnormal increase in network traffic,and their diverse types of attacks,along with a severe data imbalance,make it difficult for traditional classification methods to effectively identify a small number of attack types.To solve this problem,this paper proposes a DDoS recognition method CVWGG(Conditional Variational Autoencoder-Wasserstein Generative Adversarial Network-gradient penalty-Gated Recurrent Unit)for unbalanced data,which generates less noisy data and high data quality compared with existing methods.CVWGG mainly includes unbalanced data processing for CVWG,feature extraction,and classification.CVWGG uses the CVAE(Conditional Variational Autoencoder)to improve the WGAN(Wasserstein Generative Adversarial Network)and introduces a GP(gradient penalty)term to design the loss function to generate balanced data,which enhances the learning ability and stability of the data.Subsequently,the GRU(Gated Recurrent Units)are used to capture the temporal features and patterns of the data.Finally,the logsoftmax function is used to differentiate DDoS attack categories.Using PyCharm and Python 3.10 for programming and evaluating performance with metrics such as accuracy and precision,the results show that the method achieved accuracy rates of 96.0%and 97.3%on two datasets,respectively.Additionally,comparison and ablation experiment results demonstrate that CVWGG effectively mitigates the imbalance between DDoS attack categories,significantly improves the classification accuracy of different types of attacks and provides a valuable reference for network security defense.
文摘Future 6G communications will open up opportunities for innovative applications,including Cyber-Physical Systems,edge computing,supporting Industry 5.0,and digital agriculture.While automation is creating efficiencies,it can also create new cyber threats,such as vulnerabilities in trust and malicious node injection.Denialof-Service(DoS)attacks can stop many forms of operations by overwhelming networks and systems with data noise.Current anomaly detection methods require extensive software changes and only detect static threats.Data collection is important for being accurate,but it is often a slow,tedious,and sometimes inefficient process.This paper proposes a new wavelet transformassisted Bayesian deep learning based probabilistic(WT-BDLP)approach tomitigate malicious data injection attacks in 6G edge networks.The proposed approach combines outlier detection based on a Bayesian learning conditional variational autoencoder(Bay-LCVariAE)and traffic pattern analysis based on continuous wavelet transform(CWT).The Bay-LCVariAE framework allows for probabilistic modelling of generative features to facilitate capturing how features of interest change over time,spatially,and for recognition of anomalies.Similarly,CWT allows emphasizing the multi-resolution spectral analysis and permits temporally relevant frequency pattern recognition.Experimental testing showed that the flexibility of the Bayesian probabilistic framework offers a vast improvement in anomaly detection accuracy over existing methods,with a maximum accuracy of 98.21%recognizing anomalies.
基金co-supported by the National Key Project of China(No.GJXM92579)the National Natural Science Foundation of China(Nos.92052203,61903178 and61906081)。
文摘Inverse design has long been an efficient and powerful design tool in the aircraft industry.In this paper,a novel inverse design method for supercritical airfoils is proposed based on generative models in deep learning.A Conditional Variational Auto Encoder(CVAE)and an integrated generative network CVAE-GAN that combines the CVAE with the Wasserstein Generative Adversarial Networks(WGAN),are conducted as generative models.They are used to generate target wall Mach distributions for the inverse design that matches specified features,such as locations of suction peak,shock and aft loading.Qualitative and quantitative results show that both adopted generative models can generate diverse and realistic wall Mach number distributions satisfying the given features.The CVAE-GAN model outperforms the CVAE model and achieves better reconstruction accuracies for all the samples in the dataset.Furthermore,a deep neural network for nonlinear mapping is adopted to obtain the airfoil shape corresponding to the target wall Mach number distribution.The performances of the designed deep neural network are fully demonstrated and a smoothness measurement is proposed to quantify small oscillations in the airfoil surface,proving the authenticity and accuracy of the generated airfoil shapes.
文摘Stocks that are fundamentally connected with each other tend to move together.Considering such common trends is believed to benefit stock movement forecasting tasks.However,such signals are not trivial to model because the connections among stocks are not physically presented and need to be estimated from volatile data.Motivated by this observation,we propose a framework that incorporates the inter-connection of firms to forecast stock prices.To effectively utilize a large set of fundamental features,we further design a novel pipeline.First,we use variational autoencoder(VAE)to reduce the dimension of stock fundamental information and then cluster stocks into a graph structure(fundamentally clustering).Second,a hybrid model of graph convolutional network and long-short term memory network(GCN-LSTM)with an adjacency graph matrix(learnt from VAE)is proposed for graph-structured stock market forecasting.Experiments on minute-level U.S.stock market data demonstrate that our model effectively captures both spatial and temporal signals and achieves superior improvement over baseline methods.The proposed model is promising for other applications in which there is a possible but hidden spatial dependency to improve time-series prediction.
基金the National Natural Science Foundations of China(Nos.91860125,51705398)the National Key Basic Research Program of China(No.2015CB057400)the Shaanxi Province 2020 Natural Science Basic Research Plan(No.2020JQ-042).
文摘Weak feature extraction is of great importance for condition monitoring and intelligent diagnosis of aeroengine.Aimed at achieving intelligent diagnosis of aero-engine main shaft bearing,an enhanced sparsity-assisted intelligent condition monitoring method is proposed in this paper.Through analyzing the weakness of convex sparse model,i.e.the tradeoff between noise reduction and feature reconstruction,this paper proposes an enhanced-sparsity nonconvex regularized convex model based on Moreau envelope to achieve weak feature extraction.Accordingly,a sparsity-assisted deep convolutional variational autoencoders network is proposed,which achieves the intelligent identification of fault state through training denoised normal data.Finally,the effectiveness of the proposed method is verified through aero-engine bearing run-to-failure experiment.The comparison results show that the proposed method is good at abnormal pattern recognition,showing a good potential for weak fault intelligent diagnosis of aero-engine main shaft bearings.
基金Fundamental Research Funds for the Central Universities of China,Grant/Award Number:CUC220B009National Natural Science Foundation of China,Grant/Award Numbers:62207029,62271454,72274182。
文摘With the development of intelligent agents pursuing humanisation,artificial intelligence must consider emotion,the most basic spiritual need in human interaction.Traditional emotional dialogue systems usually use an external emotional dictionary to select appropriate emotional words to add to the response or concatenate emotional tags and semantic features in the decoding step to generate appropriate responses.However,selecting emotional words from a fixed emotional dictionary may result in loss of the diversity and consistency of the response.We propose a semantic and emotion-based dual latent variable generation model(Dual-LVG)for dialogue systems,which is able to generate appropriate emotional responses without an emotional dictionary.Different from previous work,the conditional variational autoencoder(CVAE)adopts the standard transformer structure.Then,Dual-LVG regularises the CVAE latent space by introducing a dual latent space of semantics and emotion.The content diversity and emotional accuracy of the generated responses are improved by learning emotion and semantic features respectively.Moreover,the average attention mechanism is adopted to better extract semantic features at the sequence level,and the semi-supervised attention mechanism is used in the decoding step to strengthen the fusion of emotional features of the model.Experimental results show that Dual-LVG can successfully achieve the effect of generating different content by controlling emotional factors.
基金supported in part by the National Natural Science Foundation of China under Grant 52372393,62003238in part by the DongfengTechnology Center(Research and Application of Next-Generation Low-Carbonntelligent Architecture Technology).
文摘Ensuring the safe and efficient operation of self-driving vehicles relies heavily on accurately predicting their future trajectories.Existing approaches commonly employ an encoder-decoder neural network structure to enhance information extraction during the encoding phase.However,these methods often neglect the inclusion of road rule constraints during trajectory formulation in the decoding phase.This paper proposes a novel method that combines neural networks and rule-based constraints in the decoder stage to improve trajectory prediction accuracy while ensuring compliance with vehicle kinematics and road rules.The approach separates vehicle trajectories into lateral and longitudinal routes and utilizes conditional variational autoencoder(CVAE)to capture trajectory uncertainty.The evaluation results demonstrate a reduction of 32.4%and 27.6%in the average displacement error(ADE)for predicting the top five and top ten trajectories,respectively,compared to the baseline method.
基金This work was supported by the National Research Foundation of Korea(NRF)grant funded by the Korean government(MSIT)(No.2021R1A2C2011391)was supported by the Institute of Information&communications Technology Planning&Evaluation(IITP)grant funded by the Korea government(MSIT)(No.2021-0-01806Development of security by design and security management technology in smart factory).
文摘Contemporary attackers,mainly motivated by financial gain,consistently devise sophisticated penetration techniques to access important information or data.The growing use of Internet of Things(IoT)technology in the contemporary convergence environment to connect to corporate networks and cloud-based applications only worsens this situation,as it facilitates multiple new attack vectors to emerge effortlessly.As such,existing intrusion detection systems suffer from performance degradation mainly because of insufficient considerations and poorly modeled detection systems.To address this problem,we designed a blended threat detection approach,considering the possible impact and dimensionality of new attack surfaces due to the aforementioned convergence.We collectively refer to the convergence of different technology sectors as the internet of blended environment.The proposed approach encompasses an ensemble of heterogeneous probabilistic autoencoders that leverage the corresponding advantages of a convolutional variational autoencoder and long short-term memory variational autoencoder.An extensive experimental analysis conducted on the TON_IoT dataset demonstrated 96.02%detection accuracy.Furthermore,performance of the proposed approach was compared with various single model(autoencoder)-based network intrusion detection approaches:autoencoder,variational autoencoder,convolutional variational autoencoder,and long short-term memory variational autoencoder.The proposed model outperformed all compared models,demonstrating F1-score improvements of 4.99%,2.25%,1.92%,and 3.69%,respectively.
文摘<div style="text-align:justify;"> Most existing image dehazing methods based learning are less able to perform well to real hazy images. An important reason is that they are trained on synthetic hazy images whose distribution is different from real hazy images. To relieve this issue, this paper proposes a new hazy scene generation model based on domain adaptation, which uses a variational autoencoder to encode the synthetic hazy image pairs and the real hazy images into the latent space to adapt. The synthetic hazy image pairs guide the model to learn the mapping of clear images to hazy images, the real hazy images are used to adapt the synthetic hazy images’ latent space to real hazy images through generative adversarial loss, so as to make the generative hazy images’ distribution as close to the real hazy images’ distribution as possible. By comparing the results of the model with traditional physical scattering models and Adobe Lightroom CC software, the hazy images generated in this paper is more realistic. Our end-to-end domain adaptation model is also very convenient to synthesize hazy images without depth map. Using traditional method to dehaze the synthetic hazy images generated by this paper, both SSIM and PSNR have been improved, proved that the effectiveness of our method. The non-reference haze density evaluation algorithm and other quantitative evaluation also illustrate the advantages of our method in synthetic hazy images. </div>
文摘Anomaly detection is one of the many challenging areas in cybersecurity.The anomaly can occur in many forms,such as fraudulent credit card transactions,network intrusions,and anomalous imageries or documents.One of the most common challenges in anomaly detection is the obscurity of the normal state and the lack of anomalous samples.Traditionally,this problem is tackled by using resampling techniques or choosing models that approximate the distribution of the normal states.Variational AutoEncoder(VAE)has been studied in anomaly detections despite being more suitable in generative tasks.This study aims to explore the usage of VAE in credit card anomaly detection and evaluate latent space sampling techniques.In this study,we evaluate the usage of the convolutional network-based VAE model on a credit card transaction dataset.We train two VAE models,one with a large number of normal data and one with a small number of anomalous data.We compare the performance of both VAE models and evaluate the latent space of both VAE models by rescaling them with reconstruction error vectors.We also compare the effectiveness of the VAE model with other anomaly detection models when they are trained on imbalanced dataset.
