The Informer model leverages its innovative ProbSparse self-attention mechanism to demonstrate significant performance advantages in long-sequence time-series forecasting tasks.However,when confronted with time-series...The Informer model leverages its innovative ProbSparse self-attention mechanism to demonstrate significant performance advantages in long-sequence time-series forecasting tasks.However,when confronted with time-series data exhibiting multi-scale characteristics and substantial noise,the model’s attention mechanism reveals inherent limitations.Specifically,the model is susceptible to interference from local noise or irrelevant patterns,leading to diminished focus on globally critical information and consequently impairing forecasting accuracy.To address this challenge,this study proposes an enhanced architecture that integrates a Gated Attention mechanism into the original Informer framework.This mechanism employs learnable gating functions to dynamically and selectively impose differentiated weighting on crucial temporal segments and discriminative feature dimensions within the input sequence.This adaptive weighting strategy is designed to effectively suppress noise interference while amplifying the capture of core dynamic patterns.Consequently,it substantially strengthens the model’s capability to represent complex temporal dynamics and ultimately elevates its predictive performance.展开更多
Liver cancer has the second highest incidence rate among all types of malignant tumors,and currently,its diagnosis heavily depends on doctors’manual labeling of CT scan images,a process that is time-consuming and sus...Liver cancer has the second highest incidence rate among all types of malignant tumors,and currently,its diagnosis heavily depends on doctors’manual labeling of CT scan images,a process that is time-consuming and susceptible to subjective errors.To address the aforementioned issues,we propose an automatic segmentation model for liver and tumors called Res2Swin Unet,which is based on the Unet architecture.The model combines Attention-Res2 and Swin Transformer modules for liver and tumor segmentation,respectively.Attention-Res2 merges multiple feature map parts with an Attention gate via skip connections,while Swin Transformer captures long-range dependencies and models the input globally.And the model uses deep supervision and a hybrid loss function for faster convergence.On the LiTS2017 dataset,it achieves better segmentation performance than other models,with an average Dice coefficient of 97.0%for liver segmentation and 81.2%for tumor segmentation.展开更多
Mechanical vibration defect is the key factor leading to sudden failure of gas-insulated switchgear(GIS)equipment.It is important to realise effective prediction of the me-chanical vibration state development trend of...Mechanical vibration defect is the key factor leading to sudden failure of gas-insulated switchgear(GIS)equipment.It is important to realise effective prediction of the me-chanical vibration state development trend of GIS equipment in order to improve its active safety protection level.This paper carried out research on the accurate prediction method and experimental validation of the mechanical vibration state and its defect severity development trend for the GIS equipment.Firstly,the deep and shallow vibration feature parameters for different mechanical defect signals were jointly extracted by time-domain features and deep belief network methods.Secondly,a new prediction model,incorporating the attention mechanism and the bidirectional gated recurrent unit(BiGRU),was constructed with the deep and shallow vibration feature parameters as inputs.Finally,the prediction trend effectiveness was verified based on the real-type GIS mechanical simulation platform and the field operation GIS equipment.Results show that the deep and shallow vibration feature extraction method proposed in this paper can characterise the mechanical defect information more comprehensively.The new prediction method of the vibration state trend based on the attention-BiGRU model shows ideal accuracy,and the predicted vibration state development trend is highly consistent with the actual,with an average absolute error of 0.063.The root mean square error(ERMSE)value of the prediction method is<5%,which reduces the relative error value at least 37% compared with the traditional prediction models.This paper provides a valuable reference for the proactive defence of GIS mechanical failure.展开更多
In lung nodules there is a huge variation in structural properties like Shape, Surface Texture. Even the spatial properties vary, where they can be found attached to lung walls, blood vessels in complex non-homogenous...In lung nodules there is a huge variation in structural properties like Shape, Surface Texture. Even the spatial properties vary, where they can be found attached to lung walls, blood vessels in complex non-homogenous lung structures. Moreover, the nodules are of small size at their early stage of development. This poses a serious challenge to develop a Computer aided diagnosis (CAD) system with better false positive reduction. Hence, to reduce the false positives per scan and to deal with the challenges mentioned, this paper proposes a set of three diverse 3D Attention based CNN architectures (3D ACNN) whose predictions on given low dose Volumetric Computed Tomography (CT) scans are fused to achieve more effective and reliable results. Attention mechanism is employed to selectively concentrate/weigh more on nodule specific features and less weight age over other irrelevant features. By using this attention based mechanism in CNN unlike traditional methods there was a significant gain in the classification performance. Contextual dependencies are also taken into account by giving three patches of different sizes surrounding the nodule as input to the ACNN architectures. The system is trained and validated using a publicly available LUNA16 dataset in a 10 fold cross validation approach where a competition performance metric (CPM) score of 0.931 is achieved. The experimental results demonstrate that either a single patch or a single architecture in a one-to-one fashion that is adopted in earlier methods cannot achieve a better performance and signifies the necessity of fusing different multi patched architectures. Though the proposed system is mainly designed for pulmonary nodule detection it can be easily extended to classification tasks of any other 3D medical diagnostic computed tomography images where there is a huge variation and uncertainty in classification.展开更多
Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularl...Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.展开更多
文摘The Informer model leverages its innovative ProbSparse self-attention mechanism to demonstrate significant performance advantages in long-sequence time-series forecasting tasks.However,when confronted with time-series data exhibiting multi-scale characteristics and substantial noise,the model’s attention mechanism reveals inherent limitations.Specifically,the model is susceptible to interference from local noise or irrelevant patterns,leading to diminished focus on globally critical information and consequently impairing forecasting accuracy.To address this challenge,this study proposes an enhanced architecture that integrates a Gated Attention mechanism into the original Informer framework.This mechanism employs learnable gating functions to dynamically and selectively impose differentiated weighting on crucial temporal segments and discriminative feature dimensions within the input sequence.This adaptive weighting strategy is designed to effectively suppress noise interference while amplifying the capture of core dynamic patterns.Consequently,it substantially strengthens the model’s capability to represent complex temporal dynamics and ultimately elevates its predictive performance.
文摘Liver cancer has the second highest incidence rate among all types of malignant tumors,and currently,its diagnosis heavily depends on doctors’manual labeling of CT scan images,a process that is time-consuming and susceptible to subjective errors.To address the aforementioned issues,we propose an automatic segmentation model for liver and tumors called Res2Swin Unet,which is based on the Unet architecture.The model combines Attention-Res2 and Swin Transformer modules for liver and tumor segmentation,respectively.Attention-Res2 merges multiple feature map parts with an Attention gate via skip connections,while Swin Transformer captures long-range dependencies and models the input globally.And the model uses deep supervision and a hybrid loss function for faster convergence.On the LiTS2017 dataset,it achieves better segmentation performance than other models,with an average Dice coefficient of 97.0%for liver segmentation and 81.2%for tumor segmentation.
基金National Key R&D Program of China,Grant/Award Numbers:2022YFB2403700,2022YFB2403705Natural Science Foundation of Chongqing,Grant/Award Number:CSTB2022NSCQ-MSX1247。
文摘Mechanical vibration defect is the key factor leading to sudden failure of gas-insulated switchgear(GIS)equipment.It is important to realise effective prediction of the me-chanical vibration state development trend of GIS equipment in order to improve its active safety protection level.This paper carried out research on the accurate prediction method and experimental validation of the mechanical vibration state and its defect severity development trend for the GIS equipment.Firstly,the deep and shallow vibration feature parameters for different mechanical defect signals were jointly extracted by time-domain features and deep belief network methods.Secondly,a new prediction model,incorporating the attention mechanism and the bidirectional gated recurrent unit(BiGRU),was constructed with the deep and shallow vibration feature parameters as inputs.Finally,the prediction trend effectiveness was verified based on the real-type GIS mechanical simulation platform and the field operation GIS equipment.Results show that the deep and shallow vibration feature extraction method proposed in this paper can characterise the mechanical defect information more comprehensively.The new prediction method of the vibration state trend based on the attention-BiGRU model shows ideal accuracy,and the predicted vibration state development trend is highly consistent with the actual,with an average absolute error of 0.063.The root mean square error(ERMSE)value of the prediction method is<5%,which reduces the relative error value at least 37% compared with the traditional prediction models.This paper provides a valuable reference for the proactive defence of GIS mechanical failure.
文摘In lung nodules there is a huge variation in structural properties like Shape, Surface Texture. Even the spatial properties vary, where they can be found attached to lung walls, blood vessels in complex non-homogenous lung structures. Moreover, the nodules are of small size at their early stage of development. This poses a serious challenge to develop a Computer aided diagnosis (CAD) system with better false positive reduction. Hence, to reduce the false positives per scan and to deal with the challenges mentioned, this paper proposes a set of three diverse 3D Attention based CNN architectures (3D ACNN) whose predictions on given low dose Volumetric Computed Tomography (CT) scans are fused to achieve more effective and reliable results. Attention mechanism is employed to selectively concentrate/weigh more on nodule specific features and less weight age over other irrelevant features. By using this attention based mechanism in CNN unlike traditional methods there was a significant gain in the classification performance. Contextual dependencies are also taken into account by giving three patches of different sizes surrounding the nodule as input to the ACNN architectures. The system is trained and validated using a publicly available LUNA16 dataset in a 10 fold cross validation approach where a competition performance metric (CPM) score of 0.931 is achieved. The experimental results demonstrate that either a single patch or a single architecture in a one-to-one fashion that is adopted in earlier methods cannot achieve a better performance and signifies the necessity of fusing different multi patched architectures. Though the proposed system is mainly designed for pulmonary nodule detection it can be easily extended to classification tasks of any other 3D medical diagnostic computed tomography images where there is a huge variation and uncertainty in classification.
文摘Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.
