Accurate forecasting of crude oil futures prices is crucial for understanding global energy market dynamics and formulating effective macroeconomic and energy strategies.However,the strong nonlinearity and multi-scale...Accurate forecasting of crude oil futures prices is crucial for understanding global energy market dynamics and formulating effective macroeconomic and energy strategies.However,the strong nonlinearity and multi-scale temporal characteristics of crude oil prices pose significant challenges to traditional forecasting methods.To address these issues,this study proposes a hybrid CEEMDAN–HOA–Transformer–GRU model that integrates decomposition,complexity analysis,adaptive modeling,and intelligent optimization.Specifically,Complete Ensemble Empirical Mode Decomposition with Adaptive Noise(CEEMDAN)is employed to decompose the original series into multi-scale components,after which entropy-based complexity analysis quantitatively evaluates each component.A differentiated modeling strategy is then applied:Transformer networks capture long-term dependencies in high-complexity components,while Gated Recurrent Units(GRU)model short-term dynamics in relatively simple components.To further enhance robustness,the Hiking Optimization Algorithm(HOA)is used for joint hyperparameter optimization across both base learners.Empirical analysis of WTI and Brent crude oil futures demonstrates the technical effectiveness of the framework.Compared with benchmark models,the proposed method reduces RMSE by 79.16% for WTI and 77.47% for Brent.Incorporating complexity analysis further decreases RMSE by 36.51%for WTI and 34.93%for Brent,confirming the superior nonlinear modeling capacity and generalization performance of the integrated framework.Overall,this study provides not only a technically reliable tool for modeling complex financial time series but also practical guidance for improving the accuracy and stability of crude oil price forecasting,thereby supporting market monitoring,risk management,and policy formulation.展开更多
针对地图综合中建筑多边形化简方法依赖人工规则、自动化程度低且难以利用已有化简成果的问题,本文提出了一种基于Transformer机制的建筑多边形化简模型。该模型首先把建筑多边形映射至一定范围的网格空间,将建筑多边形的坐标串表达为...针对地图综合中建筑多边形化简方法依赖人工规则、自动化程度低且难以利用已有化简成果的问题,本文提出了一种基于Transformer机制的建筑多边形化简模型。该模型首先把建筑多边形映射至一定范围的网格空间,将建筑多边形的坐标串表达为网格序列,从而获取建筑多边形化简前后的Token序列,构建出建筑多边形化简样本对数据;随后采用Transformer架构建立模型,基于样本数据利用模型的掩码自注意力机制学习点序列之间的依赖关系,最终逐点生成新的简化多边形,从而实现建筑多边形的化简。在训练过程中,模型使用结构化的样本数据,设计了忽略特定索引的交叉熵损失函数以提升化简质量。试验设计包括主试验与泛化验证两部分。主试验基于洛杉矶1∶2000建筑数据集,分别采用0.2、0.3和0.5 mm 3种网格尺寸对多边形进行编码,实现了目标比例尺为1∶5000与1∶10000的化简。试验结果表明,在0.3 mm的网格尺寸下模型性能最优,验证集上的化简结果与人工标注的一致率超过92.0%,且针对北京部分区域的建筑多边形数据的泛化试验验证了模型的迁移能力;与LSTM模型的对比分析显示,在参数规模相近的条件下,LSTM模型无法形成有效收敛,并生成可用结果。本文证实了Transformer在处理空间几何序列任务中的潜力,且能够有效复用已有化简样本,为智能建筑多边形化简提供了具有工程实用价值的途径。展开更多
Current experimental and computational methods have limitations in accurately and efficiently classifying ion channels within vast protein spaces.Here we have developed a deep learning algorithm,GPT2 Ion Channel Class...Current experimental and computational methods have limitations in accurately and efficiently classifying ion channels within vast protein spaces.Here we have developed a deep learning algorithm,GPT2 Ion Channel Classifier(GPT2-ICC),which effectively distinguishing ion channels from a test set containing approximately 239 times more non-ion-channel proteins.GPT2-ICC integrates representation learning with a large language model(LLM)-based classifier,enabling highly accurate identification of potential ion channels.Several potential ion channels were predicated from the unannotated human proteome,further demonstrating GPT2-ICC’s generalization ability.This study marks a significant advancement in artificial-intelligence-driven ion channel research,highlighting the adaptability and effectiveness of combining representation learning with LLMs to address the challenges of imbalanced protein sequence data.Moreover,it provides a valuable computational tool for uncovering previously uncharacterized ion channels.展开更多
This study introduces an innovative computational framework leveraging the transformer architecture to address a critical challenge in chemical process engineering:predicting and optimizing light olefin yields in indu...This study introduces an innovative computational framework leveraging the transformer architecture to address a critical challenge in chemical process engineering:predicting and optimizing light olefin yields in industrial methanol-to-olefins(MTO)processes.Our approach integrates advanced machine learning techniques with chemical engineering principles to tackle the complexities of non-stationary,highly volatile production data in large-scale chemical manufacturing.The framework employs the maximal information coefficient(MIC)algorithm to analyze and select the significant variables from MTO process parameters,forming a robust dataset for model development.We implement a transformer-based time series forecasting model,enhanced through positional encoding and hyperparameter optimization,significantly improving predictive accuracy for ethylene and propylene yields.The model's interpretability is augmented by applying SHapley additive exPlanations(SHAP)to quantify and visualize the impact of reaction control variables on olefin yields,providing valuable insights for process optimization.Experimental results demonstrate that our model outperforms traditional statistical and machine learning methods in accuracy and interpretability,effectively handling nonlinear,non-stationary,highvolatility,and long-sequence data challenges in olefin yield prediction.This research contributes to chemical engineering by providing a novel computerized methodology for solving complex production optimization problems in the chemical industry,offering significant potential for enhancing decisionmaking in MTO system production control and fostering the intelligent transformation of manufacturing processes.展开更多
Deep learning(DL)has become a crucial technique for predicting the El Niño-Southern Oscillation(ENSO)and evaluating its predictability.While various DL-based models have been developed for ENSO predictions,many f...Deep learning(DL)has become a crucial technique for predicting the El Niño-Southern Oscillation(ENSO)and evaluating its predictability.While various DL-based models have been developed for ENSO predictions,many fail to capture the coherent multivariate evolution within the coupled ocean-atmosphere system of the tropical Pacific.To address this three-dimensional(3D)limitation and represent ENSO-related ocean-atmosphere interactions more accurately,a novel this 3D multivariate prediction model was proposed based on a Transformer architecture,which incorporates a spatiotemporal self-attention mechanism.This model,named 3D-Geoformer,offers several advantages,enabling accurate ENSO predictions up to one and a half years in advance.Furthermore,an integrated gradient method was introduced into the model to identify the sources of predictability for sea surface temperature(SST)variability in the eastern equatorial Pacific.Results reveal that the 3D-Geoformer effectively captures ENSO-related precursors during the evolution of ENSO events,particularly the thermocline feedback processes and ocean temperature anomaly pathways on and off the equator.By extending DL-based ENSO predictions from one-dimensional Niño time series to 3D multivariate fields,the 3D-Geoformer represents a significant advancement in ENSO prediction.This study provides details in the model formulation,analysis procedures,sensitivity experiments,and illustrative examples,offering practical guidance for the application of the model in ENSO research.展开更多
基金funded by the Henan Provincial Natural Science Foundation(grant no.242300421257).
文摘Accurate forecasting of crude oil futures prices is crucial for understanding global energy market dynamics and formulating effective macroeconomic and energy strategies.However,the strong nonlinearity and multi-scale temporal characteristics of crude oil prices pose significant challenges to traditional forecasting methods.To address these issues,this study proposes a hybrid CEEMDAN–HOA–Transformer–GRU model that integrates decomposition,complexity analysis,adaptive modeling,and intelligent optimization.Specifically,Complete Ensemble Empirical Mode Decomposition with Adaptive Noise(CEEMDAN)is employed to decompose the original series into multi-scale components,after which entropy-based complexity analysis quantitatively evaluates each component.A differentiated modeling strategy is then applied:Transformer networks capture long-term dependencies in high-complexity components,while Gated Recurrent Units(GRU)model short-term dynamics in relatively simple components.To further enhance robustness,the Hiking Optimization Algorithm(HOA)is used for joint hyperparameter optimization across both base learners.Empirical analysis of WTI and Brent crude oil futures demonstrates the technical effectiveness of the framework.Compared with benchmark models,the proposed method reduces RMSE by 79.16% for WTI and 77.47% for Brent.Incorporating complexity analysis further decreases RMSE by 36.51%for WTI and 34.93%for Brent,confirming the superior nonlinear modeling capacity and generalization performance of the integrated framework.Overall,this study provides not only a technically reliable tool for modeling complex financial time series but also practical guidance for improving the accuracy and stability of crude oil price forecasting,thereby supporting market monitoring,risk management,and policy formulation.
文摘针对地图综合中建筑多边形化简方法依赖人工规则、自动化程度低且难以利用已有化简成果的问题,本文提出了一种基于Transformer机制的建筑多边形化简模型。该模型首先把建筑多边形映射至一定范围的网格空间,将建筑多边形的坐标串表达为网格序列,从而获取建筑多边形化简前后的Token序列,构建出建筑多边形化简样本对数据;随后采用Transformer架构建立模型,基于样本数据利用模型的掩码自注意力机制学习点序列之间的依赖关系,最终逐点生成新的简化多边形,从而实现建筑多边形的化简。在训练过程中,模型使用结构化的样本数据,设计了忽略特定索引的交叉熵损失函数以提升化简质量。试验设计包括主试验与泛化验证两部分。主试验基于洛杉矶1∶2000建筑数据集,分别采用0.2、0.3和0.5 mm 3种网格尺寸对多边形进行编码,实现了目标比例尺为1∶5000与1∶10000的化简。试验结果表明,在0.3 mm的网格尺寸下模型性能最优,验证集上的化简结果与人工标注的一致率超过92.0%,且针对北京部分区域的建筑多边形数据的泛化试验验证了模型的迁移能力;与LSTM模型的对比分析显示,在参数规模相近的条件下,LSTM模型无法形成有效收敛,并生成可用结果。本文证实了Transformer在处理空间几何序列任务中的潜力,且能够有效复用已有化简样本,为智能建筑多边形化简提供了具有工程实用价值的途径。
基金funded by grants from the National Key Research and Development Program of China(Grant Nos.:2022YFE0205600 and 2022YFC3400504)the National Natural Science Foundation of China(Grant Nos.:82373792 and 82273857)the Fundamental Research Funds for the Central Universities,China,and the East China Normal University Medicine and Health Joint Fund,China(Grant No.:2022JKXYD07001).
文摘Current experimental and computational methods have limitations in accurately and efficiently classifying ion channels within vast protein spaces.Here we have developed a deep learning algorithm,GPT2 Ion Channel Classifier(GPT2-ICC),which effectively distinguishing ion channels from a test set containing approximately 239 times more non-ion-channel proteins.GPT2-ICC integrates representation learning with a large language model(LLM)-based classifier,enabling highly accurate identification of potential ion channels.Several potential ion channels were predicated from the unannotated human proteome,further demonstrating GPT2-ICC’s generalization ability.This study marks a significant advancement in artificial-intelligence-driven ion channel research,highlighting the adaptability and effectiveness of combining representation learning with LLMs to address the challenges of imbalanced protein sequence data.Moreover,it provides a valuable computational tool for uncovering previously uncharacterized ion channels.
基金supported by the Humanities and Social Sciences Foundation of the Ministry of Education(22YJC910011)the China Postdoctoral Science Foundation(2023M733444)the Key Research and Development Program in Artificial Intelligence of Liaoning Province(2023JH26/10200012).
文摘This study introduces an innovative computational framework leveraging the transformer architecture to address a critical challenge in chemical process engineering:predicting and optimizing light olefin yields in industrial methanol-to-olefins(MTO)processes.Our approach integrates advanced machine learning techniques with chemical engineering principles to tackle the complexities of non-stationary,highly volatile production data in large-scale chemical manufacturing.The framework employs the maximal information coefficient(MIC)algorithm to analyze and select the significant variables from MTO process parameters,forming a robust dataset for model development.We implement a transformer-based time series forecasting model,enhanced through positional encoding and hyperparameter optimization,significantly improving predictive accuracy for ethylene and propylene yields.The model's interpretability is augmented by applying SHapley additive exPlanations(SHAP)to quantify and visualize the impact of reaction control variables on olefin yields,providing valuable insights for process optimization.Experimental results demonstrate that our model outperforms traditional statistical and machine learning methods in accuracy and interpretability,effectively handling nonlinear,non-stationary,highvolatility,and long-sequence data challenges in olefin yield prediction.This research contributes to chemical engineering by providing a novel computerized methodology for solving complex production optimization problems in the chemical industry,offering significant potential for enhancing decisionmaking in MTO system production control and fostering the intelligent transformation of manufacturing processes.
基金Supported by the Laoshan Laboratory(No.LSKJ202202402)the National Natural Science Foundation of China(No.42030410)+2 种基金the Startup Foundation for Introducing Talent of Nanjing University of Information Science&Technology,and Jiangsu Innovation Research Group(No.JSSCTD 202346)supported by the China National Postdoctoral Program for Innovative Talents(No.BX20240169)the China Postdoctoral Science Foundation(No.2141062400101)。
文摘Deep learning(DL)has become a crucial technique for predicting the El Niño-Southern Oscillation(ENSO)and evaluating its predictability.While various DL-based models have been developed for ENSO predictions,many fail to capture the coherent multivariate evolution within the coupled ocean-atmosphere system of the tropical Pacific.To address this three-dimensional(3D)limitation and represent ENSO-related ocean-atmosphere interactions more accurately,a novel this 3D multivariate prediction model was proposed based on a Transformer architecture,which incorporates a spatiotemporal self-attention mechanism.This model,named 3D-Geoformer,offers several advantages,enabling accurate ENSO predictions up to one and a half years in advance.Furthermore,an integrated gradient method was introduced into the model to identify the sources of predictability for sea surface temperature(SST)variability in the eastern equatorial Pacific.Results reveal that the 3D-Geoformer effectively captures ENSO-related precursors during the evolution of ENSO events,particularly the thermocline feedback processes and ocean temperature anomaly pathways on and off the equator.By extending DL-based ENSO predictions from one-dimensional Niño time series to 3D multivariate fields,the 3D-Geoformer represents a significant advancement in ENSO prediction.This study provides details in the model formulation,analysis procedures,sensitivity experiments,and illustrative examples,offering practical guidance for the application of the model in ENSO research.
