In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with l...In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.展开更多
This paper presents methods for computing a second-order sensitivity matrix and the Hessian matrix of eigenvalues and eigenvectors of multiple parameter structures. Second-order perturbations of eigenvalues and eigenv...This paper presents methods for computing a second-order sensitivity matrix and the Hessian matrix of eigenvalues and eigenvectors of multiple parameter structures. Second-order perturbations of eigenvalues and eigenvectors are transformed into multiple parameter forms,and the second-order perturbation sensitivity matrices of eigenvalues and eigenvectors are developed.With these formulations,the efficient methods based on the second-order Taylor expansion and second-order perturbation are obtained to estimate changes of eigenvalues and eigenvectors when the design parameters are changed. The presented method avoids direct differential operation,and thus reduces difficulty for computing the second-order sensitivity matrices of eigenpairs.A numerical example is given to demonstrate application and accuracy of the proposed method.展开更多
A numerical model of thermoelectric module （TEM） is created by academic analysis,and the impacts of the resistance ratio and thermoelement size on the output power and thermoelectric efficiency of the TEM are analyz...A numerical model of thermoelectric module （TEM） is created by academic analysis,and the impacts of the resistance ratio and thermoelement size on the output power and thermoelectric efficiency of the TEM are analyzed by the MATLAB numerical calculation.The numerical model is validated by the ANSYS thermal,electrical,and structural coupling simulation.The effects of the variable physical property parameters and contact effect on the output power and thermoelectric efficiency are evaluated,and the concept of aspect ratio optimal domain is proposed,which provides a new design approach for the TEM.展开更多
The velocity profiles and separation efficiency curves of a hydrocyclone were predicted by an Euler-Euler approach using a computational fluid dynamics tool ANSYS-CFX 14.5. The Euler-Euler approach is capable of consi...The velocity profiles and separation efficiency curves of a hydrocyclone were predicted by an Euler-Euler approach using a computational fluid dynamics tool ANSYS-CFX 14.5. The Euler-Euler approach is capable of considering the particle-particle interactions and is appropriate for highly laden liquid-solid mixtures. Pre- dicted results were compared and validated with experi- mental results and showed a considerably good agreement. An increase in the particle cut size with increasing solid concentration of the inlet mixture flow was observed and discussed. In addition to this, the erosion on hydrocyclone walls constructed from stainless steel 410, eroded by sand particles （mainly SiOz）, was predicted with the Euler-La- grange approach. In this approach, the abrasive solid particles were traced in a Lagrangian reference frame as discrete particles. The increases in the input flow velocity, solid concentration, and the particle size have increased the erosion at the upper part of the cylindrical body of the hydrocyclone, where the tangential inlet flow enters the hydrocyclone. The erosion density in the area between the cylindrical to conical body area, in comparison to other parts of the hydrocyclone, also increased considerably. Moreover, it was observed that an increase in the particle shape factor from 0.1 to 1.0 leads to a decrease of almost 70 % in the average erosion density of the hydrocyclone wall surfaces.展开更多
Based on a large number of researches and engineering practices both domestic and overseas, it is shown that the building parameters to be determined during scheme phase can exert a great effect on the building energy...Based on a large number of researches and engineering practices both domestic and overseas, it is shown that the building parameters to be determined during scheme phase can exert a great effect on the building energy consumption. In this paper, through a combination of the popular design method of building parameterization at present and the design goat of energy saving during the scheme phase, the author carries out researches on the design methods and toot development which are applicable to parameterization of building energy saving in this stage. In connection with the characteristics of both modeling process of parameterization and energy saving design, and by means of steady calculation as wetl as simulation, this paper establishes an simplified model to calculate the overall energy consumption of air-conditioning, heating, lighting and equipments, and ultimately gives suggestions on design of scheme for energy saving by optimization with the genetic algorithm （GA）. On the basis of the model, a software platform is developed by computer language QTand openGL interface and is oriented to the design users and sets up the MMI （human-computer interaction） software interface for parameterization of building energy saving, which achieves automatic modeling of parameterization and promotes research on oractical design cases.展开更多
The rapid advancement of Large Language Models(LLMs)has created unprecedented opportunities for industrial automation,process optimization,and decision support systems.As industries seek to leverage LLMs for industria...The rapid advancement of Large Language Models(LLMs)has created unprecedented opportunities for industrial automation,process optimization,and decision support systems.As industries seek to leverage LLMs for industrial tasks,understanding their architecture,deployment strategies,and fine-tuning methods becomes critical.In this review,we aim to summarize the challenges,key technologies,current status,and future directions of LLM in Prognostics and Health Management(PHM).First,this review introduces deep learning for PHM.We begin by analyzing the architectural considerations and deployment strategies for industrial environments,including acceleration techniques and quantization methods that enable efficient operation on resource-constrained industrial hardware.Second,we investigate Parameter Efficient Fine-Tuning(PEFT)techniques that allow industry-specific adaptation without prohibitive computational costs.Multi-modal capabilities extending LLMs beyond text to process sensor data,images,and time-series information are also discussed.Finally,we explore emerging PHM including anomaly detection systems that identify equipment malfunctions,fault diagnosis frameworks that determine root causes,and specialized question-answering systems that empower workers with instant domain expertise.We conclude by identifying key challenges and future research directions for LLM deployment in PHM.This review provides a timely resource for researchers,engineers,and decision-makers navigating the transformative potential of language models in industry 4.0 environments.展开更多
Time series foundation models provide a universal solution for generating forecasts to support optimization problems in energy systems.Those foundation models are typically trained in a prediction-focused manner to ma...Time series foundation models provide a universal solution for generating forecasts to support optimization problems in energy systems.Those foundation models are typically trained in a prediction-focused manner to maximize forecast quality.In contrast,decision-focused learning directly improves the resulting value of the forecast in downstream optimization rather than merely maximizing forecasting quality.The practical integration of forecast values into forecasting models is challenging,particularly when addressing complex applications with diverse instances,such as buildings.This becomes even more complicated when instances possess specific characteristics that require instance-specific,tailored predictions to increase the forecast value.To tackle this challenge,we use decision-focused fine-tuning within time series foundation models to offer a scalable and efficient solution for decision-focused learning applied to the dispatchable feeder optimization problem.To obtain more robust predictions for scarce building data,we use Moirai as a state-of-the-art foundation model,which offers robust and generalized results with few-shot parameter-efficient fine-tuning.Comparing the decision-focused fine-tuned Moirai with a state-of-the-art classical prediction-focused fine-tuning Moirai,we observe an improvement of 9.45%in Average Daily Total Costs.展开更多
1 Introduction.With the rapid development of Large Language Models(LLMs)and various fine-tuning applications[1,2].Parameter-Efficient Fine-Tuning(PEFT)methods have emerged,aiming to reduce the number of learning param...1 Introduction.With the rapid development of Large Language Models(LLMs)and various fine-tuning applications[1,2].Parameter-Efficient Fine-Tuning(PEFT)methods have emerged,aiming to reduce the number of learning parameters when fine-tuning.Among all these approaches,Low-Rank Adaptation(LoRA)[3]has gained significant traction as an effective and efficient technique.展开更多
The convolution operation possesses the characteristic of translation group equivariance. To achieve more group equivariances, rotation group equivariant convolutions(RGEC) are proposed to acquire both translation and...The convolution operation possesses the characteristic of translation group equivariance. To achieve more group equivariances, rotation group equivariant convolutions(RGEC) are proposed to acquire both translation and rotation group equivariances.However, previous work paid more attention to the number of parameters and usually ignored other resource costs. In this paper, we construct our networks without introducing extra resource costs. Specifically, a convolution kernel is rotated to different orientations for feature extractions of multiple channels. Meanwhile, much fewer kernels than previous works are used to ensure that the output channel does not increase. To further enhance the orthogonality of kernels in different orientations, we construct the non-maximum-suppression loss on the rotation dimension to suppress the other directions except the most activated one. Considering that the low-level-features benefit more from the rotational symmetry, we only share weights in the shallow layers(SWSL) via RGEC. Extensive experiments on multiple datasets(i.e., Image Net, CIFAR, and MNIST) demonstrate that SWSL can effectively benefit from the higher-degree weight sharing and improve the performances of various networks, including plain and Res Net architectures. Meanwhile, the convolutional kernels and parameters are much fewer(e.g., 75%, 87.5% fewer) in the shallow layers, and no extra computation costs are introduced.展开更多
Aggregation is one of the many important processes in chemical and process engineering. Several researchers have attempted to understand this complex process in fluidized beds using the macro-model of population balan...Aggregation is one of the many important processes in chemical and process engineering. Several researchers have attempted to understand this complex process in fluidized beds using the macro-model of population balance equations （PBEs）. The aggregation kernel is an effective parameter in PBEs, and is defined as the product of the aggregation efficiency and collision frequency functions. Attempts to derive this kernel have taken different approaches, including theoretical, experimental, and empirical techniques. The present paper calculates the aggregation kernel using micro-model computer simulations, i.e., a discrete particle model. We simulate the micro-model without aggregation for various initial conditions, and observe that the collision frequency function is in good agreement with the shear kernel. We then simulate the micro-model with aggregation and calculate the aggregation efficiency rate.展开更多
基金supported by the National Science and Technology Council(NSTC),Taiwan,under Grants Numbers 112-2622-E-029-009 and 112-2221-E-029-019.
文摘In the domain of knowledge graph embedding,conventional approaches typically transform entities and relations into continuous vector spaces.However,parameter efficiency becomes increasingly crucial when dealing with large-scale knowledge graphs that contain vast numbers of entities and relations.In particular,resource-intensive embeddings often lead to increased computational costs,and may limit scalability and adaptability in practical environ-ments,such as in low-resource settings or real-world applications.This paper explores an approach to knowledge graph representation learning that leverages small,reserved entities and relation sets for parameter-efficient embedding.We introduce a hierarchical attention network designed to refine and maximize the representational quality of embeddings by selectively focusing on these reserved sets,thereby reducing model complexity.Empirical assessments validate that our model achieves high performance on the benchmark dataset with fewer parameters and smaller embedding dimensions.The ablation studies further highlight the impact and contribution of each component in the proposed hierarchical attention structure.
基金Project supported by the 985-Engineering Innovation of Graduate Students of Jilin Universitythe Science and Technology Development Foundation of Jilin Province(20070541)
文摘This paper presents methods for computing a second-order sensitivity matrix and the Hessian matrix of eigenvalues and eigenvectors of multiple parameter structures. Second-order perturbations of eigenvalues and eigenvectors are transformed into multiple parameter forms,and the second-order perturbation sensitivity matrices of eigenvalues and eigenvectors are developed.With these formulations,the efficient methods based on the second-order Taylor expansion and second-order perturbation are obtained to estimate changes of eigenvalues and eigenvectors when the design parameters are changed. The presented method avoids direct differential operation,and thus reduces difficulty for computing the second-order sensitivity matrices of eigenpairs.A numerical example is given to demonstrate application and accuracy of the proposed method.
基金Funded by Guangdong Natural Science Foundation (No.00355991220615019)
文摘A numerical model of thermoelectric module （TEM） is created by academic analysis,and the impacts of the resistance ratio and thermoelement size on the output power and thermoelectric efficiency of the TEM are analyzed by the MATLAB numerical calculation.The numerical model is validated by the ANSYS thermal,electrical,and structural coupling simulation.The effects of the variable physical property parameters and contact effect on the output power and thermoelectric efficiency are evaluated,and the concept of aspect ratio optimal domain is proposed,which provides a new design approach for the TEM.
基金“Stiftung Rheinland-Pfalz fur Innovation,Mainz,Germany,”for financial support
文摘The velocity profiles and separation efficiency curves of a hydrocyclone were predicted by an Euler-Euler approach using a computational fluid dynamics tool ANSYS-CFX 14.5. The Euler-Euler approach is capable of considering the particle-particle interactions and is appropriate for highly laden liquid-solid mixtures. Pre- dicted results were compared and validated with experi- mental results and showed a considerably good agreement. An increase in the particle cut size with increasing solid concentration of the inlet mixture flow was observed and discussed. In addition to this, the erosion on hydrocyclone walls constructed from stainless steel 410, eroded by sand particles （mainly SiOz）, was predicted with the Euler-La- grange approach. In this approach, the abrasive solid particles were traced in a Lagrangian reference frame as discrete particles. The increases in the input flow velocity, solid concentration, and the particle size have increased the erosion at the upper part of the cylindrical body of the hydrocyclone, where the tangential inlet flow enters the hydrocyclone. The erosion density in the area between the cylindrical to conical body area, in comparison to other parts of the hydrocyclone, also increased considerably. Moreover, it was observed that an increase in the particle shape factor from 0.1 to 1.0 leads to a decrease of almost 70 % in the average erosion density of the hydrocyclone wall surfaces.
文摘Based on a large number of researches and engineering practices both domestic and overseas, it is shown that the building parameters to be determined during scheme phase can exert a great effect on the building energy consumption. In this paper, through a combination of the popular design method of building parameterization at present and the design goat of energy saving during the scheme phase, the author carries out researches on the design methods and toot development which are applicable to parameterization of building energy saving in this stage. In connection with the characteristics of both modeling process of parameterization and energy saving design, and by means of steady calculation as wetl as simulation, this paper establishes an simplified model to calculate the overall energy consumption of air-conditioning, heating, lighting and equipments, and ultimately gives suggestions on design of scheme for energy saving by optimization with the genetic algorithm （GA）. On the basis of the model, a software platform is developed by computer language QTand openGL interface and is oriented to the design users and sets up the MMI （human-computer interaction） software interface for parameterization of building energy saving, which achieves automatic modeling of parameterization and promotes research on oractical design cases.
基金supported in part by the National Natural Science Foundation of China under Grant Nos.72171172 and 92367101the Aeronautical Science Foundation of China under Grant No.2023Z066038001+2 种基金the National Natural Science Foundation of China Basic Science Research Center Program under Grant No.62088101Shanghai Municipal Science and Technology Major Project under Grant No.2021SHZDZX0100Chinese Academy of Engineering,Strategic Research and Consulting Program,under Grant No.2023-XZ-65.
文摘The rapid advancement of Large Language Models(LLMs)has created unprecedented opportunities for industrial automation,process optimization,and decision support systems.As industries seek to leverage LLMs for industrial tasks,understanding their architecture,deployment strategies,and fine-tuning methods becomes critical.In this review,we aim to summarize the challenges,key technologies,current status,and future directions of LLM in Prognostics and Health Management(PHM).First,this review introduces deep learning for PHM.We begin by analyzing the architectural considerations and deployment strategies for industrial environments,including acceleration techniques and quantization methods that enable efficient operation on resource-constrained industrial hardware.Second,we investigate Parameter Efficient Fine-Tuning(PEFT)techniques that allow industry-specific adaptation without prohibitive computational costs.Multi-modal capabilities extending LLMs beyond text to process sensor data,images,and time-series information are also discussed.Finally,we explore emerging PHM including anomaly detection systems that identify equipment malfunctions,fault diagnosis frameworks that determine root causes,and specialized question-answering systems that empower workers with instant domain expertise.We conclude by identifying key challenges and future research directions for LLM deployment in PHM.This review provides a timely resource for researchers,engineers,and decision-makers navigating the transformative potential of language models in industry 4.0 environments.
基金funded by the Helmholtz Association’s Initiative and Networking Fund through Helmholtz AI,the Helmholtz Association under the Program“Energy System Design”the German Research Foundation(DFG)as part of the Research Training Group 2153“En-ergy Status Data:Informatics Methods for its Collection,Analysis and Exploitation”+1 种基金supported by the Helmholtz Association Initiative and Networking Fund on the HAICORE@KIT partitionsupport by the KIT-Publication Fund of the Karlsruhe Institute of Technology.
文摘Time series foundation models provide a universal solution for generating forecasts to support optimization problems in energy systems.Those foundation models are typically trained in a prediction-focused manner to maximize forecast quality.In contrast,decision-focused learning directly improves the resulting value of the forecast in downstream optimization rather than merely maximizing forecasting quality.The practical integration of forecast values into forecasting models is challenging,particularly when addressing complex applications with diverse instances,such as buildings.This becomes even more complicated when instances possess specific characteristics that require instance-specific,tailored predictions to increase the forecast value.To tackle this challenge,we use decision-focused fine-tuning within time series foundation models to offer a scalable and efficient solution for decision-focused learning applied to the dispatchable feeder optimization problem.To obtain more robust predictions for scarce building data,we use Moirai as a state-of-the-art foundation model,which offers robust and generalized results with few-shot parameter-efficient fine-tuning.Comparing the decision-focused fine-tuned Moirai with a state-of-the-art classical prediction-focused fine-tuning Moirai,we observe an improvement of 9.45%in Average Daily Total Costs.
基金partially supported by the National Science and Technology Major Project(Grant No.2023ZD0121103)the National Natural Science Foundation of China(Grant Nos.62376086,U23B2031).
文摘1 Introduction.With the rapid development of Large Language Models(LLMs)and various fine-tuning applications[1,2].Parameter-Efficient Fine-Tuning(PEFT)methods have emerged,aiming to reduce the number of learning parameters when fine-tuning.Among all these approaches,Low-Rank Adaptation(LoRA)[3]has gained significant traction as an effective and efficient technique.
基金supported by National Natural Science Foundation of China(Nos.61976209 and 62020106015)CAS International Collaboration Key Project(No.173211KYSB20190024)Strategic Priority Research Program of CAS(No.XDB32040000)。
文摘The convolution operation possesses the characteristic of translation group equivariance. To achieve more group equivariances, rotation group equivariant convolutions(RGEC) are proposed to acquire both translation and rotation group equivariances.However, previous work paid more attention to the number of parameters and usually ignored other resource costs. In this paper, we construct our networks without introducing extra resource costs. Specifically, a convolution kernel is rotated to different orientations for feature extractions of multiple channels. Meanwhile, much fewer kernels than previous works are used to ensure that the output channel does not increase. To further enhance the orthogonality of kernels in different orientations, we construct the non-maximum-suppression loss on the rotation dimension to suppress the other directions except the most activated one. Considering that the low-level-features benefit more from the rotational symmetry, we only share weights in the shallow layers(SWSL) via RGEC. Extensive experiments on multiple datasets(i.e., Image Net, CIFAR, and MNIST) demonstrate that SWSL can effectively benefit from the higher-degree weight sharing and improve the performances of various networks, including plain and Res Net architectures. Meanwhile, the convolutional kernels and parameters are much fewer(e.g., 75%, 87.5% fewer) in the shallow layers, and no extra computation costs are introduced.
基金supported by the Graduiertenkolleg-828,"Micro-Macro-Interactions in Structured Media and Particles Systems",Otto-von-Guericke-University Magdeburg
文摘Aggregation is one of the many important processes in chemical and process engineering. Several researchers have attempted to understand this complex process in fluidized beds using the macro-model of population balance equations （PBEs）. The aggregation kernel is an effective parameter in PBEs, and is defined as the product of the aggregation efficiency and collision frequency functions. Attempts to derive this kernel have taken different approaches, including theoretical, experimental, and empirical techniques. The present paper calculates the aggregation kernel using micro-model computer simulations, i.e., a discrete particle model. We simulate the micro-model without aggregation for various initial conditions, and observe that the collision frequency function is in good agreement with the shear kernel. We then simulate the micro-model with aggregation and calculate the aggregation efficiency rate.
