The widespread usage of rechargeable batteries in portable devices,electric vehicles,and energy storage systems has underscored the importance for accurately predicting their lifetimes.However,data scarcity often limi...The widespread usage of rechargeable batteries in portable devices,electric vehicles,and energy storage systems has underscored the importance for accurately predicting their lifetimes.However,data scarcity often limits the accuracy of prediction models,which is escalated by the incompletion of data induced by the issues such as sensor failures.To address these challenges,we propose a novel approach to accommodate data insufficiency through achieving external information from incomplete data samples,which are usually discarded in existing studies.In order to fully unleash the prediction power of incomplete data,we have investigated the Multiple Imputation by Chained Equations(MICE)method that diversifies the training data through exploring the potential data patterns.The experimental results demonstrate that the proposed method significantly outperforms the baselines in the most considered scenarios while reducing the prediction root mean square error(RMSE)by up to 18.9%.Furthermore,we have also observed that the penetration of incomplete data benefits the explainability of the prediction model through facilitating the feature selection.展开更多
Missing instances and mixed data types,including discrete and ordered(e.g.,continuous and ordinal)variables,are widespread in many datasets in the finance sector.In this domain,estimating missing instances is crucial ...Missing instances and mixed data types,including discrete and ordered(e.g.,continuous and ordinal)variables,are widespread in many datasets in the finance sector.In this domain,estimating missing instances is crucial because many data analysis pipelines require complete data,which is particularly challenging for mixed-type data.However,existing methods treat discrete and ordinal data as continuous values,which may reduce efficacy in addressing these challenges.To fill this gap,this study proposes a probabilistic imputation method for mixed-type and incomplete loan data(PMILD),using a mixed Gaussian Copula model that supports single and multiple imputations.The method models mixed discrete and ordinal data using latent Gaussian distributions,where observed features with arbitrary margins are mapped to the latent normal space,and feature correlations are approximated through the expectation-maximization process in the latent space.Empirical results on nine real-world datasets demonstrate that PMILD substantially outperforms state-of-the-art imputation methods,providing a highly effective solution for handling mixed-type and incomplete loan data.This advancement enhances both operational efficiency and credit evaluation accuracy in finance-related applications.展开更多
Data obtained from accelerated life testing(ALT)when there are two or more failure modes,which is commonly referred to as competing failure modes,are often incomplete.The incompleteness is mainly due to censoring,as w...Data obtained from accelerated life testing(ALT)when there are two or more failure modes,which is commonly referred to as competing failure modes,are often incomplete.The incompleteness is mainly due to censoring,as well as masking which might be the case that the failure time is observed,but its corresponding failure mode is not identified.Because the identification of the failure mode may be expensive,or very difficult to investigate due to lack of appropriate diagnostics.A method is proposed for analyzing incomplete data of constant stress ALT with competing failure modes.It is assumed that failure modes have s-independent latent lifetimes and the log lifetime of each failure mode can be written as a linear function of stress.The parameters of the model are estimated by using the expectation maximum(EM)algorithm with incomplete data.Simulation studies are performed to check'model validity and investigate the properties of estimates.For further validation,the method is also illustrated by an example,which shows the process of analyze incomplete data from ALT of some insulation system.Because of considering the incompleteness of data in modeling and making use of the EM algorithm in estimating,the method becomes more flexible in ALT analysis.展开更多
Energy consumption prediction of a CNC machining process is important for energy efficiency optimization strategies.To improve the generalization abilities,more and more parameters are acquired for energy prediction m...Energy consumption prediction of a CNC machining process is important for energy efficiency optimization strategies.To improve the generalization abilities,more and more parameters are acquired for energy prediction modeling.While the data collected from workshops may be incomplete because of misoperation,unstable network connections,and frequent transfers,etc.This work proposes a framework for energy modeling based on incomplete data to address this issue.First,some necessary preliminary operations are used for incomplete data sets.Then,missing values are estimated to generate a new complete data set based on generative adversarial imputation nets(GAIN).Next,the gene expression programming(GEP)algorithm is utilized to train the energy model based on the generated data sets.Finally,we test the predictive accuracy of the obtained model.Computational experiments are designed to investigate the performance of the proposed framework with different rates of missing data.Experimental results demonstrate that even when the missing data rate increases to 30%,the proposed framework can still make efficient predictions,with the corresponding RMSE and MAE 0.903 k J and 0.739 k J,respectively.展开更多
For the fault detection and diagnosis problem in largescale industrial systems, there are two important issues： the missing data samples and the non-Gaussian property of the data. However, most of the existing data-d...For the fault detection and diagnosis problem in largescale industrial systems, there are two important issues： the missing data samples and the non-Gaussian property of the data. However, most of the existing data-driven methods cannot be able to handle both of them. Thus, a new Bayesian network classifier based fault detection and diagnosis method is proposed. At first, a non-imputation method is presented to handle the data incomplete samples, with the property of the proposed Bayesian network classifier, and the missing values can be marginalized in an elegant manner. Furthermore, the Gaussian mixture model is used to approximate the non-Gaussian data with a linear combination of finite Gaussian mixtures, so that the Bayesian network can process the non-Gaussian data in an effective way. Therefore, the entire fault detection and diagnosis method can deal with the high-dimensional incomplete process samples in an efficient and robust way. The diagnosis results are expressed in the manner of probability with the reliability scores. The proposed approach is evaluated with a benchmark problem called the Tennessee Eastman process. The simulation results show the effectiveness and robustness of the proposed method in fault detection and diagnosis for large-scale systems with missing measurements.展开更多
In modern industrial processes, timely detection and diagnosis of process abnormalities are critical for monitoring process operations. Various fault detection and diagnosis(FDD) methods have been proposed and impleme...In modern industrial processes, timely detection and diagnosis of process abnormalities are critical for monitoring process operations. Various fault detection and diagnosis(FDD) methods have been proposed and implemented, the performance of which, however, could be drastically influenced by the common presence of incomplete or missing data in real industrial scenarios. This paper presents a new FDD approach based on an incomplete data imputation technique for process fault recognition. It employs the modified stacked autoencoder,a deep learning structure, in the phase of incomplete data treatment, and classifies data representations rather than the imputed complete data in the phase of fault identification. A benchmark process, the Tennessee Eastman process, is employed to illustrate the effectiveness and applicability of the proposed method.展开更多
Due to the simplicity and flexibility of the power law process,it is widely used to model the failures of repairable systems.Although statistical inference on the parameters of the power law process has been well deve...Due to the simplicity and flexibility of the power law process,it is widely used to model the failures of repairable systems.Although statistical inference on the parameters of the power law process has been well developed,numerous studies largely depend on complete failure data.A few methods on incomplete data are reported to process such data,but they are limited to their specific cases,especially to that where missing data occur at the early stage of the failures.No framework to handle generic scenarios is available.To overcome this problem,from the point of view of order statistics,the statistical inference of the power law process with incomplete data is established in this paper.The theoretical derivation is carried out and the case studies demonstrate and verify the proposed method.Order statistics offer an alternative to the statistical inference of the power law process with incomplete data as they can reformulate current studies on the left censored failure data and interval censored data in a unified framework.The results show that the proposed method has more flexibility and more applicability.展开更多
Due to the high inherent uncertainty of renewable energy,probabilistic day-ahead wind power forecasting is crucial for modeling and controlling the uncertainty of renewable energy smart grids in smart cities.However,t...Due to the high inherent uncertainty of renewable energy,probabilistic day-ahead wind power forecasting is crucial for modeling and controlling the uncertainty of renewable energy smart grids in smart cities.However,the accuracy and reliability of high-resolution day-ahead wind power forecasting are constrained by unreliable local weather prediction and incomplete power generation data.This article proposes a physics-informed artificial intelligence(AI)surrogates method to augment the incomplete dataset and quantify its uncertainty to improve wind power forecasting performance.The incomplete dataset,built with numerical weather prediction data,historical wind power generation,and weather factors data,is augmented based on generative adversarial networks.After augmentation,the enriched data is then fed into a multiple AI surrogates model constructed by two extreme learning machine networks to train the forecasting model for wind power.Therefore,the forecasting models’accuracy and generalization ability are improved by mining the implicit physics information from the incomplete dataset.An incomplete dataset gathered from a wind farm in North China,containing only 15 days of weather and wind power generation data withmissing points caused by occasional shutdowns,is utilized to verify the proposed method’s performance.Compared with other probabilistic forecastingmethods,the proposed method shows better accuracy and probabilistic performance on the same incomplete dataset,which highlights its potential for more flexible and sensitive maintenance of smart grids in smart cities.展开更多
Multiple kernel clustering is an unsupervised data analysis method that has been used in various scenarios where data is easy to be collected but hard to be labeled.However,multiple kernel clustering for incomplete da...Multiple kernel clustering is an unsupervised data analysis method that has been used in various scenarios where data is easy to be collected but hard to be labeled.However,multiple kernel clustering for incomplete data is a critical yet challenging task.Although the existing absent multiple kernel clustering methods have achieved remarkable performance on this task,they may fail when data has a high value-missing rate,and they may easily fall into a local optimum.To address these problems,in this paper,we propose an absent multiple kernel clustering(AMKC)method on incomplete data.The AMKC method rst clusters the initialized incomplete data.Then,it constructs a new multiple-kernel-based data space,referred to as K-space,from multiple sources to learn kernel combination coefcients.Finally,it seamlessly integrates an incomplete-kernel-imputation objective,a multiple-kernel-learning objective,and a kernel-clustering objective in order to achieve absent multiple kernel clustering.The three stages in this process are carried out simultaneously until the convergence condition is met.Experiments on six datasets with various characteristics demonstrate that the kernel imputation and clustering performance of the proposed method is signicantly better than state-of-the-art competitors.Meanwhile,the proposed method gains fast convergence speed.展开更多
Data with missing values,or incomplete information,brings some challenges to the development of classification,as the incompleteness may significantly affect the performance of classifiers.In this paper,we handle miss...Data with missing values,or incomplete information,brings some challenges to the development of classification,as the incompleteness may significantly affect the performance of classifiers.In this paper,we handle missing values in both training and test sets with uncertainty and imprecision reasoning by proposing a new belief combination of classifier(BCC)method based on the evidence theory.The proposed BCC method aims to improve the classification performance of incomplete data by characterizing the uncertainty and imprecision brought by incompleteness.In BCC,different attributes are regarded as independent sources,and the collection of each attribute is considered as a subset.Then,multiple classifiers are trained with each subset independently and allow each observed attribute to provide a sub-classification result for the query pattern.Finally,these sub-classification results with different weights(discounting factors)are used to provide supplementary information to jointly determine the final classes of query patterns.The weights consist of two aspects:global and local.The global weight calculated by an optimization function is employed to represent the reliability of each classifier,and the local weight obtained by mining attribute distribution characteristics is used to quantify the importance of observed attributes to the pattern classification.Abundant comparative experiments including seven methods on twelve datasets are executed,demonstrating the out-performance of BCC over all baseline methods in terms of accuracy,precision,recall,F1 measure,with pertinent computational costs.展开更多
In numerous real-world healthcare applications,handling incomplete medical data poses significant challenges for missing value imputation and subsequent clustering or classification tasks.Traditional approaches often ...In numerous real-world healthcare applications,handling incomplete medical data poses significant challenges for missing value imputation and subsequent clustering or classification tasks.Traditional approaches often rely on statistical methods for imputation,which may yield suboptimal results and be computationally intensive.This paper aims to integrate imputation and clustering techniques to enhance the classification of incomplete medical data with improved accuracy.Conventional classification methods are ill-suited for incomplete medical data.To enhance efficiency without compromising accuracy,this paper introduces a novel approach that combines imputation and clustering for the classification of incomplete data.Initially,the linear interpolation imputation method alongside an iterative Fuzzy c-means clustering method is applied and followed by a classification algorithm.The effectiveness of the proposed approach is evaluated using multiple performance metrics,including accuracy,precision,specificity,and sensitivity.The encouraging results demonstrate that our proposed method surpasses classical approaches across various performance criteria.展开更多
With the rapid development of the economy,the scale of the power grid is expanding.The number of power equipment that constitutes the power grid has been very large,which makes the state data of power equipment grow e...With the rapid development of the economy,the scale of the power grid is expanding.The number of power equipment that constitutes the power grid has been very large,which makes the state data of power equipment grow explosively.These multi-source heterogeneous data have data differences,which lead to data variation in the process of transmission and preservation,thus forming the bad information of incomplete data.Therefore,the research on data integrity has become an urgent task.This paper is based on the characteristics of random chance and the Spatio-temporal difference of the system.According to the characteristics and data sources of the massive data generated by power equipment,the fuzzy mining model of power equipment data is established,and the data is divided into numerical and non-numerical data based on numerical data.Take the text data of power equipment defects as the mining material.Then,the Apriori algorithm based on an array is used to mine deeply.The strong association rules in incomplete data of power equipment are obtained and analyzed.From the change trend of NRMSE metrics and classification accuracy,most of the filling methods combined with the two frameworks in this method usually show a relatively stable filling trend,and will not fluctuate greatly with the growth of the missing rate.The experimental results show that the proposed algorithm model can effectively improve the filling effect of the existing filling methods on most data sets,and the filling effect fluctuates greatly with the increase of the missing rate,that is,with the increase of the missing rate,the improvement effect of the model for the existing filling methods is higher than 4.3%.Through the incomplete data clustering technology studied in this paper,a more innovative state assessment of smart grid reliability operation is carried out,which has good research value and reference significance.展开更多
A generalized flexibility–based objective function utilized for structure damage identification is constructed for solving the constrained nonlinear least squares optimized problem. To begin with, the generalized fle...A generalized flexibility–based objective function utilized for structure damage identification is constructed for solving the constrained nonlinear least squares optimized problem. To begin with, the generalized flexibility matrix (GFM) proposed to solve the damage identification problem is recalled and a modal expansion method is introduced. Next, the objective function for iterative optimization process based on the GFM is formulated, and the Trust-Region algorithm is utilized to obtain the solution of the optimization problem for multiple damage cases. And then for computing the objective function gradient, the sensitivity analysis regarding design variables is derived. In addition, due to the spatial incompleteness, the influence of stiffness reduction and incomplete modal measurement data is discussed by means of two numerical examples with several damage cases. Finally, based on the computational results, it is evident that the presented approach provides good validity and reliability for the large and complicated engineering structures.展开更多
High-dimensional and incomplete(HDI) matrices are primarily generated in all kinds of big-data-related practical applications. A latent factor analysis(LFA) model is capable of conducting efficient representation lear...High-dimensional and incomplete(HDI) matrices are primarily generated in all kinds of big-data-related practical applications. A latent factor analysis(LFA) model is capable of conducting efficient representation learning to an HDI matrix,whose hyper-parameter adaptation can be implemented through a particle swarm optimizer(PSO) to meet scalable requirements.However, conventional PSO is limited by its premature issues,which leads to the accuracy loss of a resultant LFA model. To address this thorny issue, this study merges the information of each particle's state migration into its evolution process following the principle of a generalized momentum method for improving its search ability, thereby building a state-migration particle swarm optimizer(SPSO), whose theoretical convergence is rigorously proved in this study. It is then incorporated into an LFA model for implementing efficient hyper-parameter adaptation without accuracy loss. Experiments on six HDI matrices indicate that an SPSO-incorporated LFA model outperforms state-of-the-art LFA models in terms of prediction accuracy for missing data of an HDI matrix with competitive computational efficiency.Hence, SPSO's use ensures efficient and reliable hyper-parameter adaptation in an LFA model, thus ensuring practicality and accurate representation learning for HDI matrices.展开更多
The accurate identification and diagnosis of chemical process faults are crucial for ensuring the safe and stable operation of production plants.The current hot topic in industrial process fault diagnosis research is ...The accurate identification and diagnosis of chemical process faults are crucial for ensuring the safe and stable operation of production plants.The current hot topic in industrial process fault diagnosis research is data-driven methods.Most of the existing fault diagnosis methods focus on a single shallow or deep learning model.This paper proposes a novel hybrid fault diagnosis method to fully utilize various features to improve the accuracy of fault diagnosis.Furthermore,the method addresses the issue of incomplete data,which has been largely overlooked in the majority of existing research.Firstly,the variable data is effectively fitted using orthogonal non-negative matrix tri-factorization,and the missing data in the matrix is solved to construct a complete production condition relationship.Next,the support vector machine model and the deep residual contraction network model are trained in parallel to prediagnose process faults by mining linear and non-linear interaction features.Finally,a novel mapping relationship is established between the result and model levels using the multi-layer perceptron algorithm to complete the final diagnosis and evaluation of the fault.To demonstrate the effectiveness of the proposed method,we conducted extensive comparative experiments on the Tennessee Eastman dataset and the ethylene plant cracking unit dataset.The experimental results show that the method has advantages in different evaluation metrics.展开更多
In the context of intelligent manufacturing,the modern hot strip mill process(HSMP)shows characteristics such as diversification of products,multi-specification batch production,and demand-oriented customization.These...In the context of intelligent manufacturing,the modern hot strip mill process(HSMP)shows characteristics such as diversification of products,multi-specification batch production,and demand-oriented customization.These characteristics pose significant challenges to ensuring process stability and consistency of product performance.Therefore,exploring the potential relationship between product performance and the production process,and developing a comprehensive performance evaluation method adapted to modern HSMP have become an urgent issue.A comprehensive performance evaluation method for HSMP by integrating multi-task learning and stacked performance-related autoencoder is proposed to solve the problems such as incomplete performance indicators(PIs)data,insufficient real-time acquisition requirements,and coupling of multiple PIs.First,according to the existing Chinese standards,a comprehensive performance evaluation grade strategy for strip steel is designed.The random forest model is established to predict and complete the parts of PIs data that could not be obtained in real-time.Second,a stacked performance-related autoencoder(SPAE)model is proposed to extract the deep features closely related to the product performance.Then,considering the correlation between PIs,the multi-task learning framework is introduced to output the subitem ratings and comprehensive product performance rating results of the strip steel online in real-time,where each task represents a subitem of comprehensive performance.Finally,the effectiveness of the method is verified on a real HSMP dataset,and the results show that the accuracy of the proposed method is as high as 94.8%,which is superior to the other comparative methods.展开更多
A non-negative latent factor(NLF)model is able to be built efficiently via a single latent factor-dependent,non-negative and multiplicative update(SLF-NMU)algorithm for performing precise representation to high-dimens...A non-negative latent factor(NLF)model is able to be built efficiently via a single latent factor-dependent,non-negative and multiplicative update(SLF-NMU)algorithm for performing precise representation to high-dimensional and incomplete(HDI)matrix from many kinds of big-data-related applications.However,an SLF-NMU algorithm updates a latent factor relying on the current update increment only without considering past learning information,making a resultant model suffer from slow convergence.To address this issue,this study proposes a proportional integral(PI)controller-enhanced NLF(PI-NLF)model with two-fold ideas:1)Designing an increment refinement(IR)mechanism,which formulates the current and past update increments as the proportional and integral terms of a PI controller,thereby assimilating the past update information into the learning scheme smoothly with high efficiency;2)Deriving an IR-based SLF-NMU(ISN)algorithm,which updates a latent factor following the principle of an IR mechanism,thus significantly accelerating an NLF model's convergence rate.The simulation results on eight HDI matrices collected by real applications validate that a PI-NLF model outstrips several leading-edge models in both computational efficiency and accuracy when estimating missing data within an HDI matrix.The proposed PI-NLF model can be effectively applied to applications involving HDI matrix like e-commerce system,social network,and cloud service system.The code is available at https://github.com/yuanyeswu/PINLF/blob/mainIPINLF-code.zip.展开更多
Radar anti-jamming performance evaluation is a necessary link in the process of radar development,introduction and equipment. The applications of generalized rough set theory are proposed and discussed in this paper t...Radar anti-jamming performance evaluation is a necessary link in the process of radar development,introduction and equipment. The applications of generalized rough set theory are proposed and discussed in this paper to address the problems of big data, incomplete data and redundant data in the construction of evaluation index system. Firstly, a mass of real-valued data is converted to some interval-valued data to avoid an unacceptable number of equivalence classes and classification rules, and the interval similarity relation is employed to make classifications of this interval-valued data. Meanwhile, incomplete data can be solved by a new definition of the connection degree tolerance relation for both interval-valued data and single-valued data, which makes a better description of rough set than the traditional limited tolerance relation. Then, E-condition entropy-based heuristic algorithm is applied to making attribute reduction to optimize the evaluation index system, and final decision rules can be extracted for system evaluation. Finally, the feasibility and advantage of the proposed methods are testified by a real example of radar anti-jamming performance evaluation.展开更多
The accurate mathematical models for complicated structures are very difficult to construct.The work presented here provides an identification method for estimating the mass.damping,and stiffness matrices of linear dy...The accurate mathematical models for complicated structures are very difficult to construct.The work presented here provides an identification method for estimating the mass.damping,and stiffness matrices of linear dynamical systems from incomplete experimental data.The mass,stiffness and damping matrices are assumed to be real,symmetric,and positive definite The partial set of experimental complex eigenvalues and corresponding eigenvectors are given.In the proposed method the least squares algorithm is combined with the iteration technique to determine systems identified matrices and corresponding design parameters.Seeveral illustative examples,are presented to demonstrate the reliability of the proposed method .It is emphasized that the mass,damping and stiffness matrices can be identified simultaneously.展开更多
Incomplete data accompanies our life processes and covers almost all fields of scientific studies, as a result of delivery failure, no power of battery, accidental loss, etc. However, how to model, index, and query in...Incomplete data accompanies our life processes and covers almost all fields of scientific studies, as a result of delivery failure, no power of battery, accidental loss, etc. However, how to model, index, and query incomplete data in- curs big challenges. For example, the queries struggling with incomplete data usually have dissatisfying query results due to the improper incompleteness handling methods. In this pa- per, we systematically review the management of incomplete data, including modelling, indexing, querying, and handling methods in terms of incomplete data. We also overview sev- eral application scenarios of incomplete data, and summa- rize the existing systems related to incomplete data. It is our hope that this survey could provide insights to the database community on how incomplete data is managed, and inspire database researchers to develop more advanced processing techniques and tools to cope with the issues resulting from incomplete data in the real world.展开更多
文摘The widespread usage of rechargeable batteries in portable devices,electric vehicles,and energy storage systems has underscored the importance for accurately predicting their lifetimes.However,data scarcity often limits the accuracy of prediction models,which is escalated by the incompletion of data induced by the issues such as sensor failures.To address these challenges,we propose a novel approach to accommodate data insufficiency through achieving external information from incomplete data samples,which are usually discarded in existing studies.In order to fully unleash the prediction power of incomplete data,we have investigated the Multiple Imputation by Chained Equations(MICE)method that diversifies the training data through exploring the potential data patterns.The experimental results demonstrate that the proposed method significantly outperforms the baselines in the most considered scenarios while reducing the prediction root mean square error(RMSE)by up to 18.9%.Furthermore,we have also observed that the penetration of incomplete data benefits the explainability of the prediction model through facilitating the feature selection.
基金supported by Guangdong Provincial Science and Technology Plan Project(Grant No.2021B1111600001)the National Natural Science Foundation of China(Grant Nos.62272198,62276277)+1 种基金the Outstanding Innovative Talents Cultivation Funded Programs for Doctoral Students at Jinan University(Grant No.2023 CXB022)Guangdong Basic and Applied Basic Research Foundation(Grant No.2024A1515010121).
文摘Missing instances and mixed data types,including discrete and ordered(e.g.,continuous and ordinal)variables,are widespread in many datasets in the finance sector.In this domain,estimating missing instances is crucial because many data analysis pipelines require complete data,which is particularly challenging for mixed-type data.However,existing methods treat discrete and ordinal data as continuous values,which may reduce efficacy in addressing these challenges.To fill this gap,this study proposes a probabilistic imputation method for mixed-type and incomplete loan data(PMILD),using a mixed Gaussian Copula model that supports single and multiple imputations.The method models mixed discrete and ordinal data using latent Gaussian distributions,where observed features with arbitrary margins are mapped to the latent normal space,and feature correlations are approximated through the expectation-maximization process in the latent space.Empirical results on nine real-world datasets demonstrate that PMILD substantially outperforms state-of-the-art imputation methods,providing a highly effective solution for handling mixed-type and incomplete loan data.This advancement enhances both operational efficiency and credit evaluation accuracy in finance-related applications.
基金supported by Sustentation Program of National Ministries and Commissions of China(Grant No.203020102)
文摘Data obtained from accelerated life testing(ALT)when there are two or more failure modes,which is commonly referred to as competing failure modes,are often incomplete.The incompleteness is mainly due to censoring,as well as masking which might be the case that the failure time is observed,but its corresponding failure mode is not identified.Because the identification of the failure mode may be expensive,or very difficult to investigate due to lack of appropriate diagnostics.A method is proposed for analyzing incomplete data of constant stress ALT with competing failure modes.It is assumed that failure modes have s-independent latent lifetimes and the log lifetime of each failure mode can be written as a linear function of stress.The parameters of the model are estimated by using the expectation maximum(EM)algorithm with incomplete data.Simulation studies are performed to check'model validity and investigate the properties of estimates.For further validation,the method is also illustrated by an example,which shows the process of analyze incomplete data from ALT of some insulation system.Because of considering the incompleteness of data in modeling and making use of the EM algorithm in estimating,the method becomes more flexible in ALT analysis.
基金supported in part by the National Natural Science Foundation of China(51975075)Chongqing Technology Innovation and Application Program(cstc2018jszx-cyzd X0183)。
文摘Energy consumption prediction of a CNC machining process is important for energy efficiency optimization strategies.To improve the generalization abilities,more and more parameters are acquired for energy prediction modeling.While the data collected from workshops may be incomplete because of misoperation,unstable network connections,and frequent transfers,etc.This work proposes a framework for energy modeling based on incomplete data to address this issue.First,some necessary preliminary operations are used for incomplete data sets.Then,missing values are estimated to generate a new complete data set based on generative adversarial imputation nets(GAIN).Next,the gene expression programming(GEP)algorithm is utilized to train the energy model based on the generated data sets.Finally,we test the predictive accuracy of the obtained model.Computational experiments are designed to investigate the performance of the proposed framework with different rates of missing data.Experimental results demonstrate that even when the missing data rate increases to 30%,the proposed framework can still make efficient predictions,with the corresponding RMSE and MAE 0.903 k J and 0.739 k J,respectively.
基金supported by the National Natural Science Foundation of China(61202473)the Fundamental Research Funds for Central Universities(JUSRP111A49)+1 种基金"111 Project"(B12018)the Priority Academic Program Development of Jiangsu Higher Education Institutions
文摘For the fault detection and diagnosis problem in largescale industrial systems, there are two important issues： the missing data samples and the non-Gaussian property of the data. However, most of the existing data-driven methods cannot be able to handle both of them. Thus, a new Bayesian network classifier based fault detection and diagnosis method is proposed. At first, a non-imputation method is presented to handle the data incomplete samples, with the property of the proposed Bayesian network classifier, and the missing values can be marginalized in an elegant manner. Furthermore, the Gaussian mixture model is used to approximate the non-Gaussian data with a linear combination of finite Gaussian mixtures, so that the Bayesian network can process the non-Gaussian data in an effective way. Therefore, the entire fault detection and diagnosis method can deal with the high-dimensional incomplete process samples in an efficient and robust way. The diagnosis results are expressed in the manner of probability with the reliability scores. The proposed approach is evaluated with a benchmark problem called the Tennessee Eastman process. The simulation results show the effectiveness and robustness of the proposed method in fault detection and diagnosis for large-scale systems with missing measurements.
基金supported by the National Natural Science Foundation of China(61433001)Tsinghua University Initiative Scientific Research Program。
文摘In modern industrial processes, timely detection and diagnosis of process abnormalities are critical for monitoring process operations. Various fault detection and diagnosis(FDD) methods have been proposed and implemented, the performance of which, however, could be drastically influenced by the common presence of incomplete or missing data in real industrial scenarios. This paper presents a new FDD approach based on an incomplete data imputation technique for process fault recognition. It employs the modified stacked autoencoder,a deep learning structure, in the phase of incomplete data treatment, and classifies data representations rather than the imputed complete data in the phase of fault identification. A benchmark process, the Tennessee Eastman process, is employed to illustrate the effectiveness and applicability of the proposed method.
基金supported by the National Natural Science Foundation of China(51775090)。
文摘Due to the simplicity and flexibility of the power law process,it is widely used to model the failures of repairable systems.Although statistical inference on the parameters of the power law process has been well developed,numerous studies largely depend on complete failure data.A few methods on incomplete data are reported to process such data,but they are limited to their specific cases,especially to that where missing data occur at the early stage of the failures.No framework to handle generic scenarios is available.To overcome this problem,from the point of view of order statistics,the statistical inference of the power law process with incomplete data is established in this paper.The theoretical derivation is carried out and the case studies demonstrate and verify the proposed method.Order statistics offer an alternative to the statistical inference of the power law process with incomplete data as they can reformulate current studies on the left censored failure data and interval censored data in a unified framework.The results show that the proposed method has more flexibility and more applicability.
基金funded by the National Natural Science Foundation of China under Grant 62273022.
文摘Due to the high inherent uncertainty of renewable energy,probabilistic day-ahead wind power forecasting is crucial for modeling and controlling the uncertainty of renewable energy smart grids in smart cities.However,the accuracy and reliability of high-resolution day-ahead wind power forecasting are constrained by unreliable local weather prediction and incomplete power generation data.This article proposes a physics-informed artificial intelligence(AI)surrogates method to augment the incomplete dataset and quantify its uncertainty to improve wind power forecasting performance.The incomplete dataset,built with numerical weather prediction data,historical wind power generation,and weather factors data,is augmented based on generative adversarial networks.After augmentation,the enriched data is then fed into a multiple AI surrogates model constructed by two extreme learning machine networks to train the forecasting model for wind power.Therefore,the forecasting models’accuracy and generalization ability are improved by mining the implicit physics information from the incomplete dataset.An incomplete dataset gathered from a wind farm in North China,containing only 15 days of weather and wind power generation data withmissing points caused by occasional shutdowns,is utilized to verify the proposed method’s performance.Compared with other probabilistic forecastingmethods,the proposed method shows better accuracy and probabilistic performance on the same incomplete dataset,which highlights its potential for more flexible and sensitive maintenance of smart grids in smart cities.
基金funded by National Natural Science Foundation of China under Grant Nos.61972057 and U1836208Hunan Provincial Natural Science Foundation of China under Grant No.2019JJ50655+3 种基金Scientic Research Foundation of Hunan Provincial Education Department of China under Grant No.18B160Open Fund of Hunan Key Laboratory of Smart Roadway and Cooperative Vehicle Infrastructure Systems(Changsha University of Science and Technology)under Grant No.kfj180402the“Double First-class”International Cooperation and Development Scientic Research Project of Changsha University of Science and Technology under Grant No.2018IC25the Researchers Supporting Project No.(RSP-2020/102)King Saud University,Riyadh,Saudi Arabia.
文摘Multiple kernel clustering is an unsupervised data analysis method that has been used in various scenarios where data is easy to be collected but hard to be labeled.However,multiple kernel clustering for incomplete data is a critical yet challenging task.Although the existing absent multiple kernel clustering methods have achieved remarkable performance on this task,they may fail when data has a high value-missing rate,and they may easily fall into a local optimum.To address these problems,in this paper,we propose an absent multiple kernel clustering(AMKC)method on incomplete data.The AMKC method rst clusters the initialized incomplete data.Then,it constructs a new multiple-kernel-based data space,referred to as K-space,from multiple sources to learn kernel combination coefcients.Finally,it seamlessly integrates an incomplete-kernel-imputation objective,a multiple-kernel-learning objective,and a kernel-clustering objective in order to achieve absent multiple kernel clustering.The three stages in this process are carried out simultaneously until the convergence condition is met.Experiments on six datasets with various characteristics demonstrate that the kernel imputation and clustering performance of the proposed method is signicantly better than state-of-the-art competitors.Meanwhile,the proposed method gains fast convergence speed.
基金supported in part by the Center-initiated Research Project and Research Initiation Project of Zhejiang Laboratory(113012-AL2201,113012-PI2103)the National Natural Science Foundation of China(61300167,61976120)+2 种基金the Natural Science Foundation of Jiangsu Province(BK20191445)the Natural Science Key Foundation of Jiangsu Education Department(21KJA510004)Qing Lan Project of Jiangsu Province。
文摘Data with missing values,or incomplete information,brings some challenges to the development of classification,as the incompleteness may significantly affect the performance of classifiers.In this paper,we handle missing values in both training and test sets with uncertainty and imprecision reasoning by proposing a new belief combination of classifier(BCC)method based on the evidence theory.The proposed BCC method aims to improve the classification performance of incomplete data by characterizing the uncertainty and imprecision brought by incompleteness.In BCC,different attributes are regarded as independent sources,and the collection of each attribute is considered as a subset.Then,multiple classifiers are trained with each subset independently and allow each observed attribute to provide a sub-classification result for the query pattern.Finally,these sub-classification results with different weights(discounting factors)are used to provide supplementary information to jointly determine the final classes of query patterns.The weights consist of two aspects:global and local.The global weight calculated by an optimization function is employed to represent the reliability of each classifier,and the local weight obtained by mining attribute distribution characteristics is used to quantify the importance of observed attributes to the pattern classification.Abundant comparative experiments including seven methods on twelve datasets are executed,demonstrating the out-performance of BCC over all baseline methods in terms of accuracy,precision,recall,F1 measure,with pertinent computational costs.
基金supported by the Researchers Supporting Project number(RSP2024R 34),King Saud University,Riyadh,Saudi Arabia。
文摘In numerous real-world healthcare applications,handling incomplete medical data poses significant challenges for missing value imputation and subsequent clustering or classification tasks.Traditional approaches often rely on statistical methods for imputation,which may yield suboptimal results and be computationally intensive.This paper aims to integrate imputation and clustering techniques to enhance the classification of incomplete medical data with improved accuracy.Conventional classification methods are ill-suited for incomplete medical data.To enhance efficiency without compromising accuracy,this paper introduces a novel approach that combines imputation and clustering for the classification of incomplete data.Initially,the linear interpolation imputation method alongside an iterative Fuzzy c-means clustering method is applied and followed by a classification algorithm.The effectiveness of the proposed approach is evaluated using multiple performance metrics,including accuracy,precision,specificity,and sensitivity.The encouraging results demonstrate that our proposed method surpasses classical approaches across various performance criteria.
文摘With the rapid development of the economy,the scale of the power grid is expanding.The number of power equipment that constitutes the power grid has been very large,which makes the state data of power equipment grow explosively.These multi-source heterogeneous data have data differences,which lead to data variation in the process of transmission and preservation,thus forming the bad information of incomplete data.Therefore,the research on data integrity has become an urgent task.This paper is based on the characteristics of random chance and the Spatio-temporal difference of the system.According to the characteristics and data sources of the massive data generated by power equipment,the fuzzy mining model of power equipment data is established,and the data is divided into numerical and non-numerical data based on numerical data.Take the text data of power equipment defects as the mining material.Then,the Apriori algorithm based on an array is used to mine deeply.The strong association rules in incomplete data of power equipment are obtained and analyzed.From the change trend of NRMSE metrics and classification accuracy,most of the filling methods combined with the two frameworks in this method usually show a relatively stable filling trend,and will not fluctuate greatly with the growth of the missing rate.The experimental results show that the proposed algorithm model can effectively improve the filling effect of the existing filling methods on most data sets,and the filling effect fluctuates greatly with the increase of the missing rate,that is,with the increase of the missing rate,the improvement effect of the model for the existing filling methods is higher than 4.3%.Through the incomplete data clustering technology studied in this paper,a more innovative state assessment of smart grid reliability operation is carried out,which has good research value and reference significance.
文摘A generalized flexibility–based objective function utilized for structure damage identification is constructed for solving the constrained nonlinear least squares optimized problem. To begin with, the generalized flexibility matrix (GFM) proposed to solve the damage identification problem is recalled and a modal expansion method is introduced. Next, the objective function for iterative optimization process based on the GFM is formulated, and the Trust-Region algorithm is utilized to obtain the solution of the optimization problem for multiple damage cases. And then for computing the objective function gradient, the sensitivity analysis regarding design variables is derived. In addition, due to the spatial incompleteness, the influence of stiffness reduction and incomplete modal measurement data is discussed by means of two numerical examples with several damage cases. Finally, based on the computational results, it is evident that the presented approach provides good validity and reliability for the large and complicated engineering structures.
基金supported in part by the National Natural Science Foundation of China (62372385, 62272078, 62002337)the Chongqing Natural Science Foundation (CSTB2022NSCQ-MSX1486, CSTB2023NSCQ-LZX0069)the Deanship of Scientific Research at King Abdulaziz University, Jeddah, Saudi Arabia (RG-12-135-43)。
文摘High-dimensional and incomplete(HDI) matrices are primarily generated in all kinds of big-data-related practical applications. A latent factor analysis(LFA) model is capable of conducting efficient representation learning to an HDI matrix,whose hyper-parameter adaptation can be implemented through a particle swarm optimizer(PSO) to meet scalable requirements.However, conventional PSO is limited by its premature issues,which leads to the accuracy loss of a resultant LFA model. To address this thorny issue, this study merges the information of each particle's state migration into its evolution process following the principle of a generalized momentum method for improving its search ability, thereby building a state-migration particle swarm optimizer(SPSO), whose theoretical convergence is rigorously proved in this study. It is then incorporated into an LFA model for implementing efficient hyper-parameter adaptation without accuracy loss. Experiments on six HDI matrices indicate that an SPSO-incorporated LFA model outperforms state-of-the-art LFA models in terms of prediction accuracy for missing data of an HDI matrix with competitive computational efficiency.Hence, SPSO's use ensures efficient and reliable hyper-parameter adaptation in an LFA model, thus ensuring practicality and accurate representation learning for HDI matrices.
文摘The accurate identification and diagnosis of chemical process faults are crucial for ensuring the safe and stable operation of production plants.The current hot topic in industrial process fault diagnosis research is data-driven methods.Most of the existing fault diagnosis methods focus on a single shallow or deep learning model.This paper proposes a novel hybrid fault diagnosis method to fully utilize various features to improve the accuracy of fault diagnosis.Furthermore,the method addresses the issue of incomplete data,which has been largely overlooked in the majority of existing research.Firstly,the variable data is effectively fitted using orthogonal non-negative matrix tri-factorization,and the missing data in the matrix is solved to construct a complete production condition relationship.Next,the support vector machine model and the deep residual contraction network model are trained in parallel to prediagnose process faults by mining linear and non-linear interaction features.Finally,a novel mapping relationship is established between the result and model levels using the multi-layer perceptron algorithm to complete the final diagnosis and evaluation of the fault.To demonstrate the effectiveness of the proposed method,we conducted extensive comparative experiments on the Tennessee Eastman dataset and the ethylene plant cracking unit dataset.The experimental results show that the method has advantages in different evaluation metrics.
基金supported by the National Natural Science Foundation of China(NSFC)under Grants(Nos.U21A20483,62373040 and 62273031).
文摘In the context of intelligent manufacturing,the modern hot strip mill process(HSMP)shows characteristics such as diversification of products,multi-specification batch production,and demand-oriented customization.These characteristics pose significant challenges to ensuring process stability and consistency of product performance.Therefore,exploring the potential relationship between product performance and the production process,and developing a comprehensive performance evaluation method adapted to modern HSMP have become an urgent issue.A comprehensive performance evaluation method for HSMP by integrating multi-task learning and stacked performance-related autoencoder is proposed to solve the problems such as incomplete performance indicators(PIs)data,insufficient real-time acquisition requirements,and coupling of multiple PIs.First,according to the existing Chinese standards,a comprehensive performance evaluation grade strategy for strip steel is designed.The random forest model is established to predict and complete the parts of PIs data that could not be obtained in real-time.Second,a stacked performance-related autoencoder(SPAE)model is proposed to extract the deep features closely related to the product performance.Then,considering the correlation between PIs,the multi-task learning framework is introduced to output the subitem ratings and comprehensive product performance rating results of the strip steel online in real-time,where each task represents a subitem of comprehensive performance.Finally,the effectiveness of the method is verified on a real HSMP dataset,and the results show that the accuracy of the proposed method is as high as 94.8%,which is superior to the other comparative methods.
基金supported in part by the National Natural Science Foundation of China(62372385,62272078)the Chongqing Natural Science Foundation(CSTB2023NSCQ-LZX0069).
文摘A non-negative latent factor(NLF)model is able to be built efficiently via a single latent factor-dependent,non-negative and multiplicative update(SLF-NMU)algorithm for performing precise representation to high-dimensional and incomplete(HDI)matrix from many kinds of big-data-related applications.However,an SLF-NMU algorithm updates a latent factor relying on the current update increment only without considering past learning information,making a resultant model suffer from slow convergence.To address this issue,this study proposes a proportional integral(PI)controller-enhanced NLF(PI-NLF)model with two-fold ideas:1)Designing an increment refinement(IR)mechanism,which formulates the current and past update increments as the proportional and integral terms of a PI controller,thereby assimilating the past update information into the learning scheme smoothly with high efficiency;2)Deriving an IR-based SLF-NMU(ISN)algorithm,which updates a latent factor following the principle of an IR mechanism,thus significantly accelerating an NLF model's convergence rate.The simulation results on eight HDI matrices collected by real applications validate that a PI-NLF model outstrips several leading-edge models in both computational efficiency and accuracy when estimating missing data within an HDI matrix.The proposed PI-NLF model can be effectively applied to applications involving HDI matrix like e-commerce system,social network,and cloud service system.The code is available at https://github.com/yuanyeswu/PINLF/blob/mainIPINLF-code.zip.
基金the Opening Project of the State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System(No.CEMEE2014K0301A)
文摘Radar anti-jamming performance evaluation is a necessary link in the process of radar development,introduction and equipment. The applications of generalized rough set theory are proposed and discussed in this paper to address the problems of big data, incomplete data and redundant data in the construction of evaluation index system. Firstly, a mass of real-valued data is converted to some interval-valued data to avoid an unacceptable number of equivalence classes and classification rules, and the interval similarity relation is employed to make classifications of this interval-valued data. Meanwhile, incomplete data can be solved by a new definition of the connection degree tolerance relation for both interval-valued data and single-valued data, which makes a better description of rough set than the traditional limited tolerance relation. Then, E-condition entropy-based heuristic algorithm is applied to making attribute reduction to optimize the evaluation index system, and final decision rules can be extracted for system evaluation. Finally, the feasibility and advantage of the proposed methods are testified by a real example of radar anti-jamming performance evaluation.
文摘The accurate mathematical models for complicated structures are very difficult to construct.The work presented here provides an identification method for estimating the mass.damping,and stiffness matrices of linear dynamical systems from incomplete experimental data.The mass,stiffness and damping matrices are assumed to be real,symmetric,and positive definite The partial set of experimental complex eigenvalues and corresponding eigenvectors are given.In the proposed method the least squares algorithm is combined with the iteration technique to determine systems identified matrices and corresponding design parameters.Seeveral illustative examples,are presented to demonstrate the reliability of the proposed method .It is emphasized that the mass,damping and stiffness matrices can be identified simultaneously.
文摘Incomplete data accompanies our life processes and covers almost all fields of scientific studies, as a result of delivery failure, no power of battery, accidental loss, etc. However, how to model, index, and query incomplete data in- curs big challenges. For example, the queries struggling with incomplete data usually have dissatisfying query results due to the improper incompleteness handling methods. In this pa- per, we systematically review the management of incomplete data, including modelling, indexing, querying, and handling methods in terms of incomplete data. We also overview sev- eral application scenarios of incomplete data, and summa- rize the existing systems related to incomplete data. It is our hope that this survey could provide insights to the database community on how incomplete data is managed, and inspire database researchers to develop more advanced processing techniques and tools to cope with the issues resulting from incomplete data in the real world.
