In this paper, we present a cluster-based algorithm for time series outlier mining.We use discrete Fourier transformation (DFT) to transform time series from time domain to frequency domain. Time series thus can be ma...In this paper, we present a cluster-based algorithm for time series outlier mining.We use discrete Fourier transformation (DFT) to transform time series from time domain to frequency domain. Time series thus can be mapped as the points in k -dimensional space.For these points, a cluster-based algorithm is developed to mine the outliers from these points.The algorithm first partitions the input points into disjoint clusters and then prunes the clusters,through judgment that can not contain outliers.Our algorithm has been run in the electrical load time series of one steel enterprise and proved to be effective.展开更多
Data Mining (DM) methods are being increasingly used in prediction with time series data, in addition to traditional statistical approaches. This paper presents a literature review of the use of DM with time series da...Data Mining (DM) methods are being increasingly used in prediction with time series data, in addition to traditional statistical approaches. This paper presents a literature review of the use of DM with time series data, focusing on shorttime stocks prediction. This is an area that has been attracting a great deal of attention from researchers in the field. The main contribution of this paper is to provide an outline of the use of DM with time series data, using mainly examples related with short-term stocks prediction. This is important to a better understanding of the field. Some of the main trends and open issues will also be introduced.展开更多
The mining of the rules from the electrical load time series data which are collected from the EMS (Energy Management System) is discussed. The data from the EMS are too huge and sophisticated to be understood and use...The mining of the rules from the electrical load time series data which are collected from the EMS (Energy Management System) is discussed. The data from the EMS are too huge and sophisticated to be understood and used by the power system engineer, while useful information is hidden in the electrical load data. The authors discuss the use of fuzzy linguistic summary as data mining method to induce the rules from the electrical load time series. The data preprocessing techniques are also discussed in the paper.展开更多
Data Mining has become an important technique for the exploration and extraction of data in numerous and various research projects in different fields (technology, information technology, business, the environment, ec...Data Mining has become an important technique for the exploration and extraction of data in numerous and various research projects in different fields (technology, information technology, business, the environment, economics, etc.). In the context of the analysis and visualisation of large amounts of data extracted using Data Mining on a temporary basis (time-series), free software such as R has appeared in the international context as a perfect inexpensive and efficient tool of exploitation and visualisation of time series. This has allowed the development of models, which help to extract the most relevant information from large volumes of data. In this regard, a script has been developed with the goal of implementing ARIMA models, showing these as useful and quick mechanisms for the extraction, analysis and visualisation of large data volumes, in addition to presenting the great advantage of being applied in multiple branches of knowledge from economy, demography, physics, mathematics and fisheries among others. Therefore, ARIMA models appear as a Data Mining technique, offering reliable, robust and high-quality results, to help validate and sustain the research carried out.展开更多
This paper adopts data mining(DM) technique and fuzzy system theory for robust time series forecasting.By introducing DM technique,the fuzzy rule extraction algorithm is improved to be more robust with the noises and ...This paper adopts data mining(DM) technique and fuzzy system theory for robust time series forecasting.By introducing DM technique,the fuzzy rule extraction algorithm is improved to be more robust with the noises and outliers in time series.Then,the constructed fuzzy inference system(FIS) is optimized with a partition refining strategy to balance the system's accuracy and complexity.The proposed algorithm is compared with the WangMendel(WM) method,a benchmark method for building FIS,in comprehensive analysis of robustness.In the classical Mackey-Glass time series forecasting,the simulation results prove that the proposed method is able to predict time series with random perturbation more accurately.For the practical application,the proposed FIS is applied to predicting the time series of ship maneuvering motion.To obtain actual time series data records,the ship maneuvering motion trial is conducted in the Yukun ship of Dalian Maritime University in China.The time series forecasting results show that the FIS constructed with DM concepts can forecast ship maneuvering motion robustly and effectively.展开更多
The UCR time series archive–introduced in 2002,has become an important resource in the time series data mining community,with at least one thousand published papers making use of at least one data set from the archiv...The UCR time series archive–introduced in 2002,has become an important resource in the time series data mining community,with at least one thousand published papers making use of at least one data set from the archive.The original incarnation of the archive had sixteen data sets but since that time,it has gone through periodic expansions.The last expansion took place in the summer of 2015 when the archive grew from 45 to 85 data sets.This paper introduces and will focus on the new data expansion from 85 to 128 data sets.Beyond expanding this valuable resource,this paper offers pragmatic advice to anyone who may wish to evaluate a new algorithm on the archive.Finally,this paper makes a novel and yet actionable claim:of the hundreds of papers that show an improvement over the standard baseline(1-nearest neighbor classification),a fraction might be mis-attributing the reasons for their improvement.Moreover,the improvements claimed by these papers might have been achievable with a much simpler modification,requiring just a few lines of code.展开更多
To predict the trend of chaotic time series in time series analysis and time series data mining fields,a novel predicting algorithm of chaotic time series trend is presented,and an on-line segmenting algorithm is prop...To predict the trend of chaotic time series in time series analysis and time series data mining fields,a novel predicting algorithm of chaotic time series trend is presented,and an on-line segmenting algorithm is proposed to convert a time series into a binary string according to ascending or descending trend of each subsequence.The on-line segmenting algorithm is independent of the prior knowledge about time series.The naive Bayesian algorithm is then employed to predict the trend of chaotic time series according to the binary string.The experimental results of three chaotic time series demonstrate that the proposed method predicts the ascending or descending trend of chaotic time series with few error.展开更多
A fundamental problem in whole sequence matching and subsequence matching is the problem of representation of time series.In the last decade many high level representations of time series have been proposed for data m...A fundamental problem in whole sequence matching and subsequence matching is the problem of representation of time series.In the last decade many high level representations of time series have been proposed for data mining which involve a trade-off between accuracy and compactness.In this paper the author proposes a novel time series representation called Grid Minimum Bounding Rectangle(GMBR) and based on Minimum Bounding Rectangle.In this paper,the binary idea is applied into the Minimum Bounding Rectangle.The experiments have been performed on synthetic,as well as real data sequences to evaluate the proposed method.The experiment demonstrates that 69%-92% of irrelevant sequences are pruned using the proposed method.展开更多
Pattern discovery from time series is of fundamental importance. Most of the algorithms of pattern discovery in time series capture the values of time series based on some kinds of similarity measures. Affected by the...Pattern discovery from time series is of fundamental importance. Most of the algorithms of pattern discovery in time series capture the values of time series based on some kinds of similarity measures. Affected by the scale and baseline, value-based methods bring about problem when the objective is to capture the shape. Thus, a similarity measure based on shape, Sh measure, is originally proposed, andthe properties of this similarity and corresponding proofs are given. Then a time series shape pattern discovery algorithm based on Sh measure is put forward. The proposed algorithm is terminated in finite iteration with given computational and storage complexity. Finally the experiments on synthetic datasets and sunspot datasets demonstrate that the time series shape pattern algorithm is valid.展开更多
Time series analysis is widely used in the fields of finance, medical, and climate monitoring. However, the high dimension characteristic of time series brings a lot of inconvenience to its application. In order to so...Time series analysis is widely used in the fields of finance, medical, and climate monitoring. However, the high dimension characteristic of time series brings a lot of inconvenience to its application. In order to solve the high dimensionality problem of time series, symbolic representation, a method of time series feature representation is proposed, which plays an important role in time series classification and clustering, pattern matching, anomaly detection and others. In this paper, existing symbolization representation methods of time series were reviewed and compared. Firstly, the classical symbolic aggregate approximation (SAX) principle and its deficiencies were analyzed. Then, several SAX improvement methods, including aSAX, SMSAX, ESAX and some others, were introduced and classified;Meanwhile, an experiment evaluation of the existing SAX methods was given. Finally, some unresolved issues of existing SAX methods were summed up for future work.展开更多
This paper presents a fault-detection method based on the phase space reconstruction and data mining approaches for the complex electronic system. The approach for the phase space reconstruction of chaotic time series...This paper presents a fault-detection method based on the phase space reconstruction and data mining approaches for the complex electronic system. The approach for the phase space reconstruction of chaotic time series is a combination algorithm of multiple autocorrelation and F-test, by which the quasi-optimal embedding dimension and time delay can be obtained. The data mining algorithm, which calculates the radius of gyration of unit-mass point around the centre of mass in the phase space, can distinguish the fault parameter from the chaotic time series output by the tested system. The experimental results depict that this fault detection method can correctly detect the fault phenomena of electronic system.展开更多
The explosion of online information with the recent advent of digital technology in information processing,information storing,information sharing,natural language processing,and text mining techniques has enabled sto...The explosion of online information with the recent advent of digital technology in information processing,information storing,information sharing,natural language processing,and text mining techniques has enabled stock investors to uncover market movement and volatility from heterogeneous content.For example,a typical stock market investor reads the news,explores market sentiment,and analyzes technical details in order to make a sound decision prior to purchasing or selling a particular company’s stock.However,capturing a dynamic stock market trend is challenging owing to high fluctuation and the non-stationary nature of the stock market.Although existing studies have attempted to enhance stock prediction,few have provided a complete decision-support system for investors to retrieve real-time data from multiple sources and extract insightful information for sound decision-making.To address the above challenge,we propose a unified solution for data collection,analysis,and visualization in real-time stock market prediction to retrieve and process relevant financial data from news articles,social media,and company technical information.We aim to provide not only useful information for stock investors but also meaningful visualization that enables investors to effectively interpret storyline events affecting stock prices.Specifically,we utilize an ensemble stacking of diversified machine-learning-based estimators and innovative contextual feature engineering to predict the next day’s stock prices.Experiment results show that our proposed stock forecasting method outperforms a traditional baseline with an average mean absolute percentage error of 0.93.Our findings confirm that leveraging an ensemble scheme of machine learning methods with contextual information improves stock prediction performance.Finally,our study could be further extended to a wide variety of innovative financial applications that seek to incorporate external insight from contextual information such as large-scale online news articles and social media data.展开更多
It is difficult if not impossible to appropriately and effectively select from among the vast pool of existing neural network machine learning predictive models for industrial incorporation or academic research explor...It is difficult if not impossible to appropriately and effectively select from among the vast pool of existing neural network machine learning predictive models for industrial incorporation or academic research exploration and enhancement. When all models outperform all the others under disparate circumstances, none of the models do. Selecting the ideal model becomes a matter of ill-supported opinion ungrounded on the extant real world environment. This paper proposes a novel grouping of the model pool grounded along a non-stationary real world data line into two groups: Permanent Data Learning and Reversible Data Learning. This paper further proposes a novel approach towards qualitatively and quantitatively demonstrating their significant differences based on how they alternatively approach dynamic and raw real world data vs static and prescient data mining biased laboratory data. The results across 2040 separate simulation runs using 15,600 data points in realistically operationally controlled data environments show that the two-group division is effective and significant with clear qualitative, quantitative and theoretical support. Results across the empirical and theoretical spectrum are internally and externally consistent yet demonstrative of why and how this result is non-obvious.展开更多
The burgeoning field of intelligent transportation systems(ITS)has been pivotal in addressing contemporary traffic challenges,significantly benefiting from the evolution of computational capabilities and sensor techno...The burgeoning field of intelligent transportation systems(ITS)has been pivotal in addressing contemporary traffic challenges,significantly benefiting from the evolution of computational capabilities and sensor technologies.This surge in technical advancement has paved the way for extensive reliance on deep-learning methodologies to exploit largescale traffic data.Such efforts are directed toward decoding the intricate spatiotemporal dynamics inherent in traffic prediction.This study delves into the realm of traffic prediction,encompassing time series,spatiotemporal,and origin-destination(OD)predictions,to dissect the nuances among various predictive methodologies.Through a meticulous examination,this paper highlights the efficacy of spatiotemporal coupling techniques in enhancing prediction accuracy.Furthermore,it scrutinizes the existing challenges and delineates open and new questions within the traffic prediction domain,thereby charting out prospective avenues for future research endeavors.展开更多
The Dynamic Time Warping(DTW)algorithm is widely used in finding the global alignment of time series.Many time series data mining and analytical problems can be solved by the DTW algorithm.However,using the DTW algori...The Dynamic Time Warping(DTW)algorithm is widely used in finding the global alignment of time series.Many time series data mining and analytical problems can be solved by the DTW algorithm.However,using the DTW algorithm to find similar subsequences is computationally expensive or unable to perform accurate analysis.Hence,in the literature,the parallelisation technique is used to speed up the DTW algorithm.However,due to the nature of DTW algorithm,parallelizing this algorithm remains an open challenge.In this paper,we first propose a novel method that finds the similar local subsequence.Our algorithm first searches for the possible start positions of subsequence,and then finds the best-matching alignment from these positions.Moreover,we parallelize the proposed algorithm on GPUs using CUDA and further propose an optimization technique to improve the performance of our parallelization implementation on GPU.We conducted the extensive experiments to evaluate the proposed method.Experimental results demonstrate that the proposed algorithm is able to discover time series subsequences efficiently and that the proposed GPU-based parallelization technique can further speedup the processing.展开更多
文摘In this paper, we present a cluster-based algorithm for time series outlier mining.We use discrete Fourier transformation (DFT) to transform time series from time domain to frequency domain. Time series thus can be mapped as the points in k -dimensional space.For these points, a cluster-based algorithm is developed to mine the outliers from these points.The algorithm first partitions the input points into disjoint clusters and then prunes the clusters,through judgment that can not contain outliers.Our algorithm has been run in the electrical load time series of one steel enterprise and proved to be effective.
文摘Data Mining (DM) methods are being increasingly used in prediction with time series data, in addition to traditional statistical approaches. This paper presents a literature review of the use of DM with time series data, focusing on shorttime stocks prediction. This is an area that has been attracting a great deal of attention from researchers in the field. The main contribution of this paper is to provide an outline of the use of DM with time series data, using mainly examples related with short-term stocks prediction. This is important to a better understanding of the field. Some of the main trends and open issues will also be introduced.
文摘The mining of the rules from the electrical load time series data which are collected from the EMS (Energy Management System) is discussed. The data from the EMS are too huge and sophisticated to be understood and used by the power system engineer, while useful information is hidden in the electrical load data. The authors discuss the use of fuzzy linguistic summary as data mining method to induce the rules from the electrical load time series. The data preprocessing techniques are also discussed in the paper.
文摘Data Mining has become an important technique for the exploration and extraction of data in numerous and various research projects in different fields (technology, information technology, business, the environment, economics, etc.). In the context of the analysis and visualisation of large amounts of data extracted using Data Mining on a temporary basis (time-series), free software such as R has appeared in the international context as a perfect inexpensive and efficient tool of exploitation and visualisation of time series. This has allowed the development of models, which help to extract the most relevant information from large volumes of data. In this regard, a script has been developed with the goal of implementing ARIMA models, showing these as useful and quick mechanisms for the extraction, analysis and visualisation of large data volumes, in addition to presenting the great advantage of being applied in multiple branches of knowledge from economy, demography, physics, mathematics and fisheries among others. Therefore, ARIMA models appear as a Data Mining technique, offering reliable, robust and high-quality results, to help validate and sustain the research carried out.
基金the Fundamental Research Funds for the Central Universities,China(No.01750307)the Doctoral Scientific Research Foundation of Liaoning Province,China(No.201501188)
文摘This paper adopts data mining(DM) technique and fuzzy system theory for robust time series forecasting.By introducing DM technique,the fuzzy rule extraction algorithm is improved to be more robust with the noises and outliers in time series.Then,the constructed fuzzy inference system(FIS) is optimized with a partition refining strategy to balance the system's accuracy and complexity.The proposed algorithm is compared with the WangMendel(WM) method,a benchmark method for building FIS,in comprehensive analysis of robustness.In the classical Mackey-Glass time series forecasting,the simulation results prove that the proposed method is able to predict time series with random perturbation more accurately.For the practical application,the proposed FIS is applied to predicting the time series of ship maneuvering motion.To obtain actual time series data records,the ship maneuvering motion trial is conducted in the Yukun ship of Dalian Maritime University in China.The time series forecasting results show that the FIS constructed with DM concepts can forecast ship maneuvering motion robustly and effectively.
文摘The UCR time series archive–introduced in 2002,has become an important resource in the time series data mining community,with at least one thousand published papers making use of at least one data set from the archive.The original incarnation of the archive had sixteen data sets but since that time,it has gone through periodic expansions.The last expansion took place in the summer of 2015 when the archive grew from 45 to 85 data sets.This paper introduces and will focus on the new data expansion from 85 to 128 data sets.Beyond expanding this valuable resource,this paper offers pragmatic advice to anyone who may wish to evaluate a new algorithm on the archive.Finally,this paper makes a novel and yet actionable claim:of the hundreds of papers that show an improvement over the standard baseline(1-nearest neighbor classification),a fraction might be mis-attributing the reasons for their improvement.Moreover,the improvements claimed by these papers might have been achievable with a much simpler modification,requiring just a few lines of code.
文摘To predict the trend of chaotic time series in time series analysis and time series data mining fields,a novel predicting algorithm of chaotic time series trend is presented,and an on-line segmenting algorithm is proposed to convert a time series into a binary string according to ascending or descending trend of each subsequence.The on-line segmenting algorithm is independent of the prior knowledge about time series.The naive Bayesian algorithm is then employed to predict the trend of chaotic time series according to the binary string.The experimental results of three chaotic time series demonstrate that the proposed method predicts the ascending or descending trend of chaotic time series with few error.
基金National Natural Science Foundation of China (No.60674088)Shandong Education Committee 2007 Scientific Research Development Plan (No.J07WJ20)
文摘A fundamental problem in whole sequence matching and subsequence matching is the problem of representation of time series.In the last decade many high level representations of time series have been proposed for data mining which involve a trade-off between accuracy and compactness.In this paper the author proposes a novel time series representation called Grid Minimum Bounding Rectangle(GMBR) and based on Minimum Bounding Rectangle.In this paper,the binary idea is applied into the Minimum Bounding Rectangle.The experiments have been performed on synthetic,as well as real data sequences to evaluate the proposed method.The experiment demonstrates that 69%-92% of irrelevant sequences are pruned using the proposed method.
文摘Pattern discovery from time series is of fundamental importance. Most of the algorithms of pattern discovery in time series capture the values of time series based on some kinds of similarity measures. Affected by the scale and baseline, value-based methods bring about problem when the objective is to capture the shape. Thus, a similarity measure based on shape, Sh measure, is originally proposed, andthe properties of this similarity and corresponding proofs are given. Then a time series shape pattern discovery algorithm based on Sh measure is put forward. The proposed algorithm is terminated in finite iteration with given computational and storage complexity. Finally the experiments on synthetic datasets and sunspot datasets demonstrate that the time series shape pattern algorithm is valid.
基金the National Natural Science Foundation of China [grant numbers 61602279, 61472229]Shandong Province Postdoctoral Innovation Project [grant number 201603056]+2 种基金the Sci.& Tech. Development Fund of Shandong Province of China [grant number 2016ZDJS02A11 and Grant ZR2017MF027]the SDUST Research Fund [grant number 2015TDJH102]and the Fund of Oceanic telemetry Engineering and Technology Research Center, State Oceanic Administration (grant number 2018002).
文摘Time series analysis is widely used in the fields of finance, medical, and climate monitoring. However, the high dimension characteristic of time series brings a lot of inconvenience to its application. In order to solve the high dimensionality problem of time series, symbolic representation, a method of time series feature representation is proposed, which plays an important role in time series classification and clustering, pattern matching, anomaly detection and others. In this paper, existing symbolization representation methods of time series were reviewed and compared. Firstly, the classical symbolic aggregate approximation (SAX) principle and its deficiencies were analyzed. Then, several SAX improvement methods, including aSAX, SMSAX, ESAX and some others, were introduced and classified;Meanwhile, an experiment evaluation of the existing SAX methods was given. Finally, some unresolved issues of existing SAX methods were summed up for future work.
文摘This paper presents a fault-detection method based on the phase space reconstruction and data mining approaches for the complex electronic system. The approach for the phase space reconstruction of chaotic time series is a combination algorithm of multiple autocorrelation and F-test, by which the quasi-optimal embedding dimension and time delay can be obtained. The data mining algorithm, which calculates the radius of gyration of unit-mass point around the centre of mass in the phase space, can distinguish the fault parameter from the chaotic time series output by the tested system. The experimental results depict that this fault detection method can correctly detect the fault phenomena of electronic system.
基金supported by Mahidol University(Grant No.MU-MiniRC02/2564)We also appreciate the partial computing resources from Grant No.RSA6280105funded by Thailand Science Research and Innovation(TSRI),(formerly known as the Thailand Research Fund(TRF)),and the National Research Council of Thailand(NRCT).
文摘The explosion of online information with the recent advent of digital technology in information processing,information storing,information sharing,natural language processing,and text mining techniques has enabled stock investors to uncover market movement and volatility from heterogeneous content.For example,a typical stock market investor reads the news,explores market sentiment,and analyzes technical details in order to make a sound decision prior to purchasing or selling a particular company’s stock.However,capturing a dynamic stock market trend is challenging owing to high fluctuation and the non-stationary nature of the stock market.Although existing studies have attempted to enhance stock prediction,few have provided a complete decision-support system for investors to retrieve real-time data from multiple sources and extract insightful information for sound decision-making.To address the above challenge,we propose a unified solution for data collection,analysis,and visualization in real-time stock market prediction to retrieve and process relevant financial data from news articles,social media,and company technical information.We aim to provide not only useful information for stock investors but also meaningful visualization that enables investors to effectively interpret storyline events affecting stock prices.Specifically,we utilize an ensemble stacking of diversified machine-learning-based estimators and innovative contextual feature engineering to predict the next day’s stock prices.Experiment results show that our proposed stock forecasting method outperforms a traditional baseline with an average mean absolute percentage error of 0.93.Our findings confirm that leveraging an ensemble scheme of machine learning methods with contextual information improves stock prediction performance.Finally,our study could be further extended to a wide variety of innovative financial applications that seek to incorporate external insight from contextual information such as large-scale online news articles and social media data.
文摘It is difficult if not impossible to appropriately and effectively select from among the vast pool of existing neural network machine learning predictive models for industrial incorporation or academic research exploration and enhancement. When all models outperform all the others under disparate circumstances, none of the models do. Selecting the ideal model becomes a matter of ill-supported opinion ungrounded on the extant real world environment. This paper proposes a novel grouping of the model pool grounded along a non-stationary real world data line into two groups: Permanent Data Learning and Reversible Data Learning. This paper further proposes a novel approach towards qualitatively and quantitatively demonstrating their significant differences based on how they alternatively approach dynamic and raw real world data vs static and prescient data mining biased laboratory data. The results across 2040 separate simulation runs using 15,600 data points in realistically operationally controlled data environments show that the two-group division is effective and significant with clear qualitative, quantitative and theoretical support. Results across the empirical and theoretical spectrum are internally and externally consistent yet demonstrative of why and how this result is non-obvious.
基金supported by the National Natural Science Foundation of China(62273057)。
文摘The burgeoning field of intelligent transportation systems(ITS)has been pivotal in addressing contemporary traffic challenges,significantly benefiting from the evolution of computational capabilities and sensor technologies.This surge in technical advancement has paved the way for extensive reliance on deep-learning methodologies to exploit largescale traffic data.Such efforts are directed toward decoding the intricate spatiotemporal dynamics inherent in traffic prediction.This study delves into the realm of traffic prediction,encompassing time series,spatiotemporal,and origin-destination(OD)predictions,to dissect the nuances among various predictive methodologies.Through a meticulous examination,this paper highlights the efficacy of spatiotemporal coupling techniques in enhancing prediction accuracy.Furthermore,it scrutinizes the existing challenges and delineates open and new questions within the traffic prediction domain,thereby charting out prospective avenues for future research endeavors.
基金supported by the National Natural Science Foundation of China(No.61602215)the Science Foundation of Jiangsu Province(No.BK20150527)the EU Horizon 2020—Marie Sklodowska-Curie Actions through the project entitled Computer Vision Enabled Multimedia Forensics and People Identification(Project No.690907,Acronym:IDENTITY).
文摘The Dynamic Time Warping(DTW)algorithm is widely used in finding the global alignment of time series.Many time series data mining and analytical problems can be solved by the DTW algorithm.However,using the DTW algorithm to find similar subsequences is computationally expensive or unable to perform accurate analysis.Hence,in the literature,the parallelisation technique is used to speed up the DTW algorithm.However,due to the nature of DTW algorithm,parallelizing this algorithm remains an open challenge.In this paper,we first propose a novel method that finds the similar local subsequence.Our algorithm first searches for the possible start positions of subsequence,and then finds the best-matching alignment from these positions.Moreover,we parallelize the proposed algorithm on GPUs using CUDA and further propose an optimization technique to improve the performance of our parallelization implementation on GPU.We conducted the extensive experiments to evaluate the proposed method.Experimental results demonstrate that the proposed algorithm is able to discover time series subsequences efficiently and that the proposed GPU-based parallelization technique can further speedup the processing.
