Maximum frequent pattern generation from a large database of transactions and items for association rule mining is an important research topic in data mining. Association rule mining aims to discover interesting corre...Maximum frequent pattern generation from a large database of transactions and items for association rule mining is an important research topic in data mining. Association rule mining aims to discover interesting correlations, frequent patterns, associations, or causal structures between items hidden in a large database. By exploiting quantum computing, we propose an efficient quantum search algorithm design to discover the maximum frequent patterns. We modified Grover’s search algorithm so that a subspace of arbitrary symmetric states is used instead of the whole search space. We presented a novel quantum oracle design that employs a quantum counter to count the maximum frequent items and a quantum comparator to check with a minimum support threshold. The proposed derived algorithm increases the rate of the correct solutions since the search is only in a subspace. Furthermore, our algorithm significantly scales and optimizes the required number of qubits in design, which directly reflected positively on the performance. Our proposed design can accommodate more transactions and items and still have a good performance with a small number of qubits.展开更多
The Apriori algorithm is a classical method of association rules mining.Based on analysis of this theory,the paper provides an improved Apriori algorithm.The paper puts foward with algorithm combines HASH table techni...The Apriori algorithm is a classical method of association rules mining.Based on analysis of this theory,the paper provides an improved Apriori algorithm.The paper puts foward with algorithm combines HASH table technique and reduction of candidate item sets to enhance the usage efficiency of resources as well as the individualized service of the data library.展开更多
Data mining techniques offer great opportunities for developing ethics lines whose main aim is to ensure improvements and compliance with the values, conduct and commitments making up the code of ethics. The aim of th...Data mining techniques offer great opportunities for developing ethics lines whose main aim is to ensure improvements and compliance with the values, conduct and commitments making up the code of ethics. The aim of this study is to suggest a process for exploiting the data generated by the data generated and collected from an ethics line by extracting rules of association and applying the Apriori algorithm. This makes it possible to identify anomalies and behaviour patterns requiring action to review, correct, promote or expand them, as appropriate.展开更多
Association rules and C4.5 rules can overcome the shortage of the traditional land evaluation methods and improve the intelligibility and efficiency of the land evaluation knowledge.In order to compare these two kinds...Association rules and C4.5 rules can overcome the shortage of the traditional land evaluation methods and improve the intelligibility and efficiency of the land evaluation knowledge.In order to compare these two kinds of classification rules in the application,two fuzzy classifiers were established by combining with fuzzy decision algorithm especially based on Second General Soil Survey of Guangdong Province.The results of experiments demonstrated that the fuzzy classifier based on association rules obtain a higher accuracy rate,but with more complex calculation process and more computational overhead;the fuzzy classifier based on C4.5 rules obtain a slightly lower accuracy,but with fast computation and simpler calculation.展开更多
A method for mining frequent itemsets by evaluating their probability of supports based on asso-ciation analysis is presented.This paper obtains the probability of every 1-itemset by scanning the database,then evaluat...A method for mining frequent itemsets by evaluating their probability of supports based on asso-ciation analysis is presented.This paper obtains the probability of every 1-itemset by scanning the database,then evaluates the probability of every 2-itemset,every 3-itemset,every k-itemset from the frequent 1-itemsets and gains all the candidate frequent itemsets.This paper also scans the database for verifying the support of the candidate frequent itemsets.Last,the frequent itemsets are mined.The method reduces a lot of time of scanning database and shortens the computation time of the algorithm.展开更多
Discovering cyclic generalized association rules from transaction datbases can reveal the relationship of differ-ent levels of the taxonomies and display cyclic variations over time.Information about such variations i...Discovering cyclic generalized association rules from transaction datbases can reveal the relationship of differ-ent levels of the taxonomies and display cyclic variations over time.Information about such variations is great use of better identifying trends in associations and forecast-ing.Because cyclic rules are quite sensitive to a littlenoise,this paper uses the noise-ratio as the criterion of i-dentifing cydclic itemsets for dealing with the problem and utilizes the cycle-pruning technique to reduce the comput-ing time of the data mining process by exploiting the real-tionship between the cycle and generalized frequent item-sets.The paper gives the algorithm of mining cyclic gen-eralized itemsets(CGI).Experiment shows that the CGI algorithm can efficiently yield results.展开更多
The market trends rapidly changed over the last two decades.The primary reason is the newly created opportunities and the increased number of competitors competing to grasp market share using business analysis techniq...The market trends rapidly changed over the last two decades.The primary reason is the newly created opportunities and the increased number of competitors competing to grasp market share using business analysis techniques.Market Basket Analysis has a tangible effect in facilitating current change in the market.Market Basket Analysis is one of the famous fields that deal with Big Data and Data Mining applications.MBA initially uses Association Rule Learning(ARL)as a mean for realization.ARL has a beneficial effect in providing a plenty benefit in analyzing the market data and understanding customers’behavior.An important motive of using such techniques is maximizing the business profit as well as matching the exact customer needs as closely as possible.In this survey paper,we discussed several applications and methods of MBA based on ARL.Also,we reviewed some association rule learning measurements including trust,lift,leverage,and others.Furthermore,we discuss some open issues and future topics in the area of market basket analysis and association rule learning.展开更多
At present, most of the association rules algorithms are based on the Boolean attribute and single-level association rules mining. But data of the real world has various types, the multi-level and quantitative attribu...At present, most of the association rules algorithms are based on the Boolean attribute and single-level association rules mining. But data of the real world has various types, the multi-level and quantitative attributes are got more and more attention. And the most important step is to mine frequent sets. In this paper, we propose an algorithm that is called fuzzy multiple-level association （FMA） rules to mine frequent sets. It is based on the improved Eclat algorithm that is different to many researchers’ proposed algorithms thatused the Apriori algorithm. We analyze quantitative data’s frequent sets by using the fuzzy theory, dividing the hierarchy of concept and softening the boundary of attributes’ values and frequency. In this paper, we use the vertical-style data and the improved Eclat algorithm to describe the proposed method, we use this algorithm to analyze the data of Beijing logistics route. Experiments show that the algorithm has a good performance, it has better effectiveness and high efficiency.展开更多
Hotspots (active fires) indicate spatial distribution of fires. A study on determining influence factors for hotspot occurrence is essential so that fire events can be predicted based on characteristics of a certain a...Hotspots (active fires) indicate spatial distribution of fires. A study on determining influence factors for hotspot occurrence is essential so that fire events can be predicted based on characteristics of a certain area. This study discovers the possible influence factors on the occurrence of fire events using the association rule algorithm namely Apriori in the study area of Rokan Hilir Riau Province Indonesia. The Apriori algorithm was applied on a forest fire dataset which containeddata on physical environment (land cover, river, road and city center), socio-economic (income source, population, and number of school), weather (precipitation, wind speed, and screen temperature), and peatlands. The experiment results revealed 324 multidimensional association rules indicating relationships between hotspots occurrence and other factors.The association among hotspots occurrence with other geographical objects was discovered for the minimum support of 10% and the minimum confidence of 80%. The results show that strong relations between hotspots occurrence and influence factors are found for the support about 12.42%, the confidence of 1, and the lift of 2.26. These factors are precipitation greater than or equal to 3 mm/day, wind speed in [1m/s, 2m/s), non peatland area, screen temperature in [297K, 298K), the number of school in 1 km2 less than or equal to 0.1, and the distance of each hotspot to the nearest road less than or equal to 2.5 km.展开更多
In the privacy preservation of association rules, sensitivity analysis should be reported after the quantification of items in terms of their occurrence. The traditional methodologies, used for preserving confidential...In the privacy preservation of association rules, sensitivity analysis should be reported after the quantification of items in terms of their occurrence. The traditional methodologies, used for preserving confidentiality of association rules, are based on the assumptions while safeguarding susceptible information rather than recognition of insightful items. Therefore, it is time to go one step ahead in order to remove such assumptions in the protection of responsive information especially in XML association rule mining. Thus, we focus on this central and highly researched area in terms of generating XML association rule mining without arguing on the disclosure risks involvement in such mining process. Hence, we described the identification of susceptible items in order to hide the confidential information through a supervised learning technique. These susceptible items show the high dependency on other items that are measured in terms of statistical significance with Bayesian Network. Thus, we proposed two methodologies based on items probabilistic occurrence and mode of items. Additionally, all this information is modeled and named PPDM (Privacy Preservation in Data Mining) model for XARs. Furthermore, the PPDM model is helpful for sharing markets information among competitors with a lower chance of generating monopoly. Finally, PPDM model introduces great accuracy in computing sensitivity of items and opens new dimensions to the academia for the standardization of such NP-hard problems.展开更多
In this paper,association rule mining algorithm is utilized to analyze the correlations of various factors of causing traffic accidents,from which the relationship model of dangerous driving behaviors is established.I...In this paper,association rule mining algorithm is utilized to analyze the correlations of various factors of causing traffic accidents,from which the relationship model of dangerous driving behaviors is established.In this model,the factors and their correlations include:ability of risk control,ability of driving self-confidence,individual characteristics,and incorrect driving operations.By selecting the drivers in the city of Chengdu to be the objects of investigation,a group of valid sample data is obtained.Based on these data,the Support and Confidence for association rules are analyzed.In the analysis,the two stage computing of Apriori algorithm programming is simulated,and from which some important rules are obtained.With these rules,departments of traffic administration can focus on these key factors in their processing of traffic transactions.By the training of drivers’skills and their physical and mental behaviors,the incorrect driving operations can be greatly reduced and the traffic safety can be effectively guaranteed.展开更多
Scattered storage means an item can be stored in multiple inventory bins. The scattered storage assignment problem based on association rules in Kiva mobile fulfillment system is investigated, which aims to decide the...Scattered storage means an item can be stored in multiple inventory bins. The scattered storage assignment problem based on association rules in Kiva mobile fulfillment system is investigated, which aims to decide the pods for each item to put on so as to minimize the number of pods to be moved when picking a batch of orders. This problem is formulated into an integer programming model. A genetic algorithm is developed to solve the large-sized problems. Computational experiments and comparison between the scattered storage strategy and random storage strategy are conducted to evaluate the performance of the model and algorithm.展开更多
The traditional Apriori applied in books management system causes slow system operation due to frequent scanning of database and excessive quantity of candidate item-sets, so an information recommendation book managem...The traditional Apriori applied in books management system causes slow system operation due to frequent scanning of database and excessive quantity of candidate item-sets, so an information recommendation book management system based on improved Apriori data mining algorithm is designed, in which the C/S (client/server) architecture and B/S (browser/server) architecture are integrated, so as to open the book information to library staff and borrowers. The related information data of the borrowers and books can be extracted from books lending database by the data preprocessing sub-module in the system function module. After the data is cleaned, converted and integrated, the association rule mining sub-module is used to mine the strong association rules with support degree greater than minimum support degree threshold and confidence coefficient greater than minimum confidence coefficient threshold according to the processed data and by means of the improved Apriori data mining algorithm to generate association rule database. The association matching is performed by the personalized recommendation sub-module according to the borrower and his selected books in the association rule database. The book information associated with the books read by borrower is recommended to him to realize personalized recommendation of the book information. The experimental results show that the system can effectively recommend book related information, and its CPU occupation rate is only 6.47% under the condition that 50 clients are running it at the same time. Anyway, it has good performance.展开更多
Because data warehouse is frequently changing, incremental data leads to old knowledge which is mined formerly unavailable. In order to maintain the discovered knowledge and patterns dynamically, this study presents a...Because data warehouse is frequently changing, incremental data leads to old knowledge which is mined formerly unavailable. In order to maintain the discovered knowledge and patterns dynamically, this study presents a novel algorithm updating for global frequent patterns-IPARUC. A rapid clustering method is introduced to divide database into n parts in IPARUC firstly, where the data are similar in the same part. Then, the nodes in the tree are adjusted dynamically in inserting process by ＂pruning and laying back＂ to keep the frequency descending order so that they can be shared to approaching optimization. Finally local frequent itemsets mined from each local dataset are merged into global frequent itemsets. The results of experimental study are very encouraging. It is obvious from experiment that IPARUC is more effective and efficient than other two contrastive methods. Furthermore, there is significant application potential to a prototype of Web log Analyzer in web usage mining that can help us to discover useful knowledge effectively, even help managers making decision.展开更多
Association rule mining plays an important role in knowledge and information discovery.Often for a dataset,a huge number of rules can be extracted,but many of them are redundant,especially in the case of multi-level d...Association rule mining plays an important role in knowledge and information discovery.Often for a dataset,a huge number of rules can be extracted,but many of them are redundant,especially in the case of multi-level datasets.Mining non-redundant rules is a promising approach to solve this problem.However,existing work(Pasquier et al.2005,Xu&Li 2007)is only focused on single level datasets.In this paper,we firstly present a definition for redundancy and a concise representation called Reliable basis for representing non-redundant association rules,then we propose an extension to the previous work that can remove hierarchically redundant rules from multi-level datasets.We also show that the resulting concise representation of non-redundant association rules is lossless since all association rules can be derived from the representation.Experiments show that our extension can effectively generate multilevel non-redundant rules.展开更多
This paper aims to mine the knowledge and rules on compatibility of drugs from the prescriptions for curing arrhythmia in the Chinese traditional medicine database by Apriori algorithm. For data preparation, 1 113 pre...This paper aims to mine the knowledge and rules on compatibility of drugs from the prescriptions for curing arrhythmia in the Chinese traditional medicine database by Apriori algorithm. For data preparation, 1 113 prescriptions for arrhythmia, including 535 herbs （ totally 10884 counts of herbs） were collected into the database. The prescription data were preprocessed through redundancy reduction, normalized storage, and knowledge induction according to the pretreatment demands of data mining. Then the Apriori algorithm was used to analyze the data and form the related technical rules and treatment procedures. The experimental result of compatibility of drugs for curing arrhythmia from the Chinese traditional medicine database shows that the prescription compatibility obtained by Apriori algorithm generally accords with the basic law of traditional Chinese medicine for arrhythmia. Some special compatibilities unreported were also discovered in the experiment, which may be used as the basis for developing new prescriptions for arrhythmia.展开更多
With the development of smart agriculture,the accumulation of data in the field of pesticide regulation has a certain scale.The pesticide transaction data collected by the Pesticide National Data Center alone produces...With the development of smart agriculture,the accumulation of data in the field of pesticide regulation has a certain scale.The pesticide transaction data collected by the Pesticide National Data Center alone produces more than 10 million records daily.However,due to the backward technical means,the existing pesticide supervision data lack deep mining and usage.The Apriori algorithm is one of the classic algorithms in association rule mining,but it needs to traverse the transaction database multiple times,which will cause an extra IO burden.Spark is an emerging big data parallel computing framework with advantages such as memory computing and flexible distributed data sets.Compared with the Hadoop MapReduce computing framework,IO performance was greatly improved.Therefore,this paper proposed an improved Apriori algorithm based on Spark framework,ICAMA.The MapReduce process was used to support the candidate set and then to generate the candidate set.After experimental comparison,when the data volume exceeds 250 Mb,the performance of Spark-based Apriori algorithm was 20%higher than that of the traditional Hadoop-based Apriori algorithm,and with the increase of data volume,the performance improvement was more obvious.展开更多
文摘Maximum frequent pattern generation from a large database of transactions and items for association rule mining is an important research topic in data mining. Association rule mining aims to discover interesting correlations, frequent patterns, associations, or causal structures between items hidden in a large database. By exploiting quantum computing, we propose an efficient quantum search algorithm design to discover the maximum frequent patterns. We modified Grover’s search algorithm so that a subspace of arbitrary symmetric states is used instead of the whole search space. We presented a novel quantum oracle design that employs a quantum counter to count the maximum frequent items and a quantum comparator to check with a minimum support threshold. The proposed derived algorithm increases the rate of the correct solutions since the search is only in a subspace. Furthermore, our algorithm significantly scales and optimizes the required number of qubits in design, which directly reflected positively on the performance. Our proposed design can accommodate more transactions and items and still have a good performance with a small number of qubits.
文摘The Apriori algorithm is a classical method of association rules mining.Based on analysis of this theory,the paper provides an improved Apriori algorithm.The paper puts foward with algorithm combines HASH table technique and reduction of candidate item sets to enhance the usage efficiency of resources as well as the individualized service of the data library.
文摘Data mining techniques offer great opportunities for developing ethics lines whose main aim is to ensure improvements and compliance with the values, conduct and commitments making up the code of ethics. The aim of this study is to suggest a process for exploiting the data generated by the data generated and collected from an ethics line by extracting rules of association and applying the Apriori algorithm. This makes it possible to identify anomalies and behaviour patterns requiring action to review, correct, promote or expand them, as appropriate.
基金Supported by Science and Technology Plan Project of Guangdong Province (2009B010900026,2009CD058,2009CD078,2009CD079,2009CD080)Special Funds for Support Program of Development of Modern Information Service Industry of Guangdong Province(06120840B0370124)Funded Fund Project of South China Agricultural University (2007K017)~~
文摘Association rules and C4.5 rules can overcome the shortage of the traditional land evaluation methods and improve the intelligibility and efficiency of the land evaluation knowledge.In order to compare these two kinds of classification rules in the application,two fuzzy classifiers were established by combining with fuzzy decision algorithm especially based on Second General Soil Survey of Guangdong Province.The results of experiments demonstrated that the fuzzy classifier based on association rules obtain a higher accuracy rate,but with more complex calculation process and more computational overhead;the fuzzy classifier based on C4.5 rules obtain a slightly lower accuracy,but with fast computation and simpler calculation.
基金Funded by the National 973 Project(No.2003CB415205).
文摘A method for mining frequent itemsets by evaluating their probability of supports based on asso-ciation analysis is presented.This paper obtains the probability of every 1-itemset by scanning the database,then evaluates the probability of every 2-itemset,every 3-itemset,every k-itemset from the frequent 1-itemsets and gains all the candidate frequent itemsets.This paper also scans the database for verifying the support of the candidate frequent itemsets.Last,the frequent itemsets are mined.The method reduces a lot of time of scanning database and shortens the computation time of the algorithm.
文摘Discovering cyclic generalized association rules from transaction datbases can reveal the relationship of differ-ent levels of the taxonomies and display cyclic variations over time.Information about such variations is great use of better identifying trends in associations and forecast-ing.Because cyclic rules are quite sensitive to a littlenoise,this paper uses the noise-ratio as the criterion of i-dentifing cydclic itemsets for dealing with the problem and utilizes the cycle-pruning technique to reduce the comput-ing time of the data mining process by exploiting the real-tionship between the cycle and generalized frequent item-sets.The paper gives the algorithm of mining cyclic gen-eralized itemsets(CGI).Experiment shows that the CGI algorithm can efficiently yield results.
文摘The market trends rapidly changed over the last two decades.The primary reason is the newly created opportunities and the increased number of competitors competing to grasp market share using business analysis techniques.Market Basket Analysis has a tangible effect in facilitating current change in the market.Market Basket Analysis is one of the famous fields that deal with Big Data and Data Mining applications.MBA initially uses Association Rule Learning(ARL)as a mean for realization.ARL has a beneficial effect in providing a plenty benefit in analyzing the market data and understanding customers’behavior.An important motive of using such techniques is maximizing the business profit as well as matching the exact customer needs as closely as possible.In this survey paper,we discussed several applications and methods of MBA based on ARL.Also,we reviewed some association rule learning measurements including trust,lift,leverage,and others.Furthermore,we discuss some open issues and future topics in the area of market basket analysis and association rule learning.
基金supported by the Fundamental Research Funds for the Central Universities under Grants No.ZYGX2014J051 and No.ZYGX2014J066Science and Technology Projects in Sichuan Province under Grants No.2015JY0178,No.2016FZ0002,No.2014GZ0109,No.2015KZ002 and No.2015JY0030China Postdoctoral Science Foundation under Grant No.2015M572464
文摘At present, most of the association rules algorithms are based on the Boolean attribute and single-level association rules mining. But data of the real world has various types, the multi-level and quantitative attributes are got more and more attention. And the most important step is to mine frequent sets. In this paper, we propose an algorithm that is called fuzzy multiple-level association （FMA） rules to mine frequent sets. It is based on the improved Eclat algorithm that is different to many researchers’ proposed algorithms thatused the Apriori algorithm. We analyze quantitative data’s frequent sets by using the fuzzy theory, dividing the hierarchy of concept and softening the boundary of attributes’ values and frequency. In this paper, we use the vertical-style data and the improved Eclat algorithm to describe the proposed method, we use this algorithm to analyze the data of Beijing logistics route. Experiments show that the algorithm has a good performance, it has better effectiveness and high efficiency.
文摘Hotspots (active fires) indicate spatial distribution of fires. A study on determining influence factors for hotspot occurrence is essential so that fire events can be predicted based on characteristics of a certain area. This study discovers the possible influence factors on the occurrence of fire events using the association rule algorithm namely Apriori in the study area of Rokan Hilir Riau Province Indonesia. The Apriori algorithm was applied on a forest fire dataset which containeddata on physical environment (land cover, river, road and city center), socio-economic (income source, population, and number of school), weather (precipitation, wind speed, and screen temperature), and peatlands. The experiment results revealed 324 multidimensional association rules indicating relationships between hotspots occurrence and other factors.The association among hotspots occurrence with other geographical objects was discovered for the minimum support of 10% and the minimum confidence of 80%. The results show that strong relations between hotspots occurrence and influence factors are found for the support about 12.42%, the confidence of 1, and the lift of 2.26. These factors are precipitation greater than or equal to 3 mm/day, wind speed in [1m/s, 2m/s), non peatland area, screen temperature in [297K, 298K), the number of school in 1 km2 less than or equal to 0.1, and the distance of each hotspot to the nearest road less than or equal to 2.5 km.
文摘In the privacy preservation of association rules, sensitivity analysis should be reported after the quantification of items in terms of their occurrence. The traditional methodologies, used for preserving confidentiality of association rules, are based on the assumptions while safeguarding susceptible information rather than recognition of insightful items. Therefore, it is time to go one step ahead in order to remove such assumptions in the protection of responsive information especially in XML association rule mining. Thus, we focus on this central and highly researched area in terms of generating XML association rule mining without arguing on the disclosure risks involvement in such mining process. Hence, we described the identification of susceptible items in order to hide the confidential information through a supervised learning technique. These susceptible items show the high dependency on other items that are measured in terms of statistical significance with Bayesian Network. Thus, we proposed two methodologies based on items probabilistic occurrence and mode of items. Additionally, all this information is modeled and named PPDM (Privacy Preservation in Data Mining) model for XARs. Furthermore, the PPDM model is helpful for sharing markets information among competitors with a lower chance of generating monopoly. Finally, PPDM model introduces great accuracy in computing sensitivity of items and opens new dimensions to the academia for the standardization of such NP-hard problems.
文摘In this paper,association rule mining algorithm is utilized to analyze the correlations of various factors of causing traffic accidents,from which the relationship model of dangerous driving behaviors is established.In this model,the factors and their correlations include:ability of risk control,ability of driving self-confidence,individual characteristics,and incorrect driving operations.By selecting the drivers in the city of Chengdu to be the objects of investigation,a group of valid sample data is obtained.Based on these data,the Support and Confidence for association rules are analyzed.In the analysis,the two stage computing of Apriori algorithm programming is simulated,and from which some important rules are obtained.With these rules,departments of traffic administration can focus on these key factors in their processing of traffic transactions.By the training of drivers’skills and their physical and mental behaviors,the incorrect driving operations can be greatly reduced and the traffic safety can be effectively guaranteed.
文摘Scattered storage means an item can be stored in multiple inventory bins. The scattered storage assignment problem based on association rules in Kiva mobile fulfillment system is investigated, which aims to decide the pods for each item to put on so as to minimize the number of pods to be moved when picking a batch of orders. This problem is formulated into an integer programming model. A genetic algorithm is developed to solve the large-sized problems. Computational experiments and comparison between the scattered storage strategy and random storage strategy are conducted to evaluate the performance of the model and algorithm.
文摘The traditional Apriori applied in books management system causes slow system operation due to frequent scanning of database and excessive quantity of candidate item-sets, so an information recommendation book management system based on improved Apriori data mining algorithm is designed, in which the C/S (client/server) architecture and B/S (browser/server) architecture are integrated, so as to open the book information to library staff and borrowers. The related information data of the borrowers and books can be extracted from books lending database by the data preprocessing sub-module in the system function module. After the data is cleaned, converted and integrated, the association rule mining sub-module is used to mine the strong association rules with support degree greater than minimum support degree threshold and confidence coefficient greater than minimum confidence coefficient threshold according to the processed data and by means of the improved Apriori data mining algorithm to generate association rule database. The association matching is performed by the personalized recommendation sub-module according to the borrower and his selected books in the association rule database. The book information associated with the books read by borrower is recommended to him to realize personalized recommendation of the book information. The experimental results show that the system can effectively recommend book related information, and its CPU occupation rate is only 6.47% under the condition that 50 clients are running it at the same time. Anyway, it has good performance.
基金Supported by the National Natural Science Foundation of China(60472099)Ningbo Natural Science Foundation(2006A610017)
文摘Because data warehouse is frequently changing, incremental data leads to old knowledge which is mined formerly unavailable. In order to maintain the discovered knowledge and patterns dynamically, this study presents a novel algorithm updating for global frequent patterns-IPARUC. A rapid clustering method is introduced to divide database into n parts in IPARUC firstly, where the data are similar in the same part. Then, the nodes in the tree are adjusted dynamically in inserting process by ＂pruning and laying back＂ to keep the frequency descending order so that they can be shared to approaching optimization. Finally local frequent itemsets mined from each local dataset are merged into global frequent itemsets. The results of experimental study are very encouraging. It is obvious from experiment that IPARUC is more effective and efficient than other two contrastive methods. Furthermore, there is significant application potential to a prototype of Web log Analyzer in web usage mining that can help us to discover useful knowledge effectively, even help managers making decision.
文摘Association rule mining plays an important role in knowledge and information discovery.Often for a dataset,a huge number of rules can be extracted,but many of them are redundant,especially in the case of multi-level datasets.Mining non-redundant rules is a promising approach to solve this problem.However,existing work(Pasquier et al.2005,Xu&Li 2007)is only focused on single level datasets.In this paper,we firstly present a definition for redundancy and a concise representation called Reliable basis for representing non-redundant association rules,then we propose an extension to the previous work that can remove hierarchically redundant rules from multi-level datasets.We also show that the resulting concise representation of non-redundant association rules is lossless since all association rules can be derived from the representation.Experiments show that our extension can effectively generate multilevel non-redundant rules.
文摘This paper aims to mine the knowledge and rules on compatibility of drugs from the prescriptions for curing arrhythmia in the Chinese traditional medicine database by Apriori algorithm. For data preparation, 1 113 prescriptions for arrhythmia, including 535 herbs （ totally 10884 counts of herbs） were collected into the database. The prescription data were preprocessed through redundancy reduction, normalized storage, and knowledge induction according to the pretreatment demands of data mining. Then the Apriori algorithm was used to analyze the data and form the related technical rules and treatment procedures. The experimental result of compatibility of drugs for curing arrhythmia from the Chinese traditional medicine database shows that the prescription compatibility obtained by Apriori algorithm generally accords with the basic law of traditional Chinese medicine for arrhythmia. Some special compatibilities unreported were also discovered in the experiment, which may be used as the basis for developing new prescriptions for arrhythmia.
基金supported by National Natural Science Foundation of China(No.61601471)。
文摘With the development of smart agriculture,the accumulation of data in the field of pesticide regulation has a certain scale.The pesticide transaction data collected by the Pesticide National Data Center alone produces more than 10 million records daily.However,due to the backward technical means,the existing pesticide supervision data lack deep mining and usage.The Apriori algorithm is one of the classic algorithms in association rule mining,but it needs to traverse the transaction database multiple times,which will cause an extra IO burden.Spark is an emerging big data parallel computing framework with advantages such as memory computing and flexible distributed data sets.Compared with the Hadoop MapReduce computing framework,IO performance was greatly improved.Therefore,this paper proposed an improved Apriori algorithm based on Spark framework,ICAMA.The MapReduce process was used to support the candidate set and then to generate the candidate set.After experimental comparison,when the data volume exceeds 250 Mb,the performance of Spark-based Apriori algorithm was 20%higher than that of the traditional Hadoop-based Apriori algorithm,and with the increase of data volume,the performance improvement was more obvious.
