The proliferation of textual data in society currently is overwhelming, in particular, unstructured textual data is being constantly generated via call centre logs, emails, documents on the web, blogs, tweets, custome...The proliferation of textual data in society currently is overwhelming, in particular, unstructured textual data is being constantly generated via call centre logs, emails, documents on the web, blogs, tweets, customer comments, customer reviews, etc.While the amount of textual data is increasing rapidly, users ability to summarise, understand, and make sense of such data for making better business/living decisions remains challenging. This paper studies how to analyse textual data, based on layered software patterns, for extracting insightful user intelligence from a large collection of documents and for using such information to improve user operations and performance.展开更多
In computational physics proton transfer phenomena could be viewed as pattern classification problems based on a set of input features allowing classification of the proton motion into two categories： transfer ＇occu...In computational physics proton transfer phenomena could be viewed as pattern classification problems based on a set of input features allowing classification of the proton motion into two categories： transfer ＇occurred＇ and transfer ＇not occurred＇. The goal of this paper is to evaluate the use of artificial neural networks in the classification of proton transfer events, based on the feed-forward back propagation neural network, used as a classifier to distinguish between the two transfer cases. In this paper, we use a new developed data mining and pattern recognition tool for automating, controlling, and drawing charts of the output data of an Empirical Valence Bond existing code. The study analyzes the need for pattern recognition in aqueous proton transfer processes and how the learning approach in error back propagation （multilayer perceptron algorithms） could be satisfactorily employed in the present case. We present a tool for pattern recognition and validate the code including a real physical case study. The results of applying the artificial neural networks methodology to crowd patterns based upon selected physical properties （e.g., temperature, density） show the abilities of the network to learn proton transfer patterns corresponding to properties of the aqueous environments, which is in turn proved to be fully compatible with previous proton transfer studies.展开更多
Anomaly detection has been an active research topic in the field of network intrusion detection for many years. A novel method is presented for anomaly detection based on system calls into the kernels of Unix or Linux...Anomaly detection has been an active research topic in the field of network intrusion detection for many years. A novel method is presented for anomaly detection based on system calls into the kernels of Unix or Linux systems. The method uses the data mining technique to model the normal behavior of a privileged program and uses a variable-length pattern matching algorithm to perform the comparison of the current behavior and historic normal behavior, which is more suitable for this problem than the fixed-length pattern matching algorithm proposed by Forrest et al. At the detection stage, the particularity of the audit data is taken into account, and two alternative schemes could be used to distinguish between normalities and intrusions. The method gives attention to both computational efficiency and detection accuracy and is especially applicable for on-line detection. The performance of the method is evaluated using the typical testing data set, and the results show that it is significantly better than the anomaly detection method based on hidden Markov models proposed by Yan et al. and the method based on fixed-length patterns proposed by Forrest and Hofmeyr. The novel method has been applied to practical hosted-based intrusion detection systems and achieved high detection performance.展开更多
In order to explore the travel characteristics and space-time distribution of different groups of bikeshare users,an online analytical processing(OLAP)tool called data cube was used for treating and displaying multi-d...In order to explore the travel characteristics and space-time distribution of different groups of bikeshare users,an online analytical processing(OLAP)tool called data cube was used for treating and displaying multi-dimensional data.We extended and modified the traditionally threedimensional data cube into four dimensions,which are space,date,time,and user,each with a user-specified hierarchy,and took transaction numbers and travel time as two quantitative measures.The results suggest that there are two obvious transaction peaks during the morning and afternoon rush hours on weekdays,while the volume at weekends has an approximate even distribution.Bad weather condition significantly restricts the bikeshare usage.Besides,seamless smartcard users generally take a longer trip than exclusive smartcard users;and non-native users ride faster than native users.These findings not only support the applicability and efficiency of data cube in the field of visualizing massive smartcard data,but also raise equity concerns among bikeshare users with different demographic backgrounds.展开更多
A variety of faulty radar echoes may cause serious problems with radar data applications,especially radar data assimilation and quantitative precipitation estimates.In this study,＂test pattern＂ caused by test signal...A variety of faulty radar echoes may cause serious problems with radar data applications,especially radar data assimilation and quantitative precipitation estimates.In this study,＂test pattern＂ caused by test signal or radar hardware failures in CINRAD （China New Generation Weather Radar） SA and SB radar operational observations are investigated.In order to distinguish the test pattern from other types of radar echoes,such as precipitation,clear air and other non-meteorological echoes,five feature parameters including the effective reflectivity data percentage （Rz）,velocity RF （range folding） data percentage （RRF）,missing velocity data percentage （RM）,averaged along-azimuth reflectivity fluctuation （RNr,z） and averaged along-beam reflectivity fluctuation （RNa,z） are proposed.Based on the fuzzy logic method,a test pattern identification algorithm is developed,and the statistical results from all the different kinds of radar echoes indicate the performance of the algorithm.Analysis of two typical cases with heavy precipitation echoes located inside the test pattern are performed.The statistical results show that the test pattern identification algorithm performs well,since the test pattern is recognized in most cases.Besides,the algorithm can effectively remove the test pattern signal and retain strong precipitation echoes in heavy rainfall events.展开更多
The distance-based outlier detection method detects the implied outliers by calculating the distance of the points in the dataset, but the computational complexity is particularly high when processing multidimensional...The distance-based outlier detection method detects the implied outliers by calculating the distance of the points in the dataset, but the computational complexity is particularly high when processing multidimensional datasets. In addition, the traditional outlier detection method does not consider the frequency of subsets occurrence, thus, the detected outliers do not fit the definition of outliers (i.e., rarely appearing). The pattern mining-based outlier detection approaches have solved this problem, but the importance of each pattern is not taken into account in outlier detection process, so the detected outliers cannot truly reflect some actual situation. Aimed at these problems, a two-phase minimal weighted rare pattern mining-based outlier detection approach, called MWRPM-Outlier, is proposed to effectively detect outliers on the weight data stream. In particular, a method called MWRPM is proposed in the pattern mining phase to fast mine the minimal weighted rare patterns, and then two deviation factors are defined in outlier detection phase to measure the abnormal degree of each transaction on the weight data stream. Experimental results show that the proposed MWRPM-Outlier approach has excellent performance in outlier detection and MWRPM approach outperforms in weighted rare pattern mining.展开更多
Based on the night light data, urban area data, and economic data of Wuhan Urban Agglomeration from 2009 to 2015, we use spatial correlation dimension, spatial self-correlation analysis and weighted standard deviation...Based on the night light data, urban area data, and economic data of Wuhan Urban Agglomeration from 2009 to 2015, we use spatial correlation dimension, spatial self-correlation analysis and weighted standard deviation ellipse to identify the general characteristics and dynamic evolution characteristics of urban spatial pattern and economic disparity pattern. The research results prove that: between 2009 and 2013, Wuhan Urban Agglomeration expanded gradually from northwest to southeast and presented the dynamic evolution features of “along the river and the road”. The spatial structure is obvious, forming the pattern of “core-periphery”. The development of Wuhan Urban Agglomeration has obvious imbalance in economic geography space, presenting the development tendency of “One prominent, stronger in the west and weaker in the east”. The contract within Wuhan Urban Agglomeration is gradually decreased. Wuhan city and its surrounding areas have stronger economic growth strength as well as the cities along The Yangtze River. However, the relative development rate of Wuhan city area is still far higher than other cities and counties.展开更多
In order to reduce the computational and spatial complexity in rerunning algorithm of sequential patterns query, this paper proposes sequential patterns based and projection database based algorithm for fast interacti...In order to reduce the computational and spatial complexity in rerunning algorithm of sequential patterns query, this paper proposes sequential patterns based and projection database based algorithm for fast interactive sequential patterns mining algorithm （FISP）, in which the number of frequent items of the projection databases constructed by the correct mining which based on the previously mined sequences has been reduced. Furthermore, the algorithm＇s iterative running times are reduced greatly by using global-threshold. The results of experiments testify that FISP outperforms PrefixSpan in interactive mining展开更多
Mining frequent pattern in transaction database, time series databases, and many other kinds of databases have been studied popularly in data mining research. Most of the previous studies adopt Apriori like candidat...Mining frequent pattern in transaction database, time series databases, and many other kinds of databases have been studied popularly in data mining research. Most of the previous studies adopt Apriori like candidate set generation and test approach. However, candidate set generation is very costly. Han J. proposed a novel algorithm FP growth that could generate frequent pattern without candidate set. Based on the analysis of the algorithm FP growth, this paper proposes a concept of equivalent FP tree and proposes an improved algorithm, denoted as FP growth * , which is much faster in speed, and easy to realize. FP growth * adopts a modified structure of FP tree and header table, and only generates a header table in each recursive operation and projects the tree to the original FP tree. The two algorithms get the same frequent pattern set in the same transaction database, but the performance study on computer shows that the speed of the improved algorithm, FP growth * , is at least two times as fast as that of FP growth.展开更多
It is nontrivial to maintain such discovered frequent query patterns in real XML-DBMS because the transaction database of queries may allow frequent updates and such updates may not only invalidate some existing frequ...It is nontrivial to maintain such discovered frequent query patterns in real XML-DBMS because the transaction database of queries may allow frequent updates and such updates may not only invalidate some existing frequent query patterns but also generate some new frequent query patterns. In this paper, two incremental updating algorithms, FUX-QMiner and FUXQMiner, are proposed for efficient maintenance of discovered frequent query patterns and generation the new frequent query patterns when new XMI, queries are added into the database. Experimental results from our implementation show that the proposed algorithms have good performance. Key words XML - frequent query pattern - incremental algorithm - data mining CLC number TP 311 Foudation item: Supported by the Youthful Foundation for Scientific Research of University of Shanghai for Science and TechnologyBiography: PENG Dun-lu (1974-), male, Associate professor, Ph.D, research direction: data mining, Web service and its application, peerto-peer computing.展开更多
The objective,connotations and research issues of big geodata mining were discussed to address its significance to geographical research in this paper.Big geodata may be categorized into two domains:big earth observat...The objective,connotations and research issues of big geodata mining were discussed to address its significance to geographical research in this paper.Big geodata may be categorized into two domains:big earth observation data and big human behavior data.A description of big geodata includes,in addition to the“5Vs”(volume,velocity,value,variety and veracity),a further five features,that is,granularity,scope,density,skewness and precision.Based on this approach,the essence of mining big geodata includes four aspects.First,flow space,where flow replaces points in traditional space,will become the new presentation form for big human behavior data.Second,the objectives for mining big geodata are the spatial patterns and the spatial relationships.Third,the spatiotemporal distributions of big geodata can be viewed as overlays of multiple geographic patterns and the characteristics of the data,namely heterogeneity and homogeneity,may change with scale.Fourth,data mining can be seen as a tool for discovery of geographic patterns and the patterns revealed may be attributed to human-land relationships.The big geodata mining methods may be categorized into two types in view of the mining objective,i.e.,classification mining and relationship mining.Future research will be faced by a number of issues,including the aggregation and connection of big geodata,the effective evaluation of the mining results and the challenge for mining to reveal“non-trivial”knowledge.展开更多
This paper proposes a new computer aided pattern design system(CAPDS)in which a uniformmathematical expression is used to construct different objects. In the geometric model, 3 B-splineis adopted as a line model. The ...This paper proposes a new computer aided pattern design system(CAPDS)in which a uniformmathematical expression is used to construct different objects. In the geometric model, 3 B-splineis adopted as a line model. The new system with uniform mathematical expression has the distin-guishing features: curve smoothness and fidelity, convenience to process graphic data in thedatabase and etc.. This paper presents the data structure, the program structure and also the implementation.展开更多
文摘The proliferation of textual data in society currently is overwhelming, in particular, unstructured textual data is being constantly generated via call centre logs, emails, documents on the web, blogs, tweets, customer comments, customer reviews, etc.While the amount of textual data is increasing rapidly, users ability to summarise, understand, and make sense of such data for making better business/living decisions remains challenging. This paper studies how to analyse textual data, based on layered software patterns, for extracting insightful user intelligence from a large collection of documents and for using such information to improve user operations and performance.
基金Dr. Steve Jones, Scientific Advisor of the Canon Foundation for Scientific Research (7200 The Quorum, Oxford Business Park, Oxford OX4 2JZ, England). Canon Foundation for Scientific Research funded the UPC 2013 tuition fees of the corresponding author during her writing this article
文摘In computational physics proton transfer phenomena could be viewed as pattern classification problems based on a set of input features allowing classification of the proton motion into two categories： transfer ＇occurred＇ and transfer ＇not occurred＇. The goal of this paper is to evaluate the use of artificial neural networks in the classification of proton transfer events, based on the feed-forward back propagation neural network, used as a classifier to distinguish between the two transfer cases. In this paper, we use a new developed data mining and pattern recognition tool for automating, controlling, and drawing charts of the output data of an Empirical Valence Bond existing code. The study analyzes the need for pattern recognition in aqueous proton transfer processes and how the learning approach in error back propagation （multilayer perceptron algorithms） could be satisfactorily employed in the present case. We present a tool for pattern recognition and validate the code including a real physical case study. The results of applying the artificial neural networks methodology to crowd patterns based upon selected physical properties （e.g., temperature, density） show the abilities of the network to learn proton transfer patterns corresponding to properties of the aqueous environments, which is in turn proved to be fully compatible with previous proton transfer studies.
基金supported by the National Grand Fundamental Research "973" Program of China (2004CB318109)the National High-Technology Research and Development Plan of China (2006AA01Z452)the National Information Security "242"Program of China (2005C39).
文摘Anomaly detection has been an active research topic in the field of network intrusion detection for many years. A novel method is presented for anomaly detection based on system calls into the kernels of Unix or Linux systems. The method uses the data mining technique to model the normal behavior of a privileged program and uses a variable-length pattern matching algorithm to perform the comparison of the current behavior and historic normal behavior, which is more suitable for this problem than the fixed-length pattern matching algorithm proposed by Forrest et al. At the detection stage, the particularity of the audit data is taken into account, and two alternative schemes could be used to distinguish between normalities and intrusions. The method gives attention to both computational efficiency and detection accuracy and is especially applicable for on-line detection. The performance of the method is evaluated using the typical testing data set, and the results show that it is significantly better than the anomaly detection method based on hidden Markov models proposed by Yan et al. and the method based on fixed-length patterns proposed by Forrest and Hofmeyr. The novel method has been applied to practical hosted-based intrusion detection systems and achieved high detection performance.
基金Supported by Projects of International Cooperation and Exchange of the National Natural Science Foundation of China(51561135003)Key Project of National Natural Science Foundation of China(51338003)Scientific Research Foundation of Graduated School of Southeast University(YBJJ1842)
文摘In order to explore the travel characteristics and space-time distribution of different groups of bikeshare users,an online analytical processing(OLAP)tool called data cube was used for treating and displaying multi-dimensional data.We extended and modified the traditionally threedimensional data cube into four dimensions,which are space,date,time,and user,each with a user-specified hierarchy,and took transaction numbers and travel time as two quantitative measures.The results suggest that there are two obvious transaction peaks during the morning and afternoon rush hours on weekdays,while the volume at weekends has an approximate even distribution.Bad weather condition significantly restricts the bikeshare usage.Besides,seamless smartcard users generally take a longer trip than exclusive smartcard users;and non-native users ride faster than native users.These findings not only support the applicability and efficiency of data cube in the field of visualizing massive smartcard data,but also raise equity concerns among bikeshare users with different demographic backgrounds.
基金supported by the National Key Program for Developing Basic Sciences under Grant 2012CB417202the National Natural Science Foundation of China under Grant No. 41175038, No. 41305088 and No. 41075023+4 种基金the Meteorological Special Project "Radar network observation technology and QC"the CMA Key project "Radar Operational Software Engineering"the Chinese Academy of Meteorological Sciences Basic ScientificOperational Projects "Observation and retrieval methods of micro-physics and dynamic parameters of cloud and precipitation with multi-wavelength Remote Sensing"Project of the State Key Laboratory of Severe Weather grant 2012LASW-B04
文摘A variety of faulty radar echoes may cause serious problems with radar data applications,especially radar data assimilation and quantitative precipitation estimates.In this study,＂test pattern＂ caused by test signal or radar hardware failures in CINRAD （China New Generation Weather Radar） SA and SB radar operational observations are investigated.In order to distinguish the test pattern from other types of radar echoes,such as precipitation,clear air and other non-meteorological echoes,five feature parameters including the effective reflectivity data percentage （Rz）,velocity RF （range folding） data percentage （RRF）,missing velocity data percentage （RM）,averaged along-azimuth reflectivity fluctuation （RNr,z） and averaged along-beam reflectivity fluctuation （RNa,z） are proposed.Based on the fuzzy logic method,a test pattern identification algorithm is developed,and the statistical results from all the different kinds of radar echoes indicate the performance of the algorithm.Analysis of two typical cases with heavy precipitation echoes located inside the test pattern are performed.The statistical results show that the test pattern identification algorithm performs well,since the test pattern is recognized in most cases.Besides,the algorithm can effectively remove the test pattern signal and retain strong precipitation echoes in heavy rainfall events.
基金supported by Fundamental Research Funds for the Central Universities (No. 2018XD004)
文摘The distance-based outlier detection method detects the implied outliers by calculating the distance of the points in the dataset, but the computational complexity is particularly high when processing multidimensional datasets. In addition, the traditional outlier detection method does not consider the frequency of subsets occurrence, thus, the detected outliers do not fit the definition of outliers (i.e., rarely appearing). The pattern mining-based outlier detection approaches have solved this problem, but the importance of each pattern is not taken into account in outlier detection process, so the detected outliers cannot truly reflect some actual situation. Aimed at these problems, a two-phase minimal weighted rare pattern mining-based outlier detection approach, called MWRPM-Outlier, is proposed to effectively detect outliers on the weight data stream. In particular, a method called MWRPM is proposed in the pattern mining phase to fast mine the minimal weighted rare patterns, and then two deviation factors are defined in outlier detection phase to measure the abnormal degree of each transaction on the weight data stream. Experimental results show that the proposed MWRPM-Outlier approach has excellent performance in outlier detection and MWRPM approach outperforms in weighted rare pattern mining.
文摘Based on the night light data, urban area data, and economic data of Wuhan Urban Agglomeration from 2009 to 2015, we use spatial correlation dimension, spatial self-correlation analysis and weighted standard deviation ellipse to identify the general characteristics and dynamic evolution characteristics of urban spatial pattern and economic disparity pattern. The research results prove that: between 2009 and 2013, Wuhan Urban Agglomeration expanded gradually from northwest to southeast and presented the dynamic evolution features of “along the river and the road”. The spatial structure is obvious, forming the pattern of “core-periphery”. The development of Wuhan Urban Agglomeration has obvious imbalance in economic geography space, presenting the development tendency of “One prominent, stronger in the west and weaker in the east”. The contract within Wuhan Urban Agglomeration is gradually decreased. Wuhan city and its surrounding areas have stronger economic growth strength as well as the cities along The Yangtze River. However, the relative development rate of Wuhan city area is still far higher than other cities and counties.
基金Supported by the National Natural Science Funda-tion of China (70371015) andthe Natural Science Foundation of Jian-gsu Province (BK2004058)
文摘In order to reduce the computational and spatial complexity in rerunning algorithm of sequential patterns query, this paper proposes sequential patterns based and projection database based algorithm for fast interactive sequential patterns mining algorithm （FISP）, in which the number of frequent items of the projection databases constructed by the correct mining which based on the previously mined sequences has been reduced. Furthermore, the algorithm＇s iterative running times are reduced greatly by using global-threshold. The results of experiments testify that FISP outperforms PrefixSpan in interactive mining
基金theFundoftheNationalManagementBureauofTraditionalChineseMedicine(No .2 0 0 0 J P 5 4 )
文摘Mining frequent pattern in transaction database, time series databases, and many other kinds of databases have been studied popularly in data mining research. Most of the previous studies adopt Apriori like candidate set generation and test approach. However, candidate set generation is very costly. Han J. proposed a novel algorithm FP growth that could generate frequent pattern without candidate set. Based on the analysis of the algorithm FP growth, this paper proposes a concept of equivalent FP tree and proposes an improved algorithm, denoted as FP growth * , which is much faster in speed, and easy to realize. FP growth * adopts a modified structure of FP tree and header table, and only generates a header table in each recursive operation and projects the tree to the original FP tree. The two algorithms get the same frequent pattern set in the same transaction database, but the performance study on computer shows that the speed of the improved algorithm, FP growth * , is at least two times as fast as that of FP growth.
基金Supported by National Natural Science Foundation of China(60675039)National High Technology Research and Development Program of China(863 Program)(2006AA04Z217)Hundred Talents Program of Chinese Academy of Sciences
文摘It is nontrivial to maintain such discovered frequent query patterns in real XML-DBMS because the transaction database of queries may allow frequent updates and such updates may not only invalidate some existing frequent query patterns but also generate some new frequent query patterns. In this paper, two incremental updating algorithms, FUX-QMiner and FUXQMiner, are proposed for efficient maintenance of discovered frequent query patterns and generation the new frequent query patterns when new XMI, queries are added into the database. Experimental results from our implementation show that the proposed algorithms have good performance. Key words XML - frequent query pattern - incremental algorithm - data mining CLC number TP 311 Foudation item: Supported by the Youthful Foundation for Scientific Research of University of Shanghai for Science and TechnologyBiography: PENG Dun-lu (1974-), male, Associate professor, Ph.D, research direction: data mining, Web service and its application, peerto-peer computing.
基金National Natural Science Foundation of China,No.41525004,No.41421001。
文摘The objective,connotations and research issues of big geodata mining were discussed to address its significance to geographical research in this paper.Big geodata may be categorized into two domains:big earth observation data and big human behavior data.A description of big geodata includes,in addition to the“5Vs”(volume,velocity,value,variety and veracity),a further five features,that is,granularity,scope,density,skewness and precision.Based on this approach,the essence of mining big geodata includes four aspects.First,flow space,where flow replaces points in traditional space,will become the new presentation form for big human behavior data.Second,the objectives for mining big geodata are the spatial patterns and the spatial relationships.Third,the spatiotemporal distributions of big geodata can be viewed as overlays of multiple geographic patterns and the characteristics of the data,namely heterogeneity and homogeneity,may change with scale.Fourth,data mining can be seen as a tool for discovery of geographic patterns and the patterns revealed may be attributed to human-land relationships.The big geodata mining methods may be categorized into two types in view of the mining objective,i.e.,classification mining and relationship mining.Future research will be faced by a number of issues,including the aggregation and connection of big geodata,the effective evaluation of the mining results and the challenge for mining to reveal“non-trivial”knowledge.
文摘This paper proposes a new computer aided pattern design system(CAPDS)in which a uniformmathematical expression is used to construct different objects. In the geometric model, 3 B-splineis adopted as a line model. The new system with uniform mathematical expression has the distin-guishing features: curve smoothness and fidelity, convenience to process graphic data in thedatabase and etc.. This paper presents the data structure, the program structure and also the implementation.
