Recently,internet stimulates the explosive progress of knowledge discovery in big volume data resource,to dig the valuable and hidden rules by computing.Simultaneously,the wireless channel measurement data reveals big...Recently,internet stimulates the explosive progress of knowledge discovery in big volume data resource,to dig the valuable and hidden rules by computing.Simultaneously,the wireless channel measurement data reveals big volume feature,considering the massive antennas,huge bandwidth and versatile application scenarios.This article firstly presents a comprehensive survey of channel measurement and modeling research for mobile communication,especially for 5th Generation(5G) and beyond.Considering the big data research progress,then a cluster-nuclei based model is proposed,which takes advantages of both the stochastical model and deterministic model.The novel model has low complexity with the limited number of cluster-nuclei while the cluster-nuclei has the physical mapping to real propagation objects.Combining the channel properties variation principles with antenna size,frequency,mobility and scenario dug from the channel data,the proposed model can be expanded in versatile application to support future mobile research.展开更多
These last years we have been witnessing a tremendous growth in the volume and availability of data. This fact results primarily from the emergence of a multitude of sources (e.g. computers, mobile devices, sensors or...These last years we have been witnessing a tremendous growth in the volume and availability of data. This fact results primarily from the emergence of a multitude of sources (e.g. computers, mobile devices, sensors or social networks) that are continuously producing either structured, semi-structured or unstructured data. Database Management Systems and Data Warehouses are no longer the only technologies used to store and analyze datasets, namely due to the volume and complex structure of nowadays data that degrade their performance and scalability. Big Data is one of the recent challenges, since it implies new requirements in terms of data storage, processing and visualization. Despite that, analyzing properly Big Data can constitute great advantages because it allows discovering patterns and correlations in datasets. Users can use this processed information to gain deeper insights and to get business advantages. Thus, data modeling and data analytics are evolved in a way that we are able to process huge amounts of data without compromising performance and availability, but instead by “relaxing” the usual ACID properties. This paper provides a broad view and discussion of the current state of this subject with a particular focus on data modeling and data analytics, describing and clarifying the main differences between the three main approaches in what concerns these aspects, namely: operational databases, decision support databases and Big Data technologies.展开更多
This paper provides a new obstacle avoidance control method for cars based on big data and just-in-time modeling. Just-in-time modeling is a new kind of data-driven control technique in the age of big data and is used...This paper provides a new obstacle avoidance control method for cars based on big data and just-in-time modeling. Just-in-time modeling is a new kind of data-driven control technique in the age of big data and is used in various real systems. The main property of the proposed method is that a gain and a control time which are parameters in the control input to avoid an encountered obstacle are computed from a database which includes a lot of driving data in various situations. Especially, the important advantage of the method is small computation time, and hence it realizes real-time obstacle avoidance control for cars. From some numerical simulations, it is showed that the new control method can make the car avoid various obstacles efficiently in comparison with the previous method.展开更多
The 19th National Congress of the Communist Party of China has put forward higher requirements for Chinese government governance. The government governance has developed to a higher stage. Meanwhile, it faces more cha...The 19th National Congress of the Communist Party of China has put forward higher requirements for Chinese government governance. The government governance has developed to a higher stage. Meanwhile, it faces more challenges, like lack of top-level design and information sharing. To develop a government governance decision-making innovation model, we should make good use of big data to mine in the grassroots government data management network. Both the characteristics of the times and the experience of the practice have proven that big data can empower government governance and promote the construction of a service-oriented government.展开更多
Facing the development of future 5 G, the emerging technologies such as Internet of things, big data, cloud computing, and artificial intelligence is enhancing an explosive growth in data traffic. Radical changes in c...Facing the development of future 5 G, the emerging technologies such as Internet of things, big data, cloud computing, and artificial intelligence is enhancing an explosive growth in data traffic. Radical changes in communication theory and implement technologies, the wireless communications and wireless networks have entered a new era. Among them, wireless big data(WBD) has tremendous value, and artificial intelligence(AI) gives unthinkable possibilities. However, in the big data development and artificial intelligence application groups, the lack of a sound theoretical foundation and mathematical methods is regarded as a real challenge that needs to be solved. From the basic problem of wireless communication, the interrelationship of demand, environment and ability, this paper intends to investigate the concept and data model of WBD, the wireless data mining, the wireless knowledge and wireless knowledge learning(WKL), and typical practices examples, to facilitate and open up more opportunities of WBD research and developments. Such research is beneficial for creating new theoretical foundation and emerging technologies of future wireless communications.展开更多
The proliferation of textual data in society currently is overwhelming, in particular, unstructured textual data is being constantly generated via call centre logs, emails, documents on the web, blogs, tweets, custome...The proliferation of textual data in society currently is overwhelming, in particular, unstructured textual data is being constantly generated via call centre logs, emails, documents on the web, blogs, tweets, customer comments, customer reviews, etc.While the amount of textual data is increasing rapidly, users ability to summarise, understand, and make sense of such data for making better business/living decisions remains challenging. This paper studies how to analyse textual data, based on layered software patterns, for extracting insightful user intelligence from a large collection of documents and for using such information to improve user operations and performance.展开更多
Damage caused by people and organizations unconnected with the pipeline management is a major risk faced by pipelines,and its consequences can have a huge impact.However,the present measures to monitor this have major...Damage caused by people and organizations unconnected with the pipeline management is a major risk faced by pipelines,and its consequences can have a huge impact.However,the present measures to monitor this have major problems such as time delays,overlooking threats,and false alarms.To overcome the disadvantages of these methods,analysis of big location data from mobile phone systems was applied to prevent third-party damage to pipelines,and a third-party damage prevention system was developed for pipelines including encryption mobile phone data,data preprocessing,and extraction of characteristic patterns.By applying this to natural gas pipelines,a large amount of location data was collected for data feature recognition and model analysis.Third-party illegal construction and occupation activities were discovered in a timely manner.This is important for preventing third-party damage to pipelines.展开更多
Big data challenges current information technologies (IT landscape) while promising a more competitive and efficient contributions to business organizations. What big data can contribute to is what organizations have ...Big data challenges current information technologies (IT landscape) while promising a more competitive and efficient contributions to business organizations. What big data can contribute to is what organizations have been wanted for a long time ago. This paper presents the nature of big data and how organizations can advance their systems with big data technologies. By improving the efficiency and effectiveness of organizations, people can benefit the can take advantages of a more convenient life contributed by Information Technology.展开更多
We all live on one planet and geology has no borders.Countries that reside on different continents share the same architecture beneath the surface;they were once neighbors with common foundations.Interoperable geologi...We all live on one planet and geology has no borders.Countries that reside on different continents share the same architecture beneath the surface;they were once neighbors with common foundations.Interoperable geological data are now freely available to everyone for the benefit of society,demonstrating that geoscience can address both global and regional problems.Whilst increasingly large datasets("Big Data")provide clear opportunities(e.g.,Spina,2018).展开更多
Big data is becoming a well-known buzzword and in active use in many areas. Because of the velocity, variety, and volume of big data, security and privacy issues are magnified, which results in the traditional protect...Big data is becoming a well-known buzzword and in active use in many areas. Because of the velocity, variety, and volume of big data, security and privacy issues are magnified, which results in the traditional protection mechanisms for structured small scale data are inadequate for big data. Sensitivities around big data security and privacy are a hurdle that organizations need to overcome. In this paper, we review the current data security in big data and analysis its feasibilities and obstacles. Besides, we also introduced intelligent analytics to enhance security with the proposed security intelligence model. This research aims to summarize, organize and classify the information available in the literature to identify any gaps in current research and suggest areas for scholars and security researchers for further investigation.展开更多
Construction project is not a standalone engineering maneuver.It is closely linked to the well-being of local communities in concern.The city renovation in Beijing down center for Olympic 2008 transformed many antique...Construction project is not a standalone engineering maneuver.It is closely linked to the well-being of local communities in concern.The city renovation in Beijing down center for Olympic 2008 transformed many antique architecture and regional landscape.It gave a world-recognized achievement in China s modem development and manifested a major milestone in China's economic development.In the course of metro construction projects,there are substantial interwoven municipal structures influencing the success of the projects,which including,but the least,all underground cables and ducts,sewage system,the power consumption of construction works,traffic diversion,air pollution,expatriate business activities and social security.There are many US and UK project insurance companies moving into Asia Pacific.They are doing re-insurance business on major construction guarantee,such as machinery damage,project on-time,power consumption,claims from contractors and communities.Environmental information,such as water quality,indoor and outdoor air quality,people inflow and lift waiting time play deterministic roles in construction's fit-touse.Big Data is a contemporary buzzword since 2013,and the key competence is to provide real time response to heuristic syndrome in order to make short-term prediction.This paper attempts to develop a conceptual model in big data for construction展开更多
There are a lot of biological and experimental data from genomics, proteomics, drug screening, medicinal chemistry, etc. A large amount of data must be analyzed by special methods of statistics, bioinformatics, and co...There are a lot of biological and experimental data from genomics, proteomics, drug screening, medicinal chemistry, etc. A large amount of data must be analyzed by special methods of statistics, bioinformatics, and computer science. Big data analysis is an effective way to build scientific hypothesis and explore internal mechanism.Here, gene expression is taken as an example to illustrate the basic procedure of the big data analysis.展开更多
Quantitative analysis of digital images requires detection and segmentation of the borders of the object of interest. Accurate segmentation is required for volume determination, 3D rendering, radiation therapy, and su...Quantitative analysis of digital images requires detection and segmentation of the borders of the object of interest. Accurate segmentation is required for volume determination, 3D rendering, radiation therapy, and surgery planning. In medical images, segmentation has traditionally been done by human experts. Substantial computational and storage requirements become especially acute when object orientation and scale have to be considered. Therefore, automated or semi-automated segmentation techniques are essential if these software applications are ever to gain widespread clinical use. Many methods have been proposed to detect and segment 2D shapes, most of which involve template matching. Advanced segmentation techniques called Snakes or active contours have been used, considering deformable models or templates. The main purpose of this work is to apply segmentation techniques for the definition of 3D organs (anatomical structures) when big data information has been stored and must be organized by the doctors for medical diagnosis. The processes would be implemented in the CT images from patients with COVID-19.展开更多
As more and more application systems related to big data were developed, NoSQL (Not Only SQL) database systems are becoming more and more popular. In order to add transaction features for some NoSQL database systems, ...As more and more application systems related to big data were developed, NoSQL (Not Only SQL) database systems are becoming more and more popular. In order to add transaction features for some NoSQL database systems, many scholars have tried different techniques. Unfortunately, there is a lack of research on Redis’s transaction in the existing literatures. This paper proposes a transaction model for key-value NoSQL databases including Redis to make possible allowing users to access data in the ACID (Atomicity, Consistency, Isolation and Durability) way, and this model is vividly called the surfing concurrence transaction model. The architecture, important features and implementation principle are described in detail. The key algorithms also were given in the form of pseudo program code, and the performance also was evaluated. With the proposed model, the transactions of Key-Value NoSQL databases can be performed in a lock free and MVCC (Multi-Version Concurrency Control) free manner. This is the result of further research on the related topic, which fills the gap ignored by relevant scholars in this field to make a little contribution to the further development of NoSQL technology.展开更多
In this paper, the statistical properties of parameters of each path in wireless channel models are analyzed to prove that there is the static part in channel state information(CSI) which can be extracted from huge am...In this paper, the statistical properties of parameters of each path in wireless channel models are analyzed to prove that there is the static part in channel state information(CSI) which can be extracted from huge amounts of CSI data. Based on the analysis, the concept of the Tomographic Channel Model(TCM) is presented. With cluster algorithms, the static CSI database can be built in an off-line manner. The static CSI database can provide prior information to help pilot design to reduce overhead and improve accuracy in channel estimation. A new CSI prediction method and a new channel estimation method between different frequency bands are introduced based on the static CSI database. Using measurement data, the performance of the new channel prediction method is compared with that of the Auto Regression(AR) predictor. The results indicate that the prediction range of the new method is better than that of the AR method and the new method can predict with fewer pilot symbols. Using measurement data, the new channel estimation method between different frequency bands can estimate the CSI of one frequency band based on known CSI of another frequency band without any feedback.展开更多
A large amount of information is frequently encountered when characterizing the sample model in chemical process.A fault diagnosis method based on dynamic modeling of feature engineering is proposed to effectively rem...A large amount of information is frequently encountered when characterizing the sample model in chemical process.A fault diagnosis method based on dynamic modeling of feature engineering is proposed to effectively remove the nonlinear correlation redundancy of chemical process in this paper.From the whole process point of view,the method makes use of the characteristic of mutual information to select the optimal variable subset.It extracts the correlation among variables in the whitening process without limiting to only linear correlations.Further,PCA(Principal Component Analysis)dimension reduction is used to extract feature subset before fault diagnosis.The application results of the TE(Tennessee Eastman)simulation process show that the dynamic modeling process of MIFE(Mutual Information Feature Engineering)can accurately extract the nonlinear correlation relationship among process variables and can effectively reduce the dimension of feature detection in process monitoring.展开更多
Challenges in Big Data analysis arise due to the way the data are recorded, maintained, processed and stored. We demonstrate that a hierarchical, multivariate, statistical machine learning algorithm, namely Boosted Re...Challenges in Big Data analysis arise due to the way the data are recorded, maintained, processed and stored. We demonstrate that a hierarchical, multivariate, statistical machine learning algorithm, namely Boosted Regression Tree (BRT) can address Big Data challenges to drive decision making. The challenge of this study is lack of interoperability since the data, a collection of GIS shapefiles, remotely sensed imagery, and aggregated and interpolated spatio-temporal information, are stored in monolithic hardware components. For the modelling process, it was necessary to create one common input file. By merging the data sources together, a structured but noisy input file, showing inconsistencies and redundancies, was created. Here, it is shown that BRT can process different data granularities, heterogeneous data and missingness. In particular, BRT has the advantage of dealing with missing data by default by allowing a split on whether or not a value is missing as well as what the value is. Most importantly, the BRT offers a wide range of possibilities regarding the interpretation of results and variable selection is automatically performed by considering how frequently a variable is used to define a split in the tree. A comparison with two similar regression models (Random Forests and Least Absolute Shrinkage and Selection Operator, LASSO) shows that BRT outperforms these in this instance. BRT can also be a starting point for sophisticated hierarchical modelling in real world scenarios. For example, a single or ensemble approach of BRT could be tested with existing models in order to improve results for a wide range of data-driven decisions and applications.展开更多
基金supported in part by National Natural Science Foundation of China (61322110, 6141101115)Doctoral Fund of Ministry of Education (201300051100013)
文摘Recently,internet stimulates the explosive progress of knowledge discovery in big volume data resource,to dig the valuable and hidden rules by computing.Simultaneously,the wireless channel measurement data reveals big volume feature,considering the massive antennas,huge bandwidth and versatile application scenarios.This article firstly presents a comprehensive survey of channel measurement and modeling research for mobile communication,especially for 5th Generation(5G) and beyond.Considering the big data research progress,then a cluster-nuclei based model is proposed,which takes advantages of both the stochastical model and deterministic model.The novel model has low complexity with the limited number of cluster-nuclei while the cluster-nuclei has the physical mapping to real propagation objects.Combining the channel properties variation principles with antenna size,frequency,mobility and scenario dug from the channel data,the proposed model can be expanded in versatile application to support future mobile research.
文摘These last years we have been witnessing a tremendous growth in the volume and availability of data. This fact results primarily from the emergence of a multitude of sources (e.g. computers, mobile devices, sensors or social networks) that are continuously producing either structured, semi-structured or unstructured data. Database Management Systems and Data Warehouses are no longer the only technologies used to store and analyze datasets, namely due to the volume and complex structure of nowadays data that degrade their performance and scalability. Big Data is one of the recent challenges, since it implies new requirements in terms of data storage, processing and visualization. Despite that, analyzing properly Big Data can constitute great advantages because it allows discovering patterns and correlations in datasets. Users can use this processed information to gain deeper insights and to get business advantages. Thus, data modeling and data analytics are evolved in a way that we are able to process huge amounts of data without compromising performance and availability, but instead by “relaxing” the usual ACID properties. This paper provides a broad view and discussion of the current state of this subject with a particular focus on data modeling and data analytics, describing and clarifying the main differences between the three main approaches in what concerns these aspects, namely: operational databases, decision support databases and Big Data technologies.
文摘This paper provides a new obstacle avoidance control method for cars based on big data and just-in-time modeling. Just-in-time modeling is a new kind of data-driven control technique in the age of big data and is used in various real systems. The main property of the proposed method is that a gain and a control time which are parameters in the control input to avoid an encountered obstacle are computed from a database which includes a lot of driving data in various situations. Especially, the important advantage of the method is small computation time, and hence it realizes real-time obstacle avoidance control for cars. From some numerical simulations, it is showed that the new control method can make the car avoid various obstacles efficiently in comparison with the previous method.
文摘The 19th National Congress of the Communist Party of China has put forward higher requirements for Chinese government governance. The government governance has developed to a higher stage. Meanwhile, it faces more challenges, like lack of top-level design and information sharing. To develop a government governance decision-making innovation model, we should make good use of big data to mine in the grassroots government data management network. Both the characteristics of the times and the experience of the practice have proven that big data can empower government governance and promote the construction of a service-oriented government.
文摘Facing the development of future 5 G, the emerging technologies such as Internet of things, big data, cloud computing, and artificial intelligence is enhancing an explosive growth in data traffic. Radical changes in communication theory and implement technologies, the wireless communications and wireless networks have entered a new era. Among them, wireless big data(WBD) has tremendous value, and artificial intelligence(AI) gives unthinkable possibilities. However, in the big data development and artificial intelligence application groups, the lack of a sound theoretical foundation and mathematical methods is regarded as a real challenge that needs to be solved. From the basic problem of wireless communication, the interrelationship of demand, environment and ability, this paper intends to investigate the concept and data model of WBD, the wireless data mining, the wireless knowledge and wireless knowledge learning(WKL), and typical practices examples, to facilitate and open up more opportunities of WBD research and developments. Such research is beneficial for creating new theoretical foundation and emerging technologies of future wireless communications.
文摘The proliferation of textual data in society currently is overwhelming, in particular, unstructured textual data is being constantly generated via call centre logs, emails, documents on the web, blogs, tweets, customer comments, customer reviews, etc.While the amount of textual data is increasing rapidly, users ability to summarise, understand, and make sense of such data for making better business/living decisions remains challenging. This paper studies how to analyse textual data, based on layered software patterns, for extracting insightful user intelligence from a large collection of documents and for using such information to improve user operations and performance.
基金supported by Pipeline Management Data Analysis and Typical Model Research [Grant Number 2016B-3105-0501]CNPC (China National Petroleum Corporation) project, Research on Oil and Gas Pipeline Safety and Reliability Operating [Grant Number 2015-B025-0628]
文摘Damage caused by people and organizations unconnected with the pipeline management is a major risk faced by pipelines,and its consequences can have a huge impact.However,the present measures to monitor this have major problems such as time delays,overlooking threats,and false alarms.To overcome the disadvantages of these methods,analysis of big location data from mobile phone systems was applied to prevent third-party damage to pipelines,and a third-party damage prevention system was developed for pipelines including encryption mobile phone data,data preprocessing,and extraction of characteristic patterns.By applying this to natural gas pipelines,a large amount of location data was collected for data feature recognition and model analysis.Third-party illegal construction and occupation activities were discovered in a timely manner.This is important for preventing third-party damage to pipelines.
文摘Big data challenges current information technologies (IT landscape) while promising a more competitive and efficient contributions to business organizations. What big data can contribute to is what organizations have been wanted for a long time ago. This paper presents the nature of big data and how organizations can advance their systems with big data technologies. By improving the efficiency and effectiveness of organizations, people can benefit the can take advantages of a more convenient life contributed by Information Technology.
基金granted by National Natural Science Foundation of China(Grant Nos.41572154,41820104004)the National Key R&D Plan(Grant No.2017YFC0601405)the Strategic Priority Research Program(B)of the Chinese Academy of Sciences(Grant No.XDB18000000).
文摘We all live on one planet and geology has no borders.Countries that reside on different continents share the same architecture beneath the surface;they were once neighbors with common foundations.Interoperable geological data are now freely available to everyone for the benefit of society,demonstrating that geoscience can address both global and regional problems.Whilst increasingly large datasets("Big Data")provide clear opportunities(e.g.,Spina,2018).
文摘Big data is becoming a well-known buzzword and in active use in many areas. Because of the velocity, variety, and volume of big data, security and privacy issues are magnified, which results in the traditional protection mechanisms for structured small scale data are inadequate for big data. Sensitivities around big data security and privacy are a hurdle that organizations need to overcome. In this paper, we review the current data security in big data and analysis its feasibilities and obstacles. Besides, we also introduced intelligent analytics to enhance security with the proposed security intelligence model. This research aims to summarize, organize and classify the information available in the literature to identify any gaps in current research and suggest areas for scholars and security researchers for further investigation.
文摘Construction project is not a standalone engineering maneuver.It is closely linked to the well-being of local communities in concern.The city renovation in Beijing down center for Olympic 2008 transformed many antique architecture and regional landscape.It gave a world-recognized achievement in China s modem development and manifested a major milestone in China's economic development.In the course of metro construction projects,there are substantial interwoven municipal structures influencing the success of the projects,which including,but the least,all underground cables and ducts,sewage system,the power consumption of construction works,traffic diversion,air pollution,expatriate business activities and social security.There are many US and UK project insurance companies moving into Asia Pacific.They are doing re-insurance business on major construction guarantee,such as machinery damage,project on-time,power consumption,claims from contractors and communities.Environmental information,such as water quality,indoor and outdoor air quality,people inflow and lift waiting time play deterministic roles in construction's fit-touse.Big Data is a contemporary buzzword since 2013,and the key competence is to provide real time response to heuristic syndrome in order to make short-term prediction.This paper attempts to develop a conceptual model in big data for construction
文摘There are a lot of biological and experimental data from genomics, proteomics, drug screening, medicinal chemistry, etc. A large amount of data must be analyzed by special methods of statistics, bioinformatics, and computer science. Big data analysis is an effective way to build scientific hypothesis and explore internal mechanism.Here, gene expression is taken as an example to illustrate the basic procedure of the big data analysis.
文摘Quantitative analysis of digital images requires detection and segmentation of the borders of the object of interest. Accurate segmentation is required for volume determination, 3D rendering, radiation therapy, and surgery planning. In medical images, segmentation has traditionally been done by human experts. Substantial computational and storage requirements become especially acute when object orientation and scale have to be considered. Therefore, automated or semi-automated segmentation techniques are essential if these software applications are ever to gain widespread clinical use. Many methods have been proposed to detect and segment 2D shapes, most of which involve template matching. Advanced segmentation techniques called Snakes or active contours have been used, considering deformable models or templates. The main purpose of this work is to apply segmentation techniques for the definition of 3D organs (anatomical structures) when big data information has been stored and must be organized by the doctors for medical diagnosis. The processes would be implemented in the CT images from patients with COVID-19.
文摘As more and more application systems related to big data were developed, NoSQL (Not Only SQL) database systems are becoming more and more popular. In order to add transaction features for some NoSQL database systems, many scholars have tried different techniques. Unfortunately, there is a lack of research on Redis’s transaction in the existing literatures. This paper proposes a transaction model for key-value NoSQL databases including Redis to make possible allowing users to access data in the ACID (Atomicity, Consistency, Isolation and Durability) way, and this model is vividly called the surfing concurrence transaction model. The architecture, important features and implementation principle are described in detail. The key algorithms also were given in the form of pseudo program code, and the performance also was evaluated. With the proposed model, the transactions of Key-Value NoSQL databases can be performed in a lock free and MVCC (Multi-Version Concurrency Control) free manner. This is the result of further research on the related topic, which fills the gap ignored by relevant scholars in this field to make a little contribution to the further development of NoSQL technology.
基金supported by the National Natural Science Foundation of China (No.61631013)National Key Basic Research Program of China (973 Program)(No. 2013CB329002)National Major Project (NO. 2018ZX03001006003)
文摘In this paper, the statistical properties of parameters of each path in wireless channel models are analyzed to prove that there is the static part in channel state information(CSI) which can be extracted from huge amounts of CSI data. Based on the analysis, the concept of the Tomographic Channel Model(TCM) is presented. With cluster algorithms, the static CSI database can be built in an off-line manner. The static CSI database can provide prior information to help pilot design to reduce overhead and improve accuracy in channel estimation. A new CSI prediction method and a new channel estimation method between different frequency bands are introduced based on the static CSI database. Using measurement data, the performance of the new channel prediction method is compared with that of the Auto Regression(AR) predictor. The results indicate that the prediction range of the new method is better than that of the AR method and the new method can predict with fewer pilot symbols. Using measurement data, the new channel estimation method between different frequency bands can estimate the CSI of one frequency band based on known CSI of another frequency band without any feedback.
基金Supported by the National Natural Science Foundation of China(21576143).
文摘A large amount of information is frequently encountered when characterizing the sample model in chemical process.A fault diagnosis method based on dynamic modeling of feature engineering is proposed to effectively remove the nonlinear correlation redundancy of chemical process in this paper.From the whole process point of view,the method makes use of the characteristic of mutual information to select the optimal variable subset.It extracts the correlation among variables in the whitening process without limiting to only linear correlations.Further,PCA(Principal Component Analysis)dimension reduction is used to extract feature subset before fault diagnosis.The application results of the TE(Tennessee Eastman)simulation process show that the dynamic modeling process of MIFE(Mutual Information Feature Engineering)can accurately extract the nonlinear correlation relationship among process variables and can effectively reduce the dimension of feature detection in process monitoring.
文摘Challenges in Big Data analysis arise due to the way the data are recorded, maintained, processed and stored. We demonstrate that a hierarchical, multivariate, statistical machine learning algorithm, namely Boosted Regression Tree (BRT) can address Big Data challenges to drive decision making. The challenge of this study is lack of interoperability since the data, a collection of GIS shapefiles, remotely sensed imagery, and aggregated and interpolated spatio-temporal information, are stored in monolithic hardware components. For the modelling process, it was necessary to create one common input file. By merging the data sources together, a structured but noisy input file, showing inconsistencies and redundancies, was created. Here, it is shown that BRT can process different data granularities, heterogeneous data and missingness. In particular, BRT has the advantage of dealing with missing data by default by allowing a split on whether or not a value is missing as well as what the value is. Most importantly, the BRT offers a wide range of possibilities regarding the interpretation of results and variable selection is automatically performed by considering how frequently a variable is used to define a split in the tree. A comparison with two similar regression models (Random Forests and Least Absolute Shrinkage and Selection Operator, LASSO) shows that BRT outperforms these in this instance. BRT can also be a starting point for sophisticated hierarchical modelling in real world scenarios. For example, a single or ensemble approach of BRT could be tested with existing models in order to improve results for a wide range of data-driven decisions and applications.
