A direction-of-arrival （DOA） estimation algorithm based on direct data domain （D3） approach is presented. This method can accuracy estimate DOA using one snapshot modified data, called the temporal and spatial two...A direction-of-arrival （DOA） estimation algorithm based on direct data domain （D3） approach is presented. This method can accuracy estimate DOA using one snapshot modified data, called the temporal and spatial two-dimensional vector reconstruction （TSR） method. The key idea is to apply the D3 approach which can extract the signal of given frequency but null out other frequency signals in temporal domain. Then the spatial vector reconstruction processing is used to estimate the angle of the spatial coherent signal source based on extract signal data. Compared with the common temporal and spatial processing approach, the TSR method has a lower computational load, higher real-time performance, robustness and angular accuracy of DOA. The proposed algorithm can be directly applied to the phased array radar of coherent pulses. Simulation results demonstrate the performance of the proposed technique.展开更多
In non-homogeneous environment, traditional space-time adaptive processing doesn＇t effectively suppress interference and detect target, because the secondary data don＇ t exactly reflect the statistical characteristi...In non-homogeneous environment, traditional space-time adaptive processing doesn＇t effectively suppress interference and detect target, because the secondary data don＇ t exactly reflect the statistical characteristic of the range cell under test. A ravel methodology utilizing the direct data domain approach to space-time adaptive processing （ STAP ） in airbome radar non-homogeneous environments is presented. The deterministic least squares adaptive signal processing technique operates on a ＂snapshot-by-snapshot＂ basis to dethrone the adaptive adaptive weights for nulling interferences and estimating signal of interest （SOI）. Furthermore, this approach eliminates the requirement for estimating the covariance through the data of neighboring range cell, which eliminates calculating the inverse of covariance, and can be implemented to operate in real-time. Simulation results illustrate the efficiency of interference suppression in non-homogeneous environment.展开更多
A robust phase-only Direct Data Domain Least Squares (D3LS) algorithm based on gen- eralized Rayleigh quotient optimization using hybrid Genetic Algorithm (GA) is presented in this letter. The optimization efficiency ...A robust phase-only Direct Data Domain Least Squares (D3LS) algorithm based on gen- eralized Rayleigh quotient optimization using hybrid Genetic Algorithm (GA) is presented in this letter. The optimization efficiency and computational speed are improved via the hybrid GA com- posed of standard GA and Nelder-Mead simplex algorithms. First, the objective function, with a form of generalized Rayleigh quotient, is derived via the standard D3LS algorithm. It is then taken as a fitness function and the unknown phases of all adaptive weights are taken as decision variables. Then, the nonlinear optimization is performed via the hybrid GA to obtain the optimized solution of phase-only adaptive weights. As a phase-only adaptive algorithm, the proposed algorithm is sim- pler than conventional algorithms when it comes to hardware implementation. Moreover, it proc- esses only a single snapshot data as opposed to forming sample covariance matrix and operating matrix inversion. Simulation results show that the proposed algorithm has a good signal recovery and interferences nulling performance, which are superior to that of the phase-only D3LS algorithm based on standard GA.展开更多
With the explosive growth of data available, there is an urgent need to develop continuous data mining which reduces manual interaction evidently. A novel model for data mining is proposed in evolving environment. Fir...With the explosive growth of data available, there is an urgent need to develop continuous data mining which reduces manual interaction evidently. A novel model for data mining is proposed in evolving environment. First, some valid mining task schedules are generated, and then au tonomous and local mining are executed periodically, finally, previous results are merged and refined. The framework based on the model creates a communication mechanism to in corporate domain knowledge into continuous process through ontology service. The local and merge mining are transparent to the end user and heterogeneous data ,source by ontology. Experiments suggest that the framework should be useful in guiding the continuous mining process.展开更多
Virtual data center is a new form of cloud computing concept applied to data center. As one of the most important challenges, virtual data center embedding problem has attracted much attention from researchers. In dat...Virtual data center is a new form of cloud computing concept applied to data center. As one of the most important challenges, virtual data center embedding problem has attracted much attention from researchers. In data centers, energy issue is very important for the reality that data center energy consumption has increased by dozens of times in the last decade. In this paper, we are concerned about the cost-aware multi-domain virtual data center embedding problem. In order to solve this problem, this paper first addresses the energy consumption model. The model includes the energy consumption model of the virtual machine node and the virtual switch node, to quantify the energy consumption in the virtual data center embedding process. Based on the energy consumption model above, this paper presents a heuristic algorithm for cost-aware multi-domain virtual data center embedding. The algorithm consists of two steps: inter-domain embedding and intra-domain embedding. Inter-domain virtual data center embedding refers to dividing virtual data center requests into several slices to select the appropriate single data center. Intra-domain virtual data center refers to embedding virtual data center requests in each data center. We first propose an inter-domain virtual data center embedding algorithm based on label propagation to select the appropriate single data center. We then propose a cost-aware virtual data center embedding algorithm to perform the intra-domain data center embedding. Extensive simulation results show that our proposed algorithm in this paper can effectively reduce the energy consumption while ensuring the success ratio of embedding.展开更多
In this paper, a model-free approach is presented to design an observer-based fault detection system of linear continuoustime systems based on input and output data in the time domain. The core of the approach is to d...In this paper, a model-free approach is presented to design an observer-based fault detection system of linear continuoustime systems based on input and output data in the time domain. The core of the approach is to directly identify parameters of the observer-based residual generator based on a numerically reliable data equation obtained by filtering and sampling the input and output signals.展开更多
Recent advances in computing,communications,digital storage technologies,and high-throughput data-acquisition technologies,make it possible to gather and store incredible volumes of data.It creates unprecedented oppor...Recent advances in computing,communications,digital storage technologies,and high-throughput data-acquisition technologies,make it possible to gather and store incredible volumes of data.It creates unprecedented opportunities for large-scale knowledge discovery from database.Data mining is an emerging area of computational intelligence that offers new theories,techniques,and tools for processing large volumes of data,such as data analysis,decision making,etc.There are many researchers working on designing efficient data mining techniques,methods,and algorithms.Unfortunately,most data mining researchers pay much attention to technique problems for developing data mining models and methods,while little to basic issues of data mining.In this paper,we will propose a new understanding for data mining,that is,domain-oriented data-driven data mining(3DM)model.Some data-driven data mining algorithms developed in our Lab are also presented to show its validity.展开更多
Data mining (also known as Knowledge Discovery in Databases - KDD) is defined as the nontrivial extraction of implicit, previously unknown, and potentially useful information from data. The aims and objectives of data...Data mining (also known as Knowledge Discovery in Databases - KDD) is defined as the nontrivial extraction of implicit, previously unknown, and potentially useful information from data. The aims and objectives of data mining are to discover knowledge of interest to user needs.Data mining is really a useful tool in many domains such as marketing, decision making, etc. However, some basic issues of data mining are ignored. What is data mining? What is the product of a data mining process? What are we doing in a data mining process? Is there any rule we should obey in a data mining process? In order to discover patterns and knowledge really interesting and actionable to the real world Zhang et al proposed a domain-driven human-machine-cooperated data mining process.Zhao and Yao proposed an interactive user-driven classification method using the granule network. In our work, we find that data mining is a kind of knowledge transforming process to transform knowledge from data format into symbol format. Thus, no new knowledge could be generated (born) in a data mining process. In a data mining process, knowledge is just transformed from data format, which is not understandable for human, into symbol format,which is understandable for human and easy to be used.It is similar to the process of translating a book from Chinese into English.In this translating process,the knowledge itself in the book should remain unchanged. What will be changed is the format of the knowledge only. That is, the knowledge in the English book should be kept the same as the knowledge in the Chinese one.Otherwise, there must be some mistakes in the translating proces, that is, we are transforming knowledge from one format into another format while not producing new knowledge in a data mining process. The knowledge is originally stored in data (data is a representation format of knowledge). Unfortunately, we can not read, understand, or use it, since we can not understand data. With this understanding of data mining, we proposed a data-driven knowledge acquisition method based on rough sets. It also improved the performance of classical knowledge acquisition methods. In fact, we also find that the domain-driven data mining and user-driven data mining do not conflict with our data-driven data mining. They could be integrated into domain-oriented data-driven data mining. It is just like the views of data base. Users with different views could look at different partial data of a data base. Thus, users with different tasks or objectives wish, or could discover different knowledge (partial knowledge) from the same data base. However, all these partial knowledge should be originally existed in the data base. So, a domain-oriented data-driven data mining method would help us to extract the knowledge which is really existed in a data base, and really interesting and actionable to the real world.展开更多
In order to improve the performance of classifiers in subjective domains, this paper defines a metric to measure the quality of the subjectively labelled training data (QoSTD) by means of K-means clustering. Then, the...In order to improve the performance of classifiers in subjective domains, this paper defines a metric to measure the quality of the subjectively labelled training data (QoSTD) by means of K-means clustering. Then, the QoSTD is used as a weight of the predicted class scores to adjust the likelihoods of instances. Moreover, two measurements are defined to assess the performance of the classifiers trained by the subjective labelled data. The binary classifiers of Traditional Chinese Medicine (TCM) Zhengs are trained and retrained by the real-world data set, utilizing the support vector machine (SVM) and the discrimination analysis (DA) models, so as to verify the effectiveness of the proposed method. The experimental results show that the consistency of likelihoods of instances with the corresponding observations is increased notable for the classes, especially in the cases with the relatively low QoSTD training data set. The experimental results also indicate the solution how to eliminate the miss-labelled instances from the training data set to re-train the classifiers in the subjective domains.展开更多
Frequency-domain waveform seismic tomography includes modeling of wave propagation and full waveform inversion of correcting the initial velocity model. In the forward modeling, we use direct solution based on sparse ...Frequency-domain waveform seismic tomography includes modeling of wave propagation and full waveform inversion of correcting the initial velocity model. In the forward modeling, we use direct solution based on sparse matrix factorization, combined with nine-point finite-difference for the linear system of equations. In the waveform inversion, we use preconditioned gradient method where the preconditioner is provided by the diagonal of the approximate Hessian matrix. We successfully applied waveform inversion method from low to high frequency in two sets of Marmousi data. One is the data set generated by frequencydomain finite-difference modeling, and the other is the original Marmousi shots data set. The former result is very close to the true velocity model. In the original shots data set inversion, we replace the prior source with estimated source; the result is also acceptable, and consistent with the true model.展开更多
Cloud Computing as a disruptive technology, provides a dynamic, elastic and promising computing climate to tackle the challenges of big data processing and analytics. Hadoop and MapReduce are the widely used open sour...Cloud Computing as a disruptive technology, provides a dynamic, elastic and promising computing climate to tackle the challenges of big data processing and analytics. Hadoop and MapReduce are the widely used open source frameworks in Cloud Computing for storing and processing big data in the scalable fashion. Spark is the latest parallel computing engine working together with Hadoop that exceeds MapReduce performance via its in-memory computing and high level programming features. In this paper, we present our design and implementation of a productive, domain-specific big data analytics cloud platform on top of Hadoop and Spark. To increase user’s productivity, we created a variety of data processing templates to simplify the programming efforts. We have conducted experiments for its productivity and performance with a few basic but representative data processing algorithms in the petroleum industry. Geophysicists can use the platform to productively design and implement scalable seismic data processing algorithms without handling the details of data management and the complexity of parallelism. The Cloud platform generates a complete data processing application based on user’s kernel program and simple configurations, allocates resources and executes it in parallel on top of Spark and Hadoop.展开更多
大语言模型(large language model,LLM)技术热潮对数据质量的要求提升到了一个新的高度.在现实场景中,数据通常来源不同且高度相关.但由于数据隐私安全问题,跨域异质数据往往不允许集中共享,难以被LLM高效利用.鉴于此,提出了一种LLM和...大语言模型(large language model,LLM)技术热潮对数据质量的要求提升到了一个新的高度.在现实场景中,数据通常来源不同且高度相关.但由于数据隐私安全问题,跨域异质数据往往不允许集中共享,难以被LLM高效利用.鉴于此,提出了一种LLM和知识图谱(knowledge graph,KG)协同的跨域异质数据查询框架,在LLM+KG的范式下给出跨域异质数据查询的一个治理方案.为确保LLM能够适应多场景中的跨域异质数据,首先采用适配器对跨域异质数据进行融合,并构建相应的知识图谱.为提高查询效率,引入线性知识图,并提出同源知识图抽取算法HKGE来实现知识图谱的重构,可显著提高查询性能,确保跨域异质数据治理的高效性.进而,为保证多域数据查询的高可信度,提出可信候选子图匹配算法Trust HKGM,用于检验跨域同源数据的置信度计算和可信候选子图匹配,剔除低质量节点.最后,提出基于线性知识图提示的多域数据查询算法MKLGP,实现LLM+KG范式下的高效可信跨域查询.该方法在多个真实数据集上进行了广泛实验,验证了所提方法的有效性和高效性.展开更多
文摘A direction-of-arrival （DOA） estimation algorithm based on direct data domain （D3） approach is presented. This method can accuracy estimate DOA using one snapshot modified data, called the temporal and spatial two-dimensional vector reconstruction （TSR） method. The key idea is to apply the D3 approach which can extract the signal of given frequency but null out other frequency signals in temporal domain. Then the spatial vector reconstruction processing is used to estimate the angle of the spatial coherent signal source based on extract signal data. Compared with the common temporal and spatial processing approach, the TSR method has a lower computational load, higher real-time performance, robustness and angular accuracy of DOA. The proposed algorithm can be directly applied to the phased array radar of coherent pulses. Simulation results demonstrate the performance of the proposed technique.
文摘In non-homogeneous environment, traditional space-time adaptive processing doesn＇t effectively suppress interference and detect target, because the secondary data don＇ t exactly reflect the statistical characteristic of the range cell under test. A ravel methodology utilizing the direct data domain approach to space-time adaptive processing （ STAP ） in airbome radar non-homogeneous environments is presented. The deterministic least squares adaptive signal processing technique operates on a ＂snapshot-by-snapshot＂ basis to dethrone the adaptive adaptive weights for nulling interferences and estimating signal of interest （SOI）. Furthermore, this approach eliminates the requirement for estimating the covariance through the data of neighboring range cell, which eliminates calculating the inverse of covariance, and can be implemented to operate in real-time. Simulation results illustrate the efficiency of interference suppression in non-homogeneous environment.
基金Supported by the Natural Science Foundation of Jiangsu Province (No.BK2004016).
文摘A robust phase-only Direct Data Domain Least Squares (D3LS) algorithm based on gen- eralized Rayleigh quotient optimization using hybrid Genetic Algorithm (GA) is presented in this letter. The optimization efficiency and computational speed are improved via the hybrid GA com- posed of standard GA and Nelder-Mead simplex algorithms. First, the objective function, with a form of generalized Rayleigh quotient, is derived via the standard D3LS algorithm. It is then taken as a fitness function and the unknown phases of all adaptive weights are taken as decision variables. Then, the nonlinear optimization is performed via the hybrid GA to obtain the optimized solution of phase-only adaptive weights. As a phase-only adaptive algorithm, the proposed algorithm is sim- pler than conventional algorithms when it comes to hardware implementation. Moreover, it proc- esses only a single snapshot data as opposed to forming sample covariance matrix and operating matrix inversion. Simulation results show that the proposed algorithm has a good signal recovery and interferences nulling performance, which are superior to that of the phase-only D3LS algorithm based on standard GA.
基金Supported by the National Natural Science Foun-dation of China (60173058 ,70372024)
文摘With the explosive growth of data available, there is an urgent need to develop continuous data mining which reduces manual interaction evidently. A novel model for data mining is proposed in evolving environment. First, some valid mining task schedules are generated, and then au tonomous and local mining are executed periodically, finally, previous results are merged and refined. The framework based on the model creates a communication mechanism to in corporate domain knowledge into continuous process through ontology service. The local and merge mining are transparent to the end user and heterogeneous data ,source by ontology. Experiments suggest that the framework should be useful in guiding the continuous mining process.
基金supported in part by the following funding agencies of China:National Natural Science Foundation under Grant 61602050 and U1534201National Key Research and Development Program of China under Grant 2016QY01W0200
文摘Virtual data center is a new form of cloud computing concept applied to data center. As one of the most important challenges, virtual data center embedding problem has attracted much attention from researchers. In data centers, energy issue is very important for the reality that data center energy consumption has increased by dozens of times in the last decade. In this paper, we are concerned about the cost-aware multi-domain virtual data center embedding problem. In order to solve this problem, this paper first addresses the energy consumption model. The model includes the energy consumption model of the virtual machine node and the virtual switch node, to quantify the energy consumption in the virtual data center embedding process. Based on the energy consumption model above, this paper presents a heuristic algorithm for cost-aware multi-domain virtual data center embedding. The algorithm consists of two steps: inter-domain embedding and intra-domain embedding. Inter-domain virtual data center embedding refers to dividing virtual data center requests into several slices to select the appropriate single data center. Intra-domain virtual data center refers to embedding virtual data center requests in each data center. We first propose an inter-domain virtual data center embedding algorithm based on label propagation to select the appropriate single data center. We then propose a cost-aware virtual data center embedding algorithm to perform the intra-domain data center embedding. Extensive simulation results show that our proposed algorithm in this paper can effectively reduce the energy consumption while ensuring the success ratio of embedding.
基金This work was supported was supported in part by the European Union under grant NeCST.
文摘In this paper, a model-free approach is presented to design an observer-based fault detection system of linear continuoustime systems based on input and output data in the time domain. The core of the approach is to directly identify parameters of the observer-based residual generator based on a numerically reliable data equation obtained by filtering and sampling the input and output signals.
文摘Recent advances in computing,communications,digital storage technologies,and high-throughput data-acquisition technologies,make it possible to gather and store incredible volumes of data.It creates unprecedented opportunities for large-scale knowledge discovery from database.Data mining is an emerging area of computational intelligence that offers new theories,techniques,and tools for processing large volumes of data,such as data analysis,decision making,etc.There are many researchers working on designing efficient data mining techniques,methods,and algorithms.Unfortunately,most data mining researchers pay much attention to technique problems for developing data mining models and methods,while little to basic issues of data mining.In this paper,we will propose a new understanding for data mining,that is,domain-oriented data-driven data mining(3DM)model.Some data-driven data mining algorithms developed in our Lab are also presented to show its validity.
文摘Data mining (also known as Knowledge Discovery in Databases - KDD) is defined as the nontrivial extraction of implicit, previously unknown, and potentially useful information from data. The aims and objectives of data mining are to discover knowledge of interest to user needs.Data mining is really a useful tool in many domains such as marketing, decision making, etc. However, some basic issues of data mining are ignored. What is data mining? What is the product of a data mining process? What are we doing in a data mining process? Is there any rule we should obey in a data mining process? In order to discover patterns and knowledge really interesting and actionable to the real world Zhang et al proposed a domain-driven human-machine-cooperated data mining process.Zhao and Yao proposed an interactive user-driven classification method using the granule network. In our work, we find that data mining is a kind of knowledge transforming process to transform knowledge from data format into symbol format. Thus, no new knowledge could be generated (born) in a data mining process. In a data mining process, knowledge is just transformed from data format, which is not understandable for human, into symbol format,which is understandable for human and easy to be used.It is similar to the process of translating a book from Chinese into English.In this translating process,the knowledge itself in the book should remain unchanged. What will be changed is the format of the knowledge only. That is, the knowledge in the English book should be kept the same as the knowledge in the Chinese one.Otherwise, there must be some mistakes in the translating proces, that is, we are transforming knowledge from one format into another format while not producing new knowledge in a data mining process. The knowledge is originally stored in data (data is a representation format of knowledge). Unfortunately, we can not read, understand, or use it, since we can not understand data. With this understanding of data mining, we proposed a data-driven knowledge acquisition method based on rough sets. It also improved the performance of classical knowledge acquisition methods. In fact, we also find that the domain-driven data mining and user-driven data mining do not conflict with our data-driven data mining. They could be integrated into domain-oriented data-driven data mining. It is just like the views of data base. Users with different views could look at different partial data of a data base. Thus, users with different tasks or objectives wish, or could discover different knowledge (partial knowledge) from the same data base. However, all these partial knowledge should be originally existed in the data base. So, a domain-oriented data-driven data mining method would help us to extract the knowledge which is really existed in a data base, and really interesting and actionable to the real world.
文摘In order to improve the performance of classifiers in subjective domains, this paper defines a metric to measure the quality of the subjectively labelled training data (QoSTD) by means of K-means clustering. Then, the QoSTD is used as a weight of the predicted class scores to adjust the likelihoods of instances. Moreover, two measurements are defined to assess the performance of the classifiers trained by the subjective labelled data. The binary classifiers of Traditional Chinese Medicine (TCM) Zhengs are trained and retrained by the real-world data set, utilizing the support vector machine (SVM) and the discrimination analysis (DA) models, so as to verify the effectiveness of the proposed method. The experimental results show that the consistency of likelihoods of instances with the corresponding observations is increased notable for the classes, especially in the cases with the relatively low QoSTD training data set. The experimental results also indicate the solution how to eliminate the miss-labelled instances from the training data set to re-train the classifiers in the subjective domains.
基金Supported by the National Natural Science Foundation of China (69983005)
文摘Frequency-domain waveform seismic tomography includes modeling of wave propagation and full waveform inversion of correcting the initial velocity model. In the forward modeling, we use direct solution based on sparse matrix factorization, combined with nine-point finite-difference for the linear system of equations. In the waveform inversion, we use preconditioned gradient method where the preconditioner is provided by the diagonal of the approximate Hessian matrix. We successfully applied waveform inversion method from low to high frequency in two sets of Marmousi data. One is the data set generated by frequencydomain finite-difference modeling, and the other is the original Marmousi shots data set. The former result is very close to the true velocity model. In the original shots data set inversion, we replace the prior source with estimated source; the result is also acceptable, and consistent with the true model.
文摘Cloud Computing as a disruptive technology, provides a dynamic, elastic and promising computing climate to tackle the challenges of big data processing and analytics. Hadoop and MapReduce are the widely used open source frameworks in Cloud Computing for storing and processing big data in the scalable fashion. Spark is the latest parallel computing engine working together with Hadoop that exceeds MapReduce performance via its in-memory computing and high level programming features. In this paper, we present our design and implementation of a productive, domain-specific big data analytics cloud platform on top of Hadoop and Spark. To increase user’s productivity, we created a variety of data processing templates to simplify the programming efforts. We have conducted experiments for its productivity and performance with a few basic but representative data processing algorithms in the petroleum industry. Geophysicists can use the platform to productively design and implement scalable seismic data processing algorithms without handling the details of data management and the complexity of parallelism. The Cloud platform generates a complete data processing application based on user’s kernel program and simple configurations, allocates resources and executes it in parallel on top of Spark and Hadoop.
文摘大语言模型(large language model,LLM)技术热潮对数据质量的要求提升到了一个新的高度.在现实场景中,数据通常来源不同且高度相关.但由于数据隐私安全问题,跨域异质数据往往不允许集中共享,难以被LLM高效利用.鉴于此,提出了一种LLM和知识图谱(knowledge graph,KG)协同的跨域异质数据查询框架,在LLM+KG的范式下给出跨域异质数据查询的一个治理方案.为确保LLM能够适应多场景中的跨域异质数据,首先采用适配器对跨域异质数据进行融合,并构建相应的知识图谱.为提高查询效率,引入线性知识图,并提出同源知识图抽取算法HKGE来实现知识图谱的重构,可显著提高查询性能,确保跨域异质数据治理的高效性.进而,为保证多域数据查询的高可信度,提出可信候选子图匹配算法Trust HKGM,用于检验跨域同源数据的置信度计算和可信候选子图匹配,剔除低质量节点.最后,提出基于线性知识图提示的多域数据查询算法MKLGP,实现LLM+KG范式下的高效可信跨域查询.该方法在多个真实数据集上进行了广泛实验,验证了所提方法的有效性和高效性.
