The development of forecasting models for pollution particles shows a nonlinear dynamic behavior;hence, implementation is a non-trivial process. In the literature, there have been multiple models of particulate pollut...The development of forecasting models for pollution particles shows a nonlinear dynamic behavior;hence, implementation is a non-trivial process. In the literature, there have been multiple models of particulate pollutants, which use softcomputing techniques and machine learning such as: multilayer perceptrons, neural networks, support vector machines, kernel algorithms, and so on. This paper presents a prediction pollution model using support vector machines and kernel functions, which are: Gaussian, Polynomial and Spline. Finally, the prediction results of ozone (O3), particulate matter (PM10) and nitrogen dioxide (NO2) at Mexico City are presented as a case study using these techniques.展开更多
A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controlled plant i...A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controlled plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.展开更多
A method of applying support vector machine (SVM) in speech recognition was proposed, and a speech recognition system for mandarin digits was built up by SVMs. In the system, vectors were linearly extracted from speec...A method of applying support vector machine (SVM) in speech recognition was proposed, and a speech recognition system for mandarin digits was built up by SVMs. In the system, vectors were linearly extracted from speech feature sequence to make up time-aligned input patterns for SVM, and the decisions of several 2-class SVM classifiers were employed for constructing an N-class classifier. Four kinds of SVM kernel functions were compared in the experiments of speaker-independent speech recognition of mandarin digits. And the kernel of radial basis function has the highest accurate rate of 99.33%, which is better than that of the baseline system based on hidden Markov models (HMM) (97.08%). And the experiments also show that SVM can outperform HMM especially when the samples for learning were very limited.展开更多
A new filtering method for SAR data de-noising using wavelet support vector regression （WSVR） is developed. On the basis of the grey scale distribution character of SAR imagery, the logarithmic SAR image as a noise ...A new filtering method for SAR data de-noising using wavelet support vector regression （WSVR） is developed. On the basis of the grey scale distribution character of SAR imagery, the logarithmic SAR image as a noise polluted signal is taken and the noise model assumption in logarithmic domain with Gaussian noise and impact noise is proposed. Based on the better per- formance of support vector regression （SVR） for complex signal approximation and the wavelet for signal detail expression, the wavelet kernel function is chosen as support vector kernel func- tion. Then the logarithmic SAR image is regressed with WSVR. Furthermore the regression distance is used as a judgment index of the noise type. According to the judgment of noise type every pixel can be adaptively de-noised with different filters. Through an approximation experiment for a one-dimensional complex signal, the feasibility of SAR data regression based on WSVR is con- firmed. Afterward the SAR image is treated as a two-dimensional continuous signal and filtered by an SVR with wavelet kernel function. The results show that the method proposed here reduces the radar speckle noise effectively while maintaining edge features and details well.展开更多
In this paper, a classification method based on Support Vector Machine (SVM) is given in the digital modulation signal classification. The second, fourth and sixth order cumulants of the received signals are used as c...In this paper, a classification method based on Support Vector Machine (SVM) is given in the digital modulation signal classification. The second, fourth and sixth order cumulants of the received signals are used as classification vectors firstly, then the kernel thought is used to map the feature vector to the high dimensional feature space and the optimum separating hyperplane is constructed in space to realize signal recognition. In order to build an effective and robust SVM classifier, the radial basis kernel function is selected, one against one or one against rest of multi-class classifier is designed, and method of parameter selection using cross- validation grid is adopted. Through the experiments it can be concluded that the classifier based on SVM has high performance and is more robust.展开更多
Owing to the effect of classified models was different in Protein-Protein Interaction(PPI) extraction, which was made by different single kernel functions, and only using single kernel function hardly trained the opti...Owing to the effect of classified models was different in Protein-Protein Interaction(PPI) extraction, which was made by different single kernel functions, and only using single kernel function hardly trained the optimal classified model to extract PPI, this paper presents a strategy to find the optimal kernel function from a kernel function set. The strategy is that in the kernel function set which consists of different single kernel functions, endlessly finding the last two kernel functions on the performance in PPI extraction, using their optimal kernel function to replace them, until there is only one kernel function and it’s the final optimal kernel function. Finally, extracting PPI using the classified model made by this kernel function. This paper conducted the PPI extraction experiment on AIMed corpus, the experimental result shows that the optimal convex combination kernel function this paper presents can effectively improve the extraction performance than single kernel function, and it gets the best precision which reaches 65.0 among the similar PPI extraction systems.展开更多
Kernel-based methods work by embedding the data into a feature space and then searching linear hypothesis among the embedding data points. The performance is mostly affected by which kernel is used. A promising way is...Kernel-based methods work by embedding the data into a feature space and then searching linear hypothesis among the embedding data points. The performance is mostly affected by which kernel is used. A promising way is to learn the kernel from the data automatically. A general regularized risk functional （RRF） criterion for kernel matrix learning is proposed. Compared with the RRF criterion, general RRF criterion takes into account the geometric distributions of the embedding data points. It is proven that the distance between different geometric distdbutions can be estimated by their centroid distance in the reproducing kernel Hilbert space. Using this criterion for kernel matrix learning leads to a convex quadratically constrained quadratic programming （QCQP） problem. For several commonly used loss functions, their mathematical formulations are given. Experiment results on a collection of benchmark data sets demonstrate the effectiveness of the proposed method.展开更多
The principle of the support vector regression machine（SVR） is first analysed. Then the new data-dependent kernel function is constructed from information geometry perspective. The current waveforms change regularly...The principle of the support vector regression machine（SVR） is first analysed. Then the new data-dependent kernel function is constructed from information geometry perspective. The current waveforms change regularly in accordance with the different horizontal offset when the rotational frequency of the high speed rotational arc sensor is in the range from 15 Hz to 30 Hz. The welding current data is pretreated by wavelet filtering, mean filtering and normalization treatment. The SVR model is constructed by making use of the evolvement laws, the decision function can be achieved by training the SVR and the seam offset can be identified. The experimental results show that the precision of the offset identification can be greatly improved by modifying the SVR and applying mean filteringfrom the longitudinal direction.展开更多
With the progress of deep learning research, convolutional neural networks have become the most important method in feature extraction. How to effectively classify and recognize the extracted features will directly af...With the progress of deep learning research, convolutional neural networks have become the most important method in feature extraction. How to effectively classify and recognize the extracted features will directly affect the performance of the entire network. Traditional processing methods include classification models such as fully connected network models and support vector machines. In order to solve the problem that the traditional convolutional neural network is prone to over-fitting for the classification of small samples, a CNN-TWSVM hybrid model was proposed by fusing the twin support vector machine (TWSVM) with higher computational efficiency as the CNN classifier, and it was applied to the traffic sign recognition task. In order to improve the generalization ability of the model, the wavelet kernel function is introduced to deal with the nonlinear classification task. The method uses the network initialized from the ImageNet dataset to fine-tune the specific domain and intercept the inner layer of the network to extract the high abstract features of the traffic sign image. Finally, the TWSVM based on wavelet kernel function is used to identify the traffic signs, so as to effectively solve the over-fitting problem of traffic signs classification. On GTSRB and BELGIUMTS datasets, the validity and generalization ability of the improved model is verified by comparing with different kernel functions and different SVM classifiers.展开更多
To improve performance of a support vector regression, a new method for a modified kernel function is proposed. In this method, information of all samples is included in the kernel function with conformal mapping. Thu...To improve performance of a support vector regression, a new method for a modified kernel function is proposed. In this method, information of all samples is included in the kernel function with conformal mapping. Thus the kernel function is data-dependent. With a random initial parameter, the kernel function is modified repeatedly until a satisfactory result is achieved. Compared with the conventional model, the improved approach does not need to select parameters of the kernel function. Sim- ulation is carried out for the one-dimension continuous function and a case of strong earthquakes. The results show that the improved approach has better learning ability and forecasting precision than the traditional model. With the increase of the iteration number, the figure of merit decreases and converges. The speed of convergence depends on the parameters used in the algorithm.展开更多
为提高月径流时间序列预测精度,改进最小二乘孪生支持向量回归机(Least Squares Twin Support Vector Regression,LSTSVR)性能,首次基于高斯核函数、多项式核函数、线性核函数构建混合核函数,提出4种季节趋势分解(Seasonal and Trend de...为提高月径流时间序列预测精度,改进最小二乘孪生支持向量回归机(Least Squares Twin Support Vector Regression,LSTSVR)性能,首次基于高斯核函数、多项式核函数、线性核函数构建混合核函数,提出4种季节趋势分解(Seasonal and Trend decomposition using Loess,STL)-小波包变换(Wavelet Packet Transform,WPT)-裂狐优化(Rüppell's Fox Optimizer,RFO)算法-混合核最小二乘孪生支持向量回归机(Hybrid Kernel Least Squares Twin Support Vector Regression,HLSTSVR)模型,并构建STL-WPT-RFO-LSTSVR、STL-WPT-RFO-混合核最小二乘支持向量回归机(Hybrid Kerllel Least Squares Twin Suppart Vector Regression,HLSSVR)、STL-WPT-RFO-最小二乘支持向量回归机(Least Squares Support Vector Regression,LSSVR)等17种对比分析模型,通过云南省高桥、凤屯水文站月径流时间序列预测实例对21种模型进行验证。首先利用STL-WPT二次分解技术对月径流序列进行分解处理,合理划分训练集和验证集;然后基于高斯核函数、多项式核函数、线性核函数,采用“三三”线性组合和“两两”线性组合的方式构建4种混合核函数对月径流分解分量进行空间映射;最后利用RFO寻优HLSTSVR/LSTSVR/HLSSVR/LSSVR最佳超参数,利用最佳超参数建立21种模型对实例月径流序列各分解分量进行训练、预测和重构。结果表明:①4种STL-WPT-RFO-HLSTSVR模型能适应不同尺度的月径流数据分布,具有较好的模型性能和较小的预测误差,其中STL-WPT-RFO-HLSTSVR(高斯+多项式+线性)模型对高桥、凤屯站月径流预测的平均绝对百分比误差MAPE分别为2.85%、2.19%,决定系数R2均为0.9994,预测精度最高、效果最好;②混合核函数兼顾了不同核函数优势,能在模型复杂度与泛化能力之间取得平衡,显著提升模型性能和预测精度;③STL-WPT二次分解技术能有效解决复杂时间序列的非平稳性、非线性和多尺度特征,较STL更具分解优势;④组合模型融合了STL-WPT、RFO和HLSTSVR优点,具有较好的普适性和参考价值。展开更多
The performance of the support vector regression (SVR) model is sensitive to the kernel type and its parameters.The determination of an appropriate kernel type and the associated parameters for SVR is a challenging re...The performance of the support vector regression (SVR) model is sensitive to the kernel type and its parameters.The determination of an appropriate kernel type and the associated parameters for SVR is a challenging research topic in the field of support vector learning.In this study,we present a novel method for simultaneous optimization of the SVR kernel function and its parameters,formulated as a mixed integer optimization problem and solved using the recently proposed heuristic 'extremal optimization (EO)'.We present the problem formulation for the optimization of the SVR kernel and parameters,the EO-SVR algorithm,and experimental tests with five benchmark regression problems.The results of comparison with other traditional approaches show that the proposed EO-SVR method provides better generalization performance by successfully identifying the optimal SVR kernel function and its parameters.展开更多
Histogram Intersection Kernel Support Vector Machines (SVM) was used for the image classification problem. Specifically, each image was split into blocks, and each block was represented by the Scale Invariant Feature ...Histogram Intersection Kernel Support Vector Machines (SVM) was used for the image classification problem. Specifically, each image was split into blocks, and each block was represented by the Scale Invariant Feature Transform (SIFT) descriptors;secondly, k-means cluster method was applied to separate the SIFT descriptors into groups, each group represented a visual keywords;thirdly, count the number of the SIFT descriptors in each image, and histogram of each image should be constructed;finally, Histogram Intersection Kernel should be built based on these histograms. In our experimental study, we use Corel-low images to test our method. Compared with typical RBF kernel SVM, the Histogram Intersection kernel SVM performs better than RBF kernel SVM.展开更多
文摘The development of forecasting models for pollution particles shows a nonlinear dynamic behavior;hence, implementation is a non-trivial process. In the literature, there have been multiple models of particulate pollutants, which use softcomputing techniques and machine learning such as: multilayer perceptrons, neural networks, support vector machines, kernel algorithms, and so on. This paper presents a prediction pollution model using support vector machines and kernel functions, which are: Gaussian, Polynomial and Spline. Finally, the prediction results of ozone (O3), particulate matter (PM10) and nitrogen dioxide (NO2) at Mexico City are presented as a case study using these techniques.
基金This work has been supported by the National Outstanding Youth Science Foundation of China (No. 60025308) and the Teach and Research Award Program for Outstanding Young Teachers in Higher Education Institutions of MOE,China.
文摘A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controlled plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.
文摘A method of applying support vector machine (SVM) in speech recognition was proposed, and a speech recognition system for mandarin digits was built up by SVMs. In the system, vectors were linearly extracted from speech feature sequence to make up time-aligned input patterns for SVM, and the decisions of several 2-class SVM classifiers were employed for constructing an N-class classifier. Four kinds of SVM kernel functions were compared in the experiments of speaker-independent speech recognition of mandarin digits. And the kernel of radial basis function has the highest accurate rate of 99.33%, which is better than that of the baseline system based on hidden Markov models (HMM) (97.08%). And the experiments also show that SVM can outperform HMM especially when the samples for learning were very limited.
基金supported by Shanghai Science and Technology Commission Innovation Action Plan(08DZ1205708)
文摘A new filtering method for SAR data de-noising using wavelet support vector regression （WSVR） is developed. On the basis of the grey scale distribution character of SAR imagery, the logarithmic SAR image as a noise polluted signal is taken and the noise model assumption in logarithmic domain with Gaussian noise and impact noise is proposed. Based on the better per- formance of support vector regression （SVR） for complex signal approximation and the wavelet for signal detail expression, the wavelet kernel function is chosen as support vector kernel func- tion. Then the logarithmic SAR image is regressed with WSVR. Furthermore the regression distance is used as a judgment index of the noise type. According to the judgment of noise type every pixel can be adaptively de-noised with different filters. Through an approximation experiment for a one-dimensional complex signal, the feasibility of SAR data regression based on WSVR is con- firmed. Afterward the SAR image is treated as a two-dimensional continuous signal and filtered by an SVR with wavelet kernel function. The results show that the method proposed here reduces the radar speckle noise effectively while maintaining edge features and details well.
文摘In this paper, a classification method based on Support Vector Machine (SVM) is given in the digital modulation signal classification. The second, fourth and sixth order cumulants of the received signals are used as classification vectors firstly, then the kernel thought is used to map the feature vector to the high dimensional feature space and the optimum separating hyperplane is constructed in space to realize signal recognition. In order to build an effective and robust SVM classifier, the radial basis kernel function is selected, one against one or one against rest of multi-class classifier is designed, and method of parameter selection using cross- validation grid is adopted. Through the experiments it can be concluded that the classifier based on SVM has high performance and is more robust.
文摘Owing to the effect of classified models was different in Protein-Protein Interaction(PPI) extraction, which was made by different single kernel functions, and only using single kernel function hardly trained the optimal classified model to extract PPI, this paper presents a strategy to find the optimal kernel function from a kernel function set. The strategy is that in the kernel function set which consists of different single kernel functions, endlessly finding the last two kernel functions on the performance in PPI extraction, using their optimal kernel function to replace them, until there is only one kernel function and it’s the final optimal kernel function. Finally, extracting PPI using the classified model made by this kernel function. This paper conducted the PPI extraction experiment on AIMed corpus, the experimental result shows that the optimal convex combination kernel function this paper presents can effectively improve the extraction performance than single kernel function, and it gets the best precision which reaches 65.0 among the similar PPI extraction systems.
基金supported by the National Natural Science Fundation of China (60736021)the Joint Funds of NSFC-Guangdong Province(U0735003)
文摘Kernel-based methods work by embedding the data into a feature space and then searching linear hypothesis among the embedding data points. The performance is mostly affected by which kernel is used. A promising way is to learn the kernel from the data automatically. A general regularized risk functional （RRF） criterion for kernel matrix learning is proposed. Compared with the RRF criterion, general RRF criterion takes into account the geometric distributions of the embedding data points. It is proven that the distance between different geometric distdbutions can be estimated by their centroid distance in the reproducing kernel Hilbert space. Using this criterion for kernel matrix learning leads to a convex quadratically constrained quadratic programming （QCQP） problem. For several commonly used loss functions, their mathematical formulations are given. Experiment results on a collection of benchmark data sets demonstrate the effectiveness of the proposed method.
基金Supported by National Natural Science Foundation of China（ No. 50705030）.
文摘The principle of the support vector regression machine（SVR） is first analysed. Then the new data-dependent kernel function is constructed from information geometry perspective. The current waveforms change regularly in accordance with the different horizontal offset when the rotational frequency of the high speed rotational arc sensor is in the range from 15 Hz to 30 Hz. The welding current data is pretreated by wavelet filtering, mean filtering and normalization treatment. The SVR model is constructed by making use of the evolvement laws, the decision function can be achieved by training the SVR and the seam offset can be identified. The experimental results show that the precision of the offset identification can be greatly improved by modifying the SVR and applying mean filteringfrom the longitudinal direction.
文摘With the progress of deep learning research, convolutional neural networks have become the most important method in feature extraction. How to effectively classify and recognize the extracted features will directly affect the performance of the entire network. Traditional processing methods include classification models such as fully connected network models and support vector machines. In order to solve the problem that the traditional convolutional neural network is prone to over-fitting for the classification of small samples, a CNN-TWSVM hybrid model was proposed by fusing the twin support vector machine (TWSVM) with higher computational efficiency as the CNN classifier, and it was applied to the traffic sign recognition task. In order to improve the generalization ability of the model, the wavelet kernel function is introduced to deal with the nonlinear classification task. The method uses the network initialized from the ImageNet dataset to fine-tune the specific domain and intercept the inner layer of the network to extract the high abstract features of the traffic sign image. Finally, the TWSVM based on wavelet kernel function is used to identify the traffic signs, so as to effectively solve the over-fitting problem of traffic signs classification. On GTSRB and BELGIUMTS datasets, the validity and generalization ability of the improved model is verified by comparing with different kernel functions and different SVM classifiers.
基金Project supported by the National Natural Science Foundation of China (No. 50578168)the Natural Science Foundation of CQ CSTC (No. 2007BB2396)
文摘To improve performance of a support vector regression, a new method for a modified kernel function is proposed. In this method, information of all samples is included in the kernel function with conformal mapping. Thus the kernel function is data-dependent. With a random initial parameter, the kernel function is modified repeatedly until a satisfactory result is achieved. Compared with the conventional model, the improved approach does not need to select parameters of the kernel function. Sim- ulation is carried out for the one-dimension continuous function and a case of strong earthquakes. The results show that the improved approach has better learning ability and forecasting precision than the traditional model. With the increase of the iteration number, the figure of merit decreases and converges. The speed of convergence depends on the parameters used in the algorithm.
文摘为提高月径流时间序列预测精度,改进最小二乘孪生支持向量回归机(Least Squares Twin Support Vector Regression,LSTSVR)性能,首次基于高斯核函数、多项式核函数、线性核函数构建混合核函数,提出4种季节趋势分解(Seasonal and Trend decomposition using Loess,STL)-小波包变换(Wavelet Packet Transform,WPT)-裂狐优化(Rüppell's Fox Optimizer,RFO)算法-混合核最小二乘孪生支持向量回归机(Hybrid Kernel Least Squares Twin Support Vector Regression,HLSTSVR)模型,并构建STL-WPT-RFO-LSTSVR、STL-WPT-RFO-混合核最小二乘支持向量回归机(Hybrid Kerllel Least Squares Twin Suppart Vector Regression,HLSSVR)、STL-WPT-RFO-最小二乘支持向量回归机(Least Squares Support Vector Regression,LSSVR)等17种对比分析模型,通过云南省高桥、凤屯水文站月径流时间序列预测实例对21种模型进行验证。首先利用STL-WPT二次分解技术对月径流序列进行分解处理,合理划分训练集和验证集;然后基于高斯核函数、多项式核函数、线性核函数,采用“三三”线性组合和“两两”线性组合的方式构建4种混合核函数对月径流分解分量进行空间映射;最后利用RFO寻优HLSTSVR/LSTSVR/HLSSVR/LSSVR最佳超参数,利用最佳超参数建立21种模型对实例月径流序列各分解分量进行训练、预测和重构。结果表明:①4种STL-WPT-RFO-HLSTSVR模型能适应不同尺度的月径流数据分布,具有较好的模型性能和较小的预测误差,其中STL-WPT-RFO-HLSTSVR(高斯+多项式+线性)模型对高桥、凤屯站月径流预测的平均绝对百分比误差MAPE分别为2.85%、2.19%,决定系数R2均为0.9994,预测精度最高、效果最好;②混合核函数兼顾了不同核函数优势,能在模型复杂度与泛化能力之间取得平衡,显著提升模型性能和预测精度;③STL-WPT二次分解技术能有效解决复杂时间序列的非平稳性、非线性和多尺度特征,较STL更具分解优势;④组合模型融合了STL-WPT、RFO和HLSTSVR优点,具有较好的普适性和参考价值。
文摘The performance of the support vector regression (SVR) model is sensitive to the kernel type and its parameters.The determination of an appropriate kernel type and the associated parameters for SVR is a challenging research topic in the field of support vector learning.In this study,we present a novel method for simultaneous optimization of the SVR kernel function and its parameters,formulated as a mixed integer optimization problem and solved using the recently proposed heuristic 'extremal optimization (EO)'.We present the problem formulation for the optimization of the SVR kernel and parameters,the EO-SVR algorithm,and experimental tests with five benchmark regression problems.The results of comparison with other traditional approaches show that the proposed EO-SVR method provides better generalization performance by successfully identifying the optimal SVR kernel function and its parameters.
文摘Histogram Intersection Kernel Support Vector Machines (SVM) was used for the image classification problem. Specifically, each image was split into blocks, and each block was represented by the Scale Invariant Feature Transform (SIFT) descriptors;secondly, k-means cluster method was applied to separate the SIFT descriptors into groups, each group represented a visual keywords;thirdly, count the number of the SIFT descriptors in each image, and histogram of each image should be constructed;finally, Histogram Intersection Kernel should be built based on these histograms. In our experimental study, we use Corel-low images to test our method. Compared with typical RBF kernel SVM, the Histogram Intersection kernel SVM performs better than RBF kernel SVM.
