In this paper, sixty-eight research articles published between 2000 and 2017 as well as textbooks which employed four classification algorithms: K-Nearest-Neighbor (KNN), Support Vector Machines (SVM), Random Forest (...In this paper, sixty-eight research articles published between 2000 and 2017 as well as textbooks which employed four classification algorithms: K-Nearest-Neighbor (KNN), Support Vector Machines (SVM), Random Forest (RF) and Neural Network (NN) as the main statistical tools were reviewed. The aim was to examine and compare these nonparametric classification methods on the following attributes: robustness to training data, sensitivity to changes, data fitting, stability, ability to handle large data sizes, sensitivity to noise, time invested in parameter tuning, and accuracy. The performances, strengths and shortcomings of each of the algorithms were examined, and finally, a conclusion was arrived at on which one has higher performance. It was evident from the literature reviewed that RF is too sensitive to small changes in the training dataset and is occasionally unstable and tends to overfit in the model. KNN is easy to implement and understand but has a major drawback of becoming significantly slow as the size of the data in use grows, while the ideal value of K for the KNN classifier is difficult to set. SVM and RF are insensitive to noise or overtraining, which shows their ability in dealing with unbalanced data. Larger input datasets will lengthen classification times for NN and KNN more than for SVM and RF. Among these nonparametric classification methods, NN has the potential to become a more widely used classification algorithm, but because of their time-consuming parameter tuning procedure, high level of complexity in computational processing, the numerous types of NN architectures to choose from and the high number of algorithms used for training, most researchers recommend SVM and RF as easier and wieldy used methods which repeatedly achieve results with high accuracies and are often faster to implement.展开更多
Arrhythmia beat classification is an active area of research in ECG based clinical decision support systems. In this paper, Pruned Fuzzy K-nearest neighbor (PFKNN) classifier is proposed to classify six types of beats...Arrhythmia beat classification is an active area of research in ECG based clinical decision support systems. In this paper, Pruned Fuzzy K-nearest neighbor (PFKNN) classifier is proposed to classify six types of beats present in the MIT-BIH Arrhythmia database. We have tested our classifier on ~ 103100 beats for six beat types present in the database. Fuzzy KNN (FKNN) can be implemented very easily but large number of training examples used for classification can be very time consuming and requires large storage space. Hence, we have proposed a time efficient Arif-Fayyaz pruning algorithm especially suitable for FKNN which can maintain good classification accuracy with appropriate retained ratio of training data. By using Arif-Fayyaz pruning algorithm with Fuzzy KNN, we have achieved a beat classification accuracy of 97% and geometric mean of sensitivity of 94.5% with only 19% of the total training examples. The accuracy and sensitivity is comparable to FKNN when all the training data is used. Principal Component Analysis is used to further reduce the dimension of feature space from eleven to six without compromising the accuracy and sensitivity. PFKNN was found to robust against noise present in the ECG data.展开更多
Collaborative representation-based classification(CRC) is a distance based method, and it obtains the original contributions from all samples to solve the sparse representation coefficient. We find out that it helps t...Collaborative representation-based classification(CRC) is a distance based method, and it obtains the original contributions from all samples to solve the sparse representation coefficient. We find out that it helps to enhance the discrimination in classification by integrating other distance based features and/or adding signal preprocessing to the original samples. In this paper, we propose an improved version of the CRC method which uses the Gabor wavelet transformation to preprocess the samples and also adapts the nearest neighbor(NN)features, and hence we call it GNN-CRC. Firstly, Gabor wavelet transformation is applied to minimize the effects from the background in face images and build Gabor features into the input data. Secondly, the distances solved by NN and CRC are fused together to obtain a more discriminative classification. Extensive experiments are conducted to evaluate the proposed method for face recognition with different instantiations. The experimental results illustrate that our method outperforms the naive CRC as well as some other state-of-the-art algorithms.展开更多
Classification of the patterns is a crucial structure of research and applications. Using fuzzy set theory, classifying the patterns has become of great interest because of its ability to understand the parameters. On...Classification of the patterns is a crucial structure of research and applications. Using fuzzy set theory, classifying the patterns has become of great interest because of its ability to understand the parameters. One of the problemsobserved in the fuzzification of an unknown pattern is that importance is givenonly to the known patterns but not to their features. In contrast, features of thepatterns play an essential role when their respective patterns overlap. In this paper,an optimal fuzzy nearest neighbor model has been introduced in which a fuzzifi-cation process has been carried out for the unknown pattern using k nearest neighbor. With the help of the fuzzification process, the membership matrix has beenformed. In this membership matrix, fuzzification has been carried out of the features of the unknown pattern. Classification results are verified on a completelyllabelled Telugu vowel data set, and the accuracy is compared with the differentmodels and the fuzzy k nearest neighbor algorithm. The proposed model gives84.86% accuracy on 50% training data set and 89.35% accuracy on 80% trainingdata set. The proposed classifier learns well enough with a small amount of training data, resulting in an efficient and faster approach.展开更多
In wireless sensor networks, target classification differs from that in centralized sensing systems because of the distributed detection, wireless communication and limited resources. We study the classification probl...In wireless sensor networks, target classification differs from that in centralized sensing systems because of the distributed detection, wireless communication and limited resources. We study the classification problem of moving vehicles in wireless sensor networks using acoustic signals emitted from vehicles. Three algorithms including wavelet decomposition, weighted k-nearest-neighbor and Dempster-Shafer theory are combined in this paper. Finally, we use real world experimental data to validate the classification methods. The result shows that wavelet based feature extraction method can extract stable features from acoustic signals. By fusion with Dempster's rule, the classification performance is improved.展开更多
The performance of scene classification of satellite images strongly relies on the discriminative power of the low-level and mid-level feature representation. This paper presents a novel approach, named multi-level ma...The performance of scene classification of satellite images strongly relies on the discriminative power of the low-level and mid-level feature representation. This paper presents a novel approach, named multi-level max-margin analysis （M 3 DA） for semantic classification for high-resolution satellite images. In our M 3 DA model, the maximum entropy discrimination latent Dirichlet allocation （MedLDA） model is applied to learn the topic-level features first, and then based on a bag-of-words repre- sentation of low-level local image features, the large margin nearest neighbor （LMNN） classifier is used to optimize a multiple soft label composed of word-level features （generated by SVM classifier） and topic-level features. The categorization performances on 21-class land-use dataset have demonstrated that the proposed model in multi-level max-margin scheme can distinguish different categories of land-use scenes reasonably.展开更多
In numerous real-world healthcare applications,handling incomplete medical data poses significant challenges for missing value imputation and subsequent clustering or classification tasks.Traditional approaches often ...In numerous real-world healthcare applications,handling incomplete medical data poses significant challenges for missing value imputation and subsequent clustering or classification tasks.Traditional approaches often rely on statistical methods for imputation,which may yield suboptimal results and be computationally intensive.This paper aims to integrate imputation and clustering techniques to enhance the classification of incomplete medical data with improved accuracy.Conventional classification methods are ill-suited for incomplete medical data.To enhance efficiency without compromising accuracy,this paper introduces a novel approach that combines imputation and clustering for the classification of incomplete data.Initially,the linear interpolation imputation method alongside an iterative Fuzzy c-means clustering method is applied and followed by a classification algorithm.The effectiveness of the proposed approach is evaluated using multiple performance metrics,including accuracy,precision,specificity,and sensitivity.The encouraging results demonstrate that our proposed method surpasses classical approaches across various performance criteria.展开更多
In this paper, a new shape classification system based on singular value decomposition (SVD) transform using nearest neighbour classifier was proposed. The gray scale image of the shape object was converted into a bla...In this paper, a new shape classification system based on singular value decomposition (SVD) transform using nearest neighbour classifier was proposed. The gray scale image of the shape object was converted into a black and white image. The squared Euclidean distance transform on binary image was applied to extract the boundary image of the shape. SVD transform features were extracted from the the boundary of the object shapes. In this paper, the proposed classification system based on SVD transform feature extraction method was compared with classifier based on moment invariants using nearest neighbour classifier. The experimental results showed the advantage of our proposed classification system.展开更多
In this paper, a new approach is presented to find the reference set for the nearest neighbor classifier. The optimal reference set, which has minimum sample size and satisfies a certain error rate threshold, is obtai...In this paper, a new approach is presented to find the reference set for the nearest neighbor classifier. The optimal reference set, which has minimum sample size and satisfies a certain error rate threshold, is obtained through a Tabu search algorithm. When the error rate threshold is set to zero, the algorithm obtains a near minimal consistent subset of a given training set. While the threshold is set to a small appropriate value, the obtained reference set may compensate the bias of the nearest neighbor estimate. An aspiration criterion for Tabu search is introduced, which aims to prevent the search process from the inefficient wandering between the feasible and infeasible regions in the search space and speed up the convergence. Experimental results based on a number of typical data sets are presented and analyzed to illustrate the benefits of the proposed method. Compared to conventional methods, such as CNN and Dasarathy's algorithm, the size of the reduced reference sets is much smaller, and the nearest neighbor classification performance is better, especially when the error rate thresholds are set to appropriate nonzero values. The experimental results also illustrate that the MCS (minimal consistent set) of Dasarathy's algorithm is not minimal, and its candidate consistent set is not always ensured to reduce monotonically. A counter example is also given to confirm this claim.展开更多
文摘In this paper, sixty-eight research articles published between 2000 and 2017 as well as textbooks which employed four classification algorithms: K-Nearest-Neighbor (KNN), Support Vector Machines (SVM), Random Forest (RF) and Neural Network (NN) as the main statistical tools were reviewed. The aim was to examine and compare these nonparametric classification methods on the following attributes: robustness to training data, sensitivity to changes, data fitting, stability, ability to handle large data sizes, sensitivity to noise, time invested in parameter tuning, and accuracy. The performances, strengths and shortcomings of each of the algorithms were examined, and finally, a conclusion was arrived at on which one has higher performance. It was evident from the literature reviewed that RF is too sensitive to small changes in the training dataset and is occasionally unstable and tends to overfit in the model. KNN is easy to implement and understand but has a major drawback of becoming significantly slow as the size of the data in use grows, while the ideal value of K for the KNN classifier is difficult to set. SVM and RF are insensitive to noise or overtraining, which shows their ability in dealing with unbalanced data. Larger input datasets will lengthen classification times for NN and KNN more than for SVM and RF. Among these nonparametric classification methods, NN has the potential to become a more widely used classification algorithm, but because of their time-consuming parameter tuning procedure, high level of complexity in computational processing, the numerous types of NN architectures to choose from and the high number of algorithms used for training, most researchers recommend SVM and RF as easier and wieldy used methods which repeatedly achieve results with high accuracies and are often faster to implement.
文摘Arrhythmia beat classification is an active area of research in ECG based clinical decision support systems. In this paper, Pruned Fuzzy K-nearest neighbor (PFKNN) classifier is proposed to classify six types of beats present in the MIT-BIH Arrhythmia database. We have tested our classifier on ~ 103100 beats for six beat types present in the database. Fuzzy KNN (FKNN) can be implemented very easily but large number of training examples used for classification can be very time consuming and requires large storage space. Hence, we have proposed a time efficient Arif-Fayyaz pruning algorithm especially suitable for FKNN which can maintain good classification accuracy with appropriate retained ratio of training data. By using Arif-Fayyaz pruning algorithm with Fuzzy KNN, we have achieved a beat classification accuracy of 97% and geometric mean of sensitivity of 94.5% with only 19% of the total training examples. The accuracy and sensitivity is comparable to FKNN when all the training data is used. Principal Component Analysis is used to further reduce the dimension of feature space from eleven to six without compromising the accuracy and sensitivity. PFKNN was found to robust against noise present in the ECG data.
基金the National Natural Science Foundation of China(No.61502208)the Natural Science Foundation of Jiangsu Province of China(No.BK20150522)+1 种基金the Scientific and Technical Program of City of Huizhou(Nos.2016X0422037 and 2017C0405021)the Natural Science Foundation of Huizhou University(Nos.hzux1201606 and hzu201701)
文摘Collaborative representation-based classification(CRC) is a distance based method, and it obtains the original contributions from all samples to solve the sparse representation coefficient. We find out that it helps to enhance the discrimination in classification by integrating other distance based features and/or adding signal preprocessing to the original samples. In this paper, we propose an improved version of the CRC method which uses the Gabor wavelet transformation to preprocess the samples and also adapts the nearest neighbor(NN)features, and hence we call it GNN-CRC. Firstly, Gabor wavelet transformation is applied to minimize the effects from the background in face images and build Gabor features into the input data. Secondly, the distances solved by NN and CRC are fused together to obtain a more discriminative classification. Extensive experiments are conducted to evaluate the proposed method for face recognition with different instantiations. The experimental results illustrate that our method outperforms the naive CRC as well as some other state-of-the-art algorithms.
基金supported by the Taif University Researchers Supporting Project Number(TURSP-2020/79),Taif University,Taif,Saudi Arabia.
文摘Classification of the patterns is a crucial structure of research and applications. Using fuzzy set theory, classifying the patterns has become of great interest because of its ability to understand the parameters. One of the problemsobserved in the fuzzification of an unknown pattern is that importance is givenonly to the known patterns but not to their features. In contrast, features of thepatterns play an essential role when their respective patterns overlap. In this paper,an optimal fuzzy nearest neighbor model has been introduced in which a fuzzifi-cation process has been carried out for the unknown pattern using k nearest neighbor. With the help of the fuzzification process, the membership matrix has beenformed. In this membership matrix, fuzzification has been carried out of the features of the unknown pattern. Classification results are verified on a completelyllabelled Telugu vowel data set, and the accuracy is compared with the differentmodels and the fuzzy k nearest neighbor algorithm. The proposed model gives84.86% accuracy on 50% training data set and 89.35% accuracy on 80% trainingdata set. The proposed classifier learns well enough with a small amount of training data, resulting in an efficient and faster approach.
基金Supported in part by Science & Technology Department of Shanghai (05dz15004)
文摘In wireless sensor networks, target classification differs from that in centralized sensing systems because of the distributed detection, wireless communication and limited resources. We study the classification problem of moving vehicles in wireless sensor networks using acoustic signals emitted from vehicles. Three algorithms including wavelet decomposition, weighted k-nearest-neighbor and Dempster-Shafer theory are combined in this paper. Finally, we use real world experimental data to validate the classification methods. The result shows that wavelet based feature extraction method can extract stable features from acoustic signals. By fusion with Dempster's rule, the classification performance is improved.
基金Supported by the Open Projects Program of National Laboratory of Pattern Recognition and the National Natural Science Foundation of China(91338113)
文摘The performance of scene classification of satellite images strongly relies on the discriminative power of the low-level and mid-level feature representation. This paper presents a novel approach, named multi-level max-margin analysis （M 3 DA） for semantic classification for high-resolution satellite images. In our M 3 DA model, the maximum entropy discrimination latent Dirichlet allocation （MedLDA） model is applied to learn the topic-level features first, and then based on a bag-of-words repre- sentation of low-level local image features, the large margin nearest neighbor （LMNN） classifier is used to optimize a multiple soft label composed of word-level features （generated by SVM classifier） and topic-level features. The categorization performances on 21-class land-use dataset have demonstrated that the proposed model in multi-level max-margin scheme can distinguish different categories of land-use scenes reasonably.
基金supported by the Researchers Supporting Project number(RSP2024R 34),King Saud University,Riyadh,Saudi Arabia。
文摘In numerous real-world healthcare applications,handling incomplete medical data poses significant challenges for missing value imputation and subsequent clustering or classification tasks.Traditional approaches often rely on statistical methods for imputation,which may yield suboptimal results and be computationally intensive.This paper aims to integrate imputation and clustering techniques to enhance the classification of incomplete medical data with improved accuracy.Conventional classification methods are ill-suited for incomplete medical data.To enhance efficiency without compromising accuracy,this paper introduces a novel approach that combines imputation and clustering for the classification of incomplete data.Initially,the linear interpolation imputation method alongside an iterative Fuzzy c-means clustering method is applied and followed by a classification algorithm.The effectiveness of the proposed approach is evaluated using multiple performance metrics,including accuracy,precision,specificity,and sensitivity.The encouraging results demonstrate that our proposed method surpasses classical approaches across various performance criteria.
基金This paper received financial support towards the cost of its publication from the Deanship of Research and Graduate Studies at Applied Science University, Amman, Jordan.
文摘In this paper, a new shape classification system based on singular value decomposition (SVD) transform using nearest neighbour classifier was proposed. The gray scale image of the shape object was converted into a black and white image. The squared Euclidean distance transform on binary image was applied to extract the boundary image of the shape. SVD transform features were extracted from the the boundary of the object shapes. In this paper, the proposed classification system based on SVD transform feature extraction method was compared with classifier based on moment invariants using nearest neighbour classifier. The experimental results showed the advantage of our proposed classification system.
基金he National Natural Science Foundation of China (No.69675007) and Beijing MunicipalNatural Science Foundation (No.4972008).
文摘In this paper, a new approach is presented to find the reference set for the nearest neighbor classifier. The optimal reference set, which has minimum sample size and satisfies a certain error rate threshold, is obtained through a Tabu search algorithm. When the error rate threshold is set to zero, the algorithm obtains a near minimal consistent subset of a given training set. While the threshold is set to a small appropriate value, the obtained reference set may compensate the bias of the nearest neighbor estimate. An aspiration criterion for Tabu search is introduced, which aims to prevent the search process from the inefficient wandering between the feasible and infeasible regions in the search space and speed up the convergence. Experimental results based on a number of typical data sets are presented and analyzed to illustrate the benefits of the proposed method. Compared to conventional methods, such as CNN and Dasarathy's algorithm, the size of the reduced reference sets is much smaller, and the nearest neighbor classification performance is better, especially when the error rate thresholds are set to appropriate nonzero values. The experimental results also illustrate that the MCS (minimal consistent set) of Dasarathy's algorithm is not minimal, and its candidate consistent set is not always ensured to reduce monotonically. A counter example is also given to confirm this claim.
