Knowledge distillation techniques have been widely used in the field of deep learning,usually by extracting valid information from a neural network with a large number of parameters and a high learning capacity(the te...Knowledge distillation techniques have been widely used in the field of deep learning,usually by extracting valid information from a neural network with a large number of parameters and a high learning capacity(the teacher model)to a neural network with a small number of parameters and a low learning capacity(the student model).However,there are inefficiencies in the transfer of knowledge between teacher and student.The student model does not fully learn all the knowledge of the teacher model.Therefore,we aim to achieve knowledge distillation of our network layer by a single model,i.e.,self-distillation.We also apply the idea of self-distillation to the object detection task and propose a multi-scale self-distillation approach,where we argue that knowledge distillation of the information contained in feature maps at different scales can help the model better detect small targets.In addition,we propose a Gaussian mask based on the target region as an auxiliary detection method to improve the accuracy of target position detection in the distillation process.We then validate our approach on the KITTI dataset using a single-stage detector YOLO.The results demonstrate a 2.8%improvement in accuracy over the baseline model without the use of a teacher model.展开更多
In this paper, a new medical image classification scheme is proposed using selforganizing map (SOM) combined with multiscale technique. It addresses the problem of the handling of edge pixels in the traditional multis...In this paper, a new medical image classification scheme is proposed using selforganizing map (SOM) combined with multiscale technique. It addresses the problem of the handling of edge pixels in the traditional multiscale SOM classifiers. First, to solve the difficulty in manual selection of edge pixels, a multiscale edge detection algorithm based on wavelet transform is proposed. Edge pixels detected are then selected into the training set as a new class and a mu1tiscale SoM classifier is trained using this training set. In this new scheme, the SoM classifier can perform both the classification on the entire image and the edge detection simultaneously. On the other hand, the misclassification of the traditional multiscale SoM classifier in regions near edges is greatly reduced and the correct classification is improved at the same time.展开更多
基金supported by the New Generation AI Major Project of Ministry of Science and Technology of China(No.2018AAA0102501).
文摘Knowledge distillation techniques have been widely used in the field of deep learning,usually by extracting valid information from a neural network with a large number of parameters and a high learning capacity(the teacher model)to a neural network with a small number of parameters and a low learning capacity(the student model).However,there are inefficiencies in the transfer of knowledge between teacher and student.The student model does not fully learn all the knowledge of the teacher model.Therefore,we aim to achieve knowledge distillation of our network layer by a single model,i.e.,self-distillation.We also apply the idea of self-distillation to the object detection task and propose a multi-scale self-distillation approach,where we argue that knowledge distillation of the information contained in feature maps at different scales can help the model better detect small targets.In addition,we propose a Gaussian mask based on the target region as an auxiliary detection method to improve the accuracy of target position detection in the distillation process.We then validate our approach on the KITTI dataset using a single-stage detector YOLO.The results demonstrate a 2.8%improvement in accuracy over the baseline model without the use of a teacher model.
文摘In this paper, a new medical image classification scheme is proposed using selforganizing map (SOM) combined with multiscale technique. It addresses the problem of the handling of edge pixels in the traditional multiscale SOM classifiers. First, to solve the difficulty in manual selection of edge pixels, a multiscale edge detection algorithm based on wavelet transform is proposed. Edge pixels detected are then selected into the training set as a new class and a mu1tiscale SoM classifier is trained using this training set. In this new scheme, the SoM classifier can perform both the classification on the entire image and the edge detection simultaneously. On the other hand, the misclassification of the traditional multiscale SoM classifier in regions near edges is greatly reduced and the correct classification is improved at the same time.
