Deep Neural Networks(DNNs)have become the tool of choice for machine learning practitioners today.One important aspect of designing a neural network is the choice of the activation function to be used at the neurons o...Deep Neural Networks(DNNs)have become the tool of choice for machine learning practitioners today.One important aspect of designing a neural network is the choice of the activation function to be used at the neurons of the different layers.In this work,we introduce a four-output activation function called the Reflected Rectified Linear Unit(RRe LU)activation which considers both a feature and its negation during computation.Our activation function is"sparse",in that only two of the four possible outputs are active at a given time.We test our activation function on the standard MNIST and CIFAR-10 datasets,which are classification problems,as well as on a novel Computational Fluid Dynamics(CFD)dataset which is posed as a regression problem.On the baseline network for the MNIST dataset,having two hidden layers,our activation function improves the validation accuracy from 0.09 to 0.97 compared to the well-known Re LU activation.For the CIFAR-10 dataset,we use a deep baseline network that achieves 0.78 validation accuracy with 20 epochs but overfits the data.Using the RRe LU activation,we can achieve the same accuracy without overfitting the data.For the CFD dataset,we show that the RRe LU activation can reduce the number of epochs from 100(using Re LU)to 10 while obtaining the same levels of performance.展开更多
修正线性单元(rectified linear unit,ReLU)是深度卷积神经网络常用的激活函数,但当输入为负数时,ReLU的输出为零,造成了零梯度问题;且当输入为正数时,ReLU的输出保持输入不变,使得ReLU函数的平均值恒大于零,引起了偏移现象,从而限制了...修正线性单元(rectified linear unit,ReLU)是深度卷积神经网络常用的激活函数,但当输入为负数时,ReLU的输出为零,造成了零梯度问题;且当输入为正数时,ReLU的输出保持输入不变,使得ReLU函数的平均值恒大于零,引起了偏移现象,从而限制了深度卷积神经网络的学习速率和学习效果.针对ReLU函数的零梯度问题和偏移现象,根据"输出均值接近零的激活函数能够提升神经网络学习性能"原理对其进行改进,提出SLU(softplus linear unit)函数.首先,对负数输入部分进行softplus处理,使得负数输入时SLU函数的输出为负,从而输出平均值更接近于零,减缓了偏移现象;其次,为保证梯度平稳,对SLU的参数进行约束,并固定正数部分的参数;最后,根据SLU对正数部分的处理调整负数部分的参数,确保激活函数在零点处连续可导,信息得以双向传播.设计深度自编码模型在数据集MINST上进行无监督学习,设计网中网卷积神经网络模型在数据集CIFAR-10上进行监督学习.实验结果表明,与ReLU及其相关改进单元相比,基于SLU函数的神经网络模型具有更好的特征学习能力和更高的学习精度.展开更多
Currently,there are many limitations to classify images of small objects.In addition,there are limitations such as error detection due to external factors,and there is also a disadvantage that it is difficult to accur...Currently,there are many limitations to classify images of small objects.In addition,there are limitations such as error detection due to external factors,and there is also a disadvantage that it is difficult to accurately distinguish between various objects.This paper uses a convolutional neural network(CNN)algorithm to recognize and classify object images of very small moths and obtain precise data images.A convolution neural network algorithm is used for image data classification,and the classified image is transformed into image data to learn the topological structure of the image.To improve the accuracy of the image classification and reduce the loss rate,a parameter for finding a fast-optimal point of image classification is set by a convolutional neural network and a pixel image as a preprocessor.As a result of this study,we applied a convolution neural network algorithm to classify the images of very small moths by capturing precise images of the moths.Experimental results showed that the accuracy of classification of very small moths was more than 90%.展开更多
由于背景环境复杂,检测物体易受部分遮挡、天气以及光线变化等因素的影响,传统目标检测方法存在提取特征难、检测准确率低、检测耗时长等缺陷.为了改善传统目标检测方法存在的缺陷,实现快速准确的目标检测,提出了一种基于快速区域卷积...由于背景环境复杂,检测物体易受部分遮挡、天气以及光线变化等因素的影响,传统目标检测方法存在提取特征难、检测准确率低、检测耗时长等缺陷.为了改善传统目标检测方法存在的缺陷,实现快速准确的目标检测,提出了一种基于快速区域卷积神经网络(faster regions with convolutional neural network,Faster-RCNN)算法的轻量化改进方法,即针对算法Inception-V2特征提取网络进行轻量化改进,并以带泄露线性整流(leaky rectified linear unit,Leaky ReLU)作为激活函数,解决使用线性整流(rectified linear unit,ReLU)激活函数存在的神经元输入为负数时输出为0的问题.基于上述改进方法,选择沙滩废弃物的检测为案例以验证方法的有效性,并且结合不同特征提取网络在检测沙滩废弃物时的表现,对比了SSD(single shot multibox detector)与Faster-RCNN算法.实验结果表明:所提改进算法在实际检测中有较好的综合性能,且相比原算法Faster-RCNN_Inception-V2,轻量化改进后的Inception-V2特征提取网络卷积计算量减少51.8%,模型训练耗时缩短了9.1%,检测耗时减少了10.9%,各类别AP的平均值(mean average precision,mAP)增加了1.02%,可见所提的改进方法能够有效提高目标检测的准确率,减少检测耗时,并在沙滩废弃物检测上得到成功应用,为海滨城市的沙滩清理维护提供了技术支持与保障.展开更多
文摘Deep Neural Networks(DNNs)have become the tool of choice for machine learning practitioners today.One important aspect of designing a neural network is the choice of the activation function to be used at the neurons of the different layers.In this work,we introduce a four-output activation function called the Reflected Rectified Linear Unit(RRe LU)activation which considers both a feature and its negation during computation.Our activation function is"sparse",in that only two of the four possible outputs are active at a given time.We test our activation function on the standard MNIST and CIFAR-10 datasets,which are classification problems,as well as on a novel Computational Fluid Dynamics(CFD)dataset which is posed as a regression problem.On the baseline network for the MNIST dataset,having two hidden layers,our activation function improves the validation accuracy from 0.09 to 0.97 compared to the well-known Re LU activation.For the CIFAR-10 dataset,we use a deep baseline network that achieves 0.78 validation accuracy with 20 epochs but overfits the data.Using the RRe LU activation,we can achieve the same accuracy without overfitting the data.For the CFD dataset,we show that the RRe LU activation can reduce the number of epochs from 100(using Re LU)to 10 while obtaining the same levels of performance.
文摘修正线性单元(rectified linear unit,ReLU)是深度卷积神经网络常用的激活函数,但当输入为负数时,ReLU的输出为零,造成了零梯度问题;且当输入为正数时,ReLU的输出保持输入不变,使得ReLU函数的平均值恒大于零,引起了偏移现象,从而限制了深度卷积神经网络的学习速率和学习效果.针对ReLU函数的零梯度问题和偏移现象,根据"输出均值接近零的激活函数能够提升神经网络学习性能"原理对其进行改进,提出SLU(softplus linear unit)函数.首先,对负数输入部分进行softplus处理,使得负数输入时SLU函数的输出为负,从而输出平均值更接近于零,减缓了偏移现象;其次,为保证梯度平稳,对SLU的参数进行约束,并固定正数部分的参数;最后,根据SLU对正数部分的处理调整负数部分的参数,确保激活函数在零点处连续可导,信息得以双向传播.设计深度自编码模型在数据集MINST上进行无监督学习,设计网中网卷积神经网络模型在数据集CIFAR-10上进行监督学习.实验结果表明,与ReLU及其相关改进单元相比,基于SLU函数的神经网络模型具有更好的特征学习能力和更高的学习精度.
文摘弱小船舶目标实时检测因在海上搜救、无人船和海上交通管理等领域中的众多应用而备受关注。虽然基于深度学习的目标检测算法,如YOLO(you only look once)和SSD(single shot multibox detector)等取得了不错的目标检测性能,但是它们仍然无法实时有效检测出海上弱小船舶运动目标。针对此问题,文章提出了一种改进的深度学习网络结构,结合SELU(scaled exponential linear units)激活函数,有效解决了已有的YOLOv2算法对弱小目标检测率较低的不足以及YOLOv3算法中残差网络结构冗余的问题。实验表明,该文提出的方法在海上弱小船舶目标检测上,比原YOLO算法具有更高的检测精度、更快的检测速度和更优良的鲁棒性。该方法在低配硬件环境中仍具有实时性的特点,因此对算法的推广应用具有实际的意义。
文摘Currently,there are many limitations to classify images of small objects.In addition,there are limitations such as error detection due to external factors,and there is also a disadvantage that it is difficult to accurately distinguish between various objects.This paper uses a convolutional neural network(CNN)algorithm to recognize and classify object images of very small moths and obtain precise data images.A convolution neural network algorithm is used for image data classification,and the classified image is transformed into image data to learn the topological structure of the image.To improve the accuracy of the image classification and reduce the loss rate,a parameter for finding a fast-optimal point of image classification is set by a convolutional neural network and a pixel image as a preprocessor.As a result of this study,we applied a convolution neural network algorithm to classify the images of very small moths by capturing precise images of the moths.Experimental results showed that the accuracy of classification of very small moths was more than 90%.
文摘由于背景环境复杂,检测物体易受部分遮挡、天气以及光线变化等因素的影响,传统目标检测方法存在提取特征难、检测准确率低、检测耗时长等缺陷.为了改善传统目标检测方法存在的缺陷,实现快速准确的目标检测,提出了一种基于快速区域卷积神经网络(faster regions with convolutional neural network,Faster-RCNN)算法的轻量化改进方法,即针对算法Inception-V2特征提取网络进行轻量化改进,并以带泄露线性整流(leaky rectified linear unit,Leaky ReLU)作为激活函数,解决使用线性整流(rectified linear unit,ReLU)激活函数存在的神经元输入为负数时输出为0的问题.基于上述改进方法,选择沙滩废弃物的检测为案例以验证方法的有效性,并且结合不同特征提取网络在检测沙滩废弃物时的表现,对比了SSD(single shot multibox detector)与Faster-RCNN算法.实验结果表明:所提改进算法在实际检测中有较好的综合性能,且相比原算法Faster-RCNN_Inception-V2,轻量化改进后的Inception-V2特征提取网络卷积计算量减少51.8%,模型训练耗时缩短了9.1%,检测耗时减少了10.9%,各类别AP的平均值(mean average precision,mAP)增加了1.02%,可见所提的改进方法能够有效提高目标检测的准确率,减少检测耗时,并在沙滩废弃物检测上得到成功应用,为海滨城市的沙滩清理维护提供了技术支持与保障.
