针对室外街道的行人检测与跟踪,提出一种改进YOLOv3与简单在线实时跟踪(simple online and real-time tracking,SORT)算法相结合的检测及跟踪方法。首先,引入距离和比例交并比(distance and proportional-IOU,DPIOU)损失,将原有的损失...针对室外街道的行人检测与跟踪,提出一种改进YOLOv3与简单在线实时跟踪(simple online and real-time tracking,SORT)算法相结合的检测及跟踪方法。首先,引入距离和比例交并比(distance and proportional-IOU,DPIOU)损失,将原有的损失函数中的均方误差(mean square error,MSE)部分进行变化,从而得到更精确的检测框;其次,将网络结构中的RestNet进行优化,改变下采样区域,增加池化层,进而减少特征信息的丢失;最后将检测结果输入SORT算法进行建模和匹配。实验结果表明,在室外街道的场景下,改进的算法与YOLOv3相比较,损失值收敛更快,平均准确率高出4.85%,跟踪准确率上升3.4%,同时,模型的速度有所提高,最快可达14.39 FPS。展开更多
Compared with manual sorting of citrus fruit,vision-based sorting solutions can help achieve higher accuracy and efficiency.In this study,we present a vision system based on CNN-LSTM,which can cooperate with robotic g...Compared with manual sorting of citrus fruit,vision-based sorting solutions can help achieve higher accuracy and efficiency.In this study,we present a vision system based on CNN-LSTM,which can cooperate with robotic grippers for real-time sorting and is readily applicable to various citrus processing plants.A CNN-based detector was adopted to detect the defective oranges in view and temporarily classify them into corresponding types,and an LSTM-based predictor was used to predict the position of the oranges in a future frame based on image sequential data.The fusion of CNN and LSTM networks enabled the system to track defective ones during rotation and identify their true types,and their future path was also predicted which is vital for predictive control of visually guided robotic grasping.High detection accuracy of 94.1%was obtained based on experimental results,and the error for path prediction was within 4.33 pixels 40 frames later.The average time to process a frame was between 28 and 62 frames per second,which also satisfied real-time performance.The results proved the potential of the proposed system for automated citrus sorting with good precision and efficiency,and it can be readily extended to other fruit crops featuring high versatility.展开更多
文摘针对室外街道的行人检测与跟踪,提出一种改进YOLOv3与简单在线实时跟踪(simple online and real-time tracking,SORT)算法相结合的检测及跟踪方法。首先,引入距离和比例交并比(distance and proportional-IOU,DPIOU)损失,将原有的损失函数中的均方误差(mean square error,MSE)部分进行变化,从而得到更精确的检测框;其次,将网络结构中的RestNet进行优化,改变下采样区域,增加池化层,进而减少特征信息的丢失;最后将检测结果输入SORT算法进行建模和匹配。实验结果表明,在室外街道的场景下,改进的算法与YOLOv3相比较,损失值收敛更快,平均准确率高出4.85%,跟踪准确率上升3.4%,同时,模型的速度有所提高,最快可达14.39 FPS。
基金National Key R&D Program(2020YFD1000101)and Special Funds for the Construction of Industrial Technology System of Modern Agriculture(Citrus)(CARS-26)Construction Project of Citrus Whole Course Mechanized Scientifific Research Base(Agricultural Development Facility 297[2017]19),Hubei Agricultural Science and Technology Innovation Action Project.
文摘Compared with manual sorting of citrus fruit,vision-based sorting solutions can help achieve higher accuracy and efficiency.In this study,we present a vision system based on CNN-LSTM,which can cooperate with robotic grippers for real-time sorting and is readily applicable to various citrus processing plants.A CNN-based detector was adopted to detect the defective oranges in view and temporarily classify them into corresponding types,and an LSTM-based predictor was used to predict the position of the oranges in a future frame based on image sequential data.The fusion of CNN and LSTM networks enabled the system to track defective ones during rotation and identify their true types,and their future path was also predicted which is vital for predictive control of visually guided robotic grasping.High detection accuracy of 94.1%was obtained based on experimental results,and the error for path prediction was within 4.33 pixels 40 frames later.The average time to process a frame was between 28 and 62 frames per second,which also satisfied real-time performance.The results proved the potential of the proposed system for automated citrus sorting with good precision and efficiency,and it can be readily extended to other fruit crops featuring high versatility.
