摘要
获取渔业养殖鱼类生长态势的人工测量方法费时费力,且影响鱼的正常生长。为了实现水下鱼体信息动态感知和快速无损检测,该研究提出立体视觉下动态鱼体尺寸测量方法。通过双目立体视觉技术获取三维信息,再通过Mask-RCNN(Mask Region Convolution Neural Network)网络进行鱼体检测与精细分割,最后生成鱼表面的三维点云数据,计算得到自由活动下多条鱼的外形尺寸。试验结果表明,长度和宽度的平均相对误差分别在4.7%和9.2%左右。该研究满足了水产养殖环境下进行可视化管理、无接触测量鱼体尺寸的需要,可以为养殖过程中分级饲养和合理投饵提供参考依据。
Fish dimension information,especially length,is very important for aquaculture,which can be used for grading and developing bait strategy.In order to acquire accurate information on fish size,the traditional method of measurement has to take the fish out of the water,which is not only time-consuming and laborious but also may influence the growth rates of fishes.In this study,a dynamic measurement method for fish body dimension based on stereo vision was proposed,which could calculate dimension information of multiple fishes simultaneously without restricting their movements.It was implemented and verified by an intelligent monitor system designed and built by ourselves considering the hardware compatibility with satisfied integral performance.Through this system,the videos of underwater fish were captured and uploaded to the remote cloud server for further processing.Then three main procedures were developed including 3 D reconstruction,fish detection and segmentation,3 D points cloud processing,which was designed for size acquirement of fishes swimming freely in a real aquaculture environment.In the 3 D reconstruction part,in order to acquire the data for modeling,3 D information was restored from binocular images by camera calibration,stereo rectification,stereo matching in sequence.Firstly,the binocular was calibrated with a chessboard to get camera parameters including intrinsic matrix as well as relative translation and rotation of the left and right camera.Then,the captured binocular images were rectified to row-aligned according to parameters of the calibrated binocular camera.Finally,stereo matching based on the semi-global block matching method(SGBM)was applied to extract accurate 3 D information from rectified binocular image pairs and achieved 3 D reconstruction.In the fish detection and segmentation part,a Mask Region Convolution Neural Network(Mask-RCNN)was trained as a model to locate fishes in the image with a bounding box and extract pixels of fish in each bounding box to get raw segmentation.The raw segmentation was refined with an interactive segmentation method called GrabCut combining with some morphological processing algorithms to correct bias around the edge.In the 3 D points cloud processing part,two coordinate transformations were carried out to unify the cloud points of fishes with various locations and orientations.The transformation parameters were calculated based on three-dimension plane fitting of the contour points cloud and rotated ellipse fitting of the transformed points cloud respectively.After transformation,the length and width of the fish points cloud were parallel to axes.Therefore,the length and width of fish were the range of points cloud along the abscissa and ordinate axes.Experiments were conducted using the self-designed system and results including various species and sizes of fish were compared with those of manual measurements.It turned out that the average relative estimation error of length was about 4.7%and the average relative estimation error of width was about 9.2%.In terms of running time,the developed measurement system could process 2.5 frames per second for fish dimensions calculation.The experiment results also showed that the trained Mask-RCNN model achieved the precision of 0.88 and the recall of 84%with satisfied generalization performance.After segmentation refinement,the mean intersection over union increased from 78%to 81%,which exhibited the effectiveness of the refinement method.It also showed that the longer the fish length,the smaller the average relative error of the measurement.These results demonstrated that the proposed method was able to measure multiple underwater fish dimensions based on a stereoscopic vision method by using deep learning-based image segmentation algorithms and coordinates transformation method.This study could provide a novel idea for flexible measurement of fish body size and improve the level of dynamic information perception technology for rapid and non-destructive detection of underwater fish in aquaculture.
作者
李艳君
黄康为
项基
Li Yanjun;Huang Kangwei;Xiang Ji(Zhejiang University City College,Hangzhou 310015,China;College of Control Science and Engineering,Zhejiang University,Hangzhou 310027,China;College of Electrical Engineering,Zhejiang University,Hangzhou 310027,China)
出处
《农业工程学报》
EI
CAS
CSCD
北大核心
2020年第21期220-226,共7页
Transactions of the Chinese Society of Agricultural Engineering
基金
浙江省重点研发计划项目(2019C01150)。
关键词
鱼
机器视觉
三维重建
图像分割
深度学习
Mask-RCNN
三维点云处理
fish
machine vision
three-dimensional reconstruction
image segmentation
deep learning
Mask-RCNN
3D cloud points processing
