Mature soybean phenotyping is an important process in soybean breeding;however, the manual process is time-consuming and labor-intensive. Therefore, a novel approach that is rapid, accurate and highly precise is requi...Mature soybean phenotyping is an important process in soybean breeding;however, the manual process is time-consuming and labor-intensive. Therefore, a novel approach that is rapid, accurate and highly precise is required to obtain the phenotypic data of soybean stems, pods and seeds. In this research, we propose a mature soybean phenotype measurement algorithm called Soybean Phenotype Measure-instance Segmentation(SPM-IS). SPM-IS is based on a feature pyramid network, Principal Component Analysis(PCA) and instance segmentation. We also propose a new method that uses PCA to locate and measure the length and width of a target object via image instance segmentation. After 60,000 iterations, the maximum mean Average Precision(m AP) of the mask and box was able to reach 95.7%. The correlation coefficients R^(2) of the manual measurement and SPM-IS measurement of the pod length, pod width, stem length, complete main stem length, seed length and seed width were 0.9755, 0.9872, 0.9692, 0.9803,0.9656, and 0.9716, respectively. The correlation coefficients R^(2) of the manual counting and SPM-IS counting of pods, stems and seeds were 0.9733, 0.9872, and 0.9851, respectively. The above results show that SPM-IS is a robust measurement and counting algorithm that can reduce labor intensity, improve efficiency and speed up the soybean breeding process.展开更多
Instance segmentation plays an important role in image processing.The Deep Snake algorithm based on contour iteration deforms an initial bounding box to an instance contour end-to-end,which can improve the performance...Instance segmentation plays an important role in image processing.The Deep Snake algorithm based on contour iteration deforms an initial bounding box to an instance contour end-to-end,which can improve the performance of instance segmentation,but has defects such as slow segmentation speed and sub-optimal initial contour.To solve these problems,a real-time instance segmentation algorithm based on contour learning was proposed.Firstly,ShuffleNet V2 was used as backbone network,and the receptive field of the model was expanded by using a 5×5 convolution kernel.Secondly,a lightweight up-sampling module,multi-stage aggregation(MSA),performs residual fusion of multi-layer features,which not only improves segmentation speed,but also extracts effective features more comprehensively.Thirdly,a contour initialization method for network learning was designed,and a global contour feature aggregation mechanism was used to return a coarse contour,which solves the problem of excessive error between manually initialized contour and real contour.Finally,the Snake deformation module was used to iteratively optimize the coarse contour to obtain the final instance contour.The experimental results showed that the proposed method improved the instance segmentation accuracy on semantic boundaries dataset(SBD),Cityscapes and Kins datasets,and the average precision reached 55.8 on the SBD;Compared with Deep Snake,the model parameters were reduced by 87.2%,calculation amount was reduced by 78.3%,and segmentation speed reached 39.8 frame·s^(−1) when instance segmentation was performed on an image with a size of 512×512 pixels on a 2080Ti GPU.The proposed method can reduce resource consumption,realize instance segmentation tasks quickly and accurately,and therefore is more suitable for embedded platforms with limited resources.展开更多
In actual traffic scenarios,precise recognition of traffic participants,such as vehicles and pedestrians,is crucial for intelligent transportation.This study proposes an improved algorithm built on Mask-RCNN to enhanc...In actual traffic scenarios,precise recognition of traffic participants,such as vehicles and pedestrians,is crucial for intelligent transportation.This study proposes an improved algorithm built on Mask-RCNN to enhance the ability of autonomous driving systems to recognize traffic participants.The algorithmincorporates long and shortterm memory networks and the fused attention module(GSAM,GCT,and Spatial Attention Module)to enhance the algorithm’s capability to process both global and local information.Additionally,to increase the network’s initial operation stability,the original network activation function was replaced with Gaussian error linear unit.Experiments were conducted using the publicly available Cityscapes dataset.Comparing the test results,it was observed that the revised algorithmoutperformed the original algorithmin terms of AP_(50),AP_(75),and othermetrics by 8.7%and 9.6%for target detection and 12.5%and 13.3%for segmentation.展开更多
The application of robotic grasping for agricultural products pushes automation in agriculture-related industries.Cucumber,a common vegetable in greenhouses and supermarkets,often needs to be grasped from a cluttered ...The application of robotic grasping for agricultural products pushes automation in agriculture-related industries.Cucumber,a common vegetable in greenhouses and supermarkets,often needs to be grasped from a cluttered scene.In order to realize efficient grasping in cluttered scenes,a fully automatic cucumber recognition,grasping,and palletizing robot system was constructed in this paper.The system adopted Yolact++deep learning network to segment cucumber instances.An early fusion method of F-RGBD was proposed,which increases the algorithm's discriminative ability for these appearance-similar cucumbers at different depths,and at different occlusion degrees.The results of the comparative experiment of the F-RGBD dataset and the common RGB dataset on Yolact++prove the positive effect of the F-RGBD fusion method.Its segmentation masks have higher quality,are more continuous,and are less false positive for prioritizing-grasping prediction.Based on the segmentation result,a 4D grab line prediction method was proposed for cucumber grasping.And the cucumber detection experiment in cluttered scenarios is carried out in the real world.The success rate is 93.67%and the average sorting time is 9.87 s.The effectiveness of the cucumber segmentation and grasping pose acquisition method is verified by experiments.展开更多
Objective In tongue diagnosis,the location,color,and distribution of spots can be used to speculate on the viscera and severity of the heat evil.This work focuses on the image analysis method of artificial intelligenc...Objective In tongue diagnosis,the location,color,and distribution of spots can be used to speculate on the viscera and severity of the heat evil.This work focuses on the image analysis method of artificial intelligence(AI)to study the spotted tongue recognition of traditional Chinese medicine(TCM).Methods A model of spotted tongue recognition and extraction is designed,which is based on the principle of image deep learning and instance segmentation.This model includes multiscale feature map generation,region proposal searching,and target region recognition.Firstly,deep convolution network is used to build multiscale low-and high-abstraction feature maps after which,target candidate box generation algorithm and selection strategy are used to select high-quality target candidate regions.Finally,classification network is used for classifying target regions and calculating target region pixels.As a result,the region segmentation of spotted tongue is obtained.Under non-standard illumination conditions,various tongue images were taken by mobile phones,and experiments were conducted.Results The spotted tongue recognition achieved an area under curve(AUC)of 92.40%,an accuracy of 84.30%with a sensitivity of 88.20%,a specificity of 94.19%,a recall of 88.20%,a regional pixel accuracy index pixel accuracy(PA)of 73.00%,a mean pixel accuracy(m PA)of73.00%,an intersection over union(Io U)of 60.00%,and a mean intersection over union(mIo U)of 56.00%.Conclusion The results of the study verify that the model is suitable for the application of the TCM tongue diagnosis system.Spotted tongue recognition via multiscale convolutional neural network(CNN)would help to improve spot classification and the accurate extraction of pixels of spot area as well as provide a practical method for intelligent tongue diagnosis of TCM.展开更多
Instance co-segmentation aims to segment the co-occurrent instances among two images.This task heavily relies on instance-related cues provided by co-peaks,which are generally estimated by exhaustively exploiting all ...Instance co-segmentation aims to segment the co-occurrent instances among two images.This task heavily relies on instance-related cues provided by co-peaks,which are generally estimated by exhaustively exploiting all paired candidates in point-to-point patterns.However,such patterns could yield a high number of false-positive co-peaks,resulting in over-segmentation whenever there are mutual occlusions.To tackle with this issue,this paper proposes an instance co-segmentation method via tensor-based salient co-peak search(TSCPS-ICS).The proposed method explores high-order correlations via triple-to-triple matching among feature maps to find reliable co-peaks with the help of co-saliency detection.The proposed method is shown to capture more accurate intra-peaks and inter-peaks among feature maps,reducing the false-positive rate of co-peak search.Upon having accurate co-peaks,one can efficiently infer responses of the targeted instance.Experiments on four benchmark datasets validate the superior performance of the proposed method.展开更多
Cell instance segmentation is a fundamental task for many biological applications,especially for packed cells in three-dimensional(3D)microscope images that can fully display cellular morphology.Image processing algor...Cell instance segmentation is a fundamental task for many biological applications,especially for packed cells in three-dimensional(3D)microscope images that can fully display cellular morphology.Image processing algorithms based on neural networks and feature engineering have enabled great progress in two-dimensional(2D)instance segmentation.However,current methods cannot achieve high segmentation accuracy for irregular cells in 3D images.In this study,we introduce a universal,morphology-based 3D instance segmentation algorithm called Crop Once Merge Twice(C1M2),which can segment cells from a wide range of image types and does not require nucleus images.C1M2 can be extended to quantify the fluorescence intensity of fluorescent proteins and antibodies and automatically annotate their expression levels in individual cells.Our results suggest that C1M2 can serve as a tissue cytometry for 3D histopathological assays by quantifying fluorescence intensity with spatial localization and morphological information.展开更多
现有基于红外图像的绝缘子发热缺陷识别方法,存在目标区域提取精度有限、温度提取受环境因素影响较大等问题。为此,本文提出一种复合绝缘子过热识别新方法:首先改进单阶段绝缘子实例分割算法You Only Look At CoefficienTs(YOLACT),引...现有基于红外图像的绝缘子发热缺陷识别方法,存在目标区域提取精度有限、温度提取受环境因素影响较大等问题。为此,本文提出一种复合绝缘子过热识别新方法:首先改进单阶段绝缘子实例分割算法You Only Look At CoefficienTs(YOLACT),引入嵌有Efficient Local Attention(ELA)机制的MobileNetV2作主干网络提升检测速度,融合特征金字塔网络(Feature Pyramid Network,FPN)各层特征图并加入聚焦纯卷积特征提取模块提高特征图质量;然后使用改进算法识别红外图像中复合绝缘子外轮廓,定位其棒芯位置;最后依据红外图像热矩阵获取棒芯温度矩阵,对比温度变化判断是否异常。实际生产环境中,本文方法整体准确率达到975,算法总耗时125ms;改进实例分割算法平均交互比(mIOU)为9297,平均像素准确率(mPA)为9615,每秒帧数(FPS)为19。结果显示,此方法分割定位效果好,能滤除多数环境因素导致的温度识别误差,为绝缘子温度异常识别提供新方案。展开更多
基金supported by the National Natural Science Foundation of China (31400074, 31471516, 31271747, and 30971809)the Natural Science Foundation of Heilongjiang Province of China(ZD201213)the Heilongjiang Postdoctoral Science Foundation(LBH-Q18025)。
文摘Mature soybean phenotyping is an important process in soybean breeding;however, the manual process is time-consuming and labor-intensive. Therefore, a novel approach that is rapid, accurate and highly precise is required to obtain the phenotypic data of soybean stems, pods and seeds. In this research, we propose a mature soybean phenotype measurement algorithm called Soybean Phenotype Measure-instance Segmentation(SPM-IS). SPM-IS is based on a feature pyramid network, Principal Component Analysis(PCA) and instance segmentation. We also propose a new method that uses PCA to locate and measure the length and width of a target object via image instance segmentation. After 60,000 iterations, the maximum mean Average Precision(m AP) of the mask and box was able to reach 95.7%. The correlation coefficients R^(2) of the manual measurement and SPM-IS measurement of the pod length, pod width, stem length, complete main stem length, seed length and seed width were 0.9755, 0.9872, 0.9692, 0.9803,0.9656, and 0.9716, respectively. The correlation coefficients R^(2) of the manual counting and SPM-IS counting of pods, stems and seeds were 0.9733, 0.9872, and 0.9851, respectively. The above results show that SPM-IS is a robust measurement and counting algorithm that can reduce labor intensity, improve efficiency and speed up the soybean breeding process.
基金supported by National Key Research and Development Program(No.2022YFE0112400)National Natural Science Foundation of China(No.21706096)Natural Science Foundation of Jiangsu Province(No.BK20160162).
文摘Instance segmentation plays an important role in image processing.The Deep Snake algorithm based on contour iteration deforms an initial bounding box to an instance contour end-to-end,which can improve the performance of instance segmentation,but has defects such as slow segmentation speed and sub-optimal initial contour.To solve these problems,a real-time instance segmentation algorithm based on contour learning was proposed.Firstly,ShuffleNet V2 was used as backbone network,and the receptive field of the model was expanded by using a 5×5 convolution kernel.Secondly,a lightweight up-sampling module,multi-stage aggregation(MSA),performs residual fusion of multi-layer features,which not only improves segmentation speed,but also extracts effective features more comprehensively.Thirdly,a contour initialization method for network learning was designed,and a global contour feature aggregation mechanism was used to return a coarse contour,which solves the problem of excessive error between manually initialized contour and real contour.Finally,the Snake deformation module was used to iteratively optimize the coarse contour to obtain the final instance contour.The experimental results showed that the proposed method improved the instance segmentation accuracy on semantic boundaries dataset(SBD),Cityscapes and Kins datasets,and the average precision reached 55.8 on the SBD;Compared with Deep Snake,the model parameters were reduced by 87.2%,calculation amount was reduced by 78.3%,and segmentation speed reached 39.8 frame·s^(−1) when instance segmentation was performed on an image with a size of 512×512 pixels on a 2080Ti GPU.The proposed method can reduce resource consumption,realize instance segmentation tasks quickly and accurately,and therefore is more suitable for embedded platforms with limited resources.
基金the National Natural Science Foundation of China(52175236)Qingdao People’s Livelihood Science and Technology Plan(19-6-1-88-nsh).
文摘In actual traffic scenarios,precise recognition of traffic participants,such as vehicles and pedestrians,is crucial for intelligent transportation.This study proposes an improved algorithm built on Mask-RCNN to enhance the ability of autonomous driving systems to recognize traffic participants.The algorithmincorporates long and shortterm memory networks and the fused attention module(GSAM,GCT,and Spatial Attention Module)to enhance the algorithm’s capability to process both global and local information.Additionally,to increase the network’s initial operation stability,the original network activation function was replaced with Gaussian error linear unit.Experiments were conducted using the publicly available Cityscapes dataset.Comparing the test results,it was observed that the revised algorithmoutperformed the original algorithmin terms of AP_(50),AP_(75),and othermetrics by 8.7%and 9.6%for target detection and 12.5%and 13.3%for segmentation.
基金supported by the Beijing Innovation Consortium of Agriculture Research System (BAIC12).
文摘The application of robotic grasping for agricultural products pushes automation in agriculture-related industries.Cucumber,a common vegetable in greenhouses and supermarkets,often needs to be grasped from a cluttered scene.In order to realize efficient grasping in cluttered scenes,a fully automatic cucumber recognition,grasping,and palletizing robot system was constructed in this paper.The system adopted Yolact++deep learning network to segment cucumber instances.An early fusion method of F-RGBD was proposed,which increases the algorithm's discriminative ability for these appearance-similar cucumbers at different depths,and at different occlusion degrees.The results of the comparative experiment of the F-RGBD dataset and the common RGB dataset on Yolact++prove the positive effect of the F-RGBD fusion method.Its segmentation masks have higher quality,are more continuous,and are less false positive for prioritizing-grasping prediction.Based on the segmentation result,a 4D grab line prediction method was proposed for cucumber grasping.And the cucumber detection experiment in cluttered scenarios is carried out in the real world.The success rate is 93.67%and the average sorting time is 9.87 s.The effectiveness of the cucumber segmentation and grasping pose acquisition method is verified by experiments.
基金Anhui Province College Natural Science Fund Key Project of China(KJ2020ZD77)the Project of Education Department of Anhui Province(KJ2020A0379)。
文摘Objective In tongue diagnosis,the location,color,and distribution of spots can be used to speculate on the viscera and severity of the heat evil.This work focuses on the image analysis method of artificial intelligence(AI)to study the spotted tongue recognition of traditional Chinese medicine(TCM).Methods A model of spotted tongue recognition and extraction is designed,which is based on the principle of image deep learning and instance segmentation.This model includes multiscale feature map generation,region proposal searching,and target region recognition.Firstly,deep convolution network is used to build multiscale low-and high-abstraction feature maps after which,target candidate box generation algorithm and selection strategy are used to select high-quality target candidate regions.Finally,classification network is used for classifying target regions and calculating target region pixels.As a result,the region segmentation of spotted tongue is obtained.Under non-standard illumination conditions,various tongue images were taken by mobile phones,and experiments were conducted.Results The spotted tongue recognition achieved an area under curve(AUC)of 92.40%,an accuracy of 84.30%with a sensitivity of 88.20%,a specificity of 94.19%,a recall of 88.20%,a regional pixel accuracy index pixel accuracy(PA)of 73.00%,a mean pixel accuracy(m PA)of73.00%,an intersection over union(Io U)of 60.00%,and a mean intersection over union(mIo U)of 56.00%.Conclusion The results of the study verify that the model is suitable for the application of the TCM tongue diagnosis system.Spotted tongue recognition via multiscale convolutional neural network(CNN)would help to improve spot classification and the accurate extraction of pixels of spot area as well as provide a practical method for intelligent tongue diagnosis of TCM.
基金supported in part by the National Natural Science Foundation of China (Grant Nos.U21A20520,62172112)the Key-Area Research and Development of Guangdong Province (2022A0505050014,2020B1111190001)+1 种基金the National Key Research and Development Program of China (2022YFE0112200)the Key-Area Research and Development Program of Guangzhou City (202206030009).
文摘Instance co-segmentation aims to segment the co-occurrent instances among two images.This task heavily relies on instance-related cues provided by co-peaks,which are generally estimated by exhaustively exploiting all paired candidates in point-to-point patterns.However,such patterns could yield a high number of false-positive co-peaks,resulting in over-segmentation whenever there are mutual occlusions.To tackle with this issue,this paper proposes an instance co-segmentation method via tensor-based salient co-peak search(TSCPS-ICS).The proposed method explores high-order correlations via triple-to-triple matching among feature maps to find reliable co-peaks with the help of co-saliency detection.The proposed method is shown to capture more accurate intra-peaks and inter-peaks among feature maps,reducing the false-positive rate of co-peak search.Upon having accurate co-peaks,one can efficiently infer responses of the targeted instance.Experiments on four benchmark datasets validate the superior performance of the proposed method.
基金the National Key Research and Development Program of China(2017YFA0700403,2017YFA0700402)the National Natural Science Foundation of China(62061160490)+2 种基金the Applied Fundamental Research of Wuhan(2020010601012167)the Fundamental Research Funds for the Central Universities(2019kfy XMBZ022)the Innovation Fund of Wuhan National Laboratory for Optoelectronics(WNLO)。
文摘Cell instance segmentation is a fundamental task for many biological applications,especially for packed cells in three-dimensional(3D)microscope images that can fully display cellular morphology.Image processing algorithms based on neural networks and feature engineering have enabled great progress in two-dimensional(2D)instance segmentation.However,current methods cannot achieve high segmentation accuracy for irregular cells in 3D images.In this study,we introduce a universal,morphology-based 3D instance segmentation algorithm called Crop Once Merge Twice(C1M2),which can segment cells from a wide range of image types and does not require nucleus images.C1M2 can be extended to quantify the fluorescence intensity of fluorescent proteins and antibodies and automatically annotate their expression levels in individual cells.Our results suggest that C1M2 can serve as a tissue cytometry for 3D histopathological assays by quantifying fluorescence intensity with spatial localization and morphological information.
文摘现有基于红外图像的绝缘子发热缺陷识别方法,存在目标区域提取精度有限、温度提取受环境因素影响较大等问题。为此,本文提出一种复合绝缘子过热识别新方法:首先改进单阶段绝缘子实例分割算法You Only Look At CoefficienTs(YOLACT),引入嵌有Efficient Local Attention(ELA)机制的MobileNetV2作主干网络提升检测速度,融合特征金字塔网络(Feature Pyramid Network,FPN)各层特征图并加入聚焦纯卷积特征提取模块提高特征图质量;然后使用改进算法识别红外图像中复合绝缘子外轮廓,定位其棒芯位置;最后依据红外图像热矩阵获取棒芯温度矩阵,对比温度变化判断是否异常。实际生产环境中,本文方法整体准确率达到975,算法总耗时125ms;改进实例分割算法平均交互比(mIOU)为9297,平均像素准确率(mPA)为9615,每秒帧数(FPS)为19。结果显示,此方法分割定位效果好,能滤除多数环境因素导致的温度识别误差,为绝缘子温度异常识别提供新方案。
