Ensuring food security has become a global challenge owing to climate change and population growth.High-throughput phenotyping can effectively drive crop genetic enhancement,which can potentially solve food crisis.Phe...Ensuring food security has become a global challenge owing to climate change and population growth.High-throughput phenotyping can effectively drive crop genetic enhancement,which can potentially solve food crisis.Phenotyping robot is an essential part of crop ground phenotyping information monitoring,although there are challenges such as the inability to adjust the fixed track width,poor load capacity of the detection robotic arm,and inability to fuse information in real-time.This study reports a phenotyping robot with a gantry-style chassis featuring an adjustable wheeltrack(1400-1600 mm)to adapt to different row spacing arrangements and reduced damage,and function effectively in both dry field and paddy field environments.A six-degree-of-freedom sensor gimbal with high payload capacity is also developed to enable precise height(1016-2096 mm)and angle ad-justments.Additionally,this study introduces an enhanced method for data acquisition from multiple imaging sensors through registration and fusion using Zhang's calibration and feature point extraction algorithm,calcu-lating a homography matrix for high-throughput data collection at fixed positions and heights.The experimental validation results demonstrate that the RMSE of the registration algorithm does not exceed 3 pixels.The gimbal data strongly correlated with that of a handheld instrument data(r^(2)>0.90).The robot is practical,reliable,and fully functional,offering a solid theoretical foundation and equipment support for high-throughput phenotyping.展开更多
High-throughput phenotyping collection technology is important in affecting the efficiency of crop breeding.This study introduces a novel autonomous navigation method for phenotyping robots that leverages ground-air c...High-throughput phenotyping collection technology is important in affecting the efficiency of crop breeding.This study introduces a novel autonomous navigation method for phenotyping robots that leverages ground-air collaboration to meet the demands of unmanned crop phenotypic data collection.The proposed method employs a UAV equipped with a Real-Time Kinematic(RTK)module for the construction of high-precision Field maps.It utilizes SegFormor-B0 semantic segmentation models to detect crop rows,and extracts key coordinate points of these rows,and generates navigation paths for the phenotyping robots by mapping these points to actual geographic coordinates.Furthermore,an adaptive controller based on the Pure Pursuit algorithm is proposed,which dynamically adjusts the steering angle of the phenotyping robot in real-time,according to the distance(),angular deviation()and the lateral deviation()between the robot's current position and its target position.This enables the robot to accurately trace paths in field environments.The results demonstrate that the mean absolute error(MAE)of the proposed method in extracting the centerline of potted plants area's rows is 2.83 cm,and the cropland's rows is 4.51 cm.The majority of global path tracking errors stay within 2 cm.In the potted plants area,99.1%of errors lie within this range,with a mean absolute error of 0.62 cm and a maximum error of 2.59 cm.In the cropland,72.4%of errors remain within this range,with a mean absolute error of 1.51 cm and a maximum error of 4.22 cm.Compared with traditional GNSS-based navigation methods and single vision methods,this method shows significant advantages in adapting to the dynamic growth of crops and complex field environments,which not only ensures that the phenotyping robot accurately travels along the crop rows during field operations to avoid damage to the crops,but also provides an efficient and accurate means of data acquisition for crop phenotyping.展开更多
Understanding the genetic basis of quantitative traits related to crop growth,yield,and stress response requires the acquisition of large-scale,high-quality phenotypic datasets.High-throughput phenotyping platforms ha...Understanding the genetic basis of quantitative traits related to crop growth,yield,and stress response requires the acquisition of large-scale,high-quality phenotypic datasets.High-throughput phenotyping platforms have become effective tools for meeting this requirement.Autonomous mobile robots have gained prominence owing to their ability to carry heavy payloads,their operational flexibility,and their proximity to crops,which allows for higher imaging resolution.In this study,we introduce PhenoRob-F(a phenotyping robot for the field),a cross-row,wheeled robot designed for efficient and automated phenotyping under field conditions.The mobile platform and phenotyping module of the robot were engineered to meet the specific demands of field pheno-typing,with integrated visual and satellite navigation systems enabling autonomous operation.We validated the performance of the robot through a series of experiments involving various crop canopies.By capturing RGB images of rice and wheat,we independently performed wheat ear detection and rice panicle segmentation.For wheat ear detection,we achieve a precision of 0.783,a recall of 0.822,and a mean average precision(mAP)of 0.853 when the YOLOv8m model is used.For rice panicle segmentation,the SegFormer_BO model yielded a mean intersection over union(mIoU)of 0.949 and an accuracy of 0.987.Additionally,by capturing RGB-D data of maize canopies,we performed 3D reconstructions to calculate plant height,achieving an R^(2) of 0.99 compared with manual measurements.Similar experiments with rapeseed yielded an R^(2) of 0.97.Near-infrared spectral data collected from drought-stressed rice plants enabled the classification of drought severity into five categories,with classification accuracies ranging from 0.977 to 0.996.Our results reveal that PhenoRob-F is an effective tool for high-throughput phenotyping and is capable of providing precise data to support phenotypic trait analysis and the selection of superior crop genotypes.展开更多
The continuous development of robot technology has made phenotype detection robots a key for extracting and analyzing phenotyping data in agriculture and forestry.The different applications of agricultural robots and ...The continuous development of robot technology has made phenotype detection robots a key for extracting and analyzing phenotyping data in agriculture and forestry.The different applications of agricultural robots and phenotype detection robots were discussed in this article.Further,the structural characteristics and information interaction modes of the current phenotype detection robots were summarized from the viewpoint of agriculture and forestry.The publications with keywords related to clustering distribution were analyzed and the currently available phenotype robots were classified.Additionally,a conclusion on the design criteria and evaluation system of plant phenotype detection robots was summarized and obtained,and the challenges and future development direction were proposed,which can provide a reference for the design and applications of agriculture and forestry robots.展开更多
In order to address the challenge of non-destructive detection of tomato fruit ripeness in controlled environments,this study proposed a real-time instance segmentation method based on the edge device.This method comb...In order to address the challenge of non-destructive detection of tomato fruit ripeness in controlled environments,this study proposed a real-time instance segmentation method based on the edge device.This method combined the principles of phenotype robots and machine vision based on deep learning.A compact and remotely controllable phenotype detection robot was employed to acquire precise data on tomato ripeness.The video data were then processed by using an efficient backbone and the FeatFlowNet structure for feature extraction and analysis of key-frame to non-key-frame mapping from video data.To enhance the diversity of training datasets and the generalization of the model,an innovative approach was chosen by using random enhancement techniques.Besides,the PolyLoss optimization technique was applied to further improve the accuracy of the ripeness multi-class detection tasks.Through validation,the method of this study achieved real-time processing speeds of 90.1 fps(RTX 3070Ti)and 65.5 fps(RTX 2060 S),with an average detection accuracy of 97%compared to manually measured results.This is more accurate and efficient than other instance segmentation models according to actual testing in a greenhouse.Therefore,the results of this research can be deployed in edge devices and provide technical support for unmanned greenhouse monitoring devices or fruit-picking robots in facility environments.展开更多
基金The work was supported by the National Key Research and Development Program of China(Grant No.2021YFD2000101).
文摘Ensuring food security has become a global challenge owing to climate change and population growth.High-throughput phenotyping can effectively drive crop genetic enhancement,which can potentially solve food crisis.Phenotyping robot is an essential part of crop ground phenotyping information monitoring,although there are challenges such as the inability to adjust the fixed track width,poor load capacity of the detection robotic arm,and inability to fuse information in real-time.This study reports a phenotyping robot with a gantry-style chassis featuring an adjustable wheeltrack(1400-1600 mm)to adapt to different row spacing arrangements and reduced damage,and function effectively in both dry field and paddy field environments.A six-degree-of-freedom sensor gimbal with high payload capacity is also developed to enable precise height(1016-2096 mm)and angle ad-justments.Additionally,this study introduces an enhanced method for data acquisition from multiple imaging sensors through registration and fusion using Zhang's calibration and feature point extraction algorithm,calcu-lating a homography matrix for high-throughput data collection at fixed positions and heights.The experimental validation results demonstrate that the RMSE of the registration algorithm does not exceed 3 pixels.The gimbal data strongly correlated with that of a handheld instrument data(r^(2)>0.90).The robot is practical,reliable,and fully functional,offering a solid theoretical foundation and equipment support for high-throughput phenotyping.
基金supported by the National Key Research and Development Program of China(2022YFD2002304,2021YFD1200504)the National Natural Science Foundation of China(32471992)Key Core Technology Project in Agriculture of Hubei Province(HBNYHXGG2023-9).
文摘High-throughput phenotyping collection technology is important in affecting the efficiency of crop breeding.This study introduces a novel autonomous navigation method for phenotyping robots that leverages ground-air collaboration to meet the demands of unmanned crop phenotypic data collection.The proposed method employs a UAV equipped with a Real-Time Kinematic(RTK)module for the construction of high-precision Field maps.It utilizes SegFormor-B0 semantic segmentation models to detect crop rows,and extracts key coordinate points of these rows,and generates navigation paths for the phenotyping robots by mapping these points to actual geographic coordinates.Furthermore,an adaptive controller based on the Pure Pursuit algorithm is proposed,which dynamically adjusts the steering angle of the phenotyping robot in real-time,according to the distance(),angular deviation()and the lateral deviation()between the robot's current position and its target position.This enables the robot to accurately trace paths in field environments.The results demonstrate that the mean absolute error(MAE)of the proposed method in extracting the centerline of potted plants area's rows is 2.83 cm,and the cropland's rows is 4.51 cm.The majority of global path tracking errors stay within 2 cm.In the potted plants area,99.1%of errors lie within this range,with a mean absolute error of 0.62 cm and a maximum error of 2.59 cm.In the cropland,72.4%of errors remain within this range,with a mean absolute error of 1.51 cm and a maximum error of 4.22 cm.Compared with traditional GNSS-based navigation methods and single vision methods,this method shows significant advantages in adapting to the dynamic growth of crops and complex field environments,which not only ensures that the phenotyping robot accurately travels along the crop rows during field operations to avoid damage to the crops,but also provides an efficient and accurate means of data acquisition for crop phenotyping.
基金This work was supported by the National Key Research and Development Program of China(2021YFD1200504,2022YFD2002304)the National Natural Science Foundation of China(32471992)+1 种基金the Key Core Technology Project in Agriculture of Hubei Province(HBNYHXGG2023-9)the Supporting Project for High-Quality Development of the Seed Industry of Hubei Province(HBZY2023B001-06).
文摘Understanding the genetic basis of quantitative traits related to crop growth,yield,and stress response requires the acquisition of large-scale,high-quality phenotypic datasets.High-throughput phenotyping platforms have become effective tools for meeting this requirement.Autonomous mobile robots have gained prominence owing to their ability to carry heavy payloads,their operational flexibility,and their proximity to crops,which allows for higher imaging resolution.In this study,we introduce PhenoRob-F(a phenotyping robot for the field),a cross-row,wheeled robot designed for efficient and automated phenotyping under field conditions.The mobile platform and phenotyping module of the robot were engineered to meet the specific demands of field pheno-typing,with integrated visual and satellite navigation systems enabling autonomous operation.We validated the performance of the robot through a series of experiments involving various crop canopies.By capturing RGB images of rice and wheat,we independently performed wheat ear detection and rice panicle segmentation.For wheat ear detection,we achieve a precision of 0.783,a recall of 0.822,and a mean average precision(mAP)of 0.853 when the YOLOv8m model is used.For rice panicle segmentation,the SegFormer_BO model yielded a mean intersection over union(mIoU)of 0.949 and an accuracy of 0.987.Additionally,by capturing RGB-D data of maize canopies,we performed 3D reconstructions to calculate plant height,achieving an R^(2) of 0.99 compared with manual measurements.Similar experiments with rapeseed yielded an R^(2) of 0.97.Near-infrared spectral data collected from drought-stressed rice plants enabled the classification of drought severity into five categories,with classification accuracies ranging from 0.977 to 0.996.Our results reveal that PhenoRob-F is an effective tool for high-throughput phenotyping and is capable of providing precise data to support phenotypic trait analysis and the selection of superior crop genotypes.
基金funded by the Construction of Collaborative Innovation Center of Beijing Academy of Agricultural and Forestry Sciences(KJCX201917)Beijing Nova Program(Z211100002121065)Science and Technology Innovation Special Construction Funded Program of Beijing Academy of Agriculture and Forestry Sciences(KJCX20210413).
文摘The continuous development of robot technology has made phenotype detection robots a key for extracting and analyzing phenotyping data in agriculture and forestry.The different applications of agricultural robots and phenotype detection robots were discussed in this article.Further,the structural characteristics and information interaction modes of the current phenotype detection robots were summarized from the viewpoint of agriculture and forestry.The publications with keywords related to clustering distribution were analyzed and the currently available phenotype robots were classified.Additionally,a conclusion on the design criteria and evaluation system of plant phenotype detection robots was summarized and obtained,and the challenges and future development direction were proposed,which can provide a reference for the design and applications of agriculture and forestry robots.
基金funded by the National Key R&D Program(Grant No.2022YFD2002305)Beijing Nova Program(Grant No.Z211100002121065,20220484202)Collaborative Innovation Center of Beijing Academy of Agricultural and Forestry Sciences(Grant No.KJCX201917).
文摘In order to address the challenge of non-destructive detection of tomato fruit ripeness in controlled environments,this study proposed a real-time instance segmentation method based on the edge device.This method combined the principles of phenotype robots and machine vision based on deep learning.A compact and remotely controllable phenotype detection robot was employed to acquire precise data on tomato ripeness.The video data were then processed by using an efficient backbone and the FeatFlowNet structure for feature extraction and analysis of key-frame to non-key-frame mapping from video data.To enhance the diversity of training datasets and the generalization of the model,an innovative approach was chosen by using random enhancement techniques.Besides,the PolyLoss optimization technique was applied to further improve the accuracy of the ripeness multi-class detection tasks.Through validation,the method of this study achieved real-time processing speeds of 90.1 fps(RTX 3070Ti)and 65.5 fps(RTX 2060 S),with an average detection accuracy of 97%compared to manually measured results.This is more accurate and efficient than other instance segmentation models according to actual testing in a greenhouse.Therefore,the results of this research can be deployed in edge devices and provide technical support for unmanned greenhouse monitoring devices or fruit-picking robots in facility environments.
