Computer vision is increasingly used in farmers'fields and agricultural experiments to quantify important traits.Imaging setups with a sub-millimeter ground sampling distance enable the detection and tracking of p...Computer vision is increasingly used in farmers'fields and agricultural experiments to quantify important traits.Imaging setups with a sub-millimeter ground sampling distance enable the detection and tracking of plant features,including size,shape,and colour.Although today's AI-driven foundation models segment almost any object in an image,they still fail for complex plant canopies.To improve model performance,the global wheat dataset consortium assembled a diverse set of images from experiments around the globe.After the head detection dataset(GWHD),the new dataset targets a full semantic segmentation(GWFSS)of organs(leaves,stems and spikes)covering all developmental stages.Images were collected by 11 institutions using a wide range of imaging setups.Two datasets are provided:ⅰ)a set of 1096 diverse images in which all organs were labelled at the pixel level,and(ⅱ)a dataset of 52,078 images without annotations available for additional training.The labelled set was used to train segmentation models based on DeepLabV3Plus and Segformer.Our Segformer model performed slightly better than DeepLabV3Plus with a mIOU for leaves and spikes of ca.90%.However,the precision for stems with 54%was rather lower.The major advantages over published models are:ⅰ)the exclusion of weeds from the wheat canopy,ⅱ)the detection of all wheat features including necrotic and se-nescent tissues and its separation from crop residues.This facilitates further development in classifying healthy vs.unhealthy tissue to address the increasing need for accurate quantification of senescence and diseases in wheat canopies.展开更多
Deep learning has shown potential in domains with large-scale annotated datasets.However,manual annotation is expensive,time-consuming,and tedious.Pixel-level annotations are particularly costly for semantic segmentat...Deep learning has shown potential in domains with large-scale annotated datasets.However,manual annotation is expensive,time-consuming,and tedious.Pixel-level annotations are particularly costly for semantic segmentation in images with dense irregular patterns of object instances,such as in plant images.In this work,we propose a method for developing high-performing deep learning models for semantic segmentation of such images utilizing little manual annotation.As a use case,we focus on wheat head segmentation.We synthesize a computationally annotated dataset—using a few annotated images,a short unannotated video clip of a wheat field,and several video clips with no wheat—to train a customized U-Net model.Considering the distribution shift between the synthesized and real images,we apply three domain adaptation steps to gradually bridge the domain gap.Only using two annotated images,we achieved a Dice score of 0.89 on the internal test set.When further evaluated on a diverse external dataset collected from 18 different domains across five countries,this model achieved a Dice score of 0.73.To expose the model to images from different growth stages and environmental conditions,we incorporated two annotated images from each of the 18 domains to further fine-tune the model.This increased the Dice score to 0.91.The result highlights the utility of the proposed approach in the absence of large-annotated datasets.Although our use case is wheat head segmentation,the proposed approach can be extended to other segmentation tasks with similar characteristics of irregularly repeating patterns of object instances.展开更多
基金Global wheat was directly supported by Analytics for the Australian Grains Industry(AAGI).
文摘Computer vision is increasingly used in farmers'fields and agricultural experiments to quantify important traits.Imaging setups with a sub-millimeter ground sampling distance enable the detection and tracking of plant features,including size,shape,and colour.Although today's AI-driven foundation models segment almost any object in an image,they still fail for complex plant canopies.To improve model performance,the global wheat dataset consortium assembled a diverse set of images from experiments around the globe.After the head detection dataset(GWHD),the new dataset targets a full semantic segmentation(GWFSS)of organs(leaves,stems and spikes)covering all developmental stages.Images were collected by 11 institutions using a wide range of imaging setups.Two datasets are provided:ⅰ)a set of 1096 diverse images in which all organs were labelled at the pixel level,and(ⅱ)a dataset of 52,078 images without annotations available for additional training.The labelled set was used to train segmentation models based on DeepLabV3Plus and Segformer.Our Segformer model performed slightly better than DeepLabV3Plus with a mIOU for leaves and spikes of ca.90%.However,the precision for stems with 54%was rather lower.The major advantages over published models are:ⅰ)the exclusion of weeds from the wheat canopy,ⅱ)the detection of all wheat features including necrotic and se-nescent tissues and its separation from crop residues.This facilitates further development in classifying healthy vs.unhealthy tissue to address the increasing need for accurate quantification of senescence and diseases in wheat canopies.
文摘Deep learning has shown potential in domains with large-scale annotated datasets.However,manual annotation is expensive,time-consuming,and tedious.Pixel-level annotations are particularly costly for semantic segmentation in images with dense irregular patterns of object instances,such as in plant images.In this work,we propose a method for developing high-performing deep learning models for semantic segmentation of such images utilizing little manual annotation.As a use case,we focus on wheat head segmentation.We synthesize a computationally annotated dataset—using a few annotated images,a short unannotated video clip of a wheat field,and several video clips with no wheat—to train a customized U-Net model.Considering the distribution shift between the synthesized and real images,we apply three domain adaptation steps to gradually bridge the domain gap.Only using two annotated images,we achieved a Dice score of 0.89 on the internal test set.When further evaluated on a diverse external dataset collected from 18 different domains across five countries,this model achieved a Dice score of 0.73.To expose the model to images from different growth stages and environmental conditions,we incorporated two annotated images from each of the 18 domains to further fine-tune the model.This increased the Dice score to 0.91.The result highlights the utility of the proposed approach in the absence of large-annotated datasets.Although our use case is wheat head segmentation,the proposed approach can be extended to other segmentation tasks with similar characteristics of irregularly repeating patterns of object instances.
