Ocean internal waves appear as irregular bright and dark stripes on synthetic aperture radar(SAR)remote sensing images.Ocean internal waves detection in SAR images consequently constituted a difficult and popular rese...Ocean internal waves appear as irregular bright and dark stripes on synthetic aperture radar(SAR)remote sensing images.Ocean internal waves detection in SAR images consequently constituted a difficult and popular research topic.In this paper,ocean internal waves are detected in SAR images by employing the faster regions with convolutional neural network features(Faster R-CNN)framework;for this purpose,888 internal wave samples are utilized to train the convolutional network and identify internal waves.The experimental results demonstrate a 94.78%recognition rate for internal waves,and the average detection speed is 0.22 s/image.In addition,the detection results of internal wave samples under different conditions are analyzed.This paper lays a foundation for detecting ocean internal waves using convolutional neural networks.展开更多
This paper proposes a solution to localization and classification of rice grains in an image.All existing related works rely on conventional based machine learning approaches.However,those techniques do not do well fo...This paper proposes a solution to localization and classification of rice grains in an image.All existing related works rely on conventional based machine learning approaches.However,those techniques do not do well for the problem designed in this paper,due to the high similarities between different types of rice grains.The deep learning based solution is developed in the proposed solution.It contains pre-processing steps of data annotation using the watershed algorithm,auto-alignment using the major axis orientation,and image enhancement using the contrast-limited adaptive histogram equalization(CLAHE)technique.Then,the mask region-based convolutional neural networks(R-CNN)is trained to localize and classify rice grains in an input image.The performance is enhanced by using the transfer learning and the dropout regularization for overfitting prevention.The proposed method is validated using many scenarios of experiments,reported in the forms of mean average precision(mAP)and a confusion matrix.It achieves above 80%mAP for main scenarios in the experiments.It is also shown to perform outstanding,when compared to human experts.展开更多
Rod insulators are vital parts of the catenary of high speed railways(HSRs).There are many different catenary insulators,and the background of the insulator image is complicated.It is difficult to recognise insulators...Rod insulators are vital parts of the catenary of high speed railways(HSRs).There are many different catenary insulators,and the background of the insulator image is complicated.It is difficult to recognise insulators and detect defects automatically.In this paper,we propose a catenary intelligent defect detection algorithm based on Mask region-convolutional neural network(R-CNN)and an image processing model.Vertical projection technology is used to achieve single shed positioning and precise cutting of the insulator.Gradient,texture,and gray feature fusion(GTGFF)and a K-means clustering analysis model(KCAM)are proposed to detect broken insulators,dirt,foreign bodies,and flashover.Using this model,insulator recognition and defect detection can achieve a high recall rate and accuracy,and generalized defect detection.The algorithm is tested and verified on a dataset of realistic insulator images,and the accuracy and reliability of the algorithm satisfy current requirements for HSR catenary automatic inspection and intelligent maintenance.展开更多
Automatic road damage detection using image processing is an important aspect of road maintenance.It is also a challenging problem due to the inhomogeneity of road damage and complicated background in the road images....Automatic road damage detection using image processing is an important aspect of road maintenance.It is also a challenging problem due to the inhomogeneity of road damage and complicated background in the road images.In recent years,deep convolutional neural network based methods have been used to address the challenges of road damage detection and classification.In this paper,we propose a new approach to address those challenges.This approach uses densely connected convolution networks as the backbone of the Mask R-CNN to effectively extract image feature,a feature pyramid network for combining multiple scales features,a region proposal network to generate the road damage region,and a fully convolutional neural network to classify the road damage region and refine the region bounding box.This method can not only detect and classify the road damage,but also create a mask of the road damage.Experimental results show that the proposed approach can achieve better results compared with other existing methods.展开更多
In order to solve the problem of small objects detection in unmanned aerial vehicle(UAV)aerial images with complex background,a general detection method for multi-scale small objects based on Faster region-based convo...In order to solve the problem of small objects detection in unmanned aerial vehicle(UAV)aerial images with complex background,a general detection method for multi-scale small objects based on Faster region-based convolutional neural network(Faster R-CNN)is proposed.The bird’s nest on the high-voltage tower is taken as the research object.Firstly,we use the improved convolutional neural network ResNet101 to extract object features,and then use multi-scale sliding windows to obtain the object region proposals on the convolution feature maps with different resolutions.Finally,a deconvolution operation is added to further enhance the selected feature map with higher resolution,and then it taken as a feature mapping layer of the region proposals passing to the object detection sub-network.The detection results of the bird’s nest in UAV aerial images show that the proposed method can precisely detect small objects in aerial images.展开更多
In order to improve the accuracy of threaded hole object detection,combining a dual camera vision system with the Hough transform circle detection,we propose an object detection method of artifact threaded hole based ...In order to improve the accuracy of threaded hole object detection,combining a dual camera vision system with the Hough transform circle detection,we propose an object detection method of artifact threaded hole based on Faster region-ased convolutional neural network(Faster R-CNN).First,a dual camera image acquisition system is established.One industrial camera placed at a high position is responsible for collecting the whole image of the workpiece,and the suspected screw hole position on the workpiece can be preliminarily selected by Hough transform detection algorithm.Then,the other industrial camera is responsible for collecting the local images of the suspected screw holes that have been detected by Hough transform one by one.After that,ResNet50-based Faster R-CNN object detection model is trained on the self-built screw hole data set.Finally,the local image of the threaded hole is input into the trained Faster R-CNN object detection model for further identification and location.The experimental results show that the proposed method can effectively avoid small object detection of threaded holes,and compared with the method that only uses Hough transform or Faster RCNN object detection alone,it has high recognition and positioning accuracy.展开更多
An otoscope is traditionally used to examine the eardrum and ear canal.A diagnosis of otitis media(OM)relies on the experience of clinicians.If an examiner lacks experience,the examination may be difficult and time-co...An otoscope is traditionally used to examine the eardrum and ear canal.A diagnosis of otitis media(OM)relies on the experience of clinicians.If an examiner lacks experience,the examination may be difficult and time-consuming.This paper presents an ear disease classification method using middle ear images based on a convolutional neural network(CNN).Especially the segmentation and classification networks are used to classify an otoscopic image into six classes:normal,acute otitis media(AOM),otitis media with effusion(OME),chronic otitis media(COM),congenital cholesteatoma(CC)and traumatic perforations(TMPs).The Mask R-CNN is utilized for the segmentation network to extract the region of interest(ROI)from otoscopic images.The extracted ROIs are used as guiding features for the classification.The classification is based on transfer learning with an ensemble of two CNN classifiers:EfficientNetB0 and Inception-V3.The proposed model was trained with a 5-fold cross-validation technique.The proposed method was evaluated and achieved a classification accuracy of 97.29%.展开更多
Rock classification plays a crucial role in various fields such as geology,engineering,and environmental studies.Employing deep learning AI(artificial intelligence)methods has a high potential to significantly improve...Rock classification plays a crucial role in various fields such as geology,engineering,and environmental studies.Employing deep learning AI(artificial intelligence)methods has a high potential to significantly improve the accuracy and efficiency of this task.The paper delves into the exploration of two cuttingedge AI techniques,namely Mask DINO and Mask R-CNN(convolutional neural network),as means to identify rock weathering grades and rock types.The results demonstrate that Mask DINO,which is a Detection Transformer(DETR),outperforms Mask R-CNN for the aforementioned purposes.Mask DINO achieved f-1 scores of 91% and 86% in weathering grade detection and rock type detection,as opposed to the Mask R-CNN's f-1 scores of 84% and 75%,respectively.These findings underscore the substantial potential of employing DETR algorithms like Mask DINO for automatic classification of both rock type and weathering states.Although the study examines only two AI models,the data processing and other techniques developed in this study may serve as a foundation for future advancements in the field.By incorporating these advanced AI techniques,logging personnel can obtain valuable references to aid their work,ultimately contributing to the advancement of geological and related fields.展开更多
Most image-based object detection methods employ horizontal bounding boxes(HBBs)to capture objects in tunnel images.However,these bounding boxes often fail to effectively enclose objects oriented in arbitrary directio...Most image-based object detection methods employ horizontal bounding boxes(HBBs)to capture objects in tunnel images.However,these bounding boxes often fail to effectively enclose objects oriented in arbitrary directions,resulting in reduced accuracy and suboptimal detection performance.Moreover,HBBs cannot provide directional information for rotated objects.This study proposes a rotated detection method for identifying apparent defects in shield tunnels.Specifically,the oriented region-convolutional neural network(oriented R-CNN)is utilized to detect rotated objects in tunnel images.To enhance feature extraction,a novel hybrid backbone combining CNN-based networks with Swin Transformers is proposed.A feature fusion strategy is employed to integrate features extracted from both networks.Additionally,a neck network based on the bidirectional-feature pyramid network(Bi-FPN)is designed to combine multi-scale object features.The bolt hole dataset is curated to evaluate the efficacyof the proposed method.In addition,a dedicated pre-processing approach is developed for large-sized images to accommodate the rotated,dense,and small-scale characteristics of objects in tunnel images.Experimental results demonstrate that the proposed method achieves a more than 4%improvement in mAP_(50-95)compared to other rotated detectors and a 6.6%-12.7%improvement over mainstream horizontal detectors.Furthermore,the proposed method outperforms mainstream methods by 6.5%-14.7%in detecting leakage bolt holes,underscoring its significant engineering applicability.展开更多
This paper help with leguminous seeds detection and smart farming. There are hundreds of kinds of seeds and itcan be very difficult to distinguish between them. Botanists and those who study plants, however, can ident...This paper help with leguminous seeds detection and smart farming. There are hundreds of kinds of seeds and itcan be very difficult to distinguish between them. Botanists and those who study plants, however, can identifythe type of seed at a glance. As far as we know, this is the first work to consider leguminous seeds images withdifferent backgrounds and different sizes and crowding. Machine learning is used to automatically classify andlocate 11 different seed types. We chose Leguminous seeds from 11 types to be the objects of this study. Thosetypes are of different colors, sizes, and shapes to add variety and complexity to our research. The images datasetof the leguminous seeds was manually collected, annotated, and then split randomly into three sub-datasetstrain, validation, and test (predictions), with a ratio of 80%, 10%, and 10% respectively. The images consideredthe variability between different leguminous seed types. The images were captured on five different backgrounds: white A4 paper, black pad, dark blue pad, dark green pad, and green pad. Different heights and shootingangles were considered. The crowdedness of the seeds also varied randomly between 1 and 50 seeds per image.Different combinations and arrangements between the 11 types were considered. Two different image-capturingdevices were used: a SAMSUNG smartphone camera and a Canon digital camera. A total of 828 images wereobtained, including 9801 seed objects (labels). The dataset contained images of different backgrounds, heights,angles, crowdedness, arrangements, and combinations. The TensorFlow framework was used to construct theFaster Region-based Convolutional Neural Network (R-CNN) model and CSPDarknet53 is used as the backbonefor YOLOv4 based on DenseNet designed to connect layers in convolutional neural. Using the transfer learningmethod, we optimized the seed detection models. The currently dominant object detection methods, Faster RCNN, and YOLOv4 performances were compared experimentally. The mAP (mean average precision) of the FasterR-CNN and YOLOv4 models were 84.56% and 98.52% respectively. YOLOv4 had a significant advantage in detection speed over Faster R-CNN which makes it suitable for real-time identification as well where high accuracy andlow false positives are needed. The results showed that YOLOv4 had better accuracy, and detection ability, as wellas faster detection speed beating Faster R-CNN by a large margin. The model can be effectively applied under avariety of backgrounds, image sizes, seed sizes, shooting angles, and shooting heights, as well as different levelsof seed crowding. It constitutes an effective and efficient method for detecting different leguminous seeds incomplex scenarios. This study provides a reference for further seed testing and enumeration applications.展开更多
To pursue the ideal of a safe high-tech society in a time when traffic accidents are frequent,the traffic signs detection system has become one of the necessary topics in recent years and in the future.The ultimate go...To pursue the ideal of a safe high-tech society in a time when traffic accidents are frequent,the traffic signs detection system has become one of the necessary topics in recent years and in the future.The ultimate goal of this research is to identify and classify the types of traffic signs in a panoramic image.To accomplish this goal,the paper proposes a new model for traffic sign detection based on the Convolutional Neural Network for com-prehensive traffic sign classification and Mask Region-based Convolutional Neural Networks(R-CNN)implementation for identifying and extracting signs in panoramic images.Data augmentation and normalization of the images are also applied to assist in classifying better even if old traffic signs are degraded,and considerably minimize the rates of discovering the extra boxes.The proposed model is tested on both the testing dataset and the actual images and gets 94.5%of the correct signs recognition rate,the classification rate of those signs discovered was 99.41%and the rate of false signs was only around 0.11.展开更多
基金Supported by the National Natural Science Foundation of China(No.61471136)the Special Project for Global Change and Air-sea Interaction of Ministry of Natural Resources(No.GASI-02-SCS-YGST2-04)the Chinese Association of Ocean Mineral Resources R&D(No.DY135-E2-4)
文摘Ocean internal waves appear as irregular bright and dark stripes on synthetic aperture radar(SAR)remote sensing images.Ocean internal waves detection in SAR images consequently constituted a difficult and popular research topic.In this paper,ocean internal waves are detected in SAR images by employing the faster regions with convolutional neural network features(Faster R-CNN)framework;for this purpose,888 internal wave samples are utilized to train the convolutional network and identify internal waves.The experimental results demonstrate a 94.78%recognition rate for internal waves,and the average detection speed is 0.22 s/image.In addition,the detection results of internal wave samples under different conditions are analyzed.This paper lays a foundation for detecting ocean internal waves using convolutional neural networks.
文摘This paper proposes a solution to localization and classification of rice grains in an image.All existing related works rely on conventional based machine learning approaches.However,those techniques do not do well for the problem designed in this paper,due to the high similarities between different types of rice grains.The deep learning based solution is developed in the proposed solution.It contains pre-processing steps of data annotation using the watershed algorithm,auto-alignment using the major axis orientation,and image enhancement using the contrast-limited adaptive histogram equalization(CLAHE)technique.Then,the mask region-based convolutional neural networks(R-CNN)is trained to localize and classify rice grains in an input image.The performance is enhanced by using the transfer learning and the dropout regularization for overfitting prevention.The proposed method is validated using many scenarios of experiments,reported in the forms of mean average precision(mAP)and a confusion matrix.It achieves above 80%mAP for main scenarios in the experiments.It is also shown to perform outstanding,when compared to human experts.
基金supported by the National Natural Science Foundation of China(Nos.51677171,51637009,51577166 and 51827810)the National Key R&D Program of China(No.2018YFB0606000)+2 种基金the China Scholarship Council(No.201708330502)the Fund of Shuohuang Railway Development Limited Liability Company(No.SHTL-2020-13)the Fund of State Key Laboratory of Industrial Control Technology(No.ICT2022B29),China。
文摘Rod insulators are vital parts of the catenary of high speed railways(HSRs).There are many different catenary insulators,and the background of the insulator image is complicated.It is difficult to recognise insulators and detect defects automatically.In this paper,we propose a catenary intelligent defect detection algorithm based on Mask region-convolutional neural network(R-CNN)and an image processing model.Vertical projection technology is used to achieve single shed positioning and precise cutting of the insulator.Gradient,texture,and gray feature fusion(GTGFF)and a K-means clustering analysis model(KCAM)are proposed to detect broken insulators,dirt,foreign bodies,and flashover.Using this model,insulator recognition and defect detection can achieve a high recall rate and accuracy,and generalized defect detection.The algorithm is tested and verified on a dataset of realistic insulator images,and the accuracy and reliability of the algorithm satisfy current requirements for HSR catenary automatic inspection and intelligent maintenance.
基金supported by the School Doctoral Fund of Zhengzhou University of Light Industry No.2015BSJJ051.
文摘Automatic road damage detection using image processing is an important aspect of road maintenance.It is also a challenging problem due to the inhomogeneity of road damage and complicated background in the road images.In recent years,deep convolutional neural network based methods have been used to address the challenges of road damage detection and classification.In this paper,we propose a new approach to address those challenges.This approach uses densely connected convolution networks as the backbone of the Mask R-CNN to effectively extract image feature,a feature pyramid network for combining multiple scales features,a region proposal network to generate the road damage region,and a fully convolutional neural network to classify the road damage region and refine the region bounding box.This method can not only detect and classify the road damage,but also create a mask of the road damage.Experimental results show that the proposed approach can achieve better results compared with other existing methods.
基金National Defense Pre-research Fund Project(No.KMGY318002531)。
文摘In order to solve the problem of small objects detection in unmanned aerial vehicle(UAV)aerial images with complex background,a general detection method for multi-scale small objects based on Faster region-based convolutional neural network(Faster R-CNN)is proposed.The bird’s nest on the high-voltage tower is taken as the research object.Firstly,we use the improved convolutional neural network ResNet101 to extract object features,and then use multi-scale sliding windows to obtain the object region proposals on the convolution feature maps with different resolutions.Finally,a deconvolution operation is added to further enhance the selected feature map with higher resolution,and then it taken as a feature mapping layer of the region proposals passing to the object detection sub-network.The detection results of the bird’s nest in UAV aerial images show that the proposed method can precisely detect small objects in aerial images.
文摘In order to improve the accuracy of threaded hole object detection,combining a dual camera vision system with the Hough transform circle detection,we propose an object detection method of artifact threaded hole based on Faster region-ased convolutional neural network(Faster R-CNN).First,a dual camera image acquisition system is established.One industrial camera placed at a high position is responsible for collecting the whole image of the workpiece,and the suspected screw hole position on the workpiece can be preliminarily selected by Hough transform detection algorithm.Then,the other industrial camera is responsible for collecting the local images of the suspected screw holes that have been detected by Hough transform one by one.After that,ResNet50-based Faster R-CNN object detection model is trained on the self-built screw hole data set.Finally,the local image of the threaded hole is input into the trained Faster R-CNN object detection model for further identification and location.The experimental results show that the proposed method can effectively avoid small object detection of threaded holes,and compared with the method that only uses Hough transform or Faster RCNN object detection alone,it has high recognition and positioning accuracy.
基金This study was supported by a Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Science,ICT&Future Planning NRF-2020R1A2C1014829the Soonchunhyang University Research Fund.
文摘An otoscope is traditionally used to examine the eardrum and ear canal.A diagnosis of otitis media(OM)relies on the experience of clinicians.If an examiner lacks experience,the examination may be difficult and time-consuming.This paper presents an ear disease classification method using middle ear images based on a convolutional neural network(CNN).Especially the segmentation and classification networks are used to classify an otoscopic image into six classes:normal,acute otitis media(AOM),otitis media with effusion(OME),chronic otitis media(COM),congenital cholesteatoma(CC)and traumatic perforations(TMPs).The Mask R-CNN is utilized for the segmentation network to extract the region of interest(ROI)from otoscopic images.The extracted ROIs are used as guiding features for the classification.The classification is based on transfer learning with an ensemble of two CNN classifiers:EfficientNetB0 and Inception-V3.The proposed model was trained with a 5-fold cross-validation technique.The proposed method was evaluated and achieved a classification accuracy of 97.29%.
基金supported by the Construction Industry Council(Grant No.CICR/01/22)the support from the General Research Fund(Grant No.17206822)of the Research Grants Council(Hong Kong).
文摘Rock classification plays a crucial role in various fields such as geology,engineering,and environmental studies.Employing deep learning AI(artificial intelligence)methods has a high potential to significantly improve the accuracy and efficiency of this task.The paper delves into the exploration of two cuttingedge AI techniques,namely Mask DINO and Mask R-CNN(convolutional neural network),as means to identify rock weathering grades and rock types.The results demonstrate that Mask DINO,which is a Detection Transformer(DETR),outperforms Mask R-CNN for the aforementioned purposes.Mask DINO achieved f-1 scores of 91% and 86% in weathering grade detection and rock type detection,as opposed to the Mask R-CNN's f-1 scores of 84% and 75%,respectively.These findings underscore the substantial potential of employing DETR algorithms like Mask DINO for automatic classification of both rock type and weathering states.Although the study examines only two AI models,the data processing and other techniques developed in this study may serve as a foundation for future advancements in the field.By incorporating these advanced AI techniques,logging personnel can obtain valuable references to aid their work,ultimately contributing to the advancement of geological and related fields.
基金support from the National Natural Science Foundation of China(Grant Nos.52025084 and 52408420)the Beijing Natural Science Foundation(Grant No.8244058).
文摘Most image-based object detection methods employ horizontal bounding boxes(HBBs)to capture objects in tunnel images.However,these bounding boxes often fail to effectively enclose objects oriented in arbitrary directions,resulting in reduced accuracy and suboptimal detection performance.Moreover,HBBs cannot provide directional information for rotated objects.This study proposes a rotated detection method for identifying apparent defects in shield tunnels.Specifically,the oriented region-convolutional neural network(oriented R-CNN)is utilized to detect rotated objects in tunnel images.To enhance feature extraction,a novel hybrid backbone combining CNN-based networks with Swin Transformers is proposed.A feature fusion strategy is employed to integrate features extracted from both networks.Additionally,a neck network based on the bidirectional-feature pyramid network(Bi-FPN)is designed to combine multi-scale object features.The bolt hole dataset is curated to evaluate the efficacyof the proposed method.In addition,a dedicated pre-processing approach is developed for large-sized images to accommodate the rotated,dense,and small-scale characteristics of objects in tunnel images.Experimental results demonstrate that the proposed method achieves a more than 4%improvement in mAP_(50-95)compared to other rotated detectors and a 6.6%-12.7%improvement over mainstream horizontal detectors.Furthermore,the proposed method outperforms mainstream methods by 6.5%-14.7%in detecting leakage bolt holes,underscoring its significant engineering applicability.
文摘This paper help with leguminous seeds detection and smart farming. There are hundreds of kinds of seeds and itcan be very difficult to distinguish between them. Botanists and those who study plants, however, can identifythe type of seed at a glance. As far as we know, this is the first work to consider leguminous seeds images withdifferent backgrounds and different sizes and crowding. Machine learning is used to automatically classify andlocate 11 different seed types. We chose Leguminous seeds from 11 types to be the objects of this study. Thosetypes are of different colors, sizes, and shapes to add variety and complexity to our research. The images datasetof the leguminous seeds was manually collected, annotated, and then split randomly into three sub-datasetstrain, validation, and test (predictions), with a ratio of 80%, 10%, and 10% respectively. The images consideredthe variability between different leguminous seed types. The images were captured on five different backgrounds: white A4 paper, black pad, dark blue pad, dark green pad, and green pad. Different heights and shootingangles were considered. The crowdedness of the seeds also varied randomly between 1 and 50 seeds per image.Different combinations and arrangements between the 11 types were considered. Two different image-capturingdevices were used: a SAMSUNG smartphone camera and a Canon digital camera. A total of 828 images wereobtained, including 9801 seed objects (labels). The dataset contained images of different backgrounds, heights,angles, crowdedness, arrangements, and combinations. The TensorFlow framework was used to construct theFaster Region-based Convolutional Neural Network (R-CNN) model and CSPDarknet53 is used as the backbonefor YOLOv4 based on DenseNet designed to connect layers in convolutional neural. Using the transfer learningmethod, we optimized the seed detection models. The currently dominant object detection methods, Faster RCNN, and YOLOv4 performances were compared experimentally. The mAP (mean average precision) of the FasterR-CNN and YOLOv4 models were 84.56% and 98.52% respectively. YOLOv4 had a significant advantage in detection speed over Faster R-CNN which makes it suitable for real-time identification as well where high accuracy andlow false positives are needed. The results showed that YOLOv4 had better accuracy, and detection ability, as wellas faster detection speed beating Faster R-CNN by a large margin. The model can be effectively applied under avariety of backgrounds, image sizes, seed sizes, shooting angles, and shooting heights, as well as different levelsof seed crowding. It constitutes an effective and efficient method for detecting different leguminous seeds incomplex scenarios. This study provides a reference for further seed testing and enumeration applications.
文摘To pursue the ideal of a safe high-tech society in a time when traffic accidents are frequent,the traffic signs detection system has become one of the necessary topics in recent years and in the future.The ultimate goal of this research is to identify and classify the types of traffic signs in a panoramic image.To accomplish this goal,the paper proposes a new model for traffic sign detection based on the Convolutional Neural Network for com-prehensive traffic sign classification and Mask Region-based Convolutional Neural Networks(R-CNN)implementation for identifying and extracting signs in panoramic images.Data augmentation and normalization of the images are also applied to assist in classifying better even if old traffic signs are degraded,and considerably minimize the rates of discovering the extra boxes.The proposed model is tested on both the testing dataset and the actual images and gets 94.5%of the correct signs recognition rate,the classification rate of those signs discovered was 99.41%and the rate of false signs was only around 0.11.
