Unmanned aerial vehicle(UAV)photography has become the main power system inspection method;however,automated fault detection remains a major challenge.Conventional algorithms encounter difficulty in processing all the...Unmanned aerial vehicle(UAV)photography has become the main power system inspection method;however,automated fault detection remains a major challenge.Conventional algorithms encounter difficulty in processing all the detected objects in the power transmission lines simultaneously.The object detection method involving deep learning provides a new method for fault detection.However,the traditional non-maximum suppression(NMS)algorithm fails to delete redundant annotations when dealing with objects having two labels such as insulators and dampers.In this study,we propose an area-based non-maximum suppression(A-NMS)algorithm to solve the problem of one object having multiple labels.The A-NMS algorithm is used in the fusion stage of cropping detection to detect small objects.Experiments prove that A-NMS and cropping detection achieve a mean average precision and recall of 88.58%and 91.23%,respectively,in case of the aerial image datasets and realize multi-object fault detection in aerial images.展开更多
Referring expression comprehension(REC)aims to locate a specific region in an image described by a natural language.Existing two-stage methods generate multiple candidate proposals in the first stage,followed by selec...Referring expression comprehension(REC)aims to locate a specific region in an image described by a natural language.Existing two-stage methods generate multiple candidate proposals in the first stage,followed by selecting one of these proposals as the grounding result in the second stage.Nevertheless,the number of candidate proposals generated in the first stage significantly exceeds ground truth and the recall of critical objects is inadequate,thereby enormously limiting the overall network performance.To address the above issues,the authors propose an innovative method termed Separate Non-Maximum Suppression(Sep-NMS)for two-stage REC.Particularly,Sep-NMS models information from the two stages independently and collaboratively,ultimately achieving an overall improvement in comprehension and identification of the target objects.Specifically,the authors propose a Ref-Relatedness module for filtering referent proposals rigorously,decreasing the redundancy of referent proposals.A CLIP†Relatedness module based on robust multimodal pre-trained encoders is built to precisely assess the relevance between language and proposals to improve the recall of critical objects.It is worth mentioning that the authors are the pioneers in utilising a multimodal pre-training model for proposal filtering in the first stage.Moreover,an Information Fusion module is designed to effectively amalgamate the multimodal information across two stages,ensuring maximum uti-lisation of the available information.Extensive experiments demonstrate that the approach achieves competitive performance with previous state-of-the-art methods.The datasets used are publicly available:RefCOCO,RefCOCO+:https://doi.org/10.1007/978-3-319-46475-6_5 and RefCOCOg:https://doi.org/10.1109/CVPR.2016.9.展开更多
As the basic work of image stitching and object recognition,image registration played an important part in the image processing field.Much previous work in registration accuracy and realtime performance progressed ver...As the basic work of image stitching and object recognition,image registration played an important part in the image processing field.Much previous work in registration accuracy and realtime performance progressed very slowly,especially in registrating images with line feature.An innovative method for image registration based on lines is proposed,it can effectively improve the accuracy and real-time performance of image registration.The line feature can deal with some registration problems where point feature does not work.Our registration process is divided into two parts.The first part determines the rough registration transformation relation between reference image and test image.Then the similarity degree among different transformation and modified nonmaximum suppression(MNMS)algorithms are obtained,which produce local optimal solution to optimize the rough registration transformation.The final optimal registration relation can be obtained from two registration parts according to the match scores.The experimental results show that the proposed method makes a more accurate registration relation and performs better in real-time situation.展开更多
The increasing trend towards independent fruit packaging demands a high appearance quality of individually packed fruits.In this paper,we propose an improved YOLOv5-based model,YOLO-Banana,to effectively grade banana ...The increasing trend towards independent fruit packaging demands a high appearance quality of individually packed fruits.In this paper,we propose an improved YOLOv5-based model,YOLO-Banana,to effectively grade banana appearance quality based on the number of banana defect points.Due to the minor and dense defects on the surface of bananas,existing detection algorithms have poor detection results and high missing rates.To address this,we propose a densitybased spatial clustering of applications with noise(DBSCAN)and K-means fusion clustering method that utilizes refined anchor points to obtain better initial anchor values,thereby enhancing the network’s recognition accuracy.Moreover,the optimized progressive aggregated network(PANet)enables better multi-level feature fusion.Additionally,the non-maximum suppression function is replaced with a weighted non-maximum suppression(weighted NMS)function based on distance intersection over union(DIoU).Experimental results show that the model’s accuracy is improved by 2.3%compared to the original YOLOv5 network model,thereby effectively grading the banana appearance quality.展开更多
In order to avoid the problem of poor illumination characteristics and inaccurate positioning accuracy, this paper proposed a pedestrian detection algorithm suitable for low-light environments. The algorithm first app...In order to avoid the problem of poor illumination characteristics and inaccurate positioning accuracy, this paper proposed a pedestrian detection algorithm suitable for low-light environments. The algorithm first applied the multi-scale Retinex image enhancement algorithm to the sample pre-processing of deep learning to improve the image resolution. Then the paper used the faster regional convolutional neural network to train the pedestrian detection model, extracted the pedestrian characteristics, and obtained the bounding boxes through classification and position regression. Finally, the pedestrian detection process was carried out by introducing the Soft-NMS algorithm, and the redundant bounding box was eliminated to obtain the best pedestrian detection position. The experimental results showed that the proposed detection algorithm achieves an average accuracy of 89.74% on the low-light dataset, and the pedestrian detection effect was more significant.展开更多
In edge detection algorithms, there is a common redundancy problem, especially when the gradient direction is close to -135°, -45°, 45°, and 135°. Double edge effect appears on the edges around the...In edge detection algorithms, there is a common redundancy problem, especially when the gradient direction is close to -135°, -45°, 45°, and 135°. Double edge effect appears on the edges around these directions. This is caused by the discrete calculation of non-maximum suppression. Many algorithms use edge points as feature for further task such as line extraction, curve detection, matching and recognition. Redundancy is a very important factor of algorithm speed and accuracy. We find that most edge detection algorithms have redundancy of 50% in the worst case and 0% in the best case depending on the edge direction distribution. The common redundancy rate on natural images is approximately between 15% and 20%. Based on Canny’s framework, we propose a restriction in the hysteresis step. Our experiment shows that proposed restricted hysteresis reduce the redundancy successfully.展开更多
Balinese carvings are cultural objects that adorn sacred buildings. The carvings consist of several motifs,each representing the values adopted by the Balinese people. Detection of Balinese carving motifs ischallengin...Balinese carvings are cultural objects that adorn sacred buildings. The carvings consist of several motifs,each representing the values adopted by the Balinese people. Detection of Balinese carving motifs ischallenging due to the unavailability of a Balinese carving dataset for detection tasks, high variance,and tiny-size carving motifs. This research aims to improve carving motif detection performance onchallenging Balinese carving motifs detection task through a modification of YOLOv5 to support adigital carving conservation system. We proposed CARVING-DETC, a deep learning-based Balinesecarving detection method consisting of three steps. First, the data generation step performs dataaugmentation and annotation on Balinese carving images. Second, we proposed a network scalingstrategy on the YOLOv5 model and performed non-maximum suppression (NMS) on the modelensemble to generate the most optimal predictions. The ensemble model utilizes NMS to producehigher performance by optimizing the detection results based on the highest confidence score andsuppressing other overlap predictions with a lower confidence score. Third, performance evaluation onscaled-YOLOv5 versions and NMS ensemble models. The research findings are beneficial in conservingthe cultural heritage and as a reference for other researchers. In addition, this study proposed a novelBalinese carving dataset through data collection, augmentation, and annotation. To our knowledge,it is the first Balinese carving dataset for the object detection task. Based on experimental results,CARVING-DETC achieved a detection performance of 98%, which outperforms the baseline model.展开更多
基金the National Grid Corporation Headquarters Science and Technology Project:Key Technology Research,Equipment Development and Engineering Demonstration of Artificial Smart Drived Electric Vehicle Smart Travel Service(No.52020118000G).
文摘Unmanned aerial vehicle(UAV)photography has become the main power system inspection method;however,automated fault detection remains a major challenge.Conventional algorithms encounter difficulty in processing all the detected objects in the power transmission lines simultaneously.The object detection method involving deep learning provides a new method for fault detection.However,the traditional non-maximum suppression(NMS)algorithm fails to delete redundant annotations when dealing with objects having two labels such as insulators and dampers.In this study,we propose an area-based non-maximum suppression(A-NMS)algorithm to solve the problem of one object having multiple labels.The A-NMS algorithm is used in the fusion stage of cropping detection to detect small objects.Experiments prove that A-NMS and cropping detection achieve a mean average precision and recall of 88.58%and 91.23%,respectively,in case of the aerial image datasets and realize multi-object fault detection in aerial images.
基金funded by the National Natural Science Foundation of China(No.62076032).
文摘Referring expression comprehension(REC)aims to locate a specific region in an image described by a natural language.Existing two-stage methods generate multiple candidate proposals in the first stage,followed by selecting one of these proposals as the grounding result in the second stage.Nevertheless,the number of candidate proposals generated in the first stage significantly exceeds ground truth and the recall of critical objects is inadequate,thereby enormously limiting the overall network performance.To address the above issues,the authors propose an innovative method termed Separate Non-Maximum Suppression(Sep-NMS)for two-stage REC.Particularly,Sep-NMS models information from the two stages independently and collaboratively,ultimately achieving an overall improvement in comprehension and identification of the target objects.Specifically,the authors propose a Ref-Relatedness module for filtering referent proposals rigorously,decreasing the redundancy of referent proposals.A CLIP†Relatedness module based on robust multimodal pre-trained encoders is built to precisely assess the relevance between language and proposals to improve the recall of critical objects.It is worth mentioning that the authors are the pioneers in utilising a multimodal pre-training model for proposal filtering in the first stage.Moreover,an Information Fusion module is designed to effectively amalgamate the multimodal information across two stages,ensuring maximum uti-lisation of the available information.Extensive experiments demonstrate that the approach achieves competitive performance with previous state-of-the-art methods.The datasets used are publicly available:RefCOCO,RefCOCO+:https://doi.org/10.1007/978-3-319-46475-6_5 and RefCOCOg:https://doi.org/10.1109/CVPR.2016.9.
文摘As the basic work of image stitching and object recognition,image registration played an important part in the image processing field.Much previous work in registration accuracy and realtime performance progressed very slowly,especially in registrating images with line feature.An innovative method for image registration based on lines is proposed,it can effectively improve the accuracy and real-time performance of image registration.The line feature can deal with some registration problems where point feature does not work.Our registration process is divided into two parts.The first part determines the rough registration transformation relation between reference image and test image.Then the similarity degree among different transformation and modified nonmaximum suppression(MNMS)algorithms are obtained,which produce local optimal solution to optimize the rough registration transformation.The final optimal registration relation can be obtained from two registration parts according to the match scores.The experimental results show that the proposed method makes a more accurate registration relation and performs better in real-time situation.
基金supported by the Beijing Science Foundation(No.9232005)the Beijing Municipal Philosophy and Social Science Foundation of China(No.19GLB036)the Beijing Science and Technology Project(No.Z221100005822014)。
文摘The increasing trend towards independent fruit packaging demands a high appearance quality of individually packed fruits.In this paper,we propose an improved YOLOv5-based model,YOLO-Banana,to effectively grade banana appearance quality based on the number of banana defect points.Due to the minor and dense defects on the surface of bananas,existing detection algorithms have poor detection results and high missing rates.To address this,we propose a densitybased spatial clustering of applications with noise(DBSCAN)and K-means fusion clustering method that utilizes refined anchor points to obtain better initial anchor values,thereby enhancing the network’s recognition accuracy.Moreover,the optimized progressive aggregated network(PANet)enables better multi-level feature fusion.Additionally,the non-maximum suppression function is replaced with a weighted non-maximum suppression(weighted NMS)function based on distance intersection over union(DIoU).Experimental results show that the model’s accuracy is improved by 2.3%compared to the original YOLOv5 network model,thereby effectively grading the banana appearance quality.
文摘In order to avoid the problem of poor illumination characteristics and inaccurate positioning accuracy, this paper proposed a pedestrian detection algorithm suitable for low-light environments. The algorithm first applied the multi-scale Retinex image enhancement algorithm to the sample pre-processing of deep learning to improve the image resolution. Then the paper used the faster regional convolutional neural network to train the pedestrian detection model, extracted the pedestrian characteristics, and obtained the bounding boxes through classification and position regression. Finally, the pedestrian detection process was carried out by introducing the Soft-NMS algorithm, and the redundant bounding box was eliminated to obtain the best pedestrian detection position. The experimental results showed that the proposed detection algorithm achieves an average accuracy of 89.74% on the low-light dataset, and the pedestrian detection effect was more significant.
文摘In edge detection algorithms, there is a common redundancy problem, especially when the gradient direction is close to -135°, -45°, 45°, and 135°. Double edge effect appears on the edges around these directions. This is caused by the discrete calculation of non-maximum suppression. Many algorithms use edge points as feature for further task such as line extraction, curve detection, matching and recognition. Redundancy is a very important factor of algorithm speed and accuracy. We find that most edge detection algorithms have redundancy of 50% in the worst case and 0% in the best case depending on the edge direction distribution. The common redundancy rate on natural images is approximately between 15% and 20%. Based on Canny’s framework, we propose a restriction in the hysteresis step. Our experiment shows that proposed restricted hysteresis reduce the redundancy successfully.
基金the Directorate General of Higher Education,Research,and Technology,Republic of Indonesia under the grand number 3/E1/KP.PTNBH/2021.
文摘Balinese carvings are cultural objects that adorn sacred buildings. The carvings consist of several motifs,each representing the values adopted by the Balinese people. Detection of Balinese carving motifs ischallenging due to the unavailability of a Balinese carving dataset for detection tasks, high variance,and tiny-size carving motifs. This research aims to improve carving motif detection performance onchallenging Balinese carving motifs detection task through a modification of YOLOv5 to support adigital carving conservation system. We proposed CARVING-DETC, a deep learning-based Balinesecarving detection method consisting of three steps. First, the data generation step performs dataaugmentation and annotation on Balinese carving images. Second, we proposed a network scalingstrategy on the YOLOv5 model and performed non-maximum suppression (NMS) on the modelensemble to generate the most optimal predictions. The ensemble model utilizes NMS to producehigher performance by optimizing the detection results based on the highest confidence score andsuppressing other overlap predictions with a lower confidence score. Third, performance evaluation onscaled-YOLOv5 versions and NMS ensemble models. The research findings are beneficial in conservingthe cultural heritage and as a reference for other researchers. In addition, this study proposed a novelBalinese carving dataset through data collection, augmentation, and annotation. To our knowledge,it is the first Balinese carving dataset for the object detection task. Based on experimental results,CARVING-DETC achieved a detection performance of 98%, which outperforms the baseline model.
