This study investigates the application of Learnable Memory Vision Transformers(LMViT)for detecting metal surface flaws,comparing their performance with traditional CNNs,specifically ResNet18 and ResNet50,as well as o...This study investigates the application of Learnable Memory Vision Transformers(LMViT)for detecting metal surface flaws,comparing their performance with traditional CNNs,specifically ResNet18 and ResNet50,as well as other transformer-based models including Token to Token ViT,ViT withoutmemory,and Parallel ViT.Leveraging awidely-used steel surface defect dataset,the research applies data augmentation and t-distributed stochastic neighbor embedding(t-SNE)to enhance feature extraction and understanding.These techniques mitigated overfitting,stabilized training,and improved generalization capabilities.The LMViT model achieved a test accuracy of 97.22%,significantly outperforming ResNet18(88.89%)and ResNet50(88.90%),aswell as the Token to TokenViT(88.46%),ViT without memory(87.18),and Parallel ViT(91.03%).Furthermore,LMViT exhibited superior training and validation performance,attaining a validation accuracy of 98.2%compared to 91.0%for ResNet 18,96.0%for ResNet50,and 89.12%,87.51%,and 91.21%for Token to Token ViT,ViT without memory,and Parallel ViT,respectively.The findings highlight the LMViT’s ability to capture long-range dependencies in images,an areawhere CNNs struggle due to their reliance on local receptive fields and hierarchical feature extraction.The additional transformer-based models also demonstrate improved performance in capturing complex features over CNNs,with LMViT excelling particularly at detecting subtle and complex defects,which is critical for maintaining product quality and operational efficiency in industrial applications.For instance,the LMViT model successfully identified fine scratches and minor surface irregularities that CNNs often misclassify.This study not only demonstrates LMViT’s potential for real-world defect detection but also underscores the promise of other transformer-based architectures like Token to Token ViT,ViT without memory,and Parallel ViT in industrial scenarios where complex spatial relationships are key.Future research may focus on enhancing LMViT’s computational efficiency for deployment in real-time quality control systems.展开更多
Efficient detection of surface defects is primary for ensuring product quality during manufacturing processes.To enhance the performance of deep learning-based methods in practical applications,the authors propose Den...Efficient detection of surface defects is primary for ensuring product quality during manufacturing processes.To enhance the performance of deep learning-based methods in practical applications,the authors propose Dense-YOLO,a fast surface defect detection network that combines the strengths of DenseNet and you only look once version 3(YOLOv3).The authors design a lightweight backbone network with improved densely connected blocks,optimising the utilisation of shallow features while maintaining high detection speeds.Additionally,the authors refine the feature pyramid network of YOLOv3 to increase the recall of tiny defects and overall positioning accuracy.Furthermore,an online multi-angle template matching technique is introduced based on normalised cross-correlation to precisely locate the detection area.This refined template matching method not only accelerates detection speed but also mitigates the influence of the background.To validate the effectiveness of our enhancements,the authors conduct comparative experiments across two private datasets and one public dataset.Results show that Dense-YOLO outperforms existing methods,such as faster R-CNN,YOLOv3,YOLOv5s,YOLOv7,and SSD,in terms of mean average precision(mAP)and detection speed.Moreover,Dense-YOLO outperforms networks inherited from VGG and ResNet,including improved faster R-CNN,FCOS,M2Det-320 and FRCN,in mAP.展开更多
Detecting surface defects on unused rails is crucial for evaluating rail quality and durability to ensure the safety of rail transportation.However,existing detection methods often struggle with challenges such as com...Detecting surface defects on unused rails is crucial for evaluating rail quality and durability to ensure the safety of rail transportation.However,existing detection methods often struggle with challenges such as complex defect morphology,texture similarity,and fuzzy edges,leading to poor accuracy and missed detections.In order to resolve these problems,we propose MSCM-Net(Multi-Scale Cross-Modal Network),a multiscale cross-modal framework focused on detecting rail surface defects.MSCM-Net introduces an attention mechanism to dynamically weight the fusion of RGB and depth maps,effectively capturing and enhancing features at different scales for each modality.To further enrich feature representation and improve edge detection in blurred areas,we propose a multi-scale void fusion module that integrates multi-scale feature information.To improve cross-modal feature fusion,we develop a cross-enhanced fusion module that transfers fused features between layers to incorporate interlayer information.We also introduce a multimodal feature integration module,which merges modality-specific features from separate decoders into a shared decoder,enhancing detection by leveraging richer complementary information.Finally,we validate MSCM-Net on the NEU RSDDS-AUG RGB-depth dataset,comparing it against 12 leading methods,and the results show that MSCM-Net achieves superior performance on all metrics.展开更多
The service life of internal combustion engines is significantly influenced by surface defects in cylinder liners.To address the limitations of traditional detection methods,we propose an enhanced YOLOv8 model with Sw...The service life of internal combustion engines is significantly influenced by surface defects in cylinder liners.To address the limitations of traditional detection methods,we propose an enhanced YOLOv8 model with Swin Transformer as the backbone network.This approach leverages Swin Transformer's multi-head self-attention mechanism for improved feature extraction of defects spanning various scales.Integrated with the YOLOv8 detection head,our model achieves a mean average precision of 85.1%on our dataset,outperforming baseline methods by 1.4%.The model's effectiveness is further demonstrated on a steel-surface defect dataset,indicating its broad applicability in industrial surface defect detection.Our work highlights the potential of combining Swin Transformer and YOLOv8 for accurate and efficient defect detection.展开更多
In response to themissed and false detections that are easily caused by the large variety of and significant differences among aluminum surface defects,a detection algorithm based on an improved You Only Look Once(YOL...In response to themissed and false detections that are easily caused by the large variety of and significant differences among aluminum surface defects,a detection algorithm based on an improved You Only Look Once(YOLO)v8n network is proposed.First,a C2f_DWR_DRB module is constructed by introducing a dilation-wise residual(DWR)module and a dilated reparameterization block(DRB)to replace the C2f module at the high level of the backbone network,enriching the gradient flow information and increasing the effective receptive field(ERF).Second,an efficient local attention(ELA)mechanism is fused with the high-level screening-feature pyramid networks(HS-FPN)module,and an ELA_HSFPN is designed to replace the original feature fusion module,enhancing the ability of the network to cope with multiscale detection tasks.Moreover,a lightweight shared convolutional detection head(SCDH)is introduced to reduce the number of parameters and the computational complexity of the module while enhancing the performance and generalizability of the model.Finally,the soft intersection over union(SIoU)replaces the original loss function to improve the convergence speed and prediction accuracy of the model.Experimental results show that compared with that of the original YOLOv8n model,the mAP@0.5 of the improved algorithm is increased by 5.1%,the number of parameters and computational complexity are reduced by 33.3%and 32.1%,respectively,and the FPS is increased by 4.9%.Compared with other mainstream object detection algorithms,the improved algorithm still leads in terms of core indicators and has good generalizability for surface defects encountered in other industrial scenarios.展开更多
A novel dual-branch decoding fusion convolutional neural network model(DDFNet)specifically designed for real-time salient object detection(SOD)on steel surfaces is proposed.DDFNet is based on a standard encoder–decod...A novel dual-branch decoding fusion convolutional neural network model(DDFNet)specifically designed for real-time salient object detection(SOD)on steel surfaces is proposed.DDFNet is based on a standard encoder–decoder architecture.DDFNet integrates three key innovations:first,we introduce a novel,lightweight multi-scale progressive aggregation residual network that effectively suppresses background interference and refines defect details,enabling efficient salient feature extraction.Then,we propose an innovative dual-branch decoding fusion structure,comprising the refined defect representation branch and the enhanced defect representation branch,which enhance accuracy in defect region identification and feature representation.Additionally,to further improve the detection of small and complex defects,we incorporate a multi-scale attention fusion module.Experimental results on the public ESDIs-SOD dataset show that DDFNet,with only 3.69 million parameters,achieves detection performance comparable to current state-of-the-art models,demonstrating its potential for real-time industrial applications.Furthermore,our DDFNet-L variant consistently outperforms leading methods in detection performance.The code is available at https://github.com/13140W/DDFNet.展开更多
The detection of surface defects in concrete bridges using deep learning is of significant importance for reducing operational risks,saving maintenance costs,and driving the intelligent transformation of bridge defect...The detection of surface defects in concrete bridges using deep learning is of significant importance for reducing operational risks,saving maintenance costs,and driving the intelligent transformation of bridge defect detection.In contrast to the subjective and inefficient manual visual inspection,deep learning-based algorithms for concrete defect detection exhibit remarkable advantages,emerging as a focal point in recent research.This paper comprehensively analyzes the research progress of deep learning algorithms in the field of surface defect detection in concrete bridges in recent years.It introduces the early detection methods for surface defects in concrete bridges and the development of deep learning.Subsequently,it provides an overview of deep learning-based concrete bridge surface defect detection research from three aspects:image classification,object detection,and semantic segmentation.The paper summarizes the strengths and weaknesses of existing methods and the challenges they face.Additionally,it analyzes and prospects the development trends of surface defect detection in concrete bridges.展开更多
Aiming at the problems of low detection efficiency and difficult positioning of traditional steel surface defect detection methods,a lightweight steel surface defect detection model based on you only look once version...Aiming at the problems of low detection efficiency and difficult positioning of traditional steel surface defect detection methods,a lightweight steel surface defect detection model based on you only look once version 7(YOLOv7)is proposed.First,a cascading style sheets(CSS)block module is proposed,which uses more lightweight operations to obtain redundant information in the feature map,reduces the amount of computation,and effectively improves the detection speed.Secondly,the improved spatial pyramid pooling with cross stage partial convolutions(SPPCSPC)structure is adopted to ensure that the model can also pay attention to the defect location information while predicting the defect category information,obtain richer defect features.In addition,the convolution operation in the original model is simplified,which significantly reduces the size of the model and helps to improve the detection speed.Finally,using efficient intersection over union(EIOU)loss to focus on high-quality anchors,speed up convergence and improve positioning accuracy.Experiments were carried out on the Northeastern University-defect(NEU-DET)steel surface defect dataset.Compared with the original YOLOv7 model,the number of parameters of this model was reduced by 40%,the frames per second(FPS)reached 112,and the average accuracy reached 79.1%,the detection accuracy and speed have been improved,which can meet the needs of steel surface defect detection.展开更多
Surface defects can affect the quality of steel plate.Many methods based on computer vision are currently applied to surface defect detection of steel plate.However,their real-time performance and object detection of ...Surface defects can affect the quality of steel plate.Many methods based on computer vision are currently applied to surface defect detection of steel plate.However,their real-time performance and object detection of small defect are still unsatisfactory.An improved object detection network based on You Only Look One-level Feature(YOLOF)is proposed to show excellent performance in surface defect detection of steel plate,called DLF-YOLOF.First,the anchor-free detector is used to reduce the network hyperparameters.Secondly,deformable convolution network and local spatial attention module are introduced into the feature extraction network to increase the contextual information in the feature maps.Also,the soft non-maximum suppression is used to improve detection accuracy significantly.Finally,data augmentation is performed for small defect objects during training to improve detection accuracy.Experiments show the average precision and average precision for small objects are 42.7%and 33.5%at a detection speed of 62 frames per second on a single GPU,respectively.This shows that DLF-YOLOF has excellent performance to meet the needs of industrial real-time detection.展开更多
Ensuring high product quality is of paramount importance in pharmaceutical drug manufacturing,as it is subject to rigorous regulatory practices.This study presents a research focused on the development of an on-line d...Ensuring high product quality is of paramount importance in pharmaceutical drug manufacturing,as it is subject to rigorous regulatory practices.This study presents a research focused on the development of an on-line detection method and system for identifying surface defects in pharmaceutical products packaged in aluminum-plastic blisters.Firstly,the aluminum-plastic blister packages exhibit multi-scale features and inter-class indistinction.To address this,the deep semantic network with boundary refinement(DSN-BR)model is proposed,which leverages semantic segmentation domain knowledge,to accurately segment the defects in pixel level.Additionally,a specialized image acquisition module that minimizes the impact of ambient light is established,ensuring high-quality image capture.Finally,the image acquisition module,image detection module,and data management module are designed to construct a comprehensive online surface defect detection system.To validate the effectiveness of our approach,we employ a real dataset for instance verification on the implemented system.The experimental results substantiate the outstanding performance of the DSN-BR,achieving the mean intersection over union(MIoU)of 90.5%.Furthermore,the proposed system achieves an inference speed of up to 14.12 f/s,while attaining an F1-Score of 98.25%.These results demonstrate that the system meets the actual needs of the enterprise and provides theoretical and methodological support for intelligent inspection of product surface quality.By standardizing the control process of pharmaceutical manufacturing and improving the management capability of the manufacturing process,our approach holds significant market application prospects.展开更多
Segment Anything Model(SAM)is a cutting-edge model that has shown impressive performance in general object segmentation.The birth of the segment anything is a groundbreaking step towards creating a universal intellige...Segment Anything Model(SAM)is a cutting-edge model that has shown impressive performance in general object segmentation.The birth of the segment anything is a groundbreaking step towards creating a universal intelligent model.Due to its superior performance in general object segmentation,it quickly gained attention and interest.This makes SAM particularly attractive in industrial surface defect segmentation,especially for complex industrial scenes with limited training data.However,its segmentation ability for specific industrial scenes remains unknown.Therefore,in this work,we select three representative and complex industrial surface defect detection scenarios,namely strip steel surface defects,tile surface defects,and rail surface defects,to evaluate the segmentation performance of SAM.Our results show that although SAM has great potential in general object segmentation,it cannot achieve satisfactory performance in complex industrial scenes.Our test results are available at:https://github.com/VDT-2048/SAM-IS.展开更多
Printed Circuit Board(PCB)surface tiny defect detection is a difficult task in the integrated circuit industry,especially since the detection of tiny defects on PCB boards with large-size complex circuits has become o...Printed Circuit Board(PCB)surface tiny defect detection is a difficult task in the integrated circuit industry,especially since the detection of tiny defects on PCB boards with large-size complex circuits has become one of the bottlenecks.To improve the performance of PCB surface tiny defects detection,a PCB tiny defects detection model based on an improved attention residual network(YOLOX-AttResNet)is proposed.First,the unsupervised clustering performance of the K-means algorithm is exploited to optimize the channel weights for subsequent operations by feeding the feature mapping into the SENet(Squeeze and Excitation Network)attention network;then the improved K-means-SENet network is fused with the directly mapped edges of the traditional ResNet network to form an augmented residual network(AttResNet);and finally,the AttResNet module is substituted for the traditional ResNet structure in the backbone feature extraction network of mainstream excellent detection models,thus improving the ability to extract small features from the backbone of the target detection network.The results of ablation experiments on a PCB surface defect dataset show that AttResNet is a reliable and efficient module.In Torify the performance of AttResNet for detecting small defects in large-size complex circuit images,a series of comparison experiments are further performed.The results show that the AttResNet module combines well with the five best existing target detection frameworks(YOLOv3,YOLOX,Faster R-CNN,TDD-Net,Cascade R-CNN),and all the combined new models have improved detection accuracy compared to the original model,which suggests that the AttResNet module proposed in this paper can help the detection model to extract target features.Among them,the YOLOX-AttResNet model proposed in this paper performs the best,with the highest accuracy of 98.45% and the detection speed of 36 FPS(Frames Per Second),which meets the accuracy and real-time requirements for the detection of tiny defects on PCB surfaces.This study can provide some new ideas for other real-time online detection tasks of tiny targets with high-resolution images.展开更多
The accumulation of defects on wind turbine blade surfaces can lead to irreversible damage,impacting the aero-dynamic performance of the blades.To address the challenge of detecting and quantifying surface defects on ...The accumulation of defects on wind turbine blade surfaces can lead to irreversible damage,impacting the aero-dynamic performance of the blades.To address the challenge of detecting and quantifying surface defects on wind turbine blades,a blade surface defect detection and quantification method based on an improved Deeplabv3+deep learning model is proposed.Firstly,an improved method for wind turbine blade surface defect detection,utilizing Mobilenetv2 as the backbone feature extraction network,is proposed based on an original Deeplabv3+deep learning model to address the issue of limited robustness.Secondly,through integrating the concept of pre-trained weights from transfer learning and implementing a freeze training strategy,significant improvements have been made to enhance both the training speed and model training accuracy of this deep learning model.Finally,based on segmented blade surface defect images,a method for quantifying blade defects is proposed.This method combines image stitching algorithms to achieve overall quantification and risk assessment of the entire blade.Test results show that the improved Deeplabv3+deep learning model reduces training time by approximately 43.03%compared to the original model,while achieving mAP and MIoU values of 96.87%and 96.93%,respectively.Moreover,it demonstrates robustness in detecting different surface defects on blades across different back-grounds.The application of a blade surface defect quantification method enables the precise quantification of dif-ferent defects and facilitates the assessment of risk levels associated with defect measurements across the entire blade.This method enables non-contact,long-distance,high-precision detection and quantification of surface defects on the blades,providing a reference for assessing surface defects on wind turbine blades.展开更多
Precision steel balls are critical components in precision bearings.Surface defects on the steel balls will significantly reduce their useful life and cause linear or rotational transmission errors.Human visual inspec...Precision steel balls are critical components in precision bearings.Surface defects on the steel balls will significantly reduce their useful life and cause linear or rotational transmission errors.Human visual inspection of precision steel balls demands significant labor work.Besides,human inspection cannot maintain consistent quality assurance.To address these limitations and reduce inspection time,a convolutional neural network(CNN)based optical inspection system has been developed that automatically detects steel ball defects using a novel designated vertical mechanism.During image detection processing,two key challenges were addressed and resolved.They are the reflection caused by the coaxial light onto the ball center and the image deformation appearing at the edge of the steel balls.The special vertical rotating mechanism utilizing a spinning rod along with a spiral track was developed to enable successful and reliable full steel ball surface inspection during the rod rotation.The combination of the spinning rod and the spiral rotating component effectively rotates the steel ball to facilitate capturing complete surface images.Geometric calculations demonstrate that the steel balls can be completely inspected through specific rotation degrees,with the surface fully captured in 12 photo shots.These images are then analyzed by a CNN to determine surface quality defects.This study presents a new inspection method that enables the entire examination of steel ball surfaces.The successful development of this innovative automated optical inspection system with CNN represents a significant advancement in inspection quality control for precision steel balls.展开更多
Real-time detection of surface defects on cables is crucial for ensuring the safe operation of power systems.However,existing methods struggle with small target sizes,complex backgrounds,low-quality image acquisition,...Real-time detection of surface defects on cables is crucial for ensuring the safe operation of power systems.However,existing methods struggle with small target sizes,complex backgrounds,low-quality image acquisition,and interference from contamination.To address these challenges,this paper proposes the Real-time Cable Defect Detection Network(RC2DNet),which achieves an optimal balance between detection accuracy and computational efficiency.Unlike conventional approaches,RC2DNet introduces a small object feature extraction module that enhances the semantic representation of small targets through feature pyramids,multi-level feature fusion,and an adaptive weighting mechanism.Additionally,a boundary feature enhancement module is designed,incorporating boundary-aware convolution,a novel boundary attention mechanism,and an improved loss function to significantly enhance boundary localization accuracy.Experimental results demonstrate that RC2DNet outperforms state-of-the-art methods in precision,recall,F1-score,mean Intersection over Union(mIoU),and frame rate,enabling real-time and highly accurate cable defect detection in complex backgrounds.展开更多
A more effective and accurate improved Sobel algorithm has been developed to detect surface defects on heavy rails. The proposed method can make up for the mere sensitivity to X and Y directions of the Sobel algorithm...A more effective and accurate improved Sobel algorithm has been developed to detect surface defects on heavy rails. The proposed method can make up for the mere sensitivity to X and Y directions of the Sobel algorithm by adding six templates at different directions. Meanwhile, an experimental platform for detecting surface defects consisting of the bed-jig, image-forming system with CCD cameras and light sources, parallel computer system and cable system has been constructed. The detection results of the backfin defects show that the improved Sobel algorithm can achieve an accurate and efficient positioning with decreasing interference noises to the defect edge. It can also extract more precise features and characteristic parameters of the backfin defect. Furthermore, the BP neural network adopted for defects classification with the inputting characteristic parameters of improved Sobel algorithm can obtain the optimal training precision of 0.0095827 with 106 iterative steps and time of 3 s less than Sobel algorithm with 146 steps and 5 s. Finally, an enhanced identification rate of 10% for the defects is also confirmed after the Sobel algorithm is improved.展开更多
Feature extraction is essential to the classification of surface defect images. The defects of hot-rolled steels distribute in different directions. Therefore, the methods of multi-scale geometric analysis （MGA） wer...Feature extraction is essential to the classification of surface defect images. The defects of hot-rolled steels distribute in different directions. Therefore, the methods of multi-scale geometric analysis （MGA） were employed to decompose the image into several directional subba^ds at several scales. Then, the statistical features of each subband were calculated to produce a high-dimensional feature vector, which was reduced to a lower-dimensional vector by graph embedding algorithms. Finally, support vector machine （SVM） was used for defect classification. The multi-scale feature extraction method was implemented via curvelet transform and kernel locality preserving projections （KLPP）. Experiment results show that the proposed method is effective for classifying the surface defects of hot-rolled steels and the total classification rate is up to 97.33%.展开更多
Surface defects of the cold-rolled sheets of Ti-IF steel were studied and analyzed. After analyzing surface defects of cold-rolled sheets, such as shelling defects, holes and sliver defects by SEM/EDS, a variety of in...Surface defects of the cold-rolled sheets of Ti-IF steel were studied and analyzed. After analyzing surface defects of cold-rolled sheets, such as shelling defects, holes and sliver defects by SEM/EDS, a variety of inclusions were found. In addition, the distribution of macro-inclusions in slabs was analyzed by MIDAS method. The results show the macroscopic inclusion bands of head slabs and normal slabs are in 1/8 slab thickness regions of both inner arc side and outer arc side. The formation process of the defects in the cold-rolled sheets was simulated with an experimental cold-rolling machine for comparison. The results show that there were three kinds of inclusions underneath the surface defects: Al2O3, SiO2 and particles from slag entrainment, which were the main reason for defect formation during cold rolling.展开更多
Inefficient charge separation and limited light absorption are two critical issues associated with high‐efficiency photocatalytic H2production using TiO2.Surface defects within a certain concentration range in photoc...Inefficient charge separation and limited light absorption are two critical issues associated with high‐efficiency photocatalytic H2production using TiO2.Surface defects within a certain concentration range in photocatalyst materials are beneficial for photocatalytic activity.In this study,surface defects(oxygen vacancies and metal cation replacement defects)were induced with a facile and effective approach by surface doping with low‐cost transition metals(Co,Ni,Cu,and Mn)on ultrafine TiO2.The obtained surface‐defective TiO2exhibited a3–4‐fold improved activity compared to that of the original ultrafine TiO2.In addition,a H2production rate of3.4μmol/h was obtained using visible light(λ>420nm)irradiation.The apparent quantum yield(AQY)at365nm reached36.9%over TiO2‐Cu,significantly more than the commercial P25TiO2.The enhancement of photocatalytic H2production activity can be attributed to improved rapid charge separation efficiency andexpanded light absorption window.This hydrothermal treatment with transition metal was proven to be a very facile and effective method for obtaining surface defects.展开更多
The formations of defective MgC12 surfaces, and subsequent adsorption of Ti species and electron donor, as well as propylene polymerization over the Ziegler-Natta catalyst have been investigated using density function...The formations of defective MgC12 surfaces, and subsequent adsorption of Ti species and electron donor, as well as propylene polymerization over the Ziegler-Natta catalyst have been investigated using density functional theory （DFT） method. Twelve possible support models of regular and defective MgC12 （110） and （100） surfaces were built. The individual adsorptions of titanium chlorides as mononuclear or dinuclear, and ethyl benzoate （EB） as electron donor, on these models were evaluated. The analysis of energies presented the cases of EB adsorption were generally more stable than titanium chlorides on both surfaces. Thus, EB as internal electron donor mainly prevented TIC14 from coordinating on the MgC12 surfaces where mostly non-stereospecific active sites could be formed. Exceptionally, A5 the site model with terminal Cl-vacancy on the MgC12 support, presented stronger adsorption of TiCl4 than that of EB on （110） surface. Since the TIC14 and ethyl benzoate （EB） would compete to adsorb on the support surface, it seems reasonable to assume that TIC14 might predominately occupy this site, which can act as the most plausible active site for propylene polymerization. The first insertion of propylene monomer into the A5 active site model showed that it exhibited good regioselectivity but poor stereospecificity in the absence of electron donor.展开更多
基金funded by Woosong University Academic Research 2024.
文摘This study investigates the application of Learnable Memory Vision Transformers(LMViT)for detecting metal surface flaws,comparing their performance with traditional CNNs,specifically ResNet18 and ResNet50,as well as other transformer-based models including Token to Token ViT,ViT withoutmemory,and Parallel ViT.Leveraging awidely-used steel surface defect dataset,the research applies data augmentation and t-distributed stochastic neighbor embedding(t-SNE)to enhance feature extraction and understanding.These techniques mitigated overfitting,stabilized training,and improved generalization capabilities.The LMViT model achieved a test accuracy of 97.22%,significantly outperforming ResNet18(88.89%)and ResNet50(88.90%),aswell as the Token to TokenViT(88.46%),ViT without memory(87.18),and Parallel ViT(91.03%).Furthermore,LMViT exhibited superior training and validation performance,attaining a validation accuracy of 98.2%compared to 91.0%for ResNet 18,96.0%for ResNet50,and 89.12%,87.51%,and 91.21%for Token to Token ViT,ViT without memory,and Parallel ViT,respectively.The findings highlight the LMViT’s ability to capture long-range dependencies in images,an areawhere CNNs struggle due to their reliance on local receptive fields and hierarchical feature extraction.The additional transformer-based models also demonstrate improved performance in capturing complex features over CNNs,with LMViT excelling particularly at detecting subtle and complex defects,which is critical for maintaining product quality and operational efficiency in industrial applications.For instance,the LMViT model successfully identified fine scratches and minor surface irregularities that CNNs often misclassify.This study not only demonstrates LMViT’s potential for real-world defect detection but also underscores the promise of other transformer-based architectures like Token to Token ViT,ViT without memory,and Parallel ViT in industrial scenarios where complex spatial relationships are key.Future research may focus on enhancing LMViT’s computational efficiency for deployment in real-time quality control systems.
基金Program for Young Excellent Talents in the University of Fujian Province,Grant/Award Number:201847The National Key Research and Development Program of China,Grant/Award Number:2022YFB3206605Natural Science Foundation of Xiamen Municipality,Grant/Award Number:3502Z20227189。
文摘Efficient detection of surface defects is primary for ensuring product quality during manufacturing processes.To enhance the performance of deep learning-based methods in practical applications,the authors propose Dense-YOLO,a fast surface defect detection network that combines the strengths of DenseNet and you only look once version 3(YOLOv3).The authors design a lightweight backbone network with improved densely connected blocks,optimising the utilisation of shallow features while maintaining high detection speeds.Additionally,the authors refine the feature pyramid network of YOLOv3 to increase the recall of tiny defects and overall positioning accuracy.Furthermore,an online multi-angle template matching technique is introduced based on normalised cross-correlation to precisely locate the detection area.This refined template matching method not only accelerates detection speed but also mitigates the influence of the background.To validate the effectiveness of our enhancements,the authors conduct comparative experiments across two private datasets and one public dataset.Results show that Dense-YOLO outperforms existing methods,such as faster R-CNN,YOLOv3,YOLOv5s,YOLOv7,and SSD,in terms of mean average precision(mAP)and detection speed.Moreover,Dense-YOLO outperforms networks inherited from VGG and ResNet,including improved faster R-CNN,FCOS,M2Det-320 and FRCN,in mAP.
基金funded by the National Natural Science Foundation of China(grant number 62306186)the Technology Plan Joint Foundation of Liaoning Province(grant number 2023-MSLH-246)the Technology Plan Joint Foundation of Liaoning Province(grant number 2023-BSBA-238).
文摘Detecting surface defects on unused rails is crucial for evaluating rail quality and durability to ensure the safety of rail transportation.However,existing detection methods often struggle with challenges such as complex defect morphology,texture similarity,and fuzzy edges,leading to poor accuracy and missed detections.In order to resolve these problems,we propose MSCM-Net(Multi-Scale Cross-Modal Network),a multiscale cross-modal framework focused on detecting rail surface defects.MSCM-Net introduces an attention mechanism to dynamically weight the fusion of RGB and depth maps,effectively capturing and enhancing features at different scales for each modality.To further enrich feature representation and improve edge detection in blurred areas,we propose a multi-scale void fusion module that integrates multi-scale feature information.To improve cross-modal feature fusion,we develop a cross-enhanced fusion module that transfers fused features between layers to incorporate interlayer information.We also introduce a multimodal feature integration module,which merges modality-specific features from separate decoders into a shared decoder,enhancing detection by leveraging richer complementary information.Finally,we validate MSCM-Net on the NEU RSDDS-AUG RGB-depth dataset,comparing it against 12 leading methods,and the results show that MSCM-Net achieves superior performance on all metrics.
基金supported by the Scientific and technological key project in Henan Province 22210224002the Natural Science Foundation of Henan Polytechnic University B2021-38.
文摘The service life of internal combustion engines is significantly influenced by surface defects in cylinder liners.To address the limitations of traditional detection methods,we propose an enhanced YOLOv8 model with Swin Transformer as the backbone network.This approach leverages Swin Transformer's multi-head self-attention mechanism for improved feature extraction of defects spanning various scales.Integrated with the YOLOv8 detection head,our model achieves a mean average precision of 85.1%on our dataset,outperforming baseline methods by 1.4%.The model's effectiveness is further demonstrated on a steel-surface defect dataset,indicating its broad applicability in industrial surface defect detection.Our work highlights the potential of combining Swin Transformer and YOLOv8 for accurate and efficient defect detection.
基金supported by the Jiangsu Province Science and Technology Policy Guidance Program(Industry-University-Research Cooperation)/Forward-Looking Joint Research Project(BY2016005-05).
文摘In response to themissed and false detections that are easily caused by the large variety of and significant differences among aluminum surface defects,a detection algorithm based on an improved You Only Look Once(YOLO)v8n network is proposed.First,a C2f_DWR_DRB module is constructed by introducing a dilation-wise residual(DWR)module and a dilated reparameterization block(DRB)to replace the C2f module at the high level of the backbone network,enriching the gradient flow information and increasing the effective receptive field(ERF).Second,an efficient local attention(ELA)mechanism is fused with the high-level screening-feature pyramid networks(HS-FPN)module,and an ELA_HSFPN is designed to replace the original feature fusion module,enhancing the ability of the network to cope with multiscale detection tasks.Moreover,a lightweight shared convolutional detection head(SCDH)is introduced to reduce the number of parameters and the computational complexity of the module while enhancing the performance and generalizability of the model.Finally,the soft intersection over union(SIoU)replaces the original loss function to improve the convergence speed and prediction accuracy of the model.Experimental results show that compared with that of the original YOLOv8n model,the mAP@0.5 of the improved algorithm is increased by 5.1%,the number of parameters and computational complexity are reduced by 33.3%and 32.1%,respectively,and the FPS is increased by 4.9%.Compared with other mainstream object detection algorithms,the improved algorithm still leads in terms of core indicators and has good generalizability for surface defects encountered in other industrial scenarios.
基金supported in part by the National Key R&D Program of China(Grant No.2023YFB3307604)the Shanxi Province Basic Research Program Youth Science Research Project(Grant Nos.202303021212054 and 202303021212046)+3 种基金the Key Projects Supported by Hebei Natural Science Foundation(Grant No.E2024203125)the National Science Foundation of China(Grant No.52105391)the Hebei Provincial Science and Technology Major Project(Grant No.23280101Z)the National Key Laboratory of Metal Forming Technology and Heavy Equipment Open Fund(Grant No.S2308100.W17).
文摘A novel dual-branch decoding fusion convolutional neural network model(DDFNet)specifically designed for real-time salient object detection(SOD)on steel surfaces is proposed.DDFNet is based on a standard encoder–decoder architecture.DDFNet integrates three key innovations:first,we introduce a novel,lightweight multi-scale progressive aggregation residual network that effectively suppresses background interference and refines defect details,enabling efficient salient feature extraction.Then,we propose an innovative dual-branch decoding fusion structure,comprising the refined defect representation branch and the enhanced defect representation branch,which enhance accuracy in defect region identification and feature representation.Additionally,to further improve the detection of small and complex defects,we incorporate a multi-scale attention fusion module.Experimental results on the public ESDIs-SOD dataset show that DDFNet,with only 3.69 million parameters,achieves detection performance comparable to current state-of-the-art models,demonstrating its potential for real-time industrial applications.Furthermore,our DDFNet-L variant consistently outperforms leading methods in detection performance.The code is available at https://github.com/13140W/DDFNet.
基金supported by the Key Research and Development Program of Shaanxi Province-International Science and Technology Cooperation Program Project (No.2020KW-001)the Contract for Xi'an Municipal Science and Technology Plan Project-Xi'an City Strong Foundation Innovation Plan (No.21XJZZ0074)the Key Project of Graduate Student Innovation Fund at Xi'an University of Posts and Telecommunications (No.CXJJZL2023013)。
文摘The detection of surface defects in concrete bridges using deep learning is of significant importance for reducing operational risks,saving maintenance costs,and driving the intelligent transformation of bridge defect detection.In contrast to the subjective and inefficient manual visual inspection,deep learning-based algorithms for concrete defect detection exhibit remarkable advantages,emerging as a focal point in recent research.This paper comprehensively analyzes the research progress of deep learning algorithms in the field of surface defect detection in concrete bridges in recent years.It introduces the early detection methods for surface defects in concrete bridges and the development of deep learning.Subsequently,it provides an overview of deep learning-based concrete bridge surface defect detection research from three aspects:image classification,object detection,and semantic segmentation.The paper summarizes the strengths and weaknesses of existing methods and the challenges they face.Additionally,it analyzes and prospects the development trends of surface defect detection in concrete bridges.
基金supported by the National Natural Science Foundation of China(No.62103298)the Natural Science Foundation of Hebei Province(No.F2018209289)。
文摘Aiming at the problems of low detection efficiency and difficult positioning of traditional steel surface defect detection methods,a lightweight steel surface defect detection model based on you only look once version 7(YOLOv7)is proposed.First,a cascading style sheets(CSS)block module is proposed,which uses more lightweight operations to obtain redundant information in the feature map,reduces the amount of computation,and effectively improves the detection speed.Secondly,the improved spatial pyramid pooling with cross stage partial convolutions(SPPCSPC)structure is adopted to ensure that the model can also pay attention to the defect location information while predicting the defect category information,obtain richer defect features.In addition,the convolution operation in the original model is simplified,which significantly reduces the size of the model and helps to improve the detection speed.Finally,using efficient intersection over union(EIOU)loss to focus on high-quality anchors,speed up convergence and improve positioning accuracy.Experiments were carried out on the Northeastern University-defect(NEU-DET)steel surface defect dataset.Compared with the original YOLOv7 model,the number of parameters of this model was reduced by 40%,the frames per second(FPS)reached 112,and the average accuracy reached 79.1%,the detection accuracy and speed have been improved,which can meet the needs of steel surface defect detection.
基金supported by the Natural Science Foundation of Liaoning Province(No.2022-MS-353)Basic Scientific Research Project of Education Department of Liaoning Province(Nos.2020LNZD06 and LJKMZ20220640)。
文摘Surface defects can affect the quality of steel plate.Many methods based on computer vision are currently applied to surface defect detection of steel plate.However,their real-time performance and object detection of small defect are still unsatisfactory.An improved object detection network based on You Only Look One-level Feature(YOLOF)is proposed to show excellent performance in surface defect detection of steel plate,called DLF-YOLOF.First,the anchor-free detector is used to reduce the network hyperparameters.Secondly,deformable convolution network and local spatial attention module are introduced into the feature extraction network to increase the contextual information in the feature maps.Also,the soft non-maximum suppression is used to improve detection accuracy significantly.Finally,data augmentation is performed for small defect objects during training to improve detection accuracy.Experiments show the average precision and average precision for small objects are 42.7%and 33.5%at a detection speed of 62 frames per second on a single GPU,respectively.This shows that DLF-YOLOF has excellent performance to meet the needs of industrial real-time detection.
文摘Ensuring high product quality is of paramount importance in pharmaceutical drug manufacturing,as it is subject to rigorous regulatory practices.This study presents a research focused on the development of an on-line detection method and system for identifying surface defects in pharmaceutical products packaged in aluminum-plastic blisters.Firstly,the aluminum-plastic blister packages exhibit multi-scale features and inter-class indistinction.To address this,the deep semantic network with boundary refinement(DSN-BR)model is proposed,which leverages semantic segmentation domain knowledge,to accurately segment the defects in pixel level.Additionally,a specialized image acquisition module that minimizes the impact of ambient light is established,ensuring high-quality image capture.Finally,the image acquisition module,image detection module,and data management module are designed to construct a comprehensive online surface defect detection system.To validate the effectiveness of our approach,we employ a real dataset for instance verification on the implemented system.The experimental results substantiate the outstanding performance of the DSN-BR,achieving the mean intersection over union(MIoU)of 90.5%.Furthermore,the proposed system achieves an inference speed of up to 14.12 f/s,while attaining an F1-Score of 98.25%.These results demonstrate that the system meets the actual needs of the enterprise and provides theoretical and methodological support for intelligent inspection of product surface quality.By standardizing the control process of pharmaceutical manufacturing and improving the management capability of the manufacturing process,our approach holds significant market application prospects.
基金supported by the National Natural Science Foundation of China(51805078)Project of National Key Laboratory of Advanced Casting Technologies(CAT2023-002)the 111 Project(B16009).
文摘Segment Anything Model(SAM)is a cutting-edge model that has shown impressive performance in general object segmentation.The birth of the segment anything is a groundbreaking step towards creating a universal intelligent model.Due to its superior performance in general object segmentation,it quickly gained attention and interest.This makes SAM particularly attractive in industrial surface defect segmentation,especially for complex industrial scenes with limited training data.However,its segmentation ability for specific industrial scenes remains unknown.Therefore,in this work,we select three representative and complex industrial surface defect detection scenarios,namely strip steel surface defects,tile surface defects,and rail surface defects,to evaluate the segmentation performance of SAM.Our results show that although SAM has great potential in general object segmentation,it cannot achieve satisfactory performance in complex industrial scenes.Our test results are available at:https://github.com/VDT-2048/SAM-IS.
基金supported by the National Natural Science Foundation of China(No.61976083)Hubei Province Key R&D Program of China(No.2022BBA0016).
文摘Printed Circuit Board(PCB)surface tiny defect detection is a difficult task in the integrated circuit industry,especially since the detection of tiny defects on PCB boards with large-size complex circuits has become one of the bottlenecks.To improve the performance of PCB surface tiny defects detection,a PCB tiny defects detection model based on an improved attention residual network(YOLOX-AttResNet)is proposed.First,the unsupervised clustering performance of the K-means algorithm is exploited to optimize the channel weights for subsequent operations by feeding the feature mapping into the SENet(Squeeze and Excitation Network)attention network;then the improved K-means-SENet network is fused with the directly mapped edges of the traditional ResNet network to form an augmented residual network(AttResNet);and finally,the AttResNet module is substituted for the traditional ResNet structure in the backbone feature extraction network of mainstream excellent detection models,thus improving the ability to extract small features from the backbone of the target detection network.The results of ablation experiments on a PCB surface defect dataset show that AttResNet is a reliable and efficient module.In Torify the performance of AttResNet for detecting small defects in large-size complex circuit images,a series of comparison experiments are further performed.The results show that the AttResNet module combines well with the five best existing target detection frameworks(YOLOv3,YOLOX,Faster R-CNN,TDD-Net,Cascade R-CNN),and all the combined new models have improved detection accuracy compared to the original model,which suggests that the AttResNet module proposed in this paper can help the detection model to extract target features.Among them,the YOLOX-AttResNet model proposed in this paper performs the best,with the highest accuracy of 98.45% and the detection speed of 36 FPS(Frames Per Second),which meets the accuracy and real-time requirements for the detection of tiny defects on PCB surfaces.This study can provide some new ideas for other real-time online detection tasks of tiny targets with high-resolution images.
基金supported by the National Science Foundation of China(Grant Nos.52068049 and 51908266)the Science Fund for Distinguished Young Scholars of Gansu Province(No.21JR7RA267)Hongliu Outstanding Young Talents Program of Lanzhou University of Technology.
文摘The accumulation of defects on wind turbine blade surfaces can lead to irreversible damage,impacting the aero-dynamic performance of the blades.To address the challenge of detecting and quantifying surface defects on wind turbine blades,a blade surface defect detection and quantification method based on an improved Deeplabv3+deep learning model is proposed.Firstly,an improved method for wind turbine blade surface defect detection,utilizing Mobilenetv2 as the backbone feature extraction network,is proposed based on an original Deeplabv3+deep learning model to address the issue of limited robustness.Secondly,through integrating the concept of pre-trained weights from transfer learning and implementing a freeze training strategy,significant improvements have been made to enhance both the training speed and model training accuracy of this deep learning model.Finally,based on segmented blade surface defect images,a method for quantifying blade defects is proposed.This method combines image stitching algorithms to achieve overall quantification and risk assessment of the entire blade.Test results show that the improved Deeplabv3+deep learning model reduces training time by approximately 43.03%compared to the original model,while achieving mAP and MIoU values of 96.87%and 96.93%,respectively.Moreover,it demonstrates robustness in detecting different surface defects on blades across different back-grounds.The application of a blade surface defect quantification method enables the precise quantification of dif-ferent defects and facilitates the assessment of risk levels associated with defect measurements across the entire blade.This method enables non-contact,long-distance,high-precision detection and quantification of surface defects on the blades,providing a reference for assessing surface defects on wind turbine blades.
文摘Precision steel balls are critical components in precision bearings.Surface defects on the steel balls will significantly reduce their useful life and cause linear or rotational transmission errors.Human visual inspection of precision steel balls demands significant labor work.Besides,human inspection cannot maintain consistent quality assurance.To address these limitations and reduce inspection time,a convolutional neural network(CNN)based optical inspection system has been developed that automatically detects steel ball defects using a novel designated vertical mechanism.During image detection processing,two key challenges were addressed and resolved.They are the reflection caused by the coaxial light onto the ball center and the image deformation appearing at the edge of the steel balls.The special vertical rotating mechanism utilizing a spinning rod along with a spiral track was developed to enable successful and reliable full steel ball surface inspection during the rod rotation.The combination of the spinning rod and the spiral rotating component effectively rotates the steel ball to facilitate capturing complete surface images.Geometric calculations demonstrate that the steel balls can be completely inspected through specific rotation degrees,with the surface fully captured in 12 photo shots.These images are then analyzed by a CNN to determine surface quality defects.This study presents a new inspection method that enables the entire examination of steel ball surfaces.The successful development of this innovative automated optical inspection system with CNN represents a significant advancement in inspection quality control for precision steel balls.
基金supported by the National Natural Science Foundation of China under Grant 62306128the Basic Science Research Project of Jiangsu Provincial Department of Education under Grant 23KJD520003the Leading Innovation Project of Changzhou Science and Technology Bureau under Grant CQ20230072.
文摘Real-time detection of surface defects on cables is crucial for ensuring the safe operation of power systems.However,existing methods struggle with small target sizes,complex backgrounds,low-quality image acquisition,and interference from contamination.To address these challenges,this paper proposes the Real-time Cable Defect Detection Network(RC2DNet),which achieves an optimal balance between detection accuracy and computational efficiency.Unlike conventional approaches,RC2DNet introduces a small object feature extraction module that enhances the semantic representation of small targets through feature pyramids,multi-level feature fusion,and an adaptive weighting mechanism.Additionally,a boundary feature enhancement module is designed,incorporating boundary-aware convolution,a novel boundary attention mechanism,and an improved loss function to significantly enhance boundary localization accuracy.Experimental results demonstrate that RC2DNet outperforms state-of-the-art methods in precision,recall,F1-score,mean Intersection over Union(mIoU),and frame rate,enabling real-time and highly accurate cable defect detection in complex backgrounds.
基金Project(51174151)supported by the National Natural Science Foundation of ChinaProject(2010Z19003)supported by the Major Scientific Research Program of Hubei Provincial Department of Education,ChinaProject(2010CDB03403)supported by the Natural Science Foundation of Science and Technology Department of Hubei Province,China
文摘A more effective and accurate improved Sobel algorithm has been developed to detect surface defects on heavy rails. The proposed method can make up for the mere sensitivity to X and Y directions of the Sobel algorithm by adding six templates at different directions. Meanwhile, an experimental platform for detecting surface defects consisting of the bed-jig, image-forming system with CCD cameras and light sources, parallel computer system and cable system has been constructed. The detection results of the backfin defects show that the improved Sobel algorithm can achieve an accurate and efficient positioning with decreasing interference noises to the defect edge. It can also extract more precise features and characteristic parameters of the backfin defect. Furthermore, the BP neural network adopted for defects classification with the inputting characteristic parameters of improved Sobel algorithm can obtain the optimal training precision of 0.0095827 with 106 iterative steps and time of 3 s less than Sobel algorithm with 146 steps and 5 s. Finally, an enhanced identification rate of 10% for the defects is also confirmed after the Sobel algorithm is improved.
基金supports by the Program for New Century Excellent Talents in Chinese Universities (No.NCET-08-0726)Beijing Nova Program (No. 2007B027)the Fundamental Research Funds for the Central Universities (No. FRF-TP-09-027B)
文摘Feature extraction is essential to the classification of surface defect images. The defects of hot-rolled steels distribute in different directions. Therefore, the methods of multi-scale geometric analysis （MGA） were employed to decompose the image into several directional subba^ds at several scales. Then, the statistical features of each subband were calculated to produce a high-dimensional feature vector, which was reduced to a lower-dimensional vector by graph embedding algorithms. Finally, support vector machine （SVM） was used for defect classification. The multi-scale feature extraction method was implemented via curvelet transform and kernel locality preserving projections （KLPP）. Experiment results show that the proposed method is effective for classifying the surface defects of hot-rolled steels and the total classification rate is up to 97.33%.
文摘Surface defects of the cold-rolled sheets of Ti-IF steel were studied and analyzed. After analyzing surface defects of cold-rolled sheets, such as shelling defects, holes and sliver defects by SEM/EDS, a variety of inclusions were found. In addition, the distribution of macro-inclusions in slabs was analyzed by MIDAS method. The results show the macroscopic inclusion bands of head slabs and normal slabs are in 1/8 slab thickness regions of both inner arc side and outer arc side. The formation process of the defects in the cold-rolled sheets was simulated with an experimental cold-rolling machine for comparison. The results show that there were three kinds of inclusions underneath the surface defects: Al2O3, SiO2 and particles from slag entrainment, which were the main reason for defect formation during cold rolling.
基金supported by the Double First‐rate Subject‐Food Science and Engineering Program of Hebei Province (2018SPGCA18)Young Tip‐top Talents Plan of Universities and Colleges in Hebei Province of China (BJ2017026)the Specific Foundation for Doctor in Hebei Agriculture University of China (ZD201709)~~
文摘Inefficient charge separation and limited light absorption are two critical issues associated with high‐efficiency photocatalytic H2production using TiO2.Surface defects within a certain concentration range in photocatalyst materials are beneficial for photocatalytic activity.In this study,surface defects(oxygen vacancies and metal cation replacement defects)were induced with a facile and effective approach by surface doping with low‐cost transition metals(Co,Ni,Cu,and Mn)on ultrafine TiO2.The obtained surface‐defective TiO2exhibited a3–4‐fold improved activity compared to that of the original ultrafine TiO2.In addition,a H2production rate of3.4μmol/h was obtained using visible light(λ>420nm)irradiation.The apparent quantum yield(AQY)at365nm reached36.9%over TiO2‐Cu,significantly more than the commercial P25TiO2.The enhancement of photocatalytic H2production activity can be attributed to improved rapid charge separation efficiency andexpanded light absorption window.This hydrothermal treatment with transition metal was proven to be a very facile and effective method for obtaining surface defects.
基金supported by the Program of Introducing Talents of Discipline to Universities (B08021)Fundamental Research Funds for the Central Universities
文摘The formations of defective MgC12 surfaces, and subsequent adsorption of Ti species and electron donor, as well as propylene polymerization over the Ziegler-Natta catalyst have been investigated using density functional theory （DFT） method. Twelve possible support models of regular and defective MgC12 （110） and （100） surfaces were built. The individual adsorptions of titanium chlorides as mononuclear or dinuclear, and ethyl benzoate （EB） as electron donor, on these models were evaluated. The analysis of energies presented the cases of EB adsorption were generally more stable than titanium chlorides on both surfaces. Thus, EB as internal electron donor mainly prevented TIC14 from coordinating on the MgC12 surfaces where mostly non-stereospecific active sites could be formed. Exceptionally, A5 the site model with terminal Cl-vacancy on the MgC12 support, presented stronger adsorption of TiCl4 than that of EB on （110） surface. Since the TIC14 and ethyl benzoate （EB） would compete to adsorb on the support surface, it seems reasonable to assume that TIC14 might predominately occupy this site, which can act as the most plausible active site for propylene polymerization. The first insertion of propylene monomer into the A5 active site model showed that it exhibited good regioselectivity but poor stereospecificity in the absence of electron donor.
