Defect inspection of specular curved surface is a challenging job. Taking steel balls for example, a new method based on reflected pattern integrity recognition is put forward. The specular steel ball surfac...Defect inspection of specular curved surface is a challenging job. Taking steel balls for example, a new method based on reflected pattern integrity recognition is put forward. The specular steel ball surface will totally reflect the patterns when it is placed inside a dome-shaped light source, whose inner wall is modified by patterns with certain regular. Distortion or intermittence of reflected pattern will occur at the defective part, which indicates the pattern has lost its integrity. Based on the integrity analysis of reflected pattern images？ surface defects can be revealed. In this paper, a set of concentric circles are used as the pattern and an image processing algorithm is customized to extract the surface defects. Results show that the proposed method is effective for the specular curved surface defect inspection展开更多
The growing demand for electronic devices, smart devices, and the Internet of Things constitutes the primary driving force for marching down the path of decreased critical dimension and increased circuit intricacy of ...The growing demand for electronic devices, smart devices, and the Internet of Things constitutes the primary driving force for marching down the path of decreased critical dimension and increased circuit intricacy of integrated circuits. However, as sub-10 nm high-volume manufacturing is becoming the mainstream, there is greater awareness that defects introduced by original equipment manufacturer components impact yield and manufacturing costs. The identification, positioning, and classification of these defects, including random particles and systematic defects, are becoming more and more challenging at the 10 nm node and beyond.Very recently, the combination of conventional optical defect inspection with emerging techniques such as nanophotonics, optical vortices, computational imaging, quantitative phase imaging, and deep learning is giving the field a new possibility. Hence, it is extremely necessary to make a thorough review for disclosing new perspectives and exciting trends, on the foundation of former great reviews in the field of defect inspection methods. In this article, we give a comprehensive review of the emerging topics in the past decade with a focus on three specific areas:(a) the defect detectability evaluation,(b) the diverse optical inspection systems,and(c) the post-processing algorithms. We hope, this work can be of importance to both new entrants in the field and people who are seeking to use it in interdisciplinary work.展开更多
In the proposed system for online inspection of steel balls, a diffuse illumination is developed to enhance defect appearances and produce high quality images. To fully view the entire sphere, a novel unfolding method...In the proposed system for online inspection of steel balls, a diffuse illumination is developed to enhance defect appearances and produce high quality images. To fully view the entire sphere, a novel unfolding method is put forward based on geometrical analysis, which only requires one-dimensional movement of the balls and a pair of cameras to capture images from different directions. Moreover, a realtime inspection algorithm is customized to improve both accuracy and efficiency. The precision and recall of the sample set were 87.7% and 98%, respectively. The average time cost on image processing and analysis for a steel ball was 47 ms, and the total time cost was less than 200 ms plus the cost of image acquisition and balls' movement. The system can sort 18 000 balls per hour with a spatial resolution higher than 0.01 mm.展开更多
Additive manufacturing(AM) technology is considered one of the most promising manufacturing technologies in the aerospace and defense industries. However, AM components are known to have various internal defects, such...Additive manufacturing(AM) technology is considered one of the most promising manufacturing technologies in the aerospace and defense industries. However, AM components are known to have various internal defects, such as powder agglomeration, balling, porosity,internal cracks and thermal/internal stress, which can significantly affect the quality, mechanical properties and safety of final parts. Therefore, defect inspection methods are important for reducing manufactured defects and improving the surface quality and mechanical properties of AM components. This paper describes defect inspection technologies and their applications in AM processes. The architecture of defects in AM processes is reviewed. Traditional defect detection technology and the surface defect detection methods based on deep learning are summarized, and future aspects are suggested.展开更多
An intelligent camera for surface defect inspection is presented which can pre-process the surface image of a rolled strip and pick defective areas out at a spead of 1 600 meters per minute. The camera is made up of a...An intelligent camera for surface defect inspection is presented which can pre-process the surface image of a rolled strip and pick defective areas out at a spead of 1 600 meters per minute. The camera is made up of a high speed line CCD, a 60 Mb/s CCD digitizer with correlated double sampling function, and a field programmable gate array(FPGA), which can quickly distinguish defective areas using a perceptron embedded in FPGA thus the data to be further processed would dramatically be reduced. Some experiments show that the camera can meet high producing speed, and reduce cost and complexity of automation surface inspection systems.展开更多
Purpose–This research aims to improve the performance of rail fastener defect inspection method for multi railways,to effectively ensure the safety of railway operation.Design/methodology/approach–Firstly,a fastener...Purpose–This research aims to improve the performance of rail fastener defect inspection method for multi railways,to effectively ensure the safety of railway operation.Design/methodology/approach–Firstly,a fastener region location method based on online learning strategy was proposed,which can locate fastener regions according to the prior knowledge of track image and template matching method.Online learning strategy is used to update the template library dynamically,so that the method not only can locate fastener regions in the track images of multi railways,but also can automatically collect and annotate fastener samples.Secondly,a fastener defect recognition method based on deep convolutional neural network was proposed.The structure of recognition network was designed according to the smaller size and the relatively single content of the fastener region.The data augmentation method based on the sample random sorting strategy is adopted to reduce the impact of the imbalance of sample size on recognition performance.Findings–Test verification of the proposed method is conducted based on the rail fastener datasets of multi railways.Specifically,fastener location module has achieved an average detection rate of 99.36%,and fastener defect recognition module has achieved an average precision of 96.82%.Originality/value–The proposed method can accurately locate fastener regions and identify fastener defect in the track images of different railways,which has high reliability and strong adaptability to multi railways.展开更多
In integrated circuit(IC)manufacturing,fast,nondestructive,and precise detection of defects in patterned wafers,realized by bright-field microscopy,is one of the critical factors for ensuring the final performance and...In integrated circuit(IC)manufacturing,fast,nondestructive,and precise detection of defects in patterned wafers,realized by bright-field microscopy,is one of the critical factors for ensuring the final performance and yields of chips.With the critical dimensions of IC nanostructures continuing to shrink,directly imaging or classifying deep-subwavelength defects by bright-field microscopy is challenging due to the well-known diffraction barrier,the weak scattering effect,and the faint correlation between the scattering cross-section and the defect morphology.Herein,we propose an optical far-field inspection method based on the form-birefringence scattering imaging of the defective nanostructure,which can identify and classify various defects without requiring optical super-resolution.The technique is built upon the principle of breaking the optical form birefringence of the original periodic nanostructures by the defect perturbation under the anisotropic illumination modes,such as the orthogonally polarized plane waves,then combined with the high-order difference of far-field images.We validated the feasibility and effectiveness of the proposed method in detecting deep subwavelength defects through rigid vector imaging modeling and optical detection experiments of various defective nanostructures based on polarization microscopy.On this basis,an intelligent classification algorithm for typical patterned defects based on a dual-channel AlexNet neural network has been proposed,stabilizing the classification accuracy ofλ/16-sized defects with highly similar features at more than 90%.The strong classification capability of the two-channel network on typical patterned defects can be attributed to the high-order difference image and its transverse gradient being used as the network’s input,which highlights the polarization modulation difference between different patterned defects more significantly than conventional bright-field microscopy results.This work will provide a new but easy-to-operate method for detecting and classifying deep-subwavelength defects in patterned wafers or photomasks,which thus endows current online inspection equipment with more missions in advanced IC manufacturing.展开更多
Precision steel balls are critical components in precision bearings.Surface defects on the steel balls will significantly reduce their useful life and cause linear or rotational transmission errors.Human visual inspec...Precision steel balls are critical components in precision bearings.Surface defects on the steel balls will significantly reduce their useful life and cause linear or rotational transmission errors.Human visual inspection of precision steel balls demands significant labor work.Besides,human inspection cannot maintain consistent quality assurance.To address these limitations and reduce inspection time,a convolutional neural network(CNN)based optical inspection system has been developed that automatically detects steel ball defects using a novel designated vertical mechanism.During image detection processing,two key challenges were addressed and resolved.They are the reflection caused by the coaxial light onto the ball center and the image deformation appearing at the edge of the steel balls.The special vertical rotating mechanism utilizing a spinning rod along with a spiral track was developed to enable successful and reliable full steel ball surface inspection during the rod rotation.The combination of the spinning rod and the spiral rotating component effectively rotates the steel ball to facilitate capturing complete surface images.Geometric calculations demonstrate that the steel balls can be completely inspected through specific rotation degrees,with the surface fully captured in 12 photo shots.These images are then analyzed by a CNN to determine surface quality defects.This study presents a new inspection method that enables the entire examination of steel ball surfaces.The successful development of this innovative automated optical inspection system with CNN represents a significant advancement in inspection quality control for precision steel balls.展开更多
Skin defect inspection is one of the most significant tasks in the conventional process of aircraft inspection.This paper proposes a vision-based method of pixel-level defect detection,which is based on the Mask Scori...Skin defect inspection is one of the most significant tasks in the conventional process of aircraft inspection.This paper proposes a vision-based method of pixel-level defect detection,which is based on the Mask Scoring R-CNN.First,an attention mechanism and a feature fusion module are introduced,to improve feature representation.Second,a new classifier head—consisting of four convolutional layers and a fully connected layer—is proposed,to reduce the influence of information around the area of the defect.Third,to evaluate the proposed method,a dataset of aircraft skin defects was constructed,containing 276 images with a resolution of 960×720 pixels.Experimental results show that the proposed classifier head improves the detection and segmentation accuracy,for aircraft skin defect inspection,more effectively than the attention mechanism and feature fusion module.Compared with the Mask R-CNN and Mask Scoring R-CNN,the proposed method increased the segmentation precision by approximately 21%and 19.59%,respectively.These results demonstrate that the proposed method performs favorably against the other two methods of pixellevel aircraft skin defect detection.展开更多
In manufacture of precise optical products, it is important to inspect and classify the potential defects existing on the products’ surfaces after precise machining in order to obtain high quality in both functionali...In manufacture of precise optical products, it is important to inspect and classify the potential defects existing on the products’ surfaces after precise machining in order to obtain high quality in both functionality and aesthetics. The existing methods for detecting and classifying defects all are low accuracy or efficiency or high cost in inspection process. In this paper, a new inspection system based on machine vision has been introduced, which uses automatic focusing and image mosaic technologies to rapidly acquire distinct surface image, and employs Case-Based Reasoning(CBR)method in defects classification. A modificatory fuzzy similarity algorithm in CBR has been adopted for more quick and robust need of pattern recognition in practice inspection. Experiments show that the system can inspect surface diameter of 500mm in half an hour with resolving power of 0.8μm diameter according to digs or 0.5μm transverse width according to scratches. The proposed inspection principles and methods not only have meet manufacturing requirements of precise optical products, but also have great potential applications in other fields of precise surface inspection.展开更多
Solder bumps are widely used in surface mount components, which provide electrical and mechanical connection between the chip/package and the substrate. As the solder bump getting smaller in dimension and pitch, it be...Solder bumps are widely used in surface mount components, which provide electrical and mechanical connection between the chip/package and the substrate. As the solder bump getting smaller in dimension and pitch, it becomes more difficult to inspect the solder defects hidden in the IC package. In this paper, an intelligent inspection method using the scanning acoustic microscopy(SAM) and the fuzzy C-means(FCM) algorithm was investigated. A flip chip package of FA10 was chosen as the test sample. The SAM tests of FA10 were carried out in C-scan mode. The sub-image of every solder bump was segmented from the SAM image. The statistical features were then calculated and adopted for clustering of solder bumps using the FCM algorithm. The recognition results of FCM reached a high accuracy of 94.3%. The intelligent system is effective for defect inspection in high density packages.展开更多
The manual visual inspections of façade building defects are posing a high and increasing cost for building asset managers,particularly when inspections delay projects or require asset outages,visits to decommiss...The manual visual inspections of façade building defects are posing a high and increasing cost for building asset managers,particularly when inspections delay projects or require asset outages,visits to decommissioned sites or work within hazardous environments.This paper reports on the development,testing and delivery of a working mobile app prototype to facilitate the inspections and documentation of building facade condition monitoring.The work presented builds upon the development of an online platform for remote building inspection based on the integration of methodologies and tools,including VR(virtual reality),and digital photogrammetry to collect real-time data that support automated decision making.The mobile app:(i)allows the user to import 3D models and 2D building plans;(ii)provides the means of first-person exploration of models via a VR headset;and(iii)captures,records and catalogues images of façade defect types,and the date and time.An inspection case study was used to demonstrate and evaluate the mobile app prototype.The Building Inspector app allows building professionals to manage inspections and to track past and ongoing monitoring of the condition of building façades.展开更多
Wafer bin map(WBM)inspection is a critical approach for evaluating the semiconductor manufacturing process.An excellent inspection algorithm can improve the production efficiency and yield.This paper proposes a WBM de...Wafer bin map(WBM)inspection is a critical approach for evaluating the semiconductor manufacturing process.An excellent inspection algorithm can improve the production efficiency and yield.This paper proposes a WBM defect pattern inspection strategy based on the DenseNet deep learning model,the structure and training loss function are improved according to the characteristics of the WBM.In addition,a constrained mean filtering algorithm is proposed to filter the noise grains.In model prediction,an entropy-based Monte Carlo dropout algorithm is employed to quantify the uncertainty of the model decision.The experimental results show that the recognition ability of the improved DenseNet is better than that of traditional algorithms in terms of typical WBM defect patterns.Analyzing the model uncertainty can not only effectively reduce the miss or false detection rate but also help to identify new patterns.展开更多
A robust system for backlit keyboard inspection is revealed. The backlit keyboard not only has changeable diverse colors but also has the laser marking keys. The keys on the keyboard can be divided into regions of fun...A robust system for backlit keyboard inspection is revealed. The backlit keyboard not only has changeable diverse colors but also has the laser marking keys. The keys on the keyboard can be divided into regions of function keys, normal keys, and number keys. However, there might have some types of defects： incorrect illuminating area, non-uniform illumination of specified inspection region（IR）, and incorrect luminance and intensity of individual key. Since the illumination features of backlit keyboard are too complex to inspect for human inspector in the production line, an auto-mated inspection system for the backlit keyboard is proposed in this paper. The system was designed into the operation module and inspection module. A set of image processing methods were developed for these defects inspection. Some experimental results demonstrate the robustness and effectiveness of the proposed system.展开更多
A novel automatic ultrasonic system used for the inspection of pipeline girth welds is developed, in which a linear phased array transducer using electronic scan is adopted. Optimal array parameters are determined bas...A novel automatic ultrasonic system used for the inspection of pipeline girth welds is developed, in which a linear phased array transducer using electronic scan is adopted. Optimal array parameters are determined based on a mathematical model of acoustic field for linear phased army derived from Huygens＇ principle. The testing method and the system structure are introduced. The experimental results show that the phased array transducer system has the same detectability as that of conventional ultrasonic transducer system, but the system architecture can be simplified greatly, and the testing flexibility and the testing speed can be improved greatly.展开更多
With the development of wood industry, the processing of wood products becomemore significant. This paper discusses the developmen of machine vision system used to inspect andclassny the various types of defects of wo...With the development of wood industry, the processing of wood products becomemore significant. This paper discusses the developmen of machine vision system used to inspect andclassny the various types of defects of wood suxface. The surface defeds means the variations ofcolour and textUre. The machine vision system is to dated undesirable 'defecs' that can appear onthe surface of rough wood lwnber. A neural network was used within the Blackboard framework fora labeling verification step of the high-level recognition module of vision system. The system hasbere successfully tested on a number of boards from several different species.展开更多
Defect inspection is critical in semiconductor manufacturing for product quality improvement at reduced production costs.A whole new manufacturing process is often associated with a new set of defects that can cause s...Defect inspection is critical in semiconductor manufacturing for product quality improvement at reduced production costs.A whole new manufacturing process is often associated with a new set of defects that can cause serious damage to the manufacturing system.Therefore,classifying existing defects and new defects provides crucial clues to fix the issue in the newly introduced manufacturing process.We present a multi-task hybrid transformer(MT-former)that distinguishes novel defects from the known defects in electron microscope images of semiconductors.MT-former consists of upstream and downstream training stages.In the upstream stage,an encoder of a hybrid transformer is trained by solving both classification and reconstruction tasks for the existing defects.In the downstream stage,the shared encoder is fine-tuned by simultaneously learning the classification as well as a deep support vector domain description(Deep-SVDD)to detect the new defects among the existing ones.With focal loss,we also design a hybrid-transformer using convolutional and an efficient self-attention module.Our model is evaluated on real-world data from SK Hynix and on publicly available data from magnetic tile defects and HAM10000.For SK Hynix data,MT-former achieved higher AUC as compared with a Deep-SVDD model,by 8.19%for anomaly detection and by 9.59%for classifying the existing classes.Furthermore,the best AUC(magnetic tile defect 67.9%,HAM1000070.73%)on the public dataset achieved with the proposed model implies that MT-former would be a useful model for classifying the new types of defects from the existing ones.展开更多
基金Tianjin Research Program of Application Foundation and Advanced Technology(No.14JCYBJC18600,No.14JCZDJC39700)National Key Scientific Instrument and Equipment Development Project(No.2013YQ17053903)
文摘Defect inspection of specular curved surface is a challenging job. Taking steel balls for example, a new method based on reflected pattern integrity recognition is put forward. The specular steel ball surface will totally reflect the patterns when it is placed inside a dome-shaped light source, whose inner wall is modified by patterns with certain regular. Distortion or intermittence of reflected pattern will occur at the defective part, which indicates the pattern has lost its integrity. Based on the integrity analysis of reflected pattern images？ surface defects can be revealed. In this paper, a set of concentric circles are used as the pattern and an image processing algorithm is customized to extract the surface defects. Results show that the proposed method is effective for the specular curved surface defect inspection
基金funded by the National Natural Science Foundation of China(Grant Nos.52175509 and 52130504)the National Key Research and Development Program of China(2017YFF0204705)+1 种基金the Key Research and Development Plan of Hubei Province(2021BAA013)the National Science and Technology Major Project(2017ZX02101006-004)。
文摘The growing demand for electronic devices, smart devices, and the Internet of Things constitutes the primary driving force for marching down the path of decreased critical dimension and increased circuit intricacy of integrated circuits. However, as sub-10 nm high-volume manufacturing is becoming the mainstream, there is greater awareness that defects introduced by original equipment manufacturer components impact yield and manufacturing costs. The identification, positioning, and classification of these defects, including random particles and systematic defects, are becoming more and more challenging at the 10 nm node and beyond.Very recently, the combination of conventional optical defect inspection with emerging techniques such as nanophotonics, optical vortices, computational imaging, quantitative phase imaging, and deep learning is giving the field a new possibility. Hence, it is extremely necessary to make a thorough review for disclosing new perspectives and exciting trends, on the foundation of former great reviews in the field of defect inspection methods. In this article, we give a comprehensive review of the emerging topics in the past decade with a focus on three specific areas:(a) the defect detectability evaluation,(b) the diverse optical inspection systems,and(c) the post-processing algorithms. We hope, this work can be of importance to both new entrants in the field and people who are seeking to use it in interdisciplinary work.
文摘In the proposed system for online inspection of steel balls, a diffuse illumination is developed to enhance defect appearances and produce high quality images. To fully view the entire sphere, a novel unfolding method is put forward based on geometrical analysis, which only requires one-dimensional movement of the balls and a pair of cameras to capture images from different directions. Moreover, a realtime inspection algorithm is customized to improve both accuracy and efficiency. The precision and recall of the sample set were 87.7% and 98%, respectively. The average time cost on image processing and analysis for a steel ball was 47 ms, and the total time cost was less than 200 ms plus the cost of image acquisition and balls' movement. The system can sort 18 000 balls per hour with a spatial resolution higher than 0.01 mm.
基金financial support of the National Key R&D Program of China (Project Nos. 2017YFA0701200, 2016YFF0102003)the Shanghai Science and Technology Committee Innovation Grant (Grant Nos. 19ZR1404600, 17JC1400601)the Science Challenging Program of CAEP (Grant No. JCKY2016212A506-0106)。
文摘Additive manufacturing(AM) technology is considered one of the most promising manufacturing technologies in the aerospace and defense industries. However, AM components are known to have various internal defects, such as powder agglomeration, balling, porosity,internal cracks and thermal/internal stress, which can significantly affect the quality, mechanical properties and safety of final parts. Therefore, defect inspection methods are important for reducing manufactured defects and improving the surface quality and mechanical properties of AM components. This paper describes defect inspection technologies and their applications in AM processes. The architecture of defects in AM processes is reviewed. Traditional defect detection technology and the surface defect detection methods based on deep learning are summarized, and future aspects are suggested.
文摘An intelligent camera for surface defect inspection is presented which can pre-process the surface image of a rolled strip and pick defective areas out at a spead of 1 600 meters per minute. The camera is made up of a high speed line CCD, a 60 Mb/s CCD digitizer with correlated double sampling function, and a field programmable gate array(FPGA), which can quickly distinguish defective areas using a perceptron embedded in FPGA thus the data to be further processed would dramatically be reduced. Some experiments show that the camera can meet high producing speed, and reduce cost and complexity of automation surface inspection systems.
基金funded by the Key Research and Development Project of China Academy of Railway Sciences Corporation Limited(2021YJ310).
文摘Purpose–This research aims to improve the performance of rail fastener defect inspection method for multi railways,to effectively ensure the safety of railway operation.Design/methodology/approach–Firstly,a fastener region location method based on online learning strategy was proposed,which can locate fastener regions according to the prior knowledge of track image and template matching method.Online learning strategy is used to update the template library dynamically,so that the method not only can locate fastener regions in the track images of multi railways,but also can automatically collect and annotate fastener samples.Secondly,a fastener defect recognition method based on deep convolutional neural network was proposed.The structure of recognition network was designed according to the smaller size and the relatively single content of the fastener region.The data augmentation method based on the sample random sorting strategy is adopted to reduce the impact of the imbalance of sample size on recognition performance.Findings–Test verification of the proposed method is conducted based on the rail fastener datasets of multi railways.Specifically,fastener location module has achieved an average detection rate of 99.36%,and fastener defect recognition module has achieved an average precision of 96.82%.Originality/value–The proposed method can accurately locate fastener regions and identify fastener defect in the track images of different railways,which has high reliability and strong adaptability to multi railways.
基金funded by National Natural Science Foundation of China(Grant Nos.52130504,52305577,and 52175509)the Key Research and Development Plan of Hubei Province(Grant No.2022BAA013)+4 种基金the Major Program(JD)of Hubei Province(Grant No.2023BAA008-2)the Interdisciplinary Research Program of Huazhong University of Science and Technology(2023JCYJ047)the Innovation Project of Optics Valley Laboratory(Grant No.OVL2023PY003)the Postdoctoral Fellowship Program(Grade B)of China Postdoctoral Science Foundation(Grant No.GZB20230244)the fellowship from the China Postdoctoral Science Foundation(2024M750995)。
文摘In integrated circuit(IC)manufacturing,fast,nondestructive,and precise detection of defects in patterned wafers,realized by bright-field microscopy,is one of the critical factors for ensuring the final performance and yields of chips.With the critical dimensions of IC nanostructures continuing to shrink,directly imaging or classifying deep-subwavelength defects by bright-field microscopy is challenging due to the well-known diffraction barrier,the weak scattering effect,and the faint correlation between the scattering cross-section and the defect morphology.Herein,we propose an optical far-field inspection method based on the form-birefringence scattering imaging of the defective nanostructure,which can identify and classify various defects without requiring optical super-resolution.The technique is built upon the principle of breaking the optical form birefringence of the original periodic nanostructures by the defect perturbation under the anisotropic illumination modes,such as the orthogonally polarized plane waves,then combined with the high-order difference of far-field images.We validated the feasibility and effectiveness of the proposed method in detecting deep subwavelength defects through rigid vector imaging modeling and optical detection experiments of various defective nanostructures based on polarization microscopy.On this basis,an intelligent classification algorithm for typical patterned defects based on a dual-channel AlexNet neural network has been proposed,stabilizing the classification accuracy ofλ/16-sized defects with highly similar features at more than 90%.The strong classification capability of the two-channel network on typical patterned defects can be attributed to the high-order difference image and its transverse gradient being used as the network’s input,which highlights the polarization modulation difference between different patterned defects more significantly than conventional bright-field microscopy results.This work will provide a new but easy-to-operate method for detecting and classifying deep-subwavelength defects in patterned wafers or photomasks,which thus endows current online inspection equipment with more missions in advanced IC manufacturing.
文摘Precision steel balls are critical components in precision bearings.Surface defects on the steel balls will significantly reduce their useful life and cause linear or rotational transmission errors.Human visual inspection of precision steel balls demands significant labor work.Besides,human inspection cannot maintain consistent quality assurance.To address these limitations and reduce inspection time,a convolutional neural network(CNN)based optical inspection system has been developed that automatically detects steel ball defects using a novel designated vertical mechanism.During image detection processing,two key challenges were addressed and resolved.They are the reflection caused by the coaxial light onto the ball center and the image deformation appearing at the edge of the steel balls.The special vertical rotating mechanism utilizing a spinning rod along with a spiral track was developed to enable successful and reliable full steel ball surface inspection during the rod rotation.The combination of the spinning rod and the spiral rotating component effectively rotates the steel ball to facilitate capturing complete surface images.Geometric calculations demonstrate that the steel balls can be completely inspected through specific rotation degrees,with the surface fully captured in 12 photo shots.These images are then analyzed by a CNN to determine surface quality defects.This study presents a new inspection method that enables the entire examination of steel ball surfaces.The successful development of this innovative automated optical inspection system with CNN represents a significant advancement in inspection quality control for precision steel balls.
基金National Natural Science Foundation of China(Nos.U2033201 and U1633105)。
文摘Skin defect inspection is one of the most significant tasks in the conventional process of aircraft inspection.This paper proposes a vision-based method of pixel-level defect detection,which is based on the Mask Scoring R-CNN.First,an attention mechanism and a feature fusion module are introduced,to improve feature representation.Second,a new classifier head—consisting of four convolutional layers and a fully connected layer—is proposed,to reduce the influence of information around the area of the defect.Third,to evaluate the proposed method,a dataset of aircraft skin defects was constructed,containing 276 images with a resolution of 960×720 pixels.Experimental results show that the proposed classifier head improves the detection and segmentation accuracy,for aircraft skin defect inspection,more effectively than the attention mechanism and feature fusion module.Compared with the Mask R-CNN and Mask Scoring R-CNN,the proposed method increased the segmentation precision by approximately 21%and 19.59%,respectively.These results demonstrate that the proposed method performs favorably against the other two methods of pixellevel aircraft skin defect detection.
文摘In manufacture of precise optical products, it is important to inspect and classify the potential defects existing on the products’ surfaces after precise machining in order to obtain high quality in both functionality and aesthetics. The existing methods for detecting and classifying defects all are low accuracy or efficiency or high cost in inspection process. In this paper, a new inspection system based on machine vision has been introduced, which uses automatic focusing and image mosaic technologies to rapidly acquire distinct surface image, and employs Case-Based Reasoning(CBR)method in defects classification. A modificatory fuzzy similarity algorithm in CBR has been adopted for more quick and robust need of pattern recognition in practice inspection. Experiments show that the system can inspect surface diameter of 500mm in half an hour with resolving power of 0.8μm diameter according to digs or 0.5μm transverse width according to scratches. The proposed inspection principles and methods not only have meet manufacturing requirements of precise optical products, but also have great potential applications in other fields of precise surface inspection.
基金supported by the National Natural Science Foundation of China(Grant Nos.51675250&51705203)the Natural Science Foundation of Jiangsu Province(Grant No.BK20160183)the Open Foundation of State Key Lab of Digital Manufacturing Equipment&Technology(Grant No.DMETKF2016005)
文摘Solder bumps are widely used in surface mount components, which provide electrical and mechanical connection between the chip/package and the substrate. As the solder bump getting smaller in dimension and pitch, it becomes more difficult to inspect the solder defects hidden in the IC package. In this paper, an intelligent inspection method using the scanning acoustic microscopy(SAM) and the fuzzy C-means(FCM) algorithm was investigated. A flip chip package of FA10 was chosen as the test sample. The SAM tests of FA10 were carried out in C-scan mode. The sub-image of every solder bump was segmented from the SAM image. The statistical features were then calculated and adopted for clustering of solder bumps using the FCM algorithm. The recognition results of FCM reached a high accuracy of 94.3%. The intelligent system is effective for defect inspection in high density packages.
文摘The manual visual inspections of façade building defects are posing a high and increasing cost for building asset managers,particularly when inspections delay projects or require asset outages,visits to decommissioned sites or work within hazardous environments.This paper reports on the development,testing and delivery of a working mobile app prototype to facilitate the inspections and documentation of building facade condition monitoring.The work presented builds upon the development of an online platform for remote building inspection based on the integration of methodologies and tools,including VR(virtual reality),and digital photogrammetry to collect real-time data that support automated decision making.The mobile app:(i)allows the user to import 3D models and 2D building plans;(ii)provides the means of first-person exploration of models via a VR headset;and(iii)captures,records and catalogues images of façade defect types,and the date and time.An inspection case study was used to demonstrate and evaluate the mobile app prototype.The Building Inspector app allows building professionals to manage inspections and to track past and ongoing monitoring of the condition of building façades.
基金Project(Z135060009002)supported by the Ministry of Industry and Information Technology of ChinaProject(KZ202010005004)supported by Beijing Municipal Commission of Education and Beijing Municipal Natural Science Foundation of China。
文摘Wafer bin map(WBM)inspection is a critical approach for evaluating the semiconductor manufacturing process.An excellent inspection algorithm can improve the production efficiency and yield.This paper proposes a WBM defect pattern inspection strategy based on the DenseNet deep learning model,the structure and training loss function are improved according to the characteristics of the WBM.In addition,a constrained mean filtering algorithm is proposed to filter the noise grains.In model prediction,an entropy-based Monte Carlo dropout algorithm is employed to quantify the uncertainty of the model decision.The experimental results show that the recognition ability of the improved DenseNet is better than that of traditional algorithms in terms of typical WBM defect patterns.Analyzing the model uncertainty can not only effectively reduce the miss or false detection rate but also help to identify new patterns.
文摘A robust system for backlit keyboard inspection is revealed. The backlit keyboard not only has changeable diverse colors but also has the laser marking keys. The keys on the keyboard can be divided into regions of function keys, normal keys, and number keys. However, there might have some types of defects： incorrect illuminating area, non-uniform illumination of specified inspection region（IR）, and incorrect luminance and intensity of individual key. Since the illumination features of backlit keyboard are too complex to inspect for human inspector in the production line, an auto-mated inspection system for the backlit keyboard is proposed in this paper. The system was designed into the operation module and inspection module. A set of image processing methods were developed for these defects inspection. Some experimental results demonstrate the robustness and effectiveness of the proposed system.
文摘A novel automatic ultrasonic system used for the inspection of pipeline girth welds is developed, in which a linear phased array transducer using electronic scan is adopted. Optimal array parameters are determined based on a mathematical model of acoustic field for linear phased army derived from Huygens＇ principle. The testing method and the system structure are introduced. The experimental results show that the phased array transducer system has the same detectability as that of conventional ultrasonic transducer system, but the system architecture can be simplified greatly, and the testing flexibility and the testing speed can be improved greatly.
文摘With the development of wood industry, the processing of wood products becomemore significant. This paper discusses the developmen of machine vision system used to inspect andclassny the various types of defects of wood suxface. The surface defeds means the variations ofcolour and textUre. The machine vision system is to dated undesirable 'defecs' that can appear onthe surface of rough wood lwnber. A neural network was used within the Blackboard framework fora labeling verification step of the high-level recognition module of vision system. The system hasbere successfully tested on a number of boards from several different species.
基金supported by SK Hynix AICC(P23.03)by the National Research Foundation of Korea(NRF)grant funded by the Ministry of Science and ICT(2023R1A2C3004880)+4 种基金the Ministry of Education(2020R1A6A1A03047902 and 2022R1A6A1A03052954)by Basic Science Research Program through the NRF funded by the Ministry of Education(RS-2024-00415450)by Institute of Information&communications Technology Planning&Evaluation(IITP)grant funded by the Korea government(MSIT)(No.RS-2019-II191906,Artificial Intelligence Graduate School Program(POSTECH))by the BK21 FOUR projectby Glocal University 30 projects.
文摘Defect inspection is critical in semiconductor manufacturing for product quality improvement at reduced production costs.A whole new manufacturing process is often associated with a new set of defects that can cause serious damage to the manufacturing system.Therefore,classifying existing defects and new defects provides crucial clues to fix the issue in the newly introduced manufacturing process.We present a multi-task hybrid transformer(MT-former)that distinguishes novel defects from the known defects in electron microscope images of semiconductors.MT-former consists of upstream and downstream training stages.In the upstream stage,an encoder of a hybrid transformer is trained by solving both classification and reconstruction tasks for the existing defects.In the downstream stage,the shared encoder is fine-tuned by simultaneously learning the classification as well as a deep support vector domain description(Deep-SVDD)to detect the new defects among the existing ones.With focal loss,we also design a hybrid-transformer using convolutional and an efficient self-attention module.Our model is evaluated on real-world data from SK Hynix and on publicly available data from magnetic tile defects and HAM10000.For SK Hynix data,MT-former achieved higher AUC as compared with a Deep-SVDD model,by 8.19%for anomaly detection and by 9.59%for classifying the existing classes.Furthermore,the best AUC(magnetic tile defect 67.9%,HAM1000070.73%)on the public dataset achieved with the proposed model implies that MT-former would be a useful model for classifying the new types of defects from the existing ones.
