Coal is heterogeneous in nature,and thus the characterization of coal is essential before its use for a specific purpose.Thus,the current study aims to develop a machine vision system for automated coal characterizati...Coal is heterogeneous in nature,and thus the characterization of coal is essential before its use for a specific purpose.Thus,the current study aims to develop a machine vision system for automated coal characterizations.The model was calibrated using 80 image samples that are captured for different coal samples in different angles.All the images were captured in RGB color space and converted into five other color spaces(HSI,CMYK,Lab,xyz,Gray)for feature extraction.The intensity component image of HSI color space was further transformed into four frequency components(discrete cosine transform,discrete wavelet transform,discrete Fourier transform,and Gabor filter)for the texture features extraction.A total of 280 image features was extracted and optimized using a step-wise linear regression-based algorithm for model development.The datasets of the optimized features were used as an input for the model,and their respective coal characteristics(analyzed in the laboratory)were used as outputs of the model.The R-squared values were found to be 0.89,0.92,0.92,and 0.84,respectively,for fixed carbon,ash content,volatile matter,and moisture content.The performance of the proposed artificial neural network model was also compared with the performances of performances of Gaussian process regression,support vector regression,and radial basis neural network models.The study demonstrates the potential of the machine vision system in automated coal characterization.展开更多
An optical inspection method of the Ball Grid Array package(BGA) was proposed by using a machine vision system. The developed machine vision system could get main critical factors for BGA quality evaluation, such as t...An optical inspection method of the Ball Grid Array package(BGA) was proposed by using a machine vision system. The developed machine vision system could get main critical factors for BGA quality evaluation, such as the height of solder ball, diameter, pitch and coplanarity. The experiment has proved that this system is available for BGA failure detection.展开更多
To improve the identification for visual defect of TFF-LCD, a new machine vision system is proposed, which is superior to human eye inspection. The system respectively employs a CCD camera to capture the image of TFT-...To improve the identification for visual defect of TFF-LCD, a new machine vision system is proposed, which is superior to human eye inspection. The system respectively employs a CCD camera to capture the image of TFT-LCD panel and an image processing system to identify potential visual defects. Image pre-processing, such as average filtering and geometric correction, was performed on the captured image, and then a candidate area of defect was segmented from the background. Feature information extracted from the area of interest entered a fuzzy rule-based classifier that simulated the defect inspection of TFT-LCD undertaken by experienced technicians. Experiment results show that the machine vision system can obtain fast, objective and accurate inspection compared with subjective and inaccurate human eye inspection.展开更多
This study assessed the feasibility of developing a machine vision system equipped with ultraviolet (UV) light, using changes in fish-surface color to predict aerobic plate count (APC, a standard freshness indicator) ...This study assessed the feasibility of developing a machine vision system equipped with ultraviolet (UV) light, using changes in fish-surface color to predict aerobic plate count (APC, a standard freshness indicator) during storage. The APC values were tested and images of the fish surface were taken when fish were stored at room temperature. Then, images</span></span><span style="font-family:Verdana;"><span style="font-family:Verdana;"><span style="font-family:Verdana;">’</span></span></span><span><span><span><span> color-space conversion among RGB, HSV, and L*a*b* color spaces was carried out and analyzed. The results revealed that a* and b* values from the UV-light image decreased linearly during storage. A further regression analysis of these two parameters with APC value demonstrated a good exponential relationship between the a* value and the APC value (R</span><sup><span>2</span></sup><span> = 0.97), followed by the b* (R</span><sup><span>2</span></sup><span> = 0.85). Therefore, our results suggest that the change in color of the fish surface under UV light can be used to assess fish freshness during storage.展开更多
With the development of wood industry, the processing of wood products becomemore significant. This paper discusses the developmen of machine vision system used to inspect andclassny the various types of defects of wo...With the development of wood industry, the processing of wood products becomemore significant. This paper discusses the developmen of machine vision system used to inspect andclassny the various types of defects of wood suxface. The surface defeds means the variations ofcolour and textUre. The machine vision system is to dated undesirable 'defecs' that can appear onthe surface of rough wood lwnber. A neural network was used within the Blackboard framework fora labeling verification step of the high-level recognition module of vision system. The system hasbere successfully tested on a number of boards from several different species.展开更多
With the rapid development of computer vision technology,artificial intelligence algorithms,and high-performance computing platforms,machine vision technology has gradually shown its great potential in automated produ...With the rapid development of computer vision technology,artificial intelligence algorithms,and high-performance computing platforms,machine vision technology has gradually shown its great potential in automated production lines,especially in defect detection.Machine vision technology can be applied in many industries such as semiconductor,automobile manufacturing,aerospace,food,and drugs,which can significantly improve detection efficiency and accuracy,reduce labor costs,improve product quality,enhance market competitiveness,and provide strong support for the arrival of Industry 4.0 era.In this article,the concept,advantages,and disadvantages of machine vision and the algorithm framework of machine vision in the defect detection system are briefly described,aiming to promote the rapid development of industry and strengthen China’s industry.展开更多
Machine vision systems(MVSs)are an important component of intelligent systems,such as autonomous vehicles and robots.However,with the continuous increase in data and new application scenarios,new requirements are put ...Machine vision systems(MVSs)are an important component of intelligent systems,such as autonomous vehicles and robots.However,with the continuous increase in data and new application scenarios,new requirements are put forward for the next generation of MVS.There is an urgent need to find new material systems to complement the existing semiconductor technology based on thin-film materials,and new architectures must be explored to improve efficiency.Because of their unique physical properties,two-dimensional(2D)materials have received extensive attention for use in MVSs,especially in biomimetic ones:the human visual system,which can process complex visual information with low power consumption,provides a model for next-generation MVSs.This review paper summarizes the progress and challenges of applying 2D material photodetectors in sense-memory-computational integration and biomimetic image sensors for machine vision.展开更多
Bioinspired neuromorphic machine vision system(NMVS)that integrates retinomorphic sensing and neuromorphic computing into one monolithic system is regarded as the most promising architecture for visual perception.Howe...Bioinspired neuromorphic machine vision system(NMVS)that integrates retinomorphic sensing and neuromorphic computing into one monolithic system is regarded as the most promising architecture for visual perception.However,the large intensity range of natural lights and complex illumination conditions in actual scenarios always require the NMVS to dynamically adjust its sensitivity according to the environmental conditions,just like the visual adaptation function of the human retina.Although some opto-sensors with scotopic or photopic adaption have been developed,NMVSs,especially fully flexible NMVSs,with both scotopic and photopic adaptation functions are rarely reported.Here we propose an ion-modulation strategy to dynamically adjust the photosensitivity and time-varying activation/inhibition characteristics depending on the illumination conditions,and develop a flexible ionmodulated phototransistor array based on MoS_(2)/graphdiyne heterostructure,which can execute both retinomorphic sensing and neuromorphic computing.By controlling the intercalated Li^(+) ions in graphdiyne,both scotopic and photopic adaptation functions are demonstrated successfully.A fully flexible NMVS consisting of front-end retinomorphic vision sensors and a back-end convolutional neural network is constructed based on the as-fabricated 28×28 device array,demonstrating quite high recognition accuracies for both dim and bright images and robust flexibility.This effort for fully flexible and monolithic NMVS paves the way for its applications in wearable scenarios.展开更多
Monitring pest populations in paddy fields is important to effectively implement integrated pest management.Light traps are widely used to monitor field pests all over the world.Most conventional light traps still inv...Monitring pest populations in paddy fields is important to effectively implement integrated pest management.Light traps are widely used to monitor field pests all over the world.Most conventional light traps still involve manual identification of target pests from lots of trapped insects,which is time-consuming,labor-intensive and error-prone,especially in pest peak periods.In this paper,we developed an automatic monitoring system for rice light-trap pests based on machine vision.This system is composed of an itelligent light trap,a computer or mobile phone client platform and a cloud server.The light trap firstly traps,kills and disperses insects,then collects images of trapped insects and sends each image to the cloud server.Five target pests in images are automatically identifed and counted by pest identification models loaded in the server.To avoid light-trap insects piling up,a vibration plate and a moving rotation conveyor belt are adopted to disperse these trapped insects.There was a close correlation(r=0.92)between our automatic and manual identification methods based on the daily pest number of one-year images from one light trap.Field experiments demonstrated the effectiveness and accuracy of our automatic light trap monitoring system.展开更多
Estimation of fruit size in tree fruit crops is essential for selective robotic harvesting and crop-load estimation.Machine vision systems for fruit detection and localization have been studied widely for robotic harv...Estimation of fruit size in tree fruit crops is essential for selective robotic harvesting and crop-load estimation.Machine vision systems for fruit detection and localization have been studied widely for robotic harvesting and crop-load estimation.However,only a few studies have been carried out to estimate fruit size in orchards using machine vision systems.This study was carried out to develop a machine vision system consisting of a color CCD camera and a time-of-flight(TOF)light-based 3D camera for estimating apple size in tree canopies.As a measure of fruit size,the major axis(longest axis)was estimated based on(i)the 3D coordinates of pixels on corresponding apple surfaces,and(ii)the 2D size of individual pixels within apple surfaces.In the 3D coordinates-based method,the distance between pairs of pixels within apple regions were calculated using 3D coordinates,and the maximum distance between all pixel pairs within an apple region was estimated to be the major axis.The accuracy of estimating the major axis using 3D coordinates was 69.1%.In the pixel-size-based method,the physical sizes of pixels were estimated using a calibration model developed based on pixel coordinates and the distance to pixels from the camera.The major axis length was then estimated by summing the size of individual pixels along the major axis of the fruit.The accuracy of size estimation increased to 84.8%when the pixel size-based method was used.The results showed the potential for estimating fruit size in outdoor environments using a 3D machine vision system.展开更多
Text detection and recognition is a hot topic in computer vision,which is considered to be the further development of the traditional optical character recognition(OCR)technology.With the rapid development of machine ...Text detection and recognition is a hot topic in computer vision,which is considered to be the further development of the traditional optical character recognition(OCR)technology.With the rapid development of machine vision system and the wide application of deep learning algorithms,text recognition has achieved excellent performance.In contrast,detecting text block from complex natural scenes is still a challenging task.At present,many advanced natural scene text detection algorithms have been proposed,but most of them run slow due to the complexity of the detection pipeline and can not be applied to industrial scenes.In this paper,we proposed a CCD based machine vision system for realtime text detection in invoice images.In this system,we applied optimizations from several aspects including the optical system,the hardware architecture,and the deep learning algorithm to improve the speed performance of the machine vision system.The experimental data confirms that the optimization methods can significantly improve the running speed of the machine vision system and make it meeting the real-time text detection requirements in industrial scenarios.展开更多
In manufacture of precise optical products, it is important to inspect and classify the potential defects existing on the products’ surfaces after precise machining in order to obtain high quality in both functionali...In manufacture of precise optical products, it is important to inspect and classify the potential defects existing on the products’ surfaces after precise machining in order to obtain high quality in both functionality and aesthetics. The existing methods for detecting and classifying defects all are low accuracy or efficiency or high cost in inspection process. In this paper, a new inspection system based on machine vision has been introduced, which uses automatic focusing and image mosaic technologies to rapidly acquire distinct surface image, and employs Case-Based Reasoning(CBR)method in defects classification. A modificatory fuzzy similarity algorithm in CBR has been adopted for more quick and robust need of pattern recognition in practice inspection. Experiments show that the system can inspect surface diameter of 500mm in half an hour with resolving power of 0.8μm diameter according to digs or 0.5μm transverse width according to scratches. The proposed inspection principles and methods not only have meet manufacturing requirements of precise optical products, but also have great potential applications in other fields of precise surface inspection.展开更多
During the sizing process,yarn congestion fault occurs at the reed teeth of a sizing machine.At present,the yarn congestion fault is generally handled by manual detection.The sizing production line operates on a large...During the sizing process,yarn congestion fault occurs at the reed teeth of a sizing machine.At present,the yarn congestion fault is generally handled by manual detection.The sizing production line operates on a large scale and runs continuously.Untimely handling of the yarn congestion fault causes a large amount of yarn waste.In this research,a machine vision-based algorithm for yarn congestion fault detection is developed.Through the analysis of the congestion fault and interference contour characteristics,the basic idea of image phase subtraction to identify the congestion fault is determined.To address the interference information appearing after image phase subtraction,the image pre-processing methods of Canny edge extraction and mean filtering are employed.According to the fault size and location characteristics,the fault contour detection algorithm based on inter-frame difference is designed.To mitigate the camera vibration interference,the anti-vibration interference algorithm based on affine transformation is studied,and the fault detection algorithm for the total yarn congestion fault is determined.The detection of 20 sets of field data is carried out,and the detection rate reaches 90%.This fault detection algorithm realizes the automatic detection of yarn congestion fault of sizing machine with certain real-time performance and accuracy.展开更多
Objective:To explore a rapid and accurate method for evaluating the quality of Prunus mandshurica(Maxim.)Koehne(P.mandshurica,Ku Xing Ren)during rancidity using machine vision and learning.Methods:Sensory evaluation a...Objective:To explore a rapid and accurate method for evaluating the quality of Prunus mandshurica(Maxim.)Koehne(P.mandshurica,Ku Xing Ren)during rancidity using machine vision and learning.Methods:Sensory evaluation and chemometrics were used to classify P.mandshurica quality grades after rancidity.Chemical indicators of the P.mandshurica quality change were determined to verify the ob-tained grades and support the subsequent modeling.The International Commission on Illumination color space was used to extract the color features of the P.mandshurica.Discrimination and prediction models based on color features combined with multiple machine learning algorithms were established using 10-fold cross-validation and external test set validation.Results:The P.mandshurica rancidity samples were allocated to three quality grades.The Bayes net model based on powder color successfully identified the P.mandshurica at different grades with an accuracy of 88.89%and 100%using two validations,and the naive Bayes model based on section color achieved the same accuracy with an receiver operating characteristic area of 0.979.The instance-based k-nearest neighbors model based on powder color performed best in predicting the amygdalin content[R^(2)=0.9801,mean absolute error(MAE)=0.2071,root mean squared error(RMSE)=0.4170],followed by the random com-mittee model in predicting the acid value(R^(2)=0.9580,MAE=1.5121,RMSE=1.9099)and the random forest model in predicting the peroxide value(R^(2)=0.8857,MAE=0.0027,RMSE=0.0035).Conclusion:This study demonstrates that color digitization analysis is a potential method for rapidly evaluating the quality of P.mandshurica across the rancidity process,providing a new reference for the quality assessment of traditional Chinese medicines.展开更多
Realizing the point-of-care tumor markers biodetection with good convenience and high sensitivity possesses great significance for prompting cancer monitoring and screening in biomedical study field.Herein,the quantum...Realizing the point-of-care tumor markers biodetection with good convenience and high sensitivity possesses great significance for prompting cancer monitoring and screening in biomedical study field.Herein,the quantum dots luminescence and microfluidic biochip with machine vision algorithm-based intelligent biosensing platform have been designed and manufactured for point-of-care tumor markers diagnostics.The employed quantum dots with excellent photoluminescent performance are modified with specific antibody as the optical labeling agents for the designed sandwich structure immunoassay.The corresponding biosensing investigations of the designed biodetection platform illustrate several advantages involving high sensitivity(~0.021 ng mL^(−1)),outstanding accessibility,and great integrability.Moreover,related test results of human-sourced artificial saliva samples demonstrate better detection capabilities compared with commercially utilized rapid test strips.Combining these infusive abilities,our elaborate biosensing platform is expected to exhibit potential applications for the future point-of-care tumor markers diagnostic area.展开更多
Vibration cutting has emerged as a promising method for creating surface functional microstructures.However,achieving precise tool setting is a time-consuming process that significantly impacts process efficiency.This...Vibration cutting has emerged as a promising method for creating surface functional microstructures.However,achieving precise tool setting is a time-consuming process that significantly impacts process efficiency.This study proposes an intelligent approach for tool setting in vibration cutting using machine vision and hearing,divided into two steps.In the first step,machine vision is employed to achieve rough precision in tool setting within tens of micrometers.Subsequently,in the second step,machine hearing utilizes sound pickup to capture vibration audio signals,enabling fine tool adjustment within 1μm precision.The relationship between the spectral intensity of vibration audio and cutting depth is analyzed to establish criteria for tool–workpiece contact.Finally,the efficacy of this approach is validated on an ultra-precision platform,demonstrating that the automated tool-setting process takes no more than 74 s.The total cost of the vision and hearing sensors is less than$1500.展开更多
The wear of work rolls significantly affects the production efficiency and product quality.However,existing methods for wear assessment fail to effectively quantify work roll surface wear conditions,thereby affecting ...The wear of work rolls significantly affects the production efficiency and product quality.However,existing methods for wear assessment fail to effectively quantify work roll surface wear conditions,thereby affecting the quality control of steel strips and maintenance strategies for rolls.To accurately assess the wear conditions of hot-rolling work rolls,this study initially established an apparatus for capturing high-precision roll surfaces images.Subsequently,a quantitative assessment of common surface wear morphologies was conducted,and a hot-rolling work roll surface wear dataset was constructed.The MobileNetV2 convolutional neural network(CNN),augmented by transfer learning,was employed to develop a MobileNetV2-wear detection and classification(WDC)surface wear grading model.A comparison with mainstream CNN models revealed that the MobileNetV2-WDC model achieved high-speed(21.92 ms)and accurate(96.86%)grading with minimal model parameters(2.27 M)and size(27 M),meeting the industrial efficiency and practicality requirements.A visual analysis of the model classification errors was conducted,outlining paths for further optimization.This study provides an efficient and accurate solution for detecting and grading surface wear on hot-rolling work rolls,enhancing product quality and extending the lifespan of rolls.展开更多
An automatic monitoring method of the 3-D deformation is presented for crustal fault based on laser and machine vision. The laser source and screen are independently set up in the headwall and footwall, the collimated...An automatic monitoring method of the 3-D deformation is presented for crustal fault based on laser and machine vision. The laser source and screen are independently set up in the headwall and footwall, the collimated laser beam creates a circular spot on the screen, meanwhile, the industrial camera captures the tiny deformation of the crustal fault by monitoring the change of the spot position. This method significantly reduces the cost of equipment and labor, provides daily sampling to ensure high continuity of data. A prototype of the automatic monitoring system is developed, and a repeatability test indicates that the error of spot jitter can be minimized by consecutive samples. Meanwhile, the environmental correction model is determined to ensure that environmental changes do not disturb the system. Furthermore, the automatic monitoring system has been applied at the deformation monitoring station(KJX02) of China Beishan underground research laboratory, where continuous deformation monitoring is underway.展开更多
In this paper, an automatic inspection system for weld surface appearance using machine vision has been developed to recognize weld surface defects such as porosities, cracks, etc. It can replace conventional manual v...In this paper, an automatic inspection system for weld surface appearance using machine vision has been developed to recognize weld surface defects such as porosities, cracks, etc. It can replace conventional manual visual inspection method, which is tedious, time-consuming, subjective, experience-depended, and sometimes biased. The system consists of a CCD camera, a self-designed annular light source, a sensor controller, a frame grabbing card, a computer and so on. After acquiring weld surface appearance images using CCD, the images are preprocessed using median filtering and a series of image enhancement algorithms. Then a dynamic threshold and morphology algorithms are applied to segment defect object. Finally, defect features information is obtained by eight neighborhoods boundary chain code algorithm. Experimental results show that the developed system is capable of inspecting most surface defects such as porosities, cracks with high reliability and accuracy.展开更多
To realize high-precision automatic measurement of two-dimensional geometric features on parts, a cooperative measurement system based on machine vision is constructed. Its hardware structure, functional composition a...To realize high-precision automatic measurement of two-dimensional geometric features on parts, a cooperative measurement system based on machine vision is constructed. Its hardware structure, functional composition and working principle are introduced. The mapping relationship between the feature image coordinates and the measuring space coordinates is established. The method of measuring path planning of small field of view （FOV） images is proposed. With the cooperation of the panoramic image of the object to be measured, the small FOV images with high object plane resolution are acquired automatically. Then, the auxiliary measuring characteristics are constructed and the parameters of the features to be measured are automatically extracted. Experimental results show that the absolute value of relative error is less than 0. 03% when applying the cooperative measurement system to gauge the hole distance of 100 mm nominal size. When the object plane resolving power of the small FOV images is 16 times that of the large FOV image, the measurement accuracy of small FOV images is improved by 14 times compared with the large FOV image. It is suitable for high-precision automatic measurement of two-dimensional complex geometric features distributed on large scale parts.展开更多
文摘Coal is heterogeneous in nature,and thus the characterization of coal is essential before its use for a specific purpose.Thus,the current study aims to develop a machine vision system for automated coal characterizations.The model was calibrated using 80 image samples that are captured for different coal samples in different angles.All the images were captured in RGB color space and converted into five other color spaces(HSI,CMYK,Lab,xyz,Gray)for feature extraction.The intensity component image of HSI color space was further transformed into four frequency components(discrete cosine transform,discrete wavelet transform,discrete Fourier transform,and Gabor filter)for the texture features extraction.A total of 280 image features was extracted and optimized using a step-wise linear regression-based algorithm for model development.The datasets of the optimized features were used as an input for the model,and their respective coal characteristics(analyzed in the laboratory)were used as outputs of the model.The R-squared values were found to be 0.89,0.92,0.92,and 0.84,respectively,for fixed carbon,ash content,volatile matter,and moisture content.The performance of the proposed artificial neural network model was also compared with the performances of performances of Gaussian process regression,support vector regression,and radial basis neural network models.The study demonstrates the potential of the machine vision system in automated coal characterization.
文摘An optical inspection method of the Ball Grid Array package(BGA) was proposed by using a machine vision system. The developed machine vision system could get main critical factors for BGA quality evaluation, such as the height of solder ball, diameter, pitch and coplanarity. The experiment has proved that this system is available for BGA failure detection.
文摘To improve the identification for visual defect of TFF-LCD, a new machine vision system is proposed, which is superior to human eye inspection. The system respectively employs a CCD camera to capture the image of TFT-LCD panel and an image processing system to identify potential visual defects. Image pre-processing, such as average filtering and geometric correction, was performed on the captured image, and then a candidate area of defect was segmented from the background. Feature information extracted from the area of interest entered a fuzzy rule-based classifier that simulated the defect inspection of TFT-LCD undertaken by experienced technicians. Experiment results show that the machine vision system can obtain fast, objective and accurate inspection compared with subjective and inaccurate human eye inspection.
文摘This study assessed the feasibility of developing a machine vision system equipped with ultraviolet (UV) light, using changes in fish-surface color to predict aerobic plate count (APC, a standard freshness indicator) during storage. The APC values were tested and images of the fish surface were taken when fish were stored at room temperature. Then, images</span></span><span style="font-family:Verdana;"><span style="font-family:Verdana;"><span style="font-family:Verdana;">’</span></span></span><span><span><span><span> color-space conversion among RGB, HSV, and L*a*b* color spaces was carried out and analyzed. The results revealed that a* and b* values from the UV-light image decreased linearly during storage. A further regression analysis of these two parameters with APC value demonstrated a good exponential relationship between the a* value and the APC value (R</span><sup><span>2</span></sup><span> = 0.97), followed by the b* (R</span><sup><span>2</span></sup><span> = 0.85). Therefore, our results suggest that the change in color of the fish surface under UV light can be used to assess fish freshness during storage.
文摘With the development of wood industry, the processing of wood products becomemore significant. This paper discusses the developmen of machine vision system used to inspect andclassny the various types of defects of wood suxface. The surface defeds means the variations ofcolour and textUre. The machine vision system is to dated undesirable 'defecs' that can appear onthe surface of rough wood lwnber. A neural network was used within the Blackboard framework fora labeling verification step of the high-level recognition module of vision system. The system hasbere successfully tested on a number of boards from several different species.
文摘With the rapid development of computer vision technology,artificial intelligence algorithms,and high-performance computing platforms,machine vision technology has gradually shown its great potential in automated production lines,especially in defect detection.Machine vision technology can be applied in many industries such as semiconductor,automobile manufacturing,aerospace,food,and drugs,which can significantly improve detection efficiency and accuracy,reduce labor costs,improve product quality,enhance market competitiveness,and provide strong support for the arrival of Industry 4.0 era.In this article,the concept,advantages,and disadvantages of machine vision and the algorithm framework of machine vision in the defect detection system are briefly described,aiming to promote the rapid development of industry and strengthen China’s industry.
基金supported by the National Natural Science Foundation of China(Grant Nos.61905266,62004207,61904184,62005303,62175045,62134009)Special grants from China Post-doctoral Science Foundation(Grant No.2021M700156)+1 种基金Youth Innovation Promotion Association CAS,Hangzhou Key Research and Development Program(Grant No.20212013B01)the Science and Technology Commission of Shanghai Municipality(Grant No.21JC1406100 and 20YF1455900).
文摘Machine vision systems(MVSs)are an important component of intelligent systems,such as autonomous vehicles and robots.However,with the continuous increase in data and new application scenarios,new requirements are put forward for the next generation of MVS.There is an urgent need to find new material systems to complement the existing semiconductor technology based on thin-film materials,and new architectures must be explored to improve efficiency.Because of their unique physical properties,two-dimensional(2D)materials have received extensive attention for use in MVSs,especially in biomimetic ones:the human visual system,which can process complex visual information with low power consumption,provides a model for next-generation MVSs.This review paper summarizes the progress and challenges of applying 2D material photodetectors in sense-memory-computational integration and biomimetic image sensors for machine vision.
基金National Natural Science Foundation of China,Grant/Award Numbers:12174207,51802220,62274119Fundamental Research Funds for the Central Universities,Grant/Award Numbers:010-63233006,010-DK2300010203。
文摘Bioinspired neuromorphic machine vision system(NMVS)that integrates retinomorphic sensing and neuromorphic computing into one monolithic system is regarded as the most promising architecture for visual perception.However,the large intensity range of natural lights and complex illumination conditions in actual scenarios always require the NMVS to dynamically adjust its sensitivity according to the environmental conditions,just like the visual adaptation function of the human retina.Although some opto-sensors with scotopic or photopic adaption have been developed,NMVSs,especially fully flexible NMVSs,with both scotopic and photopic adaptation functions are rarely reported.Here we propose an ion-modulation strategy to dynamically adjust the photosensitivity and time-varying activation/inhibition characteristics depending on the illumination conditions,and develop a flexible ionmodulated phototransistor array based on MoS_(2)/graphdiyne heterostructure,which can execute both retinomorphic sensing and neuromorphic computing.By controlling the intercalated Li^(+) ions in graphdiyne,both scotopic and photopic adaptation functions are demonstrated successfully.A fully flexible NMVS consisting of front-end retinomorphic vision sensors and a back-end convolutional neural network is constructed based on the as-fabricated 28×28 device array,demonstrating quite high recognition accuracies for both dim and bright images and robust flexibility.This effort for fully flexible and monolithic NMVS paves the way for its applications in wearable scenarios.
基金Supported by the Fundamental Public Welfare Research Program of Zhejiang Provincial Natural Science Foundation,China(LGN18C140007 and Y20C140024)the National High Technology Research and Development Program of China(863 Program,2013AA102402)the Agricultural Science and Technology Innovation Program of Chinese Academy of Agricultural Sciences.
文摘Monitring pest populations in paddy fields is important to effectively implement integrated pest management.Light traps are widely used to monitor field pests all over the world.Most conventional light traps still involve manual identification of target pests from lots of trapped insects,which is time-consuming,labor-intensive and error-prone,especially in pest peak periods.In this paper,we developed an automatic monitoring system for rice light-trap pests based on machine vision.This system is composed of an itelligent light trap,a computer or mobile phone client platform and a cloud server.The light trap firstly traps,kills and disperses insects,then collects images of trapped insects and sends each image to the cloud server.Five target pests in images are automatically identifed and counted by pest identification models loaded in the server.To avoid light-trap insects piling up,a vibration plate and a moving rotation conveyor belt are adopted to disperse these trapped insects.There was a close correlation(r=0.92)between our automatic and manual identification methods based on the daily pest number of one-year images from one light trap.Field experiments demonstrated the effectiveness and accuracy of our automatic light trap monitoring system.
基金supported in part by the USDA’s Hatch and Multistate Project Funds(Accession Nos.1005756 and 1001246)。
文摘Estimation of fruit size in tree fruit crops is essential for selective robotic harvesting and crop-load estimation.Machine vision systems for fruit detection and localization have been studied widely for robotic harvesting and crop-load estimation.However,only a few studies have been carried out to estimate fruit size in orchards using machine vision systems.This study was carried out to develop a machine vision system consisting of a color CCD camera and a time-of-flight(TOF)light-based 3D camera for estimating apple size in tree canopies.As a measure of fruit size,the major axis(longest axis)was estimated based on(i)the 3D coordinates of pixels on corresponding apple surfaces,and(ii)the 2D size of individual pixels within apple surfaces.In the 3D coordinates-based method,the distance between pairs of pixels within apple regions were calculated using 3D coordinates,and the maximum distance between all pixel pairs within an apple region was estimated to be the major axis.The accuracy of estimating the major axis using 3D coordinates was 69.1%.In the pixel-size-based method,the physical sizes of pixels were estimated using a calibration model developed based on pixel coordinates and the distance to pixels from the camera.The major axis length was then estimated by summing the size of individual pixels along the major axis of the fruit.The accuracy of size estimation increased to 84.8%when the pixel size-based method was used.The results showed the potential for estimating fruit size in outdoor environments using a 3D machine vision system.
文摘Text detection and recognition is a hot topic in computer vision,which is considered to be the further development of the traditional optical character recognition(OCR)technology.With the rapid development of machine vision system and the wide application of deep learning algorithms,text recognition has achieved excellent performance.In contrast,detecting text block from complex natural scenes is still a challenging task.At present,many advanced natural scene text detection algorithms have been proposed,but most of them run slow due to the complexity of the detection pipeline and can not be applied to industrial scenes.In this paper,we proposed a CCD based machine vision system for realtime text detection in invoice images.In this system,we applied optimizations from several aspects including the optical system,the hardware architecture,and the deep learning algorithm to improve the speed performance of the machine vision system.The experimental data confirms that the optimization methods can significantly improve the running speed of the machine vision system and make it meeting the real-time text detection requirements in industrial scenarios.
文摘In manufacture of precise optical products, it is important to inspect and classify the potential defects existing on the products’ surfaces after precise machining in order to obtain high quality in both functionality and aesthetics. The existing methods for detecting and classifying defects all are low accuracy or efficiency or high cost in inspection process. In this paper, a new inspection system based on machine vision has been introduced, which uses automatic focusing and image mosaic technologies to rapidly acquire distinct surface image, and employs Case-Based Reasoning(CBR)method in defects classification. A modificatory fuzzy similarity algorithm in CBR has been adopted for more quick and robust need of pattern recognition in practice inspection. Experiments show that the system can inspect surface diameter of 500mm in half an hour with resolving power of 0.8μm diameter according to digs or 0.5μm transverse width according to scratches. The proposed inspection principles and methods not only have meet manufacturing requirements of precise optical products, but also have great potential applications in other fields of precise surface inspection.
基金National Key Research and Development Program of China(No.2017YFB1304001)。
文摘During the sizing process,yarn congestion fault occurs at the reed teeth of a sizing machine.At present,the yarn congestion fault is generally handled by manual detection.The sizing production line operates on a large scale and runs continuously.Untimely handling of the yarn congestion fault causes a large amount of yarn waste.In this research,a machine vision-based algorithm for yarn congestion fault detection is developed.Through the analysis of the congestion fault and interference contour characteristics,the basic idea of image phase subtraction to identify the congestion fault is determined.To address the interference information appearing after image phase subtraction,the image pre-processing methods of Canny edge extraction and mean filtering are employed.According to the fault size and location characteristics,the fault contour detection algorithm based on inter-frame difference is designed.To mitigate the camera vibration interference,the anti-vibration interference algorithm based on affine transformation is studied,and the fault detection algorithm for the total yarn congestion fault is determined.The detection of 20 sets of field data is carried out,and the detection rate reaches 90%.This fault detection algorithm realizes the automatic detection of yarn congestion fault of sizing machine with certain real-time performance and accuracy.
基金funded by the National Natural Science Foundation of China(81573542)Shanxi Province's Traditional Chinese Medicine Technology Innovation Project(2100601).
文摘Objective:To explore a rapid and accurate method for evaluating the quality of Prunus mandshurica(Maxim.)Koehne(P.mandshurica,Ku Xing Ren)during rancidity using machine vision and learning.Methods:Sensory evaluation and chemometrics were used to classify P.mandshurica quality grades after rancidity.Chemical indicators of the P.mandshurica quality change were determined to verify the ob-tained grades and support the subsequent modeling.The International Commission on Illumination color space was used to extract the color features of the P.mandshurica.Discrimination and prediction models based on color features combined with multiple machine learning algorithms were established using 10-fold cross-validation and external test set validation.Results:The P.mandshurica rancidity samples were allocated to three quality grades.The Bayes net model based on powder color successfully identified the P.mandshurica at different grades with an accuracy of 88.89%and 100%using two validations,and the naive Bayes model based on section color achieved the same accuracy with an receiver operating characteristic area of 0.979.The instance-based k-nearest neighbors model based on powder color performed best in predicting the amygdalin content[R^(2)=0.9801,mean absolute error(MAE)=0.2071,root mean squared error(RMSE)=0.4170],followed by the random com-mittee model in predicting the acid value(R^(2)=0.9580,MAE=1.5121,RMSE=1.9099)and the random forest model in predicting the peroxide value(R^(2)=0.8857,MAE=0.0027,RMSE=0.0035).Conclusion:This study demonstrates that color digitization analysis is a potential method for rapidly evaluating the quality of P.mandshurica across the rancidity process,providing a new reference for the quality assessment of traditional Chinese medicines.
基金supported by grants from the Research Grants Council of the Hong Kong Special Administrative Region,China(Project No.CRF No.PolyU C5110-20G)PolyU Grants(1-CE0H,1-W30M,1-CD4S).
文摘Realizing the point-of-care tumor markers biodetection with good convenience and high sensitivity possesses great significance for prompting cancer monitoring and screening in biomedical study field.Herein,the quantum dots luminescence and microfluidic biochip with machine vision algorithm-based intelligent biosensing platform have been designed and manufactured for point-of-care tumor markers diagnostics.The employed quantum dots with excellent photoluminescent performance are modified with specific antibody as the optical labeling agents for the designed sandwich structure immunoassay.The corresponding biosensing investigations of the designed biodetection platform illustrate several advantages involving high sensitivity(~0.021 ng mL^(−1)),outstanding accessibility,and great integrability.Moreover,related test results of human-sourced artificial saliva samples demonstrate better detection capabilities compared with commercially utilized rapid test strips.Combining these infusive abilities,our elaborate biosensing platform is expected to exhibit potential applications for the future point-of-care tumor markers diagnostic area.
基金the financial support for this research provided by the National Natural Science Foundation of China(Grant Nos.52275470,124115301,and 52105458)the Natural Science Foundation of Beijing(Grant No.3222009).
文摘Vibration cutting has emerged as a promising method for creating surface functional microstructures.However,achieving precise tool setting is a time-consuming process that significantly impacts process efficiency.This study proposes an intelligent approach for tool setting in vibration cutting using machine vision and hearing,divided into two steps.In the first step,machine vision is employed to achieve rough precision in tool setting within tens of micrometers.Subsequently,in the second step,machine hearing utilizes sound pickup to capture vibration audio signals,enabling fine tool adjustment within 1μm precision.The relationship between the spectral intensity of vibration audio and cutting depth is analyzed to establish criteria for tool–workpiece contact.Finally,the efficacy of this approach is validated on an ultra-precision platform,demonstrating that the automated tool-setting process takes no more than 74 s.The total cost of the vision and hearing sensors is less than$1500.
基金Supported by National Key Research and Development Program of China(Grant No.2018YFA0707300)Hebei Natural Science Foundation(Grant No.E2023203260)+1 种基金Scientific and Technological Research Projects of Hebei Higher Education Institutions(Grant No.CXY2024009)Central Government Guide Local Science and Technology Development Fund Project(Grant No.216Z1602G)。
文摘The wear of work rolls significantly affects the production efficiency and product quality.However,existing methods for wear assessment fail to effectively quantify work roll surface wear conditions,thereby affecting the quality control of steel strips and maintenance strategies for rolls.To accurately assess the wear conditions of hot-rolling work rolls,this study initially established an apparatus for capturing high-precision roll surfaces images.Subsequently,a quantitative assessment of common surface wear morphologies was conducted,and a hot-rolling work roll surface wear dataset was constructed.The MobileNetV2 convolutional neural network(CNN),augmented by transfer learning,was employed to develop a MobileNetV2-wear detection and classification(WDC)surface wear grading model.A comparison with mainstream CNN models revealed that the MobileNetV2-WDC model achieved high-speed(21.92 ms)and accurate(96.86%)grading with minimal model parameters(2.27 M)and size(27 M),meeting the industrial efficiency and practicality requirements.A visual analysis of the model classification errors was conducted,outlining paths for further optimization.This study provides an efficient and accurate solution for detecting and grading surface wear on hot-rolling work rolls,enhancing product quality and extending the lifespan of rolls.
基金supported by Earthquake Sciences Spark Programs of China Earthquake Administration(No.XH22020YA)Science Innovation Fund granted by the First Monitoring and Application Center of China Earthquake Administration(No.FMC202309).
文摘An automatic monitoring method of the 3-D deformation is presented for crustal fault based on laser and machine vision. The laser source and screen are independently set up in the headwall and footwall, the collimated laser beam creates a circular spot on the screen, meanwhile, the industrial camera captures the tiny deformation of the crustal fault by monitoring the change of the spot position. This method significantly reduces the cost of equipment and labor, provides daily sampling to ensure high continuity of data. A prototype of the automatic monitoring system is developed, and a repeatability test indicates that the error of spot jitter can be minimized by consecutive samples. Meanwhile, the environmental correction model is determined to ensure that environmental changes do not disturb the system. Furthermore, the automatic monitoring system has been applied at the deformation monitoring station(KJX02) of China Beishan underground research laboratory, where continuous deformation monitoring is underway.
文摘In this paper, an automatic inspection system for weld surface appearance using machine vision has been developed to recognize weld surface defects such as porosities, cracks, etc. It can replace conventional manual visual inspection method, which is tedious, time-consuming, subjective, experience-depended, and sometimes biased. The system consists of a CCD camera, a self-designed annular light source, a sensor controller, a frame grabbing card, a computer and so on. After acquiring weld surface appearance images using CCD, the images are preprocessed using median filtering and a series of image enhancement algorithms. Then a dynamic threshold and morphology algorithms are applied to segment defect object. Finally, defect features information is obtained by eight neighborhoods boundary chain code algorithm. Experimental results show that the developed system is capable of inspecting most surface defects such as porosities, cracks with high reliability and accuracy.
基金The National Natural Science Foundation of China(No.51175267)the Natural Science Foundation of Jiangsu Province(No.BK2010481)+2 种基金the Ph.D.Programs Foundation of Ministry of Education of China(No.20113219120004)China Postdoctoral Science Foundation(No.20100481148)the Postdoctoral Science Foundation of Jiangsu Province(No.1001004B)
文摘To realize high-precision automatic measurement of two-dimensional geometric features on parts, a cooperative measurement system based on machine vision is constructed. Its hardware structure, functional composition and working principle are introduced. The mapping relationship between the feature image coordinates and the measuring space coordinates is established. The method of measuring path planning of small field of view （FOV） images is proposed. With the cooperation of the panoramic image of the object to be measured, the small FOV images with high object plane resolution are acquired automatically. Then, the auxiliary measuring characteristics are constructed and the parameters of the features to be measured are automatically extracted. Experimental results show that the absolute value of relative error is less than 0. 03% when applying the cooperative measurement system to gauge the hole distance of 100 mm nominal size. When the object plane resolving power of the small FOV images is 16 times that of the large FOV image, the measurement accuracy of small FOV images is improved by 14 times compared with the large FOV image. It is suitable for high-precision automatic measurement of two-dimensional complex geometric features distributed on large scale parts.
