Biological visions have inspired the development of artificial vision systems with diverse visual functional traits,however,the detected wavelength is only in visible light between 0.4 and 0.78μm,restricting their ap...Biological visions have inspired the development of artificial vision systems with diverse visual functional traits,however,the detected wavelength is only in visible light between 0.4 and 0.78μm,restricting their applications.Snakes generate a thermal image of animals due to pit organs for detecting and converting infrared,allowing them to accurately target predators or prey even under darkness.Inspired by natural infrared visualized snakes,we propose artificial vision systems with CMOS sensors-integrated upconverters to break visible light limitations to realize 3840×2160 ultra-high-resolution short-wave infrared(SWIR)and mid-wave infrared(MWIR)visualization imaging for the first time.Through colloidal quantum dot barrier heterojunction architecture design of infrared detecting units and the introduction of co-hosted emitting units,the luminance and upconversion efficiency reach up to 6388.09 cd m^(−2) and 6.41%for SWIR,1311.64 cd m^(−2) and 4.06%for MWIR at room temperature.Our artificial vision systems broaden a wide spectrum of applications within infrared,such as night vision,agricultural science,and industry inspection,marking a significant advance in bioartificial vision.展开更多
Artificial visual sensors(AVSs)with bio-inspired sensing and neuromorphic signal processing are essential for next-generation intelligent systems.Conventional optoelectronic devices employed in AVSs operate discretely...Artificial visual sensors(AVSs)with bio-inspired sensing and neuromorphic signal processing are essential for next-generation intelligent systems.Conventional optoelectronic devices employed in AVSs operate discretely in terms of sensing,processing,and memorization,and not ideal for applications necessitating shape deformation to achieve wide fields-of-view and deep depths-of-field.Here,we present stretchable artificial visual sensors(S-AVS)capable of concurrently sensing and processing optical signals while adapting to shape deformations.Specifically,these S-AVSs use a stretchable transistor structure with a meticulously engineered photosensitive semiconductor layer,comprising an organic semiconductor,thermoplastic elastomer,and cesium lead bromide quantum dots(CsPbBr_(3) QDs).They exhibit synaptic behaviors such as excitatory postsynaptic current(EPSC)and paired-pulse facilitation(PPF)under optical signals,maintaining functionality under 30%strain and repeated stretching.The nonlinear response and fading memory effect support in-sensor reservoir computing,achieving image recognition accuracies of 97.46%and 97.1%at 0%and 30%strain,respectively.展开更多
Van der Waals(vdW)ferroelectric-semiconductor heterojunction provides reconfigurable band alignment based on optical/electrical-assisted polarization switching,which shows great potential to construct artificial visua...Van der Waals(vdW)ferroelectric-semiconductor heterojunction provides reconfigurable band alignment based on optical/electrical-assisted polarization switching,which shows great potential to construct artificial visual neural systems.However,the mechanical exfoliation fabrication scheme for proof-of-concept demonstrations and fundamental studies is cumbersome and not scalable for practical application.Here,we present a synthetic strategy for the large-scale and high crystallinity growth of planar/verticalα-In_(2)Se_(3)/MoS_(2)heterojunctions by dynamically tuning the growth temperature.Furthermore,based on theα-In_(2)Se_(3)/MoS_(2)heterostructures,photo-synapse devices are designed and fabricated to simulate visual neural systems functions,including multistate storage,optical logic operation,potentiation and depression,paired-pulse facilitation(PPF),short-term memory(STM),long-term memory(LTM),and Learning-Forgetting-Relearning.By coupling the spatiotemporally relevant optical and electric information,the device can mimic the superior biological visual system’s light adaptation and Pavlovian conditioning.This work provides a strategy for dynamically tuning the orientation of ferroelectric-semiconductor heterojunction stacks and will give impetus to applying all-in-one sensing and memory-computing artificial vision systems.展开更多
Recently,for developing neuromorphic visual systems,adaptive optoelectronic devices become one of the main research directions and attract extensive focus to achieve optoelectronic transistors with high performances a...Recently,for developing neuromorphic visual systems,adaptive optoelectronic devices become one of the main research directions and attract extensive focus to achieve optoelectronic transistors with high performances and flexible func-tionalities.In this review,based on a description of the biological adaptive functions that are favorable for dynamically perceiv-ing,filtering,and processing information in the varying environment,we summarize the representative strategies for achiev-ing these adaptabilities in optoelectronic transistors,including the adaptation for detecting information,adaptive synaptic weight change,and history-dependent plasticity.Moreover,the key points of the corresponding strategies are comprehen-sively discussed.And the applications of these adaptive optoelectronic transistors,including the adaptive color detection,sig-nal filtering,extending the response range of light intensity,and improve learning efficiency,are also illustrated separately.Lastly,the challenges faced in developing adaptive optoelectronic transistor for artificial vision system are discussed.The descrip-tion of biological adaptive functions and the corresponding inspired neuromorphic devices are expected to provide insights for the design and application of next-generation artificial visual systems.展开更多
Human action recognition(HAR)is crucial for the development of efficient computer vision,where bioinspired neuromorphic perception visual systems have emerged as a vital solution to address transmission bottlenecks ac...Human action recognition(HAR)is crucial for the development of efficient computer vision,where bioinspired neuromorphic perception visual systems have emerged as a vital solution to address transmission bottlenecks across sensor-processor interfaces.However,the absence of interactions among versatile biomimicking functionalities within a single device,which was developed for specific vision tasks,restricts the computational capacity,practicality,and scalability of in-sensor vision computing.Here,we propose a bioinspired vision sensor composed of a Ga N/Al N-based ultrathin quantum-disks-in-nanowires(QD-NWs)array to mimic not only Parvo cells for high-contrast vision and Magno cells for dynamic vision in the human retina but also the synergistic activity between the two cells for in-sensor vision computing.By simply tuning the applied bias voltage on each QD-NW-array-based pixel,we achieve two biosimilar photoresponse characteristics with slow and fast reactions to light stimuli that enhance the in-sensor image quality and HAR efficiency,respectively.Strikingly,the interplay and synergistic interaction of the two photoresponse modes within a single device markedly increased the HAR recognition accuracy from 51.4%to 81.4%owing to the integrated artificial vision system.The demonstration of an intelligent vision sensor offers a promising device platform for the development of highly efficient HAR systems and future smart optoelectronics.展开更多
Inspired by the snake pit organ’s remarkable ability to perceive mid-wave infrared(MWIR)radiation,researchers have developed a biomimetic artificial vision system that integrates infrared-to-visible upconverters with...Inspired by the snake pit organ’s remarkable ability to perceive mid-wave infrared(MWIR)radiation,researchers have developed a biomimetic artificial vision system that integrates infrared-to-visible upconverters with CMOS sensors.Operating at room temperature,this platform enables direct visualization of both short-wave infrared(SWIR)and MWIR,marking a pioneering demonstration of broadband infrared imaging with high resolution.Such a breakthrough paves the way for low-cost and flexible applications in night vision,agricultural monitoring,industrial inspection,and beyond.展开更多
The emergence of the Internet-of-Things is anticipated to create a vast market for what are known as smart edge devices,opening numerous opportunities across countless domains,including personalized healthcare and adv...The emergence of the Internet-of-Things is anticipated to create a vast market for what are known as smart edge devices,opening numerous opportunities across countless domains,including personalized healthcare and advanced robotics.Leveraging 3D integration,edge devices can achieve unprecedented miniaturization while simultaneously boosting processing power and minimizing energy consumption.Here,we demonstrate a back-end-of-line compatible optoelectronic synapse with a transfer learning method on health care applications,including electroencephalogram(EEG)-based seizure prediction,electromyography(EMG)-based gesture recognition,and electrocardiogram(ECG)-based arrhythmia detection.With experiments on three biomedical datasets,we observe the classification accuracy improvement for the pretrained model with 2.93%on EEG,4.90%on ECG,and 7.92%on EMG,respectively.The optical programming property of the device enables an ultralow power(2.8×10^(-13) J)fine-tuning process and offers solutions for patient-specific issues in edge computing scenarios.Moreover,the device exhibits impressive light-sensitive characteristics that enable a range of light-triggered synaptic functions,making it promising for neuromorphic vision application.To display the benefits of these intricate synaptic properties,a 5×5 optoelectronic synapse array is developed,effectively simulating human visual perception and memory functions.The proposed flexible optoelectronic synapse holds immense potential for advancing the fields of neuromorphic physiological signal processing and artificial visual systems in wearable applications.展开更多
The traditional von Neumann architecture faces inherent limitations due to the separation of memory and computa-tion,leading to high energy consumption,significant latency,and reduced operational efficiency.Neuromorph...The traditional von Neumann architecture faces inherent limitations due to the separation of memory and computa-tion,leading to high energy consumption,significant latency,and reduced operational efficiency.Neuromorphic computing,inspired by the architecture of the human brain,offers a promising alternative by integrating memory and computational func-tions,enabling parallel,high-speed,and energy-efficient information processing.Among various neuromorphic technologies,ion-modulated optoelectronic devices have garnered attention due to their excellent ionic tunability and the availability of multi-dimensional control strategies.This review provides a comprehensive overview of recent progress in ion-modulation optoelec-tronic neuromorphic devices.It elucidates the key mechanisms underlying ionic modulation of light fields,including ion migra-tion dynamics and capture and release of charge through ions.Furthermore,the synthesis of active materials and the proper-ties of these devices are analyzed in detail.The review also highlights the application of ion-modulation optoelectronic devices in artificial vision systems,neuromorphic computing,and other bionic fields.Finally,the existing challenges and future direc-tions for the development of optoelectronic neuromorphic devices are discussed,providing critical insights for advancing this promising field.展开更多
Retinal degenerative diseases may induce the degeneration of outer retina and in turn,blindness.Nevertheless,due to the maintenance of inner retina,the coding and processing of visual neurons responses are still able ...Retinal degenerative diseases may induce the degeneration of outer retina and in turn,blindness.Nevertheless,due to the maintenance of inner retina,the coding and processing of visual neurons responses are still able to be executed naturally.Therefore,an effective retinal prosthesis device may be developed by mimicking the function of outer retina:transferring the visual light into artificial stimulus and delivering the stimulus to the retina aiming to evoke the neural responses.As two main developing directions for current retinal prosthesis,epiretinal(ER)and subretinal(SR)prosthesis are both undergoing experimental stage and possessing advantages and limitations.Further investigations in power supply,biocompatibility,etc.are still required.Additionally,suprachoroidal transretinal stimulation(STS)and neurotransmitter-induced stimulation as some other alternatives in retinal prosthesis are also considered as promising research directions,although they are not mature enough to be applied commercially,either.展开更多
Artificial visual systems can recognize desired objects and information from complex environments, and are therefore highly desired for pattern recognition, object detection, and imaging applications. However, state-o...Artificial visual systems can recognize desired objects and information from complex environments, and are therefore highly desired for pattern recognition, object detection, and imaging applications. However, state-of-the-art artificial visual systems with high recognition performances that typically consist of electronic devices face the challenges of requiring huge storage space and high power consumption owing to redundant data. Here, we report a terahertz(THz) frequency-selective surface using a graphene split-ring resonator driven by ferroelectric polarization for efficient visual system applications. The downward polarization of the ferroelectric material offers an ultrahigh electrostatic field for doping p-type graphene with an anticipated Fermi level. By optimizing the geometric parameters of the devices and modulating the carrier behaviors of graphene, our plasmonic devices exhibit a tunable spectral response in a range of 1.7–6.0 THz with continuous transmission values. The alloptical neural network using graphene plasmonic surfaces designed in this study exhibited excellent performance in visual preprocessing and convolutional filtering and achieved an ultrahigh recognition accuracy of up to 99.3% in training the Modified National Institute of Standards and Technology(MNIST) handwritten digit dataset. These features demonstrate the great potential of graphene plasmonic devices for future smart artificial vision systems.展开更多
Photonic synapses combining photosensitivity and synaptic function can efficiently perceive and memorize visual information,making them crucial for the development of artificial vision systems.However,the development ...Photonic synapses combining photosensitivity and synaptic function can efficiently perceive and memorize visual information,making them crucial for the development of artificial vision systems.However,the development of high-performance photonic synapses with low power consumption and rapid optical erasing ability remains challenging.Here,we propose a photon-modulated charging/discharging mechanism for self-powered photonic synapses.The current hysteresis enables the devices based on CsPbBr3/solvent/carbon nitride multilayer architecture to emulate synaptic behaviors,such as excitatory postsynaptic currents,paired-pulse facilitation,and long/short-term memory.Intriguingly,the unique radiation direction-dependent photocurrent endows the photonic synapses with the capability of optical writing and rapid optical erasing.Moreover,the photonic synapses exhibit exceptional performance in contrast enhancement and noise reduction owing to the notable synaptic plasticity.In simulations based on artificial neural network(ANN)algorithms,the pre-processing by our photonic synapses improves the recognition rate of handwritten digit from 11.4%(200 training epochs)to 85%(~60 training epochs).Furthermore,due to the excellent feature extraction and memory capability,an array based on the photonic synapses can imitate facial recognition of human retina without the assistance of ANN.展开更多
The artificial intelligence era has witnessed a surge of demand in detection and recognition of biometric information,with applications from financial services to information security.However,the physical separation o...The artificial intelligence era has witnessed a surge of demand in detection and recognition of biometric information,with applications from financial services to information security.However,the physical separation of sensing,memory,and computational units in traditional biometric systems introduces severe decision latency and operational power consumption.Herein,an in-sensor reservoir computing(RC)system based on MoTe_(2)/BaTiO_(3)optical synapses is proposed to detect and recognize the faces and fingerprints information.In optical operation mode,the device exhibits low energy consumption of 41.2 pJ,long retention time of 3×10^(4)s,high endurance of 10^(4)switching cycles,and multifunctional sensing-memory-computing visual simulations.The light intensity-dependent optical sensing and multilevel optical storage properties are exploited to achieve sunburned eye simulation and image memory functions.These nonlinear,multi-state,short-term storage,and long-term memory characteristics make MoTe_(2)/BaTiO_(3)optical synapses a suitable reservoir layer and readout layer,with short-term properties to project complicated input features into high-dimensional output features,and long-term properties to be used as a readout layer,thus further building an in-sensor RC system for face and fingerprint recognition.Under the 40%Gaussian noise environment,the system achieves 91.73%recognition accuracy for face and 97.50%for fingerprint images,and experimental verification is carried out,which shows potential in practical applications.These results provide a strategy for constructing a high-performance in-sensor RC system for high-accuracy biometric identification.展开更多
Biomimetic stimulation of the retina with neurotransmitters,the natural agents of communication at chemical synapses,could be more effective than electrical stimulation for treating blindness from photoreceptor degene...Biomimetic stimulation of the retina with neurotransmitters,the natural agents of communication at chemical synapses,could be more effective than electrical stimulation for treating blindness from photoreceptor degenerative diseases.Recent studies have demonstrated the feasibility of neurotransmitter stimulation by injecting glutamate,a primary retinal neurotransmitter,into the retina at isolated single sites.Here,we demonstrate spatially patterned multisite stimulation of the retina with glutamate,offering the first experimental evidence for applicability of this strategy for translating visual patterns into afferent neural signals.To accomplish pattern stimulation,we fabricated a special microfluidic device comprising an array of independently addressable microports connected to tiny on-chip glutamate reservoirs via microchannels.The device prefilled with glutamate was interfaced with explanted rat retinas placed over a multielectrode array(MEA)with the retinal ganglion cells(RGC)contacting the electrodes and photoreceptor surface contacting the microports.By independently and simultaneously activating a subset of the microports with modulated pressure pulses,small boluses of glutamate were convectively injected at multiple sites in alphabet patterns over the photoreceptor surface.We found that the glutamate-driven RGC responses recorded through the MEA system were robust and spatially laid out in patterns strongly resembling the injection patterns.The stimulations were also highly localized with spatial resolutions comparable to or better than electrical retinal prostheses.Our findings suggest that surface stimulation of the retina with neurotransmitters in pixelated patterns of visual images is feasible and an artificial chemical synapse chip based on this approach could potentially circumvent the limitations of electrical retinal prostheses.展开更多
Halide perovskites are considered as promising memristive materials for nextgeneration optoelectronic devices.This review concisely summarizes the recent development of halide perovskite memristors and highlights thei...Halide perovskites are considered as promising memristive materials for nextgeneration optoelectronic devices.This review concisely summarizes the recent development of halide perovskite memristors and highlights their advancements in optoelectronic applications:light‐induced low power switches,optoelectronic logic operations,optoelectronic neuromorphic computation,and artificial vision systems.Finally,we address the challenges and future development prospects of halide perovskites‐based memristors.This review highlights the promising potential of halide perovskite materials for future optoelectronic memory and computing applications.展开更多
Through the supply chain,the quality or quality change of the products can generate important losses.The quality control in some steps is made manually that supposes a high level of subjectivity,controlling the qualit...Through the supply chain,the quality or quality change of the products can generate important losses.The quality control in some steps is made manually that supposes a high level of subjectivity,controlling the quality and its evolution using automatic systems can suppose a reduction of the losses.Testing some automatic image analysis techniques in the case of tomatoes and zucchini is the main objective of this study.Two steps in the supply chain are considered,the feeding of the raw products into the handling chain(because low quality generates a reduction of the chain productivity)and the cool storage of the processed products(as the value at the market is reduced).It was proposed to analyze the incoming products at the head the processing line using CCD cameras to detect low quality and/or dirty products(corresponding to specific farmers/suppliers,it should be asked to improve to maintain the productivity of the line).The second stage is analyzing the evolution of the products along the cool chain(storage and transport),the use of an App developed to be use under Android was proposed to substitute the“visual”evaluation used in practice.The algorithms used,including stages of pre-treatment,segmentation,analysis and presentation of the results take account of the short time available and the limited capacity of the batteries.High performance techniques were applied to the homography stage to discard some of the images,resulting in better performance.Also threads and renderscript kernels were created to parallelize the methods used on the resulting images being able to inspect faster the products.The proposed method achieves success rates comparable to,and improving,the expert inspection.展开更多
Two-dimensional metal chalcogenides have garnered significant attention as promising candidates for novel neuromorphic synaptic devices due to their exceptional structural and optoelectronic properties.However,achievi...Two-dimensional metal chalcogenides have garnered significant attention as promising candidates for novel neuromorphic synaptic devices due to their exceptional structural and optoelectronic properties.However,achieving large-scale integration and practical applications of synaptic chips has proven to be challenging due to significant hurdles in materials preparation and the absence of effective nanofabrication techniques.In a recent breakthrough,we introduced a revolutionary allopatric defect-modulated Fe_(7)S_(8)@MoS_(2)synaptic heterostructure,which demonstrated remarkable optoelectronic synaptic response capabilities.Building upon this achievement,our current study takes a step further by presenting a sulfurization-seeding synergetic growth strategy,enabling the large-scale and arrayed preparation of Fe_(7)S_(8)@MoS_(2)heterostructures.Moreover,a three-dimensional vertical integration technique was developed for the fabrication of arrayed optoelectronic synaptic chips.Notably,we have successfully simulated the visual persistence function of the human eye with the adoption of the arrayed chip.Our synaptic devices exhibit a remarkable ability to replicate the preprocessing functions of the human visual system,resulting in significantly improved noise reduction and image recognition efficiency.This study might mark an important milestone in advancing the field of optoelectronic synaptic devices,which significantly prompts the development of mature integrated visual perception chips.展开更多
In recent years,smart agriculture has gained strength due to the application of industry 4.0 technologies in agriculture.As a result,efforts are increasing in proposing artificial vision applications to solvemany prob...In recent years,smart agriculture has gained strength due to the application of industry 4.0 technologies in agriculture.As a result,efforts are increasing in proposing artificial vision applications to solvemany problems.However,many of these applications are developed separately.Many academic works have proposed solutions integrating image classification techniques through IoT platforms.For this reason,this paper aims to answer the following research questions:(1)What are themain problems to be solvedwith smart farming IoT platforms that incorporate images?(2)What are the main strategies for incorporating image classification methods in smart agriculture IoT platforms?and(3)What are the main image acquisition,preprocessing,transmission,and classification technologies used in smart agriculture IoT platforms?This study adopts a Systematic Literature Review(SLR)approach.We searched Scopus,Web of Science,IEEE Xplore,and Springer Link databases from January 2018 to July 2022.Fromwhich we could identify five domains corresponding to(1)disease and pest detection,(2)crop growth and health monitoring,(3)irrigation and crop protectionmanagement,(4)intrusion detection,and(5)fruits and plant counting.There are three types of strategies to integrate image data into smart agriculture IoT platforms:(1)classification process in the edge,(2)classification process in the cloud,and(3)classification process combined.The main advantage of the first is obtaining data in real-time,and its main disadvantage is the cost of implementation.On the other hand,the main advantage of the second is the ability to process high-resolution images,and its main disadvantage is the need for high-bandwidth connectivity.Finally,themixed strategy can significantly benefit infrastructure investment,butmostworks are experimental.展开更多
基金National Key R&D Program of China(2021YFA0717600)National Natural Science Foundation of China(NSFC No.62305022,NSFC No.62035004,NSFC No.U22A2081)Young Elite Scientists Sponsorship Program by CAST(No.YESS20200163).
文摘Biological visions have inspired the development of artificial vision systems with diverse visual functional traits,however,the detected wavelength is only in visible light between 0.4 and 0.78μm,restricting their applications.Snakes generate a thermal image of animals due to pit organs for detecting and converting infrared,allowing them to accurately target predators or prey even under darkness.Inspired by natural infrared visualized snakes,we propose artificial vision systems with CMOS sensors-integrated upconverters to break visible light limitations to realize 3840×2160 ultra-high-resolution short-wave infrared(SWIR)and mid-wave infrared(MWIR)visualization imaging for the first time.Through colloidal quantum dot barrier heterojunction architecture design of infrared detecting units and the introduction of co-hosted emitting units,the luminance and upconversion efficiency reach up to 6388.09 cd m^(−2) and 6.41%for SWIR,1311.64 cd m^(−2) and 4.06%for MWIR at room temperature.Our artificial vision systems broaden a wide spectrum of applications within infrared,such as night vision,agricultural science,and industry inspection,marking a significant advance in bioartificial vision.
基金supported by the the Innovation Program of Shanghai Municipal Education Commission(No.2021-01-07-00-07-E00096)the National Natural Science Foundation of China(Nos.62074111 and 62374115)the National Key Research and Development Program of China(No.2022YFB3203502).
文摘Artificial visual sensors(AVSs)with bio-inspired sensing and neuromorphic signal processing are essential for next-generation intelligent systems.Conventional optoelectronic devices employed in AVSs operate discretely in terms of sensing,processing,and memorization,and not ideal for applications necessitating shape deformation to achieve wide fields-of-view and deep depths-of-field.Here,we present stretchable artificial visual sensors(S-AVS)capable of concurrently sensing and processing optical signals while adapting to shape deformations.Specifically,these S-AVSs use a stretchable transistor structure with a meticulously engineered photosensitive semiconductor layer,comprising an organic semiconductor,thermoplastic elastomer,and cesium lead bromide quantum dots(CsPbBr_(3) QDs).They exhibit synaptic behaviors such as excitatory postsynaptic current(EPSC)and paired-pulse facilitation(PPF)under optical signals,maintaining functionality under 30%strain and repeated stretching.The nonlinear response and fading memory effect support in-sensor reservoir computing,achieving image recognition accuracies of 97.46%and 97.1%at 0%and 30%strain,respectively.
基金supported by the National Natural Science Foundation of China(Nos.52371245,12174237,12241403)the National Key Research and Development Program of China(No.2022YFB3505301).
文摘Van der Waals(vdW)ferroelectric-semiconductor heterojunction provides reconfigurable band alignment based on optical/electrical-assisted polarization switching,which shows great potential to construct artificial visual neural systems.However,the mechanical exfoliation fabrication scheme for proof-of-concept demonstrations and fundamental studies is cumbersome and not scalable for practical application.Here,we present a synthetic strategy for the large-scale and high crystallinity growth of planar/verticalα-In_(2)Se_(3)/MoS_(2)heterojunctions by dynamically tuning the growth temperature.Furthermore,based on theα-In_(2)Se_(3)/MoS_(2)heterostructures,photo-synapse devices are designed and fabricated to simulate visual neural systems functions,including multistate storage,optical logic operation,potentiation and depression,paired-pulse facilitation(PPF),short-term memory(STM),long-term memory(LTM),and Learning-Forgetting-Relearning.By coupling the spatiotemporally relevant optical and electric information,the device can mimic the superior biological visual system’s light adaptation and Pavlovian conditioning.This work provides a strategy for dynamically tuning the orientation of ferroelectric-semiconductor heterojunction stacks and will give impetus to applying all-in-one sensing and memory-computing artificial vision systems.
基金the National Key Research and Development Program of China(2021YFA0717900)National Natural Science Foundation of China(62471251,62405144,62288102,22275098,and 62174089)+1 种基金Basic Research Program of Jiangsu(BK20240033,BK20243057)Jiangsu Funding Program for Excellent Postdoctoral Talent(2022ZB402).
文摘Recently,for developing neuromorphic visual systems,adaptive optoelectronic devices become one of the main research directions and attract extensive focus to achieve optoelectronic transistors with high performances and flexible func-tionalities.In this review,based on a description of the biological adaptive functions that are favorable for dynamically perceiv-ing,filtering,and processing information in the varying environment,we summarize the representative strategies for achiev-ing these adaptabilities in optoelectronic transistors,including the adaptation for detecting information,adaptive synaptic weight change,and history-dependent plasticity.Moreover,the key points of the corresponding strategies are comprehen-sively discussed.And the applications of these adaptive optoelectronic transistors,including the adaptive color detection,sig-nal filtering,extending the response range of light intensity,and improve learning efficiency,are also illustrated separately.Lastly,the challenges faced in developing adaptive optoelectronic transistor for artificial vision system are discussed.The descrip-tion of biological adaptive functions and the corresponding inspired neuromorphic devices are expected to provide insights for the design and application of next-generation artificial visual systems.
基金funded by the National Natural Science Foundation of China(Grant Nos.62322410,52272168,624B2135,61804047)the Fundamental Research Funds for the Central Universities(No.WK2030000103)。
文摘Human action recognition(HAR)is crucial for the development of efficient computer vision,where bioinspired neuromorphic perception visual systems have emerged as a vital solution to address transmission bottlenecks across sensor-processor interfaces.However,the absence of interactions among versatile biomimicking functionalities within a single device,which was developed for specific vision tasks,restricts the computational capacity,practicality,and scalability of in-sensor vision computing.Here,we propose a bioinspired vision sensor composed of a Ga N/Al N-based ultrathin quantum-disks-in-nanowires(QD-NWs)array to mimic not only Parvo cells for high-contrast vision and Magno cells for dynamic vision in the human retina but also the synergistic activity between the two cells for in-sensor vision computing.By simply tuning the applied bias voltage on each QD-NW-array-based pixel,we achieve two biosimilar photoresponse characteristics with slow and fast reactions to light stimuli that enhance the in-sensor image quality and HAR efficiency,respectively.Strikingly,the interplay and synergistic interaction of the two photoresponse modes within a single device markedly increased the HAR recognition accuracy from 51.4%to 81.4%owing to the integrated artificial vision system.The demonstration of an intelligent vision sensor offers a promising device platform for the development of highly efficient HAR systems and future smart optoelectronics.
文摘Inspired by the snake pit organ’s remarkable ability to perceive mid-wave infrared(MWIR)radiation,researchers have developed a biomimetic artificial vision system that integrates infrared-to-visible upconverters with CMOS sensors.Operating at room temperature,this platform enables direct visualization of both short-wave infrared(SWIR)and MWIR,marking a pioneering demonstration of broadband infrared imaging with high resolution.Such a breakthrough paves the way for low-cost and flexible applications in night vision,agricultural monitoring,industrial inspection,and beyond.
基金financial support by the Semiconductor Initiative at the King Abdullah University of Science and Technologysupported by King Abdullah University of Science and Technology(KAUST)Research Funding(KRF)under Award No.ORA-2022-5314.
文摘The emergence of the Internet-of-Things is anticipated to create a vast market for what are known as smart edge devices,opening numerous opportunities across countless domains,including personalized healthcare and advanced robotics.Leveraging 3D integration,edge devices can achieve unprecedented miniaturization while simultaneously boosting processing power and minimizing energy consumption.Here,we demonstrate a back-end-of-line compatible optoelectronic synapse with a transfer learning method on health care applications,including electroencephalogram(EEG)-based seizure prediction,electromyography(EMG)-based gesture recognition,and electrocardiogram(ECG)-based arrhythmia detection.With experiments on three biomedical datasets,we observe the classification accuracy improvement for the pretrained model with 2.93%on EEG,4.90%on ECG,and 7.92%on EMG,respectively.The optical programming property of the device enables an ultralow power(2.8×10^(-13) J)fine-tuning process and offers solutions for patient-specific issues in edge computing scenarios.Moreover,the device exhibits impressive light-sensitive characteristics that enable a range of light-triggered synaptic functions,making it promising for neuromorphic vision application.To display the benefits of these intricate synaptic properties,a 5×5 optoelectronic synapse array is developed,effectively simulating human visual perception and memory functions.The proposed flexible optoelectronic synapse holds immense potential for advancing the fields of neuromorphic physiological signal processing and artificial visual systems in wearable applications.
基金supported by National Natural Science Foundation of China(62174164,U23A20568,and U22A2075)National Key Research and Development Project(2021YFA1202600)+2 种基金Talent Plan of Shanghai Branch,Chinese Academy of Sciences(CASSHB-QNPD-2023-022)Ningbo Technology Project(2022A-007-C)Ningbo Key Research and Development Project(2023Z021).
文摘The traditional von Neumann architecture faces inherent limitations due to the separation of memory and computa-tion,leading to high energy consumption,significant latency,and reduced operational efficiency.Neuromorphic computing,inspired by the architecture of the human brain,offers a promising alternative by integrating memory and computational func-tions,enabling parallel,high-speed,and energy-efficient information processing.Among various neuromorphic technologies,ion-modulated optoelectronic devices have garnered attention due to their excellent ionic tunability and the availability of multi-dimensional control strategies.This review provides a comprehensive overview of recent progress in ion-modulation optoelec-tronic neuromorphic devices.It elucidates the key mechanisms underlying ionic modulation of light fields,including ion migra-tion dynamics and capture and release of charge through ions.Furthermore,the synthesis of active materials and the proper-ties of these devices are analyzed in detail.The review also highlights the application of ion-modulation optoelectronic devices in artificial vision systems,neuromorphic computing,and other bionic fields.Finally,the existing challenges and future direc-tions for the development of optoelectronic neuromorphic devices are discussed,providing critical insights for advancing this promising field.
文摘Retinal degenerative diseases may induce the degeneration of outer retina and in turn,blindness.Nevertheless,due to the maintenance of inner retina,the coding and processing of visual neurons responses are still able to be executed naturally.Therefore,an effective retinal prosthesis device may be developed by mimicking the function of outer retina:transferring the visual light into artificial stimulus and delivering the stimulus to the retina aiming to evoke the neural responses.As two main developing directions for current retinal prosthesis,epiretinal(ER)and subretinal(SR)prosthesis are both undergoing experimental stage and possessing advantages and limitations.Further investigations in power supply,biocompatibility,etc.are still required.Additionally,suprachoroidal transretinal stimulation(STS)and neurotransmitter-induced stimulation as some other alternatives in retinal prosthesis are also considered as promising research directions,although they are not mature enough to be applied commercially,either.
基金supported by the National Natural Science Foundation of China(Grant No. 62201096)the Engineering Research Center of Digital Imaging and Display, Ministry of Education, Soochow University(Grant No. SDGC2246)the Open Project Program of Shanxi Key Laboratory of Advanced Semiconductor Optoelectronic Devices and Integrated Systems(Grant No. 2023SZKF12)。
文摘Artificial visual systems can recognize desired objects and information from complex environments, and are therefore highly desired for pattern recognition, object detection, and imaging applications. However, state-of-the-art artificial visual systems with high recognition performances that typically consist of electronic devices face the challenges of requiring huge storage space and high power consumption owing to redundant data. Here, we report a terahertz(THz) frequency-selective surface using a graphene split-ring resonator driven by ferroelectric polarization for efficient visual system applications. The downward polarization of the ferroelectric material offers an ultrahigh electrostatic field for doping p-type graphene with an anticipated Fermi level. By optimizing the geometric parameters of the devices and modulating the carrier behaviors of graphene, our plasmonic devices exhibit a tunable spectral response in a range of 1.7–6.0 THz with continuous transmission values. The alloptical neural network using graphene plasmonic surfaces designed in this study exhibited excellent performance in visual preprocessing and convolutional filtering and achieved an ultrahigh recognition accuracy of up to 99.3% in training the Modified National Institute of Standards and Technology(MNIST) handwritten digit dataset. These features demonstrate the great potential of graphene plasmonic devices for future smart artificial vision systems.
基金supported by the Natural Science Foundation of Shandong Province(ZR2021YQ32)the China Postdoctoral Science Foundation(2023M740472)+2 种基金the National Natural Science Foundation of China(62175162,62205214,and 61901222)the Taishan Scholars Program of Shandong Province(tsqn201909117)the Special Fund for Science and Technology Innovation Teams of Shanxi Province and Foundation of Shenzhen Science and Technology(20200814100534001).
文摘Photonic synapses combining photosensitivity and synaptic function can efficiently perceive and memorize visual information,making them crucial for the development of artificial vision systems.However,the development of high-performance photonic synapses with low power consumption and rapid optical erasing ability remains challenging.Here,we propose a photon-modulated charging/discharging mechanism for self-powered photonic synapses.The current hysteresis enables the devices based on CsPbBr3/solvent/carbon nitride multilayer architecture to emulate synaptic behaviors,such as excitatory postsynaptic currents,paired-pulse facilitation,and long/short-term memory.Intriguingly,the unique radiation direction-dependent photocurrent endows the photonic synapses with the capability of optical writing and rapid optical erasing.Moreover,the photonic synapses exhibit exceptional performance in contrast enhancement and noise reduction owing to the notable synaptic plasticity.In simulations based on artificial neural network(ANN)algorithms,the pre-processing by our photonic synapses improves the recognition rate of handwritten digit from 11.4%(200 training epochs)to 85%(~60 training epochs).Furthermore,due to the excellent feature extraction and memory capability,an array based on the photonic synapses can imitate facial recognition of human retina without the assistance of ANN.
基金supported by the National Key R&D Plan“Nano Frontier”Key Special Project(Grant No.2021YFA1200502)Cultivation Projects of National Major R&D Project(Grant No.92164109)+13 种基金the National Natural Science Foundation of China(Grant Nos.61874158,62004056,and 62104058)the Special Project of Strategic Leading Science and Technology of Chinese Academy of Sciences(Grant No.XDB44000000-7)Key R&D Plan Projects in Hebei Province(Grant No.22311101D)Hebei Basic Research Special Key Project(Grant No.F2021201045)the Support Program for the Top Young Talents of Hebei Province(Grant No.70280011807)the Supporting Plan for 100 Excellent Innovative Talents in Colleges and Universities of Hebei Province(Grant No.SLRC2019018)the Interdisciplinary Research Program of Natural Science of Hebei University(No.DXK202101)the Institute of Life Sciences and Green Development(No.521100311)the Natural Science Foundation of Hebei Province(Nos.F2022201054 and F2021201022)the Outstanding Young Scientific Research and Innovation Team of Hebei University(Grant No.605020521001)the Special Support Funds for National High Level Talents(Grant No.041500120001)the Advanced Talents Incubation Program of the Hebei University(Grant Nos.521000981426,521100221071,and 521000981363)the Science and Technology Project of Hebei Education Department(Grant Nos.QN2020178 and QN2021026)Postgraduate's Innovation Fund Project of Hebei Province(CXZZBS2024004).
文摘The artificial intelligence era has witnessed a surge of demand in detection and recognition of biometric information,with applications from financial services to information security.However,the physical separation of sensing,memory,and computational units in traditional biometric systems introduces severe decision latency and operational power consumption.Herein,an in-sensor reservoir computing(RC)system based on MoTe_(2)/BaTiO_(3)optical synapses is proposed to detect and recognize the faces and fingerprints information.In optical operation mode,the device exhibits low energy consumption of 41.2 pJ,long retention time of 3×10^(4)s,high endurance of 10^(4)switching cycles,and multifunctional sensing-memory-computing visual simulations.The light intensity-dependent optical sensing and multilevel optical storage properties are exploited to achieve sunburned eye simulation and image memory functions.These nonlinear,multi-state,short-term storage,and long-term memory characteristics make MoTe_(2)/BaTiO_(3)optical synapses a suitable reservoir layer and readout layer,with short-term properties to project complicated input features into high-dimensional output features,and long-term properties to be used as a readout layer,thus further building an in-sensor RC system for face and fingerprint recognition.Under the 40%Gaussian noise environment,the system achieves 91.73%recognition accuracy for face and 97.50%for fingerprint images,and experimental verification is carried out,which shows potential in practical applications.These results provide a strategy for constructing a high-performance in-sensor RC system for high-accuracy biometric identification.
基金The work presented in the paper was supported by the National Science Foundation,Emerging Frontiers in Research and Innovation(NSF-EFRI)program grant number 0938072.
文摘Biomimetic stimulation of the retina with neurotransmitters,the natural agents of communication at chemical synapses,could be more effective than electrical stimulation for treating blindness from photoreceptor degenerative diseases.Recent studies have demonstrated the feasibility of neurotransmitter stimulation by injecting glutamate,a primary retinal neurotransmitter,into the retina at isolated single sites.Here,we demonstrate spatially patterned multisite stimulation of the retina with glutamate,offering the first experimental evidence for applicability of this strategy for translating visual patterns into afferent neural signals.To accomplish pattern stimulation,we fabricated a special microfluidic device comprising an array of independently addressable microports connected to tiny on-chip glutamate reservoirs via microchannels.The device prefilled with glutamate was interfaced with explanted rat retinas placed over a multielectrode array(MEA)with the retinal ganglion cells(RGC)contacting the electrodes and photoreceptor surface contacting the microports.By independently and simultaneously activating a subset of the microports with modulated pressure pulses,small boluses of glutamate were convectively injected at multiple sites in alphabet patterns over the photoreceptor surface.We found that the glutamate-driven RGC responses recorded through the MEA system were robust and spatially laid out in patterns strongly resembling the injection patterns.The stimulations were also highly localized with spatial resolutions comparable to or better than electrical retinal prostheses.Our findings suggest that surface stimulation of the retina with neurotransmitters in pixelated patterns of visual images is feasible and an artificial chemical synapse chip based on this approach could potentially circumvent the limitations of electrical retinal prostheses.
基金Ministry of Science and Technology of the People's Republic of China,Grant/Award Number:2023YFB4402301the fund from Jilin Province,Grant/Award Number:20240101018JJ+3 种基金111 Project,Grant/Award Number:B13013Fundamental Research Funds for the Central Universities,Grant/Award Number:2412023YQ004NSFC for Distinguished Young Scholars,Grant/Award Number:52025022NSFC Program,Grant/Award Numbers:52272140,U23A20568。
文摘Halide perovskites are considered as promising memristive materials for nextgeneration optoelectronic devices.This review concisely summarizes the recent development of halide perovskite memristors and highlights their advancements in optoelectronic applications:light‐induced low power switches,optoelectronic logic operations,optoelectronic neuromorphic computation,and artificial vision systems.Finally,we address the challenges and future development prospects of halide perovskites‐based memristors.This review highlights the promising potential of halide perovskite materials for future optoelectronic memory and computing applications.
基金funded by the Controlcrop Project,P10-TEP-6174,project framework,supported by the Andalusian Ministry of Economy,Innovation and Science(Andalusia,Spain)the Spanish Ministry of Science and Innovation as well as the EUERDF funds under grant DPI2014-56364-C2-1-R,by TEAP project included in the Marie Curie Actions(PIRSES-GA-2013-612659)by Young Scientists Fund of National Natural Science Foundation of China(31401683).
文摘Through the supply chain,the quality or quality change of the products can generate important losses.The quality control in some steps is made manually that supposes a high level of subjectivity,controlling the quality and its evolution using automatic systems can suppose a reduction of the losses.Testing some automatic image analysis techniques in the case of tomatoes and zucchini is the main objective of this study.Two steps in the supply chain are considered,the feeding of the raw products into the handling chain(because low quality generates a reduction of the chain productivity)and the cool storage of the processed products(as the value at the market is reduced).It was proposed to analyze the incoming products at the head the processing line using CCD cameras to detect low quality and/or dirty products(corresponding to specific farmers/suppliers,it should be asked to improve to maintain the productivity of the line).The second stage is analyzing the evolution of the products along the cool chain(storage and transport),the use of an App developed to be use under Android was proposed to substitute the“visual”evaluation used in practice.The algorithms used,including stages of pre-treatment,segmentation,analysis and presentation of the results take account of the short time available and the limited capacity of the batteries.High performance techniques were applied to the homography stage to discard some of the images,resulting in better performance.Also threads and renderscript kernels were created to parallelize the methods used on the resulting images being able to inspect faster the products.The proposed method achieves success rates comparable to,and improving,the expert inspection.
基金supported by the National Key Research and Development Program of China(2022YFB3603802)the National Natural Science Foundation of China(62374033)Fujian Science&Technology Innovation Laboratory for Optoelectronic Information of China(2021ZZ129)。
基金supported by the National Key R&D Program of China(2021YFA1200501)National Natural Science Foundation of China(U22A20137,U21A2069,62202350)Shenzhen Science and Technology Innovation Program(JCYJ20220818102215033,GJHZ20210705142542015,JCYJ20220530160811027).
文摘Two-dimensional metal chalcogenides have garnered significant attention as promising candidates for novel neuromorphic synaptic devices due to their exceptional structural and optoelectronic properties.However,achieving large-scale integration and practical applications of synaptic chips has proven to be challenging due to significant hurdles in materials preparation and the absence of effective nanofabrication techniques.In a recent breakthrough,we introduced a revolutionary allopatric defect-modulated Fe_(7)S_(8)@MoS_(2)synaptic heterostructure,which demonstrated remarkable optoelectronic synaptic response capabilities.Building upon this achievement,our current study takes a step further by presenting a sulfurization-seeding synergetic growth strategy,enabling the large-scale and arrayed preparation of Fe_(7)S_(8)@MoS_(2)heterostructures.Moreover,a three-dimensional vertical integration technique was developed for the fabrication of arrayed optoelectronic synaptic chips.Notably,we have successfully simulated the visual persistence function of the human eye with the adoption of the arrayed chip.Our synaptic devices exhibit a remarkable ability to replicate the preprocessing functions of the human visual system,resulting in significantly improved noise reduction and image recognition efficiency.This study might mark an important milestone in advancing the field of optoelectronic synaptic devices,which significantly prompts the development of mature integrated visual perception chips.
文摘In recent years,smart agriculture has gained strength due to the application of industry 4.0 technologies in agriculture.As a result,efforts are increasing in proposing artificial vision applications to solvemany problems.However,many of these applications are developed separately.Many academic works have proposed solutions integrating image classification techniques through IoT platforms.For this reason,this paper aims to answer the following research questions:(1)What are themain problems to be solvedwith smart farming IoT platforms that incorporate images?(2)What are the main strategies for incorporating image classification methods in smart agriculture IoT platforms?and(3)What are the main image acquisition,preprocessing,transmission,and classification technologies used in smart agriculture IoT platforms?This study adopts a Systematic Literature Review(SLR)approach.We searched Scopus,Web of Science,IEEE Xplore,and Springer Link databases from January 2018 to July 2022.Fromwhich we could identify five domains corresponding to(1)disease and pest detection,(2)crop growth and health monitoring,(3)irrigation and crop protectionmanagement,(4)intrusion detection,and(5)fruits and plant counting.There are three types of strategies to integrate image data into smart agriculture IoT platforms:(1)classification process in the edge,(2)classification process in the cloud,and(3)classification process combined.The main advantage of the first is obtaining data in real-time,and its main disadvantage is the cost of implementation.On the other hand,the main advantage of the second is the ability to process high-resolution images,and its main disadvantage is the need for high-bandwidth connectivity.Finally,themixed strategy can significantly benefit infrastructure investment,butmostworks are experimental.
