Brain-machine interface (BMI) has been developed due to its possibility to cure severe body paralysis. This technology has been used to realize the direct control of prosthetic devices,such as robot arms,computer curs...Brain-machine interface (BMI) has been developed due to its possibility to cure severe body paralysis. This technology has been used to realize the direct control of prosthetic devices,such as robot arms,computer cursors,and paralyzed muscles. A variety of neural decoding algorithms have been designed to explore relationships between neural activities and movements of the limbs. In this paper,two novel neural decoding methods based on probabilistic neural network (PNN) in rats were introduced,the PNN decoder and the modified PNN (MPNN) decoder. In the ex-periment,rats were trained to obtain water by pressing a lever over a pressure threshold. Microelectrode array was implanted in the motor cortex to record neural activity,and pressure was recorded by a pressure sensor synchronously. After training,the pressure values were estimated from the neural signals by PNN and MPNN decoders. Their per-formances were evaluated by a correlation coefficient (CC) and a mean square error (MSE). The results show that the MPNN decoder,with a CC of 0.8657 and an MSE of 0.2563,outperformed the traditionally-used Wiener filter (WF) and Kalman filter (KF) decoders. It was also observed that the discretization level did not affect the MPNN performance,indicating that the MPNN decoder can handle different tasks in BMI system,including the detection of movement states and estimation of continuous kinematic parameters.展开更多
In the field of brain decoding research,reconstructing visual perception from neural recordings is a challenging but crucial task.With the use of superior algorithms,many methods have been dedicated to enhancing decod...In the field of brain decoding research,reconstructing visual perception from neural recordings is a challenging but crucial task.With the use of superior algorithms,many methods have been dedicated to enhancing decoding performance.However,these models that map neural activities onto semantically entangled feature space are difficult to interpret.It is hard to understand the connections between neural activities and these abstract features.In this paper,we propose an interpretable neural decoding model that projects neural activities onto a semantically disentangled feature space with each dimension representing distinct visual attributes,such as gender and facial pose.A two-stage algorithm is designed to achieve this goal.First,a deep generative model learns semantically-disentangled image representations in an unsupervised way.Second,neural activities are linearly embedded into the semantic space,which the generator uses to reconstruct visual stimuli.Due to modality heterogeneity,it is challenging to learn such a neural embedded high-level semantic representation.We induce pixel,feature,and semantic alignment to ensure reconstruction quality.Three experimental fMRI datasets containing handwritten digits,characters,and human face stimuli are used to evaluate the neural decoding performance of our model.We also demonstrate the model interpretability through a reconstructed image editing application.The experimental results indicate that our model maintains a competitive decoding performance while remaining interpretable.展开更多
Vision plays a peculiar role in intelligence.Visual information,forming a large part of the sensory information,is fed into the human brain to formulate various types of cognition and behaviours that make humans becom...Vision plays a peculiar role in intelligence.Visual information,forming a large part of the sensory information,is fed into the human brain to formulate various types of cognition and behaviours that make humans become intelligent agents.Recent advances have led to the development of brain-inspired algorithms and models for machine vision.One of the key components of these methods is the utilization of the computational principles underlying biological neurons.Additionally,advanced experimental neuroscience techniques have generated different types of neural signals that carry essential visual information.Thus,there is a high demand for mapping out functional models for reading out visual information from neural signals.Here,we briefly review recent progress on this issue with a focus on how machine learning techniques can help in the development of models for contending various types of neural signals,from fine-scale neural spikes and single-cell calcium imaging to coarse-scale electroencephalography(EEG)and functional magnetic resonance imaging recordings of brain signals.展开更多
This paper presents a novel method for inferring the odor based on neural activities observed from rats' main olfactory bulbs.Multi-channel extra-cellular single unit recordings are done by micro-wire electrodes(T...This paper presents a novel method for inferring the odor based on neural activities observed from rats' main olfactory bulbs.Multi-channel extra-cellular single unit recordings are done by micro-wire electrodes(Tungsten,50 μm,32 channels)implanted in the mitral/tufted cell layers of the main olfactory bulb of the anesthetized rats to obtain neural responses to various odors.Neural responses as a key feature are measured by subtraction firing rates before stimulus from after.For odor inference,a decoding method is developed based on the ML estimation.The results show that the average decoding accuracy is about 100.0%,96.0%,and 80.0% with three rats,respectively.This work has profound implications for a novel brain-machine interface system for odor inference.展开更多
The paper presents the neural decoding result of finger or wrist movements using the primary motor cortex(M1)neural activities prior to its movement.It is well known that the observations of motor commands in brain ar...The paper presents the neural decoding result of finger or wrist movements using the primary motor cortex(M1)neural activities prior to its movement.It is well known that the observations of motor commands in brain are in advance before motor movements in the central nerve system.Readiness potential(RP)for electroencephalogram(EEG)has become an important domain of research.Likewise,pre-movement neural responses in M1 primary motor cortex have been observed.The neural activity data before 1 s.were used for neural decoding when the actual movements happened around 1 s.The obtained decoding accuracy in novel method reaches as high as 95% with 30 randomly selected neurons.展开更多
Quantum error correction technology is an important solution to solve the noise interference generated during the operation of quantum computers.In order to find the best syndrome of the stabilizer code in quantum err...Quantum error correction technology is an important solution to solve the noise interference generated during the operation of quantum computers.In order to find the best syndrome of the stabilizer code in quantum error correction,we need to find a fast and close to the optimal threshold decoder.In this work,we build a convolutional neural network(CNN)decoder to correct errors in the toric code based on the system research of machine learning.We analyze and optimize various conditions that affect CNN,and use the RestNet network architecture to reduce the running time.It is shortened by 30%-40%,and we finally design an optimized algorithm for CNN decoder.In this way,the threshold accuracy of the neural network decoder is made to reach 10.8%,which is closer to the optimal threshold of about 11%.The previous threshold of 8.9%-10.3%has been slightly improved,and there is no need to verify the basic noise.展开更多
Along with the flourishing of brain-computer interface technology,the brain-to-brain information transmission between different organisms has received high attention in recent years.However,specific information transm...Along with the flourishing of brain-computer interface technology,the brain-to-brain information transmission between different organisms has received high attention in recent years.However,specific information transmission mode and implementation technology need to be further studied.In this paper,we constructed a brain-to-brain information transmission system between pigeons based on the neural information decoding and electrical stimulation encoding technologies.Our system consists of three parts:(1)the“perception pigeon”learns to distinguish different visual stimuli with two discrepant frequencies,(2)the computer decodes the stimuli based on the neural signals recorded from the“perception pigeon”through a frequency identification algorithm(neural information decoding)and encodes them into different kinds of electrical pulses,(3)the“action pigeon”receives the Intracortical Microstimulation(ICMS)and executes corresponding key-pecking actions through discriminative learning(electrical stimulation encoding).The experimental results show that our brain-to-brain system achieves information transmission from perception to action between two pigeons with the average accuracy of about 72%.Our study verifies the feasibility of information transmission between inter-brain based on neural information decoding and ICMS encoding,providing important technical methods and experimental program references for the development of brain-to-brain communication technology.展开更多
Neuroprostheses aim to repair and replace damaged sensory brain functions such as vision,hearing and touch,improve cognitive functions such as memory,and control arms through electrical stimulations in motor cortex or...Neuroprostheses aim to repair and replace damaged sensory brain functions such as vision,hearing and touch,improve cognitive functions such as memory,and control arms through electrical stimulations in motor cortex or peripheral nerves.Through review of the progress and status of different neuroprostheses,we found an increasing role of machine learning in achieving complex prosthetic functions with groundbreaking results.This article provides a perspective on the role of machine learning in neuroprostheses designs and envisions future involvement of machine learning for more capable neuroprostheses in revolutionizing the treatment of neurological disorders and disabilities.展开更多
The target detection based on electroencephalogram(EEG)signals is a new target detection method.This method recognizes the target by decoding the specific neural response when an operator observes the target,which has...The target detection based on electroencephalogram(EEG)signals is a new target detection method.This method recognizes the target by decoding the specific neural response when an operator observes the target,which has important theoretical and application values.This paper focuses on the EEG detection of low-quality video targets,which breaks through the limitation of previous target detection based on EEG signals only for high-quality video targets.We first design an experimental paradigm for EEG-based low-quality video target detection and propose an epoch extraction method based on eye movement signals to solve the asynchronous problem faced by low-quality video target detection.Then,the neural representation in the process of operator recognition is analyzed based on the time domain,frequency domain,and source space domain,respectively.We design the time-frequency features based on continuous wavelet transform according to the neural representation and obtain an average decoding test accuracy of 84.56%.The research results of this paper lay the foundation for the development of a video target detection system based on EEG signals in the future.展开更多
基金Project supported by the National Natural Science Foundation of China (Nos. 30800287 and 60703038)the Natural Science Foundation of Zhejiang Province, China (No. Y2090707)
文摘Brain-machine interface (BMI) has been developed due to its possibility to cure severe body paralysis. This technology has been used to realize the direct control of prosthetic devices,such as robot arms,computer cursors,and paralyzed muscles. A variety of neural decoding algorithms have been designed to explore relationships between neural activities and movements of the limbs. In this paper,two novel neural decoding methods based on probabilistic neural network (PNN) in rats were introduced,the PNN decoder and the modified PNN (MPNN) decoder. In the ex-periment,rats were trained to obtain water by pressing a lever over a pressure threshold. Microelectrode array was implanted in the motor cortex to record neural activity,and pressure was recorded by a pressure sensor synchronously. After training,the pressure values were estimated from the neural signals by PNN and MPNN decoders. Their per-formances were evaluated by a correlation coefficient (CC) and a mean square error (MSE). The results show that the MPNN decoder,with a CC of 0.8657 and an MSE of 0.2563,outperformed the traditionally-used Wiener filter (WF) and Kalman filter (KF) decoders. It was also observed that the discretization level did not affect the MPNN performance,indicating that the MPNN decoder can handle different tasks in BMI system,including the detection of movement states and estimation of continuous kinematic parameters.
基金supported in part by the National Key R&D Program of China(No.2022ZD0116500)in part by the National Natural Science Foundation of China(No.62206284)。
文摘In the field of brain decoding research,reconstructing visual perception from neural recordings is a challenging but crucial task.With the use of superior algorithms,many methods have been dedicated to enhancing decoding performance.However,these models that map neural activities onto semantically entangled feature space are difficult to interpret.It is hard to understand the connections between neural activities and these abstract features.In this paper,we propose an interpretable neural decoding model that projects neural activities onto a semantically disentangled feature space with each dimension representing distinct visual attributes,such as gender and facial pose.A two-stage algorithm is designed to achieve this goal.First,a deep generative model learns semantically-disentangled image representations in an unsupervised way.Second,neural activities are linearly embedded into the semantic space,which the generator uses to reconstruct visual stimuli.Due to modality heterogeneity,it is challenging to learn such a neural embedded high-level semantic representation.We induce pixel,feature,and semantic alignment to ensure reconstruction quality.Three experimental fMRI datasets containing handwritten digits,characters,and human face stimuli are used to evaluate the neural decoding performance of our model.We also demonstrate the model interpretability through a reconstructed image editing application.The experimental results indicate that our model maintains a competitive decoding performance while remaining interpretable.
基金supported by National Natural Science Foundation of China(Nos.62176003 and 62088102)the Royal Society Newton Advanced Fellowship of the UK(No.NAF-R1-191082)。
文摘Vision plays a peculiar role in intelligence.Visual information,forming a large part of the sensory information,is fed into the human brain to formulate various types of cognition and behaviours that make humans become intelligent agents.Recent advances have led to the development of brain-inspired algorithms and models for machine vision.One of the key components of these methods is the utilization of the computational principles underlying biological neurons.Additionally,advanced experimental neuroscience techniques have generated different types of neural signals that carry essential visual information.Thus,there is a high demand for mapping out functional models for reading out visual information from neural signals.Here,we briefly review recent progress on this issue with a focus on how machine learning techniques can help in the development of models for contending various types of neural signals,from fine-scale neural spikes and single-cell calcium imaging to coarse-scale electroencephalography(EEG)and functional magnetic resonance imaging recordings of brain signals.
基金supported by the MKE(The Ministry of Knowledge Economy,Korea)theITRC(Information Technology Research Center)support program(NIPA-2010-C1090-1021-0010)
文摘This paper presents a novel method for inferring the odor based on neural activities observed from rats' main olfactory bulbs.Multi-channel extra-cellular single unit recordings are done by micro-wire electrodes(Tungsten,50 μm,32 channels)implanted in the mitral/tufted cell layers of the main olfactory bulb of the anesthetized rats to obtain neural responses to various odors.Neural responses as a key feature are measured by subtraction firing rates before stimulus from after.For odor inference,a decoding method is developed based on the ML estimation.The results show that the average decoding accuracy is about 100.0%,96.0%,and 80.0% with three rats,respectively.This work has profound implications for a novel brain-machine interface system for odor inference.
基金MKE(the Ministry of Knowledge Economy),Korea,under the ITRC(Infor mation Technology Research Center)support program supervised by the NIPA(National ITIndustry Promotion Agency)(NIPA-2011-C1090-1121-0010)
文摘The paper presents the neural decoding result of finger or wrist movements using the primary motor cortex(M1)neural activities prior to its movement.It is well known that the observations of motor commands in brain are in advance before motor movements in the central nerve system.Readiness potential(RP)for electroencephalogram(EEG)has become an important domain of research.Likewise,pre-movement neural responses in M1 primary motor cortex have been observed.The neural activity data before 1 s.were used for neural decoding when the actual movements happened around 1 s.The obtained decoding accuracy in novel method reaches as high as 95% with 30 randomly selected neurons.
基金the National Natural Science Foundation of China(Grant Nos.11975132 and 61772295)the Natural Science Foundation of Shandong Province,China(Grant No.ZR2019YQ01)the Project of Shandong Province Higher Educational Science and Technology Program,China(Grant No.J18KZ012).
文摘Quantum error correction technology is an important solution to solve the noise interference generated during the operation of quantum computers.In order to find the best syndrome of the stabilizer code in quantum error correction,we need to find a fast and close to the optimal threshold decoder.In this work,we build a convolutional neural network(CNN)decoder to correct errors in the toric code based on the system research of machine learning.We analyze and optimize various conditions that affect CNN,and use the RestNet network architecture to reduce the running time.It is shortened by 30%-40%,and we finally design an optimized algorithm for CNN decoder.In this way,the threshold accuracy of the neural network decoder is made to reach 10.8%,which is closer to the optimal threshold of about 11%.The previous threshold of 8.9%-10.3%has been slightly improved,and there is no need to verify the basic noise.
基金supported by the National Natural Science Foundation of China(62301496)the Postdoctoral Fellowship Program of China Postdoctoral Science Foundation(GZC20232447)+1 种基金the Program for Science and Technology of Henan Province of China(242300421411)the Key Scientific and Technological Projects of Henan Province(232102210072,232102210098).
文摘Along with the flourishing of brain-computer interface technology,the brain-to-brain information transmission between different organisms has received high attention in recent years.However,specific information transmission mode and implementation technology need to be further studied.In this paper,we constructed a brain-to-brain information transmission system between pigeons based on the neural information decoding and electrical stimulation encoding technologies.Our system consists of three parts:(1)the“perception pigeon”learns to distinguish different visual stimuli with two discrepant frequencies,(2)the computer decodes the stimuli based on the neural signals recorded from the“perception pigeon”through a frequency identification algorithm(neural information decoding)and encodes them into different kinds of electrical pulses,(3)the“action pigeon”receives the Intracortical Microstimulation(ICMS)and executes corresponding key-pecking actions through discriminative learning(electrical stimulation encoding).The experimental results show that our brain-to-brain system achieves information transmission from perception to action between two pigeons with the average accuracy of about 72%.Our study verifies the feasibility of information transmission between inter-brain based on neural information decoding and ICMS encoding,providing important technical methods and experimental program references for the development of brain-to-brain communication technology.
基金supported in part by Science and Technology Innovation(STI)2030 Major Projects,China(No.2021ZD0200400).
文摘Neuroprostheses aim to repair and replace damaged sensory brain functions such as vision,hearing and touch,improve cognitive functions such as memory,and control arms through electrical stimulations in motor cortex or peripheral nerves.Through review of the progress and status of different neuroprostheses,we found an increasing role of machine learning in achieving complex prosthetic functions with groundbreaking results.This article provides a perspective on the role of machine learning in neuroprostheses designs and envisions future involvement of machine learning for more capable neuroprostheses in revolutionizing the treatment of neurological disorders and disabilities.
基金supported by Basic Research Plan under Grant JCKY2022602C024.
文摘The target detection based on electroencephalogram(EEG)signals is a new target detection method.This method recognizes the target by decoding the specific neural response when an operator observes the target,which has important theoretical and application values.This paper focuses on the EEG detection of low-quality video targets,which breaks through the limitation of previous target detection based on EEG signals only for high-quality video targets.We first design an experimental paradigm for EEG-based low-quality video target detection and propose an epoch extraction method based on eye movement signals to solve the asynchronous problem faced by low-quality video target detection.Then,the neural representation in the process of operator recognition is analyzed based on the time domain,frequency domain,and source space domain,respectively.We design the time-frequency features based on continuous wavelet transform according to the neural representation and obtain an average decoding test accuracy of 84.56%.The research results of this paper lay the foundation for the development of a video target detection system based on EEG signals in the future.
