One of the most significant challenges in the neuroscience community is to understand how the human brain works.Recent progress in neuroimaging techniques have validated that it is possible to decode a person′s thoug...One of the most significant challenges in the neuroscience community is to understand how the human brain works.Recent progress in neuroimaging techniques have validated that it is possible to decode a person′s thoughts,memories,and emotions via functional magnetic resonance imaging(i.e.,fMRI)since it can measure the neural activation of human brains with satisfied spatiotemporal resolutions.However,the unprecedented scale and complexity of the fMRI data have presented critical computational bottlenecks requiring new scientific analytic tools.Given the increasingly important role of machine learning in neuroscience,a great many machine learning algorithms are presented to analyze brain activities from the fMRI data.In this paper,we mainly provide a comprehensive and up-to-date review of machine learning methods for analyzing neural activities with the following three aspects,i.e.,brain image functional alignment,brain activity pattern analysis,and visual stimuli reconstruction.In addition,online resources and open research problems on brain pattern analysis are also provided for the convenience of future research.展开更多
The BRAIN project recently announced by the president Obama is the reflection of unrelenting human quest for cracking the brain code, the patterns of neuronal activity that define who we are and what we are. While the...The BRAIN project recently announced by the president Obama is the reflection of unrelenting human quest for cracking the brain code, the patterns of neuronal activity that define who we are and what we are. While the Brain Activity Mapping proposal has rightly emphasized on the need to develop new technologies for measuring every spike from every neuron, it might be helpful to consider both the theoretical and experimental aspects that would accelerate our search for the organizing principles of the brain code. Here we share several insights and lessons from the similar proposal, namely, Brain Decoding Project that we initiated since 2007. We provide a specific example in our initial mapping of real-time memory traces from one part of the memory circuit, namely, the CA1 region of the mouse hippocampus. We show how innovative behavioral tasks and appropriate mathematical analyses of large datasets can play equally, if not more, important roles in uncovering the specific-to-general feature-coding cell assembly mechanism by which episodic memory, semantic knowledge, and imagination are generated and organized. Our own experiences suggest that the bottleneck of the Brain Project is not only at merely developing additional new technologies, but also the lack of efficient avenues to disseminate cutting edge platforms and decoding expertise to neuroscience community. Therefore, we propose that in order to harness unique insights and extensive knowledge from various investigators working in diverse neuroscience subfields, ranging from perception and emotion to memory and social behaviors, the BRAIN project should create a set of International and National Brain Decoding Centers at which cutting-edge recording technologies and expertise on analyzing large datasets analyses can be made readily available to the entire community of neuroscientists who can apply and schedule to perform cutting-edge research.展开更多
Brain decoding based on functional magnetic resonance imaging has recently enabled the identification of visual perception and mental states.However,due to the limitations of sample size and the lack of an effective r...Brain decoding based on functional magnetic resonance imaging has recently enabled the identification of visual perception and mental states.However,due to the limitations of sample size and the lack of an effective reconstruction model,accurate reconstruction of natural images is still a major challenge.The current,rapid development of deep learning models provides the possibility of overcoming these obstacles.Here,we propose a deep learning-based framework that includes a latent feature extractor,a latent feature decoder,and a natural image generator,to achieve the accurate reconstruction of natural images from brain activity.The latent feature extractor is used to extract the latent features of natural images.The latent feature decoder predicts the latent features of natural images based on the response signals from the higher visual cortex.The natural image generatoris applied to generate reconstructed images from the predicted latent features of natural images and the response signals from the visual cortex.Quantitative and qualitative evaluations were conducted with test images.The results showed that the reconstructed image achieved comparable,accurate reproduction of the presented image in both highlevel semantic category information and low-level pixel information.The framework we propose shows promise for decoding the brain activity.展开更多
基金This work was supported by National Natural Science Foundation of China(Nos.61876082,61861130366,6173-2006 and 61902183)National Key Research and Development Program of China(Nos.2018 YFC2001600,2018YFC 2001602)+1 种基金the Royal Society-Academy of Medical Sciences Newton Advanced Fellowship(No.NAF\R1\180371)China Postdoctoral Science Foundation funded project(No.2019M661831).
文摘One of the most significant challenges in the neuroscience community is to understand how the human brain works.Recent progress in neuroimaging techniques have validated that it is possible to decode a person′s thoughts,memories,and emotions via functional magnetic resonance imaging(i.e.,fMRI)since it can measure the neural activation of human brains with satisfied spatiotemporal resolutions.However,the unprecedented scale and complexity of the fMRI data have presented critical computational bottlenecks requiring new scientific analytic tools.Given the increasingly important role of machine learning in neuroscience,a great many machine learning algorithms are presented to analyze brain activities from the fMRI data.In this paper,we mainly provide a comprehensive and up-to-date review of machine learning methods for analyzing neural activities with the following three aspects,i.e.,brain image functional alignment,brain activity pattern analysis,and visual stimuli reconstruction.In addition,online resources and open research problems on brain pattern analysis are also provided for the convenience of future research.
基金Georgia Research Alliance for funding the Brain Decoding Initiative (2007 present)Yunnan Province Department of Science and Technology for the support of our work
文摘The BRAIN project recently announced by the president Obama is the reflection of unrelenting human quest for cracking the brain code, the patterns of neuronal activity that define who we are and what we are. While the Brain Activity Mapping proposal has rightly emphasized on the need to develop new technologies for measuring every spike from every neuron, it might be helpful to consider both the theoretical and experimental aspects that would accelerate our search for the organizing principles of the brain code. Here we share several insights and lessons from the similar proposal, namely, Brain Decoding Project that we initiated since 2007. We provide a specific example in our initial mapping of real-time memory traces from one part of the memory circuit, namely, the CA1 region of the mouse hippocampus. We show how innovative behavioral tasks and appropriate mathematical analyses of large datasets can play equally, if not more, important roles in uncovering the specific-to-general feature-coding cell assembly mechanism by which episodic memory, semantic knowledge, and imagination are generated and organized. Our own experiences suggest that the bottleneck of the Brain Project is not only at merely developing additional new technologies, but also the lack of efficient avenues to disseminate cutting edge platforms and decoding expertise to neuroscience community. Therefore, we propose that in order to harness unique insights and extensive knowledge from various investigators working in diverse neuroscience subfields, ranging from perception and emotion to memory and social behaviors, the BRAIN project should create a set of International and National Brain Decoding Centers at which cutting-edge recording technologies and expertise on analyzing large datasets analyses can be made readily available to the entire community of neuroscientists who can apply and schedule to perform cutting-edge research.
基金supported by the National Natural Science Foundation of China(61773094,61533006,U1808204,31730039,31671133,and 61876114)the Ministry of Science and Technology of China(2015CB351701)+1 种基金the National Major Scientific Instruments and Equipment Development Project(ZDYZ2015-2)a Chinese Academy of Sciences Strategic Priority Research Program B grant(XDB32010300)。
文摘Brain decoding based on functional magnetic resonance imaging has recently enabled the identification of visual perception and mental states.However,due to the limitations of sample size and the lack of an effective reconstruction model,accurate reconstruction of natural images is still a major challenge.The current,rapid development of deep learning models provides the possibility of overcoming these obstacles.Here,we propose a deep learning-based framework that includes a latent feature extractor,a latent feature decoder,and a natural image generator,to achieve the accurate reconstruction of natural images from brain activity.The latent feature extractor is used to extract the latent features of natural images.The latent feature decoder predicts the latent features of natural images based on the response signals from the higher visual cortex.The natural image generatoris applied to generate reconstructed images from the predicted latent features of natural images and the response signals from the visual cortex.Quantitative and qualitative evaluations were conducted with test images.The results showed that the reconstructed image achieved comparable,accurate reproduction of the presented image in both highlevel semantic category information and low-level pixel information.The framework we propose shows promise for decoding the brain activity.
