Understanding the neural underpinning of human gait and balance is one of the most pertinent challenges for 21st-century translational neuroscience due to the profound impact that falls and mobility disturbances have ...Understanding the neural underpinning of human gait and balance is one of the most pertinent challenges for 21st-century translational neuroscience due to the profound impact that falls and mobility disturbances have on our aging population.Posture and gait control does not happen automatically,as previously believed,but rather requires continuous involvement of central nervous mechanisms.To effectively exert control over the body,the brain must integrate multiple streams of sensory information,including visual,vestibular,and somatosensory signals.The mechanisms which underpin the integration of these multisensory signals are the principal topic of the present work.Existing multisensory integration theories focus on how failure of cognitive processes thought to be involved in multisensory integration leads to falls in older adults.Insufficient emphasis,however,has been placed on specific contributions of individual sensory modalities to multisensory integration processes and cross-modal interactions that occur between the sensory modalities in relation to gait and balance.In the present work,we review the contributions of somatosensory,visual,and vestibular modalities,along with their multisensory intersections to gait and balance in older adults and patients with Parkinson’s disease.We also review evidence of vestibular contributions to multisensory temporal binding windows,previously shown to be highly pertinent to fall risk in older adults.Lastly,we relate multisensory vestibular mechanisms to potential neural substrates,both at the level of neurobiology(concerning positron emission tomography imaging)and at the level of electrophysiology(concerning electroencephalography).We hope that this integrative review,drawing influence across multiple subdisciplines of neuroscience,paves the way for novel research directions and therapeutic neuromodulatory approaches,to improve the lives of older adults and patients with neurodegenerative diseases.展开更多
Artificial multisensory devices play a key role in human-computer interaction in the field of artificial intelligence(AI).In this work,we have designed and constructed a novel olfactory-visual bimodal neuromorphic car...Artificial multisensory devices play a key role in human-computer interaction in the field of artificial intelligence(AI).In this work,we have designed and constructed a novel olfactory-visual bimodal neuromorphic carbon nanotube thin film transistor(TFT)arrays for artificial olfactory-visual multisensory synergy recognition with a very low power consumption of 25 aJ for a single pulse,employing semiconducting single-walled carbon nanotubes(sc-SWCNTs)as channel materials and gas sensitive materials,and poly[[4,8-bis[5-(2-ethylhexyl)-2-thienyl]benzo[1,2-b:4,5-b0]dithiophene-2,6-diyl]-2,5-thiophenediyl-[5,7-bis(2-ethylhexyl)-4,8-dioxo-4H,8H-benzo[1,2-c:4,5-c0]dithio-phene-1,3-diyl]](PBDB-T)as the photosensitive material.It is noted that it is the first time to realize the simulation of olfactory and visual senses(from 280 nm to 650 nm)with the wide operating temperature range(0-150℃)in a single SWCNT TFT device and successfully simulate the recovery of olfactory senses after COVID-19 by olfactory-visual synergy.Furthermore,our SWCNT neuromorphic TFT devices with a high IOn/IOff ratio(up to 10^(6))at a low operating voltage(−2 to 0.5 V)can mimic not only the basic biological synaptic functions of olfaction and vision(such as paired-pulse facilitation,short-term plasticity,and long-term plasticity),but also optical wireless communication by Morse code.The proposed multisensory,broadband light-responsive,low-power synaptic devices provide great potential for developing AI robots to face complex external environments.展开更多
Objective To examine the effect of multisensory exercise on balance disorders.Methods PubMed,Scopus and Web of Science were searched to identify eligible studies published before January 1,2020.Eligible studies includ...Objective To examine the effect of multisensory exercise on balance disorders.Methods PubMed,Scopus and Web of Science were searched to identify eligible studies published before January 1,2020.Eligible studies included randomized control trials(RCTs),non-randomized studies,case-control studies,and cohort studies.The methodological quality of the included studies was evaluated using JBI Critical Appraisal Checklists for RCTs and for Quasi-Experimental Studies by two researchers independently.A narrative synthesis of intervention characteristics and health-related outcomes was performed.Results A total of 11 non-randomized studies and 9 RCTs were eligible,including 667 participants.The results supported our assumption that multisensory exercise improved balance in people with balance disorders.All of the 20 studies were believed to be of high or moderate quality.Conclusion Our study confirmed that multisensory exercise was effective in improving balance in people with balance disorders.Multisensory exercises could lower the risk of fall and enhance confidence level to improve the quality of life.Further research is needed to investigate the optimal strategy of multisensory exercises and explore the underlying neural and molecular mechanisms of balance improvement brought by multisensory exercises.展开更多
Multisensory enhancement,as a facilitation phenomenon,is responsible for superior behavioral performance when an individual is responding to cross-modal versus modality-specific stimuli.However,the event-related poten...Multisensory enhancement,as a facilitation phenomenon,is responsible for superior behavioral performance when an individual is responding to cross-modal versus modality-specific stimuli.However,the event-related potential（ERP） counterparts of behavioral multisensory enhancement are not well known.We recorded ERPs and behavioral data from 14 healthy volunteers with three types of target stimuli（modality-specific,bimodal,and trimodal） to examine the spatio-temporal electrophysiological characteristics of multisensory enhancement by comparing behavioral data with ERPs.We found a strong correlation between P3 latency and behavioral performance in terms of reaction time（RT）（R = 0.98,P ＆lt;0.001）,suggesting that P3 latency constitutes a temporal measure of behavioral multisensory enhancement.In addition,a fast RT and short P3 latency were found when comparing the modality-specific visual target with the modality-specific auditory and somatosensory targets.Our results indicate that behavioral multisensory enhancement can be identified by the latency and source distribution of the P3 component.These findings may advance our understanding of the neuronal mechanisms of multisensory enhancement.展开更多
Accurate self-motion perception,which is critical for organisms to survive,is a process involving multiple sensory cues.The two most powerful cues are visual(optic flow)and vestibular(inertial motion).Psychophysical s...Accurate self-motion perception,which is critical for organisms to survive,is a process involving multiple sensory cues.The two most powerful cues are visual(optic flow)and vestibular(inertial motion).Psychophysical studies have indicated that humans and nonhuman primates integrate the two cues to improve the estimation of self-motion direction,often in a statistically Bayesian-optimal way.In the last decade,single-unit recordings in awake,behaving animals have provided valuable neurophysiological data with a high spatial and temporal resolution,giving insight into possible neural mechanisms underlying multisensory self-motion perception.Here,we review these findings,along with new evidence from the most recent studies focusing on the temporal dynamics of signals in different modalities.We show that,in light of new data,conventional thoughts about the cortical mechanisms underlying visuo-vestibular integration for linear self-motion are challenged.We propose that different temporal component signals may mediate different functions,a possibility that requires future studies.展开更多
Progress in understanding multisensory integration in human have suggested researchers that the integration may result into the enhancement or depression of incoming signals.It is evident based on different psychologi...Progress in understanding multisensory integration in human have suggested researchers that the integration may result into the enhancement or depression of incoming signals.It is evident based on different psychological and behavioral experiments that stimuli coming from different perceptual modalities at the same time or from the same place,the signal having more strength under the influence of emotions effects the response accordingly.Current research inmultisensory integration has not studied the effect of emotions despite its significance and natural influence in multisensory enhancement or depression.Therefore,there is a need to integrate the emotional state of the agent with incoming stimuli for signal enhancement or depression.In this study,two different neural network-based learning algorithms have been employed to learn the impact of emotions on signal enhancement or depression.It was observed that the performance of a proposed system for multisensory integration increases when emotion features were present during enhancement or depression of multisensory signals.展开更多
Energy consumption of the EU has a crucial environmental impact;several efforts are nowadays thus directed into massively reducing energy consumption by envelope improvement,system efficiency and smart control.On the ...Energy consumption of the EU has a crucial environmental impact;several efforts are nowadays thus directed into massively reducing energy consumption by envelope improvement,system efficiency and smart control.On the other hand,the indoor thermal and lighting conditions significantly influence users’wellbeing and productivity,which is especially important when dealing with educational and working facilities.Strategies to enhance system efficiency are focused on design and construction aspects.These strategies ease to promote a powerful approach which is needed when focusing on existing buildings in need of retrofit measures.When dealing with new or refurbished buildings,energy saving has a further step to achieve.In the last years,research trends moved towards an increasing inclusion of human factors in energy evaluation.This allows to account for the occupancy variability in the energy analyses,considering how to bridge the performance gap between predictive models and actual consumptions due to indoor thermal settings.In empty buildings energy consumption is huge and economic efforts are wasted,due to unconscious energy-wasting behaviors.Previous studies with Interactive Virtual Environments confirmed that indoor environments illuminated by different color lights lead to the perception of different levels of thermal comfort.The results of the present paper replicate previous evidence collected in real conditions,suggesting that Virtual Reality is a valid and reliable tool to assess thermal comfort more quickly and cheaply.This paper provides a further perspective on this topic,including also the use of different fragrances to understand how the indoor environment could be enhanced and manipulated to increase wellbeing,thermal perception and finally energy saving.展开更多
Background:The concept of stochastic facilitation suggests that the addition of precise amounts of white noise can improve the perceptibility of a stimulus of weak amplitude.We know from previous research that tactile...Background:The concept of stochastic facilitation suggests that the addition of precise amounts of white noise can improve the perceptibility of a stimulus of weak amplitude.We know from previous research that tactile and auditory noise can facilitate visual perception,respectively.Here we wanted to see if the effects of stochastic facilitation generalise to a reaction time paradigm,and if reaction times are correlated with tactile thresholds.We know that when multiple sensory systems are stimulated simultaneously,reaction times are faster than either stimulus alone,and also faster than the sum of reaction times(known as the race model).Methods:Five participants were re-tested in five blocks each of which contained a different background noise levels,randomly ordered across sessions.At each noise level,they performed a tactile threshold detection task and a tactile reaction time task.Results:Both tactile threshold and tactile reaction times were significantly affected by the background white noise.While the preferred amplitude for the white noise was different for every participant,the average lowest threshold was obtained with white noise presented binaurally at 70 db.The reaction times were analysed by fitting an ex-Gaussian,the sum of a Gaussian function and an exponential decay function.The white noise significantly affected the exponential parameter(tau)in a way that is compatible with the facilitation of thresholds.Conclusions:We therefore conclude that multisensory reaction time facilitation can,at least in part,be explained by stochastic facilitation of the neural signals.展开更多
Multisensory lab based in Peking University,has carried out basic studies in multisensory space and time processing,intersensory binding and haptic/tactile perception.We exploited a typical paradigm of multisensory il...Multisensory lab based in Peking University,has carried out basic studies in multisensory space and time processing,intersensory binding and haptic/tactile perception.We exploited a typical paradigm of multisensory illusion-temporal ventriloquist effect and applied it in a wide range of multisensory interactions(mainly focused on temporal processing).In this work,we summarized how the tactile stimuli were exploited to compose tactile cues and as tactile apparent motion to interface with other sensory stimuli(visual and auditory stimuli)to examine the underlying perceptual organization in a multisensory context.Moreover,we introduced two examples of wearable haptic/tactile perception in our lab,by using two customized tactile devices and discussed the potential applications in this field.展开更多
Objectives:Emotional stress is a significant public health challenge.Virtual reality(VR)offers the potential for aiding emotional recovery.This study explores the impact of VR environment design factors on emotional r...Objectives:Emotional stress is a significant public health challenge.Virtual reality(VR)offers the potential for aiding emotional recovery.This study explores the impact of VR environment design factors on emotional recovery,examining underlying mechanisms through physiological indicators and behavioral responses.Methods:Two experimentswere conducted.Experiment 1 employed a 4[Scene Type:real environment(RE),virtual scenes that restore the RE(VR),virtual scenes that incorporate natural window view design(VR-W),and a no-scene control condition(CTL)]×3(Experimental Phase:baseline,emotion arousal,recovery)mixed design(N=33).Participants viewed a 4-min anxiety-inducing video followed by a 3-min scene exposure.State-Trait Anxiety Inventory-State Form(STAIS),galvanic skin response(GSR),and blood-volume pulse(BVP)frequency were analyzed with linear mixed-effects models.Experiment 2 used a 3(Motion-control mode:Unnatural,Semi-natural,Natural)×2(Sound form:Spatial positioning,Surround)×3(Experimental Phase:baseline,emotion arousal,recovery)mixed design(N=42).Presence was analyzed with the Scheirer–Ray–Hare test;phase efficacy was verified with Friedman tests.Results:Experiment 1 showed significant Scene Type×Experimental Phase interactions for GSR(F=8.006,p<0.001,η^(2)_(p)=0.624)and BVP frequency(F=11.491,p<0.001,η^(2)_(p)=0.704).VR-Wproduced the largest recovery(ΔGSR=–1.26;ΔBVP=–5.80;Hedges g≥0.83)vs.RE and VR.STAI-S returned to baseline across all Scene Types.Experiment 2 revealedmain effects of Motion-control mode(F=8.55,p=0.001,η^(2)_(p)=0.32)and Sound form(F=4.35,p=0.044,η^(2)_(p)=0.11)on Presence(Semi-natural+Spatial positioning highest).The greatest physiological recovery occurred with the UnnaturalMotioncontrolmode(GSRH=20.17,p<0.001,ε^(2)=0.49;BVPH=7.92,p=0.019),amplified by Spatial positioning Sound form only in this mode.Design factors did not influence STAI-S change.Conclusions:VR scenes are as restorative as RE;embedding VR-W accelerates recovery.Maximal Presence is not essential:Unnatural Motion-control mode induced the largest physiological recovery,especially combined with Spatial positioning Sound form.展开更多
A visual and tactile multisensory integrated system is essential for human walking due to the demand for real-time interactions between perception and action.Here,a piezoresistor and MoS_(2)field effect transistor are...A visual and tactile multisensory integrated system is essential for human walking due to the demand for real-time interactions between perception and action.Here,a piezoresistor and MoS_(2)field effect transistor are combined to construct an artificial integration nervous system to simulate perception and synaptic plasticity.The key characteristics of synaptic plasticity are successfully demonstrated by individual pressure signals,individual optical signals,the synergy of optical and pressure signals,which are based on the electron trapping–detrapping mechanism at the MoS_(2)/SiO_(2)interface.We demonstrate that perception under synergy is stronger than perception under optical or pressure signal alone,which is similar to a biological system.Moreover,various distinguishable motion scenarios(combination of the following conditions:external lighting environment of day or night,flat or rough road,movement state of walking or running)are simulated and verified by adjusting the amplitude and frequency of the optical and pressure signals.展开更多
Multi-sensory neuromorphic devices(MND)have broad potential in overcoming the structural bottleneck of von Neumann in the era of big data.However,the current multisensory artificial neuromorphic system is mainly based...Multi-sensory neuromorphic devices(MND)have broad potential in overcoming the structural bottleneck of von Neumann in the era of big data.However,the current multisensory artificial neuromorphic system is mainly based on unitary nonvolatile memory or volatile synaptic devices without intrinsic thermal sensitivity,which limits the range of biological multisensory perception and the flexibility and computational efficiency of the neural morphological computing system.Here,a temperature-dependent memory/synaptic hybrid artificial neuromorphic device based on floating gate phototransistors(FGT)is fabricated.The CsPbBr_(3)/TiO_(2)core–shell nanocrystals(NCs)prepared by in-situ pre-protection low-temperature solvothermal method were used as the photosensitive layer.The device exhibits remarkable multi-level visual memory with a large memory window of 59.6 V at room temperature.Surprisingly,when the temperature varies from 20 to 120℃back and forth,the device can switch between nonvolatile memory and volatile synaptic device with reconfigurable and reversible behaviors,which contributes to the efficient visual/thermal fusion perception.This work expands the sensory range of multisensory devices and promotes the development of memory and neuromorphic devices based on organic field-effect transistors(OFET).展开更多
Over the last decade virtual reality （VR） setups for rodents have been developed and utilized to in-vestigate the neural foundations of behavior. Such VR systems became very popular since theyallow the use of state-...Over the last decade virtual reality （VR） setups for rodents have been developed and utilized to in-vestigate the neural foundations of behavior. Such VR systems became very popular since theyallow the use of state-of-the-art techniques to measure neural activity in behaving rodents that can-not be easily used with classical behavior setups. Here, we provide an overview of rodent VR tech-nologies and review recent results from related research. We discuss commonalities and differ-ences as well as merits and issues of different approaches. A special focus is given to experimental（behavioral） paradigms in use. Finally we comment on possible use cases that may further exploitthe potential of VR in rodent research and hence inspire future studies.展开更多
Urbanization of animal habitats has the potential to affect the natural communication systems of any species able to survive in the changed environment. Urban animals such as squirrels use multiple signal channels to ...Urbanization of animal habitats has the potential to affect the natural communication systems of any species able to survive in the changed environment. Urban animals such as squirrels use multiple signal channels to communicate, but it is un- known how ttrbanization has affected these behaviors. Multimodal commtmication, involving more than one sensory modality, can be studied by use of biomimetic mechanical animal models that are designed to simulate the multimodal signals and be pre- sented to animal subjects in the field. In this way the responses to the various signal components can be compared and contrasted to determine whether the multimodal signal is made up of redundant or nonredundant components. In this study, we presented wild gray squirrels in relatively urban and relatively rural habitats in Western Massachusetts with a biomimetic squirrel model that produced tail flags and alarm barks in a variety of combinations. We found that the squirrels responded to each unimodal component on its own, the bark and tail flag, but they responded most to the complete multimodal signal, containing both the acoustic and the moving visual components, providing evidence that in this context the signal components are redundant and that their combination elicits multimodal enhancement. We expanded on the results of Partan et al. （2009） by providing data on sig- naling behavior in the presence and absence of conspecifics, suggesting that alarm signaling is more likely if conspecifics are present. We found that the squirrels were more active in the urban habitats and that they responded more to tail flagging in the urban habitats as compared to the rural ones, suggesting the interesting possibility of a multimodal shift from reliance on audio to visual signals in noisier more crowded urban habitats [Current Zoology 56 （3）： 313-326, 2010].展开更多
In the robotic community more and more hands are developed. Based on theexperience of HIT Hand and DLR Hand II, a smaller and easier manufactured dexterous robot hand withmultisen-sory function and high integration is...In the robotic community more and more hands are developed. Based on theexperience of HIT Hand and DLR Hand II, a smaller and easier manufactured dexterous robot hand withmultisen-sory function and high integration is jointly developed. The prototype of the hand issuccessfully built. It has 4 fingers in total 13-DOFs (degree of freedom). Each finger has 3-DOFsand 4 joints, the last 2 joints are mechanically coupled by means of four-bar linkage mechanism. Italso has an additional DOF to realize motion of the thumb relative to the palm. The fingertip forcecan reach up to 10 N. Full integration of mechanical body, actuation system, multisensory system andelectronics is a significant feature. DSP based control system is implemented in PCI busarchitecture and the serial communication between the hand and DSP needs only 2 lines.展开更多
This paper presents a new developed anthropomorphic robot dexterous hand： HIT/DLR Hand II. The hand is composed of an independent palm and five identical modular fingers, and each finger has three degree of freedom ...This paper presents a new developed anthropomorphic robot dexterous hand： HIT/DLR Hand II. The hand is composed of an independent palm and five identical modular fingers, and each finger has three degree of freedom （DOFs） and four joints. All the actuators and electronics are integrated in the finger body and the palm. Owing to using a new actuator, drivers and a novel arrangement, both the length and width of the finger is about two third of its formner version. By using the wire coupling mecha- nism, the distal phalanx transmission ratio is kept exactly 1 ： 1 in the whole movement range. The packing mechanism which is implemented directly in the finger body and palm not only reduces the size of whole hand but also make it more anthropomorphic. Additionally, the new designed force/torque and position sensors are integrated in the hand for increasing muhisensory capability. To evaluate the performances of the finger mechanism, the position and impedance control experiments are conducted.展开更多
Cross-modal semantic mapping and cross-media retrieval are key problems of the multimedia search engine.This study analyzes the hierarchy,the functionality,and the structure in the visual and auditory sensations of co...Cross-modal semantic mapping and cross-media retrieval are key problems of the multimedia search engine.This study analyzes the hierarchy,the functionality,and the structure in the visual and auditory sensations of cognitive system,and establishes a brain-like cross-modal semantic mapping framework based on cognitive computing of visual and auditory sensations.The mechanism of visual-auditory multisensory integration,selective attention in thalamo-cortical,emotional control in limbic system and the memory-enhancing in hippocampal were considered in the framework.Then,the algorithms of cross-modal semantic mapping were given.Experimental results show that the framework can be effectively applied to the cross-modal semantic mapping,and also provides an important significance for brain-like computing of non-von Neumann structure.展开更多
The present study observed the effects of the non-invasive, mirror neuron system application on neurorehabilitation. In primate studies, mirror neurons have been shown to fire when the subject observes or performs a s...The present study observed the effects of the non-invasive, mirror neuron system application on neurorehabilitation. In primate studies, mirror neurons have been shown to fire when the subject observes or performs a specific action, thereby allowing for observation of motor cortex activation. This activation of the mirror neuron system could serve as a treatment for stroke patients. In the present study, the combination of a mirror neuron system-based therapy was introduced for the treatment of patients with motor-deficits, who could not perform rehabilitation exercises. The results also indicate that this therapeutic method plays a positive role in emotional regulation in the same patients.展开更多
文摘Understanding the neural underpinning of human gait and balance is one of the most pertinent challenges for 21st-century translational neuroscience due to the profound impact that falls and mobility disturbances have on our aging population.Posture and gait control does not happen automatically,as previously believed,but rather requires continuous involvement of central nervous mechanisms.To effectively exert control over the body,the brain must integrate multiple streams of sensory information,including visual,vestibular,and somatosensory signals.The mechanisms which underpin the integration of these multisensory signals are the principal topic of the present work.Existing multisensory integration theories focus on how failure of cognitive processes thought to be involved in multisensory integration leads to falls in older adults.Insufficient emphasis,however,has been placed on specific contributions of individual sensory modalities to multisensory integration processes and cross-modal interactions that occur between the sensory modalities in relation to gait and balance.In the present work,we review the contributions of somatosensory,visual,and vestibular modalities,along with their multisensory intersections to gait and balance in older adults and patients with Parkinson’s disease.We also review evidence of vestibular contributions to multisensory temporal binding windows,previously shown to be highly pertinent to fall risk in older adults.Lastly,we relate multisensory vestibular mechanisms to potential neural substrates,both at the level of neurobiology(concerning positron emission tomography imaging)and at the level of electrophysiology(concerning electroencephalography).We hope that this integrative review,drawing influence across multiple subdisciplines of neuroscience,paves the way for novel research directions and therapeutic neuromodulatory approaches,to improve the lives of older adults and patients with neurodegenerative diseases.
基金supported by the National Key Research and Development Program of China(2020YFA0714700)Natural Science Foundation of China(62274174)+3 种基金Key Research and Development Program of Jiangsu Province(BK20232009)a fellowship from the China Postdoctoral Science Foundation(NO:2023M742559)the Cooperation Project of Vacuum Interconnect Research Facility(NANO-X)of Suzhou Institute of Nano-Tech and Nano-Bionics,Chinese Academy of Sciences(F2208)the technical support for Nano-X from Suzhou Institute of Nano-Tech and Nano-Bionics,Chinese Academy of Sciences(SINANO)。
文摘Artificial multisensory devices play a key role in human-computer interaction in the field of artificial intelligence(AI).In this work,we have designed and constructed a novel olfactory-visual bimodal neuromorphic carbon nanotube thin film transistor(TFT)arrays for artificial olfactory-visual multisensory synergy recognition with a very low power consumption of 25 aJ for a single pulse,employing semiconducting single-walled carbon nanotubes(sc-SWCNTs)as channel materials and gas sensitive materials,and poly[[4,8-bis[5-(2-ethylhexyl)-2-thienyl]benzo[1,2-b:4,5-b0]dithiophene-2,6-diyl]-2,5-thiophenediyl-[5,7-bis(2-ethylhexyl)-4,8-dioxo-4H,8H-benzo[1,2-c:4,5-c0]dithio-phene-1,3-diyl]](PBDB-T)as the photosensitive material.It is noted that it is the first time to realize the simulation of olfactory and visual senses(from 280 nm to 650 nm)with the wide operating temperature range(0-150℃)in a single SWCNT TFT device and successfully simulate the recovery of olfactory senses after COVID-19 by olfactory-visual synergy.Furthermore,our SWCNT neuromorphic TFT devices with a high IOn/IOff ratio(up to 10^(6))at a low operating voltage(−2 to 0.5 V)can mimic not only the basic biological synaptic functions of olfaction and vision(such as paired-pulse facilitation,short-term plasticity,and long-term plasticity),but also optical wireless communication by Morse code.The proposed multisensory,broadband light-responsive,low-power synaptic devices provide great potential for developing AI robots to face complex external environments.
基金supported by grants from the National Twelfth Five-Year Research Program of China(No.2012BAI12B02)the National Natural Science Foundation of China(No.81873701)the Two-hundred Talent Fund of Medical College of Shanghai Jiao Tong University(No.20191921).
文摘Objective To examine the effect of multisensory exercise on balance disorders.Methods PubMed,Scopus and Web of Science were searched to identify eligible studies published before January 1,2020.Eligible studies included randomized control trials(RCTs),non-randomized studies,case-control studies,and cohort studies.The methodological quality of the included studies was evaluated using JBI Critical Appraisal Checklists for RCTs and for Quasi-Experimental Studies by two researchers independently.A narrative synthesis of intervention characteristics and health-related outcomes was performed.Results A total of 11 non-randomized studies and 9 RCTs were eligible,including 667 participants.The results supported our assumption that multisensory exercise improved balance in people with balance disorders.All of the 20 studies were believed to be of high or moderate quality.Conclusion Our study confirmed that multisensory exercise was effective in improving balance in people with balance disorders.Multisensory exercises could lower the risk of fall and enhance confidence level to improve the quality of life.Further research is needed to investigate the optimal strategy of multisensory exercises and explore the underlying neural and molecular mechanisms of balance improvement brought by multisensory exercises.
基金partly supported by the National Natural Science Foundation of China(81271685)Key Project of Science and Technology Fund of Tianjin Municipality,China(10JCZDJC16100)+1 种基金the China Postdoctoral Science Foundation(2012M510754)the Independent Innovative Fund of Tianjin University,Tianjin Municipality,China(1102)
文摘Multisensory enhancement,as a facilitation phenomenon,is responsible for superior behavioral performance when an individual is responding to cross-modal versus modality-specific stimuli.However,the event-related potential（ERP） counterparts of behavioral multisensory enhancement are not well known.We recorded ERPs and behavioral data from 14 healthy volunteers with three types of target stimuli（modality-specific,bimodal,and trimodal） to examine the spatio-temporal electrophysiological characteristics of multisensory enhancement by comparing behavioral data with ERPs.We found a strong correlation between P3 latency and behavioral performance in terms of reaction time（RT）（R = 0.98,P ＆lt;0.001）,suggesting that P3 latency constitutes a temporal measure of behavioral multisensory enhancement.In addition,a fast RT and short P3 latency were found when comparing the modality-specific visual target with the modality-specific auditory and somatosensory targets.Our results indicate that behavioral multisensory enhancement can be identified by the latency and source distribution of the P3 component.These findings may advance our understanding of the neuronal mechanisms of multisensory enhancement.
基金supported by grants from the National Science and Technology Innovation 2030 Major Program(2022ZD0205000)the Strategic Priority Research Program of CAS(XDB32070000)+1 种基金the Shanghai Municipal Science and Technology Major Project(2018SHZDZX05)the Shanghai Academic Research Leader Program(21XD1404000).
文摘Accurate self-motion perception,which is critical for organisms to survive,is a process involving multiple sensory cues.The two most powerful cues are visual(optic flow)and vestibular(inertial motion).Psychophysical studies have indicated that humans and nonhuman primates integrate the two cues to improve the estimation of self-motion direction,often in a statistically Bayesian-optimal way.In the last decade,single-unit recordings in awake,behaving animals have provided valuable neurophysiological data with a high spatial and temporal resolution,giving insight into possible neural mechanisms underlying multisensory self-motion perception.Here,we review these findings,along with new evidence from the most recent studies focusing on the temporal dynamics of signals in different modalities.We show that,in light of new data,conventional thoughts about the cortical mechanisms underlying visuo-vestibular integration for linear self-motion are challenged.We propose that different temporal component signals may mediate different functions,a possibility that requires future studies.
基金This work was supported by the GRRC program of Gyeonggi province.[GRRC-Gachon2020(B04),Development of AI-based Healthcare Devices].
文摘Progress in understanding multisensory integration in human have suggested researchers that the integration may result into the enhancement or depression of incoming signals.It is evident based on different psychological and behavioral experiments that stimuli coming from different perceptual modalities at the same time or from the same place,the signal having more strength under the influence of emotions effects the response accordingly.Current research inmultisensory integration has not studied the effect of emotions despite its significance and natural influence in multisensory enhancement or depression.Therefore,there is a need to integrate the emotional state of the agent with incoming stimuli for signal enhancement or depression.In this study,two different neural network-based learning algorithms have been employed to learn the impact of emotions on signal enhancement or depression.It was observed that the performance of a proposed system for multisensory integration increases when emotion features were present during enhancement or depression of multisensory signals.
文摘Energy consumption of the EU has a crucial environmental impact;several efforts are nowadays thus directed into massively reducing energy consumption by envelope improvement,system efficiency and smart control.On the other hand,the indoor thermal and lighting conditions significantly influence users’wellbeing and productivity,which is especially important when dealing with educational and working facilities.Strategies to enhance system efficiency are focused on design and construction aspects.These strategies ease to promote a powerful approach which is needed when focusing on existing buildings in need of retrofit measures.When dealing with new or refurbished buildings,energy saving has a further step to achieve.In the last years,research trends moved towards an increasing inclusion of human factors in energy evaluation.This allows to account for the occupancy variability in the energy analyses,considering how to bridge the performance gap between predictive models and actual consumptions due to indoor thermal settings.In empty buildings energy consumption is huge and economic efforts are wasted,due to unconscious energy-wasting behaviors.Previous studies with Interactive Virtual Environments confirmed that indoor environments illuminated by different color lights lead to the perception of different levels of thermal comfort.The results of the present paper replicate previous evidence collected in real conditions,suggesting that Virtual Reality is a valid and reliable tool to assess thermal comfort more quickly and cheaply.This paper provides a further perspective on this topic,including also the use of different fragrances to understand how the indoor environment could be enhanced and manipulated to increase wellbeing,thermal perception and finally energy saving.
文摘Background:The concept of stochastic facilitation suggests that the addition of precise amounts of white noise can improve the perceptibility of a stimulus of weak amplitude.We know from previous research that tactile and auditory noise can facilitate visual perception,respectively.Here we wanted to see if the effects of stochastic facilitation generalise to a reaction time paradigm,and if reaction times are correlated with tactile thresholds.We know that when multiple sensory systems are stimulated simultaneously,reaction times are faster than either stimulus alone,and also faster than the sum of reaction times(known as the race model).Methods:Five participants were re-tested in five blocks each of which contained a different background noise levels,randomly ordered across sessions.At each noise level,they performed a tactile threshold detection task and a tactile reaction time task.Results:Both tactile threshold and tactile reaction times were significantly affected by the background white noise.While the preferred amplitude for the white noise was different for every participant,the average lowest threshold was obtained with white noise presented binaurally at 70 db.The reaction times were analysed by fitting an ex-Gaussian,the sum of a Gaussian function and an exponential decay function.The white noise significantly affected the exponential parameter(tau)in a way that is compatible with the facilitation of thresholds.Conclusions:We therefore conclude that multisensory reaction time facilitation can,at least in part,be explained by stochastic facilitation of the neural signals.
基金Supported by Natural Science Foundation of China(NSFC61527804)NSFC and the German Research Foundation(DFG)in Project Crossmodal Learning(NSFC 61621136008/DFG TRR-169)Research fund from brain lab,TAL education group,China.
文摘Multisensory lab based in Peking University,has carried out basic studies in multisensory space and time processing,intersensory binding and haptic/tactile perception.We exploited a typical paradigm of multisensory illusion-temporal ventriloquist effect and applied it in a wide range of multisensory interactions(mainly focused on temporal processing).In this work,we summarized how the tactile stimuli were exploited to compose tactile cues and as tactile apparent motion to interface with other sensory stimuli(visual and auditory stimuli)to examine the underlying perceptual organization in a multisensory context.Moreover,we introduced two examples of wearable haptic/tactile perception in our lab,by using two customized tactile devices and discussed the potential applications in this field.
基金funded by the General Project of the National Social Science Fund(NSSF)of China in Art(Project No.23BG133)the Scientific Research Foundation ofWuhan Institute of Technology(Project No.K2023064)+2 种基金the Open Fund of Key Research Base of Humanities and Social Sciences in Hubei Province—Research Center for College Students’Development and Innovation Education:“VR Healing Intervention for College Students’Academic Anxiety—VR Healing Based on Intention Dialogue”(Project No.DXS2023017)the Research Project of China Youth and Children Research Society(Project No.2025B11)the Major Strategic Consulting Projects of the Chinese Academy of Engineering between Academy and Local Regions(Project No.2024-DFZD-41).
文摘Objectives:Emotional stress is a significant public health challenge.Virtual reality(VR)offers the potential for aiding emotional recovery.This study explores the impact of VR environment design factors on emotional recovery,examining underlying mechanisms through physiological indicators and behavioral responses.Methods:Two experimentswere conducted.Experiment 1 employed a 4[Scene Type:real environment(RE),virtual scenes that restore the RE(VR),virtual scenes that incorporate natural window view design(VR-W),and a no-scene control condition(CTL)]×3(Experimental Phase:baseline,emotion arousal,recovery)mixed design(N=33).Participants viewed a 4-min anxiety-inducing video followed by a 3-min scene exposure.State-Trait Anxiety Inventory-State Form(STAIS),galvanic skin response(GSR),and blood-volume pulse(BVP)frequency were analyzed with linear mixed-effects models.Experiment 2 used a 3(Motion-control mode:Unnatural,Semi-natural,Natural)×2(Sound form:Spatial positioning,Surround)×3(Experimental Phase:baseline,emotion arousal,recovery)mixed design(N=42).Presence was analyzed with the Scheirer–Ray–Hare test;phase efficacy was verified with Friedman tests.Results:Experiment 1 showed significant Scene Type×Experimental Phase interactions for GSR(F=8.006,p<0.001,η^(2)_(p)=0.624)and BVP frequency(F=11.491,p<0.001,η^(2)_(p)=0.704).VR-Wproduced the largest recovery(ΔGSR=–1.26;ΔBVP=–5.80;Hedges g≥0.83)vs.RE and VR.STAI-S returned to baseline across all Scene Types.Experiment 2 revealedmain effects of Motion-control mode(F=8.55,p=0.001,η^(2)_(p)=0.32)and Sound form(F=4.35,p=0.044,η^(2)_(p)=0.11)on Presence(Semi-natural+Spatial positioning highest).The greatest physiological recovery occurred with the UnnaturalMotioncontrolmode(GSRH=20.17,p<0.001,ε^(2)=0.49;BVPH=7.92,p=0.019),amplified by Spatial positioning Sound form only in this mode.Design factors did not influence STAI-S change.Conclusions:VR scenes are as restorative as RE;embedding VR-W accelerates recovery.Maximal Presence is not essential:Unnatural Motion-control mode induced the largest physiological recovery,especially combined with Spatial positioning Sound form.
基金the National Key Research and Development Program of China(No.2019YFB2204400)the Natural Science Basic Research Program of Shaanxi(No.2022JQ-650).
文摘A visual and tactile multisensory integrated system is essential for human walking due to the demand for real-time interactions between perception and action.Here,a piezoresistor and MoS_(2)field effect transistor are combined to construct an artificial integration nervous system to simulate perception and synaptic plasticity.The key characteristics of synaptic plasticity are successfully demonstrated by individual pressure signals,individual optical signals,the synergy of optical and pressure signals,which are based on the electron trapping–detrapping mechanism at the MoS_(2)/SiO_(2)interface.We demonstrate that perception under synergy is stronger than perception under optical or pressure signal alone,which is similar to a biological system.Moreover,various distinguishable motion scenarios(combination of the following conditions:external lighting environment of day or night,flat or rough road,movement state of walking or running)are simulated and verified by adjusting the amplitude and frequency of the optical and pressure signals.
基金the National Natural Science Foundation of China(Nos.62274035,U21A20497,61974029,and 11604051)the National Key Research and Development Program of China(Nos.2022YFB3603803 and 2022YFB3603802)+1 种基金the Natural Science Foundation of Fujian Province(Nos.2020J05104 and 2020J06012)Fujian Science&Technology Innovation Laboratory for Optoelectronic Information of China(Nos.2021ZZ129 and 2021ZZ130).
文摘Multi-sensory neuromorphic devices(MND)have broad potential in overcoming the structural bottleneck of von Neumann in the era of big data.However,the current multisensory artificial neuromorphic system is mainly based on unitary nonvolatile memory or volatile synaptic devices without intrinsic thermal sensitivity,which limits the range of biological multisensory perception and the flexibility and computational efficiency of the neural morphological computing system.Here,a temperature-dependent memory/synaptic hybrid artificial neuromorphic device based on floating gate phototransistors(FGT)is fabricated.The CsPbBr_(3)/TiO_(2)core–shell nanocrystals(NCs)prepared by in-situ pre-protection low-temperature solvothermal method were used as the photosensitive layer.The device exhibits remarkable multi-level visual memory with a large memory window of 59.6 V at room temperature.Surprisingly,when the temperature varies from 20 to 120℃back and forth,the device can switch between nonvolatile memory and volatile synaptic device with reconfigurable and reversible behaviors,which contributes to the efficient visual/thermal fusion perception.This work expands the sensory range of multisensory devices and promotes the development of memory and neuromorphic devices based on organic field-effect transistors(OFET).
文摘Over the last decade virtual reality （VR） setups for rodents have been developed and utilized to in-vestigate the neural foundations of behavior. Such VR systems became very popular since theyallow the use of state-of-the-art techniques to measure neural activity in behaving rodents that can-not be easily used with classical behavior setups. Here, we provide an overview of rodent VR tech-nologies and review recent results from related research. We discuss commonalities and differ-ences as well as merits and issues of different approaches. A special focus is given to experimental（behavioral） paradigms in use. Finally we comment on possible use cases that may further exploitthe potential of VR in rodent research and hence inspire future studies.
文摘Urbanization of animal habitats has the potential to affect the natural communication systems of any species able to survive in the changed environment. Urban animals such as squirrels use multiple signal channels to communicate, but it is un- known how ttrbanization has affected these behaviors. Multimodal commtmication, involving more than one sensory modality, can be studied by use of biomimetic mechanical animal models that are designed to simulate the multimodal signals and be pre- sented to animal subjects in the field. In this way the responses to the various signal components can be compared and contrasted to determine whether the multimodal signal is made up of redundant or nonredundant components. In this study, we presented wild gray squirrels in relatively urban and relatively rural habitats in Western Massachusetts with a biomimetic squirrel model that produced tail flags and alarm barks in a variety of combinations. We found that the squirrels responded to each unimodal component on its own, the bark and tail flag, but they responded most to the complete multimodal signal, containing both the acoustic and the moving visual components, providing evidence that in this context the signal components are redundant and that their combination elicits multimodal enhancement. We expanded on the results of Partan et al. （2009） by providing data on sig- naling behavior in the presence and absence of conspecifics, suggesting that alarm signaling is more likely if conspecifics are present. We found that the squirrels were more active in the urban habitats and that they responded more to tail flagging in the urban habitats as compared to the rural ones, suggesting the interesting possibility of a multimodal shift from reliance on audio to visual signals in noisier more crowded urban habitats [Current Zoology 56 （3）： 313-326, 2010].
文摘In the robotic community more and more hands are developed. Based on theexperience of HIT Hand and DLR Hand II, a smaller and easier manufactured dexterous robot hand withmultisen-sory function and high integration is jointly developed. The prototype of the hand issuccessfully built. It has 4 fingers in total 13-DOFs (degree of freedom). Each finger has 3-DOFsand 4 joints, the last 2 joints are mechanically coupled by means of four-bar linkage mechanism. Italso has an additional DOF to realize motion of the thumb relative to the palm. The fingertip forcecan reach up to 10 N. Full integration of mechanical body, actuation system, multisensory system andelectronics is a significant feature. DSP based control system is implemented in PCI busarchitecture and the serial communication between the hand and DSP needs only 2 lines.
基金supported by the National High Technology Research and Development Programme of China(2006AA04Z255)the 111 Project(B307018)
文摘This paper presents a new developed anthropomorphic robot dexterous hand： HIT/DLR Hand II. The hand is composed of an independent palm and five identical modular fingers, and each finger has three degree of freedom （DOFs） and four joints. All the actuators and electronics are integrated in the finger body and the palm. Owing to using a new actuator, drivers and a novel arrangement, both the length and width of the finger is about two third of its formner version. By using the wire coupling mecha- nism, the distal phalanx transmission ratio is kept exactly 1 ： 1 in the whole movement range. The packing mechanism which is implemented directly in the finger body and palm not only reduces the size of whole hand but also make it more anthropomorphic. Additionally, the new designed force/torque and position sensors are integrated in the hand for increasing muhisensory capability. To evaluate the performances of the finger mechanism, the position and impedance control experiments are conducted.
基金Supported by the National Natural Science Foundation of China(No.61305042,61202098)Projects of Center for Remote Sensing Mission Study of China National Space Administration(No.2012A03A0939)Science and Technological Research of Key Projects of Education Department of Henan Province of China(No.13A520071)
文摘Cross-modal semantic mapping and cross-media retrieval are key problems of the multimedia search engine.This study analyzes the hierarchy,the functionality,and the structure in the visual and auditory sensations of cognitive system,and establishes a brain-like cross-modal semantic mapping framework based on cognitive computing of visual and auditory sensations.The mechanism of visual-auditory multisensory integration,selective attention in thalamo-cortical,emotional control in limbic system and the memory-enhancing in hippocampal were considered in the framework.Then,the algorithms of cross-modal semantic mapping were given.Experimental results show that the framework can be effectively applied to the cross-modal semantic mapping,and also provides an important significance for brain-like computing of non-von Neumann structure.
文摘The present study observed the effects of the non-invasive, mirror neuron system application on neurorehabilitation. In primate studies, mirror neurons have been shown to fire when the subject observes or performs a specific action, thereby allowing for observation of motor cortex activation. This activation of the mirror neuron system could serve as a treatment for stroke patients. In the present study, the combination of a mirror neuron system-based therapy was introduced for the treatment of patients with motor-deficits, who could not perform rehabilitation exercises. The results also indicate that this therapeutic method plays a positive role in emotional regulation in the same patients.
