Microscopy,crucial for exploring biological structures,often uses polarizing microscopes to observe tissue anisotropy and reconstruct label-free images.However,these images typically show low contrast,and while fluore...Microscopy,crucial for exploring biological structures,often uses polarizing microscopes to observe tissue anisotropy and reconstruct label-free images.However,these images typically show low contrast,and while fluorescence imaging offers higher contrast,it is phototoxic and can disrupt natural assembly dynamics.This study focuses on reconstructing fluores-cence images from label-free ones using a complex nonlinear transformation,specifically aiming to identify organelles within diverse optical properties of tissues.A multimodal deep learning model,3DTransMDL,was developed,employing the Stokes vector to analyze the sample's retardance,phase,and orientation.This model incorporates isotropy and anisot-ropy to differentiate organelles,enhancing the input with structures'varied optical properties.Additionally,techniques like background distortion normalization and covariate shift methods were applied to reduce noise and overfitting,improving model generalization.The approach was tested on mouse kidney and human brain tissues,successfully identifying specific organelles and demonstrating superior performance in reconstructing 3D images,significantly reducing artifacts compared to 2D predictions.Evaluation metrics such as SSIM,PCC,and R2 score confirm the model's efficacy,with improvements observed in multi-modality input setups.This advancement suggests potential applications in molecular dynamics,aiming for further enhancements in future studies.展开更多
Gesture recognition technology enables machines to read human gestures and has significant application prospects in the fields of human-computer interaction and sign language translation.Existing researches usually us...Gesture recognition technology enables machines to read human gestures and has significant application prospects in the fields of human-computer interaction and sign language translation.Existing researches usually use convolutional neural networks to extract features directly from raw gesture data for gesture recognition,but the networks are affected by much interference information in the input data and thus fit to some unimportant features.In this paper,we proposed a novel method for encoding spatio-temporal information,which can enhance the key features required for gesture recognition,such as shape,structure,contour,position and hand motion of gestures,thereby improving the accuracy of gesture recognition.This encoding method can encode arbitrarily multiple frames of gesture data into a single frame of the spatio-temporal feature map and use the spatio-temporal feature map as the input to the neural network.This can guide the model to fit important features while avoiding the use of complex recurrent network structures to extract temporal features.In addition,we designed two sub-networks and trained the model using a sub-network pre-training strategy that trains the sub-networks first and then the entire network,so as to avoid the subnetworks focusing too much on the information of a single category feature and being overly influenced by each other’s features.Experimental results on two public gesture datasets show that the proposed spatio-temporal information encoding method achieves advanced accuracy.展开更多
Language input in foreign language classrooms is no longer restricted to one particular mode of transmission,but it can be achieved by use of different modes.Multimodality or the combination of various modes has been ...Language input in foreign language classrooms is no longer restricted to one particular mode of transmission,but it can be achieved by use of different modes.Multimodality or the combination of various modes has been applied to stimulate foreign language learning.The present paper aims to verify the effectiveness of multimodal input in language learning.It demonstrates the experiment conducted in two freshman classes in a university.The two classes used the same textbook supported by audio CD containing the recorded material.The experiment class,however,additionally has some visuals in listening classes to strengthen the linguistic input.The research lasted for one semester and findings from the study show that multimodal input has significantly improved the listening and speaking abilities of the students and also has positive impacts on the learners’attitudes towards language study.展开更多
A 1×8 multimode interference power splitter with multimode input/output waveguides in SOI material is designed by the beam propagation method and fabricated by the inductive coupled plasma etching technology for...A 1×8 multimode interference power splitter with multimode input/output waveguides in SOI material is designed by the beam propagation method and fabricated by the inductive coupled plasma etching technology for use in fiber optics communication systems.The fabricated device exhibits low loss and good coupling uniformity.The excess loss is lower than 0 8dB,and the uniformity is 0 45dB at the wavelength of 1550nm.Moreover,the polarization dependent loss is lower than 0 7dB at 1550nm.The device size is only 2mm×10mm.展开更多
基金supported by NSFCs(Nos.52361145714)the fund of the Beijing Municipal Education Commission,China,under grant number 22019821001the fund of Climbing Program Foundation from Beijing Institute of Petrochemical Technology(Project No.BIPTAAI-2021007).
文摘Microscopy,crucial for exploring biological structures,often uses polarizing microscopes to observe tissue anisotropy and reconstruct label-free images.However,these images typically show low contrast,and while fluorescence imaging offers higher contrast,it is phototoxic and can disrupt natural assembly dynamics.This study focuses on reconstructing fluores-cence images from label-free ones using a complex nonlinear transformation,specifically aiming to identify organelles within diverse optical properties of tissues.A multimodal deep learning model,3DTransMDL,was developed,employing the Stokes vector to analyze the sample's retardance,phase,and orientation.This model incorporates isotropy and anisot-ropy to differentiate organelles,enhancing the input with structures'varied optical properties.Additionally,techniques like background distortion normalization and covariate shift methods were applied to reduce noise and overfitting,improving model generalization.The approach was tested on mouse kidney and human brain tissues,successfully identifying specific organelles and demonstrating superior performance in reconstructing 3D images,significantly reducing artifacts compared to 2D predictions.Evaluation metrics such as SSIM,PCC,and R2 score confirm the model's efficacy,with improvements observed in multi-modality input setups.This advancement suggests potential applications in molecular dynamics,aiming for further enhancements in future studies.
基金This work was supported,in part,by the National Nature Science Foundation of China under grant numbers 62272236in part,by the Natural Science Foundation of Jiangsu Province under grant numbers BK20201136,BK20191401in part,by the Priority Academic Program Development of Jiangsu Higher Education Institutions(PAPD)fund.
文摘Gesture recognition technology enables machines to read human gestures and has significant application prospects in the fields of human-computer interaction and sign language translation.Existing researches usually use convolutional neural networks to extract features directly from raw gesture data for gesture recognition,but the networks are affected by much interference information in the input data and thus fit to some unimportant features.In this paper,we proposed a novel method for encoding spatio-temporal information,which can enhance the key features required for gesture recognition,such as shape,structure,contour,position and hand motion of gestures,thereby improving the accuracy of gesture recognition.This encoding method can encode arbitrarily multiple frames of gesture data into a single frame of the spatio-temporal feature map and use the spatio-temporal feature map as the input to the neural network.This can guide the model to fit important features while avoiding the use of complex recurrent network structures to extract temporal features.In addition,we designed two sub-networks and trained the model using a sub-network pre-training strategy that trains the sub-networks first and then the entire network,so as to avoid the subnetworks focusing too much on the information of a single category feature and being overly influenced by each other’s features.Experimental results on two public gesture datasets show that the proposed spatio-temporal information encoding method achieves advanced accuracy.
基金My sincere thanks should be expressed to Martha Michieka,associate professor at East Tennessee State University,Johnson City,TN,USA,who has provided valuable advice for me in writing this paper.
文摘Language input in foreign language classrooms is no longer restricted to one particular mode of transmission,but it can be achieved by use of different modes.Multimodality or the combination of various modes has been applied to stimulate foreign language learning.The present paper aims to verify the effectiveness of multimodal input in language learning.It demonstrates the experiment conducted in two freshman classes in a university.The two classes used the same textbook supported by audio CD containing the recorded material.The experiment class,however,additionally has some visuals in listening classes to strengthen the linguistic input.The research lasted for one semester and findings from the study show that multimodal input has significantly improved the listening and speaking abilities of the students and also has positive impacts on the learners’attitudes towards language study.
文摘A 1×8 multimode interference power splitter with multimode input/output waveguides in SOI material is designed by the beam propagation method and fabricated by the inductive coupled plasma etching technology for use in fiber optics communication systems.The fabricated device exhibits low loss and good coupling uniformity.The excess loss is lower than 0 8dB,and the uniformity is 0 45dB at the wavelength of 1550nm.Moreover,the polarization dependent loss is lower than 0 7dB at 1550nm.The device size is only 2mm×10mm.
