Realtime speech communications require high efficient compression algorithms to encode speech signals. As the compressed speech parameters are highly sensitive to transmission errors, robust source and channel decodin...Realtime speech communications require high efficient compression algorithms to encode speech signals. As the compressed speech parameters are highly sensitive to transmission errors, robust source and channel decoding and demodulation schemes are both important and of practical use. In this paper, an it- erative joint souree-channel decoding and demodulation algorithm is proposed for mixed excited linear pre- diction （MELP） vocoder by both exploiting the residual redundancy and passing soft information through- out the receiver while introducing systematic global iteration process to further enhance the performance. Being fully compatible with existing transmitter structure, the proposed algorithm does not introduce addi- tional bandwidth expansion and transmission delay. Simulations show substantial error correcting perfor- mance and synthesized speech quality improvement over conventional separate designed systems in delay and bandwidth constraint channels by using the joint source-channel decoding and demodulation （JSCCM） algorithm.展开更多
针对单通道语音增强中主流编解码结构面临的声学特征提取不充分、通道信息丢失和幅度相位补偿困难等问题,提出一种融合不同维度语音特征的异构双分支解码单通道语音增强模型——HDBMV(Heterogeneous DualBranch with Multi-View)。该模...针对单通道语音增强中主流编解码结构面临的声学特征提取不充分、通道信息丢失和幅度相位补偿困难等问题,提出一种融合不同维度语音特征的异构双分支解码单通道语音增强模型——HDBMV(Heterogeneous DualBranch with Multi-View)。该模型通过信息融合编码器(IFE)、时频残差Conformer(TFRC)模块、多视角注意力(MVA)模块和异构双分支解码器(HDBD)等机制,提升单通道语音增强的性能。首先,IFE联合处理振幅与复数特征,捕捉全局依赖和局部相关,生成紧凑的特征表示;其次,TFRC模块有效捕捉时间维度和频域维度上的相关性,同时降低计算复杂度;再次,MVA模块重构通道域和时频域信息,进一步增强模型对信息的多视角多层次的表征能力;最后,HDBD分别处理幅度特征和细化复数特征,解决幅度相位补偿问题,提升解码鲁棒性。实验结果表明,HDBMV在公开数据集VoiceBank+DEMAND、大数据集DNS Challenge 2020和自建的藏语数据集BodSpeDB上的语音质量感知评估(PESQ)分别达到了3.00、3.12和2.09,短时目标可理解度(STOI)分别达到了0.96、0.97和0.81。可见,HDBMV以最小的参数量和较高的计算效率获得了最佳的语音增强性能和较强的泛化能力。展开更多
Lip-reading technology,based on visual speech decoding and automatic speech recognition,offers a promising solution to overcoming communication barriers,particularly for individuals with temporary or permanent speech ...Lip-reading technology,based on visual speech decoding and automatic speech recognition,offers a promising solution to overcoming communication barriers,particularly for individuals with temporary or permanent speech impairments.However,most Visual Speech Recognition(VSR)research has primarily focused on the English language and general-purpose applications,limiting its practical applicability in medical and rehabilitative settings.This study introduces the first Deep Learning(DL)based lip-reading system for the Italian language designed to assist individuals with vocal cord pathologies in daily interactions,facilitating communication for patients recovering from vocal cord surgeries,whether temporarily or permanently impaired.To ensure relevance and effectiveness in real-world scenarios,a carefully curated vocabulary of twenty-five Italian words was selected,encompassing critical semantic fields such as Needs,Questions,Answers,Emergencies,Greetings,Requests,and Body Parts.These words were chosen to address both essential daily communication and urgent medical assistance requests.Our approach combines a spatiotemporal Convolutional Neural Network(CNN)with a bidirectional Long Short-Term Memory(BiLSTM)recurrent network,and a Connectionist Temporal Classification(CTC)loss function to recognize individual words,without requiring predefined words boundaries.The experimental results demonstrate the system’s robust performance in recognizing target words,reaching an average accuracy of 96.4%in individual word recognition,suggesting that the system is particularly well-suited for offering support in constrained clinical and caregiving environments,where quick and reliable communication is critical.In conclusion,the study highlights the importance of developing language-specific,application-driven VSR solutions,particularly for non-English languages with limited linguistic resources.By bridging the gap between deep learning-based lip-reading and real-world clinical needs,this research advances assistive communication technologies,paving the way for more inclusive and medically relevant applications of VSR in rehabilitation and healthcare.展开更多
基金Supported by the National Natural Science Foundation of China （No. 60572081 ）
文摘Realtime speech communications require high efficient compression algorithms to encode speech signals. As the compressed speech parameters are highly sensitive to transmission errors, robust source and channel decoding and demodulation schemes are both important and of practical use. In this paper, an it- erative joint souree-channel decoding and demodulation algorithm is proposed for mixed excited linear pre- diction （MELP） vocoder by both exploiting the residual redundancy and passing soft information through- out the receiver while introducing systematic global iteration process to further enhance the performance. Being fully compatible with existing transmitter structure, the proposed algorithm does not introduce addi- tional bandwidth expansion and transmission delay. Simulations show substantial error correcting perfor- mance and synthesized speech quality improvement over conventional separate designed systems in delay and bandwidth constraint channels by using the joint source-channel decoding and demodulation （JSCCM） algorithm.
文摘Lip-reading technology,based on visual speech decoding and automatic speech recognition,offers a promising solution to overcoming communication barriers,particularly for individuals with temporary or permanent speech impairments.However,most Visual Speech Recognition(VSR)research has primarily focused on the English language and general-purpose applications,limiting its practical applicability in medical and rehabilitative settings.This study introduces the first Deep Learning(DL)based lip-reading system for the Italian language designed to assist individuals with vocal cord pathologies in daily interactions,facilitating communication for patients recovering from vocal cord surgeries,whether temporarily or permanently impaired.To ensure relevance and effectiveness in real-world scenarios,a carefully curated vocabulary of twenty-five Italian words was selected,encompassing critical semantic fields such as Needs,Questions,Answers,Emergencies,Greetings,Requests,and Body Parts.These words were chosen to address both essential daily communication and urgent medical assistance requests.Our approach combines a spatiotemporal Convolutional Neural Network(CNN)with a bidirectional Long Short-Term Memory(BiLSTM)recurrent network,and a Connectionist Temporal Classification(CTC)loss function to recognize individual words,without requiring predefined words boundaries.The experimental results demonstrate the system’s robust performance in recognizing target words,reaching an average accuracy of 96.4%in individual word recognition,suggesting that the system is particularly well-suited for offering support in constrained clinical and caregiving environments,where quick and reliable communication is critical.In conclusion,the study highlights the importance of developing language-specific,application-driven VSR solutions,particularly for non-English languages with limited linguistic resources.By bridging the gap between deep learning-based lip-reading and real-world clinical needs,this research advances assistive communication technologies,paving the way for more inclusive and medically relevant applications of VSR in rehabilitation and healthcare.
