摘要
提出了一种基于双层码本的语音驱动视觉语音合成系统,该系统以矢量量化的思想为基础,建立语音特征空间到视觉语音特征空间的粗耦合映射关系。为加强语音和视觉语音的关联性,系统分别根据语音特征与视觉语音特征的相似性两次对样本数据进行自动聚类,构造同时反映语音之间与视觉语音之间相似性的双层映射码本。数据预处理阶段,提出一种能反映视觉语音几何形状特征与牙齿可见度的联合特征模型,并在语音特征LPCC及MFCC基础上采用遗传算法提取视觉语音相关的语音特征模型。合成的视频中图像数据与原始视频中图像数据的比较结果表明,合成结果能在一定程度上逼近原始数据,取得了很好的效果。
The paper proposed a hi-level codebook based speech-driven visual-speech synthesis system. The system uses the vector quantization principle to establish a coarse-coupling mapping relationship from the speech feature space to the visual speech feature space, In order to enhance the relationship between the speech and the visual speech, the system makes the unsupervising-clustering on the sample data according to the similarity of both the acoustic speech and the visual speech and constructs the hi-level mapping codebook reflecting the similarity of both the acoustic speech and the visual speech. At the stage of preproeessing, the paper proposed a joint feature model, which reflects the geometric character and the visibility of teeth. The paper also proposed an approach to extract the visual speech correlative speech feature from the speech features of LPCC and MFCC on the basis of genetic algorithm. The comparison results between the synthesis image sequences with the original one show that the synthesis one can approximate the original one and the result is good. In the future research, the restriction between the visual speech contexts should be considered to im- prove the smoothness of the synthesis results.
出处
《计算机科学》
CSCD
北大核心
2014年第1期100-104,共5页
Computer Science
基金
国家自然科学基金(61070117)
北京市自然科学基金(4122004)资助
关键词
双层码本
视觉语音合成
视觉语音特征
语音特征
Bi-level codebook, Visual speech synthesis, Visual speech feature, Speech feature
