摘要
通过单一模态数据对教师的课堂行为进行分析存在视角单一、结果不全面等问题。为解决该问题,文章以国际中文教师为研究对象,提出一种多模态数据驱动的课堂行为分析框架,并实现对中文教师的语言、表情、姿态三个维度的课堂行为分析。采用基于BERT-ResNet50的融合方法提取教师语言和面部表情特征进行多模态的教师课堂情感分析,其中BERT模型用来对文本进行编码,ResNet50模型用来对面部图像进行编码,再通过Transformer编码器进行特征融合,将来自面部表情和文本的两个独立特征进行联合编码。实验结果表明:相对于对语言、面部表情进行单模态的数据分析,模型的准确率分别提升了17.76%和16.39%;采用YOLOv8n模型分析教师姿态,在测试集上的平均精度均值(mean Average Precision,mAP)达到90%。最后将分析结果以可视化的形式呈现给教师,同时接入DeepSeek大模型生成分析报告,进而帮助国际中文教师可以及时改进自身的课堂行为。
The analysis of teachers'classroom behaviors through single-modal data has problems such as a single perspective and incomplete results.To solve this problem,we take international Chinese language teachers as the research object,propose a multimodal data-driven classroom behavior analysis framework,and realize the classroom behavior analysis of Chinese language teachers in three dimensions:language,expression and posture.The fusion method based on BERT-ResNet50 is adopted to extract the language and facial expression features of teachers for multimodal classroom sentiment analysis of teachers.Among them,the BERT model is used to encode the text,and the ResNet50 model is used to encode the facial images.Then,feature fusion is carried out through the Transformer encoder.The two independent features from facial expressions and text are jointly encoded.The experimental results show that compared with the single-modal data analysis of language and facial expressions,the accuracy of the model has increased by 17.76%and 16.39%respectively.The YOLOv8n model was adopted to analyze the postures of teachers,and the mean Average Precision(mAP)on the test set reached 90%.Finally,the analysis results are presented to the teachers in a visual form,and at the same time,the DeepSeek large model is connected to generate the analysis report,thereby helping international Chinese language teachers to improve their classroom behaviors in a timely manner.
作者
景宏伟
徐娟
JING Hong-wei;XU Juan(School of Information Science,Beijing Language and Culture University,Beijing 100083,China)
出处
《计算机技术与发展》
2025年第12期58-66,共9页
Computer Technology and Development
基金
国家社会科学基金项目(24BYY044)
教育部中外语言合作交流中心国际中文教育研究课题重大项目(24YH03A)
北京语言大学研究生创新基金项目(25YCX117)。
关键词
人工智能
多模态数据
情感分析
国际中文教育
教师行为分析
DeepSeek
artificial intelligence
multimodal data
sentiment analysis
international Chinese language education
teacher behavior analysis
DeepSeek
