Teacher emotion recognition(TER)has a significant impact on student engagement,classroom atmosphere,and teaching quality,which is a research hotspot in the smart education area.However,existing studies lack high-quali...Teacher emotion recognition(TER)has a significant impact on student engagement,classroom atmosphere,and teaching quality,which is a research hotspot in the smart education area.However,existing studies lack high-quality multimodal datasets and neglect common and discriminative features of multimodal data in emotion expression.To address these challenges,this research constructs a multimodal TER dataset suitable for real classroom teaching scenarios.TER dataset contains a total of 102 lessons and 2,170 video segments from multiple educational stages and subjects,innovatively labelled with emotional tags that characterize teacher‒student interactions,such as satisfaction and questions.To explore the characteristics of multimodal data in emotion expression,this research proposes an emotion dual-space network(EDSN)that establishes an emotion commonality space construction(ECSC)module and an emotion discrimination space construction(EDSC)module.Specifically,the EDSN utilizes central moment differences to measure the similarity to assess the correlation between multiple modalities within the emotion commonality space.On this basis,the gradient reversal layer and orthogonal projection are further utilized to construct the EDSC to extract unique emotional information and remove redundant information from each modality.Experimental results demonstrate that the EDSN achieves an accuracy of 0.770 and a weighted F1 score of 0.769 on the TER dataset,outperforming other comparative models.展开更多
基金supported by the National Natural Science Foundation of China(Grant Nos.62377007 and 62407009)the Chongqing University Graduate Education Teaching Reform Research Key Project,China(Grant No.232073)+1 种基金the Scientific and Technological Research Program of Chongqing Municipal Education Commission,China(Grant Nos.KJZD-M202400606 and KJZD-M202300603)the Chongqing Natural Science Foundation Joint Key Project for Innovation and Development,China(Grant No.2024NSCQ-LZX0057).
文摘Teacher emotion recognition(TER)has a significant impact on student engagement,classroom atmosphere,and teaching quality,which is a research hotspot in the smart education area.However,existing studies lack high-quality multimodal datasets and neglect common and discriminative features of multimodal data in emotion expression.To address these challenges,this research constructs a multimodal TER dataset suitable for real classroom teaching scenarios.TER dataset contains a total of 102 lessons and 2,170 video segments from multiple educational stages and subjects,innovatively labelled with emotional tags that characterize teacher‒student interactions,such as satisfaction and questions.To explore the characteristics of multimodal data in emotion expression,this research proposes an emotion dual-space network(EDSN)that establishes an emotion commonality space construction(ECSC)module and an emotion discrimination space construction(EDSC)module.Specifically,the EDSN utilizes central moment differences to measure the similarity to assess the correlation between multiple modalities within the emotion commonality space.On this basis,the gradient reversal layer and orthogonal projection are further utilized to construct the EDSC to extract unique emotional information and remove redundant information from each modality.Experimental results demonstrate that the EDSN achieves an accuracy of 0.770 and a weighted F1 score of 0.769 on the TER dataset,outperforming other comparative models.
