摘要
Q矩阵是认知诊断的核心,专家构建的Q矩阵通常存在一定错误,会降低估计精度,因而需要进行修正。采用随机森林(random forest,RF)算法,以PVAF、对数似然值、改造后的R统计量为特征训练模型,提出了机器学习视角的Q矩阵修正新方法(RF-P、RF-L、RF-R),并开展模拟与实证研究验证性能。研究结果表明:(1)三种模型的准确率、召回率、精确率、F1、Kappa等评估指标均在0.75以上;(2)模拟研究中,基于三种随机森林模型的新方法在整体上均比最新发表的Wald-XPD法有更好的修正表现,其中以RF-R表现最佳;(3)实证研究中,RF-R修正出了更合理的Q矩阵,有最优的模型-数据拟合结果。
Q-matrix is the core of cognitive diagnosis,and the Q-matrix constructed by experts usually has certain misspecifications,which will reduce the estimation accuracy and thus needs to be validated.New machine learning-based Q-matrix validation methods(RF-P,RF-L,and RF-R)is proposed using the random forest(RF)algorithm with PVAF,log-likelihood,and modified R statistics as the feature training models,and simulation and empirical studies are conducted to verify the performance.The results show that(1)the accuracy,recall,precision,F1,Kappa of the three models are above 0.75;(2)in the simulation study,the new methods based on the three RF models have better validation performance than the Wald-XPD method,which was latest published,among which RF-R has the best performance;(3)in the empirical study,RF-R suggested a more reasonable Q matrix with optimal model-data fitting results.
作者
秦海江
郭磊
QIN Haijiang;Guo Lei(Faculty of Psychology,Southwest University,Chongqing 400715,China;Southwest University Branch,Collaborative Innovation Center of Assessment toward Basic Education Quality,Chongqing 400715,China)
出处
《心理技术与应用》
2023年第11期685-704,共20页
Psychology(Techniques and Applications)
基金
国家自然科学基金青年项目(31900793)
中央高校基本科研业务费专项资金(SWU2109222)
西南大学2035先导计划项目(SWUPilotPlan006)。
