摘要
矩阵取样测验包含多个题册,单个题册的总分不能直接作为匹配变量用于DIF检测。本研究首先基于模拟数据,同时采用IRT_Δb法,以及用IRT模型估计的考生能力作为匹配变量修订后的LR法对矩阵取样测验进行DIF检测,分析二者进行DIF检测的有效性及其相关影响因素;并根据已有的LR法DIF判断标准划定出IRT_Δb法分类标准;最后使用实证数据加以验证。结果显示:矩阵取样测验中,IRT_Δb法和修正LR法均能较好地区分DIF量不同的题目;样本量、题册中DIF题目的比例和考生群体间真实能力的差异对两种方法的检验力、犯I类错误的概率和分类结果都有较大影响。
Matrix sampling is a useful technique widely used in large-scale educational assessments. In an assessment with matrix sampling design, each examinee takes one of the multiple booklets with partial items. A critical problem of detecting differential item functioning （DIF） in such scenario has gained a lot of attention in recent years, which is, it is not appropriate to take the observed total score obtained from individual booklet as the matching variable in detecting the DIF. Therefore, the traditional detecting methods, such as Mantel-Haenszel （MH）, SIBTEST, as well as Logistic Regression （LR） are not suitable. IRT_△b might be an alternative due to its abilities to provide valid matching variable. However, the DIF classification criterion of IRT_△b was not well established yet. Thus, the purpose of this study were： 1） to investigate the efficiency and robustness of using ability parameters obtained from Item Response Theory （IRT） model as the matching variable, comparing with the way using traditional observed raw total scores ; 2） to further identify what factors will influence the abilities in detecting DIF of two methods; 3） to propose a DIF classification criteria for IRT_△b. Simulated and empirical data were both employed in this study to explore the robustness and the efficiency of the two prevailing DIF detecting methods, which were the IRT_△b method and the adapted LR method with the estimation of group-level ability based on IRT model as the matching variable. In the Monte Carlo study, a matrix sampling test was generated, and various experimental conditions were simulated as follows： 1） different proportions of DIF items; 2） different actual examinee ability distributions; 3） different sample sizes; 4） different size of DIF. Two DIF detection methods were then applied and results were compared. In addition, power functions were established in order to derive DIF classification rule for IRT Ab based on current rules for LR. In the empirical study, through conducting a DIF analysis for American and Korean mathematics tests from Programme for International Student Assessment （PISA） 2003, the consistency of the classification rules between IRT Ab and LR were further examined. The results indicated that in the matrix sampling design, both IRT_△b method and adjusted LR method were sensitive to the diverse DIF magnitude. It was also found that the power, type I error, and the final classification of both methods were also influenced by the sample size, percentage of items with DIF, and ability differences between the focused group and the reference group. In conclusion, it was found that both the IRT_△b method and adjusted LR method can be used to detect DIF in matrix sampling tests. A classification rule for IRT_△b was proposed, which are： 0.85 between negligible DIF（A） and intermediate DIF（B）, 1.23 between intermediate DIF（B） and large DIF（C）. Meanwhile, it was suggested that researchers would take this rule as a tentative principle since the AR2 was limited between a narrow interval and the classification rule of LR was very flexible compared to classification rule of MH. Further studies could be conducted to take MH, IRT_△b as well as LR into consideration simultaneously to give more comparable and consistent classification rules for different methods.
出处
《心理学报》
CSSCI
CSCD
北大核心
2013年第8期921-934,共14页
Acta Psychologica Sinica
