Comparison of Methods for Detecting Differential Distractor Functioning

This study examined the effectiveness of the odds-ratio method (Penfield, 2008) and the multinomial logistic regression method (Kato, Moen, & Thurlow, 2009) for measuring differential distractor functioning (DDF) effects in comparison to the standardized distractor analysis approach (Schmitt & Bleistein, 1987). Students classified as participating in free and reduced-price lunch programs served as the focal group and students not participating in these programs served as the reference group. The comparisons were conducted in such a way as to provide insight into two research questions: 1) whether the magnitude and pattern of the DDF effect is constant across all methods, and 2) whether the pattern of DDF effects support differential item functioning (DIF) findings. Measures of effect size are reported. In addition, the relationship between item characteristics and DIF and DDF effects were explored for patterns. Comparisons of three methods for detecting DDF were conducted in this study. The standardized distractor analysis and odds-ratio methods for detecting DDF were found to have very highly related results, with regard to both the magnitude and pattern of DDF effects. The multinomial logistic regression DDF results also were highly related to the standardized distractor analysis approach, but yielded slightly different patterns across distractors. The odds ratio and multinomial logistic regression methods are easily implemented with available software, such as the SPSS software package used in this study, unlike the standardized distractor analysis method which must be programmed. Despite these and the other discussed differences, all three methods present a viable option for use in improving test items included in statewide assessment programs.