Differential Item Functioning Model for Testlet-Based Items Using A Bi-Factor Multidimensional Item Response Theory Model

In this dissertation, a DIF detection method for testlet-based data was developed and evaluated. The proposed DIF model is an extension of a bi-factor multidimensional IRT model for testlets. Unlike other IRT-based DIF detection models, the proposed model is capable of taking LID due to testlets into account and thus estimating DIF magnitude more accurately when a test is composed of testlets. A fully Bayesian estimation method was adopted for parameter estimation. Estimating parameters for the proposed DIF detection model with traditional MLE methods is computationally expensive since the MLE methods adopt approximation methods such as Gaussian quadrature. An MCMC estimation method implemented in this study does not have such a limitation. The proposed DIF model was evaluated by comparing parameter recovery with an IRT-based DIF model with simulated data. Simulation factors in this study were a) magnitude of testlet effect, b) magnitude of DIF, and c) magnitude of item discrimination. Three different levels were considered for the first simulation factor, while the other two factors had two levels. As a result, there were 12 simulation conditions in this study. For each simulation condition, simulation was replicated 100 times. Evaluation criteria were bias, SE, and RMSE of each parameter in the proposed DIF model, as well as DIF detection rate and DIF detection error rate. The proposed DIF model was also applied to a statewide assessment dataset to confirm the utility of the model. Educational implications and limitations of this study were also discussed.