Bayesian Hierarchical Mixture Approach to Model Timing Data with Application to Writing Assessment

Deane (2011) proposed a multi-layer cognitive writing model. The 2009 Cognitively Based Assessment of, for, and as learning (CBAL) Writing pilot assessment was designed to support that multi-layer model of writing. One of the features of the assessment was that the keystroke activity of students writing essays were logged to computer files. The preliminary analysis (Almond, et al., 2012) developed an algorithm to classify the pauses in writing based on the keystroke and suggested the distribution for pause events is a mixture of lognormal distribution. This early research was a promising effort to tie the mixture components to the layers in the multi-layer writing model. However, the preliminary analysis with sample size of 68 needed to be repeated with the large data set. Moreover, the data needed to be hierarchically modeled so that the data can fit mixture components when the number of pause events is small per essay. To address these problems, the first part of this thesis aims to fit the large data set (CBAL Writing pilot 2009) in a mixture of lognormal distribution. Then, a distributional analysis was carried out to evaluate the fit of the model to the data. The result confirms the preliminary analysis result. Namely, the two-component mixture model provides an adequate description of the data. The second part of the thesis aims to estimate mixture parameters by using the Bayesian hierarchical model suggested in the preliminary analysis (Almond, et al., 2012). In the Bayesian framework, the hierarchical model is useful when the number of observed pause events is small per essay. Therefore, the proposed model becomes a mixture of k univariate lognormal heteroscedastic components in the Bayesian hierarchical framework. The evaluation of Bayesian model implementation primarily requires the convergence of the Markov chain Monte Carlo (MCMC) sampler. Diagnostic tools were used to assess the convergence of the MCMC sampler. The results show that the MCMC sampler converged for both the two- and the three-component models.