Evaluating the Effectiveness of the Expectation-Maximization (EM) Algorithm for Bayesian Network Calibration

Educators use various statistical techniques to explain relationships between latent and observable variables. One way to model these relationships is to use Bayesian networks as a scoring model. However, adjusting the conditional probability tables (CPT-parameters) to fit a set of observations is still a challenge when using Bayesian networks. A CPT provides the conditional probabilities of a single discrete variable with respect to other discrete variables. In general Bayesian networks, the CPTs that link the proficiency variable and observable outcomes are not necessarily monotonic, but they are often constrained to be monotonic in educational applications. The monotonicity constraint states that if an examinee shows an improvement on a proficiency variable (parent variable), the individual performance on an observable (child variable) should improve. For example, if a student has a higher writing skill, then this student is likely to score better on an essay task. For educational research, building parametric models (i.e., DiBello models) with the Expectation-Maximization algorithm provides monotonic conditional probability tables (CPT). This dissertation explored the effectiveness of the EM algorithm within the DiBello parameterization under different sample sizes, test forms, and item structures. The data generation model specifies two skill variables with a different number of items depending on the test forms. The outcome measures were the relative bias of the parameters to assess parameter recovery, Kullback-Leibler distance to evaluate the distance between CPTs, and Cohen's κ to assess classification agreement between data generation and estimation models. The simulation study results showed that a minimum sample size of 400 was sufficient to produce acceptable parameter bias and KL distance. A balanced distribution of simple and integrated type items produced less bias compared to an unbalanced item distribution. The parameterized EM algorithm stabilized the estimates for cells small sizes in CPTs, providing minimal KL distance values. However, the classification agreement between generated and estimated models was low.