This paper presents results from a randomized experimental design replicated over four semesters that compared students' performance in understanding landform evolution processes as measured by the pretest to posttest score growth between two treatment methods: an online interactive simulation tool and a paper-based exercise. While both methods…

Common items with inconsistent b-parameter estimates may have a serious impact on item response theory (IRT)--based equating results. To find a better way to deal with the outlier common items with inconsistent b-parameters, the current study investigated the comparability of 10 variations of four IRT-based equating methods (i.e., concurrent…

Drawing on game-design principles and an underlying situated theoretical perspective, we developed and researched a 3D game-based curriculum designed to teach water quality concepts. We compared undergraduate student dyads assigned randomly to four different instructional design conditions where the content had increasingly level of…

Exact testing was evaluated as a method for conducting Mantel-Haenszel differential item functioning (DIF) analyses with relatively small samples. A series of computer simulations found that the asymptotic Mantel-Haenszel and the exact method yielded very similar results across sample size, levels of DIF, and data sets. (SLD)

Comparisons are made between BILOG version 2.2 and LOGIST 5.0 version 2.5 in estimating the item parameters, traits, item characteristic functions, and test characteristic functions for the three-parameter logistic model. Speed and accuracy are reported for a number of 10, 20, and 40-item tests. (Author/GDC)

A Monte Carlo simulation was used to compare the Mahalanobis "D" Squared and the Comrey "Dk" methods of detecting outliers in data sets. Under the conditions investigated, the "D" Squared technique was preferable as an outlier removal statistic. (SLD)

This study applied the maximum expected information (MEI) and the maximum posterior-weighted information (MPI) approaches of computer adaptive testing item selection to the case of a test using polytomous items following the partial credit model. The MEI and MPI approaches are described. A simulation study compared the efficiency of ability…

This simulation investigated two procedures that reduce differences between paper-and-pencil testing and computerized adaptive testing (CAT) by making CAT content sensitive. Results indicate that the price in terms of additional test items of using constrained CAT for content balancing is much smaller than that of using testlets. (SLD)

This paper develops and applies three nonparametric comparisons of the shapes of two item characteristic surfaces: (1) proportional latent odds; (2) uniform relative difficulty; and (3) item sensitivity. A method is presented for comparing the relative shapes of two item characteristic curves in two examinee populations who were administered an…

In this study, the effects of different types of support for learning from multiple representations in a simulation-based learning environment were examined. The study extends known research by examining the use of dynamic representations instead of static representations and it examines the role of the complexity of the domain and the learning…

Two order theoretic techniques were presented and compared. Ordering theory of Krus and Bart (1974) and an extended Takeya's item relational structure analysis (IRS) by Tatsuoka and Tatsuoka (1981) were used to extract the hierarchical item structure from three datasets. Directed graphs were constructed and both methods were assessed as to how…

Three heuristic methods of assessing the dimensionality of binary item pools were evaluated in a Monte Carlo investigation. The indices were based on (1) the local independence of unidimensional tests; (2) patterns of second-factor loadings derived from simplex theory; and (3) the shape of the curve of successive eigenvalues. (SLD)

The purpose of this study was to compare two non-parametric procedures, the Mantel-Haenszel (MH) procedure and the simultaneous item bias (SIB) procedure, with respect to their Type I error rates and power, and to investigate the conditions under which asymptotic distributional properties of the SIB and MH were obtained. Data were simulated to…

Responses to a sample test were simulated for examinees under free-response and multiple-choice formats. Test score sets were correlated with randomly generated sets of unit-normal measures. The extent of superiority of free response tests was sufficiently small so that other considerations might justifiably dictate format choice. (Author/DWH)

Using one simulated and two real data sets, the effects of the systematic variation of the item-selection procedure and the stepsize method on the operating characteristics of computerized adaptive testing (CAT) for instruments with polychotomously scored rating scale items were studied. The six rating scale CAT procedures used performed well.…