In this paper, the SOLO taxonomy is used to identify different levels of student understanding of the statistical concepts associated with sampling distribution. This study was part of a research project investigating students' conceptual understanding of concepts of hypothesis testing taught with the support of simulation learning activities. The…

Policymakers are increasingly interested in the extent to which experimental results generalize from a sample to a population of inference. When the sample is not randomly selected, propensity score methods are used to reweight the sample. Subclassification by propensity score is commonly used in which the population is partitioned into strata…

Classification evaluation metrics are often used to evaluate adaptive tutoring systems-- programs that teach and adapt to humans. Unfortunately, it is not clear how intuitive these metrics are for practitioners with little machine learning background. Moreover, our experiments suggest that existing convention for evaluating tutoring systems may…

This paper reports the results of a research about the strategies and difficulties developed by university students in the process of modeling and simulating of random phenomena in an environment of a spreadsheet. The results indicate that students had difficulties to identify key components of the problems, which are crucial to formulate a…

The Monte Carlo study compared the performance of predictive discriminant analysis (PDA) and that of logistic regression (LR) for the two-group classification problem. Prior probabilities were used for classification, but the cost of misclassification was assumed to be equal. The study used a fully crossed three-factor experimental design (with…

The purpose of this study was to estimate the extent to which recording variability among standardized patients (SPs) has an impact on classification consistency with data sets simulated to reflect performances on a large-scale clinical skills examination. SPs are laypersons trained to portray patients in clinical encounters (cases) and to record…

Two types of qualitative dependent variables are presented for use in counseling research: choices from an unordered set of categorical alternatives and ordered, categorical counseling outcomes. To investigate school choice behavior, the conditional logit model and analysis are introduced. The conditional logit model can include the attributes of…

Cut scores, quartile ranking, sample size, and overall classification scheme were studied as personnel selection procedures in two samples. The first was 120 simulated observations of employee scores based on actual selection procedures for applicants for administrative assistant positions. The other sample was composed of test results for 73…

This study focused on the robustness of unidimensional item response theory (UIRT) models in computerized classification testing against violation of the unidimensionality assumption. The study addressed whether UIRT models remain acceptable under various testing conditions and dimensionality strengths. Monte Carlo simulation techniques were used…

This paper proposes a new Information-Time index as the basis for item selection in computerized classification testing (CCT) and investigates how this new item selection algorithm can help improve test efficiency for item pools with mixed item types. It also investigates how practical constraints such as item exposure rate control, test…

This paper discusses the principles underlying discriminant analysis and constructs a simulated data set to illustrate its methods. Discriminant analysis is a multivariate technique for identifying the best combination of variables to maximally discriminate between groups. Discriminant functions are established on existing groups and used to…

This paper describes a study, now in progress, of new methods for representing the sampling variability of Mantel-Haenszel differential item functioning (DIF) results, based on the system for categorizing the severity of DIF that is now in place at the Educational Testing Service. The methods, which involve a Bayesian elaboration of procedures…

This study compared the model of J. E. Grizzle, C. F. Starmer, and G. G. Koch (GSK, 1969) and log-linear model-based approaches for testing hypotheses in r x c contingency tables. Tables were simulated under various conditions of table, sample, row-effect size, and column-effect size. Test statistics for column (main) and interaction effects were…

Many state and local entities are developing and using performance assessment programs. Because these initiatives are so diverse, it is very difficult to understand what they are doing, or to compare them in any meaningful way. Multiple-choice tests are contrasted with performance assessments, and preliminary classifications are suggested to…