While it is imperative that attempts be made to assess the predictive accuracy of any prediction model, traditional measures of predictive accuracy have been criticized as suffering from "the base rate problem." The base rate refers to the relative frequency of occurrence of the event being studied in the population of interest, and the…

Two types of qualitative dependent variables are presented for use in counseling research: choices from an unordered set of categorical alternatives and ordered, categorical counseling outcomes. To investigate school choice behavior, the conditional logit model and analysis are introduced. The conditional logit model can include the attributes of…

This study investigated the effect on examinees' ability estimate under item response theory (IRT) when they are presented an item, have ample time to answer the item, but decide not to respond to the item. Simulation data were modeled on an empirical data set of 25,546 examinees that was calibrated using the 3-parameter logistic model. The study…

To ensure fairness, it is of critical importance that testing programs make sure that essay items given to examinees are equivalent in difficulty. The purpose of this study was to evaluate the stability and accuracy of a logistic regression based polytomous essay difficulty index. Preliminary results from a simulation study (9 conditions with a…

This study investigated the applicability to computerized adaptive testing (CAT) data of a differential item functioning (DIF) analysis that involves an empirical Bayes (EB) enhancement of the popular Mantel Haenszel (MH) DIF analysis method. The computerized Law School Admission Test (LSAT) assumed for this study was similar to that currently…

This paper applies specific information item selection using a method developed by T. Davey and M. Fan (2000) to a multiple-choice passage-based reading test that is being developed for computer administration. Data used to calibrate the multidimensional item parameters for the simulation study consisted of item responses from randomly equivalent…

This study sought to answer the question, Do all students benefit equally from the use of a hypermedia Goal-Based Scenario (GBS)? GBS is a subcategory of anchored instruction. The correlation between the demographic variables and achievement and specific cognitive variables and achievement was explored using a lesson on DNA, and was tested on…

Monte Carlo simulation compared chi-square statistics, parameter estimates, and root mean square error of approximation values using normal and elliptical estimation methods. Three research conditions were imposed on the simulated data: sample size, population contamination percent, and kurtosis. A Bentler-Weeks structural model established the…

A simulation study was performed to determine whether a group's average percent correct in a content domain could be accurately estimated for groups taking a single test form and not the entire domain of items. Six Item Response Theory (IRT)-based domain score estimation methods were evaluated, under conditions of few items per content area per…

A procedure was developed to analyze data with missing observations by extracting data from a sparsely filled data matrix into analyzable smaller subsets of data. This subdividing method, based on the conceptual framework of meta-analysis, was accomplished by creating data sets that exhibit structural designs and then pooling variance components…

In a computerized adaptive test (CAT), it would be desirable to obtain an acceptable precision of the proficiency level estimate using an optimal number of items. Decreasing the number of items is accompanied, however, by a certain degree of bias when the true proficiency level differs significantly from the a priori estimate. G. Raiche (2000) has…

Whether one should use null hypothesis testing, confidence intervals, and/or effect sizes is a source of continuing controversy in educational research. An alternative to testing for statistical significance, known as equivalence testing, is little used in educational research. Equivalence testing is useful in situations where the researcher…

The accuracy of Gibbs sampling, a Markov chain Monte Carlo procedure, was considered for estimation of item and ability parameters under the two-parameter logistic model. Memory test data were analyzed to illustrate the Gibbs sampling procedure. Simulated data sets were analyzed using Gibbs sampling and the marginal Bayesian method. The marginal…

This material provides students with information to prepare for a mock trial. The defendant in this case has been accused of the crime of driving under the influence of alcoholic beverages causing severe bodily injury. Case materials include stipulated facts, jury instructions, depositions, and other related materials. (EH)

This paper suggests that statistical significance testing and effect size are two sides of the same coin; they complement each other, but do not substitute for one another. Good research practice requires that both should be taken into consideration to make sound quantitative decisions. A Monte Carlo simulation experiment was conducted, and a…