In this commentary, University of North Carolina's associate dean of research and assessment at the School of Education Terry Ackerman poses questions and shares his thoughts on David Thissen's essay, "Bad Questions: An Essay Involving Item Response Theory" (this issue). Ackerman begins by considering the two purposes of Item Response…

Many latent traits in social sciences display a hierarchical structure, such as intelligence, cognitive ability, or personality. Usually a second-order factor is linearly related to a group of first-order factors (also called domain abilities in cognitive ability measures), and the first-order factors directly govern the actual item responses.…

This paper focuses on two likelihood-based indices of person fit, the index "l[subscript z]" and the Snijders's modified index "l[subscript z]*". The first one is commonly used in practical assessment of person fit, although its asymptotic standard normal distribution is not valid when true abilities are replaced by sample…

The use of these three model selection information criteria for latent class models was studied for nonnested models: (1) Akaike's information criterion (H. Akaike, 1973) (AIC); (2) the Schwarz information (G. Schwarz, 1978) (SIC) criterion; and (3) the Bozdogan version of the AIC (CAIC) (H. Bozdogan, 1987). Situations in which each is preferable…

Showed how Taylor approximation can be used to generate a linear approximation to a logistic item characteristic curve and a linear ability estimator. Demonstrated how, for a specific simulation, this could result in the special case of a Robbins-Monro item selection procedure for adaptive testing. (SLD)

A set of conditions is presented for the validity of inference for Item Response Theory (IRT) models applied to data collected from examinations that allow students to choose a subset of items. Common low-dimensional IRT models estimated by standard methods do not resolve the difficult problems posed by choice-based data. (SLD)

A rigorous method is outlined for using information from a previous study and explicitly taking into account the variability of an effect size estimate when determining sample size for a chi-squared test. This approach assures that the average power of all experiments in a discipline attains the desired level. (SLD)

Proposes a mixture item response model for investigating individual differences in the selection of response categories in multiple choice items. A real data example illustrates how the model can be used to distinguish examinees disproportionately attracted to different types of distractors, and a simulation study evaluates item parameter recovery…

A model is proposed to account for publication bias explicitly using a weight function that describes probability of publication for a particular study in terms of a selection parameter. A Bayesian analysis of this model using Gibbs sampling is conducted, and the model is applied to a published meta-analysis. (SLD)

A selection model for meta-analysis is proposed that models the selection process and corrects for the consequences of selection by publication on estimates of the mean and variance of the effect parameters. Simulation studies show that the model substantially reduces bias when the model specification is correct. (SLD)

Some alternative criteria for item selection in adaptive testing are proposed that take into account uncertainty in the ability estimates. A simulation study shows that the likelihood weighted information criterion is a good alternative to the maximum information criterion. Another good alternative uses a Bayesian expected a posteriori estimator.…

Considers an approach to validating the selection of education indicators by incorporating them into a multilevel structural model and using the estimates from that model in policy-relevant simulations. The potential of this approach is demonstrated with data from the National Education Longitudinal Study of 1988. (SLD)