In selected-response assessments such as attitude surveys with Likert-type rating scales, examinees often select from rating scale categories to reflect their locations on a construct. Researchers have observed that some examinees exhibit "response styles," which are systematic patterns of responses in which examinees are more likely to…

BASIE is a framework for interpreting impact estimates from evaluations. It is an alternative to null hypothesis significance testing. This guide walks researchers through the key steps of applying BASIE, including selecting prior evidence, reporting impact estimates, interpreting impact estimates, and conducting sensitivity analyses. The guide…

In psychological research, one often aims at explaining individual differences in S-R profiles, that is, individual differences in the responses (R) with which people react to specific stimuli (S). To this end, researchers often postulate an underlying sequential process, which boils down to the specification of a set of mediating variables (M)…

Orlando and Thissen (2000) developed an item fit statistic for binary item response theory (IRT) models known as S-X[superscript 2]. This article generalizes their statistic to polytomous unfolding models. Four alternative formulations of S-X[superscript 2] are developed for the generalized graded unfolding model (GGUM). The GGUM is a…

The performance of five simple multiple imputation methods for dealing with missing data were compared. In addition, random imputation and multivariate normal imputation were used as lower and upper benchmark, respectively. Test data were simulated and item scores were deleted such that they were either missing completely at random, missing at…

This article introduces a bootstrap generalization to the Modified Parallel Analysis (MPA) method of test dimensionality assessment using factor analysis. This methodology, based on the use of Marginal Maximum Likelihood nonlinear factor analysis, provides for the calculation of a test statistic based on a parametric bootstrap using the MPA…

Cluster randomized trials (CRTs) have been widely used in field experiments treating a cluster of individuals as the unit of randomization. This study focused particularly on situations where CRTs are accompanied by a common complication, namely, treatment noncompliance or, more generally, intervention nonadherence. In CRTs, compliance may be…

Recent work has shown that multidimensionally scoring responses from different tests can provide better ability estimates. For educational assessment data, applications of this approach have been limited to binary scores. Of the different variants, the de la Torre and Patz model is considered more general because implementing the scoring procedure…

Numerous studies have shown increasing item reliabilities as an effect of the item position in personality scales. Traditionally, these context effects are analyzed based on item-total correlations. This approach neglects that trends in item reliabilities can be caused either by an increase in true score variance or by a decrease in error…

Requirements for the games and simulations available in economics education surveyed range from simple equipment to computer facilities. Some attempts at game and simulation evaluation are summarized and criticized in general and in the context of economics education. Comments on the evolving literature on principles of game design are included.…

Discussion of difficulties in evaluating simulations and games as teaching techniques includes range of claims made for the techniques; variety available; lack of specificity of technique objectives at design stage; limitations of objective-oriented research; inherent difficulties of comparative evaluation against alternative techniques; and…

This study presents an overview of Monte Carlo studies in discriminant analysis. Some common questions about the use of Monte Carlo techniques are answered through a brief literature review of articles on discriminant analysis in which Monte Carlo methods are used. The articles cover many research points, such as comparing error rate estimates,…

The purpose of this research is to demonstrate that a systematic approach to the graphical analysis of Rasch model residuals can lead to an increased understanding of ordered response data, and that residual patterns do change in predictable ways, and that summary statistics need not be the only piece of evidence for assuring the fit between model…

Sampling distributions for ten tests for comparing population variances in a two group design were generated for several combinations of equal and unequal sample sizes, population means, and group variances when distributional forms differed. The ten procedures included: (1) O'Brien's (OB); (2) O'Brien's with adjusted degrees of freedom; (3)…

This report for researchers in rural sociology develops a conceptual and analytic framework for evaluating policies or programs through interrelated models. Three models--structural analysis, control programming, and simulation--allow researchers to assess policy/program impact, determine why expected impacts did or did not occur, and modify…