Performed a meta-analysis of reliability estimates for Beck Depression Inventory (BDI) (A. Beck and others, 1961) scores across studies. Only 7.5% of studies reviewed (n=1,200) reported meaningful reliability estimates. Analysis results suggest that standard errors of measurement should be considered in addition to reliability estimates when…

Effects of item parameters on ability estimation were investigated through Monte Carlo studies using the Expected-A-Posteriori estimation. Results show a significant effect of item discriminating parameter on standard error of ability estimation. As the discriminating parameter increases, the standard error decreases. (SLD)

Sample sizes (n=15, 25, 50, 100) were examined (1,000 times each) with replacement from each of 5 data files from teacher subject test areas to compare statistical bias and standard error. Trivial levels of equating bias were present with small samples, and substantial increases in standard errors were found as the sample size decreased. (MAK)

Critiques H. W. Marsh and K.-T. Hau's (1996) assertion that parsimony is not always desirable when assessing model-fit on a particular counterexample drawn from Marsh's previous research. This counterexample is neither general nor valid enough to support such a thesis and it signals an oversight of extant, stochastic models justifying correlated…

In response to H. W. Marsh and K.-T. Hau's (1996) article on the potential for inferential errors when parsimony is rewarded in the evaluation of overall fit of structural equation models, a design-sensitive adjustment to the standard parsimony ratio is proposed. This ratio renders a more reasonable upper bound than does the standard parsimony…

Suggests that there is a tendency in social science and program evaluation to adhere to some methodological practices by force of custom rather than because of their reasoned applicability. These ideas include regression artifacts, random measurement error, and change or gain scores. (Author/SLD)

Agrees with L. Mohr that researchers are too quick to assume that measurement error is random, but disagrees that the idea of regression toward the mean has been a distraction and the notion that change scores analysis should be avoided in favor of regression analysis. (SLD)

Responds to C. S. Reichardt's discussion of regression artifacts, random measurement error, and change scores. Emphasizes that attention to regression artifacts in program evaluation is almost bound to be problematic and proposes some arguments in support of this position. (SLD)

A general approach to the analysis of nominal-scale ratings is discussed that is based on a simple measurement error model for a rater's judgments. The basic measurement error model gives rise to an agreement model for the agreement matrix of two or more raters. (SLD)

Studied the appropriateness and implications of incorporating a testlet definition into the estimation of procedures of the conditional standard error of measurement (SEM) for tests composed of testlets. Simulation results for several methods show that an item-based method using a generalizability theory model provided good estimates of the…

Studied incorporating adjusted model fit information into the root mean square error of approximation fit index (RMSEA). Monte Carlo simulation results show that incorporating robust information into the RMSEA may yield improved performance for assessing model fit under nonnormal data situations. (SLD)

Describes iterative and noniterative Rasch parameter estimation methods. Discusses various approaches to estimation and aspects of estimation error and standard errors of measures. Discusses conditions that result in statistically equivalent estimates. (SLD)

Studied the accuracy and trends of deviation noted in undergraduates' self-reported Scholastic Assessment Test (SAT) and grade point average (GPA) values. Results for 89 undergraduates show that students had highly reliable ratings of cumulative GPA, but the overall accuracies of self-reported SAT scores were considerably lower than the accuracy…

Provides and illustrates a method to compute the expected number of misclassifications of examinees using three-parameter item response theory and two state classifications (mastery or nonmastery). The method uses the standard error and the expected examinee ability distribution. (SLD)

Multidimensional unfolding methods suffer from the degeneracy problem in almost all circumstances. Most degeneracies are easily recognized: the solutions are perfect but trivial, characterized by approximately equal distances between points from different sets. A definition of an absolutely degenerate solution is proposed, which makes clear that…