Although often ignored, reliability is critical when interpreting study effects and test results. Accordingly, this article focuses on the most commonly used estimate of reliability, internal consistency coefficients, with emphasis on coefficient alpha. An interpretive framework is provided for applied researchers and others seeking a conceptual…

Compared bootstrap standard errors of five item response theory (IRT) equating methods for the common-item nonequivalent groups design using test results for 1,493 and 1,793 examinees taking a professional certification test. Results suggest that standard errors of equating less than 0.1 standard deviation units could be obtained with any of the…

A decision-theoretic approach to the question of reliability in categorically scored examinations is explored. The concepts of true scores and errors are discussed as they deviate from conventional psychometric definitions and measurement error in categorical scores is cast in terms of misclassifications. A reliability measure based on…

A widely and readily applicable covariance structure modeling approach is outlined that allows point and interval estimation of scale reliability with fixed components. The procedure employs only linear constraints introduced in a congeneric model, which after reparameterization permit expression of composite reliability as a function of…

The phenomenon of nonlinear resonance (sometimes called the "jump phenomenon") is examined and second-order van der Pol plane analysis is employed to indicate that this phenomenon is not a feature of the equation, but rather the result of accumulated round-off error, truncation error and algorithm error that distorts the true bounded solution onto…

Past estimates of the effect of family income on child development have often been plagued by endogeneity and measurement error. In this paper, we use two simulated instrumental variables strategies to estimate the causal effect of income on children's math and reading achievement. Our identification derives from the large, non-linear changes…

Full account of the latent regression model for the National Assessment of Educational Progress is given. The treatment includes derivation of the EM algorithm, Newton-Raphson method, and the asymptotic standard errors. The paper also features the use of the adaptive Gauss-Hermite numerical integration method as a basic tool to evaluate…

The procedures required to obtain the approximate posterior standard deviations of the parameters in the three commonly used item response models for dichotomous items are described and used to generate values for some common situations. The results were compared with those obtained from maximum likelihood estimation. It is shown that the use of…

"Choice and Education across the States," published by the Heartland Institute, is an advocacy document that assigns letter grades to states based on the extensiveness of each state's school choice system. The report asserts, based on a faulty use of past research, that an increase in school choice will strengthen accountability and improve…

In this study, the authors compare the validity of three nonverbal tests for the purpose of identifying academically gifted English-language learners (ELLs). Participants were 1,198 elementary children (approximately 40% ELLs). All were administered the Raven Standard Progressive Matrices (Raven), the Naglieri Nonverbal Ability Test (NNAT), and…

Confirmatory factor analytic (CFA) procedures can be used to provide evidence of measurement invariance. However, empirical evaluation has not focused on the accuracy of common CFA steps used to detect a lack of invariance across groups. This investigation examined procedures for detection of test structure differences across groups under several…

This study is a partial replication of L. Hu and P. M. Bentler's (1999) fit criteria work. The purpose of this study was twofold: (a) to determine whether cut-off values vary according to which model is the true population model for a dataset and (b) to identify which of 13 fit indexes behave optimally by retaining all of the correct models while…

This study evaluated the impact of unequal reliability on test equating methods in the nonequivalent groups with anchor test (NEAT) design. Classical true score-based models were compared in terms of their assumptions about how reliability impacts test scores. These models were related to treatment of population ability differences by different…

Short tests containing at most 15 items are used in clinical and health psychology, medicine, and psychiatry for making decisions about patients. Because short tests have large measurement error, the authors ask whether they are reliable enough for classifying patients into a treatment and a nontreatment group. For a given certainty level,…

Evidence-based practice approaches to interventions has come of age and promises to provide a new standard of excellence for school psychologists. This article describes several definitions of evidence-based practice and the problems associated with traditional statistical analyses that rely on rejection of the null hypothesis for the…