Educational assessment data are often collected from a set of test centers across various geographic regions, and therefore the data samples contain clusters. Such cluster-based data may result in clustering effects in variance estimation. However, in many grouped jackknife variance estimation applications, jackknife groups are often formed by a…

Background: Despite the importance of lifelong learning as a key to individual and societal prosperity, we know little about adult motivation to engage in learning across the lifespan. Building on educational psychological approaches, this article presents a measure of Motivation-to-Learn using four items from the background questionnaire of the…

The purpose of this study was to evaluate the combined effects of reduced equating sample size and shortened anchor test length on item response theory (IRT)-based linking and equating results. Data from two independent operational forms of a large-scale testing program were used to establish the baseline results for evaluating the results from…

The focus of this presidential address is on the contemporary treatment of reliability and validity in educational assessment. Highlights on reliability are provided under the classical true-score model using tools from latent trait modeling to clarify important assumptions and procedures for reliability estimation. In addition to reliability,…

Two studies, both using Type 12 sampling, are presented in which the effects of violating the assumption of essential parallelism in setting confidence intervals are studied. Results indicate that as long as data manifest properties of essential parallelism, the two methods studied maintain precise Type I error control. (SLD)

Develops two procedures for estimating individual-level conditional standard errors of measurement for scale scores, assuming tests of dichotomously scored items. Compares the two procedures to a polynomial procedure and a procedure developed by L. Feldt and A. Qualls (1998) using data from the Iowa Tests of Basic Skills. Contains 22 references.…

Provides a comprehensive and integrated treatment of both conditional absolute standard errors of measurement (SEM) and conditional relative SEMs from the perspective of generalizability theory. Illustrates the approach with examples from commercial standardized tests. Examples support the conclusion that both types of conditional SEMs tend to be…

Studied sources of measurement error variance in the Teacher Efficacy Scale (TES) (Gibson and Dembo, 1984). Used reliability generalization to characterize the typical score reliability for the TES and potential sources of measurement error variance across 43 studies. Also examined related instruments for measurement integrity. (SLD)

The focus on the individual that is possible in analyzing behavioral data provides the possibility of investigating sequencing effects. Autocorrelation--as illustrated with classroom data from a previous study--can cause standard procedures to underestimate the magnitude of measurement error. Recommendations are made to reduce the effects of…

Used meta-analysis to cumulate reliabilities of personality scale scores, using 848 coefficients of stability and 1,359 internal consistency reliabilities across the Big Five factors of personality. The dimension of personality being measured does not appear to moderate strongly either internal consistency or the test-retest reliabilities.…

Describes four procedures previously developed for estimating conditional standard errors of measurement for scale scores and compares them in a simulation study. All four procedures appear viable. Recommends that test users select a procedure based on various factors such as the type of scale score of concern, test characteristics, assumptions…

Two possible standard error formulas for the polytomous differential item functioning index proposed by N. J. Dorans and A. P. Schmitt (1991) were derived. These standard errors, and associated hypothesis-testing procedures, were evaluated through simulated data. The standard error that performed better is based on N. Mantel's (1963)…

Describes the temporal consistency of school classification observed in the Kentucky, and secondarily in the Charlotte-Mecklinberg (North Carolina), school-based performance award programs. Data from the Kentucky Department of Education show the extent to which temporal inconsistency could be due to measurement error. (SLD)

Restrictions due to loss of freedom in estimation, the targeting of the instrument, and the presence of misfit in the data were studied through simulation as factors that influence the asymptotic standard errors for person measures. The underestimation of the observed standard deviation of ability in simulated data is discussed. (SLD)

The uses and abuses of cut scores are examined. The article demonstrates (1) that cut scores always entail judgment; (2) that cut scores inherently result in misclassification; (3) that cut scores impose an artificial dichotomy on an essentially continuous distribution of knowledge, skill, or ability; and (4) that no true cut scores exist. (SLD)