American educators are participating in a harmful, unwinnable contest--the score-boosting game. Standardized tests yield an inaccurate picture of school staff's instructional practices. Administrators should provide assessment-literacy training, brief educational policymakers, encourage autonomous parent groups, analyze test items, and create…

This study investigates item parameter recovery, standard error estimates, and fit statistics yielded by the WINSTEPS program under the Rasch model and the rating scale model through Monte Carlo simulations. The independent variables were item response model, test length, and sample size. WINSTEPS yielded practically unbiased estimates for the…

As item response theory (IRT) becomes popular in educational and psychological testing, there is a need of reporting IRT-based effect size measures. In this study, we show how the standardized mean difference can be generalized into such a measure. A disattenuation procedure based on the IRT test reliability is proposed to correct the attenuation…

Intraindividual variability in positive and negative affect was assessed by the positive affect (Contentment, Joy, Vigor, Love, and Excitement) and negative affect (Depression, Hostility, Anxiety, Agitation, and Social Anxiety) subscales of the state version of the Comprehensive Personality and Affect Scales (COPAS) during a 3-week period. Using…

Research in structured equation modeling (SEM) suggests that nonnormal data will invalidate chi-square tests and produce erroneous standard errors. However, much remains unknown about the extent to which, and the conditions under which nonnormal data can affect SEM application, especially when excessive skewness and kurtosis are present in data.…

School-effectiveness research has not yet been able to identify the factors of effective and noneffective schools, the real contribution of the significant factors, the true sizes of school effects, and the generalizability of school-effectiveness results. This paper presents findings of a meta analysis, the Dutch PSO programme, that was used to…

This report provides an overview of a selection of measurement error studies conducted on National Center for Education Statistics (NCES) surveys. Its intent is not to offer new analyses of program data, but to summarize information from internal memoranda, working papers, and adjudicated reports about errors of measurement that occur during five…

Technical aspects of the first followup survey for the National Education Longitudinal Study of 1988 (NELS:88) are documented and summarized. Some information overlaps materials in the users' manuals for this followup, such as the overview and general descriptions of data collection, sampling, weighting, variance estimation, nonresponse patterns,…

Large sample standard errors of linear equating for the single-group design are derived without making the normality assumption. Two general methods based on the delta method of M. Kendall and A. Stuart (1977) are described. One method uses the exact partial derivatives, and the other uses numerical derivatives. Simulation using the beta-binomial…

Some specific computer-based tools for collecting and examining audit trails, i.e., data that describe a learner's path through a computer-based instruction (CBI) lesson, are detailed. Data analysis and measurement issues related to audit trails are also discussed. A particular set of embedded computer-based tools currently being used at San Diego…

Previous studies have shown that, when administered a self-adapted test, a few examinees will choose item difficulty levels that are not well-matched to their proficiencies, resulting in high standard errors of proficiency estimation. This study investigated whether the previously observed effects of a self-adapted test--lower anxiety and higher…

A measurement approach called generalizability theory (G-theory) is an important alternative to the more familiar classical measurement theory that yields less useful coefficients such as alpha or the KR-20 coefficient. G-theory is a theory about the dependability of behavioral measurements that allows the simultaneous estimation of multiple…

The use of stepwise regression has been criticized for both interpretive misunderstandings and statistical aberrations. A major statistical problem with stepwise regression and other procedures that involve multiple significance tests is the inflation of the Type I error rate. General approaches to control the family-wise error rate such as the…

Given the complexity of classroom reading observations, maintenance of measurement reliability is a concern to researchers. Sources of variation contributing to the unreliability of many measurements may be either (1) lasting and specific, (2) lasting and general, (3) temporary and specific, or (4) temporary and general. Lasting-specific sources…

Use of clustering methods with data from the 1987 High School Transcript (HST) component of the 1986 National Assessment of Educational Progress (NAEP) is discussed in an attempt to persuade data analysts and researchers that they should be interested in problems traditionally turned over to samplers. Nearly all surveys used by the National Center…