Evidence is presented on the generalizability and convergent validity of performance assessments using data from six studies of student achievement that sampled a wide range of measurement facets and methods. Results at individual and school levels indicate that task-sampling variability is the major source of measurement error. (SLD)

Examined the generalizability of written-response scores on the English 30 diploma examination administered to Alberta 12th-grade students. Student scores differed as a function of rater, but this variance component was small across two tasks and two administrations; score generalizability was high using a two-rater system; and scale variability…

A meta-analysis for six weight-loss studies comparing the efficacy of cognitive-behavior therapy (CBT) alone to CBT plus hypnotherapy. Notes that "the addition of hypnosis substantially enhanced treatment outcome." Concludes that the addition of hypnosis to CBT for weight loss results in, at most, a small enhancement of treatment…

D.V. Cicchetti, A.S. Kaufman, and S.S. Sparrow (this issue) examine various technical issues related to six studies of perinatal PCB exposure and neurodevelopment and one study of adult PCB exposure and motor function. They raise questions about possible imperfections of the studies, but many of their assertions are unsupported or frankly…

Cox (1972) proposed a discrete-time survival model that is somewhat analogous to the proportional hazards model for continuous time. Efron (1988) showed that this model can be estimated using ordinary logistic regression software, and Singer and Willett (1993) provided a detailed illustration of a particularly flexible form of the model that…

The diagnosis of developmental dyslexia (DD) is reliant on a discrepancy between intellectual functioning and reading achievement. Discrepancy-based formulae have frequently been employed to establish the significance of the difference between "intelligence" and "actual" reading achievement. These formulae, however, often fail to take into…

Generalizability (G) theory was used with a sample of 37 third-grade students to assess the variability in words correct per minute (WCPM) scores caused by student skill and passage variability. Reliability-like coefficients and the SEM based on a specific number of assessments using different combinations of passages demonstrated how manipulating…

The authors describe 2 efficiency (planned missing data) designs for measurement: the 3-form design and the 2-method measurement design. The 3-form design, a kind of matrix sampling, allows researchers to leverage limited resources to collect data for 33% more survey questions than can be answered by any 1 respondent. Power tables for estimating…

This is a construct validation study of a second language speaking assessment that reported a language profile based on analytic rating scales and a composite score. The study addressed three key issues: score dependability, convergent/discriminant validity of analytic rating scales and the weighting of analytic ratings in the composite score.…

Global economic transformations have never been as dramatic as in the twentieth century. Most countries have experienced radical changes in the standards of income per capita, technology, fertility, mortality, income inequality and the extent of democracy in the course of the past century. It is the goal of many disciplines--economics, history,…

An attempt was made to evaluate the standard error of equating (at the mean of the scores) in an ongoing testing program. The interest in estimating the empirical standard error of equating is occasioned by some discomfort with the error normally reported for test scores. Data used for this evaluation came from the Admissions Testing Program of…

Abuses and misuses of statistics are frequent. This digest attempts to warn against these in three broad classes of pitfalls: sources of bias, errors of methodology, and misinterpretation of results. Sources of bias are conditions or circumstances that affect the external validity of statistical results. In order for a researcher to make…

Recent changes in the National Assessment of Educational Progress (NAEP) that lead to its division into a trend assessment and a main assessment jeopardize the information the NAEP can provide about trends, especially the trends for racial and ethnic groups. This study for the Technical Review Panel addressed whether the trend assessment provides…

This manual has been produced to familiarize data users with the procedures followed for data collection and processing of the second follow-up school component of the National Education Longitudinal Study of 1988 (NELS:88) and to provide the necessary documentation for use of the data file. Chapter 1 contains an overview and the history of the…

A case is presented for adjusting the scores for free response items in the Advanced Placement (AP) tests. Using information about the rating process from the reliability studies, administrations of the AP test for three subject areas, psychology, computer science, and English language and composition, are analyzed. In the reliability studies, 299…