It is argued in this paper that generalizability theory provides a uniquely useful framework for defining and quantifying the dependability of data for decision making. It does so by requiring careful specification of the conditions of measurement and the anticipated sources of variation in the results of the measurement procedure. A distinction…

Describes the development of the Early Adolescent version of the Home Observation for Measurement of the Environment (EA-HOME) Inventory. Presents information on its usefulness with African Americans, Chinese Americans, European Americans, Mexican Americans, and Dominican Americans. Notes findings indicating high interobserver agreement, with…

Studied the amount of measurement error in a state's performance-based writing task as it relates to high-stakes decision reproducibility. Using 175 eighth-grade writing samples, the study finds moderate correlations between the two raters' scores, with significant differences for the rates for the handwritten, but not the typed, essays.(SLD)

In this paper, formulas for computing the weights that maximize the reliability of a test with multiple parts are derived using a congeneric model. A direct derivation for the three-part test and case and a two-step derivation for the n-part case are presented, and results for these two approaches are shown to be consistent for the three-part…

Person fit statistics are generated when item response theory is used to construct measures. While person fit statistics are well grounded in theory, their utility in aggregate reporting of survey data has not been demonstrated. This study evaluated effects on reliability and validity of including and excluding misfitting person response patterns,…

Three aspects of the usual approach to assessing local item dependency, Yen's "Q" (H. Huynh, H. Michaels, and S. Ferrara, 1995), deserve further investigation. Pearson correlation coefficients do not distribute normally when the coefficients are large, and thus cannot quantify the dependency well. In the second place, the accuracy of…

Previous studies have indicated that the reliability of test scores composed of testlets might be overestimated by conventional item-based reliability estimation methods (R. Thorndike, 1953; A. Anastasi, 1988; S. Sireci, D. Thissen, and H. Wainer, 1991; H. Wainer and D. Thissen, 1996). This study used generalizability theory to investigate the…

The benefits of item response theory (IRT) will only accrue to a testing program to the extent that model assumptions are met. Obtaining accurate item parameter estimates is a critical first step. However, the sample sizes required for stable parameter estimation are often difficult to obtain in practice, particularly for the more complex models.…

Noting that the amount of reading students do is related to their reading achievement, this booklet presents an instrument designed to measure the amount and breadth of students' reading in and out of school. The first part of the booklet discusses the Reading Activity Inventory (RAI) and how it differs from other reading activity measures, uses…

Using data collected through telephone interviews with a national sample of adults, this study searched for evidence as to whether interviewers have stronger effects on the responses given to a wide range of questions by older people than on the responses of younger people. Responses to 30 items for which significant interviewer effects had…

Evaluated with 12 children (9- to 25-months-old) were the efficacy and reliability of auditory stimulus-response control training and assessment procedures. (Author/LS)

This study determined internal consistency and test-retest reliability coefficients for the Willoughby Personality Schedule, currently used as an outcome measure in research and in clinical practice. The Hoyt analysis of variance yielded an internal consistency reliability coefficient of .90 on the first testing. The test-retest reliability…

Research indicates that a special version of the cloze procedure is a reliable test of reading comprehension. (RB)

Contrary to common belief, reliability estimates of number-right multiple-choice tests are not inflated by speededness. Because examinees guess on questions when they run out of time, the responses to these questions show less consistency with the responses of other questions, and the reliability of the test will be decreased. The surprising…