This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

Connecticut has two statewide student assessment programs. The Connecticut Assessment of Educational Progress uses sampling procedures to examine state program effectiveness periodically in 11 mandated subject areas. The Connecticut Mastery Tests annually monitor mathematics and language arts progress for every fourth-, sixth-, and eighth- grade…

Provides details on the instrument development, sample design, data collection, and data analysis procedures for the 1998 National Assessment of Educational Progress (NAEP) national and state assessments. Includes information necessary to show adherence to testing standards of professional organizations. (SLD)

One method to correct for selection bias in comparing Scholastic Aptitude Test (SAT) scores among states is presented, which is a modification of J. J. Heckman's Selection Bias Correction (1976, 1979). Empirical results suggest that sample selection bias is present in SAT score regressions. (SLD)

Describes the teacher-developed California Assessment Program (CAP) writing measure, designed to support California's reform curriculum and based on matrix sampling techniques. This program will be supplemented by literature and mathematics assessments. The greatest challenge is designing an assessment to match the state's new history and social…

California Assessment Program's application of matrix sampling and item response curve theory to the scaling and reporting of state assessment data is described. It is designed to express educational outcomes in an efficient and interpretable form that is both immediately informative and suited to analysis over extended periods of time.…

This study provides empirical evidence about the sampling variability and generalizability (reliability) of a statewide performance assessment for grade six. Results for 600 students at individual and school levels indicate that task-sampling variability was the major source of measurement error. Rater-sampling variability was negligible. (SLD)